A method of processing video data in a video coding system, wherein the video data uses YUV422 or YUV444 chroma sub-sampling pattern, the method comprising: receiving video data associated with a luma coding unit, CU, having size 2Nx2N and N is a positive integer; receiving the video data associated with a chroma CU having size Nx2N or 2Nx2N, wherein the chroma CU corresponds to a same pixel area associated with the luma CU; generating luma residue data corresponding to the luma CU by applying prediction process on the video data associated with the luma CU; generating chroma residue data corresponding to the chroma CU by applying the prediction process on the video data associated with the chroma CU; applying transform process on the luma residue data corresponding to the luma CU to form at least one luma transform unit, TU; applying transform process on the chroma residue data corresponding to the chroma CU to form at least one chroma TU, wherein the transform process is independent of prediction block size or prediction mode associated with the prediction process; generating luma transform coefficients for each luma TU by applying two-dimensional transformation to the luma residue data corresponding to each luma TU; and generating chroma transform coefficients for each chroma TU by applying the two-dimensional transformation to the chroma residue data corresponding to each chroma TU, characterized in that in the transform process on the chroma residue data the at least one chroma TU is restricted to have a square shape, and the transform process on the luma residue data determines whether to split and how to split the at least one luma TU, wherein splitting the at least one chroma TU uses a quadtree. receiving video data associated with a luma coding unit, CU, having size 2Nx2N and N is a positive integer; receiving the video data associated with a chroma CU having size Nx2N or 2Nx2N, wherein the chroma CU corresponds to a same pixel area associated with the luma CU; generating luma residue data corresponding to the luma CU by applying prediction process on the video data associated with the luma CU; generating chroma residue data corresponding to the chroma CU by applying the prediction process on the video data associated with the chroma CU; applying transform process on the luma residue data corresponding to the luma CU to form at least one luma transform unit, TU; applying transform process on the chroma residue data corresponding to the chroma CU to form at least one chroma TU, wherein the transform process is independent of prediction block size or prediction mode associated with the prediction process; generating luma transform coefficients for each luma TU by applying two-dimensional transformation to the luma residue data corresponding to each luma TU; and generating chroma transform coefficients for each chroma TU by applying the two-dimensional transformation to the chroma residue data corresponding to each chroma TU, characterized in that in the transform process on the chroma residue data the at least one chroma TU is restricted to have a square shape, and the transform process on the luma residue data determines whether to split and how to split the at least one luma TU, wherein splitting the at least one chroma TU uses a quadtree.