Patent Publication Number: US-2019182503-A1

Title: Method and image processing apparatus for video coding

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims the priority benefit of U.S. provisional application Ser. No. 62/597,938, filed on Dec. 13, 2017. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification. 
    
    
     TECHNICAL FIELD 
     The disclosure relates to technique for video coding. 
     BACKGROUND 
     As the rapid development of virtual reality and augmented reality in entertainment industry, consumer demands on high-quality images are raising to assimilate, explore, and manipulate a virtual environment for fully immersive experience. In order to provide smooth and high-quality image frames, image coding becomes one of core technologies for image data reception and transmission under storage capacity and bandwidth constraints. 
     SUMMARY OF THE DISCLOSURE 
     Accordingly, a method and an image processing apparatus for video coding are provided in the disclosure, where coding efficiency on video images would be effectively enhanced. 
     In an exemplary embodiment of the disclosure, the method is applicable to an image processing apparatus and includes the following steps. A current coding unit is received, and the number of control points of a current coding unit is set, where the number of control points is greater than or equal to 3. Next, at least one affine model is generated based on the number of control points, and an affine motion vector corresponding to each of the at least one affine model is computed. A motion vector predictor of the current coding unit is then computed based on all the at least one affine motion vector so as to accordingly perform inter-prediction coding on the current coding unit. 
     In an exemplary embodiment of the disclosure, the image processing apparatus includes a memory and a processor, where the processor is coupled to the memory. The memory is configured to store data. The processor is configured to: receive a current coding unit; set the number of control points of the current coding unit, where the number of control points is greater than or equal to 3; generate at least one affine model according to the number of control points; compute an affine motion vector respectively corresponding to each of the at least one affine model; and compute a motion vector predictor of the current coding unit based on the at least one affine motion vector so as to accordingly perform inter-prediction coding on the current coding unit. 
     In order to make the aforementioned features and advantages of the present disclosure comprehensible, preferred embodiments accompanied with figures are described in detail below. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1A - FIG. 1B  illustrate schematic diagrams of a motion vector field of a block. 
         FIG. 1C  illustrates a schematic diagram of a coding unit having multiple moving objects. 
         FIG. 2  illustrates a block diagram of an image processing apparatus in accordance with an exemplary embodiment of the disclosure. 
         FIG. 3  illustrates a flowchart of a video coding method in accordance with an exemplary embodiment of the disclosure. 
         FIG. 4A - FIG. 4D  illustrate schematic diagrams of setting methods of control points in accordance with an exemplary embodiment of the disclosure. 
         FIG. 5A  illustrates a schematic diagram of a searching method of neighboring motion vectors of a control point in accordance with an exemplary embodiment of the disclosure. 
         FIG. 5B  illustrates a schematic diagram of a current coding unit having three control points in accordance with an exemplary embodiment of the disclosure. 
         FIG. 5C  illustrates a schematic diagram of a current coding unit having five control points in accordance with an exemplary embodiment of the disclosure. 
         FIG. 6  illustrates a flowchart of a setting method of control points in accordance with an exemplary embodiment of the disclosure. 
         FIG. 7  illustrates a schematic diagram of a setting method of control points in accordance with an exemplary embodiment of the disclosure. 
     
    
    
     DESCRIPTION OF THE EMBODIMENTS 
     Reference will now be made in detail to the present preferred embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the claimed disclosure will satisfy applicable legal requirements. 
     In the Joint Video Expert Team (JVET) conference, collaboratively hosted by the Telecommunication Standardization Sector (ITU-T) and the Moving Picture Experts Group (MPEG), the Video Coding (H.266/VVC) is proposed to provide a coding standard with higher efficiency than that of High Efficiency Video Coding (H.265/HEVC). In response to the Call for Proposals (CfP) on video compression, three categories of technologies including standard dynamic range (SDR) videos, high dynamic range (HDR) videos, and 360 degree videos are discussed. Such three techniques involve prediction for frame coding. 
     The aforesaid prediction may be classified into intra-prediction and inter-prediction. The former mainly exploits the spatial correlation between neighboring blocks, and the latter mainly makes use of the temporal correlation between frames in order to perform motion-compensation prediction (MCP). A motion vector of a block between frames may be computed through motion-compensation prediction based on a translation motion model. Compared with transmitting raw data of the block, transmitting the motion vector would significantly reduce the bit number for coding. However, in the real world, there exists motions such as zoom in, zoom out, rotation, similarity transformation, spiral similarity, perspective motion, or other irregular motions. Hence, the mechanism of motion-compensation prediction based on the translation motion model would highly impact coding efficiency. 
     The Joint Exploration Test Model (JEM) has proposed affine motion compensation prediction, where a motion vector field (MVF) is described by a single affine model according to two control points to perform better prediction on a scene involving rotation, zoom in/out, or translation. As an example of a single block  100  illustrated in  FIG. 1A , a motion vector field of a sampling position (x, y) in the block  100  may be described by Eq. (1): 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             v 
                             x 
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 0 
                                  
                                 
                                     
                                 
                                  
                                 x 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             v 
                             y 
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             + 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 0 
                                  
                                 
                                     
                                 
                                  
                                 y 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     1 
                     ) 
                   
                 
               
             
           
         
       
     
     Herein, v x  denotes a horizontal motion vector of a control point, and v x  denotes a vertical motion vector of a control point. Hence, (v 0x , v 0y ) denotes a motion vector of a control point  110 , and (v 1x , v 1y ) denotes a motion vector of a control point  120 , and w is a weight with respect to the width of the block  100 . 
     To simplify the motion-compensation prediction, the block  100  may be divided into M×N sub-blocks (e.g. the block  100  illustrated in  FIG. 1B  is divided into 4×4 sub-blocks), a motion vector of a center sample of each of the sub-blocks may be derived based on Eq. (1), and a motion compensation interpolation filter may be applied on the motion vector of each of the sub-blocks to obtain the prediction thereof. After the motion-compensation prediction, the motion vector with high precision is rounded and saved as the same precision as a normal motion vector. 
     However, in order to satisfy consumer demands on high-quality videos, with an increment in video resolution, the size of each coding unit (CU) has been relatively increased. In an exemplary embodiment, it may be as large as 128×128. The existing affine motion-compensation prediction only assumes that an entire coding unit belongs to a single object. However, when a coding unit includes more than one object with different motions (e.g. a coding unit CU 1  illustrated in  FIG. 1C  includes moving objects OB 1 , OB 2 , and OB 3  with different rotation directions, where the moving object OB 1  rotates counterwisely, the moving objects OB 2  and OB 3  rotate clockwisely but with different rotation speeds), the existing mechanism could result in false prediction. The video coding technique proposed in the exemplary embodiments of the disclosure may solve the problem of insufficient efficiency in high-resolution video due to two control points and a single affine model. 
       FIG. 2  illustrates a block diagram of an image processing apparatus in accordance with an exemplary embodiment of the disclosure. However, this is merely for illustrative purposes and is not intended to limit the disclosure. 
     Referring to  FIG. 2 , in the present exemplary embodiment, an image processing device  200  would at least include a memory  210  and a processor  220 , where the processor  220  is coupled to the memory  210 . In an exemplary embodiment, the image processing device  200  may be an electronic device such as a personal computer, a laptop computer, a server computer, a tabular computer, a smart phone, a wearable device, a work station, and so forth. In an exemplary embodiment, the image processing apparatus  200  may be an encoder and/or a decoder. 
     The memory  210  would be configured to store data such as images, numerical data, programming codes, and may be, for example, any type of fixed or removable random-access memory (RAM), read-only memory (ROM), flash memory, hard disc or other similar devices, integrated circuits, and any combinations thereof. 
     The processor  220  would be configured to control an overall operation of the image processing apparatus  200  to perform video coding and may be, for example, a central processing unit (CPU), an application processor (AP), or other programmable general purpose or special purpose microprocessor, digital signal processor (DSP), image signal processor (ISP), graphics processing unit (GPU) or other similar devices, integrated circuits, and any combinations thereof. 
     As a side note, in an exemplary embodiment, the image processing apparatus  200  may optionally include an image capturing device, a transmission interface, a display, and a communication unit. The image capturing device may be, for example, a digital camera, a digital camcorder, a web camera, a surveillance camcorder, and configured to capture image data. The transmission interface may be an I/O interface that allows the processor  220  to receive image data and related information. The display may be any screen configured to display processed image data. The communication data may be a modem or a transceiver compatible to any wired or wireless communication standard and configured to receive raw image data from external sources and transmit processed image data to other apparatuses or platforms. As known per se, from an encoding perspective, the processor  220  may transmit encoded bitstreams and related information to other apparatuses or platforms having decoders via the communication unit upon the completion of encoding. Moreover, the processor  220  may also store encoded bitstreams and related information to storage medium such as a DVD disc, a hard disk, a flash drive, a memory card, and so forth. The disclosure is not limited in this regard. From a decoding perspective, once the processor  220  receives encoded bitstreams and related information, it would decode the encoded bitstreams and the related information according to the related information, and output to a player for video playing. 
       FIG. 3  illustrates a flowchart of a video coding method in accordance with an exemplary embodiment of the disclosure. The method flow in  FIG. 3  may be implemented by the image processing apparatus  200  in  FIG. 2 . In an exemplary embodiment of the disclosure, the coding may be encoding and/or decoding, and the coding method may be an encoding method and/or a decoding method. 
     In the present exemplary embodiment, the processor  220  may execute an encoding process and/or a decoding process of the image processing apparatus  200 . For example, the method flow in  FIG. 3  may be stored as programming codes in the memory  210 , and the processor  220  would execute the programming codes to perform each step in  FIG. 3 . When the processor  220  executes the encoding flow and before executing the flow in  FIG. 3 , it would receive raw video streams/frames and then perform encoding procedure thereon. When the processor  220  executes the decoding flow and before executing the flow in  FIG. 3 , it would receive encoded bitstreams and then perform decoding procedure thereon. In the following description, one of coding units (CU) in coding tree units (CTU) in the received raw video streams/frames or the encoded bitstreams as a basic processing unit would be described and referred to as “a current coding unit.” 
     Referring to  FIG. 2  and  FIG. 3 , the processor  220  of the image processing apparatus  200  would first receive a current coding unit (Step S 302 ) and set the number of control points of the current coding unit, where the number of control points is greater than or equal to 3 (Step S 304 ). The number of control points may be a preset value pre-entered by a user through an input device (not shown) or a system default value, or may be adaptively set according to a moving state of an object in the current coding unit. 
     Next, the processor  220  would generate at least one affine model according to the number of control points (Step S 306 ) and compute an affine motion vector respectively corresponding to each of the at least one affine model (Step S 308 ). The processor  220  would then compute a motion vector predictor of the current coding unit according to all of the at least one affine motion vector to accordingly perform inter-prediction coding on the current coding unit (Step  310 ). Herein, the processor  220  would apply all of the at least one affine model on all sub-blocks in the current coding unit, assign all the at least one affine motion vector to each of the sub-blocks with different weights, and thereby obtain the corresponding motion vector predictor to perform inter-prediction coding on the current coding unit. The details of Step S 304 -S 310  would be given in the following exemplary embodiments. 
       FIG. 4A  and  FIG. 4D  illustrate setting methods of control points in accordance with an exemplary embodiment of the disclosure, where the provided examples may be implemented by the image processing apparatus  200  in  FIG. 2 . 
     In the present exemplary embodiment, the processor  220  would set the number and a reference range of control points according to user settings or system defaults. The number of control points would satisfy 1+2 N , where N is a positive integer. The reference range of control points would be the number of rows and columns of neighboring sub-blocks at the left and upper sides of the current encoding unit and would be denoted as M, where M is a positive integer. As an example illustrated in  FIG. 4A , when M=1, a reference range of control points of a current coding unit CU 4 A would be neighboring sub-blocks (numbered 1-9) at a first left neighboring column and a first upper neighboring row of the current coding unit CU 4 A; when M=2, the reference range of control points of the current coding unit CU 4 A would be neighboring sub-blocks (numbered 1-20) at first two left neighboring columns and first two upper neighboring rows of the current coding unit CU 4 A; and so on. 
     Upon completion of setting the number and the reference range of control points, the processor  220  would set positions of control points. First, the processor  220  would arrange three control points at a bottom-left corner, a top-left corner, and a top-right corner of the current coding unit. As an example illustrated in  FIGS. 4B, 40B, 41B, and 42B  are three control points respectively at a top-left corner, a top-right corner, and a bottom-left corner of a current coding unit CU 4 B, i.e. corresponding to sub-blocks numbered 5, 9, and 1. From another perspective, assume that the sub-blocks numbered 1-9 are arranged at a reference line RB, since the current coding unit CU 4 B is a square, the three control points  40 B,  41 B, and  42 B would be located at two endpoints and a midpoint of the reference line RB. As another example illustrated in  FIGS. 4C, 40C, 41C, and 42C  are three control points at a top-left corner, a top-right corner, and a bottom-left corner of a current coding unit CU 4 C, i.e. corresponding to sub-blocks numbered 5, 13, and 1. From another perspective, assume that the sub-blocks numbered 1-13 are arranged at a reference line RC, since the current coding unit CU 4 C is not a square with a width greater than a length, the three control points  40 C,  41 C, and  42 C would be respectively located at two endpoints and a left portion of the reference line RC. 
     The processor  220  would determine whether to add new control points between each two of the control points according to the value of N. From another perspective, the processor  220  would determine whether the number of control points arranged at the current coding unit has reached a setting value of the number of control points. In detail, when N=1, it means that the number of control points is 3 and that the number of control points arranged at the current coding unit has reached the setting value of the number of control points. Hence, the arrangement of control points has been completed. When N=2, it means that the number of control points is 5 and that the number of control points arranged at the current coding unit has not reached the setting value of the number of control points yet. Hence, the processor  220  would add two new control points between each two adjacent control points at the current coding unit. As an example of  FIG. 4D , following  FIGS. 4B, 40B, 41B, and 42B  are three control points that have already been arranged at a current coding unit CU 4 D. The processor  220  would additionally arrange a control point  43  at a midpoint of the control point  40 B and the control point  41 B, and additionally arrange a control point  44  at a midpoint of the control point  42 B and the control point  40 B. When N=3, it means that the number of control points is nine, the processor  220  would add four new control points between each two adjacent control points such as a midpoint of the control point  42 B and the control point  44 , a midpoint of the control point  44  and the control point  40 B, a midpoint of the control point  40 B and the control point  43 , and a midpoint of the control point  43  and the control point  41 B, and so on. When the processor  220  determines that the number of the control points arranged at the current coding unit  420  has not yet reached the setting value of the number of control points, it would recursively set a new control point at a midpoint of each two adjacent arranged control points until the number of the control points arranged at the current coding unit  420  reaches the setting value of the number of control points. 
     Next, the processor  220  would generate one or more affine models according to the motion vectors of the control points. In an exemplary embodiment, when N=1 (i.e. the number of control points is 3), the number of affine models would be 1. When N&gt;1 (i.e. the number of control points is greater than 3), the number of affine models would be 1+2 N-1 . A motion vector of a control point may be computed according to coded neighboring motion vectors, where a reference frame of the coded neighboring motion vectors would be the same as a reference frame of the control point. 
     For example,  FIG. 5A  illustrates a schematic diagram of a searching method of neighboring motion vectors of control points. When M=1, the processor  220  would respectively search for coded motion vectors from neighboring sub-blocks of control points  50 A,  51 A, and  52 A of a current coding unit CU 5 A. In terms of the control point  50 A at a sub-block D, assume that sub-blocks A-C are coded sub-blocks searched out by the processor  220 , and the motion vector of the control point  50 A would be selected from the motion vectors A-C. For example, the processor  220  may determine whether each of the sub-blocks A-C and the current coding unit CU 5 A have a same reference frame in a consistent order, and the motion vector of the sub-block first satisfied such setting would be a basis for setting the motion vector of the control point  50 A. 
     On the other hand, in terms of the control point  51 A at a sub-block G, assume that sub-blocks E-F are coded sub-blocks searched out by the processor  220 , and the motion vector of the control point  51 A would be selected from the motion vectors of sub-blocks E-F. Since a sub-block H has not yet been coded, it would not be a basis for setting the motion vector of the control point  51 A. In terms of the control point  52 A at a sub-block K, assume that sub-blocks I-J are coded sub-blocks searched out by the processor  220 , and the motion vector of the control point  52 A would be selected from the motion vectors of sub-blocks I-J. Since a sub-block L has not yet been coded, it would not be a basis for setting the motion vector of the control point  52 A. 
     Moreover, when M=2, the processor  220  would respectively search for coded motion vectors from the neighboring sub-blocks of control points  50 A,  51 A, and  52 A of the current coding unit CU 5 A. Compared to M=1, more neighboring sub-blocks may be referenced for selecting and setting motion vectors of the control points  50 A,  51 A, and  52 A. For example, neighboring sub-blocks A-C and M-Q may be referenced by the control point  50 A; neighboring sub-blocks E-F, H, R-V may be referenced by the control point  51 A; and neighboring sub-blocks I, J, L, W-ZZ may be referenced by the control point  52 A. The approach for selecting and setting the motion vectors of the control points  50 A,  51 A, and  52 A may refer to the related description of M=1 and would not be repeated for brevity purposes. 
     In an exemplary embodiment, a motion vector of a control point may be computed based on motion vectors of other control points. For example, when the motion vector of the control point  52 A is not able to be obtained according to neighboring sub-blocks thereof, it may be computed according to the motion vectors of the control points  50 A and  51 A. The motion vector of the control point  52 A may be computed based on, for example, Eq. (2.01): 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             
                               mv 
                               _ 
                             
                             2 
                             x 
                           
                           = 
                           
                             
                               
                                 mv 
                                 _ 
                               
                               0 
                               x 
                             
                             - 
                             
                               h 
                                
                               
                                 
                                   ( 
                                   
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       1 
                                       y 
                                     
                                     - 
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       0 
                                       y 
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             
                               mv 
                               _ 
                             
                             2 
                             y 
                           
                           = 
                           
                             
                               
                                 mv 
                                 _ 
                               
                               0 
                               y 
                             
                             + 
                             
                               h 
                                
                               
                                 
                                   ( 
                                   
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       1 
                                       x 
                                     
                                     - 
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       0 
                                       x 
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.01 
                     ) 
                   
                 
               
             
           
         
       
     
     Herein,  mv   2   x  and  mv   2   y  denote a horizontal component and a vertical component of the motion vector of the control point  52 A;  mv   0   x  and  mv   0   y  denote a horizontal component and a vertical component of the motion vector of the control point  50 A;  mv   1   x  and  mv   1   y  denote a horizontal component and a vertical component of the motion vector of the control point  51 A; h denotes a height of the coding unit CU 5 A; and w denotes a width of the coding unit CU 5 A. 
     In another exemplary embodiment, when the motion vector of the control point  51 A is not able to be obtained from neighboring sub-blocks, it may be computed according to the control points  50 A and  52 A based on, for example, Eq. (2.02): 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             
                               mv 
                               _ 
                             
                             1 
                             x 
                           
                           = 
                           
                             
                               
                                 mv 
                                 _ 
                               
                               0 
                               x 
                             
                             + 
                             
                               h 
                                
                               
                                 
                                   ( 
                                   
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       2 
                                       y 
                                     
                                     - 
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       0 
                                       y 
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             
                               mv 
                               _ 
                             
                             1 
                             y 
                           
                           = 
                           
                             
                               
                                 mv 
                                 _ 
                               
                               0 
                               y 
                             
                             - 
                             
                               h 
                                
                               
                                 
                                   ( 
                                   
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       2 
                                       x 
                                     
                                     - 
                                     
                                       
                                         mv 
                                         _ 
                                       
                                       0 
                                       x 
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.02 
                     ) 
                   
                 
               
             
           
         
       
     
     herein,  mv   2   x  and  mv   2   y  denote a horizontal component and a vertical component of the motion vector of the control point  52 A;  mv   0   x  and  mv   0   y  denote a horizontal component and a vertical component of the motion vector of the control point  50 A;  mv   1   x  and  mv   1   y  denote a horizontal component and a vertical component of the motion vector of the control point  51 A; h denotes a height of the coding unit CU 5 A; and w denotes a width of the coding unit CU 5 A. 
     As an example,  FIG. 5B  illustrates a schematic diagram of a current coding unit CU 5 B having three control points  50 B,  51 B, and  52 B in accordance with an exemplary embodiment of the disclosure. The processor  220  may generate an affine model of the current coding unit CU 5 B according to a motion vector (v 0x , v 0y ) of the control point  50 B, a motion vector (v 1x , v 1y ) of the control point  51 B, a motion vector (v 2x , v 2y ) of the control point  52 B as expressed in Eq. (2.1): 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             v 
                             x 
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         2 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 0 
                                  
                                 
                                     
                                 
                                  
                                 x 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             v 
                             y 
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             + 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         2 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 0 
                                  
                                 
                                     
                                 
                                  
                                 y 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.1 
                     ) 
                   
                 
               
             
           
         
       
     
     Herein, (v x , v y ) denotes a motion vector field of a sub-block with a sampling position (x, y) in the current coding unit CU 5 B, and w denotes a weight with respect to a width of the sub-block. In the present exemplary embodiment, after the processor  220  applies the affine model onto all sub-blocks in the current coding unit CU 5 B, all affine motion vectors would be distributed to each of the sub-blocks with different weights, and a corresponding motion vector predictor would then be obtained. 
     As another example,  FIG. 5C  illustrates a schematic diagram of a current coding unit CU 5 C having five control points  50 C,  51 C,  52 C,  53 C, and  54 C. The processor  220  may generate three affine models of the current coding unit CU 5 C according to a motion vector (v 0x , v 0y ) of the control point v 0 , a motion vector (v 1x , v 1y ) of the control point v 1 , a motion vector (v 2x , v 2y ) of the control point v 2 , a motion vector (v 3x , v 3y ) of the control point v 3 , and a motion vector (v 4x , v 4y ) of the control point v 4 . Herein, each of the affine models may be generated from a different group of any three of the control points, where the five control points would be used, and a same control point may appear in different groups. In an exemplary embodiment, the three affine models of the current coding unit CU 5 C may be expressed by, for example, Eq. (2.2)-Eq. (2.4): 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             V 
                             
                               x 
                                
                               
                                   
                               
                                
                               1 
                             
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         4 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         2 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         4 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 4 
                                  
                                 
                                     
                                 
                                  
                                 x 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             V 
                             
                               y 
                                
                               
                                   
                               
                                
                               1 
                             
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         4 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         2 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         4 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 4 
                                  
                                 
                                     
                                 
                                  
                                 y 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.2 
                     ) 
                   
                 
               
             
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             V 
                             
                               x 
                                
                               
                                   
                               
                                
                               2 
                             
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         3 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         4 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 0 
                                  
                                 
                                     
                                 
                                  
                                 x 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             V 
                             
                               y 
                                
                               
                                   
                               
                                
                               2 
                             
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         3 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         4 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 0 
                                  
                                 
                                     
                                 
                                  
                                 y 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.3 
                     ) 
                   
                 
               
             
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             V 
                             
                               x 
                                
                               
                                   
                               
                                
                               3 
                             
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         3 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         3 
                                          
                                         
                                             
                                         
                                          
                                         x 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 3 
                                  
                                 
                                     
                                 
                                  
                                 x 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             V 
                             
                               y 
                                
                               
                                   
                               
                                
                               3 
                             
                           
                           = 
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         3 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               x 
                             
                             - 
                             
                               
                                 
                                   ( 
                                   
                                     
                                       v 
                                       
                                         0 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                     - 
                                     
                                       v 
                                       
                                         3 
                                          
                                         
                                             
                                         
                                          
                                         y 
                                       
                                     
                                   
                                   ) 
                                 
                                 w 
                               
                                
                               y 
                             
                             + 
                             
                               v 
                               
                                 3 
                                  
                                 
                                     
                                 
                                  
                                 y 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.4 
                     ) 
                   
                 
               
             
           
         
       
     
     Herein, (v x1 , v y1 ), (v x2 , v y2 ), and (v x3 , v y3 ) denote a motion vector field of a sub-block with a sampling position (x, y) in the current coding unit CU 5 B, and w denotes a weight with respect to a width of the sub-block. After the processor  220  applies the affine models onto all sub-blocks in the current coding unit CU 5 C, three affine motion vectors (v x1 , v y1 ), (v x2 , v y2 ), and (v x3 , v y3 ) would be generated, and all the affine motion vectors would be distributed to each of the sub-blocks with different weights. The processor may generate a motion vector predictor of each of the sub-blocks based on Eq. (2.5): 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             X 
                             ′ 
                           
                           = 
                           
                             
                               
                                 w 
                                 1 
                               
                                
                               
                                 V 
                                 
                                   x 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             + 
                             
                               
                                 w 
                                 2 
                               
                                
                               
                                 V 
                                 
                                   x 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             + 
                             
                               
                                 w 
                                 3 
                               
                                
                               
                                 V 
                                 
                                   x 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             Y 
                             ′ 
                           
                           = 
                           
                             
                               
                                 w 
                                 1 
                               
                                
                               
                                 V 
                                 
                                   y 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             + 
                             
                               
                                 w 
                                 2 
                               
                                
                               
                                 V 
                                 
                                   y 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             + 
                             
                               
                                 w 
                                 3 
                               
                                
                               
                                 V 
                                 
                                   y 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                    
                   
                     ( 
                     2.5 
                     ) 
                   
                 
               
             
           
         
       
     
     Herein, X′ and Y′ denote motion vector predictors of a sub-block with respect to a horizontal direction and a vertical direction, and w 1 , w 2 , and w 3  denote a weight corresponding to a distance between the sub-block and each of the three affine motion vectors. 
       FIG. 6  illustrates a flowchart of a setting method of control points in accordance with an exemplary embodiment of the disclosure, and  FIG. 7  illustrates a schematic diagram of a setting method of control points in accordance with an exemplary embodiment of the disclosure. The following setting method may be implemented by the image processing apparatus  200  for encoding or decoding. In the present exemplary embodiment, the processor  220  would adaptively set the number of control points according to a moving status of an object in a current coding unit. 
     Referring to  FIG. 2  and  FIG. 6 , the processor  220  would set three initial control points of a current coding unit (Step S 602 ). In the present exemplary embodiment, M=1 would be a preset value of a reference range for illustration, and the three initial control points would be respectively arranged at a bottom-left corner, a top-left corner, and a top-right corner of the current coding unit, referred to as a first initial control point, a second initial control point, and a third initial control point hereafter. As an example illustrated in  FIGS. 7, 7A, 7B, and 7C  of a current coding unit CU 7  respectively represent the first initial control point, the second initial control point, and the third initial control point. As a side note, in other exemplary embodiments, the processor  220  may also set the reference range according to user settings or system defaults before Step S 602 . 
     Next, the processor  220  would compute a motion vector of each of the initial control points (Step S 604 ), compute a motion vector difference between each two adjacent initial control points (Step S 606 ), and determine whether there exists any motion vector difference being greater than a preset difference and whether the number of the initial control points arranged at the current coding unit is less than the number of neighboring sub-blocks at the top and at the left of the current coding unit (Step S 608 ). It should be noted that, each two adjacent initial control points herein refers to two adjacent initial control points sequentially arranged at corners of the current coding unit. As an example illustrated in  FIG. 7 , the processor  220  would compute a motion vector difference ∥V A −V B ∥ (referred to as “a first motion vector difference”) between a motion vector V A  of the first initial control point  7 A and a motion vector V B  of the second initial control point  7 B, compute a motion vector difference ∥V B −V C ∥ (referred to as “a second motion vector difference”) between the motion vector V B  of the second initial control point  7 B and a motion vector V C  of the third control point  7 C, and determine whether any of the first motion vector difference ∥V A −V B ∥ and the second motion vector difference ∥V B −V C ∥ is greater than a preset difference d. 
     When the processor  220  determines that no motion vector difference is greater than the preset difference, in one exemplary embodiment, it means that all the motion vectors are highly similar, and the existing initial control points correspond to a same moving object. Therefore, no new control point is required to be added. Moreover, when the number of initial control points arranged at the current coding unit is not less than (or reaches) the number of neighboring sub-blocks at the top and at the left of the current coding unit, no new control point is required to be added either. The processor  220  may end the setting process of control points and generate an affine model according to the motion vectors of the initial control points. As an example illustrated in  FIG. 7 , when the processor  220  determines that ∥V A −V B ∥&lt;d and ∥V B −V C ∥&lt;d, it would generate an affine model by using the motion vector V A  of the first initial control point  7 A, the motion vector V B  of the second initial control point  7 B, and the motion vector V C  of the third initial control point  7 C, thereby compute an affine motion vector respectively corresponds to each of the affine models, and compute a motion vector predictor of the current coding unit according to all the affine motion vectors to accordingly perform inter-prediction coding on the current coding unit. 
     On the other hand, when the processor  220  determines that any of the motion vector difference is greater than the preset difference, in one exemplary embodiment, it means that the existing initial control points correspond to different moving objects. Therefore, control points may be added to comprehensively described all the moving objects in the current coding unit for a more precise prediction in the follow-up steps. Herein, when the processor  220  further determines that the number of initial control points arranged at the current coding unit is not less than (or reaches) the number of neighboring sub-blocks at the top and at the left of the current coding unit, the processor  220  would add a control point between each two adjacent initial control points (Step S 610 ) and add the newly added control points to the initial control points (Step S 612 ). In other words, the control point added between the first initial control point and the second initial control point would become a fourth initial control point, and the control point added between the second initial control point and the third initial control point would become a fifth initial control point. Next, the processor  220  would return to Step S 604  to repeat the follow-up steps until the motion vector difference of each two adjacent control points is less than the preset difference or the number of the initial control points arranged at the current coding unit reaches the number of neighboring sub-blocks at the top and at the left of the current coding unit. 
     As an example of  FIG. 7 , when the processor  220  determines that ∥V A −V B ∥&gt;d and/or ∥V B −V C ∥&gt;d, the processor  220  would add a control point  7 D at a midpoint of the first initial control point  7 A and the second initial control point  7 B as well as add a control point  7 E at a midpoint of the second initial control point  7 B and the third initial control point  7 C. Next, the processor  220  would compute a motion vector difference ∥V A −V D ∥ between the motion vector V A  of the first initial control point  7 A and the motion vector V D  of the fourth initial control point  7 D, a motion vector difference ∥V D −V B ∥ between the motion vector V D  of the fourth initial control point  7 D and the motion vector V B  of the second initial control point  7 B, a motion vector difference ∥V B −V E ∥ between the motion vector V B  of the second initial control point  7 B and the motion vector V E  of the fifth initial control point  7 E, a motion vector difference ∥V E −V C ∥ between the motion vector V E  of the fifth initial control point  7 E and the motion vector V C  of the third initial control point  7 C, and then determine whether any one of the four motion vector differences is greater than the preset difference d. When any of the fourth motion vector differences is greater than the preset difference d, the processor  220  would further add new control points at a midpoint of each two adjacent control points among the five initial control points  7 A- 7 E. In other words, when the processor  220  determines that any motion vector difference of each two adjacent initial control points of the current coding unit CU 7  is not less than the preset difference, it would recursively arrange a new control point at a midpoint of each two adjacent arranged initial control points until the motion vector difference of each two adjacent control points of the current coding unit CU 7  is less than the preset difference or the number of the initial control points arranged at the current coding unit CU 7  reaches the number of neighboring sub-blocks at the top and at the left of the current coding unit CU 7  (e.g. the number of initial control points allowed to be arranged in  FIG. 7  would be at most 9). 
     When the four differences are all less than the preset difference d, the processor  220  would generate three affine models by using the motion vector V A  of the first initial control point  7 A, the motion vector V B  of the second initial control point  7 B, the motion vector V C  of the third initial control point  7 C, the motion vector V D  of the fourth initial control point  7 D, and the motion vector V E  of the fifth initial control point  7 E, thereby generate an affine motion vector corresponding to each of the affine models respectively, and compute a motion vector predictor of the current coding unit according to all the affine motion vectors to accordingly perform inter-prediction coding on the current coding unit. 
     In summary, the video coding method and the image processing apparatus proposed in the disclosure would generate at least one affine model by using three or more control points in a coding unit to respectively compute a corresponding affine motion vector and compute a motion vector predictor of the coding unit according to the affine motion vector. The video coding technique proposed in the disclosure would solve the problem of insufficient efficiency in high-resolution video due to two control points and a single affine model so as to enhance the precision of inter-prediction coding and coding efficiency on video images. 
     Although the disclosure has been provided with embodiments as above, the embodiments are not intended to limit the disclosure. It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the disclosed embodiments without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the disclosure falls within the scope of the following claims.