Abstract:
This invention provides an image processing apparatus and an image processing method. By calculation of the pixel difference that is the difference of each corresponding pixels between the current image and the previous image with its neighbor pixel difference, this invention can determine the blending value. Furthermore, the image processing apparatus comprises a pixel buffer unit for storing the current image and the previous image; a pixel difference calculating unit for calculating a plurality of pixel differences; a moving judgment unit, coupled to the pixel difference calculating unit, for determining the pixel moving or static according to by the plurality of pixel differences; a blending value determining unit, coupled to the moving judgment unit, for determining a new blending value when the pixel is static; and an output unit, coupled to the blending value determining unit, for generating an output image by adding the pixels belonging to the same positions pixels on the current image and the previous image with a weighting according to the blending value.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    (a) Field of the Invention 
         [0002]    The present invention relates to an image processing apparatus and method, more particularly, to an image processing apparatus and method for determining a blending value according to a motion level of an image. 
         [0003]    (b) Description of the Related Art 
         [0004]    Video content typically includes a plurality of frames or fields, and the frames (or fields) are transmitted as a consecutive sequence. During transmission, if there is a moving object in the video content, pixel differences are present between current and previous frames, that is, differences between pixel values of pixels of the same location occur with respect to present and previous frames. This is because a pixel may be part of an object in the previous frame, and the same pixel may change to be part of the background in the current frame. Therefore, the error is the pixel difference between the object and the background. 
         [0005]    At present, mean absolute difference (MAD) is the most widely used technique for determining whether video content exhibits moving object characteristics. This is accomplished by comparing a MAD value with a predetermined threshold value. The larger MAD value, the greater number of pixel differences caused by object movement, and the smaller MAD value, the smaller number of pixel differences caused by object movement. Therefore, conventional approaches use the size of the numerical value of MAD to determine whether there is object movement. 
         [0006]    However, in some special cases, for example: the objects the image moving rapidly back and forth, the moving object may be considered as stationary through the calculation of MAD, therefore, the mistaking movement will occurs. Blending the image in such mistaking movement, the end result will be image streaking. 
       BRIEF SUMMARY OF THE INVENTION 
       [0007]    Therefore, in order to solve these problems, one of the purposes of this invention is to provide an image processing apparatus can determine whether image exists noise, and solve mistaken problem of the prior art. 
         [0008]    One of the purposes of this invention is to provide an image processing apparatus. The image processing apparatus comprises a pixel buffer unit, a pixel difference calculating unit, a motion level determining unit, a blending value determining unit, and an output unit. The pixel buffer unit is used to store a plurality of first pixels of a previous image and a plurality of second pixels of a current image in a sampling window. The pixel difference calculating unit for calculating a pixel difference between the first pixel and the second pixel at a position corresponding to the first pixel, and outputting a plurality of pixel differences. The motion level determining unit, for deciding a blending value of a pixel of in the sampling window and determining whether the pixel is a static pixel or a moving pixel according to the pixel difference. The blending value determining unit coupled to the motion level determining unit, for determining a new blending value when the pixel is a static pixel. The output unit coupled to the blending value determining unit, for calculating the current image and the previous image using weighted average according to the blending value so as to generate and output an output image when the pixel is static pixel; and adding together weights of the current image and the previous image according to the new blending value to generate and output the output image when the pixel is a static pixel. 
         [0009]    According to another aspect, the image processing method of this invention includes the following steps: storing a plurality of first pixels of a previous image and a plurality of second pixels of a current image in a sampling window; calculating a difference value between the first pixel and the second pixel at a position corresponding the first pixel, and outputting a plurality of pixel differences; deciding a blending value of a pixel of in the sampling window, and determining the pixel static or moving pixel according to the pixel difference; determining a new blending value when the pixel is a static pixel; and calculating the current image and the previous image using weighted average according to the blending value so as to generate and output an output image when the pixel is a moving pixel, and calculating the current image and the previous image using weighted average according to the new blending value so as to generate and output the output image when the pixel is a static pixel. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0010]      FIG. 1  shows a diagram illustrating an image processing apparatus according to one embodiment of the invention; 
           [0011]      FIG. 2  shows a diagram of a field sequence; 
           [0012]      FIG. 3  shows a diagram illustrating a of an even field sequence and an odd field sequence; 
           [0013]      FIG. 4  shows a diagram of a current sampling window of a pixel; 
           [0014]      FIG. 5  shows a diagram illustrating pixel difference matrices according to one embodiment; 
           [0015]      FIG. 6  shows a diagram illustrating pixel difference matrices according to one embodiment; 
           [0016]      FIG. 7  shows a diagram illustrating an image processing apparatus according to one embodiment of the invention; 
           [0017]      FIG. 8  shows a diagram of a field sequence of center pixel; 
           [0018]      FIG. 9  shows a diagram of center pixel and neighbor pixel; 
           [0019]      FIGS. 10A and 10B  show a flowchart of an image processing method of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0020]    Please refer to  FIG. 1 ,  FIG. 1  shows a diagram illustrating an image processing apparatus according to one embodiment of the invention. The image processing apparatus comprises a pixel buffer unit  101 , a pixel difference calculating unit  102 , a motion level determining unit  103 , a blending value determining unit  104 , and an output unit  105 . 
         [0021]    The pixel buffer unit  101  receives an input image, stores a current image  11  and a previous image  12  of a sampling window. Wherein, current image  11  and the previous image  12  respectively have a plurality of pixels, and each pixel has a pixel value which includes a luma and a chroma. 
         [0022]    The pixel difference calculating unit  102  receives current image  11  and previous image  12 , and calculates the pixel difference of each pixel according to the pixel value of the same position of the current image  11  and the previous image  12 , to gather all the pixel difference to form a pixel difference matrix. 
         [0023]    Please also refer to  FIG. 2 ,  FIG. 2  shows a diagram of a field sequence. In one embodiment of the invention, a consecutive field sequence F is formed by interposing an even field, such as field F k−2 , field F k , and F k+2 , between a pair of odd fields, such as fields F k−1  and F k+1 . 
         [0024]    The current image is a kth field (or a (k+1)th field), and the previous image is a (k−2)th field (or a (k−1)th field). Therefore, also referring to  FIG. 3 , after the even field sequence F eve  and the odd field sequence F odd  can be separated into two independent field sequences, the image processing apparatus of this invention can be used. 
         [0025]    In this embodiment, the field F k  is the current image of the even field sequence F eve , and the field F k−2  is the previous image. Similarly, the field F k+1  is the current image of the odd field sequence F odd , and the field F k−1  is the previous image. The even field sequence F eve  and the odd field sequence F odd  can be processed simultaneously by two image processing apparatuses in parallel, or using time-divisional manner to the even field sequence F eve  and the odd field sequence F odd  by one image processing apparatus 
         [0026]    The pixel difference calculating unit  102  uses each pixel as a center point to establish a corresponding sampling window. For example (the even field sequence F eve  as an example), please also refer to  FIG. 4 ,  FIG. 4  shows a diagram of a current sampling window of a pixel. Wherein, the size of the sampling window, in this embodiment, is a predetermined value m×n. In  FIG. 4 , the corresponding sampling window is W 1  for pixel P 1 . For the pixel at a boundary of an image (e.g., P 2 ), the portions outside the image of  FIG. 4  can be represented by “0” to form a completed sampling window. This method will not affect the processing result due to the portions outside the previous image and the portions outside the image of  FIG. 4  are the same. 
         [0027]    It is to be noted that, current image  11  and previous image  12  can be a frame, the current image  11  can be a kth frame, and the current image  11  can be a (k−1)th frame. Other operational principles are the same as aforementioned, detail description is omitted here for sake of brevity. 
         [0028]    Please also refer to  FIG. 5 ,  FIG. 5  shows a diagram illustrating pixel difference matrices according to one embodiment (even field sequence F eve  as an example), the actual existence of the pixels combine into the pixel matrix of the current image  11  and the previous image  12 . In other words, the pixel difference calculating unit  102  uses a pixel A of the current image  11  as a center point to establish a corresponding current sampling window W A . Wherein, the sampling window size can be a predetermined value of m×n, or the size can be defined by user. In the present embodiment, it is assumed that the sampling window size is 3×3. 
         [0029]    Meanwhile, using a pixel B of the previous image  12  as a center point to establish a corresponding previous sampling window W B . Next, calculate corresponding pixel differences on current sampling window of each pixel and the previous sampling window of each pixel having corresponding location, and then collect all the pixel differences to form a pixel difference matrix. Wherein, the location of the pixel A in current image  11  corresponds to the location of the pixel B in previous image  12 . 
         [0030]    When pixel difference calculating unit  102  is configured to subtract pixel values of previous image  12  from corresponding pixel values of current image  11 , the calculation result will be the pixel difference matrix  13  shown in the  FIG. 5 . Alternatively, when the pixel difference calculating unit  102  is configured to subtract pixels values of a current image  11  from corresponding pixel values of a previous image  12 , the calculation result will be the pixel difference matrix  13 ′ shown in the  FIG. 5 . 
         [0031]    Pixel difference matrix  13  and  13 ′ have a plurality of pixel differences respectively. The pixel difference calculation unit  102  calculates the absolute value of the sum of all the pixel differences in pixel matrix  13  or  13 ′ and uses the absolute value to be the total pixel difference D of the sampling window. Then, the motion level determining unit  103  determines a blending value of the center pixel of the sampling window, and determines whether this center pixel is moving or not. 
         [0032]    In the present embodiment, assuming the sampling window size is 3×3, the pixel difference matrix  13  is configured to subtract the current image  11  from the previous image  12 , therefore, the pixel difference matrix  13 ′ is the result as shown in  FIG. 5 . 
         [0033]    The formula for calculating the total pixel difference of the sampling windows W A  and W B  by pixel difference calculation unit  102  as shown in following: 
         [0000]      Total pixel difference  D=| 10+8−1+0+13+8+8+2+13|=61, or
 
         [0000]      Total pixel difference  D=|− 10−8+1+0−13−8−8−2−13|=61
 
         [0034]    Four predetermined threshold values are configured in the motion level determining unit  103  according to the result calculated by the pixel difference calculating unit  102 . Four predetermined threshold values are a first threshold value TH 1 , a second threshold value TH 2 , a third threshold value TH 3 , and a forth threshold value TH 4 . The relations of the four predetermined threshold values as following: the first threshold value TH 1  is smaller than the second threshold value TH 2 ; the second threshold value TH 2  is smaller than the third threshold value TH 3 ; the third threshold value TH 3  is smaller than the forth threshold value TH 4 . 
         [0035]    When the total pixel difference D of the sampling windows W A , W B  is smaller than the first threshold value TH 1 , the blending value K of the pixel difference is configured as 0; when the total pixel difference D of the sampling windows W A , W B  is greater than the first threshold value TH 1  and smaller than the second threshold value TH 2 , the blending value K of the pixel difference is configured as 0.25; when the total pixel difference D of the sampling windows W A , W B  is greater than the second threshold value TH 2  and smaller than the third threshold value TH 3 , the blending value K of the pixel difference is configured as 0.5; when the total pixel difference D of the sampling windows W A , W B  is greater than the third threshold value TH 3  and smaller than the forth threshold value TH 4 , the blending value K of the pixel difference is configured as 0.75; when the total pixel difference D of the sampling windows W A , W B  is greater than the forth threshold value TH 4 , the blending value K of the pixel difference is configured as 1. 
         [0036]    In this embodiment, it is assumed that the first threshold value TH 1 , the second threshold value TH 2 , the third threshold value TH 3 , and the forth threshold value TH 4  are 20, 50, 100, and 150 respectively. By calculating the total pixel difference D of the sampling windows W A  and W B , the total pixel difference D is 61 in this embodiment, the total pixel difference D is greater than the second threshold value TH 2  and smaller than the third threshold value TH 3 , and the motion level determining unit  103  obtains the blending value K of the center pixel of the sampling window 0.5. 
         [0037]    It is noted that, the first threshold value TH 1 , the second threshold value TH 2 , the third threshold value TH 3 , and the forth threshold value TH 4  of the present invention can be configured according to user&#39;s requirement, present invention should not be limited. 
         [0038]    In this embodiment, the center pixel of the sampling window W A  is considered as a moving pixel when the total pixel difference D of sampling windows W A , W B  is greater than 50; the center pixel of the sampling window W A  is considered as a static pixel when the total pixel difference D of the sampling windows W A , W B  is smaller than 50. In other words, the center pixel of sampling window W A  is considered as a moving pixel when the total pixel difference D of sampling windows W A , W B  is greater than the second threshold value TH 2 ; the center pixel of sampling window W A  is considered as a static pixel when the total pixel difference D of sampling windows W A , W B  is smaller than the second threshold value TH 2 . 
         [0039]    In present embodiment, because the total pixel difference D of the sampling windows W A , W B  is greater than 50, the blending value determining unit  104  doesn&#39;t change the blending value K of center pixel of sampling windows W A , the blending value K of center pixel of sampling window W A  is still 0.5. 
         [0040]    The output unit  105  calculates the center pixel of sampling window W A  of current image and the center pixel of sampling window W B  of previous image, using weighted average according to the blending value K determined by blending value determining unit  104 , so as to output an output pixel of the output image. In other words, the output unit  105  calculates corresponding output pixel according to the blending value K of each pixel. When the output pixels of all the pixels being calculated, the output pixels are collected to form an output image  14 . Then feedback the output image to pixel buffer unit  101  so as to be a previous image  12  in next calculation. The formula for calculating the output pixel as shown in following: 
         [0000]      output pixel=corresponding pixel value of current image 11× K+corresponding pixel value of previous image  12×(1 −K )
 
         [0041]    The formula for calculating the output pixel of output image  14  as shown in following: 
         [0000]      output pixel=36×0.5+49×(1−0.5)=42.5
 
         [0042]    It is noted that, the output unit  105  uses the blending value K to calculates the weighted average of center pixel of sampling window W A  of current image  11  and center pixel of sampling window W B  of previous image  12  to output the output image  14  when the center pixel is a moving pixel; and the output unit  105  uses a new blending value K′ to calculates the weighted average of center pixel of sampling window W A  of current image  11  and center pixel of sampling window W B  of previous image  12  to output the output image  14  when the center pixel is a static pixel. 
         [0043]    Please refer to  FIG. 1  and  FIG. 6 ,  FIG. 6  shows a diagram illustrating a calculation sample of pixel difference matrices according to one embodiment. After the sampling window processed by the pixel difference calculation unit  102 , the formula for calculating the pixel difference of the sampling window as shown in following: 
         [0000]      Total pixel difference  D=| 8−1−10+13+8−4+2+13+20|=49, or
 
         [0000]      Total pixel difference  D=|− 8+1+10−13−8+4−2−13−20|=49
 
         [0044]    It is understood that the total pixel difference D is greater than the first threshold value TH 1  and smaller than the second threshold value TH 2 , and therefore the motion level determining unit  103  obtains the blending value K A  of center pixel of sampling window W A  of current image 0.25. 
         [0045]    When the total pixel difference D is smaller than the second threshold value TH 2 , the center pixel of the sampling window W A  is considered as a static pixel. In the present example, the total pixel difference D is smaller than 50 (the second threshold value TH 2 ), so the blending value determining unit  104  needs to change the blending value K A  of center pixel of sampling window W A  of current image  11  and configures a new blending value K A ′. 
         [0046]    To be noted that, the center pixel of sampling window W A  of current image  11  and the center pixel of sampling window W B  of current image  12  have corresponding positions, and respectively have blending value K A  and K B . 
         [0047]    Wherein, the blending value K B  of center pixel of sampling window W B , which blending value is finally determined by the blending value determining unit  104 . 
         [0048]    In this embodiment, the new blending value K A ′ is blending value K A  when blending values K A  and K B  are equal; and the new blending value K A ′ is blending value K B  when blending values K A  and K B  are not equal. 
         [0049]    In other words, when the blending value K A  of center pixel of sampling window W A  of current image  11  is equal to the blending value K B  of the center pixel of the sampling window W B  of the previous image  12 , the new blending value K A ′ of center pixel of sampling window W A  of current image  11  is the blending value K A ; when the blending value K A  of center pixel of sampling window W A  of current image  11  is not equal to the blending value K B  of center pixel of sampling window W B  of previous image  12 , the new blending value K A ′ of center pixel of sampling window W A  of current image  11  is the blending value K B . 
         [0050]    Output unit  105  outputs the output pixel which calculation manner as aforementioned, detailed description is omitted herein for sake of brevity. 
         [0051]    In one embodiment, when blending values K A  and K B  are equal, the new blending value K A ′ is the blending value K A ; and when the blending values K A  and K B  are different, the new blending value K A ′ is a calculation of the blending values K A  and K B . In another embodiment, the new blending value K A ′ is an average of the blending values K A  and K B . In the other embodiment, the new blending value K A ′ is rounding off the average of the blending values K A  and K B . 
         [0052]    Furthermore, in one embodiment, the new blending value K A ′ is always an average of he blending values K A  and K B . In another one embodiment, the new blending value K A ′ is always rounding off the average of the blending values K A  and K B    
         [0053]    Please refer to  FIG. 7 ,  FIG. 7  shows a diagram illustrating an image processing apparatus according to one embodiment of the invention. The difference between  FIG. 1  and  FIG. 7  is that the image processing apparatus  700  further comprises a counting unit  706  to count the pixel number of the static pixel from the sampling window of the previous and current images. 
         [0054]    Please refer to  FIG. 8 , in this embodiment (taking even field sequence F eve  as an example), the field F k  of the even field sequence F eve  is the current image, and the fields F k−2 , F k−4 , and F k−6  are the previous images. 
         [0055]    Assuming a sampling window size of 3×3, the same as above-mentioned, when the center pixel of the sampling window is determined as a moving pixel by motion level determining unit  103 , the blending value determining unit  104  does not change the blending value K of the center pixel of the sampling window of the current image. 
         [0056]    Otherwise, when the center pixel of the sampling window is determined as a static pixel, the counting unit  706  calculates plurality of blending values of center pixels of previous images, and records these blending values to determine whether corresponding pixel is moving or static. 
         [0057]    For example, fields F k−2            F k−4            F k−6  respectively have center pixels P k−2            P k−4            P k−6  corresponding to center pixel of sampling window of field F k , and these center pixels P k−2            P k−4            P k−6  have corresponding previous blending values K k−2            K k−4            K k− respectively. 
         [0058]    It is assumed that there are two static pixels, and one moving pixel among center pixels P k−2            P k−4            P k−6 . In this embodiment, when the number of static pixels is greater than a predetermined value (the predetermined value is 1 in the present embodiment), the new blending value K′ of the center pixel P k  of sampling window of fields F k  is the blending value K. On the other hand, if the number of static pixels is 1, and the number of moving pixels is 2 among the center pixels P k−2            P k−4            P k−6 . In this embodiment, when the number of the static pixels is equal to or smaller than a predetermined value (the predetermined value is still 1), the new blending K′ of center pixel P k  of sampling window of the fields F k  is previous blending value K k−2 . 
         [0059]    In one embodiment, when the number of the static pixel of the center pixels P k−2            P k−4            P k−6  is equal to or smaller than 1, the new blending K′ of center pixel P k  of sampling window of fields F k  is a calculation of the previous blending values K k−2  and K k−4 . The calculation is an average of the previous blending values K k−2  and K k−4 . In another embodiment, the new blending value K′ is rounding off the average of the blending values K k−2  and K k−4 . 
         [0060]    Please refer to  FIG. 9 , in one embodiment, field F k  is current image and field F k−2  is previous image. Assuming a sampling window size is still 3×3. The center pixel of sampling window in current image is C and which corresponding location for a center pixel of previous image is C′. Around the center pixel C′ has a plurality of adjacent previous pixels, which represented by oblique lines in  FIG. 9  (there are eight adjacent previous pixels in this embodiment.) and corresponding to a plurality of adacent previous blending values (there are eight adjacent blending values in this embodiment.). When the number of static pixel is equal to or smaller than 1, the new blending value K′ of center pixel of sampling window of field F k  is a calculation of the eight adjacent blending values and the previous blending values K k−2 . In one embodiment, the calculation is an average of the eight adjacent blending values and the previous blending values K k−2 . In another embodiment, new blending value K′ is rounding off the average of blending values K k−2  and the eight adjacent blending values. 
         [0061]    Output unit  105  outputs the output image according to the new blending value K′, which calculation as mentioned above, detail description is omitted herein for sake of brevity. 
         [0062]    Referring to  FIGS. 10A and 10B , an image processing method according to this invention is used to determine that an error between the current and the previous images is caused by a moving object of an image. The image processing method includes the steps as described below. 
         [0063]    In step S 90 , setting a sampling window with a size m×n. 
         [0064]    In step S 91 , receiving a current image and a previous image, wherein the current image is a kth frame (or kth field) and the previous image is a (k−1)th frame (or (k−1)th field). 
         [0065]    In step S 92 , calculating pixel differences between center pixels at corresponding locations in the current image and the previous image, and collecting all the pixel differences to form a pixel difference matrix which size is equal to the sampling window. 
         [0066]    In step S 93 , calculating total pixel difference D. 
         [0067]    In step S 94 , determining that the center pixel is static pixel or moving pixel according to the total pixel difference D. Moving on to Step S 95 , if the center pixel is moving pixel; moving on to Step S 97 , if the center pixel is static pixel; 
         [0068]    In step S 95 , determining the blending value K of the center pixel, determining the method of determination as above-mentioned embodiment, and then moving on to step S 96 . 
         [0069]    In step S 96 , calculating the output pixel according to the blending value K of each pixel. When calculating the output pixels of all the pixels, collecting the output pixels to form an output image, which is feedback for using as a previous image in the next calculation. The formula for calculating the output pixel as shown in following: 
         [0000]      output pixel=corresponding pixel value of current image× K +corresponding pixel value of previous image×(1 −K )
 
         [0070]    In step S 97 , determining the blending value K of the center pixel, determining the method of determination as above-mentioned embodiment, and then moving on to step S 98 . 
         [0071]    In step S 98 , determining the new blending value K′ of the center pixel, and determining the method of determination as above-mentioned embodiment, and then moving on to step S 99 . 
         [0072]    In step S 99 , calculating the output pixel according to the new blending value K′ of each pixel. When calculating the output pixels of all the pixels, collecting the output pixels to form an output image, which is feedback for using as a previous image in the next calculation. The formula for calculating the output pixel is shown in following: 
         [0000]      output pixel=corresponding pixel value of current image× K ′+corresponding pixel value of previous image×(1 −K′ )
 
         [0073]    In sum, in an image sequence, when there is a moving object in previous and subsequent images, the image processing apparatus and method of this invention are capable of accurately determining between whether the total pixel difference is raised by noise interference or object motion. Therefore, this invention can prevent from the situation that mistakenly deems a moving object as a noise interference so that cause a error occur when dealing with an image sequence.