Abstract:
A method and system for image chroma suppression of an image sensor system. Chroma suppression is performed to reduce the false color phenomena. The amount of chroma suppression is performed based on the strength of spatial frequency for a current processing pixel measured during edge detection. In determining the strength of spatial frequency of the current processing pixel, only a small number of green pixel values are needed, regardless of whether or not the current processing pixel is a green pixel within the Bayer pattern. As such, line buffers from the color interpolation module of an image sensor system can also be used for the purpose of chroma suppression.

Description:
FIELD OF THE INVENTION 
     The invention relates to image sensor chroma suppression, particularly to chroma suppression that relies on spatial frequency information without needing to increase the existing image sensor line buffer size. 
     BACKGROUND 
     In demosaicking Bayer pattern images captured by an image sensor, “false color” can be induced at or near places such as an edge, a checkered pattern or a stripe pattern. As such, a need exists for reducing the false color phenomena at or near edges, checkered patterns or stripe patterns. 
    
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
       The accompanying drawings which are incorporated in and form a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention: 
         FIG. 1  shows an image sensor chip for chroma suppression in accordance with one embodiment of the invention. 
         FIG. 2  shows some of the units involved for performing chroma suppression in accordance with one embodiment of the invention. 
         FIG. 3  shows a Bayer pattern having a green current processing pixel in accordance with one embodiment of the invention. 
         FIG. 4  shows another Bayer pattern having a non-green current processing pixel in accordance with one embodiment of the invention. 
         FIG. 5  is flow chart that outlines steps for performing chroma suppression in accordance with one embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION 
     Reference is made in detail to embodiments of the invention. While the invention is described in conjunction with the embodiments, the invention is not intended to be limited by these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of the invention, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, as is obvious to one ordinarily skilled in the art, the invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so that aspects of the invention will not be obscured. 
       FIG. 1  shows an image sensor chip  100  for chroma suppression in accordance with one embodiment of the invention. 
     Image sensor chip  100  comprises an image sensor array  110  and an image processing component  150 . Image sensor array  110  is adapted for capturing and digitizing images to be processed by image processing component  150 . Image sensor chip  100  is typically used within an image-capturing device that could be, but is not limited to, a digital camcorder, a digital still camera, a videophone, a video conferencing equipment, a PC camera, a cell phone, or a security monitor. 
     Image sensor array  110  comprises an image sensor  115 , and an analog-to-digital converter (ADC)  120 . Image processing component  150  comprises a color processing component  155 , a compression engine  160  and a transceiver  165 . 
     Images are captured by sensor  115 , then digitized by ADC  120  into pixel values to be transmitted to image processing component  150 . Then, the pixel values are color processed by color processing component  155 . Color processing component  155  typically performs digital image processing that could include, but is not limited to, auto exposure control, color interpolation, edge detection, chroma suppression, auto white balancing, color correction and image sharpening. In turn, the color processed pixel values undergo compression performed by compression engine  160 . The compressed image data is then transmitted out of image sensor chip by transceiver  165 . 
     Referring now to  FIG. 2  in view of  FIG. 1 ,  FIG. 2  shows units included in color processing component  155  in accordance with one embodiment of the invention. These units comprise a color interpolation unit  210 , an edge detection unit  220 , a white balance gain unit  230 , a chroma suppression unit  240 , a color correction unit  250  and a line buffer for color interpolation unit  210 . 
     Specifically, edge detection unit  220  and chroma suppression unit  240  are involved for chroma suppression in accordance with the present embodiment. Furthermore, also shown coupled to both color interpolation unit  210  and edge detection unit is line buffer  290  that is adapted for storing pixel values to be used for performing interpolation of pixel values for a current processing pixel. Line buffer  290  can also to be used for chroma suppression in the present embodiment. As such, in the present embodiment, performing chroma suppression does not require extra line buffers in addition to that of color interpolation unit  210 . 
     Chroma suppression is adapted to reduce false color that occurs at or near an edge, a checkered pattern, or a stripe pattern in an image. As such, before chroma suppression is performed, these problematic locations needing chroma suppression are first identified. In turn, chroma suppression is performed on these problematic locations. 
     Specifically, in the present embodiment, edge detection unit  220  is adapted to indicate whether a current processing pixel is at or near an edge, a checkered pattern, or a stripe pattern. A pixel at or near these problematic image patterns typically has high spatial frequency. As such, edge detection unit  220  is adapted to determine whether the current processing pixel is in a high spatial frequency region or not in order to determine whether the current processing pixel is at or near an edge, a checkered, pattern, or a stripe pattern. If the current processing pixel is in a high spatial frequency region, then chroma suppression unit  240  is triggered to perform chroma suppression at the current processing pixel by reducing the intensity of the current processing pixel. The degree of intensity reduction can be flexibly adjusted. 
     More specifically, edge detection unit  220  is adapted to detect the presence of an edge, a checkered pattern, or a strip pattern without having to use pixel values other than some of pixel values already available in line buffer  290 , wherein pixels values in line buffer  290  are originally intended for performing color interpolation by color interpolation unit  210 . In so doing, no extra line buffer besides line buffer  290  is required by edge detection unit  220  in locating problematic locations in an image. As such, in the present embodiment, chroma suppression can be performed without extra line buffer besides line buffer  290 . 
     Moreover, edge detection unit  220  does not require any non-green pixel values in determining spatial frequency of the current processing pixel. Whether the current processing pixel is a green pixel or not, edge detection unit  220  can use pixel values of green pixels that surround the current processing pixel. 
     In view of  FIGS. 1–2 ,  FIG. 3  shows a Bayer pattern  300  having a green current processing pixel M 22  that might undergo chroma suppression in accordance with one embodiment of the invention. 
     Bayer pattern  300  comprises 16 pixels (M 00 , M 01 , . . . and M 33 ) as shown, wherein green pixel M 22  is the current processing pixel. These 16 pixels are already available for access from line buffer  290  because they are also used by color interpolation unit  210  for performing color interpolation. 
     Specifically, pixels M 00 –M 33  are arranged as a 4 by 4 Bayer pattern having green pixel M 22  as current processing pixel. As such, besides current processing pixel M 22  being a green pixel, its surrounding pixels M 00 , M 02 , M 11 , M 13 , M 20 , M 22 , M 31  and M 33  are also green pixels. 
     Continuing with  FIG. 3 , given a current processing pixel like green pixel M 22 , edge detection unit  220  uses the surrounding green pixels (i.e., M 00 , M 02 , M 11 , M 13 , M 20 , M 31  and M 33 ) of green pixel M 22  to determine whether current processing pixel M 22  is in a high frequency region or not. If current processing green pixel M 22  is found to be in a high frequency region, then chroma suppression is performed on current processing pixel M 22  by chroma suppression unit  240 , thereby reducing undesirable false color. 
     Within Bayer pattern  300 , only the values of green pixels surrounding current processing pixel M 22  need to be used by edge detection unit  220  in determining whether chroma suppression is to be performed or not. As these green pixel values are already accessible from line buffer  290 , edge detection unit  220  can share the same line buffer  290  with color interpolation unit  210 . 
     As understood herein, the invention need not be limited to using only green pixels surround green pixel M 22 . For example, in another embodiment, values of non-green pixels (stored in line buffer  290 ) surrounding green pixel M 22  are used. 
     Also, as understood herein, the Bayer pattern used need not be limited to a 4×4 block. For example, in another embodiment, a 5×5 block is used. 
     In view of  FIGS. 1–2 ,  FIG. 4  shows another Bayer pattern  400  having a non-green pixel M 22  as a current processing pixel that might undergo chroma suppression in accordance with one embodiment of the invention. 
     Bayer pattern  400  comprises 16 pixels (M 00 , M 01 , . . . and M 33 ), wherein M 22  is a current processing pixel. These 16 pixels are already available for access from line buffer  290  because they are also used by color interpolation unit  210  for performing color interpolation. 
     Specifically, pixels M 00 –M 33  are arranged as a 4 by 4 Bayer pattern having a non-green (i.e., blue or red) pixel as current processing pixel M 22 . As such, because current processing pixel M 22  is a non-green pixel, its surrounding pixels M 10 , M 30 , M 01 , M 21 , M 12 , M 32 , M 03  and M 23  are green pixels. 
     Continuing with  FIG. 4 , given a current processing pixel like non-green pixel M 22 , edge detection module  220  uses the surrounding green pixels (i.e., M 0 , M 30 , M 01 , M 21 , M 12 , M 32 , M 03  and M 23 ) of non-green current processing pixel M 22  to determine whether current processing pixel M 22  is in a high frequency region or not. If current processing non-green pixel M 22  is found to be in a high frequency region, then chroma suppression is performed on current processing pixel M 22  by chroma suppression module  240 , thereby reducing undesirable false color. 
     Within Bayer pattern  400 , only green pixels surrounding non-green current processing pixel M 22  need to be used by edge detection unit  220  in determining whether chroma suppression is to be performed or not. As these green pixel values are already accessible from line buffer  290 , edge detection unit  220  can share the same line buffer  290  with color interpolation unit  210 . 
     As understood herein, the invention need not be limited to using only green pixels surround green pixel M 22 . For example, in another embodiment, values of non-green pixels (stored in line buffer  290 ) surrounding non-green pixel M 22  are used. 
     Also, as understood herein, the Bayer pattern used need not be limited to a 4×4 block. For example, in another embodiment, a 5×5 block is used. 
     In view of  FIGS. 3–4 ,  FIG. 5  shows a flow chart  500  outlining steps for performing chroma suppression in accordance with one embodiment of the invention. 
     In query step  505 , a check is made to determine if a current processing pixel M 22  within a 4×4 Bayer pattern (as shown in  FIGS. 3 and 4 ) is a green pixel or non-green pixel. If M 22  is a green pixel, then Bayer pattern to be examined is Bayer pattern  300  shown in  FIG. 3 . In turn, step  510  is performed next. Otherwise, if M 22  is a non-green pixel, then Bayer pattern to be examined is Bayer pattern  400  shown in  FIG. 4 . In turn, step  520  is performed next. 
     In step  510  (for M 22  being a green pixel), high pass filtering is performed on green pixels surrounding the current processing pixel M 22 . 
     Specifically, the first order pixel value sums are calculated before the spatial frequency of green pixel M 22  is calculated. These first order pixel value sums are calculated with the green pixel values (i.e., M 00 , M 20 , M 11 , M 31 , M 02 , M 22 , M 13 , M 33 ) within the Bayer pattern (see Bayer pattern  300  of  FIG. 3 ). 
     First, a first plurality of 4 first order pixel value sums V[0] to V[3] are calculated, wherein for i=0 to 3 an i-th pixel value sum V[i] characterizes the pixel value sum for a pair of buffered green pixel values associated respectively with a pair of green pixels lying on the i-th vertical column of the Bayer pattern region. V[i] can be considered as an “average” of pixel values for the two green pixels lying in the i-th column of the Bayer pattern region. 
     Second, a second plurality of 4 first order pixel value sums H[0] to H[3] are calculated, wherein for j=0 to 3 an j-th pixel value sum H[j] characterizes the pixel value sum for a pair of buffered green pixel values associated respectively with a pair of green pixels lying in the j-th horizontal column of said Bayer pattern region. H[j] can be considered as an “average” of pixel values for the two green pixels lying in the j-th column of the Bayer pattern region. 
     Specifically, before the spatial frequency of green pixel M 22  is calculated, the first order pixel value sums are calculated as follow:
 
 V[ 0]=( M 00 +M 20) as an “average” of values of  M 00 and  M 20;
 
 V[ 1]=( M 11 +M 31) as an “average” of values of  M 11 and  M 31;
 
 V[ 2]=( M 02 +M 22) as an “average” of values of  M 02 and  M 22;
 
 V[ 3]=( M 13 +M 33) as an “average” of values of  M 13 and  M 33;
 
 H[ 0]=( M 00 +M 02) as an “average” of values of  M 00 and  M 02;
 
 H[ 1]=( M 11 +M 13) as an “average” of values of  M 11 and  M 13;
 
 H[ 2]=( M 20 +M 22) as an “average” of values of  M 20 and  M 22; and
 
 H[ 3]=( M 31 +M 33) as an “average” of values of  M 31 and  M 33.
 
     In turn, step  530  is performed next. 
     In step  520  (for M 22  being a non-green pixel), high pass filtering is performed on green pixels surrounding the current processing pixel M 22 . 
     Specifically, before the spatial frequency of non-green pixel M 22  is calculated, the first order pixel value sums are calculated before the spatial frequency of non-green pixel M 22  is calculated. These first order pixel value sums are calculated with the green pixels (i.e., M 0 , M 30 , M 01 , M 21 , M 12 , M 32 , M 03 , M 23 , M 01 , M 03 , M 0 , M 12 , M 21 , M 23 , M 30  and M 32 ) within the Bayer pattern (see Bayer pattern  400  of  FIG. 4 ). 
     First, a first plurality of 4 first order pixel value sums V[0] to V[3] are calculated, wherein for i=0 to 3 an i-th pixel value sum V[i] characterizes the pixel value sum for a pair of buffered green pixel values associated respectively with a pair of green pixels lying on the i-th vertical column of said Bayer pattern region. V[i] can be considered as an “average” of pixel values for the two green pixels lying in the i-th column of the Bayer pattern region. 
     Second, a second plurality of 4 first order pixel value sums H[0] to H[3] are calculated, wherein for j=0 to 3 an j-th pixel value sum H[j] characterizes the pixel value sum for a pair of buffered green pixel values associated respectively with a pair of green pixels lying in the j-th horizontal column of said Bayer pattern region. H[j] can be considered as an “average” of pixel values for the two green pixels lying in the j-th column of the Bayer pattern region. 
     Specifically, before the spatial frequency of green pixel M 22  is calculated, the first order pixel value sums are calculated as follow:
 
 V[ 0]=( M 10 +M 30) as an “average” of values of  M 10 and  M 30;
 
 V[ 1]=( M 01 +M 21) as an “average” of values of  M 01 and  M 21;
 
 V[ 2]=( M 12 +M 32) as an “average” of values of  M 12 and  M 32;
 
 V[ 3]=( M 03 +M 23) as an “average” of values of  M 03 and  M 23;
 
 H[ 0]=( M 01 +M 03) as an “average” of values of  M 01 and  M 03;
 
 H[ 1]=( M 10 +M 12) as an “average” of values of  M 10 and  M 12;
 
 H[ 2]=( M 21 +M 23) as an “average” of values of  M 21 and  M 23; and
 
 H[ 3]=( M 30 +M 32) as an “average” of values of  M 30 and  M 32.
 
     In turn, step  530  is performed next. 
     In step  530 , noise reduction is performed. 
     Specifically, the second order pixel value differences are calculated from the first order pixel value differences calculated either in step  510  (for M 22  being a green pixel) or in step  520  (for M 22  being a non-green pixel). 
     More specifically, the second order pixel value sums are calculated as follow:
 
 V 0avg=( V[ 0 ]+V[ 2]) as an “average” of  V[ 0] and V[2];
 
 V 1avg=( V[ 1 ]+V[ 3]) as an “average” of  V[ 1] and  V[ 3];
 
 H 0avg=( H[ 0 ]+H[ 2]) as an “average” of  H[ 0] and  H[ 2]; and
 
 H 1avg=( H[ 1 ]+H[ 3]) as an “average” of  H[ 1] and  H[ 3].
 
     In step  540 , preliminary values of spatial frequency for the current processing pixel are calculated. These preliminary values are the third order pixel value differences as calculated from the second pixel value sums calculated in step  530 . 
     Specifically, the third order pixel value differences are calculated as follow:
 
 V max=| V 0avg− V 1avg| as a preliminary value of spatial frequency; and
 
 H max=| H 0avg− H 1avg| as a preliminary value of spatial frequency.
 
     Vmax and Hmax will be used in determining whether or not current processing pixel M 22  is near an edge, a checkered pattern, or a stripe pattern, and thus in a high spatial frequency region. 
     In step  550 , a spatial frequency value SF[1] is assigned to current processing pixel M 22 . Specifically, SF[1] is defined as:
 
 SF [1]=MAX( V max,  H max).
 
     In step  560 , noise reduction is performed on current processing pixel M 22  by fine-tuning spatial frequency SF[1] obtained in step  550 . This noise reduction step adjusts the spatial frequency as follow:
 
 SF[ 2]=MIN(255, MAX(0, ( SF−T ))).
 
     As understood herein, the quantity SF[2] indicates if current processing pixel M 22  is in a high spatial frequency region or not. Specifically, large value of SF[2] indicates higher spatial frequency for current processing pixel M 22 . T is a parameter for tolerating low spatial deviation, wherein the range of T is from 0 to 255. Again, whether step  530  is performed following step  510  or step  520 , only pixel values of green pixels (already available from the line buffer used for interpolation) need to be used in determining the degree of spatial frequency intensity for current processing pixel M 22 . As such, exiting line buffer space of the color interpolation unit of an image sensor can be used. In turn, in evaluating whether or not M 22  is in a high spatial frequency region in need of chroma suppression, the same line buffers in the color interpolation unit of an image sensor can be used for the purpose of chroma suppression. 
     As understood herein, the invention is not limited to the formula listed in step  505 – 550  of the present embodiment. The above listed formula and their associated numerical values are for demonstrate purpose only. For example, in another embodiment of the invention, other numerical values can be used in place of the numerical values listed in steps  505 – 570 . 
     In query step  570 , a test is made to check if SF[2] indicates current processing pixel M 22  as being in a high spatial frequency region. If yes, then step  580  is performed. Otherwise, step  580  is bypassed. 
     Specifically, SF[2] is compared to a threshold value T. If SF[2]&gt;T, then SF[2] indicates that M 22  is in a high spatial frequency region. 
     In step  580 , chroma suppression is performed on current processing pixel M 22  in accordance with the strength of SF[2] as calculated in step  560 . Specifically, chroma suppression is performed by reducing the chromatic saturation of current processing pixel M 22 . 
     Assuming the spatial frequency of M 22  is greater than the threshold (i.e., SF[2]&gt;T), an attenuation factor F for chroma suppression is given as F=MIN(255, MAX(0,255−SF[2]*W)), wherein W is a parameter that quantifies the strength of chroma suppression. 
     Then, transformation from RGB space to YUV space is performed as follow:
 
 Y =(77 *R +150 *G +29 *B )/256;
 
 U=(− 44 *R −87 G+ 131 *B )/256; and
 
 V =(131 *R −110 *G −21 *B )/256.
 
As understood herein, Y is the luminance of the current processing pixel. U and V are the chroma components of current processing pixel M 22 , wherein U and V are used to determine the original chromatic saturation of current processing pixel M 22 . Moreover, because chroma suppression is understood to be the reduction of the original chromatic saturation of the current processing pixel, chroma suppression is performed by reducing U and V respectively into U′ and V′, wherein U′ and V′ are used to determine a new chromatic saturation for current processing pixel M 22 . In so doing, the new chromatic saturation determined from U′ and V′ is reduced relative to the original chromatic saturation, thereby achieving chromatic suppression for current processing pixel M 22 .
 
     Continuing with step  580 , the attenuation factor F is rescaled as F/256 (i.e., a number between 0 and 1) to perform chroma suppression by reducing U and V as follow:
 
 U′=U *( F/ 256); and
 
 V′=V *( F/ 256).
 
Again, U′ and V′ are the reduced chroma components of the current processing pixel. As such, the original chroma saturation of the current processing pixel is reduced to achieve chroma suppression on the current processing pixel.
 
     In turn, transformation from YUV space back to RGB space is performed using reduced U′ and V′ as follow:
 
 R=Y +((351 *V ′)/256;
 
 G=Y −((178 *V′ +86 *U′ )/256;
 
 B=Y +((443 *U′ )/256.
 
     As understood herein, chroma suppression in step  580  need not use the listed formula in the present embodiment. The above listed formula and their associated numerical values are for demonstrate purpose only. For example, in another embodiment of the invention, other numerical values can be used in place of the numerical values listed in step  580 . 
     The foregoing descriptions of specific embodiments of the invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to explain the principles and the application of the invention, thereby enabling others skilled in the art to utilize the invention in its various embodiments and modifications according to the particular purpose contemplated. The scope of the invention is intended to be defined by the claims appended hereto and their equivalents.