Abstract:
A digital image processing apparatus and a method thereof. The apparatus includes a CCD to photoelectrically transform an optical image which is imaged through a lens part, using a mosaic color filter array pattern, a buffer to store a color data output from the CCD by a pixel, in a predetermined unit, and an ADSE logic to color-interpolate a spatially missing color data by adaptively applying a luminance significance element value to a certain color interpolation method, the luminance significance element value indicating contribution of each color to an entire luminance with respect to each pixel color data stored in the buffer.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     This application claims the benefit under 35 U.S.C. § 119(a) from Korean Patent Application No. 2003-94376, filed Dec. 22, 2003 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference and in its entirety.  
       BACKGROUND OF THE INVENTION  
       [0002]     1. Field of the Invention  
         [0003]     The present general inventive concept relates to a digital image processing apparatus, and a method and system thereof. More specifically, the present general inventive concept pertains to a digital image processing apparatus, which can calculate spatially missing color data using color information of neighboring pixels with respect to an output image of a charge coupled device (CCD) adopting a color filter array, and a method thereof.  
         [0004]     2. Description of the Related Art  
         [0005]     In general, a digital image processing device such as a digital still camera or a digital video camera processes digital images using a single image sensor and adopts a mosaic color filter array pattern. Since an image sensor, such as a charge coupled device (CCD), basically measures only the intensity of light, a color filter array is used to present colors, and a single color is allotted to each pixel.  
         [0006]     Missing color data needs to be calculated using color information of neighboring pixels with respect to an output image of the CCD using the color filter array. This process is referred to as a color interpolation or a color demosaicing algorithm.  
         [0007]      FIG. 1  is a block diagram illustrating an example of the conventional digital image processing device. As shown in  FIG. 1 , the digital image processing device includes a lens part  10 , a CCD  20 , a buffer  30 , a digital signal processor (DSP)  40 , and an output part  50 .  
         [0008]     The lens part  10  includes a zoom lens for enlarging and reducing magnification of an object, a focus lens for adjusting focus of the object, and an iris for adjusting the intensity of radiation. The CCD  20 , which is used as a coupled device, photoelectrically transforms a photographed image into an electrical signal. The buffer  30  stores the photoelectrically-transformed image.  
         [0009]     The DSP  40  interpolates color of output data from the RGB mosaic CCD  20 , which has one of red (R), green (G), and blue (B) color at each pixel, so that each pixel can have all of the color data of R, G and B. The output part  50  converts the interpolated data of the DSP  40  to a displayable signal and outputs the converted signal.  
         [0010]      FIG. 2  is a detailed block diagram of the DSP  40  in  FIG. 1 . Referring to  FIGS. 2 and 4 , an adaptive interpolation logic  42  interpolates colors by using green color G which is similar to luminance. For example, in the RGB mosaic CCD  20  having only one of red (R), green (G), and blue (B) color at each pixel as shown in  FIG. 3 , the G 33  is interpolated with respect to the pixel R 33  based on the following equation.  
                     G   ⁢           ⁢   33     =     [       G   ⁢           ⁢   32     +       (       R   ⁢           ⁢   33     -     R   ⁢           ⁢   31       )     /   2     +       (       G   ⁢           ⁢   34     +       (       R   ⁢           ⁢   33     -     R   ⁢           ⁢   35       )     /   2       ]     /   2                     =         (       G   ⁢           ⁢   32     +     G   ⁢           ⁢   34       )     /   2     +       (         -   R     ⁢           ⁢   31     +     2   ⁢           *           ⁢   R   ⁢           ⁢   33     -     R   ⁢           ⁢   35       )     /   4                     [     Equation   ⁢           ⁢   1     ]               
         [0011]     In Equation 1, G 33  is calculated by adding a peak component of R to a linear interpolation value of G to thus generate a high-resolution signal and reduce a “zipper” effect.  FIG. 4  depicts such a color interpolation. The signal of high resolution is generated and the “zipper” effect is reduced through the color interpolation method. Difference interpolation logics  46   a  and  46   b  each interpolates a difference from G output from the adaptive interpolation logic  42  with respect to R and B using the linear interpolation, and interpolates R and B by adding G output from the adaptive interpolation logic  42 . That is, when R and B are interpolated, high frequency of G having a high frequency band is added to realize the high resolution. The above conventional digital image processing device is disclosed in US Publication No. 2003/052981, published on Mar. 20, 2003.  
         [0012]     When the conventional digital image system interpolates G, the peak component of R or B is used to enhance the resolution. Similarly, in interpolating R and B, the peak component of G is used to enhance the resolution. However, when only the peak component is used without considering influence on a luminance component, the “zipper” effect is not reduced or the resolution is not enhanced efficiently since the reduction of the “zipper” effect or the resolution enhancement is closely associated with the luminance component.  
       SUMMARY OF THE INVENTION  
       [0013]     Accordingly, the present general inventive concept provides a digital image processing apparatus capable of realizing high resolution images and reducing the so-called “zipper” effect by interpolating colors in consideration of contribution of each color with respect to the entire luminance, and a method thereof.  
         [0014]     Additional aspects and advantages of the present general inventive concept will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the general inventive concept.  
         [0015]     The foregoing and/or other aspects and advantages of the present general inventive concept are achieved by providing a digital image processing apparatus including a charge coupled device (CCD) to photoelectrically transform an optical image, which is imaged through a lens part, using a mosaic color filter array pattern, a buffer to store color data output from the CCD by a pixel into a predetermined unit, and an adaptive dual slope estimation (ADSE) logic to color-interpolate spatially missing color data by adaptively applying a luminance significance element value to a certain color interpolation method, the luminance significance element value indicating contribution of each color to an entire luminance with respect to each pixel color data stored in the buffer.  
         [0016]     The digital image processing apparatus may further include an output part to convert the color data interpolated in the ADSE logic to a displayable signal and output the displayable signal.  
         [0017]     The ADSE logic can perform the color interpolation by adding a component calculated by applying the certain color interpolation method with respect to color data of pixels neighboring a pixel to be interpolated, with a value interpolated by adaptively applying the luminance significance element value to a peak value of other color data of the neighboring pixels.  
         [0018]     The CCD may be one of a YMCG mosaic sensor, an RGB mosaic sensor, a YCGW mosaic sensor and a YCG mosaic sensor.  
         [0019]     The certain color interpolation method may be one of a linear interpolation, a bilinear interpolation, a Cubic interpolation and a Polyphase interpolation.  
         [0020]     The foregoing and/or other aspects and advantages of the present general inventive concept may also be achieved by providing a digital image processing method including photoelectrically transforming an optical image, which is imaged through a lens part, using a mosaic color filter array pattern, storing color data, output from a charge coupled device (CCD) by a pixel, in a predetermined unit, and color-interpolating a spatially missing color data by adaptively applying a luminance significance element value to a certain color interpolation method, the luminance significance element value indicating contribution of each color to an entire luminance with respect to each pixel color data stored in the buffer.  
         [0021]     The method may further include converting the color data interpolated in the ADSE logic to a displayable signal and outputting the displayable signal.  
         [0022]     The operation of color-interpolating the color data can perform the color interpolation by adding a component calculated by applying the predetermined color interpolation method with respect to color data of pixels neighboring a pixel to be interpolated, with a value interpolated by adaptively applying the luminance significance element value to a peak value of other color data of the neighboring pixels. The predetermined unit may be one of a field and a frame. The certain color interpolation method may be one of, a linear interpolation, a bilinear interpolation, a Cubic interpolation and a Polyphase interpolation. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0023]     These and/or other aspects and advantages of the present general inventive concept will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:  
         [0024]      FIG. 1  is a block diagram illustrating a conventional digital image processing device;  
         [0025]      FIG. 2  is a detailed diagram illustrating the DSP of  FIG. 1 ;  
         [0026]      FIG. 3  is a view illustrating a configuration of the RGB mosaic DDC;  
         [0027]      FIG. 4  is a graph illustrating the color interpolation by the conventional digital image processing device;  
         [0028]      FIG. 5  is a block diagram illustrating a digital image processing apparatus according to an embodiment of the present general inventive concept;  
         [0029]      FIG. 6  is a flowchart illustrating exemplary operations of the digital image processing apparatus of  FIG. 5 , according to an embodiment of the present general inventive concept;  
         [0030]      FIG. 7  is a view illustrating a configuration of a YMCG mosaic CCD;  
         [0031]      FIG. 8  is a graph illustrating the spectrum sensitivity of the CCD of  FIG. 7 ;  
         [0032]      FIG. 9  is a view illustrating the color interpolation of the digital image processing apparatus according to an embodiment of the present general inventive concept; and  
         [0033]      FIG. 10  is a view illustrating configurations of YMCG, YCG, YCGW, and RGB mosaic CCDs. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0034]     Reference will now be made in detail to the embodiments of the present general inventive concept, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present general inventive concept while referring to the drawing figures.  
         [0035]      FIG. 5  is a block diagram illustrating a digital image processing apparatus according to an embodiment of the present general inventive concept. Referring to  FIG. 5 , the digital image processing apparatus includes a lens part  100 , a charge coupled device (CCD)  120 , a buffer  130 , an adaptive dual slope estimation (ADSE) logic  140 , and an output part  150 .  
         [0036]     The lens part  100  has a zoom lens to enlarge and reduce magnification of an object, a focus lens to adjust focus of an object, and an iris to adjust the intensity of radiation. The CCD  120 , which is used as a coupled device, photoelectrically transforms a photographed image into an electrical signal using a mosaic color filter array pattern. The buffer  130  stores the photoelectrically-transformed data by a frame or a field.  
         [0037]     The ADSE logic  140  performs color interpolation by adaptively applying a luminance significance of each color with respect to data of the CCD  120  which has a single color data at each pixel stored in the buffer  130  so that each pixel has all of the color data. The output part  150  converts the color data interpolated in the ADSE logic  140  to a displayable signal and outputs the converted signal.  
         [0038]      FIG. 6  is a flowchart illustrating exemplary operations of the digital image processing apparatus of  FIG. 5 , according to another embodiment of the present general inventive concept. Referring  FIGS. 5 and 6 , the CCD  120  photoelectrically transforms an image photographed by the lens part  100  into an electric signal (operation S 200 ). If the CCD  120  is a YMCG mosaic CCD as shown in  FIG. 7 , the color data is processed based on the following equations. 
   Y=G+Cy+Mg+Ye      Cr =( Mg+Ye )−( G+Cy )    Cb =( Mg+Cy )−( G+Ye )  [Equation 2] 
         [0039]     The buffer  130  stores the photoelectrically transformed data which is output from the CCD  120  by a frame or a field (operation S 205 ).  
         [0040]     The ADSE logic  140  color-interpolates the output data of the CCD  120 , which has only one of yellow (Ye), magenta (Mg), cyan (Cy) and green (G) at each pixel, so that each pixel has all color data of Ye, Mg, Cy and G For this interpolation, an element value of the luminance significance is calculated (operation S 210 ).  
         [0041]     The element value of the luminance significance is calculated as below. Ye_Factor, Mg_Factor, Cy_Factor and G_Factor are calculated based on the following equations through integration in a wavelength range which belongs to a visible range as shown in  FIG. 8 .  
                   Ye   —     ⁢   Factor     =       ∫         Ye   _     ⁡     (   λ   )       ⁢     ⅆ   λ           ∫         Y   _     ⁡     (   λ   )       ⁢     ⅆ   λ             ⁢     
     ⁢         Mg   —     ⁢   Factor     =       ∫         Mg   _     ⁡     (   λ   )       ⁢     ⅆ   λ           ∫         Y   _     ⁡     (   λ   )       ⁢     ⅆ   λ             ⁢     
     ⁢         Cy   —     ⁢   Factor     =       ∫         Cy   _     ⁡     (   λ   )       ⁢     ⅆ   λ           ∫         Y   _     ⁡     (   λ   )       ⁢     ⅆ   λ             ⁢     
     ⁢         G   —     ⁢   Factor     =       ∫         G   _     ⁡     (   λ   )       ⁢     ⅆ   λ           ∫         Y   _     ⁡     (   λ   )       ⁢           ⁢     ⅆ   λ                     [     Equation   ⁢           ⁢   3     ]             
 
         [0042]     The luminance significance element values are obtained for the color interpolation with respect to each color, using the Ye_Factor, Mg_Factor, Cy_Factor and G_Factor calculated through Equation 3 and the following equation. 
 
 Kyc =( Ye _Factor/Luminance)/( Cy _Factor/Luminance)= Ye _Factor/ Cy _Factor 
 
 Kcy =( Cy _Factor/Luminance)/(Ye_Factor/Luminance)= Cy _Factor/ Ye _Factor 
 
 Kgm =( G _Factor/Luminance)/( Mg _Factor/Luminance)= G _Factor/ Mg _Factor 
 
 Kmg =( Mg _Factor/Luminance)/( G _Factor/Luminance)= Mg _Factor/ G _Factor  [Equation 4]
 
         [0043]     The ADSE logic  140  interpolates colors by adaptively applying the calculated luminance significance element value with respect to the data of the CCD  120  having only one color data for each pixel stored in the buffer  130  so as to obtain all color data for each pixel (operation S 215 ). That is, the luminance significance element values calculated through Equation 4 are reflected on the peak values of the related color, and is added to the calculated components through, for example, the linear interpolation to thus interpolate the color data which is spatially missing. For example, Ye 33  at the pixel Cy 33  of  FIG. 7  is calculated based on the following equation.  
               Ye   ⁢           ⁢   33     =     [     (         Ye   ⁢           ⁢   32     +     Kyc   ⁢           *           ⁢       (       Cy   ⁢           ⁢   33     -     Cy   ⁢           ⁢   31       )     /   2       +       (       Ye   ⁢           ⁢   34     +     Kyc   ⁢           *     
     ⁢           ⁢       (       Cy   ⁢           ⁢   33     -     Cy   ⁢           ⁢   35       )     /   2         ]     /   2       ⁢     
     ⁢           =         (       Ye   ⁢           ⁢   32     +     Ye   ⁢           ⁢   34       )     /   2     +     Kyc   ⁢           *           ⁢       (         -   Cy     ⁢           ⁢   31     +     2   ⁢           *     
     ⁢           ⁢   Cy   ⁢           ⁢   33     -     Cy   ⁢           ⁢   35       )     /   4                       [     Equation   ⁢           ⁢   5     ]             
 
         [0044]     In Equation 5, (Ye 32 +Ye 34 ) denotes a component calculated through the linear interpolation, and (−Cy 31 +2*Cy 33 −Cy 35 )/4 corresponds to the peak value. The interpolation is performed by adaptively applying the luminance significance element value to the peak value as shown in  FIG. 9 . Specifically, K indicates the luminance significance element value. A line between Cy 33  and Cy 31  and a line between Cy 33  and Cy 35  are shifted due to K to the location of Ye 33  to be interpolated. Accordingly, the so called “zipper” effect is reduced and the resolution is enhanced efficiently as compared with the conventional method of  FIG. 4 .  
         [0045]     The output part  150  converts the data interpolated in the ADSE logic  140  to a displayable data and outputs the converted data. The interpolated color data can then be displayed (operation S 220 ).  
         [0046]     The YMCG mosaic CCD is exemplified in the above embodiment, but other mosaic CCD sensors may be used such as RGB mosaic CCD, YCGW mosaic CCD of  FIG. 10  or YCG mosaic CCD. For example, if the RGB mosaic sensor of  FIG. 3  is used, G 33  can be calculated based on the following equation.  
               G   33     =     [         (       G   32     +     Kgr   ⁢           *           ⁢       (       R     33   -       ⁢     R   31       )     /   2         )     +       (       G   34     +     Kgr   ⁢           *     
     ⁢           ⁢       (       R   33     -     R   35       )     /   2         ]     /   2       ⁢     
     ⁢           =         (       G   32     +     G   34       )     /   2     +     Kgr   ⁢           *           ⁢       (       -     R   31       +     2   ⁢           *     
     ⁢           ⁢     R   33       -     R   35       )     /   4                     [     Equation   ⁢           ⁢   6     ]             
 
         [0047]     In Equation 6, (G 32 +G 34 )/2 denotes the linear interpolation value, (−R 31 +2*R 33 −R 35 )/4 denotes the peak value, and Kgr denotes the luminance significance element value. Kgr is calculated based on the following equation. 
 
 Kgr =( G _Factor/Luminance)/( R _Factor/Luminance)= G _Factor/ R _Factor  [Equation 7]
 
         [0048]     In Equation 7, if the integrated area is the same in the spectrum sensitivity of R, G, B mosaic CCD, the following equation can be obtained according to standard 709 of ITU-R Recommendations BT Series. 
 
 Y= 0.2126 *R+ 0.7152 *G+ 0.0722 *B   [Equation 8]
 
         [0049]     As a result, Kgr=0.7152/0.2126 since R_Factor=0.2126 G_Factor=0.7152 and B_Facotr=0.0722.  
         [0050]     As described above, the spatially missing color data can be interpolated at each pixel. Although a horizontal interpolation method is described in the above embodiment, a vertical interpolation may be applied alternatively. Also, other interpolations such as Cubic and Polyphase may be used in lieu of the linear interpolation. The digital image processing apparatus according to the embodiment of  FIG. 5  may be implemented in hardware or programmed to be executed by a computer.  
         [0051]     When the color interpolation is performed in the digital image processing apparatus which uses the mosaic CCD as the coupled device, the high resolution can be realized and the zipper effect can be reduced by adaptively applying the luminance significance with respect to each color.  
         [0052]     Although a few embodiments of the present general inventive concept have been shown and described, it will be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the general inventive concept, the scope of which is defined in the appended claims and their equivalents.