Abstract:
A dynamic range enhancement method and apparatus for an imaging device expands a dynamic range of an input image without an associated increase in the amount of image calculations. The dynamic range enhancement method includes producing brightness information and color information from an input image; applying gamma correction and inverse gamma correction to the brightness information to produce a gamma corrected image and an inverse gamma corrected image; comparing variances of the gamma corrected image and the inverse gamma corrected image at identical spots; and expanding a dynamic range of the input image by selecting one of the variances at each spot. The dynamic range of an input image is expanded by applying a post imaging process without increasing calculation amount of the imaging device.

Description:
CLAIM OF PRIORITY 
       [0001]    This application claims priority to an application entitled “DYNAMIC RANGE ENHANCEMENT METHOD AND APPARATUS” filed in the Korean Intellectual Property Office on Nov. 29, 2007 and assigned Serial No. 2007-0122645, the contents of which are incorporated herein by reference in its entirety. 
       BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The present invention relates to an imaging device and a dynamic range enhancement method and apparatus therefore to improve image quality. 
         [0004]    2. Description of the Related Art 
         [0005]    An image sensor is a device that converts an optical image to an electric signal. Typically, an image sensor comprises a Charge-Coupled Device (CCD) or a Complementary Metal-Oxide-Silicon (CMOS) sensor. A CCD image sensor typically serializes parallel analog signals, which are in arrays of photoelectric light sensors by capacitors transferring an electric charge of each pixel to a common output structure which converts the charge to a voltage and buffers and sends the voltage off-chip. On the other hand, in a CMOS image sensor, pixels represented by respective MOS transistors convert the charges to voltages and the voltages are output in accordance with switching operations of the transistors. 
         [0006]    One of the most important factors in determining product quality of an imaging device is its dynamic range of the pixels. Dynamic range is a ratio of a pixel&#39;s saturation level to its signal threshold. Typically, the dynamic range refers to the maximum signal power that the device can tolerate without distortion of the input signal. In an image sensor, as the dynamic range of a device is increased, the ability to measure the dimmest intensities in an image is improved, resulting in good image quality. 
         [0007]    In order to expand the dynamic range, a signal variation-based dynamic range expansion method has been used with limited success. The dynamic range expansion (DRE) method differentiates a signal and adjusts the differentiated signal to expand the dynamic range. In the DRE method, the dynamic range characteristic of the signal changes by varying the differential value. This signal variation-based dynamic range expansion method is useful for one-dimensional voice signals. However, this method is not suited for processing two-dimensional images signal due to large calculations of quadratic calculus and associated iterations. 
         [0008]    Another conventional dynamic range expansion method uses multiple images taken at different exposure levels. This particular method estimates a response function of the image sensor using multiple images taken at different exposure levels and compensates the image by taking an inverse function of the response function. However, this method has several drawbacks, such as requiring a large memory for storing the multiple images, and the iterative process and is restricted for applying to real time applications. 
       SUMMARY OF THE INVENTION 
       [0009]    The present invention provides a dynamic range enhancement method and apparatus that expands a dynamic range of an image without increasing a calculation amount. 
         [0010]    Also, the present invention provides a dynamic range enhancement method and apparatus for expanding a dynamic range of an image using a gamma correction and inverse gamma correction technique. 
         [0011]    Also, the present invention provides a dynamic range enhancement method and apparatus for expanding the dynamic range of an image using a weighted average of gamma corrected values and inverse gamma corrected values. 
         [0012]    In accordance with an exemplary embodiment of the present invention, a dynamic range enhancement method can include the steps of producing brightness information and color information from an input image; applying gamma correction and inverse gamma correction to the brightness information to produce a gamma corrected image and an inverse gamma corrected image; comparing variances of the gamma corrected image and the inverse gamma corrected image at identical spots; and expanding a dynamic range of the input image by selecting one of the variances at each spot. 
         [0013]    In accordance with another exemplary embodiment of the present invention, a dynamic range enhancement apparatus may include a YUV transformer for extracting brightness information and color information from an input image; a gamma corrector for correcting the brightness information; an inverse gamma corrector for correcting the brightness information inversely; and a weighted averager for taking output values of the gamma corrector and the inverse gamma corrector and expanding the dynamic range of the input image by averaging the output values using a weighted average. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    The above features and advantages of the present invention will be more apparent from the following detailed description in conjunction with the accompanying drawings, in which: 
           [0015]      FIG. 1A  is a block diagram illustrating a configuration of a portable device implemented with a dynamic range enhancement apparatus according to an exemplary embodiment of the present invention; 
           [0016]      FIG. 1B  is a block diagram illustrating a configuration of the multi-adaptive compensator of  FIG. 1A ; 
           [0017]      FIG. 2  is a graph illustrating gamma correction and inverse gamma correction curves referred for a dynamic range enhancement method according to an exemplary embodiment of the present invention 
           [0018]      FIG. 3A  is an image illustrating a result of inverse gamma correction of a dynamic range enhancement method according to an exemplary embodiment of the present invention; 
           [0019]      FIG. 3B  is an image illustrating a result of gamma correction of a dynamic range enhancement method according an exemplary embodiment of the present invention; 
           [0020]      FIG. 4  is a flowchart illustrating a dynamic range enhancement method according to an exemplary embodiment of the present invention; 
           [0021]      FIGS. 5A and 5B  are exemplary images respectively obtained by a conventional imaging device and an imaging device adopted a dynamic range enhancement apparatus according to an exemplary embodiment of the present invention; and 
           [0022]      FIGS. 6A and 6B  are another exemplary images respectively obtained by the conventional imaging device and an imaging device adopted a dynamic range enhancement apparatus according to an exemplary embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0023]    Exemplary embodiments of the present invention are described with reference to the accompanying drawings in detail. The same reference numbers are used throughout the drawings to refer to the same or like parts. Detailed descriptions of well-known functions and structures incorporated herein may be omitted to avoid obscuring appreciation of the subject matter of the present invention by a person of ordinary skill in the art. In the drawings, certain elements may be exaggerated or omitted or schematically depicted for clarity of the invention. 
         [0024]      FIG. 1A  is a block diagram illustrating a configuration of a portable device implemented with a dynamic range enhancement apparatus according to an exemplary embodiment of the present invention. 
         [0025]    Referring to  FIG. 1A , the portable device may include a key input unit  101 , a camera unit  103 , a display unit  105 , a memory unit  107 , and a control unit  109 . 
         [0026]    The key input unit  101  is provided with a plurality of keys for receiving user&#39;s key input. The key input unit  101  is configured to generate a command for capturing an image. 
         [0027]    The camera unit  103  takes an image of an object and generates an image signal. The camera unit  103  is provided with an image sensor for converting the optical image to an electric signal and a signal processor for converting the electric signal to a digital signal. The image sensor may comprise, for example CCD and CMOS image sensors. The camera unit  103  may further include a video processor for generating screen image data. The video processor is preferably provided with a video codec for compressing and decompressing the video data so as to be displayed on the display unit  105 . 
         [0028]    The display unit  105  displays operation status of the portable device and various visual information. The display unit  105  also displays the image input by the camera unit  103 . Particularly in this exemplary embodiment, the display unit  105  is configured to display a gamma corrected image and inverse gamma corrected image. 
         [0029]    Still referring to  FIG. 1A , the memory unit  107  stores application programs and data associated with the operations of the portable device. 
         [0030]    The control unit  109  controls general operations of the portable device. In this particular exemplary description, the control unit  109  controls to enhance the dynamic range of an input image. The control unit is provided with a multi-adaptive compensator  150 . The multi-adaptive compensator  150  performs the gamma correction and inverse gamma correction on the image input by the camera unit  103  and expands the dynamic range of the compensated image such that the enhanced image is displayed on the display unit  105 . The dynamic enhancement procedure is described in more detail with reference to  FIG. 1B . 
         [0031]    In the meantime, the portable device may further include a radio frequency (RF) unit  111  for allowing radio communication. The RF unit  111  may include a transceiver or a separate RF transmitter for up-converting and amplifying a signal to be transmitted and an RF receiver for low noise amplifying and down-converting a signal received through an antenna. 
         [0032]    An internal structure of the multi-adaptive compensator for enhancing the dynamic range of an image is described in more detail. 
         [0033]      FIG. 1B  is a block diagram illustrating a configuration of the multi-adaptive compensator of  FIG. 1A . 
         [0034]    Referring to  FIG. 1B , the multi-adaptive compensator  150  includes a YUV (luminance and chrominance) transformer  151 , a gamma corrector  153 , an inverse gamma corrector  155 , a weighted averager  157 , and an RGB transformer  159 . 
         [0035]    When an image is input through the camera unit  103 , the multi-adaptive compensator  150  extracts Red, Green, and Blue (RGB) information from the image. The RGB information is converted to YUV information. Here, Y information represents the brightness, U information is obtained by subtracting the Y from the blue signal of the original RGB and then scaling, and V information is created by subtracting the Y from the red and scaling by a different factor. The YUV transformer  151  uses transformation equations for obtaining the YUV information from the RGB information: 
         [0000]        Y= 0.3 R+ 0.59 G+ 0.11 B    
         [0000]        U =( B−Y )×0.493 
         [0000]        V =( R−Y )×0.877 
         [0036]    The control unit  109  controls the YUV transformer  151  to send the Y information, i.e. brightness, to the gamma corrector  153  and the inverse gamma corrector  155 . 
         [0037]    The gamma correction is performed by transforming the intensity of the light nonlinearly using a nonlinear transfer function. According to the Weber&#39;s law, human vision operates nonlinearly in response to the brightness. When light amount varies linearly in a limited expressive information amount, a human eye perceives a breakage rather than a smooth change. Accordingly, in order to show the optimal vision quality in the limited expressive information amount, a nonlinear coding should be used. The inverse gamma correction is performed by inversing the gamma correction. 
         [0038]      FIG. 2  is a graph illustrating gamma correction and inverse gamma correction curves referred for a dynamic range enhancement method according to an exemplary embodiment of the present invention. 
         [0039]    The gamma corrector  153  brightens the dark area of an image so as to make a sensor response function linear at the dark area. The inverse gamma corrector  155  darkens the bright area of the image so as to make a sensor response function linear at the bright area. In this manner, an image input by the camera unit  103  is converted to the YUV information and then compensated by the gamma corrector  153  and inverse gamma corrector  155 . 
         [0040]      FIG. 3A  is an image illustrating a result of inverse gamma correction of a dynamic range enhancement method according to an exemplary embodiment of the present invention, and  FIG. 3B  is an image illustrating a result of gamma correction of a dynamic range enhancement method according an exemplary embodiment of the present invention. Information values output by the gamma corrector  153  and inverse gamma corrector  155  are selectivelyused for compensating the sensor response function. At this time, the control unit  109  compares variances of the two images of  FIGS. 3A and 3B  at an identical local area and selects the value greater than the other. 
         [0041]    Referring to  FIGS. 3A and 3B , the control unit  109  compares the variances at the areas  1 - a  and  2 - a  of the inverse gamma corrected image of  FIG. 3A  with the variances at the corresponding areas  1 - b  and  2 - b  of the gamma corrected image of  FIG. 3B . 
         [0042]    As a result of the above comparison, it is determined that the variance of the areas  1 - a  is less than the variance of the area  1 - b  and the variance of the area  2 - a  is greater than the variance of the area  2 - b.  Accordingly, the control unit  109  selects the the value of the area  1 - b  of the gamma corrected image and the value of the area  2 - a  of the inverse gamma corrected image. 
         [0043]    Although the variances of the two local areas are compared each other in this exemplary embodiment, the gamma and inverse gamma corrected images can be, for example, globally compared with each other. 
         [0044]      FIG. 3C  is an image illustrating gamma correction and inverse gamma correction areas of an input image obtained through a dynamic range enhancement method according to another exemplary embodiment of the present invention. In  FIG. 3C , black area is the gamma compensated area and white area is the inverse gamma compensated area. 
         [0045]    Now referring again to FIB.  1 B, 2hen selecting one of the values output by the gamma corrector  153  and the inverse gamma corrector  155 , the weighted averager  157  uses the weight average of the two output values. In order to avoid the discontinuity at the boundary between the two areas when using one of the two values, the weighted averager  157  takes a weight average of the two values. The average weigh calculator  157  can be implemented, for example, as follows: 
         [0000]                                            if(var(x1)==0&amp;&amp;var(x2)==0) {              y=(x1+x2)/2           }           else{              y=(var(x1)*x1+var(x2)*x2)/(var(x1)+var(x2))           }                        
where x 1  denotes the gamma corrected value, x 2  denotes the inverse gamma corrected value, var(x 1 ) denotes the variance of the gamma corrected value, and var(x 2 ) denotes the variance of the inverse gamma corrected value. The weighted averager  157  uses the variance as weight such that the larger the variance is, the larger the weight is. Since average weigh calculator  157  has a similar variance at the boundary of two areas, a value close to the linear average of the two areas is reflected to the result. Accordingly, the discontinuous effect at the boundary, appeared when selecting the value comprehensively, can be removed.
 
         [0046]    The RGB transformer  159  transforms the YUV information so as to output the RGB information. The RGB transformer  159  receives Y′ information output by the weighted averager  157  and UV information output by the YUV transformer  151  and generates RGB information using the Y′ information and the UV information. The RGB transformer  159  transforms the YUV information to the RGB information using the following equations: 
         [0000]        R=Y+ 0.956 U+ 0.621 V    
         [0000]        G=Y+ 0.272 U+ 0.647 V    
         [0000]        B=Y+ 1.061 U+ 1.703 V.    
         [0047]      FIG. 4  is a flowchart illustrating exemplary steps of a dynamic range enhancement method according to an exemplary embodiment of the present invention. 
         [0048]    Referring to  FIG. 4 , the control unit  109  of the portable device receives RGB information and converts the RGB information into YUV information (S 401 ). At this time, the control unit  109  checks the input of an image captured by the camera and control such that the YUV transformer transforms the RGB information of the input image into the YUV information. 
         [0049]    Next, the control unit  109  extracts the Y information (brightness) and UV information (color) from the YUV information (S 403 ). Here, the control unit  109  controls the YUV transformer  151  to separate the Y information and UV information. The control unit  109  only processes the brightness information, i.e. Y information. 
         [0050]    After separating the Y information from the UV information, the control unit  109  performs gamma correction and inverse gamma correction on the Y information sequentially (S 405 ). That is, the Y information is input to the gamma corrector  153  and the inverse gamma corrector  155  so as to be output in the form of gamma corrected information and inverse gamma corrected information in parallel (see  FIG. 2 ). 
         [0051]    Still referring to the flowchart in  FIG. 4 , the control unit  109  compares the gamma corrected value and the inverse gamma corrected value, selects one of the gamma corrected value and the inverse gamma corrected value, and compensates the sensor response function linearly using the weight average (S 407 ). That is, the control unit  109  compares the values at the same local areas of the gamma corrected image output by the gamma corrector  153  and the inversed gamma corrected image output by the inverse gamma corrector  155  as shown in  FIGS. 3A and 3B  and selects one of the value. Then, the weighted averager  157  averages weight values to smoothen the discontinuous boundary. 
         [0052]    Next, the control unit  109  transforms the compensated brightness information Y′ and the color information UV so as to be output RGB information (S 409 ). The RGB transformer  159  receives the Y′ information output by the weight average  157  and the UV information output by the YUV transformer  151  and generates the RGB information using the Y′ and UV information. The RGB information is output by the RGB transformer  150  is output to, for example, display unit  105  under the control of the control unit  109 . 
         [0053]      FIGS. 5A and 5B  show exemplary images obtained by a conventional imaging device and an imaging device adopted a dynamic range enhancement apparatus according to an exemplary embodiment of the present invention, respectively.  FIGS. 6A and 6B  show another exemplary images obtained by the conventional imaging device and the imaging device adopted the dynamic range enhancement apparatus according to an exemplary embodiment of the present invention, respectively. 
         [0054]    The images of  FIGS. 5A and 6A  are photographs processed by the conventional image processing technique, and images of  FIGS. 5B and 6B  are photographs processed by the dynamic range enhancement method of the present invention. As shown in the  FIGS. 5A ,  5 B,  6 A, and  6 B, the images acquired using the dynamic range enhancement method according to an embodiment of the present invention are superior to the image acquired by the conventional technique, especially in brightness quality. 
         [0055]    Although exemplary embodiments of the present invention have been described in detail hereinabove, it should be clearly understood that many variations and/or modifications of the basic inventive concepts herein taught which may appear to those skilled in the present art will still fall within the spirit and scope of the present invention, as defined in the appended claims. 
         [0056]    As described above, the dynamic range enhancement method and apparatus of the present invention expands a dynamic range of an input image by applying a post imaging process without increasing a calculation amount of the imaging device.