Abstract:
A system includes an image analyzer to analyze an input image to generate correction parameters and a grey scale stretcher to utilize said correction parameters to perform a grey scale stretch on the image with little or no visible change in the noise level of the image.

Description:
FIELD OF THE INVENTION  
       [0001]     The present invention relates to still image processing generally and to automatic gamma correction in particular.  
       BACKGROUND OF THE INVENTION  
       [0002]     Digital images are common and are produced by many different entities. Some are produced by professional photographers but many are generated by amateurs. The latter are often shot with little thought to the composition of the photograph and thus, the resultant photograph may not look as beautiful as possible. A common error is misuse of lighting such that the photograph is too dark, too light or unevenly lit This error can be fixed through a technique known as “gamma (γ) correction”, which stretches the grey scale or dynamic range of the photograph.  
         [0003]     A common gamma correction graph is shown in  FIG. 1 , to which reference is now made. Gamma correction changes an input intensity level V i , normalized by the maximum intensity level V max  of the image, (the X axis), into an output intensity level V i , normalized by the maximum intensity level V max  of the image (the Y axis). If γ is 1 (the graph labeled  10 ), then there is no correction and the output is the same as the input. This is used for a normal looking image. If the image is dark, the image needs to be lightened and the output intensities should be raised. Thus, γ is set to less than 1.  FIG. 1  shows, in graph  12 , the curve for γ=0.5. If the image is light, the image needs to be darkened and the output intensities should be lowered. Thus, y is set to greater than 1.  FIG. 1  shows, in graph  14 , the curve for γ=2.  
         [0004]     Unfortunately, gamma correction takes a professional eye to choose the proper level of γ to fix the photograph  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0005]     The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to the principle algorithm and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:  
         [0006]      FIG. 1  is graphical illustration of a gamma correction operation;  
         [0007]      FIG. 2  is a block diagram illustration of an image improver, constructed and operative in accordance with the present invention;  
         [0008]      FIG. 3  is a graphical illustration of high pass and low pass filter operations, useful in the image improver of  FIG. 2 ;  
         [0009]      FIG. 4  is a graphical illustration of a multiplicity of exemplary histograms, useful in understanding the operation of the image improver of  FIG. 2 ;  
         [0010]      FIG. 5  is a block diagram illustration of a parameter generator forming part of the image improver of  FIG. 2 ;  
         [0011]      FIG. 6  is a graphical illustration of gamma correction implemented in the image improver of  FIG. 2 ;  
         [0012]      FIG. 7  is a block diagram illustration of a gamma processed data adaptive noise reducer forming part of the image improver of  FIG. 2 ;  
         [0013]      FIG. 8  is a block diagram illustration of a small details adaptive noise reducer forming part of the image improver of  FIG. 2 ;  
         [0014]      FIGS. 9A and 9B  are graphical illustrations of exemplary histograms, useful in understanding a second embodiment of the present invention; and  
         [0015]      FIG. 10  is a block diagram illustration of a second embodiment of the image improver of the present invention. 
     
    
       [0016]     It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.  
       DETAILED DESCRIPTION OF THE INVENTION  
       [0017]     In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.  
         [0018]     The present invention may improve improperly exposed images and may do so automatically, with no need for user (professional or otherwise) participation. Moreover, the processing may do so with little or no increase in visible noise. The method of the present invention may be applied to all types of digital images, such as those from a digital still camera, printers digital video, internet video, etc.  
         [0019]     Applicants have realized that gamma correction performed on the entire image does not produce the nicest possible image. In accordance with a preferred embodiment of the present invention, the correction performed on the elements which a viewer may perceive, such as small details and their contrast, may be different than the correction performed on the background of the image or on large details. Moreover, in accordance with a preferred embodiment of the present invention, the grey scale stretch may be provided with little or no visible change in noise level of the image.  
         [0020]     Reference is now made to  FIG. 2 , which illustrates one embodiment of an automatic image improver  20 , constructed and operative in accordance with the present invention. Image improver  20  comprises a picture parameter determiner  22 , and a plurality of color component improvers  24 , one per color component In the embodiment of  FIG. 2 , the color components are red, green and blue (R, G, B) and there are three color component improvers  24 R,  24 G and  24 B, respectively.  
         [0021]     Each color component improver  24  may comprise a low pass filter (LPF)  30 , a high pass filter (HPF)  32 , an adaptive gamma corrector  34 , a gamma processed data adaptive noise reducer  36 , a small details adaptive noise reducer  38  and an adder  40 . The elements of color component improver  24 R processing the red color component are labeled with an R, those of color component improver  24 B are labeled with a B and those of color component improver  24 G are labeled with a G.  
         [0022]     Color component improver  24  may utilize LPF  30  and BPF  32  to separate its component signal into two channels, one for large details and one for small details, respectively, and may process each channel separately.  FIG. 3 , to which reference is now briefly made, is a graphical illustration of exemplary high and low pass filters, useful in the present invention. Their cutoff frequencies are set at the expected size of the largest small detail (e.g. 4 pixels).  
         [0023]     In accordance with a preferred embodiment of the present invention, the small details (generated by HPF  32 ) may be processed by noise reducer  38 . This may reduce the noise on the small sized details that people perceive less than large details and may thus provide a sharper looking image.  
         [0024]     Applicants have real that the exposure of large details may affect the image more than the exposure of small details and that gamma correction on such large details may have a greater effect on the overall image. Thus, in accordance with a preferred embodiment of the present invention, color component improver  24  may pass the large details, generated by LPF  30 , through gamma corrector  34 . Since, as Applicants have realized, gamma correction may generate noise, the output of gamma corrector  34  may be processed by gamma noise corrector  36  to minimize the noise added by gamma corrector  34 .  
         [0025]     The output of the two channels may be combined together by adder  40  to generate the improved color component signal. Thus, if the color component being processed is the red component, the output may be the improved color component R′.  
         [0026]     In accordance with a preferred embodiment of the present invention, the parameters for gamma corrector  34 , gamma noise reducer  36  and small details noise reducer  38  are a function of the details in an input image, such as a digital image or a digitized analog image. Parameter determiner  22  may analyze the input image and may determine the gamma γ level to correct the large details of the input image. Parameter determiner  22  may also determine a gamma noise coefficient K γ  and a small details noise coefficient K t . Parameter determiner  22  may provide gamma γ to gamma correctors  34 , gamma noise coefficient K γ  to gamma noise reducers  36 , and small details noise coefficient K t  to small details noise reducer  38 .  
         [0027]     Parameter determiner  22  may comprise a luminance converter  42 , a histogram generator  44  and a parameter generator  46 . Converter  42  may convert the input RGB signal to a luminance value Y. Such a conversion is known in the art One exemplary well-used conversion equation is: 
 
 Y= 0.3 R+ 0.59 G+ 0.11 B  
 
         [0028]     Histogram generator  44  may generate a histogram H of luminance Y in the input image. Histogram is a graph of pixel quantity H(Y i ) (i.e. the number of pixels in the input image for every luminance level Y i ) in the input image. Reference is now made to  FIG. 4 , which illustrates some exemplary histograms, where the X axis is the normalized intensity Y i /Y max , and the Y axis is the normalized histogram H i /H max . Y max . may be the maximum allowable value of the intensity, such as  255 , and H max  may be the maximum number of pixels in the image.  
         [0029]     In curve  50 , the histogram has a peak  51  in the lower intensities, indicating a dark image. Curve  52  graphs the histogram for a normal image, with a peak  53  in the middle range of  FIG. 4 . Finally, curve  54  has a peak in the brighter intensities, indicating a generally much too light image.  
         [0030]     In accordance with a preferred embodiment of the present invention, parameter generator  46  ( FIG. 1 ) may divide the histogram graph into sections of different exposure quality. For example, three sections, for light, dark and normal exposures, may be defined. Alternatively, more sections, for more refined processing, may be defined. The definition may be done by a designer and may involve selecting the intensity levels (Y i /Y max ) defining the borders between sections. For the three section example, the borders might be Y D =0.3Y max  and Y L =0.7Y max . These borders are marked on  FIG. 4 . The dark section may thus be the portion of the graph with intensity levels below Y D , the light section may be the portion of the graph with intensity levels above Y L  and the normal section may be between the borders Y D  and Y L .  
         [0031]     As illustrated in  FIG. 5 , to which reference is now made, parameter generator  46  may comprise a section integrator  60 , a peak detector  62  and a controller  64 . Section integrator  60  may determine the quantity Q of pixels per section, as defined by the section division. The integration may involve summing the histogram values for the intensities in the relevant section. For the three section example, the equations may read:  
         Q   D     =       ∑   0     Y   D       ⁢           ⁢     H   ⁡     (     Y   1     )             
         Q   N     =       ∑     Y   D       Y   L       ⁢           ⁢     H   ⁡     (     Y   1     )             
         Q   L     =       ∑     Y   L       Y   max       ⁢           ⁢     H   ⁡     (     Y   1     )             
 
 Peak detector  62  may be any suitable peak detector, of which many are known in the art. In particular, peak detector  62  may find where H, the point where the histogram H is at its maximum, and Y(H max ), the intensity Y at the maximum point H max . 
 
         [0032]     Controller  64  may determine which type of exposure the input image has, in one of a number of ways. In one embodiment, controller  64  may select the section which has the largest quantity value. In another embodiment, controller  64  may have threshold values set for the dark and light sections. Thus, an image may be determined to be dark only if the dark quantity Q D  may be greater than a threshold, defined as a percentage of the total number Q M  of pixels in the image. Thus, only if Q D &gt;q D *Q M , where q D  may be, for example, between 50% and 100%, may controller  64  determine that the input image has a dark exposure. Similarly for a light exposure. If Q L &gt;q L *Q M , where q L  may be, for example, between 50% and 100%, may controller 64 determine that the input image has a light exposure.  
         [0033]     In either embodiment, once controller  64  has determined the type of exposure in the input image, controller  64  may determine the gamma γ level. If the exposure is normal, γ NP  may be 
 
γ NP =1 
 
         [0034]     Otherwise, for both dark and light exposures, the gamma γ level may be defined as:  
         γ   D     =       γ   L     =       γ   0     +       K   G     ⁢       Y   ⁡     (     H   max     )         Y   max                 
 
 where γ 0  may be a minimum γ value (γ 0  has been found empirically to be 0.6) and K G  may be a user defined coefficient Typically K G  may be close to 1.0. For dark images, Y(H max ) may be below Y D  and thus, the ratio of  
         Y   ⁡     (     H   max     )         Y   max         
 
 may be quite small. When added to γ 0  of 0.6, and using Y D  of 0.3 as in the example hereinabove, the results is a range of γ D  for the dark images of 0.6&lt;γ D &lt;0.9. For light images, Y(H max ) may be above Y L  and thus, the ratio of  
         Y   ⁡     (     H   max     )         Y   max         
 
 may be quite large. When added to γ D  of 0.6 and using Y L  of 0.7, the results is a range of γ L  for the light images of 1.3&lt;γ L &lt;1.6 (for K G =1). 
 
         [0035]     There are also pictures with complicated light distributions. For example, a picture might have a distribution Q DL  which might have a wide dark area and a small light area Another picture might have a distribution Q LD  with a wide light area and a small dark area. Similarly, there may be other distributions defined, such as dark/normal (Q DN ), normal/dark (Q ND ), light/normal (Q LN ) and normal/light (Q NL ).  
         [0036]     A picture may be considered to have the distribution Q DL  if the following conditions hold: 
 
If  Q   L &gt;[1−( Y   N   /Y   M )] Q   M 
 
And  Q   D&gt;   q   D   *Q   M  
 
 where Y M  is the maximum allowable value of the intensity, such as 255. 
 
         [0037]     Similarly, a picture may be considered of type Q LD  if. Q D &gt;(Y D /Y M )*Q M  and Q L &gt;q L *Q M    
         [0000]     where q D  and q L  is between 50% and 100%.  
         [0038]     Similar conditions may be set for Q DN , Q ND , Q LN  and Q NL .  
         [0039]     For the complicated contrast distributions, such as those described hereinabove, the gamma response may be varied, with a different response for every portion, dark, normal, or light The gamma value for each portion may be calculated in accordance with the equations of paragraphs  30  and  32 . For example, an exemplary gamma response for the dark/light distribution Q DL , is presented in  FIG. 6 , to which reference is now briefly made. The gamma response may be defined as:  
           {       V   1     /     V   max       )     out     =     {               (       V   1     /     V   max       )     in     Y   D       ,             if   ⁢           ⁢   0     &lt;     (       Y   1     /     Y   max       )     &lt;     (       Y   0     /     Y   max       )                       (       V   0     /     V   max       )       Y   D       +       [       (       V   1     /     V   max       )     -       (       V   0     /     V   max       )       Y   D         ]       Y   L         ,         otherwise               
 
 where Y o Y=H[H max (Y i )] and V o  is the relevant red (R), green (G) or blue (B) signal levels related to Y o , accordingly 
 
         [0040]     Controller  64  may also determine the noise reduction coefficients K t  and Kγ. As is known in the art noise visibility is increased for dark and normal areas and is lower for light areas. Thus, controller  64  may generate a smaller multiplicative coefficient for dark images than for light images. One exemplary equation for generating noise reduction coefficients K t  and Kγ might be:  
         K   t     =       K   γ     =       γ   0     +         Y   ⁡     (     H   max     )         Y   max       ⁢     K   F               
 
 since the gamma correction curve increases from dark images to light images. K F  may be a coefficient defining a minimal noise reduction, which a user may define. Typically KF may be close to 1.0. In addition, noise reduction coefficients Kγ and K t  may be limited to no larger than 1.0. 
 
         [0041]     Reference is now made to  FIG. 7 , which illustrates an exemplary gamma noise reducer  36 , operative on one color component. As gamma noise reducer  36  may be the same for all color components, only one will be described herein.  
         [0042]     Noise reducer  36  may reduce high frequency noise in the signal from gamma corrector  34  and may comprise a low pass filter (LPFγ)  70 , a subtractor  72 , a multiplier  74  and a summer  76 .  
         [0043]     Low pass filter  70  may generate a low frequency component Vγ LF  from an input signal Vγ from gamma corrector  34 . Subtractor  72  may subtract low frequency component Vγ LF  from the input signal Vγ, thereby producing a high frequency component Vγ HF  of input signal Vγ. The magnitude of high frequency component Vγ HF  may be changed, in multiplier  74 , by noise reduction coefficient Kγ. The resultant high frequency noise reduced signal may be added to low frequency component Vγ LF  in adder  76 , to generate the gamma noise reduced signal.  
         [0044]     Reference is now made to  FIG. 8 , which illustrates an exemplary small details noise reducer  38 . Noise reducer  38  may reduce texture noise in the high frequency color component signal produced by high pass filter  32  and may comprise a limiter  80 , a subtractor  82 , a multiplier  84  and an adder  86 .  
         [0045]     Limiter  80  may have a threshold level of 3-5 times the average noise level in the image and may generate a texture component signal V t  which may have low contrast detail data and noise (or grain). Subtractor  82  may remove texture component signal V t  from high frequency signal V HF  to generate other (contrast) components. The magnitude of texture component V t  may be changed, in multiplier  84 , by noise reduction coefficient K t . The resultant texture noise reduced signal may be added to the low contrast frequency component in adder  86 , to generate the texture part noise reduced signal.  
         [0046]     The present invention may also be utilized for images with a small dynamic range. For example, the histograms of two such images are shown in.  FIGS. 9A and 9B , to which reference is now briefly made.  FIG. 9A  shows the histogram for an image with a ‘veil’ effect, which has no dark intensities. The intensities begin at Y i /Y max =0.3. There are no intensities below that value.  FIG. 9B , on the other hand, shows the histogram for an overly dark image, where the intensities end at Y i /Y max =0.3. Neither image utilizes the full dynamic range of the camera or the film, and gamma correction, which functions over the entire dynamic range, will be unsuccessful as a result.  
         [0047]     Reference is now made to  FIG. 10 , which illustrates a further embodiment of the present invention which may handle small dynamic range images. In this embodiment, a dynamic range corrector  90  may be added before image improver  20 . Corrector  90  may determine how shrunk the dynamic range of said input image is and may shift, if necessary, and may amplify the dynamic range of the image to provide an output image with an appropriate dynamic range for image improver  20 .  
         [0048]     Corrector  90  may comprise an offset determiner  92  and a processor  94 . Offset determiner  92  may generate the histogram of the intensities and may determine the extent that the intensities are shifted above the start of the dynamic range. The start typically is at a null-point. For example, for a dynamic range of 0-255, the null-point is Y=0. Determiner  92  may then determine the size of a shift Y off , by which to correct the shift, if present, and the size of an amplification coefficient K a  by which to amplify the intensities. Processor  94  may then correct the shift using Y off  and may then amplify the possibly shifted intensities with a coefficient K a .  
         [0049]     To that end, determiner  92  may comprise luminance converter  91  (similar to luminance converter  42  of  FIG. 2 ), which may convert the input RGB signal to a luminance Y signal, histogram generator  93  (similar to histogram generator  44  of  FIG. 2 ), which may generate the histogram and a controller  100 , which may determine a minimum value Y 1 , and a maximum value Y h  of the luminance intensities and which may determine the shift Y off  and coefficient K a . therefrom Histogram generator  44  may generate the histogram using intensities rather than normalized intensities (i.e. H i  rather than H i /H max )  
         [0050]     Controller  100  may determine whether or not the minimum value Y 1  is at a null-point, such as Y=0. In the example above, the dynamic range of 0-255, if the minimum value Y 1  is above 0, then there is an offset which must be fixed. Controller  100  may then set shift Y off  to the minimum value Y 1 . Thus, if the minimum value Y 1  is 10, Y off  may become 10. If the minimum value Y 1  is at 0, then the shift Y off  may be set to 0.  
         [0051]     If the maximum value Y h  or the shifted maximum value (Y h -Y off ) is below the maximum value Y max , such as 255 in the example, the dynamic range is too small. Controller  100  may determine amplification coefficient K a  as follows: 
 
 K   a   =D*Y   max /( Y   h   −Y   off ) 
 
 where D may be less than 1 and may be a user selected value defining the amount of amplification that the user desires. 
 
         [0052]     Processor  94  may comprise an offset reducer  102  and an amplifier  104  per color component (R, G or B). Each offset reducer  102 R,  102 G or  102 B may subtract the shift value Y off  it receives from the input intensity R in , G in , or B in , respectively. Each amplifier  104  may multiply the signal it receives by coefficient K a . The result may then be three output signals R out , G out  and B out  which may then be provided as an input signal to image improver  20 .  
         [0053]     In another embodiment of the present invention, the input signal to corrector  90  may be a luminance signal Y. In this embodiment, there is no luminance converter  91  and there is only one input channel, and thus, only one of each of offset reducer  102  and amplifier  104 . Similarly, the image improver in this embodiment has no luminance converter  42  and only one input channel (and thus, only one of each of LPF  30  ( FIG. 1 ), HPF  40 , adaptive gamma corrector  34 , gamma processed data adaptive noise reducer  36 , small details adaptive noise reducer  38  and adder  40 .  
         [0054]     While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will now occur to those of ordinary skill in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.