Abstract:
The present invention provides an automatic color balancing method for digital images by essentially performing adaptive weighting of surface reflectance and illuminant spectra components of the image. The adaptive weighting mechanism is derived from fuzzy logic based inference methods, taking advantage of its ability to perform inferences from data by providing a computational framework for knowledge that is in linguistic form. It also makes it amenable for implementation on hardware because of the commercially available fuzzy logic chips that already exist in real-world systems such as camcorders for image stabilization, washing machines, etc. The present invention enables much more efficient and robust color segmentation that forms core components for several computer vision algorithms.

Description:
BACKGROUND OF THE INVENTION 
   (1) Field of the Invention 
   This invention relates to digital imagery and, more particularly, to automatic color balancing of digital images. 
   (2) Description of Related Art 
   Of the many concerns facing the design of a digital camera, the problem of color balancing is of primary importance. One might naively believe that the best way to resolve this issue is to create sensors that sample a spectral power distribution with extreme precision, creating an image that most accurately represents the color properties of the world. However, this is not the case when one is making cameras for human observers. A major problem lies in the fact that the human visual system does not “see” colors by simply calculating the precise wavelengths absorbed by the three photoreceptors (the Red, Green, and Blue (RGB) receptors). Rather, the brain sees colors in terms of the relative spectral distribution over the entire scene. That is, the human eye automatically adjusts to color temperature changes within the 2800 K (tungsten) to 5500 K (daylight) range through a well-known process called color constancy. As a result, a piece of paper looks white when viewed under a tungsten, halogen, or fluorescent lamp or when viewed outdoors. These conditions vary greatly in terms of the spectral power distribution ultimately reflected to the eye, yet the paper looks white, the letters look black, and the colors do not seem to change. There are, of course, limits in extreme low lighting conditions such as during and after sunsets. For example, a piece of paper that appears orange in the summer when the sun is a large red disk in the horizon has a bluish hue when the sun is setting under cloud cover. Despite the eye&#39;s shortcomings under extreme lighting conditions its color balancing properties are quite amazing. The exact mechanism by which the eye performs its color balancing function remains a mystery. However, this visual correction mechanism does not extend to images on a computer display or on photographic print. Simply taking raw Red, Green, and Blue (RGB) color channel data from a camera and dumping it to the screen will not produce an acceptable image. For instance, a native image captured in a tungsten-lit room will have an overwhelmingly yellowish tint and a digital camera must determine a way to color-balance such input data automatically. Put simply, the goal of color constancy is to change the appearance of the scene illuminant to one of constant spectral power density (i.e. white light). However, with only three spectral samples (e.g. Red, Green, and Blue) per pixel, a perfect reconstruction is impossible. Nevertheless, people have developed empirical methods that partially solve this problem. 
   The following equation mathematically expresses the basic process of picture taking:
 
 R ( E 1 ,E 2 ,E 3,)=( S 1 ,S 2 ,S 3)*(reflectance)*( SPD  of light source),  (1)
 
where R is the total response, the E&#39;s are the relative responses of the light sensors (to be converted to the RGB values that the camera outputs), the S&#39;s are the relative sensitivities of these sensors to various wavelengths of light, the reflectance is a property of the object being viewed (e.g. its material—wood, metal, etc.), and SPD is the spectral power distribution of the light source. The combined product of reflectance and SPD gives the actual spectral power distribution that impinges the eyes, or that the sensors of a camera absorb. The problem is that there are many possible ways of attaining the same product, and is difficult to determine the unique reflectance of an object (which determines its perceived color).
 
   Two major theories provide a mechanism for partially solving the problems associated with the above equation. One is called the Gray-World theory that assumes that since all colors in a given image should average to some shade of gray, one can scale the responses with respect to how much of that shade is present at the various sampled points in the visual field. Statistically, the Gray-World assumption makes perfect sense: in an image with similar dynamic ranges for each of its color components, which is generally true of the pictures taken by a camera, each spectral sample has an equal probability of being above or below a certain fixed value. By summing across hundreds of thousands of samples, one would expect the average to converge to that fixed value (shade of gray). 
   A camera can exploit this observation to color-balance an image. First, the camera calculates average values for the red, green, and blue components from raw image data. By the Gray-World assumption, these three components should have the same value. However, if the illuminant does not have a constant SPD, these average values will deviate from one another in a way that is characteristic of the color appearance of the illuminant. One, in essence, factors out the average assumed to represent the SPD of the light source in order to attain the relative reflectance of the objects as if viewed under a white light. For instance, data obtained under a tungsten illuminant will have higher red and green averages than blue averages. To force the Gray-World assumption to hold, the camera can scale each of the color channels independently to equalize the RGB averages. The camera begins by picking an overall value for the averages to converge. One convenient value is simply the average of the three component averages. Multiplying each color channel by the overall average divided by the component average will yield an image for which the Gray-World assumption is true. This approach is not exact, but it manages to scale the responses in terms of surrounding conditions, and is therefore a good approximation for color balancing. 
   Another such approach that also tries to approximate the lighting conditions is know as the Perfect-Reflector hypothesis that assumes that the brightest object in an image reflects the best approximation of the SPD of the light source. Therefore, the RGB values of the brightest or most reflective spot are used to scale the RGB&#39;s of all other objects in the image in order to obtain a relative reading for the entire image. Thus, if one wishes to reconstruct the color appearance of the subject under white light, one should scale the image&#39;s RGB components to render the specular white. The scale factor for each color channel is just 255/specular value for an 8-bit system. This algorithm focuses only on the specular or glossy surfaces in the image and does not take into account the overall appearance. In fact, this may be a better approach for images where there is too little color variation for the Gray-World assumption to hold. 
   There are other algorithms for color constancy in the literature. In E. H. Land et al., titled “Lightness and Retinex theory,” Journal of Optical Society of America, Vol. 61, no. 1, PP. 1-11, January 1971, the authors adopted an approach based on illuminant spatial frequency in order to compensate for variations in color. A computational model was built for modeling color constancy by L. T. Mahoney et al. and reported in the article “A Computational Model for Color Constancy,” Journal of Optical Society of America, Vol. 1, PP. 29-33, 1986, that attempted to model surface reflectance by low-dimensional families of surfaces. This model was later simplified using linear models taught by D. H. Marimont et al. and reported in the article “Linear Model for Color Constancy,” Journal of Optical Society of America, Vol. 1, PP. 1905-1913, 1992. In an article by D. H. Brained et al, titled “Bayseian Color Constancy,” Journal of Optical Society of America, vol. 14, PP. 1393-1411, 1997, the authors proposed modeling color constancy as a Bayesian model that uses the cues such as surface reflectance and illuminant spectra to make Bayesian decisions that maximize expected utility. B. V. Funt et al., as demonstrated in their article titled “Is machine color constancy good enough?” in Proc. of ECCV, PP. 445-459, 1998, found this approach inaccurate. An attempt to improve the Perfect-Reflector algorithm was made by D. A. Forsyth and was described in the article titled “Sampling, Resampling and Color Constancy,” in Proc. of CVPR, PP 300-305, 1999. The attempt proposed a resampling approach as a solution, but resulted in a method that is computationally slow and that required several frames to accurately correct for variations in color. 
   There are also several patented approaches for achieving color constancy, broadly classified into hardware-based and software-based approaches. The software-based approaches typically provide a solution based on modifying a camera sensor in some fashion to achieve color constancy. U.S. Pat. No. 6,417,863 disclosed numerous embodiments of a method for adjusting the color balance of a display. In one embodiment, the color balance of a display is adjusted by modulating two selected colors so that at some point during the modulation the two colors appear substantially the same on the display and the modulation adjusts to change the point at which the colors appear substantially the same. The color balance of the display is modified, in part, based on a measurement of the modulation adjustment. In another embodiment of a method for adjusting the color balance of a display, the reference teaches the modification of the luminance and/or chromaticity of a color from the color space of the display, and modulation of luminance and/or chromaticity of a color from a reference color space, where, over time the applied modulation results in the two colors becoming substantially the same. Thus, the modulation adjusts so that the colors appear substantially the same at a selected point during the modulation of both colors, and the color balance of the display modifies partly based on a measurement of the modulation adjustment. This approach is not automatic in its balancing because it requires the user to pre-select, either the colors or provide reference color display for modulation adjustment. Accordingly, it is less attractive due to the need for user input. 
   In U.S. Pat. No. 6,388,648, a liquid crystal display (LCD) projection unit employs a luminance and color balance system employing an LCD array characterization lookup table storing multiple sets of luminance and gamma correction values that are user-selectable to control luminance and color balance. The lookup table values are determined by measuring the transfer function of the LCD array in the projection unit for each of a set of red (“R”), green (“G”), and blue (“B”) input data values, converting the resulting S-curve responses (color responses) to corresponding sets of gamma responses and scaling the gamma responses to generate multiple R, G, and B families of luminance and gamma corrected values. Color balance is adjusted by selecting the particular R, G, and B families of luminance and gamma corrected values that cause the LCD projection unit to match a predetermined ratio of R, G, and B luminance values. Luminance adjusts by selecting sets of families of R, G, and B luminance values that maintain the color balance ratio while causing the LCD to transmit a desired overall luminance. This technique has the disadvantage of requiring the storage of a large lookup table in order to tackle variations in lighting. Furthermore, the luminance and gamma correction values have to be user-selected which implies that the approach is not automatic. 
   U.S. Pat. No. 5,231,504 disclosed an improved method for using a linear mixing space, such as a CIE (Commission Internationale de l&#39;Eclairage) color space, to determine the quantities of coloring agents needed to reproduce a color, applicable, for example, to printing a color that matches an original color on a cathode ray tube (CRT) display. The calculations match hue, saturation and reflectance so that the matching color appears like the original color and so that those characteristics of an original image are preserved. An original color&#39;s definition converts into coordinates in the linear mixing space. An achromatic region in the linear mixing space, preferably elliptically shaped, includes the coordinates of the neutral coloring agents. Only quantities of neutral coloring agents match the original colors falling within the achromatic region. This approach however, does not address the issue of color constancy of scenes as imaged by a camera under varying lighting conditions. The focus is more on reproducing a print of the colors on a CRT display. 
   U.S. Pat. No. 5,504,524 presented a method of controlling the color balance of a video signal that comprises green, red, and blue channel signals. The patent identifies pixels that have a green intensity within a peak intensity band. For these pixels, the reference examines the red and blue intensities to identify pixel subsets within respective red and blue peak bands. If the red and/or blue peak band(s) are/is greater than the green peak band, the gain of the red and/or blue channel(s) is increased. If the red and/or blue peak band(s) are/is lower than the green peak band, red and/or blue peak band(s) are/is determined for the entire video frame, and compared with the green peak band. If the red and/or blue peak band(s) for the entire frame are/is lower than that for the green channel, the gain of the red and/or blue channel(s) are/is increased. However, this algorithm does not perform well for images where there is too little color variation or for images with high specula reflections. 
   In light of the current state of the art and the drawbacks to current systems mentioned above, a need exists for a system that would automatically color balance an image to correct for various lighting situations. That is, it would minimize discrepancies that exist between the natural response of a human eye and the response of an image-capturing device. 
   BRIEF SUMMARY OF THE INVENTION 
   The present invention provides an automatic color balancing system and method for digital images that operate by effectively performing adaptive weighting of surface reflectance and illuminant spectra components of the image. The adaptive weighting mechanism of the present invention is derived from fuzzy logic-based inference methods. The advantage of fuzzy logic methods is the ability to perform inferences from data by providing a computational framework for knowledge that is in linguistic form. It also makes the present invention amenable for implementation on hardware because of the commercially available fuzzy logic chips that exist in real-world systems such as camcorders for image stabilization, in washing machines, etc. The present invention enables much more efficient and robust color segmentation, an operation that serves as a core component of many computer vision algorithms. 
   One aspect if the present invention provides both a system and a method for automatic color balancing in digital images on a data processing system by receiving a current color image into a data processing system, providing a set of Image-Wide Luminance (IL) factors, and a set of Localized Luminance (LL) factors based on the current color image. The system further computes a set of weights representing a combination application of the IL factors and the LL factors, and then computes a set of color balance factors, where each color balance factor is computed from the IL factors, the LL factors, and the weights. It then constructs a new color image using the set of color balance factors and the current color image. 
   Another aspect of the present invention computes the weights as a function of luminance. 
   In yet another aspect of the present invention the weights are generated from fuzzy membership functions, with each function having a plurality of anchor points. 
   In a further aspect, the present invention includes a maximum-weight anchor point and a minimum-weight anchor point with a sloped set of values connecting the two anchor points. 
   In still further aspect, the present invention includes both IL weights and LL weights. 
   In another aspect, the present invention provides a set of fuzzy membership functions including an IL fuzzy membership function and an LL fuzzy membership function, separated by a ratio comprised of an LL fuzzy membership function maximum-weight anchor point to an IL fuzzy membership function maximum-weight anchor point, with this ratio set to a value of at least one. 
   In yet another aspect, the present invention obtains the IL weights from the IL fuzzy membership function according to the following expression: 
   
     
       
         
           
             
               
                 
                   W 
                   IL 
                 
                 = 
                   
                 ⁢ 
                 
                   
                     1.0 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     S 
                   
                   ≺ 
                   
                     β 
                     IL 
                   
                 
               
             
           
           
             
               
                 = 
                   
                 ⁢ 
                 
                   
                     2 
                     * 
                     
                       
                         [ 
                         
                           
                             S 
                             - 
                             
                               β 
                               IL 
                             
                           
                           
                             
                               γ 
                               IL 
                             
                             - 
                             
                               β 
                               IL 
                             
                           
                         
                         ] 
                       
                       2 
                     
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       β 
                       IL 
                     
                   
                   ≤ 
                   S 
                   ≺ 
                   
                     
                       
                         γ 
                         IL 
                       
                       + 
                       
                         β 
                         IL 
                       
                     
                     2 
                   
                 
               
             
           
           
             
               
                 = 
                   
                 ⁢ 
                 
                   
                     1 
                     - 
                     
                       
                         ( 
                         
                           2 
                           * 
                           
                             
                               [ 
                               
                                 
                                   S 
                                   - 
                                   
                                     β 
                                     IL 
                                   
                                 
                                 
                                   
                                     γ 
                                     IL 
                                   
                                   - 
                                   
                                     β 
                                     IL 
                                   
                                 
                               
                               ] 
                             
                             2 
                           
                         
                         ) 
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       if 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         
                           
                             γ 
                             IL 
                           
                           + 
                           
                             β 
                             IL 
                           
                         
                         2 
                       
                     
                   
                   ≤ 
                   S 
                   ≺ 
                   
                     γ 
                     IL 
                   
                 
               
             
           
           
             
               
                 
                   = 
                     
                   ⁢ 
                   
                     
                       0.0 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       if 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       S 
                     
                     ≻ 
                     
                       γ 
                       IL 
                     
                   
                 
                 , 
               
             
           
         
       
     
   
   where W IL  represents an IL weight; 
   S represents image-wide average luminance; 
   β IL  represents a maximum-weight anchor point in the fuzzy membership function; and 
   γ IL  represents a minimum-weight anchor point of the fuzzy membership function. 
   In another aspect the present invention obtains the LL weights from the LL fuzzy membership function according to the following expression: 
   
     
       
         
           
             
               
                 
                   W 
                   LL 
                 
                 = 
                   
                 ⁢ 
                 
                   
                     1.0 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     S 
                   
                   ≻ 
                   
                     β 
                     LL 
                   
                 
               
             
           
           
             
               
                 = 
                   
                 ⁢ 
                 
                   
                     2 
                     * 
                     
                       
                         [ 
                         
                           
                             S 
                             - 
                             
                               γ 
                               LL 
                             
                           
                           
                             
                               β 
                               LL 
                             
                             - 
                             
                               γ 
                               LL 
                             
                           
                         
                         ] 
                       
                       2 
                     
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       γ 
                       LL 
                     
                   
                   ≤ 
                   S 
                   ≺ 
                   
                     
                       
                         γ 
                         LL 
                       
                       + 
                       
                         β 
                         LL 
                       
                     
                     2 
                   
                 
               
             
           
           
             
               
                 = 
                   
                 ⁢ 
                 
                   
                     1 
                     - 
                     
                       
                         ( 
                         
                           2 
                           * 
                           
                             
                               [ 
                               
                                 
                                   S 
                                   - 
                                   
                                     γ 
                                     LL 
                                   
                                 
                                 
                                   
                                     β 
                                     LL 
                                   
                                   - 
                                   
                                     γ 
                                     LL 
                                   
                                 
                               
                               ] 
                             
                             2 
                           
                         
                         ) 
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       if 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         
                           
                             γ 
                             LL 
                           
                           + 
                           
                             β 
                             LL 
                           
                         
                         2 
                       
                     
                   
                   ≤ 
                   S 
                   ≺ 
                   
                     β 
                     LL 
                   
                 
               
             
           
           
             
               
                 
                   = 
                     
                   ⁢ 
                   
                     
                       0.0 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       if 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       S 
                     
                     ≺ 
                     
                       γ 
                       LL 
                     
                   
                 
                 , 
               
             
           
         
       
     
       
       
         
           where W LL  represents an LL weight; 
           S represents image-wide average luminance; 
           β LL  represents a maximum-weight anchor point in the fuzzy membership function; and 
           γ LL  represents a minimum-weight anchor point of the fuzzy membership function. 
         
       
     
  
   In a further aspect, the present invention computes the set of color balance factors by multiplying the IL factors with the IL weights to generate IL balance factors and multiplying the LL factors with the LL weights to generate LL balance factors, and adding the IL balance factors with the LL balance factors to generate the set of color balance factors. In order to construct a new color image, the current image is multiplied by the set of color balance factors. 
   In a still further aspect, the present invention provides the computations of the IL factors based on the current color image, which comprises a set of pixels p, with each pixel p including a plurality of color channels X. Computing of IL factors is performed by first computing an average intensity value A X  for each channel X, i.e. summing an intensity for a single channel X at each pixel p in the image and dividing by the number of pixels in the image. Expressing this mathematically: 
   
     
       
         
           
             A 
             X 
           
           = 
           
             
               ( 
               
                 
                   ∑ 
                   
                     i 
                     = 
                     1 
                   
                   p 
                 
                 ⁢ 
                 
                     
                 
                 ⁢ 
                 
                   X 
                   i 
                 
               
               ) 
             
             / 
             
               p 
               . 
             
           
         
       
     
   
   Second, an overall average image intensity value A is computed by summing the average intensity value A X  for all channels and dividing by a total number of channels in accordance with the following mathematical expression: 
   
     
       
         
           A 
           = 
           
             
               
                 ∑ 
                 
                   i 
                   = 
                   1 
                 
                 m 
               
               ⁢ 
               
                   
               
               ⁢ 
               
                 A 
                 
                   X 
                   i 
                 
               
             
             m 
           
         
       
     
   
   where m represents the total number of channels. 
   Last, a set of the IL factor for each channel is computed from: 
   
     
       
         
           
             IL 
             X 
           
           = 
           
             
               A 
               
                 A 
                 X 
               
             
             . 
           
         
       
     
   
   In another aspect, the present invention provides the set of LL factors based on the current color image by selecting a color space for each pixel p, wherein the color space includes the plurality of color channels X for that pixel p. The color space is selected from a group consisting of: RGB, CMYK, YIQ, YUV, HSB, HSL, and HSV and the channels X for a particular space are normalized between 0.0 and 1.0. The LL factors compute by determining a maximum luminance pixel for each channel X, identifying pixels surrounding the maximum luminance pixel, and denoting a maximum luminance point with the maximum luminance pixel and the pixels surrounding the maximum pixel. The LL factors are computed from a luminance in the channel X at the maximum luminance point as follows: 
   
     
       
         
           
             
               LL 
               X 
             
             = 
             
               1.0 
               
                 S 
                 X 
               
             
           
           , 
           
             where 
             : 
           
         
       
     
       
       
         
           S X  is the luminance in channel X. 
         
       
     
  
   These and other features, aspects, and advantages of the invention will be apparent to those skilled in the art from the following detailed description the present invention, taken together with the drawings and the claims that follow. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. 
     It is to be understood that the drawings presented herein are to be used for the purposes of exemplary illustration only and not as a definition of the limits of the invention. Referring to the drawings in which like reference numbers present corresponding parts throughout: 
       FIG. 1  is an exemplary general overview flow diagram of the present invention; 
       FIG. 2  is an exemplary illustration of a data processing system in accordance with the present invention; 
       FIG. 3  is an exemplary illustration of a data storage unit (medium) in accordance with the present invention; 
       FIG. 4  is an exemplary flow diagram depicting operations involved in processing an image in accordance with the present invention; 
       FIG. 5  is an exemplary illustration of two trapezoidal fuzzy membership functions for Gray-World and Perfect-Reflector hypotheses according to the present invention; 
       FIG. 6  is an exemplary illustration of two fuzzy membership functions for Image-Wide Luminance measure (IL) and Localized Luminance measure (LL) hypotheses in accordance with the present invention; 
       FIG. 7  is an exemplary comparison of the color-balancing algorithms for a tulip scene in accordance with the present invention; 
       FIG. 8  is an exemplary comparison of the color-balancing algorithms for a lily scene in accordance with the present invention; 
       FIG. 9  is an exemplary comparison of the color-balancing algorithms for an indoor scene in accordance with the present invention; 
       FIG. 10  is an exemplary performance comparison of the invention for cloth scenes under severe blue and red color contamination in accordance with the present invention; 
       FIG. 11  is an exemplary performance comparison of the invention for scenes under severe blue and red color contamination for plastic, and wool and plastic objects in accordance with the present invention; and 
       FIG. 12  is an exemplary performance comparison of the invention for scenes under severe blue and red color contamination for glass and metallic objects in accordance with the present invention. 
   

   DETAILED DESCRIPTION OF THE INVENTION 
   (1) Introduction 
   In the field of image capturing, whether analog, such as through the use of traditional film-based cameras, or digital, such as through the use of charge-coupled device (CCD) or other digital imaging systems, discrepancies exist between the natural response of a human eye and the response of an image capturing device. In particular, even though calibrated to match with the eye&#39;s response under one set of lighting conditions, when the lighting conditions vary, current image capturing devices often do not maintain their calibration. Thus, pictures taken at different times, particularly under low lighting conditions, do not appear the same as the scene from which they were obtained would when viewed with the naked eye. 
   The present invention provides a solution to this problem by utilizing Image-Wide Luminance measures, Localized Luminance measures, and weights to adjust the color balance of an input image and produce a color-balanced output image. 
   Before providing details regarding each of these components of the invention, first a glossary of terms is provided as a reference for the reader. Next, a system overview is provided, describing the interconnection and operation of various major components of a system in accordance with the present invention. After the system overview, a data processing system overview is presented to acquaint the reader with the various components typically found in a data processing system that may be used in conjunction with the present invention. Next, a discussion section is provided in which the various major components presented in the system overview are discussed in detail. Finally, a results section is provided, which presents non-limiting examples of the operation of the present invention in order to provide a more tangible understanding of the operation of the invention. 
   (2) Glossary 
   In order to provide a working frame of reference, a glossary of some of the terms used in the description and claims is given as a central resource for the reader. The glossary is intended to provide the reader with a general understanding of various terms as they are used in this disclosure, and is not intended to limit the scope of these terms. Rather, the scope of the terms is intended to be construed with reference to this disclosure as a whole and with respect to the claims below. Further, the definitions provided should not be considered limiting to the extent that the terms are known in the art. These definitions are provided to assist in the understanding of the present invention. 
   Color Balancing Factor—A color balancing factor is a parameter derived from various luminescence measures required to color balance an image. 
   Current Color Image—A current color image is the input or the original data (in the form of image data). 
   Data Processing System—A data processing system as defined herein generally refers to any general or specific-purpose computer system capable of processing data in a manner compatible with the present invention. 
   Image-Wide Luminance (IL) Measure—An Image-Wide Luminance measure is a hypothetical assumption that attempts to approximate lighting conditions by using values generated from a portion of an image that may include the entire image. An example of an Image-Wide Luminance measure a measure generated by a Gray World Theory, which assumes that all colors in a given image should average to some shade of gray, and that one can scale the responses (of objects to lighting conditions) with respect to how much of that shade is present at the various sampled points in the visual field. That is, it manages to scale the responses in terms of surrounding conditions.
 
Image-Wide Luminance Scale (ILS) Factors—An Image-Wide Luminance scale factor is a ratio of the average intensity A of luminescence to the average of each color channel A ChannelM  of an image.
 
Image-Wide Luminance Fuzzy Membership Functions—Image-Wide Luminance fuzzy membership functions are fuzzy membership functions representing IL measures.
 
Image-Wide Luminance Weights—Image-Wide Luminance weights are IL confidence levels (in terms of probability).
 
Localized Luminance (LL)—Localized Luminance is a hypothetical assumption that tries to approximate lighting conditions. It assumes that the brightest object (or region) in an image reflects the best approximate of the light source.
 
Localized Luminance Scale (LLS) Factor—A Localized Luminance Scale Factors is a ratio of the normalized weight (1.0) to a color channel, given by LLS channel     —     M =1.0/L channel     —     M .
 
Localized Luminance Fuzzy Membership Functions—Localized Luminance fuzzy membership functions are fuzzy membership functions representing LL measures.
 
Localized Luminance Weights—Localized Luminance weights are LL confidence levels (in terms of probability).
 
Luminescence—Luminescence is a measure of light intensity.
 
New Color Image—A new color image is an output of the present invention, in the form of processed data (an output image).
 
Weight—A weight is a confidence level (in terms of probability).
 
   (3) System Overview 
   As illustrated in  FIG. 1 , the present invention provides an improved color balancing technique  100  that utilizes a fuzzy inference mechanism within a weight computation block  108 , combined with an Image-Wide Luminescence (IL) measure  104  and a Localized Luminescence (LL) measure  106  to provide accurate color balancing under a wide variety of lighting conditions. More specifically, upon receiving an original color image  102  into the system  100 , the IL measure computation block  104  and the LL measure computation block  106  operate to generate their respective scale factors. In addition, the weight computation block  108  takes place after receiving the location of a set L of pixels from the LL measure computation block  106 . Subsequently, the IL factors, the LL factors, and the weights are provided to the color balance computations block  110  where the weights are multiplied by the IL and LL factors to generate a set of color balance factors, which are then further processed with the original input image  102  to construct the new color image  114 . 
   As illustrated, using a fuzzy inference mechanism, the present invention combines an Image-Wide Luminescence measure (IL)  104  such as the Gray-World hypothesis with a Localized Luminescence measure (LL)  106  such as the Perfect-Reflector hypothesis for color balancing of digital images  102 . The Gray-World assumption algorithm tends to over-compensate (bright regions in an image become too bluish) whereas the Perfect-Reflector assumption algorithm tends to under-compensate (the yellowish cast in an image remains although specularities appear white). The present invention develops an automatic color balancing system and method  100  for digital images  102  by performing adaptive weighting of surface reflectance (e.g. Gray-World) and illuminant spectra (e.g. Perfect-Reflector) components of the image  102 . The adaptive weighting mechanism is derived from fuzzy logic based inference methods, which have the ability to perform inferences from data by providing a computational framework for knowledge that is in linguistic form. 
   The performance of an Image-Wide Luminescence measure (IL)  104 , such as the Gray-World hypothesis, suffers relative to a Localized Luminescence measure (LL)  106 , such as the Perfect-Reflector hypothesis, when there is little color variation in the image  102 . If the image  102  has a variety of textures or other color variations, then Image-Wide Luminescence IL measures  104  are reasonably good for color balancing. On the other hand, Localized Luminescence LL measures  106  work very well under low color variation conditions in the image  102  or when the image  102  consists of objects that are highly specular in nature. If the amount of spectral power density is limited due to a negligible amount of glossy surfaces in the image  102 , then assumptions such as Gray-World perform better color balancing. In this sense, the two approaches are complementary. The present invention provides an adaptive method to combine the strengths of each algorithm while minimizing the effects of their weakness, without much manual adjustment to system parameters (with the exception of a single manual setting  112 , discussed in details below). Furthermore, the approach is computationally simple and thus amenable for implementation as an embedded solution in many of the available digital systems, such as in digital cameras and computer processors. The core idea of the present invention is that IF the overall image pixel is not very bright OR has low overall brightness THEN apply an image wide luminescence measure (IL)  104  such as the Gray-World hypothesis; similarly, IF the image pixel is very bright OR has high overall brightness THEN apply a Localized Luminescence measure (LL)  106  such as the Perfect-Reflector hypothesis. The fuzzy logic theory provides a computational framework for the application of these rules. 
   (4) Data Processing System Overview 
   A block diagram depicting the components of a computer system used in the present invention is provided in  FIG. 2 . The data processing system  200  comprises an input  202  for receiving image input signals from any inputting mechanism, including but not limited to, an external computer connected to the system, an internet connection, or any computer readable medium  300  (illustrated in  FIG. 3 ) such as a floppy disk, Compact Disk (CD), a Digital Versatile Disk/Digital Video Disk (DVD), and a removable hard drive. The input  202  may also be configured for receiving user input from another input device such as keyboard, a mouse, or any other input device best suited for the current environment conditions (e.g. image processing). Note that the input  202  may include multiple “ports” for receiving data and user input, and may also be configured to receive information from remote databases using wired or wireless connections. The output  204  is connected with the processor  206  for providing output to the user, possibly through a video display. Output  204  may also be provided to other devices or other programs, e.g. to other software modules, for use therein, possibly serving as a wired or wireless gateway to external databases or other processing devices. The input  202  and the output  204  are both coupled with a processor  206 , which may be a general-purpose computer processor or a specialized processor designed specifically for use with the present invention. The processor  206  is coupled with a memory  208  to permit storage of data and software to be manipulated by commands to the processor. 
   (5) Discussion 
   This portion of the description focuses on the system shown in  FIG. 4 , which is a more detailed illustration of the system shown in  FIG. 1 .  FIG. 4  is an exemplary flow diagram depicting operations involved in processing an original input color image  102  in accordance with the present invention to generate a color balanced output image  114 . Also, with respect to the description of  FIG. 4 , the terms Image-Wide Luminescence measure (IL)  104  and Localized Luminescence measure (LL)  106  will be used with the understanding that specific methods such as the Gray-World and the Perfect Reflector hypotheses can respectively be substituted therefor. 
   Given the general understanding of the interaction of the Image-Wide Luminance measures module  104 , the Localized Luminescence module  106 , the weight computation module  108 , and the color balance factors for generating a color-balanced output image  114  that was provided in the system overview section with respect to  FIG. 1 , the operation of each of these individual components is described in its respective subsection below. 
   (a) Image-Wide Luminance Measures (IL) Module 
   The generation of Image-Wide Luminance (IL) Measures  104  commences with computing the average luminescence of each color channel  416  in accordance with the following: 
                     A   red     =       (       ∑     i   =   1     N     ⁢           ⁢     R   i       )     /   N       ,           (   2   )                   A   green     =       (       ∑     i   =   1     N     ⁢           ⁢     G   i       )     /   N       ,   and           (   3   )                   A   Blue     =       (       ∑     i   =   1     N     ⁢           ⁢     B   i       )     /   N       ,           (   4   )               
where A red , A green , A blue , are the averages of red (R), green (G), and blue (B) color channels, N is the number of pixels in an image, and i is an index variable from 1 to N. Block  414  receives the averages output from block  416  to compute the overall average image intensity. The overall average image intensity  414  is calculated by the following equation:
   A =( A   red   +A   green   +A   blue )/3,  (5) 
where A is the overall average image intensity  414 . The above formulas can easily generalize for M channels, and are not limited to the three-color channels of RGB color space mentioned. For example, if a gamma-scheme used has M color channels, the computation for the average luminance of each color channel  416  is as follows:
 
                     A     Channel   ⁢           ⁢   1       =       (       ∑     i   =   1     N     ⁢           ⁢     Ch   ⁢           ⁢     1   i         )     /   N       ,           (   6   )                   A     Channel   ⁢           ⁢   2       =       (       ∑     i   =   1     N     ⁢           ⁢     Ch   ⁢           ⁢     2   i         )     /   N       ,   and           (   7   )                   A     Channel   ⁢           ⁢   M       =       (       ∑     i   =   1     N     ⁢           ⁢     Ch   ⁢           ⁢     M   i         )     /   N       ,           (   8   )               
In addition, the computation of the overall average image intensity  414  of these M channel averages 416 is as follows:
   A =( A   Ch1   +A   ch2   + . . . +A   ChM )/ M   (9) 
                 A   =           ∑     i   =   1     M     ⁢           ⁢     A     Ch   ⁢           ⁢   i         M     .             (   10   )               
The average luminance for each color channel from block  416  and the overall average image intensity from block  414  are provided to block  410  to compute the Image-Wide Luminance scale (ILS) factors. The ratio of the average intensity A to the average of each color channel A Channel M  is termed the Image-Wide Luminescence scale (ILS) factor  410  (in the example discussed for the three color channels):
 ILS Red   =A/A   Red ,  (11) ILS Green   =A/A   Green , and  (12) ILS Blue   =A/A   Blue .  (13) 
Equation (13) can be further generalized to any Image-Wide Luminescence (IL) system as follows:
 ILS Channel     —     M   =A/A   Channel     —     M .  (14) 
The resulting scale factors (ILS)  410  are then provided to the color balance computation block  110  for further processing (described in detail below).
 
   (b) Localized Luminescence Measure (LL) Module 
   Computations for Localized Luminescence measure  106  (the Localized Luminescence scale (LLS) factors  412 ) for an input image  102  commences with the optional transformation of the input image  102  into another color-space (gamma scheme) at block  418 . The choice of color spaces depends on the intended use of the image. For processing images viewed by humans, one may transform the original color space (such as RGB) to a YIQ color space. In the example described here, the present invention at block  418  of  FIG. 4  transforms the original RGB color space into YIQ by using the following set of equations: 
                   (         Y           I           Q         )     ⁢     (         0.299       0.578       0.114           0.596         -   0.275           -   0.321             0.212         -   0.523         0.311         )     ⁢       (         R           G           B         )     .             (   15   )               
The YIQ space is a well-known system for performing electronic color analyses (such as those used for television). The Y component of this space provides a more stable estimate for the luminance than the regular RGB space. That is, the YIQ space is perceptually less variant to changes in illumination than the RGB color space. Hence, the YIQ better helps to balance the hue and the saturation content of an image in a perceptually more pleasing and a more human perception-like manner. Of course, other well-known color gamma schemes, non-limiting examples of which include CMY, CMYK, CIE, YUV, HSV, HSB, and HSL, may also be used, depending on the particular application for which they are intended to be used. Although each gamma scheme has a different purpose, in general, any of them may easily substitute for the exemplary YIQ system used herein.
 
   The transformed color-space of the input image  102  is provided to block  420  for identification of pixels with maximum luminescence. As further illustrated in block  420  of  FIG. 4 , the image processing system of the present invention uses the Y component for a given image to locate (identify) the pixel(s) at locations L with maximum Y value (maximum luminescence). Using these image dependent parameters, the present invention formulates the Localized Luminescence scale (LLS) factors  412  for the three-color channels:
 
LLS Red =1.0 /L   Red ,  (16)
 
LLS Green =1.0 /L   Green , and  (17)
 
LLS Blue =1.0 /L   Blue ,  (18)
 
where L Red , L Green , and L Blue  represent luminescence in each respective color channel. These equations generalize to:
 
LLS channel     —     M =1.0 /L   channel     —     M ,  (19)
 
where L channel     —     M  represents luminescence in channel M. The identified location set L from block  420  is provided to block  424  of Weights Computation module  108  (described below). The resulting scale factors (LLS)  412  are then provided to the color balance computation block  110  for further processing.
 
   (c) Weight Computation Module 
   The exemplary  FIGS. 5 and 6  graphically illustrate the derivation of weights, and serve as a framework for more clearly presenting the various processes illustrated in the weight computation block  108  of  FIG. 4 . Referring to all three  FIGS. 4 ,  5 , and  6 , in the exemplary graphical illustration of  FIG. 5  S is defined to be the total sum of Red, Green, and Blue channels of an RGB space in the color image. Two trapezoidal fuzzy membership functions are also defined, F GW  for the Gray-World hypothesis and F PR  for the Perfect-Reflector hypothesis. These fuzzy membership functions define the nature of the weighting for the Gray-World and Perfect-Reflector hypotheses for a given image. The parameter sets {α GW , β GW , γ GW } and {α PR , β PR , γ PR } illustrated are the starting, inflection, and floor points, respectively, of the individual fuzzy membership functions. The present invention provides a hybrid approach that automatically decides the weighting for linearly combining the scale factors derived from the Image-Wide Luminescence measure (IL)  104 , and the same factors derived from the Localized Luminescence measure (LL)  106 . The weights W given to each set of factors depend on each pixel&#39;s (R+G+B), or S, as shown in the  FIG. 5 . This combination of weights ensures that the specular regions in the image render as white and that the darker regions of the image are properly balanced using an Image-Wide Luminescence measure (IL)  106 . The present invention uses the appropriate amount of weighting for the two hypotheses by modifying the shape of these membership functions for each input color image by adjusting these parameters based on image statistics. As illustrated in  FIG. 6 , the fuzzy membership functions need not be trapezoidal. 
   Proportions determined by the system then weight these scale factors. The weight assignment process is formulated for the Perfect-Reflector using the fuzzy membership defined in  FIG. 5 . The figure implies that the weighting of each hypothesis is a function of the combined pixel intensity at each pixel. For regular color images, it may be assumed without the loss of generality that all the three RGB-channels normalize between 0 and 1, and thus the maximum value of S is 3.0. To relax this, two parameters S max  and S min  are defined, corresponding to the maximum and minimum value of S for a given image. The weight for the Perfect-Reflector hypothesis is set to 1.0 for all pixels with combined pixel intensity that lie between S max  and β PR . If the combined pixel intensity falls below β PR , then the weight may begin to drop linearly between β PR  and β PR . As illustrated in  FIG. 6 , weight can also drop non-linearly. For combined pixel intensity below γ PR , the weight sets to zero. This process repeats for the Gray-World hypothesis. In that case, the weight is set to 1.0 for all pixels with combined pixel intensity between S min  and γ GW . If the combined pixel intensity is greater than β GW , then the weight may begin to drop linearly between β GW  and β GW . Again, as illustrated in  FIG. 6 , the weight can also drop non-linearly. For combined pixel intensity above β GW  the weight is set to zero. The mathematical expressions for the weights illustrated in  FIGS. 5 and 6 , which are computed at the block  430  of  FIG. 4 , are as follows, for  FIG. 5 : 
                           W   GW     =       ⁢       1.0   ⁢           ⁢   if   ⁢           ⁢   S     ≺     β   GW                   =       ⁢       1.0   -         (     S   -     β   GW       )       (       γ   GW     -     β   GW       )       ⁢           ⁢   if   ⁢           ⁢     β   GW         ≤   S   ≤     γ   GW                   =       ⁢       0.0   ⁢           ⁢   if   ⁢           ⁢   S     ≻     γ   GW               ⁢     
     ⁢   and           (   20   )                       W   PR     =       ⁢       1.0   ⁢           ⁢   if   ⁢           ⁢   S     ≻     β   PR                   =       ⁢           (     S   -     γ   PR       )       (       β   PR     -     γ   PR       )       ⁢           ⁢   if   ⁢           ⁢     γ   PR       ≤   S   ≤     β   PR                   =       ⁢       0.0   ⁢           ⁢   if   ⁢           ⁢   S     ≺       γ   PR     .                     (   21   )               
For a generalized version of  FIG. 6 :
 
   
     
       
         
           
             
               
                 
                   
                     
                       
                         
                           W 
                           IL 
                         
                         = 
                           
                         ⁢ 
                         
                           
                             1.0 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             S 
                           
                           ≺ 
                           
                             β 
                             IL 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                         ⁢ 
                         
                           
                             2 
                             * 
                             
                               
                                 [ 
                                 
                                   
                                     S 
                                     - 
                                     
                                       β 
                                       IL 
                                     
                                   
                                   
                                     
                                       γ 
                                       IL 
                                     
                                     - 
                                     
                                       β 
                                       IL 
                                     
                                   
                                 
                                 ] 
                               
                               2 
                             
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               β 
                               IL 
                             
                           
                           ≤ 
                           S 
                           ≺ 
                           
                             
                               
                                 γ 
                                 IL 
                               
                               + 
                               
                                 β 
                                 IL 
                               
                             
                             2 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                         ⁢ 
                         
                           
                             1 
                             - 
                             
                               
                                 ( 
                                 
                                   2 
                                   * 
                                   
                                     
                                       [ 
                                       
                                         
                                           S 
                                           - 
                                           
                                             β 
                                             IL 
                                           
                                         
                                         
                                           
                                             γ 
                                             IL 
                                           
                                           - 
                                           
                                             β 
                                             IL 
                                           
                                         
                                       
                                       ] 
                                     
                                     2 
                                   
                                 
                                 ) 
                               
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 
                                   
                                     γ 
                                     IL 
                                   
                                   + 
                                   
                                     β 
                                     IL 
                                   
                                 
                                 2 
                               
                             
                           
                           ≤ 
                           S 
                           ≺ 
                           
                             γ 
                             IL 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                         ⁢ 
                         
                           
                             0.0 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             S 
                           
                           ≻ 
                           
                             γ 
                             IL 
                           
                         
                       
                     
                   
                 
                 ⁢ 
                 
                   
 
                 
                 ⁢ 
                 and 
               
             
             
               
                 ( 
                 22 
                 ) 
               
             
           
           
             
               
                 
                   
                     
                       
                         W 
                         LL 
                       
                       = 
                         
                       ⁢ 
                       
                         
                           1.0 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           if 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           S 
                         
                         ≻ 
                         
                           β 
                           LL 
                         
                       
                     
                   
                 
                 
                   
                     
                       = 
                         
                       ⁢ 
                       
                         
                           2 
                           * 
                           
                             
                               [ 
                               
                                 
                                   S 
                                   - 
                                   
                                     γ 
                                     LL 
                                   
                                 
                                 
                                   
                                     β 
                                     LL 
                                   
                                   - 
                                   
                                     γ 
                                     LL 
                                   
                                 
                               
                               ] 
                             
                             2 
                           
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           if 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             γ 
                             LL 
                           
                         
                         ≤ 
                         S 
                         ≺ 
                         
                           
                             
                               γ 
                               LL 
                             
                             + 
                             
                               β 
                               LL 
                             
                           
                           2 
                         
                       
                     
                   
                 
                 
                   
                     
                       = 
                         
                       ⁢ 
                       
                         
                           1 
                           - 
                           
                             
                               ( 
                               
                                 2 
                                 * 
                                 
                                   
                                     [ 
                                     
                                       
                                         S 
                                         - 
                                         
                                           γ 
                                           LL 
                                         
                                       
                                       
                                         
                                           β 
                                           LL 
                                         
                                         - 
                                         
                                           γ 
                                           LL 
                                         
                                       
                                     
                                     ] 
                                   
                                   2 
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               
                                 
                                   γ 
                                   LL 
                                 
                                 + 
                                 
                                   β 
                                   LL 
                                 
                               
                               2 
                             
                           
                         
                         ≤ 
                         S 
                         ≺ 
                         
                           β 
                           LL 
                         
                       
                     
                   
                 
                 
                   
                     
                       
                         = 
                           
                         ⁢ 
                         
                           
                             0.0 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             S 
                           
                           ≺ 
                           
                             γ 
                             LL 
                           
                         
                       
                       , 
                     
                   
                 
               
             
             
               
                 ( 
                 23 
                 ) 
               
             
           
         
       
     
   
   Using these weights, a linear combination of the scale factors (equations (14) and (19)) with the coefficients being the weights as derived in equations (20)-(23) are used to make the final adjustments to each color channel at the color balance computations block  110  of  FIG. 4 . 
   As stated above, one aspect of the present invention is the automatic mechanism to color-balance an image. This automatic mechanism at block  422  of  FIG. 4  adjusts the anchor points (α IL , β IL , γ IL , α LL , β LL , and γ LL ) on the fuzzy membership functions illustrated in  FIG. 6  for each image based on image  102  statistics. To adjust the membership functions automatically, the present invention further defines parameters β R0   max  and β R0   min , which represent the maximum and minimum values of β LL . These parameters correspond to the Localized Luminance (LL) measures, and thus depend upon the brightest pixel intensity (closer to white). These parameters compute as follows:
 
β R0   max =ξ 1 S max ,  (25)
 
β R0   min =ξ 2 S max , and  (26)
 
where ξ 1  and ξ 2  are two fixed constants. It is only important that ξ 1 &gt;ξ 2 . In the particular example discussed here, for example, the present invention sets ξ 1 =⅚ and ξ 2 =½. Another parameter needed for these calculations is the percentage of pixels connected to the brightest pixels, computed at block  424  of  FIG. 4 , using the following equation:
 
 P =(Brightest Pixels/ N ),  (27)
 
where P represents the percentage of the pixels in the image connected to the brightest pixels (located in set L as described above), and which are also very bright (within 5% of the brightest intensity as a non-limiting example), and where N represents the total number of pixels in the image. Therefore, after block  422  computes its respective weights, the weights are provided to block  426 , along with P from block  424  (equation (27) above) to compute β LL . The parameter β LL  computed at block  426  of  FIG. 4  is dependent on the Localized Luminescence (LL) measure characteristics of the image. In this example, the value of β LL  is modified based on the degree of brightness content in the image (using a linear model based on P of equation 27). This formulates as follows:
 
                   β   LL     =           (       100   ⁢   N   ⁢           ⁢     β     R   ⁢           ⁢   0     max       -     100   ⁢           ⁢     β     R   ⁢           ⁢   0     max         )     -       (     NP   -   100     )     ⁢           ⁢     (       β     R   ⁢           ⁢   0     max     -     β     R   ⁢           ⁢   0     min       )           (       100   ⁢   N     -   100     )       .             (   28   )               
The computed value of β LL  at block  426  is received by block  428 , including a manually adjusted parameter δ from block  112  (described in detail below), where additional constraints are applied to the remaining parameters of the fuzzy membership functions:
 γ IL =β LL ; γ LL =β IL   (29) and 
                     β   LL       β   IL       =     δ   .             (   30   )               
The only requirement here is that in equation (30), δ≧1.0, where δ represents the difference between the two hypotheses. Hence, the relationship illustrated in equation (29) is optional. That is, β IL ≠β LL , γ LL ≠β IL  is also possible. Higher values of 6 (manually selected at block  112 ) allow for more separability between the two hypotheses. In aspect, for example, 8 is set equal to 5.0. Thus, on an image basis the various control points of the fuzzy membership functions can be adjusted. This allows for an adaptive adjustment of the weighting parameters to achieve automatic color balancing. The δ adjustment may be either a software or hardware-based implementation that enables users to vary the value of δ before a picture is taken. A software implementation may, as a non-limiting example, be in the form of a software plug-in and the hardware implementation may be in the form of electronics in a digital camera. Block  430  receives the computations from blocks  424  and  428  to calculate the Image-Wide Luminance weight W IL  and Localized Luminance weight W LL . The resulting calculated weights at block  430  are then provided to the color balance computation block  110  for further processing.
 
   (d) Color Balance Computations Module 
   In the color balance computations block  110 , the ILS factors  410 , the LLS factors  412  and the weights W IL  and W LL  computed at block  430  are received and are used to generate a set of color balance factors. The operator  450  multiplies the Image-Wide Luminance weight W IL  by the ILS factor  410  and the operator  456  multiplies the Localized Luminance weight W LL  by the LLS factor  412 , and the operator  452  sums the two resulting products to generate a set of color balance factors. These are then provided for the construction of the new color balanced output image  114  by the operator  454  by multiplying these results with the image pixels from the original image  102  (the directed line  438  illustrated in  FIG. 4 ). As one specific example, the color adjustment to each pixel for the red channel becomes:
 
 R   NEW   =R   current *( W   IL *ILS IL   +W   LL *LLS LL )  (24)
 
where R Current  and R new  are the red channel values for a given pixel in the original and the color-balanced images, respectively. Solutions to equation (24) are calculated on a pixel-basis because the weights {W IL , W LL } and the S values vary based on the color channel values at each pixel. Similar equations are used at each pixel for any channel, e.g. the green and blue channels.
 
   (6) Results 
   In order to reduce this invention to practice, its performance has been tested on images collected using a digital camera. The careful selection of these images highlights various colors balancing problems. The first example is an image of some tulips on a farm. The original image as seen in  FIG. 7  has a severe magenta cast. The image, after correction using the Gray-World assumption, no longer has the magenta cast. However, the algorithm over-compensates at the brighter regions of the image, causing the sky to appear unacceptably greenish. It also suppresses the reds of the tulips to such a large degree that the tulips are brownish and dull. Using the Perfect-Reflector assumption, magenta was removed from the sky and, as a result, the clouds in the image appeared white. However, the magenta cast still pervades much of the rest of the image. The Forsyth algorithm performs better than the Gray-World and Perfect-Reflector algorithms. However, the magenta is not completely removed from the sky (appears dark blue) or the tulips that appear too reddish. Using the adaptive color-balancing algorithm, the present invention obtained considerable improvements. All results list the adaptive setting for β PR . The sky is a perfect blue, and the green stalks, overwhelmed by the magenta in the original image, now appear distinctly green. The red tones of the tulips appear less drowned by the magenta color and thus appear more true red. 
   In the case of lily example as shown in  FIG. 8 , there is very little blue in the image. The Gray-World algorithm attempts to equalize the blue average with the reds and greens results in an image that is unacceptably dark. The Perfect-Reflector assumption does not affect this image very much because the bright spots along the glossy stalks are small and almost white. The Forsyth algorithm also suffers from the same effect as the Gray-World albeit a little less severe. The present invention performed the better than the other techniques. It removed the extra yellow in the image without unnecessary boosting of the blue channel. The image in  FIG. 9  has a strong greenish-yellow cast. Again, the Gray-World assumption fares poorly because of the lack of blue in the object forcing the subject to take an unnatural blue cast. The Perfect-Reflector seems to have little effect on the image because the brightest region, which is the reflection of the fluorescent lighting on the glass, is particularly saturated at white and small in size. In the Forsyth algorithm, the performance is better as the yellow cast is gone and the wall now appears white. However, the algorithm has the negative side effect of de-saturating the yellow from the subject&#39;s shirt well. The present invention performed comparably with the Forsyth without washing away the yellow from the subject&#39;s shirt. The performance of the present invention was also studied with respect for objects of different materials under two extreme colored lighting conditions.  FIG. 10  shows a Mondrian type image of a colorful shirt taken under the two extreme lighting conditions. The results of the color balancing using the algorithm of the present invention (β PR =2.9) show a considerable improvement to the color content of the shirt and the consistency in the output for two different lighting conditions. Similar results can be seen for the two different material sets as shown in  FIG. 11  (β PR =2.5) and  FIG. 12  (β PR =2.27). These results show how robust the color-balancing algorithm of the present invention is to various conditions. 
   Although the invention has been described in language specific to structural features and or methodological steps, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or steps described. Rather, the specific features and steps are disclosed as preferred forms of implementing the claimed invention.