Calibrating color for an image

Calibrating color (such as adjusting color balance) for an image may include capturing a set of calibration images of a calibration target. The calibration target includes at least one color reference region illuminated by a light source having a distribution of frequencies that cycles over a fixed time period. The calibration images are captured over a calibration period that is greater than or equal to the fixed time period over which the distribution of frequencies cycles. Each of the calibration images is associated to a time identifier. A color balance adjustment is calculated for each of the calibration images, for forming a time-dependent color spectrum model of the color balance adjustments. The color spectrum model models changes in the light source over the fixed time period. In one embodiment, a subject image of a subject is captured for a determined exposure time. In the subject image, the subject is illuminated by the light source. Exposure time data is associated to the subject image. At least one subject color balance adjustment is calculated for the subject image from the exposure time data and the color spectrum model.

BACKGROUND

The present invention relates to digital imaging, and more specifically to color correction by calibrating color or adjusting color balance, which may also be known as white balance, gray balance or neutral balance.

Because light sources have varying color temperatures, images captured by standard or digital cameras often exhibit a color cast, which causes an image of a subject to appear differently from an image of the subject captured under different lighting. One may adjust the color balance attribute to remove the color casts caused by light sources. Color balance may also be referred to as white balance, gray balance, or neutral balance.

Accurate color rendition is vital in digital imaging, and correct color balance is a key factor in achieving accurate color rendition. Achieving correct color balance is typically accomplished by applying a color temperature profile to a digital image after the image is captured. In cases where the color temperature of the light source is stable over time, this technique often works well. For a group of images taken under a stable light source, the correct color temperature profile need only be determined for the first image in the group, and can then be applied to all the remaining images in the group.

Achieving correct color balance for a group of images becomes more complicated if the color temperature of the light source changes from image to image. In this case, each image must be analyzed individually to determine the correct color temperature profile to apply.

Achieving correct color balance for an image or images illuminated by fluorescent lighting is also complicated. Fluorescent lights change color significantly and continuously at the frequency of the alternating current (“AC”) which powers them. For example, fluorescent lights in the United States may change in color at a 60 times per second cycle. Images captured using exposure times shorter than one complete AC cycle may show significant color shifts with respect to each other, and so will require different color temperature profiles for each image to achieve proper white balance. Worse still, if the exposure time is sufficiently short, the images will show a color variation within each image. For example, the top portion of the image will have a different color cast from the middle portion of the image, which will have a different color cast from the bottom portion. In such cases, there is no single color temperature profile which can be applied to the entire image to achieve correct color balance.

SUMMARY OF THE INVENTION

According to one embodiment of the present invention, correcting color in an image is provided by a method for adjusting color balance for an image, that captures a set of calibration images of a calibration target. The calibration target includes a color reference region illuminated by a light source having a distribution of frequencies that cycles over a fixed time period. The calibration images are captured over a calibration period that is greater than or equal to the fixed time period over which the distribution of frequencies repeatedly varies. Each of the calibration images is associated to a time identifier. A color balance adjustment is calculated for each of the calibration images, for forming a time-dependent color spectrum model of the color balance adjustments. The color spectrum model models changes in the light source over the fixed time period.

In one embodiment, a subject image of a subject is captured for a determined exposure time. In the subject image, the subject is illuminated by the light source. Exposure time data is associated to the subject image. At least one subject color balance adjustment is calculated for the subject image from the exposure time data and the color spectrum model.

DETAILED DESCRIPTION

The following is intended to provide a detailed description of an example of the invention and should not be taken to be limiting of the invention itself. Rather, any number of variations or embodiments may fall within the scope of the invention, which is defined in the claims following the description.

Embodiments of the invention model color variation of a light source over time. Certain embodiments use this model to provide automated color balance correction of images. Embodiments of the present invention generate a time-varying color spectrum model. It is from this model that embodiments of the invention calculate a subject color balance adjustment for a subject image.

FIG. 1is a block diagram of one embodiment of the invention. A digital capture device100may be a digital single-lens-reflex camera (“dSLR”), digital video camera, a cellular phone with an embedded digital camera, an MP3 player with an embedded digital video camera, or other device that is designed to capture digital images. The digital capture device100is used to capture a series of calibration images110illuminated by a light source. The light source has a distribution of frequencies that cycles over a fixed time period. For example, the light source may be a fluorescent light with a cycle of 60 times per second.

The calibration images110include a calibration target120that contains a color reference region. The calibration target120may be a neutral grey card that is commonly used in digital imaging. The calibration target120may also be the calibration product sold by Datacolor under the trademark SPYDERCUBE, the 24-color target product sold by X-Rite under the trademark COLORCHECKER, or various other products sold for color management. In some embodiments, the calibration target120may simply be a portion of an image that is relied upon to represent a neutral gray, white, black, or other value.

The series of calibration images110are made at short exposure times and in rapid sequence. For example, the exposure time may be 1/4000 second and there may be one exposure every 1/120 second. Of course, other exposure time and sequence timings may be used. The series of calibration images110are captured over a calibration period that is greater than or equal to the fixed time period of the light source. Current technology of capture devices100may achieve these short exposure times and sequence timings by using a low pixel resolution. For example, one embodiment of the invention may capture calibration images110at a low 200 by 200 pixel resolution. Other embodiments may use lower or higher resolutions.

Calibration images110are stored electronically as files160. In some embodiments the files160are stored in any of a number of raw file formats, each of which is proprietary to a single camera manufacturer, and each of which is typically specific to a given make and model of camera. Such proprietary file formats are referred to here as “RAW” files. In other embodiments, the files160are stored as JPG format files. Other file format types may also be used. When each calibration image110is made, a time identifier is associated to the calibration image110. In some embodiments, the time identifier is stored as part of file160. For example, the time identifier may be stored as part of the RAW or JPG file. In other embodiments of the invention, the time identifier may be stored separately from the files160.

From the calibration images110and associated time identifiers, various embodiments of the invention create the time-varying color spectrum model. For each calibration image110, an embodiment of the invention uses the color reference region of the calibration target in the image and applies a color-balance technique. Such color-balance techniques are well known in the art. For example, in one embodiment, the color reference region includes a neutral gray and the invention determines the color balance adjustment value for the image. Such a color balance adjustment is the color temperature needed to map the actual color values for red, green, and blue in the calibration image to a neutral grey where the red value, the green value, and the blue value are all three equal to 128 on a scale of 0 to 255. The color balance adjustment may be expressed in kelvins.

Because a time identifier is associated with each of the calibration images110, the color balance adjustments for the images may be associated with the time identifiers. For example, a series of 64 calibration images110may result in the following data:

As explained above, the series of calibration images110are captured over a calibration period that is greater than or equal to the fixed time period of the light source. In the example from the above data table, if the light source was a fluorescent light with a 60 times per second cycle, the series of 64 calibration images captured over 1.05 seconds exceeds the fixed time period of the light source and provides sufficient sampling for various embodiments of the invention.

FIG. 2is a graph plotting color temperature color balance adjustment data points205along the Y-axis against the time identifiers on the X-axis, from data points contained in the above data table. In some embodiments, the invention determines a function305that describes the collected data points. Such a function is shown inFIG. 3, and is a analytic representation of the time-varying color spectrum model of the invention. This spectrum model comprehends how the color temperature of the light source varies over time during its entire fixed period time cycle. The information in the model is absolute with respect to time. Thus, the model can output what the color temperature of the light source was at any point in time during the session when the images were captured. For example,FIG. 4illustrates a color balance adjustment indicated by the color spectrum model ofFIG. 3. AsFIG. 4indicates, at 0.62 seconds, the color balance adjustment value405is 4198 kelvins. Thus, the model can provide information for appropriate color correction at any time during the interval in which the calibration images were captured. In addition, the model can also provide information for appropriate color correction at times after (or before) the interval in which the calibration images were captured. The invention provides the ability to extrapolate from the information captured during the calibration interval, to correct images taken before or after the calibration interval. For example, if the series of calibration images is captured between 12:45:15 and 12:45:17; the model may also be used to color-correct images taken from 12:50 through 13:50. This is possible because of the repeating cyclic nature of the variation of the light source.

FIGS. 2 through 4are only explanatory. Embodiments of the present invention do not generate such actual graphs. Rather, the calibration images110that are stored electronically as files160are processed by software190in a memory180of a computer165that has a CPU170and the time-varying color spectrum model is generated and stored as a data structure or otherwise in memory180. In some embodiments, computer165is separate from the capture device100. Such a computer165may be a local personal computer by which the capture device can be directly connected. Or such a computer165may be a remote computer that is accessed over the Internet or other network. In yet other embodiments, computer165may be within the capture device100itself.

Thus far, embodiments of the present invention that use calibration images110to generate a spectrum model have been explained. Some embodiments of the invention leverage an existing spectrum model to color-correct images. Returning now toFIG. 1, the capture device100is used to capture a subject image150of a subject130. The subject image150is illuminated by the same light source that illuminated the calibration images110. As with the calibration images110, the subject image150is stored electronically as a file160. In some embodiments the file160is a RAW file, a JPG file, or other file type. When the subject image150is made, exposure time data is associated to the subject image150. In some embodiments, the exposure time data is stored as part of file160. For example, the exposure time data may be stored as part of the RAW or JPG file. In other embodiments of the invention, the exposure time data may be stored separately from the file160. Capture device100may capture a series of subject images150, although for simplicity, only one subject image150is shown inFIG. 1.

FIG. 5Aillustrates how certain embodiments of the invention calculate a color balance adjustment for the subject image150based on the exposure time data associated with the subject image150. In the example shown inFIG. 5A, the exposure time data correlates to time 0.62 from the model. Thus, the color balance adjustment510.1is 4321 kelvins. Leveraging the color spectrum model, embodiments of the present invention may correct the color balance in a series of subject images automatically.

Embodiments of the invention may also uses the exposure time information in the image file160(often embedded as metadata) to make a decision on what type of color balance correction is needed for the image. For a subject image150receiving the same integrated light exposure, the image may require only a single color balance correction applied to the entire image file160. UsingFIG. 5Aas an example of this, a 4321 kelvin color balance may be applied to all of the pixels of the image file160.

However, for images where the exposure time is short compared to fixed time period of the light source's cycle, the subject image150will show a color cast which varies in regions of the image.FIG. 5Billustrates how an embodiment of the invention may determine that the subject image may be treated as having four horizontal regions520.1through520.4. Embodiments of the invention may calculate that a first color balance adjustment of 4321 kelvins be applied to the first horizontal region520.1, a second color balance adjustment of 3772 kelvins be applied to the second horizontal region520.2, a third color balance adjustment of 5195 kelvins be applied to the third horizontal region520.3, and a fourth color balance adjustment of 6626 kelvins be applied to the fourth horizontal region520.4. Such horizontal bands in a subject image may occur because certain dSLRs use a narrow slit moving at high speed across the surface of the imaging sensor to control exposure time. For such images, embodiments of the invention use its color spectrum model to create an appropriate color temperature gradient, and apply that gradient to the image file. Other embodiments of the invention may support vertical regions within the subject image.FIG. 5Cshows radial regions530.1,530.2,530.3and530.4that certain embodiments may support for capture devices110that have a radially-opening shutter.

FIGS. 6A and 6Bare flowcharts of exemplary steps of an embodiment of the invention. InFIG. 6A, a first step610of an embodiment of the invention involves capturing a series of calibration images110of a calibration target120having a color reference region that is illuminated by a light source having a distribution of frequencies that cycles over a fixed time period. Such calibration images110are captured over a calibration period that is greater than or equal to the fixed time period. Each calibration image110is associated to a time identifier.

In the second step620, the embodiment of the invention calculates a color balance adjustment for each of the calibration images110and from these adjustments, at step630form a color spectrum model of the color balance adjustments. The color spectrum model models the changes in the light source over its fixed time period.

FIG. 6Billustrates a flowchart of another embodiment of the invention in which after step630, at step640one or more subject images150(that are illuminated by the light source that illuminated the calibration target120) are captured for a determined exposure time. At step650exposure time data is associated to the subject image150. At step660, one or more subject color balance adjustments are calculated for the subject image150from the color spectrum model.

The embodiments of the invention so far discussed are used to correct color balance in an image that was illuminated by light source(s) that are assumed to emit frequencies that are distributed as would be expected from an ideal black-body object heated to a given temperature, per the common definition of color temperature.

As a result,FIG. 2shows that a single data point (i.e., the color temperature of the light source(s)) can be plotted for a particular point in time. One skilled in the art will recognize that the invention can be extended to provide a collection of color calibration profiles, in which a color calibration profile (rather than a color temperature) can be associated with each point in time.

Such an embodiment is used when the light source(s) do not emit a distribution of frequencies which is well described by a single color temperature. One skilled in the art will recognize that light sources can have spectra which differ from that which is well described by a single color temperature.

The previous discussion described that a single color balance adjustment may be generated by analyzing a color reference region that is known to be neutral gray. By extension, a plurality of color adjustments making up a color calibration profile may be generated by analyzing more than one known color reference region. For example, the target product sold by X-Rite under the trademark COLORCHECKER includes 24 color reference areas. For simplicity of discussion here, one color reference region may be a neutral gray, a second color reference region may be red, a third color reference region may be green, a fourth color reference region may be blue, a fifth color reference region may be orange, a sixth color reference region may be yellow, and so on. The white balance color correction determined by analysis of a single neutral gray target can be expanded to an analysis of a collection of differently colored regions from the color reference target to permit creation of a color spectrum model which is not a single number for color temperature (with its underlying assumptions about distribution of frequencies) but is a color calibration profile, which contains within it information about the actual distribution of frequencies for the light source being modeled.

As one example, such an embodiment of the invention is useful when images are taken in which the illumination is provided by high-intensity mercury lamps. Such mercury lamps do not exhibit a distribution of frequencies which is accurately described by a single color temperature. By generating a set of color calibration profiles, images may be adjusted so that the appearance of red, green, blue, orange, yellow, and so on in the image matches the various known color reference regions.

The flowcharts ofFIGS. 6A and 6Bdemonstrate how embodiments of the invention can be methods. The invention may also be embodied as a system. In such a system, software190in memory180of computer165includes a color balance adjustor module, an exposure module, a time association module, and a calculator module. Yet other embodiments of the invention are computer program products of computer readable storage medium having computer readable program code. One skilled in the art will understand from the above description and the Figures how the invention may be embodied as a method, a system, and a computer program product.