Patent ID: 12190476

DETAILED DESCRIPTION

FIG.1is a block diagram of an auto white balance adjusting system100according to an embodiment of the present invention. The auto white balance adjusting system100is capable of continuously calibrating white balance of a color shifted image under various light sources. The auto white balance adjusting system100can include an image capturing device10, a memory11, an output device12, and a processor13. The image capturing device10is used for acquiring an image. The image capturing device10can be a camera or any device having a photosensitive element. The memory11is used for saving data. The output device12is used for outputting an image with an adjusted white balance. The output device12can be a screen, an image output port, or a projection system. The processor13is coupled to the image capturing device10, the memory11, and the output device12for controlling the image capturing device10, the memory11, and the output device12. The auto white balance adjusting system100can adjust the white balance by using information of dual color spaces, thereby leading to high reliability. Further, the auto white balance adjusting system100can dynamically adjust a local white pixel area and a global white pixel area for determining an optimal reference white color for increasing the accuracy of automatically adjusting the white balance. In the auto white balance adjusting system100, after the image capturing device10acquires the image, the processor13determines a local white pixel area according to a first color temperature curve and a first brightness range of a first color space saved in the memory11. The processor13determines a global white pixel area according to the first color temperature curve and a second brightness range of the first color space saved in the memory11. The processor13selects a plurality of pixels of an image according to the local white pixel area for generating a local average color value of the first color space. The local average color value corresponds to the local white pixel area. The processor13selects a plurality of pixels of the image according to the global white pixel area for generating a global average color value of the first color space. The global average color value corresponds to the global white pixel area. The processor13converts the local average color value of the first color space into three primary color gains of a second color space. The processor13generates three primary color target gains according to the three primary color gains and a second color temperature curve of the second color space. The processor13adjusts a white balance of the image frame by frame to meet the three primary color target gains according to the local average color value of the first color space and the three primary color gains of the second color space. The processor13controls the output device12for outputting the image with the adjusted white balance. The first color space and the second color space are different.

FIG.2is an illustration of determining a local white pixel area R1and a global white pixel area R2of a first color space of the auto white balance adjusting system100. The first color space can be a Luminance-Chrominance color space (YUV color space). The second color space can be a three primary color space (RGB color space). In the YUV color space, as shown inFIG.2, Cr is a red chromaticity axis. Cb is a blue chromaticity axis. Therefore, a CrCb table ofFIG.2can be previously stored in the memory11. Based on the YUV color space, the processor13determines a local white pixel area for a local scene image according to local brightness and color variations of a high color temperature range and a low color temperature range of the first color temperature curve TC1. Such local white pixel area is called as the local white pixel area R1hereafter. Further, the local white pixel area R1can be defined by generating an area with any shape and any range around the first color temperature curve TC1. The local white pixel area R1can also be directly defined. For example, the local white pixel area R1can correspond to a local brightness range smaller than 100-200 nits. Therefore, the local white pixel area R1is a three-dimensional (Cr, Cb, Y) local white pixel space. A range of the local white pixel area R1can affect the accuracy of determining reference white pixels. Similarly, the processor13determines a global white pixel area for a global scene image according to global brightness and the color variations of the high color temperature range and the low color temperature range of the first color temperature curve TC1. Such global white pixel area is called as the global white pixel area R2hereafter. Further, the global white pixel area R2can be defined by generating an area with any shape and any range around the first color temperature curve TC1. The global white pixel area R2can also be directly defined. Similarly, the global white pixel area R2is a three-dimensional (Cr, Cb, Y) global white pixel space. Further, the local white pixel area R1is within the global white pixel area R2. After the local white pixel area R1and the global white pixel area R2are determined, the processor13can detect a plurality of corresponding white pixels in the image according to the local white pixel area R1and the global white pixel area R2. Then, the processor13can average color values of Y, U, and V of the plurality of white pixels in the local white pixel area R1for generating a local average color value (Local YUVNWP) of the first color space (YUV color space). Similarly, the processor13can average color values of Y, U, and V of the plurality of white pixels in the global white pixel area R2for generating a global average color value (Global YUVAverage) of the first color space. Then, the processor13can convert the local average color value of the first color space into three primary color gains (RGAIN,GGAIN,BGAIN) of a second color space (RGB color space). In other words, the processor13converts the local average color value of the first color space into a red color gain RGAIN, a green color gain GGAIN, and a blue color gain BGAINof the second color space. Then, the processor13can normalize the red color gain RGAIN, the green color gain GGAIN, and the blue color gain BGAIN. For example, the processor13can divide the red color gain RGAINand the blue color gain BGAINby the green color gain GGAIN.

FIG.3Ais an illustration of a second color temperature curve TC2of the second color space of the auto white balance adjusting system100.FIG.3Bis an illustration of a correlation among the second color temperature curve TC2, three primary color gains P, and three primary color target gains TP of the auto white balance adjusting system100.FIG.3Cis an illustration of a correlation among distances, weightings, and a probability function PC of the auto white balance adjusting system100. InFIG.3A, based on the second color space, the processor13can set a plurality of color temperature intervals according to the second color temperature curve TC2previously defined. For example, the processor13can set “1000K” as a range of the color temperature intervals between color temperatures 1000K and 5000K. In other words, inFIG.3A, 1000K-2000K can be defined as a color temperature interval. 2000K-3000K can be defined as a color temperature interval. 3000K-4000K can be defined as a color temperature interval. 4000K-5000K can be defined as a color temperature interval. Boundary points of a plurality of color intervals can be connected by a regression algorithm or a linear interpolation algorithm to meet the second color temperature curve TC2. Therefore, the second color temperature curve TC2can be linear or non-linear. In other words, the second color temperature curve TC2is formed by a plurality of color temperature range boundary points. X-axis is denoted as a normalized red color gain. Y-axis is denoted as a normalized blue color gain. The second color temperature curve TC2inFIG.3Acan be regarded as predetermined information saved in the memory11.

As previously illustrated, the processor13can convert the local average color value of the first color space (YUV color space) into three primary color gains of a second color space (RGB color space). Then, the processor13can generate three primary color target gains according to the three primary color gains and the second color temperature curve TC2of the second color space. Details are illustrated below. Since the second color temperature curve TC2can be formed by the plurality of color temperature range boundary points, as shown inFIG.3B, the processor13can acquire two distances between the three primary color gains P and two nearest color temperature range boundary points of the second color temperature curve TC2. For example, inFIG.3B, the processor13can acquire two nearest color temperature range boundary points corresponding to 2000K and 3000K. A distance between the three primary color gains P and a color temperature 2000K range boundary point can be determined as d1. A distance between the three primary color gains P and a color temperature 3000K range boundary point can be determined as d2. The distance can be an Euler distance, a distance derived according to a square root of an orientation vector, or a distance derived according to an absolute value of the orientation vector. Then, the processor13can generate the three primary color target gains TP by linearly combining the three primary color gains P with two nearest color temperature range boundary points according to the two distances (i.e., the distance d1and the distance d2). The distance d1is called as a first distance d1hereafter. The distance d2is called as a second distance d2hereafter. For example, the processor13can set two weightings w1and w2(hereafter, say, the first weighting w1and the second weighting w2). After the first weighting w1and the second weighting w2are linearly combined with the first distance d1and the second distance d2, coordinates of the three primary color target gains TP can be adjusted proportionally. Further, the first weighting w1and the second weighting w2can be acquired according to a look-up table or a probability function PC inFIG.3C. For example, inFIG.3C, according to the probability function PC, a weighting corresponds to the first distance d1is the first weighting w1. A weighting corresponds to the second distance d2is the second weighting w2.

In the auto white balance adjusting system100, the “white pixel area” can be adaptively updated, as illustrated below. The processor13can set a global white error threshold of the first color space. The processor13can acquire a global white error when updating the global average color value while the white balance of the image is adjusted frame by frame. For example, the processor13can estimate the global white error between the global average color value and a previous global average color value. The global white error can be derived according to a square root of an error vector or an absolute value of the error vector. When the global white error is equal to or larger than the global white error threshold, it implies that color tones of the image are varied drastically. Therefore, since the color tones of the image are varied drastically, the “original” global white pixel area R2is inappropriate. Therefore, the processor13can enlarge the global white pixel area R2and the local white pixel area R1for increasing the number of white pixels according to the global white error. Since the number of white pixels is increased, the accuracy of adjusting the white balance of the image can be improved. Further, the global white pixel area R2and the local white pixel area R1can be adjusted according to predetermined parameters or a look-up table.

When the global white error is smaller than the global white error threshold, it implies that the color tones of the image are varied gradually. Therefore, the processor13can use a local scene variation detection mode. For example, the processor13can set a white pixel (U, V) target value equal to 128 (8-bits) of the first color space. Then, the processor13can set a local white error threshold of the first color space. Further, the processor13can acquire a local white error between the local average color value and the white pixel target value. When the local white error is decreased, it implies that predicted white balance pixels approach a reference white color. When the local white error is increased, it implies that the predicted white balance pixels are far from the reference white color. Thus, the processor13can enlarge the local white pixel area R1when the local white error is equal to or larger than the local white error threshold. Conversely, the processor13can reduce the local white pixel area R1when the local white error is smaller than the local white error threshold. Similarly, the local white pixel area R1can be adjusted according to predetermined parameters or a look-up table.

In the auto white balance adjusting system100, a mechanism of adaptively generating white balance gains (i.e., the red color gain RGAINf the green color gain GGAINf and the blue color gain BGAIN) can be introduced, as illustrated below. The processor13can acquire previous three primary color target gains Curr_TP. Then, the processor13can acquire updated weightings WAWB. Then, the processor13can generate updated gains Update_Gain according to the three primary color target gains TP, the updated weightings WAWB, and the previous three primary color target gains Curr_TP.
Update_Gain=(1−WAWB)×Curr_TP+WAWB×TP

Here, the updated gains Update_Gain can be derived by linearly combining the three primary color target gains TP with the previous three primary color target gains Curr_TP according to the updated weightings WAWB. Further, the updated weightings WAWBcan be determined according to a distance between the previous three primary color target gains Curr_TP and the three primary color target gains TP, a look-up table, or a predetermined value. Further, coordinates of the updated gains Update_Gain are within the second color space (RGB). Therefore, the coordinates of the updated gains can be written as (Update BGAIN, Update RGAIN). Therefore, when the auto white balance adjusting system100adjusts the white balance frame by frame, updated components of the blue gain and the red gain can be regarded as the coordinates of the updated gains (Update BGAIN, Update RGAIN) of the second color space (RGB). Further, an updating frequency for adjusting the white balance of the image by the auto white balance adjusting system100can be customized. For example, the auto white balance adjusting system100can adjust the white balance of the image every frame or every two frames.

FIG.4is a flow chart of performing an auto white balance adjusting method by the auto white balance adjusting system100. The auto white balance adjusting method includes step S401to step S407.Step S401to step S407are illustrated below.step S401: determining the local white pixel area R1according to the first color temperature curve TC1and the first brightness range of the first color space;step S402: determining the global white pixel area R2according to the first color temperature curve TC1and the second brightness range of the first color space;step S403: selecting the plurality of pixels of the image according to the local white pixel area R1for generating the local average color value of the first color space, the local average color value being corresponding to the local white pixel area R1;step S404: selecting the plurality of pixels of the image according to the global white pixel area R2for generating the global average color value of the first color space, the global average color value being corresponding to the global white pixel area R2;step S405: converting the local average color value of the first color space into three primary color gains P of a second color space;step S406: generating three primary color target gains TP according to the three primary color gains P and the second color temperature curve TC2of the second color space;step S407: adjusting the white balance of the image frame by frame to meet the three primary color target gains TP according to the local average color value of the first color space and the three primary color gains P of the second color space.

Details of step S401to step S407are previously illustrated. Thus, they are omitted here. In step S401to step S407, the auto white balance adjusting system100can use two color spaces (i.e., such as the RGB color space and the YUV color space) for adjusting the white balance of the image. The YUV color space can be used for quickly estimating and detecting feedback signals. Further, the RGB color space can be used for quickly searching the three primary target color gains. Further, the auto white balance adjusting system100introduces a method for optimizing the white pixel area to improve white balance adjusting accuracy and convergence. Therefore, the auto white balance adjusting system100has satisfactory robustness for adjusting the white balance under any environment with various light sources.

To sum up, the present invention discloses an auto white balance adjusting system and an auto white balance adjusting method. The auto white balance adjusting system can use two color spaces for adjusting the white balance of the image. The YUV color space can be used for quickly estimating and detecting feedback signals. The RGB color space can be used for quickly searching the three primary target color gains. Further, a mechanism of adaptively generating white balance gains and a mechanism of adaptively updating white pixel areas can be introduced to the auto white balance adjusting system. Therefore, the auto white balance adjusting system has satisfactory robustness for adjusting the white balance under any environment with various light sources.

Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.