Patent Description:
<CIT> discloses a method wherein an electrical signal output from an image sensing device is converted into digital image data and stored in a noncompressed form or a lossless-compressed form, in a storage medium. A user sets parameters associated with a white balance via a setting screen. The setting of the parameter is performed either in a first adjustment mode in which a color temperature is displayed as a parameter of the white balance is displayed together with the image data on the setting screen such that the user can adjust the color temperature or in a second adjustment mode in which a parameter of the white balance other than the color temperature is adjusted in a direction different from the direction in which the color temperature is adjusted. The image data is read from the storage medium and subjected to a white balance correction according to the parameters set via the setting screen.

<CIT> discloses an image processing apparatus that includes an illumination light intensity calculating unit that determines, from a brightness value based on a pixel value of each pixel forming an original image, an illumination light intensity image having as a pixel value an illumination light component of the original image, using at least one smoothed image having the brightness value that is smoothed using at least a smoothing degree, and a light source color calculating unit that determines an illumination light source color as a light source color of illumination light in accordance with the original image and the illumination light intensity image.

<CIT> discloses an imaging device that includes a display on which a captured image of a subject is displayed, a first corrector to correct a first parameter related to shooting of the subject, a second corrector to correct a second parameter related to image processing to the captured image, and an automatic correction controller to determine a correction amount for the second parameter on the basis of a correction amount for the first parameter corrected by the first corrector, and control the second corrector to correct the second parameter by the determined correction amount.

This document describes methods and systems for a white-balance (WB) mode which is a hybrid of automatic and manual WB modes, in a camera system of an electronic device. The hybrid WB (HWB) mode has a manual mode which may provide options for a user to select a WB setting, via a camera user interface (UI) in a live-preview mode, along a continuous WB-adjustment range. In this way, the user's desired color can be applied with respect to images or video captured by the camera system. The camera UI includes a manual WB control to manually adjust the white balance. In addition, a WB module determines target WB gains corresponding to the manual WB control relative to an initial automatic WB (AWB) decision for a current frame displayed in the live-preview mode (e.g., pre-processing) of the camera application. In an example, a look-up-table (LUT) correlating to the manual WB control is used to compute the target WB gains.

In aspects, a method for an HWB mode in a camera application of an electronic device is disclosed. The method includes presenting, during a live-preview mode of the camera application, a manual white-balance (MWB) control on a camera user interface displayed on a display device of the electronic device, the MWB control configured to enable user selection of a WB setting for performing a manual adjustment on a white balance with respect to a current frame presented in the live-preview mode of the camera application. The method also includes determining, using an AWB mode, an AWB decision on a red-green-blue (RGB) scale for the current frame based on one or more characteristics of the current frame and receiving a user input that adjusts the MWB control displayed in the camera user interface. In addition, the method includes determining, using an MWB mode, target WB gains for a WB adjustment of the current frame based on the user input to the MWB control and relative to the AWB decision, the AWB decision used as initial WB gains to determine the target WB gains that correspond to the user input to the MWB control. Also, the method includes applying the target WB gains to the current frame presented in the live-preview mode of the camera application for color correction.

In aspects, a mobile electronic device comprises a display device, a camera system, one or more processors, and memory. The memory stores instructions that, when executed by the one or more processors, cause the one or more processors to implement an HWB mode for the camera system by performing the method of any one of the methods described above.

In aspects, a computer-readable medium comprising instructions which, when executed by one or more processors, cause the one or more processors to carry out the method described above.

This summary is provided to introduce simplified concepts concerning a hybrid white-balance mode in a camera system of an electronic device, which is further described below in the Detailed Description and Drawings. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

The details of one or more aspects of a hybrid white-balance mode in a camera system of an electronic device are described in this document with reference to the following drawings. The same numbers are used throughout the drawings to reference like features and components:.

This document describes methods and systems for a hybrid white-balance mode in a camera system of an electronic device. This mode provides an MWB control that uses a decision made by an AWB mode as a starting point of an MWB adjustment. Rather than predefining several preset WB illuminant points, the MWB adjustment is based on an AWB decision for a current frame and then enables a user to manually adjust the white balance to achieve a desired color correction. The MWB control provides different directions of color that can be adjusted.

For example, the MWB control can be used to adjust the white balance between red colors (higher color temperatures) and blue colors (lower color temperatures) and/or between purple and green colors. By moving the MWB control toward the red color, the color tone becomes warmer. Moving the MWB control toward the blue color results in a cooler color tone. Similarly, adjusting the MWB control toward the purple color results in a warmer color tone, while a direction toward the green color causes a cooler color tone.

The MWB control also includes different user-selectable mode-variants that enable a user input to manually adjust the white balance in different ways. For example, the user can select a touch-WB variant and then touch a location on the display device to designate a region of interest, corresponding to the user's touch, to be used for the AWB decision. In this way, the AWB decision for a current frame is calculated based on the user-selected region of interest in the current frame.

In another example, the user can select a continuous-control or "preference"-color variant, which may provide one or more sliders that the user can use to manually adjust the white balance. A user-selected location on the slider is determined using a look-up-table that defines WB gains for at least the end points of the slider. In another example, the target WB gains can be computed in real time based on the calibrated illuminants' reference lines.

In yet another example, the user can select a preset-illuminants variant, which provides a slider that the user can use to manually adjust the white balance. A user-selected location on the slider may be determined by interpolation between the nearest preset illuminant points on opposing sides of the user-selected location.

While features and concepts of the described methods and systems for a hybrid white-balance mode in a camera system of an electronic device can be implemented in any number of different environments, aspects are described in the context of the following examples.

<FIG> illustrates an example implementation <NUM> of an electronic device (e.g., electronic device <NUM>) capable of implementing a hybrid white-balance mode in a camera system of an electronic device in accordance with the techniques described herein. The electronic device <NUM> is illustrated as including a camera system <NUM>, a camera application <NUM>, and a white-balance module <NUM> ("WB module <NUM>"). In aspects, the camera application <NUM> includes a live-preview mode <NUM> and a manual-white-balance control <NUM> ("MWB control <NUM>"). Further, the WB module <NUM> includes a hybrid white-balance mode <NUM> ("HWB mode <NUM>") and a white-balance-mode control <NUM> ("WB-mode control <NUM>"). The HWB mode <NUM> includes both a manual mode (e.g., MWB mode) and an automatic mode (e.g., AWB mode) and enables the WB module <NUM> to use a WB-mode control <NUM> to switch between and/or combine functionalities of the manual and automatic WB modes.

The camera application <NUM> provides a camera UI <NUM>, which may be rendered via a display device <NUM> of the electronic device <NUM>. The live-preview mode <NUM> of the camera application <NUM> is rendered via the camera UI <NUM> to provide real-time video of a current scene within a field of view (FOV) of a lens of the camera system <NUM>, causing display device <NUM> to act as a viewfinder for the camera system <NUM>.

The WB module <NUM> is configured to automatically estimate WB settings for the camera system <NUM> to adjust color rendered in the live-preview mode <NUM> of the camera application <NUM>. In aspects, the WB module <NUM> may automatically select and apply WB gains based on one or more predefined illuminant conditions that most-closely correspond to a current illuminant condition surrounding the electronic device <NUM>.

The MWB control <NUM> is configured to enable the user to adjust one or more directions of color, e.g., between red and blue or between purple and green. In aspects that are further described herein, the user can use the MWB control <NUM> to manually adjust the WB gains along a continuous range that is not limited to specific gain settings defined by the predefined illuminant conditions.

The HWB mode <NUM> combines features and functionalities of the WB module <NUM> and the MWB control <NUM> to enhance the precision of accurate color correction, providing fine control of the color correction to the user. In an example, the HWB mode <NUM> enables the WB module <NUM> to switch between auto and manual modes for correcting the white balance of one or more frames in the live-preview mode <NUM> of the camera application <NUM>. In addition, when switching to the manual mode, the WB module <NUM> can first estimate WB gains for the current frame using the auto mode and then adjust the WB gains, according to a user input to the MWB control <NUM>, relative to the initial WB gain estimation.

In more detail, consider <FIG>, which illustrates an example implementation <NUM> of the electronic device <NUM> from <FIG>. The electronic device <NUM> of <FIG> is illustrated with a variety of example devices, including a smartphone <NUM>-<NUM>, a tablet <NUM>-<NUM>, a laptop <NUM>-<NUM>, a desktop computer <NUM>-<NUM>, a computing watch <NUM>-<NUM>, computing spectacles <NUM>-<NUM>, a gaming system <NUM>-<NUM>, a home-automation and control system <NUM>-<NUM>, and a microwave <NUM>-<NUM>. The electronic device <NUM> can also include other devices, e.g., televisions, entertainment systems, audio systems, automobiles, drones, track pads, drawing pads, netbooks, e-readers, home security systems, and other home appliances. Note that the electronic device <NUM> can be mobile, wearable, non-wearable but mobile, or relatively immobile (e.g., desktops and appliances).

The electronic device <NUM> also includes one or more computer processors <NUM> and one or more computer-readable media <NUM>. The one or more computer-readable media <NUM> includes memory media <NUM> and storage media <NUM>. Applications <NUM> and/or an operating system <NUM> implemented as computer-readable instructions on the computer-readable media <NUM> can be executed by the computer processors <NUM> to provide some or all of the functionalities described herein. For example, the computer-readable media <NUM> can include the camera application <NUM>, the WB module <NUM>, and the MWB control <NUM>.

The electronic device <NUM> also includes the camera system <NUM>, which is configured to capture images, video, and audio. Any suitable camera system <NUM> may be implemented in the electronic device <NUM>. The camera system <NUM> may be a digital camera that converts light captured by a lens to digital data representing a scene within the field of view of the lens.

The electronic device <NUM> may also include a network interface <NUM>. The electronic device <NUM> can use the network interface <NUM> for communicating data over wired, wireless, or optical networks. By way of example and not limitation, the network interface <NUM> may communicate data over a local-area-network (LAN), a wireless local-area-network (WLAN), a personal-area-network (PAN), a wide-area-network (WAN), an intranet, the Internet, a peer-to-peer network, point-to-point network, or a mesh network.

Various implementations of the camera system <NUM> can include a System-on-Chip (SoC), one or more Integrated Circuits (ICs), a processor with embedded processor instructions or configured to access processor instructions stored in memory, hardware with embedded firmware, a printed circuit board with various hardware components, or any combination thereof.

The electronic device <NUM> also includes one or more sensors <NUM>, which can include any of a variety of sensors, including an audio sensor (e.g., a microphone), a touch-input sensor (e.g., a touchscreen), an image-capture device (e.g., a camera or video-camera), proximity sensors (e.g., capacitive sensors), or an ambient light sensor (e.g., photodetector).

The electronic device <NUM> can also include a display device (e.g., the display device <NUM>). The display device <NUM> can include any suitable touch-sensitive display device, e.g., a touchscreen, a liquid crystal display (LCD), thin film transistor (TFT) LCD, an in-place switching (IPS) LCD, a capacitive touchscreen display, an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, super AMOLED display, and so forth. The display device <NUM> may be referred to as a display or a screen, such that content may be displayed on-screen.

These and other capabilities and configurations, as well as ways in which entities of <FIG> and <FIG> act and interact, are set forth in greater detail below. These entities may be further divided, combined, and so on. The implementation <NUM> and the detailed illustrations of <FIG> illustrate some of many possible environments, devices, and methods capable of employing the described techniques, whether individually or in combination with one another.

<FIG> illustrates an example implementation <NUM> of an MWB control in a camera system of an electronic device. In <FIG>, a group of preset WB illuminant gains (e.g., presets <NUM>) are shown along a range. Each of the preset WB illuminant gains may be calibrated by lab-standard illuminants. Some example preset WB illuminant gains may correspond to shade <NUM>-<NUM>, cloudy <NUM>-<NUM>, outdoor <NUM>-<NUM>, fluorescent <NUM>-<NUM>, warm fluorescent <NUM>-<NUM>, incandescent <NUM>-<NUM>, and horizon <NUM>-<NUM>. In operation, however, there exists a wide variety of illuminations that may deviate from the lab-standard illuminants. Accordingly, the techniques described herein provide manually adjustable functionality between the preset WB illuminant gains. For example, <FIG> includes a slider <NUM> that is based on the preset WB illuminant gains. The slider <NUM> may be an instance of the MWB control <NUM> in <FIG>. A user can slide an indicator <NUM> along the slider <NUM> between (and including) shade <NUM>-<NUM> and horizon <NUM>-<NUM> to determine WB gains for correcting a scene illuminant of the scene in the field of view of the camera.

The HWB mode <NUM> can be selected by passing metadata information from the camera application <NUM> to the WB module <NUM>. The metadata may include an indication (e.g., flag) of manual mode "ON. " In response to receiving such an indication, the WB module <NUM> disables the AWB mode and switches to the MWB mode. The MWB mode includes multiple preset gains corresponding to preset WB points (e.g., the presets <NUM>).

To provide a continuous WB gain computation between the presets <NUM>, an interpolation function is implemented. The interpolation function uses an interpolation ratio based on a correlated color temperature of a preset WB point (e.g., presets <NUM>). Table <NUM> illustrates example color temperatures (in Kelvin (K)) of the presets <NUM>.

Consider <FIG>, which illustrates an example <NUM> of an interpolation ratio and color correction temperature. In the illustrated example, a user-selected WB point is located between two preset WB points (e.g., MWB(i) <NUM> and MWB(i+<NUM>) <NUM>), which may correspond to two adjacent presets <NUM> from <FIG>. The term i refers to the index of the preset WB points, including those shown in Table <NUM> (e.g., i ∈ [<NUM>, <NUM>]). A distance <NUM> between MWB(i) <NUM> and MWB(i+<NUM>) <NUM> may be normalized to [<NUM>, <NUM>], with zero at MWB(i) <NUM> and one at MWB(i+<NUM>) <NUM>. When the user selects a location for the indicator <NUM> on the slider <NUM>, a corresponding WB point (e.g., user-selected WB point <NUM>) on the slider <NUM> is determined, and a ratio r (e.g., ratio <NUM>) is obtained. The MWB(i) <NUM>, the MWB(i+<NUM>) <NUM>, and the ratio <NUM> are sent to an AWB algorithm to compute target WB gains for the user-selected WB point <NUM>.

Assume the color correction temperature (CCT) of the illuminant MWB (i+<NUM>) is CCT(i+<NUM>), and the CCT of the illuminant MWB(i) is CCT(i). Then, the CCT of the target manual WB point (CCT(T)) can be computed using the following equation: <MAT>.

In Equation <NUM>, the illuminant CCT ranges from high to low (e.g., from MWB(i) to MWB(i+<NUM>)). The target WB gains can then be computed by interpolation of the preset gains of the preset WB points by using CCT as the interpolation weight. In aspects, the CCT is normalized by a Mired CCT (MCCT) before being used as the weighting for the computation of the target WB gains. The MCCT(i), MCCT(i+<NUM>), and MCCT(T) are computed using the following: <MAT>.

Target WB gains (TMWB) may include gains on a red-green-blue (RGB) scale. In aspects, Rgain and Bgain can be computed and Ggain can be kept constant. For example, TMWB(Rgain, Ggain, Bgain) can be computed using the following: <MAT> <MAT>.

<FIG> depicts an example flowchart <NUM> of a feedback loop between a camera application and the WB module for an MWB selection between preset WB points. To implement the WB mode, a feedback loop may be applied between the camera application <NUM> and the WB module <NUM> of <FIG>. At <NUM>, WB is selected via the camera application <NUM>. For example, the WB mode may have been selected or triggered by a user or a setting in the camera application <NUM>. In an example, the user adjusts the location of the indicator <NUM> on the slider <NUM> presented via the camera UI <NUM>.

The camera application <NUM> provides metadata (e.g., metadata <NUM>), associated with the user-selected WB point <NUM> on the slider <NUM>, to the WB module <NUM>. If the location of the user-selected WB point <NUM> is between two presets <NUM>, then the metadata <NUM> may include the two preset WB points (MWB(i) and MWB(i+<NUM>)) on opposing sides of the user-selected WB point <NUM>. In aspects, the two preset WB points may be the two nearest presets on opposing sides of the user-selected WB point <NUM>. The camera application <NUM> also provides a ratio r (e.g., the ratio <NUM>) based on the location of the user-selected WB point <NUM> on the slider <NUM> with respect to the two nearest preset WB points. However, if the user selected one of the presets <NUM> (e.g., the location of the user-selected WB point <NUM> corresponds to a location of a preset <NUM>), then the ratio <NUM> is zero (e.g., r=<NUM>) and the two nearest preset WB points are treated as a single point (e.g., MWB(i) = MWB(i+<NUM>)). In aspects, the metadata <NUM> also includes an indication (e.g., manual mode flag) of whether the manual WB mode is on or off.

Upon receiving the metadata <NUM>, the WB-mode control <NUM> of the WB module <NUM> determines whether to compute the WB gains using a manual mode (e.g., MWB mode <NUM>) or an auto mode (e.g., AWB mode <NUM>). If the manual mode flag is on ("YES"), then the WB module <NUM> performs a manual WB gain computation <NUM> using the MWB mode <NUM> to compute target manual WB gains <NUM> based on the user-selected WB point <NUM> (e.g., via interpolation between the two preset WB points). However, if the manual mode flag is off ("NO"), then the WB module <NUM> computes WB gains using the AWB mode <NUM>, which relies on conventional techniques for automatic WB gain computation.

The WB module <NUM> may output to the camera application <NUM> information, including the target WB gains for the user-selected WB point <NUM> (TMWB(Rgain, Ggain, Bgain)). The feedback communication between the WB module <NUM> and the camera application <NUM> may occur in real time to reduce latency and provide a user experience having a smooth WB transition process when sliding the indicator <NUM> along the slider <NUM>.

<FIG> illustrates example implementations <NUM> and <NUM> of an HWB mode in a camera system of an electronic device. In <FIG>, The HWB mode <NUM> involves using the automatic WB gain computation ("AWB decision") of the AWB mode <NUM> as a starting point of the manual WB gain computation <NUM> ("MWB gain computation <NUM>"). For example, rather than predefine several preset WB illuminant points (e.g., presets <NUM>), the manual adjustment of the white balance may be based on the AWB decision of a current frame (e.g., a current frame shown in the live-preview mode <NUM> of the camera application <NUM>). The WB module <NUM> may then apply fine or "preference" color adjustment to achieve the user's preferred color.

Directions of color adjustment generally include directions between red and blue (e.g., implementation <NUM>) and/or between purple and green (e.g., implementation <NUM>). Color tone becomes warmer (e.g., higher color temperature) by sliding the indicator <NUM> in the red direction (e.g., toward a red end point <NUM>) or the purple direction (e.g., toward a purple end point <NUM>). The color tone becomes cooler (e.g., cooler color temperature) by sliding the indicator <NUM> in the blue direction (e.g., toward a blue end point <NUM>) or the green direction (e.g., toward a green end point <NUM>).

The WB algorithm to control this manual mode is based on the predefined preference color tuning ranges in red, blue, green, and/or purple color directions. The current frame's AWB decision is defined by [Rgain, Ggain, Bgain]. Example gain adjustment tuning parameters are defined in Table <NUM>, where only Rgain and Bgain are to be adjusted. Based on the location of the user-selected WB point <NUM> on the slider <NUM> presented in the camera UI <NUM> of the camera application <NUM>, different red and blue gain adjustments can be applied in the WB algorithm.

In an example, if a user slides the indicator <NUM> toward the red end point <NUM>, the WB module <NUM> applies a preference gain adjustment (e.g., [RgainAdjust, BgainAdjust]=[<NUM>, <NUM>]) to achieve a warmer color cast. The strength of the warmer color cast depends on the user-selected WB point along the slider <NUM>. If the user slides the indicator <NUM> all the way to the red end point <NUM>, the preference gain reaches the maximum red adjustment and the minimum blue adjustment (e.g., [RmaxAdjust, BminAdjust]).

The camera UI <NUM> provides the location information of the user-selected WB point <NUM> on the slider <NUM> to enable the WB module <NUM> to provide preference-color adjusted gains. A distance from the center of the slider <NUM> to the two color ending points is defined with the location parameter [R, B] (e.g., MWB point location <NUM>). The location information at a slider center point <NUM> is defined as [R=<NUM>, B=<NUM>]. The red end point <NUM> location information is defined as [R=<NUM>, B=-<NUM>], and the blue end point <NUM> location information is defined as [R=-<NUM>, B=<NUM>]. The purple and green slider may be similarly defined using [R=<NUM>, B=<NUM>] for the purple end point <NUM> location information and [R=-<NUM>, B=-<NUM>] for the green end point <NUM> location information. The location information on the sliders is summarized in Table <NUM>:.

The camera UI <NUM> sends the MWB point location <NUM> [R, B] to the WB module <NUM> to compute the user-preferred WB gains. Preference-gain adjustment ranges are defined in WB tuning headers. Assume the current frame's AWB decision is [Rgain, Ggain, Bgain]. With this preference-color adjustment information (e.g., the MWB point location <NUM>) sent from the camera UI <NUM>, the WB module <NUM> can update the new gains by using the following, where R ∈ [-<NUM>, <NUM>] and B ∈ [-<NUM>, <NUM>]: <MAT>.

<FIG> depicts an example flowchart <NUM> of a feedback loop between a camera application and the WB module for the HWB mode based on preference color adjustment. To implement the MWB mode <NUM> described above, the feedback loop may be applied between the camera application <NUM> and the WB module <NUM> of <FIG>, similar to the feedback loop of <FIG>. At <NUM>, hybrid WB is selected via the camera application <NUM>. For example, the HWB mode <NUM> may have been selected or triggered by a user or a setting in the camera application <NUM>. In an example, the user adjusted the location of the indicator <NUM> on the slider <NUM> presented via the camera UI <NUM>.

The camera application <NUM> provides the location of the user-selected WB point <NUM> on the slider <NUM> (e.g., [R, B]). The camera application <NUM> provides metadata (e.g., metadata <NUM>) to the WB module <NUM>. The metadata <NUM> may include the red gain adjustment ratio R having a valid range of [-<NUM>, <NUM>]. The metadata <NUM> may also include the blue gain adjustment ratio B having a valid range of [-<NUM>, <NUM>]. In addition, the metadata <NUM> may include the manual mode flag, as described above.

The WB-mode control <NUM> triggers the WB module <NUM> to use the AWB mode <NUM> of the HWB mode <NUM> to compute an AWB gain computation <NUM> ("AWB decision") for a current frame of the live-preview mode <NUM> of the camera application <NUM>. The AWB gain computation <NUM> for the current frame may then be used as a starting point (e.g., [R=<NUM>, B=<NUM>]) to perform the MWB gain computation <NUM>, via the MWB mode <NUM> of the HWB mode <NUM>, using the red gain adjustment ratio R and the blue gain adjustment ratio B in the metadata. The MWB gain computation <NUM> returns an output that is provided to the camera application <NUM>, where the output includes the target manual WB gains <NUM> (e.g., TMWB(Rgain, Ggain, Bgain)) for the user-selected WB point <NUM>. The feedback communication between the camera application <NUM> and the WB module <NUM> occurs in real time to reduce latency between the movement of the MWB control <NUM> and rendered results of corresponding color correction in the content displayed via the camera UI <NUM>. Such real-time results provide a smooth WB transition process when the user moves the MWB control <NUM> in the camera application <NUM>.

<FIG> illustrates examples <NUM> and <NUM> of a camera UI having an MWB control for adjusting the white balance of frames shown in a live-preview mode of a camera application on an electronic device. In example <NUM>, the camera UI <NUM> includes slider controls for adjusting different parameters of the camera system <NUM> or the camera application <NUM>. The slider controls may include the MWB control <NUM>, which may correspond to the slider <NUM> in <FIG>, <FIG>, and <FIG>. Additional slider controls may include an exposure control <NUM> to adjust the exposure of an image sensor of the camera system <NUM>, a brightness control <NUM> to adjust brightness of the image or video, and a contrast control <NUM> to adjust the contrast of the image or video. Additional slider controls may also be implemented to enable manual adjustment of additional parameters of the camera system <NUM> or the camera application <NUM>, including focus, zoom, saturation, hue, and so forth.

The MWB control <NUM> can be presented via the camera UI <NUM> in any suitable form. In the example <NUM>, the MWB control <NUM> includes the slider <NUM> and multiple indicators on the slider <NUM>. These indicators may include an upper bound <NUM>, a lower bound <NUM>, and the indicator <NUM> representing a user-selectable target WB point (e.g., the user-selected WB point <NUM>). The slider <NUM> extends between the upper bound <NUM> and the lower bound <NUM>. Each of the indicators <NUM>, <NUM>, and <NUM> can be a UI element, including an object, an icon, a symbol, a thumbnail image, and so forth. In one example, the upper bound <NUM> indicator includes a daylight icon to indicate to the user that moving the indicator <NUM> toward the upper bound <NUM> causes the color tones of the image to become warmer. In an example, the lower bound <NUM> indicator includes an incandescent light icon to indicate to the user that moving the indicator <NUM> toward the lower bound causes the color tones of the image to become cooler. As described in <FIG>, presets <NUM> may exist between the upper bound <NUM> and the lower bound <NUM>. One or more of the presets <NUM> may also be represented by a respective UI element positioned on the slider <NUM> of the MWB control <NUM>. In aspects, the MWB control <NUM> may be displayed in the camera UI <NUM> in response to a user input that calls for the MWB control <NUM> to be displayed. The MWB control <NUM> may otherwise be hidden from the camera UI <NUM> to reduce screen clutter.

In example <NUM>, the MWB control <NUM> includes two separate sliders: a first slider <NUM> and a second slider <NUM>. The example <NUM> may correlate to the example implementations <NUM> and <NUM> described in <FIG>. For instance, the first slider <NUM> may correlate to the example implementation <NUM>, which enables manual WB adjustment between red and blue color tones. The second slider <NUM> may correlate to the example implementation <NUM>, which enables manual WB adjustment between purple and green color tones. Alternatively, the first slider <NUM> may adjust white balance between purple and green color tones, while the second slider <NUM> adjusts the white balance between red and blue color tones.

By relying on preference color ranges defined in the WB tuning headers, rather than preset WB illuminant gains (e.g., presets <NUM>), the MWB control (e.g., the MWB control <NUM>, the first slider <NUM>, or the second slider <NUM>) may be displayed as a bar with an initial AWB decision located at the center of the bar. To provide an indication of color direction associated with each MWB control, colored sliders or objects (e.g., icons) may be implemented.

<FIG> illustrates an example implementation <NUM> of a white-balance lock ("WB lock") in a camera application of an electronic device. For example, the WB module <NUM> from <FIG> may include a lock feature (e.g., WB lock), which is usable to lock the AWB gains and apply the locked AWB gains to subsequent frames in the live-preview mode <NUM>.

Some target scenes may be challenging for the WB module <NUM> to provide an accurate AWB decision. These challenges may be due to various factors, including the scene's lighting conditions or the colors of objects in the scene. To increase accuracy, the user can point the camera of the electronic device <NUM> at a grey or white object under a similar illuminant to allow the WB module <NUM> to determine an AWB decision on the current frame, and then the user can select the WB lock to prevent further changes to the white balance. While the WB lock is enabled, the WB gains are not adjustable, neither automatically nor manually. Consequently, the user can point the camera to a target scene of interest, and the white balance remains unchanged from the AWB decision made for the grey or white object. In this way, the WB module <NUM> is not misled by one or more colors in the camera's field of view of the target scene.

Using the grey or white object can be useful if such an object is available, because accurate results can be obtained without the user making further manual adjustments to the white balance. Generally, the WB module <NUM> can automatically correct the scene illuminant by using a neutral color in the camera's field of view.

In the example illustrated in <FIG>, the camera's field of view captures a scene having a neutral-color bicycle <NUM> and displays the scene in a live-preview mode of the camera UI <NUM> (e.g., camera UI <NUM>-<NUM>). The WB module <NUM> automatically determines the WB gains according to the neutral-color bicycle <NUM> (or according to a neutral-color wall behind the bicycle). To maintain the determined WB gains, the user can provide input to lock the WB gains. For example, a UI element (e.g., icon <NUM>) may be selected by a user input to initiate the WB lock. In the illustrated example, the icon <NUM> may be an unlocked lock indicating that the current WB gains are not yet locked.

Subsequent to the user input locking the WB gains, the user moves the camera field of view to target another object of interest (e.g., a chair <NUM> shown in camera UI <NUM>-<NUM>), which may be under the same illuminant. Accordingly, the same WB gains determined for the bicycle <NUM> are now applied to the capture of the chair <NUM>, regardless of how challenging the scene content is for the WB module <NUM>. In addition, the icon <NUM> can change (e.g., to icon <NUM>) to indicate that the WB gains are locked in place. In this example, the icon <NUM> is a locked lock. Because this example relies on the WB module <NUM> operating in the AWB mode, the MWB control <NUM> is not displayed. However, the lock feature described herein can be implemented in conjunction with the MWB control <NUM> to prevent unwanted adjustment of the WB gains through inadvertent user input to the MWB control <NUM>.

<FIG> depicts an example flowchart <NUM> of communication between the camera application <NUM> and the WB module <NUM> in relation to the WB lock. After the WB module <NUM> calculates an AWB decision for a current frame (e.g., a first frame), then at <NUM>, a user input selects the WB lock via the camera UI <NUM>. In response to user selection of the WB lock for the current frame, the camera application <NUM> passes a WB lock flag <NUM> to the WB module <NUM> and, at <NUM>, the WB lock is enabled. At <NUM>, the AWB auto mode locks to the current frame WB gains. In this way, the WB module <NUM> skips the AWB calculation of frames subsequent to the current frame and maintains the current frame's WB gains [Rgain, Ggain, Bgain] until the WB lock is disabled. The WB module <NUM> returns the current frame's WB gains <NUM> to the camera application <NUM>.

<FIG> depicts a flowchart <NUM> of a communication between the camera application <NUM> and the WB module <NUM> to implement a touch-WB variant of the MWB mode <NUM>. In accordance with touch WB, when the user touches the display device <NUM>, a touch region of interest (ROI) is defined. Then, in some instances, ROI information (e.g., sensor coordinate (x, y, dx, dy) on which the touch ROI is defined) is sent to the WB module <NUM>, and the WB module <NUM> uses RGB information only corresponding to the touch ROI to compute the AWB decision for the entire frame.

In many camera applications on touchscreen devices, the default touch action (e.g., tap) in the camera UI <NUM> may trigger touch auto-focus and touch auto-exposure. Thus, it may be useful for the user to have pre-knowledge on selecting a neutral object for white balancing the scene illuminant. To avoid user confusion with conventional touch gestures in many camera applications, the touch WB may be triggered separately from touch auto-focus and touch auto-exposure. Accordingly, the touch WB may be enabled in response to an explicit user input that enables touch input to adjust white balance and disabled at other times by default.

At <NUM>, a manual WB mode (specifically, touch WB) is selected via the camera UI <NUM>. The camera application <NUM> passes touch ROI information <NUM> of a user touch to the WB module <NUM>. The touch ROI information <NUM> may include a touch WB flag and touch-WB ROI. In particular, the touch-WB ROI includes the ROI information. In response to receiving the touch ROI information having the touch WB flag, the WB module <NUM>, at <NUM>, switches the MWB mode <NUM> to the touch WB. The touch WB enables a touch pipeline separate from touch focus and touch exposure. For example, by activating the touch WB of the MWB mode <NUM>, the user can touch a location on the camera UI <NUM> to be used for the AWB decision without activating or causing a change to the auto focus and/or the auto exposure.

At <NUM>, the WB module <NUM> computes a final AWB decision of the frame based on RGB information within the touch ROI. In this way, the WB module <NUM> uses the AWB mode to compute the AWB decision based on the information obtained from the MWB mode (e.g., the touch-WB variant). The WB module <NUM> then passes the final AWB decision (e.g., WB gains <NUM> [Rgain, Ggain, Bgain] based on the RGB information located inside the touch ROI) to the camera application <NUM> for color correction.

<FIG> illustrates an example implementation <NUM> of initiating a touch-WB variant of the MWB mode <NUM> via the camera UI <NUM> of the camera application <NUM>. In aspects, the MWB control <NUM> may provide selectable options, via the camera UI <NUM>, to enable the user to select a particular MWB mode from a plurality of MWB-mode variants. Some example MWB-mode variants include a touch-WB variant, a preset-illuminants variant, and a preference-color variant. The selectable objects, when selected by a user input, may initiate a corresponding MWB-mode variant. For example, in instance <NUM>-<NUM>, the camera UI <NUM> provide a UI element <NUM> having selectable options including a first icon <NUM> (e.g., touch icon), a second icon <NUM> (e.g., preference-color icon), and a third icon <NUM> (e.g., preset-colors icon). The first icon <NUM> may correspond to the touch-WB variant of the MWB mode <NUM>. The second icon <NUM> may correspond to the preference-color variant of the MWB mode <NUM>. The third icon <NUM> may correspond to the preset-illuminants variant of the MWB mode <NUM>. The MWB control <NUM> can be provided in any suitable way, including via a dropdown menu, a popup window, or an overlay. Further, the MWB control <NUM> may be accessible via the settings of the camera application <NUM>.

In the instance <NUM>-<NUM>, the user directs the camera toward their dog <NUM>, sitting on a floor <NUM> in front of a wall <NUM>. The camera UI <NUM> is currently showing a live preview of the scene of the dog <NUM>. The HWB mode <NUM> described herein enables the user to control color correction, via a WB adjustment, to the frames shown in the live-preview mode <NUM> of the camera application <NUM>.

In the instance <NUM>-<NUM>, the user touches (e.g., touch <NUM>) the first icon <NUM>, which triggers initiation of the touch-WB variant of the MWB mode <NUM>. In response, the camera application <NUM> switches to the touch-WB variant. In some cases, the MWB control <NUM> is removed from the camera UI <NUM> in response to the touch <NUM> to reduce screen clutter. In another example, the display of the MWB control <NUM> remains. Because the user has selected the touch-WB variant, the electronic device <NUM> does not perform an AWB decision until after receiving an explicit user input that selects an ROI on the display device <NUM>. In instance <NUM>-<NUM> of the illustrated example, the user input (e.g., touch <NUM>) selects the wall <NUM> behind the dog <NUM> because the wall has a neutral color. The WB module <NUM> then performs an AWB decision using RGB information only within a region (e.g., the touch ROI) of the display device <NUM> touched by the user's finger. Using the AWB decision, the camera application <NUM> can adjust the white balance of the subsequent frames shown in the live-preview mode <NUM> and, consequently, in a captured image or video captured by the camera system <NUM> for persistent storage. In addition, the camera UI <NUM> can provide a lock option (e.g., the WB lock described in <FIG>) by displaying the icon <NUM>, which is selectable by the user to lock the WB gains. Then, the user can move the camera's field of view to capture an image or video of a different object (not shown), and the camera system <NUM> uses the same WB gains as those calculated based on the selection of the wall <NUM>.

<FIG> illustrates an example implementation <NUM> of selecting a preference-color variant of the MWB mode <NUM> via the camera UI <NUM> of the camera application <NUM>. In instance <NUM>-<NUM>, a user input (e.g., touch <NUM>) selects, via the UI element <NUM>, the second icon <NUM>, which corresponds to the preference-color variant of the MWB mode <NUM>. In response to the selection of the second icon <NUM>, the camera application <NUM>, in instance <NUM>-<NUM>, provides one or more WB sliders (e.g., the first slider <NUM> and/or the second slider <NUM>) via the camera UI <NUM> to enable the user to manually adjust the white balance of the frames shown in the live-preview mode by moving the indicator(s) (e.g., indicators <NUM>-<NUM> and <NUM>-<NUM>) on the WB slider(s) (e.g., the sliders <NUM> and <NUM>). Then, as described above, the camera application determines the location of the indicators <NUM>-<NUM> and <NUM>-<NUM> on the WB slider(s) <NUM> and <NUM> and passes that location information to the WB module <NUM>. The WB module <NUM> calculates the AWB decision based on the location information and returns the AWB decision (e.g., WB gains) to the camera application <NUM> for color correction.

<FIG> illustrates an example implementation <NUM> of selecting a preset-illuminants variant of the MWB mode via the camera UI of the camera application. In instance <NUM>-<NUM>, the user input (e.g., touch <NUM>) selects, via the UI element <NUM>, the third icon <NUM>, which corresponds to the preset-illuminants variant of the MWB mode <NUM>. In response to the selection of the third icon <NUM>, the camera application <NUM>, in instance <NUM>-<NUM>, provides one or more WB controls (e.g., the MWB control <NUM>) via the camera UI <NUM> to enable the user to manually adjust the white balance of the frames shown in the live-preview mode by moving the indicator <NUM> on the MWB control <NUM>. Then, as described above, the camera application <NUM> determines the location of the indicator <NUM> on the MWB control <NUM>, such as by interpolation between the nearest preset illuminant points (e.g., presets <NUM>) on opposing sides of the indicator <NUM>. The WB module <NUM> calculates the AWB decision based on the interpolation of the preset manual WB gains corresponding to the presets <NUM> and returns the AWB decision to the camera application <NUM> for color correction.

<FIG> illustrates an example implementation <NUM> of an MWB adjustment by preference gains. Similar to the implementation <NUM> in <FIG>, the implementation <NUM> includes an MWB slider (e.g., slider <NUM>) with two end points (e.g., red end point <NUM>, blue end point <NUM>). In <FIG>, the end points <NUM> and <NUM> correspond to tunable red and blue adjustments, respectively. A preference color adjustment look-up-table (LUT) may be implemented in the tuning header and enable maximum red and blue gains (e.g., the red end point <NUM> and the blue end point <NUM>, respectively) to be tunable. In aspects, the red and blue end points <NUM> and <NUM> of the slider <NUM> are tunable to provide either full coverage of the illuminant range or only a portion of the illuminant range. The MWB control <NUM> is based on the AWB gains, which enables the user to return the manually adjusted WB gains to the original AWB decision. For example, the AWB decision is used to define the center (e.g., slider center <NUM>) of the WB slider <NUM> (e.g., slider center <NUM> is defined as R, B initial gain adjust = [<NUM>, <NUM>]). The gain adjustment LUT, which may be defined in Table <NUM>, is used to define boundary settings of the MWB control <NUM>:.

Based on the user-selected location of the indicator <NUM> on the WB slider <NUM> in the camera UI <NUM>, different red and blue gain adjustments are applied by the WB module <NUM>. To compute the real-time MWB gains and provide an output to the images or video displayed in the camera preview (e.g., the live-preview mode <NUM> of the camera application <NUM>), the information that indicates the location of the user-selected indicator <NUM> on the WB slider <NUM> is passed from the camera UI <NUM> to the WB module <NUM>. If the location of the indicator <NUM> is defined as a floating value dslider ∈ [-<NUM>, <NUM>], it can be illustrated on the slider <NUM> in <FIG>. Corresponding values of the location of the indicator <NUM> are shown in the third column (e.g., "Slider Distance Value") of Table <NUM>.

Using the location dslider, the WB module <NUM> can look up the preference gain adjustment table defined in the tuning header and locate the corresponding gain adjustment values to compute the current MWB gains. Note that the maximum and minimum preference gain adjustment values are tunable based on the needs of each camera sensor. The maximum and minimum preference gain values (e.g., red end point <NUM>, blue end point <NUM>) are defined as [Rmax, Bmin], [Rmin, Bmax], respectively. When the WB module <NUM> receives the location dslider, the corresponding adjustment gain <MAT>, for target red and blue gains respectively, can be computed using the following: <MAT>.

When the WB gain changes, the corresponding CCT and color-correction matrix (CCM) may also change. For manual mode, depending on the maximum gain adjustment range, the new CCT and CCM may be computed together.

In some implementations, a smaller range between the red and blue end points of the slider <NUM> may enable finer adjustment of the white balance, whereas a larger range enables a coarser adjustment of the white balance. Accordingly, the sensitivity of the WB slider <NUM> can be adjusted by defining the respective end points of the WB slider <NUM>. For example, the sensitivity of the WB slider <NUM> can be increased by increasing the range between the red and blue end points on the slider <NUM>. Alternatively, the sensitivity of the WB slider <NUM> can be decreased by decreasing the range between the red and blue end points on the slider <NUM>.

<FIG> illustrates an example plot <NUM> of a color-correction temperature range for preference gain adjustment. In aspects, the CCT range includes full coverage of an illuminant range. However, the CCT range may be bounded to include only a portion of the full illuminant range. The illuminant range includes sub-ranges <NUM> corresponding to at least some of the preset WB points (e.g., the presets <NUM>) and correlated color temperatures described above in Table <NUM>. Any suitable number and size of sub-ranges <NUM> can be used to represent portions of the illuminant range. In the illustrated example, the plot <NUM> includes a first range <NUM>-<NUM>, a second range <NUM>-<NUM>, a third range <NUM>-<NUM>, and a fourth range <NUM>-<NUM>. The first range <NUM>-<NUM> may be defined as a range around calibrated reference points (e.g., calibrated by lab-standard illuminants) and reference lines, including the presets shade <NUM>-<NUM>, cloudy <NUM>-<NUM>, and outdoor <NUM>-<NUM>. The second range <NUM>-<NUM> may be defined as a range around calibrated reference points and reference lines, including the presets fluorescent <NUM>-<NUM> and warm fluorescent <NUM>-<NUM>. The third range <NUM>-<NUM> may be defined as a range around calibrated reference points and reference lines including the preset incandescent <NUM>-<NUM>. The fourth range <NUM>-<NUM> may be defined as a range around calibrated reference points and reference lines including the preset horizon <NUM>-<NUM>.

Conventional camera systems may use only the preset gains (e.g., presets <NUM>). However, the preset gains may not match the true real-world illuminant, which may result in some color cast even when using a manual WB adjustment. Such issues may reduce the effectiveness of the manual WB adjustment. To address this issue, the color-correction algorithm used by the WB module <NUM> is based on the current AWB decision and may still cover the entire illuminant range. In this way, the WB slider <NUM> is centered on the current AWB decision, and the user may slide the indicator <NUM> on the WB slider <NUM> to achieve different color appearances. This approach enhances the accuracy of the color correction over conventional techniques that just rely on the presets <NUM>.

A centerline <NUM> represents a pre-calibrated line between the presets <NUM>. A reference CCT line <NUM> (represented by a dashed line) is a calibrated line that is parallel to the centerline <NUM> but for visual clarity is illustrated as being offset from the centerline <NUM>. Based on current illuminant characteristics, the WB module <NUM> computes in real time the AWB decision (e.g., AWB decision <NUM>) and how it relates to a calibrated center illuminant. The center illuminant is based on the highest color temperature and the lowest color temperature. Sliding the indicator <NUM> on the WB slider <NUM> in <FIG>, for example, toward the red end point <NUM> moves the WB gains along the reference CCT line <NUM> in a first direction <NUM> and adjusts to a warmer color tone. Sliding the indicator <NUM> on the WB slider <NUM> toward the blue end point <NUM> moves the WB gains along the reference CCT line <NUM> in a second direction <NUM>, opposite the first direction <NUM>, on the WB slider <NUM> and adjusts to a cooler color tone.

<FIG> depicts a flowchart <NUM> of an HWB mode by preference color adjustment. At <NUM>, the camera application <NUM> determines metadata <NUM> associated with the indicator <NUM> on the MWB control <NUM> (e.g., WB slider <NUM>) in the camera application <NUM>. The camera application <NUM> sends the metadata <NUM> to the WB module <NUM>. The metadata <NUM> includes location information (e.g., float value dslider) for the indicator <NUM> on the WB slider <NUM> relative to a center (e.g., the slider center <NUM>) of the WB slider <NUM>. The metadata <NUM> may also include a manual mode flag, as described above.

At <NUM>, the WB-mode control <NUM> of the WB module <NUM> receives the metadata <NUM> from the camera application <NUM> and uses the metadata <NUM> to determine whether to proceed with the AWB mode <NUM> or the MWB mode <NUM> for computation of the WB gain adjustments.

At <NUM>, the WB module <NUM> determines if the MWB mode <NUM> is enabled based on the manual mode flag included in the metadata <NUM>. If the MWB mode <NUM> is enabled ("YES" at <NUM>), then at <NUM>, the WB module <NUM> performs a manual WB computation using the location information in the metadata <NUM>. If, however, the MWB mode <NUM> is disabled ("NO" at <NUM>), then at <NUM>, the WB module <NUM> uses an AWB mode <NUM> to perform an auto WB computation based on characteristics of a current frame. Then at <NUM>, the WB module provides an output (e.g., WB gains [Rgain, Ggain, Bgain], CCT, CCM) to the camera application <NUM> to enable the camera application <NUM> to adjust the WB gains for the frames in the live-preview mode <NUM>.

<FIG> illustrates an example method <NUM> of implementing the MWB control <NUM> from <FIG> in a preview mode of a camera system of an electronic device. In the illustrated example, the electronic device <NUM> provides a colored slider <NUM> in the camera preview of a camera application rendered on the display device <NUM>. The colored slider <NUM> is dedicated for manual WB adjustment and is an instance of the slider <NUM>. In aspects, the colored slider <NUM> is provided in a vertical orientation. However, the colored slider <NUM> may be presented in a horizontal orientation or a diagonal orientation. The color toward the top (e.g., top end <NUM>) of the colored slider <NUM> may be rendered with a warm color (e.g., orange, red) to indicate a warmer color tone adjustment direction <NUM>. The color toward the bottom (e.g., bottom end <NUM>) of the colored slider <NUM> may be rendered with a cool color (e.g., blue) to indicate a cooler color tone adjustment direction <NUM>. In aspects, the color of the colored slider <NUM>, between the top end <NUM> and the bottom end <NUM>, may gradually change from substantially blue to substantially red (or substantially orange) to visually indicate a correlation between the color tone adjustment and the position of the indicator <NUM> on the colored slider <NUM>. In another example, the color of the colored slider <NUM> may change in a stepwise fashion. In this way, the user can intuitively understand the functionality of the WB slider (e.g., the correlation between the location of the indicator and the resulting color-tone effects on the preview image) without having prior knowledge of CCT or how to use the CCT for WB adjustment. Rather, the warmer and cooler color directions for MWB adjustment provide the user with an immediate impression of how to change the color appearance.

In aspects, the CCT also changes based on the gain adjustment change. For example, sliding the indicator <NUM> toward the warmer direction moves the white balance to the higher CCT direction, and sliding the indicator <NUM> toward the cooler direction moves the white balance to the lower CCT direction.

Generally, any of the components, modules, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Some operations of the example methods may be described in the general context of executable instructions stored on computer-readable storage memory that is local and/or remote to a computer processing system, and implementations can include software applications, programs, functions, and the like. Alternatively or in addition, any of the functionality described herein can be performed, at least in part, by one or more hardware logic components, including, and without limitation, Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SoCs), Complex Programmable Logic Devices (CPLDs), and the like.

Claim 1:
A method for a hybrid white-balance mode in a camera system (<NUM>) of an electronic device (<NUM>), the method comprising:
presenting, during a live-preview mode (<NUM>) of a camera application (<NUM>) of the camera system, a manual-white-balance control via a camera user interface (<NUM>) displayed on a display device (<NUM>) of the electronic device, the manual-white-balance control configured to enable user selection of a white-balance setting for performing a manual adjustment on a white balance with respect to a current frame presented in the live-preview mode of the camera application, wherein the manual-white-balance control includes a slider (<NUM>), the slider configured to provide a range between a plurality of preset white-balance gains;
determining, using an auto-white-balance mode, an auto-white-balance decision on a red-green-blue, RGB, scale for the current frame based on one or more characteristics of the current frame;
receiving a user input that adjusts the manual-white-balance control displayed in the camera user interface;
determining, using a manual-white-balance mode, target white-balance gains for a white-balance adjustment of the current frame based on the user input to the manual-white-balance control and relative to the auto-white-balance decision, the auto-white-balance decision used as initial white-balance gains to determine the target white-balance gains that correspond to the user input to the manual-white-balance control; and
applying the target white-balance gains to the current frame presented in the live-preview mode of the camera application for color correction,
wherein the method further comprises:
determining location information associated with a user-selected location on the slider; and
applying an interpolation function to determine the target white-balance gains corresponding to the location information of the user-selected location relative to two preset white-balance points on opposing sides of the user-selected location, wherein the interpolation function uses an interpolation ratio that is based on a correlated color temperature of each of the two preset white-balance points (<NUM>).