Patent ID: 12262156

DESCRIPTION OF EMBODIMENTS

In the embodiments of this application, terms “first”, “second”, “third” and “fourth” are used merely for the purpose of description, and shall not be construed as indicating or implying relative importance or implying a quantity of indicated technical features.

For ease of understanding of the embodiments of this application, related concepts involved in the embodiments of this application are first briefly described below.

1. White Balance

White balance is an indicator describing the accuracy of a white color generated by mixing three primary colors, red, green, and blue, in a display. A white balance setting in a camera device is an important guarantee to ensure an ideal picture color; and white balance processing refers to an operation of reversely eliminating an influence of a light source color in an image by accurately estimating the light source color, so as to achieve an effect of shooting with white light.

2. Bayer Pattern

A Bayer pattern is one of main technologies of implementing photographing a colorful image by a charge-coupled device (charge-coupled device, CCD) or a complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS) sensor; and the Bayer pattern may be a 4×4 pattern, for example, composed of 8 green, 4 blue, and 4 red pixels, and performing 9 operations in a 2×2 matrix during transforming a grayscale graphic into a colorful picture, to finally generate a colorful image.

3. Color Filter Array (Color Filter Array, CFA) Sensor

A color filter array sensor refers to a sensor covering a mosaic color filter array on a pixel sensor, and is configured to collect color information of an image; a general photoelectric sensor can only feel the strength of light and cannot distinguish a wavelength (color) of the light; and a filter array sensor may obtain the color information of a pixel point through color filter (color filter).

4. Color Mode

A color mode is a model of representing a certain color as a number or a manner of recording the color of an image.

5. Demosaic (Demosaic)

Demosaicing refers to an image processing process of transforming a Raw image into an RGB image.

6. Neutral Color

A neutral color may refer to black, white, and various shades of gray that are blended with black and white.

7. Neural Network

A neural network is a network formed by connecting many single neurons together. That is, an output of a neuron may be an input of another neuron. An input of each neuron may be connected with a local receptive field of a previous layer to extract a feature of the local receptive field. The local receptive field may be a region including several neurons.

8. Fully Connected Neural Network

A fully connected neural network, also known as a deep neural network (deep neural network, DNN) or a multi-layer neural network, may be understood as a neural network with a plurality of hidden layers. According to locations of different layers, neural networks inside the fully connected neural network may be classified into three types: input layers, hidden layers, and output layers. Generally, the first layer is an input layer, the last layer is an output layer, and middle layers are all hidden layers. The layers are fully connected. That is, any neuron in an ithlayer needs to be connected to any neuron in an (i+1)thlayer.

A working principle of the fully connected neural network may be expressed by the following linear relationship expression: y=a(w·x+b); where x represents an input vector, y represents an output vector, b represents an offset vector, W represents a weight matrix (also referred to as a coefficient), and a( ) represents an activation function. Each layer performs an operation through the linear expression on the input vector x to obtain the output vector y.

Because the DNN have many layers, the quantity of the coefficient w and the offset vector b are relatively large. These parameters in the DNN are defined as below:

As shown inFIG.1, assuming that in a three-layer DNN, the three-layer DNN includes the input layer (the first layer), the hidden layer (the second layer), and the output layer (the third layer); and for example, the linear coefficient of a 4thneuron in the second layer to a 1stneuron in the third layer may be defined as W78, and the subscript 78 correspondingly indicates an inputted second layer index 7 and an outputted third layer index 8.

9. Back Propagation Algorithm

A neural network may modify the size of a parameter in an initial neural network model during the training by using an error back propagation (back propagation. BP) algorithm, so that a rebuilding error loss of the neural network model becomes smaller. Specifically, an input signal is propagated forward until an output generates an error loss, and the parameter in the initial neural network model is updated through back propagation of error loss information, so that the error loss is converged. The back propagation algorithm is a back propagation motion dominated by an error loss, and is intended to obtain a parameter of an optimal neural network model, for example, a weight matrix.

The following describes a white balance processing method and an electronic device in the embodiments of this application with reference to accompanying drawings.

FIG.2shows a hardware system applicable to the electronic device according to this application.

An electronic device100may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (augmented reality, AR) device, a virtual reality (virtual reality, VR) device, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), a projector, or the like. A specific type of the electronic device100is not limited in this embodiment of this application.

The electronic device100may include a processor110, an external memory interface120, an internal memory121, a universal serial bus (universal serial bus, USB) interface130, a charging management module140, a power management module141, a battery142, an antenna 1, an antenna 2, a mobile communication module150, a wireless communication module160, an audio module170, a speaker170A, a telephone receiver170B, a microphone170C, a headset jack170D, a sensor module180, a button190, a motor191, an indicator192, a camera193, a display screen194, a subscriber identification module (subscriber identification module, SIM) card interface195, and the like. The sensor module180may include a pressure sensor180A, a gyroscope sensor180B, a barometric pressure sensor180C, a magnetic sensor180D, an acceleration sensor180E, a distance sensor180F, an optical proximity sensor180G, a fingerprint sensor180H, a temperature sensor180J, a touch sensor180K, an ambient light sensor180L, a bone conduction sensor180M, and the like.

It should be noted that, the structure shown inFIG.2does not constitute a specific limitation on the electronic device100. In some other embodiments of this application, the electronic device100may include more or fewer components than the components shown inFIG.2, or, the electronic device100may include a combination of some components in the components shown inFIG.2, or, the electronic device100may include subcomponents of some components in the components shown inFIG.2. The components shown inFIG.2may be implemented by hardware, software, or a combination of software and hardware.

The processor110may include one or more processing units. For example, the processor110may include at least one of the following processor units: an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor. DSP), a baseband processor, and a neural-network processing unit (neural-network processing unit, NPU). Different processing units may be independent components, or may be integrated components. The controller may generate an operating control signal according to an instruction operation code and a sequence signal, to complete control of fetching and executing instructions.

A memory may be further disposed in the processor110, and is configured to store instructions and data. In some embodiments, the memory in the processor110is a cache memory. The memory may store instructions or data that has just been used or recycled by the processor110. If the processor110needs to reuse the instructions or the data, the processor110may directly invoke the instructions or the data from the memory, which avoids repeated access, and reduces a waiting time of the processor110, thereby improving system efficiency.

Exemplarily, the processor110may be configured to perform the white balance processing method according to the embodiments of this application. For example: displaying a first interface, where the first interface includes a first control; detecting a first operation on the first control; obtaining a first image in response to the first operation, where the first image refers to an image of a first color space captured by a multispectral color filter array sensor; performing decomposition processing and demosaicing processing on the first image to obtain a second image and a third image, where the second image is an image in a first color mode and the third image is an image in a second color mode; obtaining a color correction matrix according to the second image and the third image, where the color correction matrix is used for representing a pixel change amount of transforming the second image into the third image; inputting the color correction matrix into a white balance model to obtain a white balance parameter; and performing image processing on the first image according to the white balance parameter to obtain a fourth image.

Exemplarily, the processor110may be configured to perform the training methods of the white balance model according to the embodiments of this application. For example: obtaining training data, where the training data includes a first sample image and a second sample image, the first sample image and the second sample image refer to images of a first color space captured by a multispectral color filter array sensor under a same light source scenario, and the second sample image includes a color checker; performing decomposition processing and demosaicing processing on the first sample image to obtain a third sample image and a fourth sample image, where the third sample image is an image in a first color mode and the fourth sample image is an image in a second color mode; obtaining a sample color correction matrix according to the third sample image and the fourth sample image, where the sample color correction matrix is used for representing a pixel change amount of transforming the third sample image into the fourth sample image; performing the decomposition processing and the demosaicing processing on the second sample image to obtain a fifth sample image, where the fifth sample image is an image in the first color mode; and training a white balance model by using the sample color correction matrix as input data and using first pixel information in the fifth sample image as target data to obtain a trained white balance model, where the white balance model is used for calculating a parameter of white balance processing, and the first pixel information refers to a pixel value corresponding to a neutral color block included m the color checker in the fifth sample image.

A connection relationship among the modules shown inFIG.2is merely an example for description, and constitutes no limitation on the connection relationship among the modules of the electronic device100. Optionally, the modules of the electronic device100may alternatively use a combination of a plurality of connection manners in the foregoing embodiment.

A wireless communication function of the electronic device100may be implemented by using the antenna 1, the antenna 2, the mobile communications module150, the wireless communications module160, the modem processor, the baseband processor, and the like.

The antenna 1 and the antenna 2 are configured to transmit and receive an electromagnetic wave signal. Each antenna in the electronic device100may be configured to cover one or more communication frequency bands. Different antennas may be multiplexed to improve antenna utilization. For example, the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas may be used with a tuning switch.

The electronic device100implements a display function by using the GPU, the display screen194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen194and the application processor. The GPU is configured to perform mathematical and geometric calculation, and is configured to render a graphic. The processor110may include one or more GPUs, and execute program instructions to generate or change display information.

The display screen194may be configured to display an image or a video.

The electronic device100may implement a photographing function by using the ISP, the camera193, the video codec, the GPU, the display screen194, the application processor, and the like.

The ISP is configured to process data fed back by the camera193. For example, during photographing, a shutter is enabled. Light is transferred to a photosensitive element of the camera through a lens, and an optical signal is converted into an electrical signal. The photosensitive element of the camera transfers the electrical signal to the ISP for processing, and the electrical signal is converted into an image visible to a naked eye. The ISP may optimize an algorithm for noise, brightness, and colors of the image, and may further optimize parameters, such as exposure and color temperature, of a photographing scene. In some embodiments, the ISP may be disposed in the camera193.

The camera193is configured to capture a static image or a video. An optical image of an object is generated through a lens and is projected onto a photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device. CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts an optical signal into an electrical signal, and then transfers the electrical signal to the ISP, to convert the electrical signal into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard image signal in a standard format such as RGB (red green blue, RGB) or YUV. In some embodiments, the electronic device100may include 1 or N cameras193, and N is a positive integer greater than 1.

The digital signal processor is configured to process a digital signal, and may further process another digital signal in addition to a digital image signal. For example, when the electronic device100performs frequency selection, the digital signal processor is configured to perform Fourier transform and the like on frequency energy.

The video codec is configured to compress or decompress a digital video. The electronic device100may support one or more video codecs. In this way, the electronic device100may play or record videos in a plurality of encoding formats, for example, moving picture experts group (moving picture experts group, MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

The gyroscope sensor180B may be configured to determine a motion posture of the electronic device100. In some embodiments, angular velocities of the electronic device100around three axes (i.e., x, y, and z axes) may be determined through the gyroscope sensor180B. The gyroscope sensor180B may be configured to prevent jitter during photographing. Exemplarily, when the shutter is pressed, the gyroscope sensor180B detects an angle at which the electronic device100jitters, and calculates, based on the angle, a distance for which a lens module needs to compensate, and allows the lens to cancel the jitter of the electronic device100through reverse motion, thereby implementing image stabilization. The gyroscope sensor180B may also be configured for scenarios such as a navigation scenario and a motion sensing game scenario.

Exemplarily, the gyroscope sensor180B in this embodiment of this application may be configured to collect jitter information, and the jitter information may be used for representing a pose change of the electronic device during photographing.

The acceleration sensor180E may detect an acceleration value of the electronic device100in all directions (generally x, y, and z axes). When the electronic device100is stationary, a magnitude and a direction of gravity may be detected. The acceleration sensor180E may be further configured to recognize a posture of the electronic device100as an input parameter of an application program such as switching between landscape orientation and portrait orientation or a pedometer.

The distance sensor180F is configured to measure a distance. The electronic device100may measure a distance through infrared light or laser. In some embodiments, for example, in a photographing scenario, the electronic device100may measure a distance by using the distance sensor180F, to implement quick focusing.

The ambient light sensor180L is configured to perceive ambient light brightness. The electronic device100may adaptively adjust brightness of the display screen194according to the perceived ambient light brightness. The ambient light sensor180L may be further configured to automatically adjust white balance during photographing. The ambient light sensor180L may further cooperate with the optical proximity sensor180G to detect whether the electronic device100is in a pocket, so as to prevent an accidental touch.

The fingerprint sensor180H is configured to collect a fingerprint. The electronic device100may implement a function such as unlock, application lock accessing, photographing, call answering, or the like by using a feature of the collected fingerprint.

The touch sensor180K is also referred to as a touch control device. The touch sensor180K may be disposed in the display screen194, and the touch sensor180K and the display screen194constitute a touchscreen, and the touchscreen is also referred to as a touch control screen. The touch sensor180K is configured to detect a touch operation on or near the touch sensor180K. The touch sensor180K may transfer the detected touch operation to the application processor to determine a touch event type. The touch sensor180K may provide a visual output related to the touch operation by using the display screen194. In some other embodiments, the touch sensor180K may alternatively be disposed on a surface of the electronic device100, and is located on a position different from that of the display screen194.

When the electronic device captures an image, the color of an object in the image will be affected by the color of a light source. For an image sensor, it is impossible to accurately determine the color of the object under the light source in any colors. Therefore, the image color needs to be adjusted through a series of operations such as white balance processing, a color transformation matrix and a three-dimensional lookup table. White balance processing refers to an operation of reversely eliminating an influence of a light source color in an image by accurately estimating the light source color, so as to achieve an effect of shooting with white light. Currently, existing image white balance processing relies on the single-frame image, for example, image information with 3 channels. For some photographing scenarios, for example, a scenario of photographing objects in pure colors in a short distance, the accuracy of the existing white balance processing still needs to be improved.

In view of this, the embodiments of this application provide a white balance processing method and an electronic device. In the embodiments of this application, the first image is captured through the multispectral color filter array sensor, and the second image and the third image in different color modes may be obtained through performing the decomposition processing and the demosaicing processing on the first image. The color correction matrix may be obtained according to the second image and the third image, and the white balance parameter may be obtained by inputting the color correction matrix into the white balance model. The fourth image is obtained by performing the image processing on the first image through the white balance parameter. Because the color information in the image (for example, a Raw image with 6 channels) captured by the multispectral color filter array sensor is more than the color information of a single-frame image (for example, an image with 3 channels), the white balance parameter obtained through the first image can improve the accuracy of the white balance processing and the color accuracy of the image.

The following exemplarily describes an application scenario of the white balance processing method according to this embodiment of this application with reference toFIG.3andFIG.4.

Exemplarily, the white balance processing method in this embodiment of this application may be applicable to the field of photography, video recording, a video call, or other image processing. The color accuracy of an image can be improved through performing white balance processing on the image through the white balance processing method according to the embodiment of this application.

Application Scenario One: The Field of Photography

As shown inFIG.3, when photographing under a dark light scenario (for example, a night scene embodiment), the light entering amount of the electronic device is small, leading to a relatively low signal-to-noise ratio of the image. Existing white balance processing, that is, a white balance algorithm based on a single-frame image, is prone to error, resulting in obvious color distortion existing in an acquired image. (a) inFIG.3shows a preview image of a photographed object210obtained by using the existing white balance processing method. (b) inFIG.3shows a preview image of the photographed object210obtained through the white balance processing method according to this embodiment of this application. Compared with the preview image shown in (a) inFIG.3, the preview image shown in (b) inFIG.3has a higher degree of color restoration of the photographed object210. Therefore, the color accuracy of an image can be improved through performing white balance processing on the image through the white balance processing method according to the embodiment of this application.

Application Scenario Two: Video Call

As shown inFIG.4, (a) inFIG.4shows a preview image of a photographed object220in a video call by using the existing white balance processing method. (b) inFIG.4obtains a preview image of the photographed object220in the video call through the white balance processing method according the embodiment of this application. Compared with the preview image shown in (a) inFIG.4, the preview image shown in (b) inFIG.4has a higher degree of color restoration of the preview image of the photographed object220. Therefore, the color accuracy of an image can be improved through performing white balance processing on the image through the white balance processing method according to the embodiment of this application.

It is to be understood that, the foregoing describes by using examples of the application scenario, and does not constitute any limitation on the application scenario of this application.

The following describes the white balance processing method according to the embodiments of this application in detail with reference toFIG.5andFIG.11.

FIG.5is a schematic diagram of a system architecture applicable to the white balance processing method according to this application.

As shown inFIG.5, the system architecture300includes a multispectral color filter array sensor310, a decomposition module320, a white balance parameter calculation module330, and an image signal processor340. The image signal processor340may further include a white balance module341.

Exemplarily, the multispectral color filter array sensor310may be configured to obtain a Raw image. For example, the Raw image captured by the multispectral color filter array sensor310may include an RGB color mode and another color mode.

Exemplarily, the Raw image captured by the multispectral color filter array sensor310may refer to an RGBCYM image, or, an RGBCYGM image, or an image in the another color mode.

In an example, the Raw image obtained by the multispectral color filter array sensor310may be an RGBCMY image, where R represents red (red), G represents green (green), B represents blue (blue), C represents cyan (cyan), M represents mayenta (mayenta), and Y represents yellow (yellow).

In an example, the Raw image obtained by the multispectral color filter array sensor310may be the RGBCYGM image, where R represents red (red), G represents green (green), B represents blue (blue), C represents cyan (cyan), Y represents yellow (yellow), and M represents mayenta (mayenta).

It is to be understood that, the foregoing describes by using the RGBCMY image and the RGBCYGM image as examples, which is not limited in this application.

Exemplarily, the Raw image obtained by the multispectral color filter array sensor310may be an image with multiple channels, for example, a Raw image with 6 channels, a Raw image with 8 channels, or a Raw image with another quantity of channels.

It is to be understood that, the Raw image captured by the multispectral color filter array sensor may be decomposed into an image in an RGB color mode and an image in another color mode, and the spectral response curves of the two images satisfy that any two curves are different.

For example,FIG.6shows spectral response curves of an RGBCMY image with 6 channels, where curve 1 represents a spectral response curve corresponding to blue (blue, B); curve 2 represents a spectral response curve corresponding to cyan (cyan, C); curve 3 represents a spectral response curve corresponding to mayenta (mayenta, M); curve 4 represents a spectral response curve corresponding to yellow (yellow, Y); curve 5 represents a spectral response curve corresponding to green (green, G), and curve 6 represents a spectral response curve corresponding to red (red, R). In the spectral response curves of the RGBCMY image shown inFIG.6, curve 6 corresponds to curve 3, curve 5 corresponds to curve 4, and curve 1 corresponds to curve 2 respectively. As can be seen fromFIG.6, a spectral range of curve 3 is wider than that of curve 6, and a light input amount of curve 3 is better than that of curve 6; a spectral range of curve 4 is wider than that of curve 5, and a light input amount of curve 4 is better than that of curve 5; and a spectral range of curve 2 is wider than that of curve 1, and a light input amount of curve 2 is better than that of curve 1.

Exemplarily, the decomposition module320is configured to decompose the Raw image captured by the multispectral color filter array sensor310into a first Bayer pattern image and a second Bayer pattern image, where the first Bayer pattern image may refer to a Bayer pattern image in a first color mode (for example, an RGB color mode), and the second Bayer pattern image may refer to a Bayer pattern image in a second color mode (for example, a CYM color mode, a CYGM color mode, or another color mode).

It should be understood that, any two curves in the spectral response curves of the first Bayer pattern image and the spectral response curves of the second Bayer pattern image are different.

Exemplarily, the decomposition module320may first perform demosaicing (demosaic) processing, and then perform resizing (resize) processing on the Raw image captured by the multispectral color filter array sensor. For example, the sizes of the first Bayer pattern image and the second Bayer pattern image may be 68*48*3.

Optionally, the first Bayer pattern image may refer to an image in the RGB color mode; and the second Bayer pattern image may refer to an image in the another color mode. For example, the second Bayer pattern image may refer to an image in the CMY color mode; or the second Bayer pattern image may refer to an image in the CYGM color mode.

Exemplarily, the Raw image captured by the multispectral color filter array sensor310may be an image with multiple channels; and the image with multiple channels may be classified into a first Bayer pattern image (for example, the RGB color mode) with 3 channels and a second Bayer pattern image with another quantity of channels.

Optionally, the Raw image captured by the multispectral color filter array sensor310may be an RGBCMY image with 6 channels, which may be decomposed into the first Bayer pattern image (an RGB image) with 3 channels and a second Bayer pattern image (a CMY image) with 3 channels.

Optionally, the Raw image captured by the multispectral color filter array sensor310may be an RGBCYGM image with 7 channels, which may be decomposed into the first Bayer pattern image (the RGB image) with 3 channels and a second Bayer pattern image (a CYGM image) with 4 channels.

Optionally, the Raw image captured by the multispectral color filter array sensor310may be a Raw image with 8 channels, which may be decomposed into the first Bayer pattern image (the RGB image) with 3 channels and a second Bayer pattern image with 5 channels.

It is to be understood that, the foregoing describes the Raw image captured by the multispectral color filter array sensor310by using examples, which is not limited in this application.

Exemplarily, the white balance parameter calculation module330is configured to calculate a white balance parameter according to the first Bayer pattern image and the second Bayer pattern image, and a specific process may refer to steps S430to S450shown in subsequentFIG.7.

Optionally, the white balance parameter calculation module330may further transmit the obtained white balance parameter to the image signal processor340.

Exemplarily, after the image signal processor340obtains the white balance parameter, the white balance module341may perform white balance processing on the image captured by the multispectral color filter array sensor310according to the white balance parameter to obtain a processed image.

For example, the white balance module341may perform automatic white balance processing (automatic white balance, AWB) on the image captured by the multispectral color filter array sensor310according to the white balance parameter.

Optionally, the white balance parameter calculation module330may be a module in a CPU, a GPU, or another hardware with computility; or, the white balance parameter calculation module330may be a module in the image signal processor340; or, a function of the white balance parameter calculation module330may alternatively be performed in the white balance module341.

It should be noted that, the system architecture shown inFIG.5may be the system architecture in the electronic device shown inFIG.2; and the system architecture shown inFIG.5may perform a white balance processing method shown inFIG.7, and the white balance processing method shown inFIG.7are described in detail below.

FIG.7is a schematic diagram of a white balance processing method according to an embodiment of this application. The method400includes steps S410to S470, and steps S410to S470are described in detail below.

Step S410: Obtain a Raw image (an example of a first image) captured by a multispectral color filter array sensor. The Raw image refers to the Raw image captured by the multispectral color filter array sensor310shown inFIG.5.

For example, a Raw image with 6 channels may be captured by the multispectral color filter array sensor310, and the Raw image may include an RGB color mode and another color mode.

It is to be understood that, the color filter array sensor refers to a sensor covering a mosaic color filter array on a pixel sensor, and is used for collecting color information of the image. A Raw image with more channels may be obtained through the color filter array sensor. A spectral range of the spectral response curves of the Raw image captured by the color filter array sensor is wider, that is, the light input amount of the color filter array sensor is larger, which makes a better brightness value of the image.

Step S420: Perform decomposition processing on the Raw image.

Optionally, the decomposition processing is performed on the Raw image captured by the multispectral color filter array sensor.

Exemplarily, two Bayer pattern images, a first Bayer pattern image and a second Bayer pattern image respectively, are extracted from a preset position of the Raw image.

Step S430: Obtain the first Bayer pattern image.

Optionally, the first Bayer pattern image is obtained according to the decomposition processing.

Step S440: Obtain the second Bayer pattern image.

Optionally, the second Bayer pattern image is obtained according to the decomposition processing.

Exemplarily, corresponding pixels (pixel) may be extracted according to an RGB mask (mask) of the Raw image to compose an RGGB Bayer pattern image to obtain a first sample Bayer pattern image; and corresponding pixels (pixel) may be extracted according to a CYM mask (mask) of the Raw image to compose a CYYM Bayer pattern image to obtain a second sample Bayer pattern image.

Exemplarily, the corresponding pixels (pixel) may be extracted according to the RGB mask (mask) of the Raw image to compose the RGGB Bayer pattern image to obtain the first sample Bayer pattern image; and corresponding pixels (pixel) may be extracted according to a CYGM mask (mask) of the Raw image to compose a CYGM Bayer pattern image to obtain a second sample Bayer pattern image.

It is to be understood that, the first Bayer pattern image refers to the RGGB image, and the second Bayer pattern image may refer to the CYYM image, the CYGM image, or an image in another color mode. The second Bayer pattern image and the first Bayer pattern image satisfy that any two curves in the spectral response curves are different, and the color mode of the second Bayer pattern image is not limited in this application.

Step S450: Calculate a color correction matrix.

Optionally, the color correction matrix is calculated according to the first Bayer pattern image and the second Bayer pattern image.

Exemplarily, demosaicing processing may be performed on the first Bayer pattern image and the second Bayer pattern image to obtain an image (an example of a second image) in a first color mode and an image (an example of a third image) in a second color mode. The color correction matrix may be obtained according to a pixel difference between the image in the first color mode and the image in the second color mode, where the color correction matrix is used for representing a pixel change amount of transforming the image in the first color mode into the image in the second color mode.

For example, the first Bayer pattern image is the RGGB image, and the second Bayer pattern image is the CMMY image; the image in the first color mode may be an RGB image, and the image in the second color mode may be a CMY image; each pixel in the RGB image may correspond to one group of RGB value (an example of a first pixel, a second pixel, and a third pixel), and each pixel in the CMY image may correspond to one group of CMY value (an example of a fourth pixel, a fifth pixel, and a sixth pixel); a 3×1 matrix (an example of a first vector) may be obtained by transforming the RGB value corresponding to an R pixel into the CMY value corresponding to a C pixel; similarly, a 3×1 matrix (an example of a second vector) may be obtained by transforming the RGB value corresponding to a G pixel into the CMY value corresponding to a M pixel; a 3×1 matrix (an example of a third vector) may be obtained by transforming the RGB value corresponding to a B pixel into the CMY value corresponding to a Y pixel; and a 3×3 color correction matrix is obtained through the 3 one dimensional matrixes.

Step S460. Input the color correction matrix into a white balance model to obtain a white balance parameter.

The white balance model may obtain the corresponding white balance parameter according to the inputted color correction matrix. The white balance model may be a pretrained fully connected neural network, and a training method of the white balance model may refer to the subsequentFIG.8.

It is to be understood that, the white balance parameter may refer to color information of a light source. For example, the white balance parameter may refer to an RGB value of the light source. The influence of the light source on the color of a photographed object may be eliminated by applying the color information of the light source to the image, thereby avoiding color distortion of the photographed object. The color information of the photographed object in the image may be modified according to the color information of the light source, thereby improving color accuracy of the photographed object in the image.

Step S470: Perform white balance processing on the Raw image according to the white balance parameter.

Optionally, the white balance processing (an example of image processing) is performed on the Raw image according to the white balance parameter to obtain a processed image (an example of a fourth image).

In this embodiment of this application, the first image is captured through the multispectral color filter array sensor, and the second image and the third image in different color modes may be obtained through performing the decomposition processing and the demosaicing processing on the first image. The color correction matrix may be obtained according to the second image and the third image, and the white balance parameter may be obtained by inputting the color correction matrix into the white balance model. The fourth image is obtained by performing the image processing on the first image through the white balance parameter. Because the color information in the image (for example, a Raw image with 6 channels) captured by the multispectral color filter array sensor is more than the color information of a single-frame image (for example, an image with 3 channels), the white balance parameter obtained through the first image can improve the accuracy of the white balance processing and the color accuracy of the image.

FIG.8is a schematic diagram of a training method of a white balance model according to an embodiment of this application. The training method may be performed by the electronic device shown inFIG.2. The training method500may include steps S510to S560, and steps S510to S560are described in detail respectively below.

Step S510: Obtain training data of different light source scenarios.

Optionally, for the same photographing scenario, the training data may include a first sample image without a color checker and a second sample image with a color checker in the same light source scenario, and the second sample image is used for marking ground truths (Ground truth) of different light sources.

It should be noted that, the color checker may be disposed under the illumination of a main light source for photographing during obtaining the sample image.

Optionally, for scenarios of a color flash and a color cast, automatic white balance of the current scenarios is relatively poor; and the capturing of the sample image may be performed focusing on the scenarios.

Step S520: Decompose the first sample image to obtain a first sample Bayer pattern image and a second sample Bayer pattern image.

Exemplarily, corresponding pixels (pixel) may be extracted according to an RGB mask (mask) of the first sample image to compose an RGGB Bayer pattern image to obtain the first sample Bayer pattern image; and corresponding pixels (pixel) may be extracted according to a CYM mask (mask) of the first sample image to compose a CYYM Bayer pattern image to obtain the second sample Bayer pattern image.

Exemplarily, the corresponding pixels (pixel) may be extracted according to the RGB mask (mask) of the first sample image to compose the RGGB Bayer pattern image to obtain the first sample Bayer pattern image; and corresponding pixels (pixel) may be extracted according to a CYGM mask (mask) of the first sample image to compose a CYGM Bayer pattern image to obtain the second sample Bayer pattern image.

It is to be understood that, the first sample Bayer pattern image may refer to the RGGB image, and the second sample Bayer pattern image may refer to the CYYM image, the CYGM image, or an image in another color mode. The second sample Bayer pattern image and the first sample Bayer pattern image satisfy that any two curves in the spectral response curves are different, and the color mode of the second sample Bayer pattern image is not limited in this application.

Step S530: Perform demosaicing processing on the first sample Bayer pattern image and the second sample Bayer pattern image to obtain a third sample image and a fourth sample image.

The third sample image may refer to an image in a first color mode, and the fourth sample image may refer to an image in a second color mode.

For example, the first color mode may refer to an RGB color mode, and the second color mode may refer to a CMY color mode; or, the second color mode may further refer to an image in another color mode.

Exemplarily, the first sample Bayer pattern image may be the RGGB image, and the demosaicing processing is performed on the RGGB image to obtain an RGB image; and the second sample Bayer pattern image may be the CYYM image, and the demosaicing processing is performed on the CYYM image to obtain a CYM image.

Exemplarily, the first sample Bayer pattern image may be the RGGB image, and the demosaicing processing is performed on the RGGB image to obtain the RGB image; and the second sample Bayer pattern image may be the CYGM image, and the demosaicing processing is performed on the CYGM image to obtain a CYGM image.

Step S540: Obtain a sample color correction matrix according to the third sample image and the fourth sample image.

For example, the sample color correction matrix may be obtained through transforming a pixel in the third sample image into a pixel in the fourth sample image.

Exemplarily, the third sample image may be the RGB image, and the fourth sample image may be the CMY image; each pixel in the RGB image may correspond to one group of RGB value, and each pixel in the CMY image may correspond to one group of CMY value; a 3×1 matrix may be obtained by transforming the RGB value corresponding to an R pixel into the CMY value corresponding to a C pixel; similarly, a 3×1 matrix may be obtained by transforming the RGB value corresponding to a G pixel into the CMY value corresponding to a M pixel; a 3×1 matrix may be obtained by transforming the RGB value corresponding to a B pixel into the CMY value corresponding to a Y pixel; and a 3×3 sample color correction matrix is obtained through the 3 one dimensional matrixes.

Optionally, to facilitate the training of a white balance model, the 3×3 sample color correction matrix may be reorganized to obtain a one-dimensional vector, that is, a 9×1 sample color correction matrix.

Step S550: Mark a color value corresponding to a neutral color block included in the color checker in the second sample image as the color value of the light source.

It is to be understood that, a neutral material of a neutral color block reflects light with various wavelengths uniformly. Therefore, any color of light that shines on the neutral color block reflects the light of that color. For example, red light that shines on the neutral color block of the color checker reflects the red light, or green light that shines on the neutral color block of the color checker reflects the green light. Therefore, a neutral color checker of the color checker may be used for marking the color of the light source.

Exemplarily, the neutral color block may refer to a gray color block in the color checker. For example, an RGB pixel value of the gray color block in a target image is marked as a light source color of the light source, that is, the RGB pixel value of the light source.

Exemplarily, a marking process of the color value of the light source includes the following steps:

Step one: Decompose the second sample image to obtain a third sample Bayer pattern image (an RGGB Bayer pattern image) and a fourth sample Bayer pattern image.

Step two: Perform the demosaicing processing on the third sample Bayer pattern image and the fourth sample Bayer pattern image to obtain an RGB image (an example of a fifth sample image) and an image in another color mode.

Step three: Mark the RGB pixel value corresponding to the neutral color block of the color checker included in the RGGB image as the color value (an example of first pixel information) of the light source.

Exemplarily, the neutral color block may refer to a gray color block in the color checker. For example, an RGB pixel value of the gray color block in a target image is marked as a light source color of the light source, that is, the RGB pixel value of the light source.

Optionally, in a possible implementation, demosaic algorithm processing may be performed on the second sample image to obtain a processed image; and a pixel value of RGB in the pixel value corresponding to the neutral color block of the color checker is obtained in the processed image.

Step S560: Train the white balance model by using the sample color correction matrix as input data and using the color value of the light source as a target value to obtain a trained white balance model.

Exemplarily, the white balance model is inputted into the sample color correction matrix to obtain a predicted white balance parameter (an example of predicted pixel information); the color value of the light source is compared according to the predicted white balance parameter; and iteration is performed on the parameter of the white balance model through a back propagation algorithm until the white balance model is converged to obtain the trained white balance model.

For example, the white balance model may be trained through back iteration using an angular error loss (Angular Error Loss) between the predicted white balance parameter and the color value of the light source.

Optionally, the white balance model in this embodiment of this application may be a fully connected neural network.

In this embodiment of this application, the first sample image without the color checker and the second sample image with the color checker are captured through the multispectral color filter array sensor under a same light source and a same scenario. The input data for training the white balance model may be obtained through processing the first sample image. The target data for training the white balance model may be obtained through the second sample image. The trained white balance model obtained through the training method of the white balance model according to the embodiment of this application may be used for white balance processing, and the white balance processing may be performed on the Raw image captured by the multispectral color filter array sensor according to the white balance parameter outputted by the white balance model. In this embodiment of this application, because the training data of training the white balance model is the image captured by the multispectral color filter array sensor, and the color information in the image (for example, a Raw image with 6 channels) captured by the multispectral color filter array sensor is more than the color information of a single-frame image (for example, an image with 3 channels), the parameter used for white balance processing outputted by the trained white balance model can improve the accuracy of the white balance processing and the color accuracy of the image.

FIG.9is a schematic diagram of an effect of a white balance processing method according to an embodiment of this application.

As shown inFIG.9, (a) inFIG.9is an output image obtained through a white balance processing method in the related art. (b) inFIG.9is an output image obtained through the white balance processing method according to this embodiment of this application. Severe distortion appearing in colors of a color checker601may be seen from a preview image shown in (a) inFIG.9. Compared with the preview image shown in (a) inFIG.9, a preview image shown in (b) inFIG.9has a higher degree of color restoration of the color checker601. That is, the color accuracy of an image can be improved through performing white balance processing on the image through the white balance processing method according to this embodiment of this application.

In an embodiment, a color restoration mode may be turned on in a camera application program of an electronic device, then the white balance parameter may be obtained in the electronic device through the white balance model according to this embodiment of this application, and the white balance processing may be performed on the Raw image captured by the multispectral color filter array sensor according to the white balance parameter, thereby outputting a processed image or video.

FIG.10AandFIG.10Bshow a graphical user interface (graphical user interface, GUI) of an electronic device.

The GUI shown in (a) inFIG.10Ais a desktop610of the electronic device. When detecting an operation that a user taps an icon620of a camera application (application, APP) on the desktop610, the electronic device may start the camera application, and display another GUI shown in (b) inFIG.10A. The GUI shown in (b) inFIG.10Amay be a display interface under a photographing mode of the camera APP, and the GUI may include a photographing interface630. The photographing interface630may include a viewfinder frame631and a control. For example, the photographing interface630may include a control632for instructing photographing and a control633for instructing setting. In a preview state, the viewfinder frame631may display a preview image in real time. The preview state may refer to a state when a user turns on the camera and does not press a photo/video button. In this case, the viewfinder frame may display the preview image in real time.

After the electronic device detects an operation of clicking and setting the control633by the user, a setting interface shown in (c) inFIG.10Bis displayed. The setting interface may include a color restoration mode control634. After detecting an operation of clicking the color restoration mode control634by the user, a color restoration mode may be turned on in the electronic device. After the color restoration mode may be turned on in the electronic device, a white balance parameter may be obtained through the white balance model according to this embodiment of this application, and white balance processing may be performed on a Raw image captured by a multispectral color filter array sensor according to the white balance parameter, thereby outputting a processed image.

In an embodiment, under a photographing mode shown inFIG.11, the photographing interface630may further include a control635, and the control635is configured to instruct to turn on/off the color restoration mode. After the electronic device detects an operation of clicking the control635by the user, the color restoration mode may be turned on in the electronic device, the white balance parameter may be obtained through the white balance model according to this embodiment of this application, and the white balance processing may be performed on the Raw image captured by the multispectral color filter array sensor according to the white balance parameter, thereby outputting a processed image or video.

It is to be understood that, the foregoing exemplary description with examples is merely intended to help a person skilled in the art to understand the embodiments of this application, and is not intended to limit the embodiments of this application to specific values or specific scenarios in the example. Obviously, a person skilled in the art may make various equivalent modifications or variations according to the foregoing exemplary description. The modifications or variations also fall with the scope of the embodiments of this application.

The training method of the white balance model and the white balance processing method according to the embodiments of this application are described above in detail with reference toFIG.1toFIG.11. An apparatus embodiment of this application is described in detail below with reference toFIG.12toFIG.14. It is to be understood that, the apparatus in this embodiment of this application may perform the methods in the foregoing embodiments of this application, that is, for specific working processes of the following products, refer to corresponding processes in the foregoing method embodiments.

FIG.12is a schematic structural diagram of an electronic device according to an embodiment of this application. The electronic device700) may perform the training method shown inFIG.8. The electronic device700includes an obtaining module710and a processing module720.

The obtaining module710is configured to obtain training data, where the training data includes a first sample image and a second sample image, the first sample image and the second sample image refer to images of a first color space captured by a multispectral color filter array sensor under a same light source scenario, and the second sample image includes a color checker. The processing module720is configured to perform decomposition processing and demosaicing processing on the first sample image to obtain a third sample image and a fourth sample image, where the third sample image is an image in a first color mode and the fourth sample image is an image in a second color mode. A sample color correction matrix is obtained according to the third sample image and the fourth sample image, where the sample color correction matrix is used for representing a pixel change amount of transforming the third sample image into the fourth sample image. The decomposition processing and the demosaicing processing are performed on the second sample image to obtain a fifth sample image, where the fifth sample image is an image in the first color mode. A white balance model is trained by using the sample color correction matrix as input data and using first pixel information in the fifth sample image as target data to obtain a trained white balance model, where the white balance model is used for calculating a parameter of white balance processing, and the first pixel information refers to a pixel value corresponding to a neutral color block included in the color checker in the fifth sample image.

Optionally, as an embodiment, the processing module720is specifically configured to:input the sample color correction matrix into the white balance model to obtain predicted pixel information; andtrain the white balance model according to the predicted pixel information and the first pixel information to obtain the trained white balance model.

Optionally, as an embodiment, the parameter of the trained white balance model is obtained through iteration using a back propagation algorithm according to a difference between the predicted pixel information and the first pixel information.

Optionally, as an embodiment, the multispectral color filter array sensor refers to a sensor covering a mosaic color filter array on a pixel sensor.

Optionally, as an embodiment, the white balance model is a fully connected neural network.

It should be noted that, the foregoing electronic device700is embodied in the form of functional modules. The term “module” may be implemented through the form of software and/or hardware, which is not specifically limited.

For example, the “module” may be a software program, a hardware circuit, or a combination of the two, to implement the foregoing functions. The hardware circuit may include an application specific integrated circuit (application specific integrated circuit, ASIC), an electronic circuit, a processor (for example, a shared processor, a dedicated processor, or a packet processor) configured to execute one or more software or firmware programs, a memory, a combined logical circuit, and/or another suitable component that supports the described functions.

FIG.13is a schematic structural diagram of an electronic device according to an embodiment of this application. The electronic device800may perform the white balance processing method shown inFIG.7. The electronic device800includes a display module810and a processing module820.

The display module810is configured to display a first interface, where the first interface includes a first control. The processing module820is configured to detect a first operation on the first control; obtain a first image in response to the first operation, where the first image refers to an image of a first color space captured by a multispectral color filter array sensor; perform decomposition processing and demosaicing processing on the first image to obtain a second image and a third image, where the second image is an image in a first color mode and the third image is an image in a second color mode; obtain a color correction matrix according to the second image and the third image, where the color correction matrix is used for representing a pixel change amount of transforming the second image into the third image; input the color correction matrix into a white balance model to obtain a white balance parameter; and perform image processing on the first image according to the white balance parameter to obtain a fourth image, where the white balance model is used for calculating a parameter of white balance processing, the white balance model is obtained through training using a sample color correction matrix as input data and first pixel information as target data, the sample color correction matrix is obtained according to a third sample image and a fourth sample image, the sample color correction matrix is used for representing a pixel change amount of transforming the third sample image into the fourth sample image, the third sample image and the fourth sample image are obtained by performing the decomposition processing and the demosaicing processing on a first sample image, the first pixel information refers to a pixel value corresponding to a neutral color block included in a color checker in a fifth sample image, the fifth sample image is obtained by performing the decomposition processing and the demosaicing processing on a second sample image, the first sample image and the second sample image refer to images of the first color space captured by the multispectral color filter array sensor under a same light source scenario, and the second sample image includes the color checker.

Optionally, as an embodiment, the processing module820is specifically configured to:perform the decomposition processing on the first image to obtain a first Bayer pattern image and a second Bayer pattern image, where the first Bayer pattern image is a Bayer pattern image in the first color mode and the second Bayer pattern image is a Bayer pattern image in the second color mode;perform the demosaicing processing on the first Bayer pattern image to obtain the second image; andperform the demosaicing processing on the second Bayer pattern image to obtain the third image.

Optionally, as an embodiment, the second image includes a first pixel, a second pixel and a third pixel, the third image includes a fourth pixel, a fifth pixel and a sixth pixel, and the processing module820is specifically configured to:obtain a first vector according to a difference between the first pixel and the fourth pixel;obtain a second vector according to a difference between the second pixel and the fifth pixel;obtain a third vector according to a difference between the third pixel and the sixth pixel; andcompose the color correction matrix by the first vector, the second vector, and the third vector.

Optionally, as an embodiment, a parameter of the white balance model is obtained through iteration using a back propagation algorithm according to a difference between predicted pixel information and the first pixel information, and the predicted pixel information refers to output information obtained by inputting the sample color correction matrix into the white balance model.

Optionally, as an embodiment, the multispectral color filter array sensor refers to a sensor covering a mosaic color filter array on a pixel sensor.

Optionally, as an embodiment, the white balance model is a fully connected neural network.

Optionally, as an embodiment, the first interface refers to a main screen interface of the electronic device, the main screen interface comprises a camera application program, and the first control refers to a control corresponding to the camera application program.

Optionally, as an embodiment, the first interface refers to a photographing interface, and the first control refers to a control used for instructing to photograph.

Optionally, as an embodiment, the first interface refers to a video shooting interface, and the first control refers to a control used for instructing to shoot a video.

It should be noted that, the foregoing electronic device800is embodied in the form of functional modules. The term “module” may be implemented through the form of software and/or hardware, which is not specifically limited.

For example, the “module” may be a software program, a hardware circuit, or a combination of the two, to implement the foregoing functions. The hardware circuit may include an application specific integrated circuit (application specific integrated circuit, ASIC), an electronic circuit, a processor (for example, a shared processor, a dedicated processor, or a packet processor) configured to execute one or more software or firmware programs, a memory, a combined logical circuit, and/or another suitable component that supports the described functions.

Therefore, the units in the examples described in the embodiments in this application may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed by hardware or software depends on particular applications and design constraint conditions of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of this application.

FIG.14shows a schematic structural diagram of an electronic device according to this application. A dashed line inFIG.14indicates that the unit or the module is optional. The electronic device900may be configured to implement the method described in the foregoing method embodiments.

The electronic device900includes one or more processors901, and the one or more processors901may support the electronic device900to implement the training method of the white balance model or the white balance processing method in the method embodiments. The processor901may be a general processor or a dedicated processor. For example, the processor901may be a central processing unit (central processing unit, CPU), a digital signal processor (digital signal processor, DSP), an application-specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA), or another programmable logic device, such as a discrete gate, a transistor logic device, or a discrete hardware component.

The processor901may be configured to control the electronic device900, execute a software program, and process data of the software program. The electronic device900may further include a communication unit905configured to implement input (receiving) and output (transmitting) of a signal.

For example, the electronic device900may be a chip, and the communication unit905may be an input and/or output circuit of the chip, or, the communication unit905may be a communication interface of the chip, and the chip may be used as a component of a terminal device or another electronic device.

For another example, the electronic device900may be a terminal device, and the communication unit905may be a transceiver of the terminal device, or, the communication unit905may be a transceiver circuit of the terminal device.

The electronic device900may include one or more memories902, storing a program904, and the program904may be run by the processor901to generate instructions903, causing the processor901to perform the training method or the white balance processing method described in the foregoing method embodiments according to the instructions903.

Optionally, the memory902may be further store data. Optionally, the processor901may further read the data stored in the memory902, and the data may be stored in the same storage address as the program904, or the data may be stored in a different storage address with the program904.

The processor901and the memory902may be provided separately, or may be integrated together, for example, integrated on a system on chip (system on chip, SOC) of the terminal device.

Exemplarily, the memory902may be configured to store the relevant program904of the training method of the white balance model provided in the embodiments of this application, and the processor901may be configured to invoke, when executing the training of the white balance model, the relevant program904of the training method of the white balance model stored in the memory902to perform the training method of the white balance model according to the embodiments of this application. For example, obtaining training data, where the training data includes a first sample image and a second sample image, the first sample image and the second sample image refer to images of a first color space captured by a multispectral color filter array sensor under a same light source scenario, and the second sample image includes a color checker; performing decomposition processing and demosaicing processing on the first sample image to obtain a third sample image and a fourth sample image, where the third sample image is an image in a first color mode and the fourth sample image is an image in a second color mode; obtaining a sample color correction matrix according to the third sample image and the fourth sample image, where the sample color correction matrix is used for representing a pixel change amount of transforming the third sample image into the fourth sample image; performing the decomposition processing and the demosaicing processing on the second sample image to obtain a fifth sample image, where the fifth sample image is an image in the first color mode; and training a white balance model by using the sample color correction matrix as input data and using first pixel information in the fifth sample image as target data to obtain a trained white balance model, where the white balance model is used for calculating a parameter of white balance processing, and the first pixel information refers to a pixel value corresponding to a neutral color block included in the color checker in the fifth sample image.

Exemplarily, the memory902may be configured to store the relevant program904of the white balance processing method provided in the embodiments of this application, and the processor901may be configured to invoke, when executing the white balance processing, the relevant program904of the white balance processing method stored in the memory902to perform the white balance processing method according to the embodiments of this application. For example: displaying a first interface, where the first interface includes a first control; detecting a first operation on the first control; obtaining a first image in response to the first operation, where the first image refers to an image of a first color space captured by a multispectral color filter array sensor; performing decomposition processing and demosaicing processing on the first image to obtain a second image and a third image, where the second image is an image in a first color mode and the third image is an image in a second color mode; obtaining a color correction matrix according to the second image and the third image, where the color correction matrix is used for representing a pixel change amount of transforming the second image into the third image; inputting the color correction matrix into a white balance model to obtain a white balance parameter; and performing image processing on the first image according to the white balance parameter to obtain a fourth image, where the white balance model is used for calculating a parameter of white balance processing, the white balance model is obtained through training using a sample color correction matrix as input data and first pixel information as target data, the sample color correction matrix is obtained according to a third sample image and a fourth sample image, the sample color correction matrix is used for representing a pixel change amount of transforming the third sample image into the fourth sample image, the third sample image and the fourth sample image are obtained by performing the decomposition processing and the demosaicing processing on a first sample image, the first pixel information refers to a pixel value corresponding to a neutral color block included in a color checker in a fifth sample image, the fifth sample image is obtained by performing the decomposition processing and the demosaicing processing on a second sample image, the first sample image and the second sample image refer to images of the first color space captured by the multispectral color filter array sensor under a same light source scenario, and the second sample image includes the color checker.

This application further provides a computer program product, where when the computer program product is executed by a processor901, the training method or the white balance processing method described in any one of the method embodiments of this application is implemented.

The computer program product may be stored in a memory902. For example, the computer program product may be a program904, where the program904is finally transformed, after processing processes such as preprocessing, compiling, assembling, and linking, into an executable target file that can be executed by the processor901.

This application further provides a computer-readable storage medium, storing a computer program. When the computer program is executed by a computer, the white balance processing method described in any one of the method embodiments of this application is implemented. The computer program may be a high-level language program, or may be an executable target program.

The computer-readable storage medium is, for example, a memory902. The memory902may be a volatile memory or a non-volatile memory, or, the memory902may include a volatile memory and a non-volatile memory. The non-volatile memory may be a read-only memory (read-only memory, ROM), a programmable ROM (programmable ROM, PROM), an erasable PROM (erasable PROM, EPROM), an electrically EPROM (electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (random access memory, RAM), and is used as an external cache. Through illustrative but not limitative description, many forms of RAMs may be used, for example, a static RAM (static RAM, SRAM), a dynamic RAM (dynamic RAM, DRAM), a synchronous DRAM (synchronous DRAM, SDRAM), a double data rate SDRAM (double data rate SDRAM. DDR SDRAM), an enhanced SDRAM (enhanced SDRAM, ESDRAM), a synchlink DRAM (synchlink DRAM, SLDRAM), and a direct rambus RAM (direct rambus RAM, DR RAM).

It may be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, for a detailed working process and a generated technical effect of the foregoing apparatus and device, refer to a corresponding process and technical effect in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided in this application, the disclosed system, apparatus, and method may be implemented in other manners. For example, some features of the described method embodiment may be ignored or not performed. The described apparatus embodiment is merely exemplary. The unit division is merely logical function division and may be other division in actual implementation. A plurality of units or components may be combined or integrated into another system. In addition, the coupling between units and the coupling between components may be direct couplings, or may be indirect couplings. The foregoing couplings include connections in electrical, mechanical, or other forms.

It should be understood that sequence numbers of the foregoing processes do not mean execution sequences in various embodiments of this application. The execution sequences of the processes should be determined according to functions and internal logic of the processes, and should not be construed as any limitation on the implementation processes of the embodiments of this application.

In addition, the terms “system” and “network” may be used interchangeably in this specification. The term “and/or” in this specification describes only an association relationship for describing associated objects and represents that three relationships may exist. For example, A and/or B may represent the following three cases: Only A exists, both A and B exist, and only B exists. In addition, the character “/” in this specification generally indicates an “or” relationship between the associated objects.

In conclusion, the foregoing descriptions am merely exemplary embodiments of this application, and are not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of this application shall fall within the protection scope of this application.