Image processing method and image processing apparatus, and program thereof

A color chart and an illumination unevenness correcting chart are shot by a digital camera 1 under the same illumination, and color data on a first image obtained by shooting the color chart and color data on a second image obtained by shooting the illumination unevenness correcting chart are input to a color processing parameter editing apparatus 2 for generating a color processing condition used in color processing to be performed on an image shot by the digital camera 1. A correcting unit 204 corrects the color data on the first image using the color data on the second image. Target color data corresponding to the corrected color data on the first image is input. A parameter optimizing unit 205 generates a color processing condition for converting the corrected color data on the first image into the target color data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing method and image processing apparatus, and program thereof for generating a color processing condition used in color processing to be performed on an image shot by an imaging apparatus.

2. Description of the Related Art

Imaging apparatuses such as conventional digital cameras are capable of storing a plurality of color processing parameters therein in advance and allowing selection among a plurality of modes. According to a user selection of one of the modes, digital cameras use a color processing parameter corresponding to the selected mode to perform color processing on shot image data.

However, the conventional digital cameras only allow selection among predetermined modes and do not allow use of other color processing parameters, for example those that realize the user's preferred color reproduction.

To address this, the present applicant has proposed a color processing parameter customizing apparatus for the purpose of providing an apparatus allowing a user to arbitrarily set color reproduction of a digital camera according to the user's preference (see Japanese Patent Laid-Open No. 2004-341923). A color chart including colors of skin, the sky, and grass and fundamental colors is shot by a digital camera. Then, an optimal color processing parameter can be determined that brings the imaging data closer to the original colors of the color chart, or that brings the colors of skin, the sky, and grass and the fundamental colors closer to the user's preferred colors (target data).

However, the apparatus performs processing on the assumption that no illumination unevenness is caused by the light emitted on the color chart when the imaging data is obtained. This requires highly precise adjustment in actual shooting, and there is a need for a simple technique for this adjustment.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the above situation and aims to provide an image processing method and image processing apparatus, and program thereof that allow precise color adjustment in a simple manner irrespective of the presence or absence of illumination unevenness at the time of image shooting.

According to the present invention, the foregoing object is attained by providing an image processing method in an image processing apparatus for generating a color processing condition used in color processing to be performed on an image shot by imaging means, comprising:

a color data input step for shooting a color chart and an illumination unevenness correcting chart by the imaging means under the same illumination and inputting color data on a first image obtained by shooting the color chart and color data on a second image obtained by shooting the illumination unevenness correcting chart to the image processing apparatus;

a correcting step for correcting the color data on the first image using the color data on the second image;

a target color data input step for inputting target color data corresponding to the corrected color data on the first image to the image processing apparatus; and

a generating step for generating a color processing condition for converting the corrected color data on the first image into the target color data.

According to another aspect of the present invention, the foregoing object is attained by providing an image processing method in an image processing apparatus for generating a color processing condition used in color processing to be performed on an image shot by imaging means, comprising:

a data input step for shooting a color chart and an illumination unevenness correcting chart by the imaging means under the same illumination and inputting color data on a first image obtained by shooting the color chart, color data on a second image obtained by shooting the illumination unevenness correcting chart, and target color data corresponding to corrected color data on the first image obtained by correcting the color data on the first image using the color data on the second image to the image processing apparatus as text data; and

a generating step for generating a color processing condition for converting the corrected color data on the first image into the target color data.

According to another aspect of the present invention, the foregoing object is attained by providing an image processing apparatus for generating a color processing condition used in color processing to be performed on an image shot by imaging means, comprising:

color data input means for shooting a color chart and an illumination unevenness correcting chart by the imaging means under the same illumination and inputting color data on a first image obtained by shooting the color chart and color data on a second image obtained by shooting the illumination unevenness correcting chart;

correcting means for correcting the color data on the first image using the color data on the second image;

target color data input means for inputting target color data corresponding to the corrected color data on the first image; and

generating means for generating a color processing condition for converting the corrected color data on the first image into the target color data.

DESCRIPTION OF THE EMBODIMENTS

First Embodiment

[Configuration of Image Processing System]

FIG. 1is a diagram showing a configuration of an image processing system according to a first embodiment of the present invention. The image processing system shown inFIG. 1includes a digital camera1, which is an imaging apparatus, and a color processing parameter editing apparatus2. The digital camera1shoots a color chart3.

The digital camera1includes an imaging unit101comprised of an imaging lens and an image sensor such as CCD, and a data input/output unit102functioning as an interface for communicating with external devices for image data and color processing parameters. The digital camera1also includes a color processing parameter storage unit104for storing the color processing parameters, and a display unit105, such as a liquid crystal display, for displaying an image that is being shot or has been shot. The digital camera1further includes an image processing unit103generating image data from a captured image obtained by the imaging unit101, by using a color processing parameter stored in the color processing parameter storage unit104and generating image data.

The color processing parameter editing apparatus2includes a data input/output unit201functioning as an interface for communicating with external devices for the image data, the color processing parameters, and target data. The color processing parameter editing apparatus2also includes a color processing parameter storage unit206for storing the color processing parameters for the digital camera1, and an input data/target data storage unit207for storing input data and the target data. The color processing parameter editing apparatus2further includes an image processing unit202generating image data from an object image obtained by the imaging unit101of the digital camera1by using a color processing parameter stored in the color processing parameter storage unit206. The color processing parameter editing apparatus2further includes an averaging unit203for calculating average RGB values of each patch of image data that is input via the data input/output unit201, and a correcting unit204for correcting the input average RGB values. The color processing parameter editing apparatus2further includes a parameter optimizing unit205for generating a color processing condition by optimizing a color processing parameter stored in the color processing parameter storage unit206using, for example, the DLS technique.

[Processing in Image Processing System]

When a user is going to shoot an object with the digital camera1, the user turns a power switch on (not shown). Next, the image processing unit103generates a display image from image data obtained by the imaging unit101by using a parameter stored in the color processing parameter storage unit104. The display image is displayed on the display unit105. The user determines the camera composition based on the image displayed on the display unit105and presses a shutter button (not shown) to shoot the image.

In response to pressing the shutter button, the image processing unit103processes the colors, brightness, and so forth of the image data captured by the imaging unit101, by using a parameter stored in the color processing parameter storage unit104. The processed image data is further output via the data input/output unit102. The image data may be directly output to the color processing parameter editing apparatus2via a cable or the like, or may be output to a portable storage medium such as CompactFlash®.

The color processing parameter storage unit104may contain a parameter that causes the image data obtained by the imaging unit101to be directly output to the data input/output unit102. With this parameter, the image data is able to be output to the color processing parameter editing apparatus2in the same form as obtained from the imaging unit101.

FIG. 2is a diagram showing an example of the color chart3that is shot by the digital camera1in this embodiment. The color chart3may be a chart for use in color design, for example the ColorChecker for DC (240 colors) or the ColorChecker (24 colors) of GretagMacbeth Ltd.

FIG. 5is a diagram showing an example of an illumination unevenness correcting chart4that is shot by the digital camera1in this embodiment. The illumination unevenness correcting chart4may be a uniformly white chart, for example the N8 uniformly white chart that is custom-designed by GretagMacbeth Ltd.

FIG. 3is a flowchart for describing image processing steps in the color processing parameter editing apparatus2according to the embodiment of the present invention. First, a color processing parameter is optimized (step S1). Details of operations in step S1will be described below with reference toFIG. 4. Next, a color processing condition generated by the optimization, such as a 3DLUT, is output from the data input/output unit201and uploaded to the digital camera1(step S2).

FIG. 4is a flowchart for describing detailed processing steps of the color processing parameter optimization (step S1) in the flowchart ofFIG. 3.

For each of the read two images, average RGB values of each patch are determined in the averaging unit203(step S12). The average RGB values of the patches determined in step S12are corrected in the correcting unit204(step S13). In this embodiment, the correction is performed in the following manner.

The average RGB values of any one patch on the color chart3are expressed as Rorg, Gorg, Borg. The average RGB values of a patch on the illumination unevenness correcting chart4corresponding to the same range (image area) as the patch on the color chart3are expressed as Rw, Gw, Bw. Further, the maximum values of measured average RGB values on the illumination unevenness correcting chart4are expressed as Rmax, Gmax, Bmax. Then, the following equation (1) is used to calculate corrected average RGB values Raft, Gaft, Baft.
Raft=Rorg*Rmax/Rw
Gaft=Gorg*Gmax/Gw
Baft=Borg*Bmax/Bw(1)

After the correction in step S13, input data is calculated (step S14). In this embodiment, the image processing unit202performs color processing on the corrected average RGB values using a color processing parameter that is set in advance in the color processing parameter storage unit206. The color processing parameter may be a 3×3 matrix, a 3×9 matrix, or a 3×9 matrix. It is assumed that the processed RGB data conforms to sRGB defined by IEC61966-2-1, and the processed RGB data converted into CIELAB is the input data.

Next, target data (target color data) is input (step S15). In this embodiment, CIEXYZ values obtained by measuring the colors of each color patch itself on the color chart3with the SpectroLino of GretagMacbeth Ltd. or the like are used as the target data.

The color processing parameter is optimized in the parameter optimizing unit205(step S16). In the optimization, it is assumed that each input RGB data conforms to sRGB defined by IEC61966-2-1. Each input RGB data corresponding to each patch is converted into CIELAB data (L*i, a*i, b*i) (“i” expresses a patch number), based on the above assumption. CIEXYZ data, which is input target data, is converted into CIELAB data in step S15. Every converted target data is expressed as (L*Ti, a*Ti, b*Ti) (“i” expresses a patch number). Next, the color processing parameter is optimized with a technique such as the DLS technique so that all color-processed input data is brought as close as possible to the corresponding target data. For example, a color parameter that minimizes the value E defined in the following equation (2) is calculated, where Eiis an evaluation function for each color patch and E is an overall evaluation function.

A weight may be individually set for each patch. In that case, the evaluation function is calculated using the following equation (3), where wiis a weight value.

First, the above 4913 (=17×17×17) data items for (R, G, B) are sequentially input. Next, the optimized color processing parameter is used to calculate R′G′B′ values. Assuming that the R′G′B′ values conforms to sRGB defined by IEC 61966-2-1, L*a*b* values are calculated by converting the R′G′B′ values into CIELAB. The calculated 4913 pairs of (R, G, B)−(L*, a*, b*) are stored in the color processing parameter storage unit206as the 3DLUT.

Now, description will be given of the method of calculating the R′G′B′ values in steps S14, S15, S16, and S17, and the conversion equations of RGB into CIEXYZ and CIEXYZ into CIELAB.

First, the input RGB values are normalized. For the input RGB values of ten bits, the normalization is performed by dividing each value by 1023 (=210−1). Next, any one of the following equations (4) to (6) is used to calculate the R′G′B′.

[Conversion of RGB into CIEXYZ]

The conversion is performed with the following equations (7) to (10).

The conversion is performed with the following equations (11) to (16).

where Xn, Yn, and Zn are values calculated from the color temperature of the illumination at the time of shooting.

Other Embodiments

In the above-described embodiment, data on two images is input to use the input data and the illumination unevenness correcting data by determining the average RGB values of each patch on each image in the apparatus. Alternatively, the processing in steps S11, S12, and S15may be performed in an external apparatus so that text data as shown inFIG. 6is input.FIG. 6is a diagram showing an example of text data where the input data, the illumination unevenness correcting data, and the target data are read from a text file.

Although only one illumination unevenness correcting chart is used in the above-described embodiment, a plurality of illumination unevenness correcting charts with different levels of luminance may also be used. In that case, for example, three charts of white, gray, and black may be provided to shoot them under the same illumination as that for the color chart3. In correcting, the average RGB values of any one patch on the color chart3are expressed as Rorg, Gorg, Borg. The average RGB values of patches on the illumination unevenness correcting charts4corresponding to the same range as the patch on the color chart3are expressed as Rw, Gw, Bw, (white), Rg, Gg, Bg(gray), and Rk, Gk, Bk(black). Further, the maximum values of measured average RGB values on the illumination unevenness correcting charts4are expressed as Rwm, Gwm, Bwm(white), Rgm, Ggm, Bgm(gray), and Rkm, Gkm, Bkm(black). Still further, corrected average RGB values are expressed as Raft, Gaft, Baft, which are calculated with the following equations (17) to (21).

In addition, although the above-described embodiment uses the equation (2) or (3) as the evaluation function, it is to be understood that ΔE94expressed by the following equation (22) or (23) may also be used.

While the exemplary embodiments have been described in detail above, the present invention may be embodied as, for example, a system, apparatus, method, program, or storage medium (recording medium). Specifically, the present invention may be applied to a system comprised of a plurality of devices or to a system as a single device.

The present invention also covers the case where a program of software for implementing the functions of the above-described embodiments (for the above embodiments, a program corresponding to the flowcharts shown in the drawings) is provided directly or remotely to a system or apparatus. A computer in the system or apparatus then reads and executes the provided program code to achieve the invention.

Therefore, the program code itself installed to the computer for implementing the functional processing of the present invention also realizes the present invention. That is, the present invention also includes the computer program itself for implementing the functional processing of the present invention.

In that case, the present invention may be embodied as object code, a program executed by an interpreter, or script data provided to an OS, as long as it has program functionality.

Examples of the recording medium for providing the program includes a floppy (R) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, and DVD (DVD-ROM and DVD-R).

Alternatively, the program may be provided by using a browser on a client computer and downloading the program from a web page on the Internet to a recording medium such as a hard disk. That is, a connection is made to the web page, from which the computer program itself of the present invention or a compressed file having an automatic installing function is downloaded. The program may also be provided by dividing the program code constituting the program of the present invention into a plurality of files so that each file is downloaded from a different web page. That is, the present invention also includes a WWW server that allows a plurality of users to download a program file for implementing the functional processing of the present invention on a computer.

The program of the present invention may also be stored in a storage medium such as a CD-ROM in an encrypted form and provided to users. Users who meet certain criteria are allowed to download decryption key information from a web page via the Internet. The users may use the key information to execute the encrypted program and install the program to their computer.

The functions of the above-described embodiments may be implemented by a computer executing the read program. It is also possible that an OS or the like operating on a computer performs part or all of the actual processing under instructions of the program, and the processing implements the functions of the above-described embodiments.

The functions of the above-described embodiments may also be implemented after the program read from the recording medium is written to memory provided on a function extension board inserted into a computer or in a function extension unit connected to a computer. That is, the functions of the above-described embodiments may also be implemented in a manner in which part or all of the actual processing is performed by a CPU or the like provided on the function extension board or in the function extension unit under instructions of the program.

The present invention allows precise color adjustment in a simple manner irrespective of the presence or absence of illumination unevenness at the time of image shooting.

This application claims the benefit of Japanese Patent Application No. 2005-264428, filed on Sep. 12, 2005, which is hereby incorporated by reference herein in its entirety.