Image processing apparatus, system, method, and program product for adjusting saturation of a skin area while maintaining converted hue

An image processing apparatus includes a skin area extraction unit that extracts a skin area from image data, a chromaticity computation unit that computes hue information and saturation information of the skin area, a hue conversion unit that converts the hue information so that the hue information is in agreement with target information, a reception unit that receives a change instruction to change the saturation information, a parameter determination unit that determines a conversion parameter based on the change instruction, and a saturation conversion unit that converts the saturation information based on the conversion parameter.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, an image processing system, an image processing method, and a recording medium.

BACKGROUND ART

Many people are accustomed to seeing an image in which a person is a photographic subject. If a color of a person's skin in the image is not properly reproduced, the reproduced image may lead to a sense of incongruity to the viewer. Moreover, in a case where a photograph of a person who wears a red hat or red clothes is taken, the person's skin may look bluish even if the skin color itself is properly reproduced. There is a visual sense characteristic that the skin color does not look proper due to the effect of contrast between the skin color and the background color.

To improve an impression of a person photographed in an image, image processing may be performed so that a person's skin color is changed to a color different from the actual condition. For example, if the color of a woman's skin in the image is made more fair or the color of a man's skin in the image is made more dark, the impression of the person within the image may be changed. In a case of images used for commercial purposes, such as advertisement and promotion, it is demanded to give an impression that a person and goods are suitably fitted to each other. In this case, image processing which adjusts the skin color to such appearance becomes more important. Moreover, in a case of a print market in which photographs are left as important recollections, such as a photograph book, the highest priority matter is to reproduce the skin color so that a person's impression in an image may improve.

An image processing method has been proposed in which color components of pixels of a skin area in an image are acquired using three attributes of brightness, saturation, and hue of the pixels and the color components of the skin area are adjusted to change partially distributions represented by two of the three attributes so that a skin color of a person in an image is adjusted. For example, see Japanese Laid-Open Patent Publication No. 2007-158824.

SUMMARY OF THE INVENTION

Problems to be Solved by the Invention

However, in the image processing method of Japanese Laid-Open Patent Publication No. 2007-158824, what is operated by a user is parameters which represent the skin color in the image, such as brightness, saturation, and hue. It is often difficult for a user who is unfamiliar with the image processing to adjust the parameters and obtain a desired skin color expression. In a case of a skin color, unlike primary colors like red or yellow, the impact of delicate color changes on the viewer is great and adjustment of color tone or saturation is very difficult. Hence, it is difficult for the above-described image processing method to reproduce a user's desired skin color expression with ease.

Moreover, in the image processing method of Japanese Laid-Open Patent Publication No. 2007-158824, image processing may be performed based on predetermined standard hue conversion parameters. However, the degree of freedom of image processing by the user will be lost and the user's desired skin color expression will not necessarily be reproduced.

Accordingly, in view of the above-described problems, the present invention aims at providing an image processing apparatus which reproduces a desired skin color expression with ease.

Means for Solving the Problems

In one aspect, the present invention provides an image processing apparatus which is capable of reproducing a desired skin color expression with ease.

In one embodiment, the present invention provides an image processing apparatus including a memory storing computer readable code that, when executed by a processor, configures the processor as: a skin area extraction unit configured to extract a skin area from image data; a chromaticity computation unit configured to compute hue information and saturation information of the skin area; a hue conversion unit configured to convert the hue information so that the hue information is in agreement with target information; a reception unit configured to receive a change instruction to change the saturation information; a parameter determination unit configured to determine a conversion parameter based on the change instruction; and a saturation conversion unit configured to convert the saturation information based on the conversion parameter.

According to one embodiment of the present invention, it is possible to provide an image processing apparatus which reproduces a desired skin color expression with ease.

Other objects, features and advantages of embodiments will be apparent from the following detailed description when read in conjunction with the accompanying drawings.

MODE FOR CARRYING OUT THE INVENTION

In the following, a description will be given of preferred embodiments with reference to the accompanying drawings.

First Embodiment

FIG. 1shows a hardware configuration of an image processing apparatus100according to a first embodiment. As shown inFIG. 1, the image processing apparatus100includes a control unit101, a main memory unit102, a secondary memory unit103, a recording medium interface unit104, a network interface unit105, an operation unit106, and a display unit107, which are interconnected by a bus B.

The control unit101may include a CPU (central processing unit) which performs control of the respective units of the image processing apparatus and performs computation and processing of data. The control unit101may include a processor which executes a program stored in the main memory unit102, and the processor receives data from an input unit or a storage unit, performs computation and processing of the data, and outputs the processed data to an output unit or a storage unit.

The main memory unit102may include a ROM (read only memory), a RAM (random access memory), etc. In the main memory unit102, an OS (operating system) as the basic software executed by the control unit101, application programs and data are stored or temporarily retained.

The secondary memory unit103may include a HDD (hard disk drive) or the like. In the secondary memory unit103, data relevant to the application programs and others are stored.

The recording medium interface unit104provides an interface between a recording medium108, such as a flash memory, and the image processing apparatus100. For example, by using a data transmission line, such as USB (universal serial bus), the recording medium interface unit104is connected to the recording medium108.

A predetermined program is stored in the recording medium108, and the recording medium108is attached to the image processing apparatus100. The predetermined program stored in the recording medium108is installed in the main memory unit102of the image processing apparatus100through the recording medium interface unit104. After the installation, the predetermined program is read from the main memory unit102and executed by the control unit101of the image processing apparatus100.

The network interface unit105provides an interface between a not-shown peripheral device and the image processing apparatus100, the peripheral device having a communication function and being connected to the image processing apparatus100via a wired or wireless network which is constructed of data transmission lines, such as LAN (local area network) or WAN (wide area network).

The operation unit106may include key switches composed of hard keys, a mouse, etc.

The display unit107is, for example, an LCD (liquid crystal display), an organic EL (electroluminescence) display, etc. Images, operational icons, etc., are displayed on the display unit107and the display unit107serves as a user interface for a user to perform various setting processes when using functions of the image processing apparatus100.

FIG. 2is a diagram showing a functional configuration of the image processing apparatus100according to the first embodiment. As shown inFIG. 2, the image processing apparatus100includes a skin area extraction unit201, a skin color information acquisition unit202, a chromaticity computation unit203, a storage unit204, a target information acquisition unit205, a hue conversion unit206, a reception unit207, a parameter determination unit208, a saturation conversion unit209, and a display input unit210.

The skin area extraction unit201, the skin color information acquisition unit202, the chromaticity computation unit203, the target information acquisition unit205, the hue conversion unit206, the parameter determination unit208, and the saturation conversion unit209are functional units of the image processing apparatus100implemented by the control unit101when executing the program stored in the main memory unit102or the like. The storage unit204is implemented by the main memory unit102and the secondary memory unit103. The reception unit207and the display input unit210are functional units of the image processing apparatus100implemented by the operation unit106and the display unit107controlled by the control unit101.

In the following, the functions of the respective units of the image processing apparatus100to perform image processing on input image data will be described with reference toFIGS. 3A through 8B.

<Skin Area Extraction and Skin Color Information Acquisition>

When image data indicating a portrait image (including a person) on which the image processing is to be performed is input to the image processing apparatus100, the skin area extraction unit201extracts a person's skin area from the input image data. In the following, the image data input to the image processing apparatus100is called “input image data.”

FIGS. 3A and 3Bare diagrams showing an input image data301which is input to the image processing apparatus100, and a skin area302in the input image data301.FIG. 3Ashows the input image data301, andFIG. 3Bshows the skin area302(a white portion) extracted from the input image data301by the skin area extraction unit201.

The skin area extraction unit201compares color information of the pixels which constitute the input image data301with a predetermined skin color threshold, and determines a skin area or a non-skin area for each of the pixels. The determination is made for all the pixels which constitute the input image data301, and a set of pixels that have been determined as being in the skin area is determined as the skin area302.

In the example shown inFIG. 3B, the skin area302(a white portion) and an area (a black portion) other than the skin area302are separated by a clear boundary line. Alternatively, the boundary line between the skin area302and the area other than the skin area302may be shaded off. Moreover, the degree of shading of the boundary line may be changed according to a location.

After the skin area302is extracted from the input image data301by the skin area extraction unit201, the skin color information acquisition unit202acquires skin color information of the pixels which constitute the skin area302.

For example, the skin color information is represented by an 8-bit gradation value (0-255) for each of RGB color components of all the pixels which constitute the skin area302. However, depending on the image processing environment, various color specification components may be applied. For example, the YUV or YCbCr color system applied to the internal processing of a display or a digital camera may be used instead. Alternatively, when image data is expressed by four color components of CMYK which is used in offset printing, halftone dot ratios of CMYK (%) may be used as the color specification components.

In the image processing apparatus100according to the present embodiment, a user may designate a skin area302using the operation unit106, while viewing the input image data301which is displayed on the display unit107. Also in this case, the skin color information acquisition unit202acquires the skin color information from the designated skin area302similarly.

After the skin color information is acquired from the skin area302by the skin color information acquisition unit202, the chromaticity computation unit203performs chromaticity computation based on the skin color information and computes hue histogram information which indicates a frequency distribution of pixels corresponding to a hue angle H in the skin area302and saturation histogram information which indicates a frequency distribution of pixels corresponding to a saturation S in the skin area302. The chromaticity may be determined by selecting an optimal one of color spaces, such as LCH, HSV, etc., according to a working environment. In the present embodiment, the HSV (hue, saturation, value) color space is used.

A frequency in the frequency distribution computed by the chromaticity computation unit203is represented by a ratio of the number of pixels corresponding to a certain hue angle H (or a certain saturation S) to the total number of pixels which constitutes the skin area302. A value of the frequency is in a range between 0 and 1.

FIGS. 4A and 4Bare diagrams showing examples of hue histogram information401and saturation histogram information402of the skin area302computed by the chromaticity computation unit203.FIG. 4Ashows the hue histogram information401, and inFIG. 4A, the lateral axis denotes the hue angle H and the vertical axis denotes the frequency of pixels to the hue angle H.FIG. 4Bshows the saturation histogram information402, and inFIG. 4B, the lateral axis denotes the saturation S and the vertical axis denotes the frequency of pixels to the saturation S.

Target information which is predetermined as being a target hue histogram is stored in the storage unit204of the image processing apparatus100, and the target information is read from the storage unit204by the target information acquisition unit205.

FIG. 5shows an example of the target information501. Similar to the hue histogram information401, the target information501shown inFIG. 5indicates a target frequency distribution of pixels corresponding to a hue angle H. InFIG. 5, the lateral axis denotes the hue angle H and the vertical axis denotes the frequency of pixels to the hue angle H.

The target information501is hue histogram information of a skin area with which a viewer of a portrait image (including a person) feels with high probability that the skin color reproduction of the person in the image is expressed in an average and comfortable fashion. Such target information501is quantitatively determined through the analysis of subjective evaluation results obtained from many test subjects using a plurality of sets of sample image data.

A plurality of target information items may be stored in the storage unit204. An optimal target information item may be prepared for each of several categories which are classified by race, age or sex. In this case, the target information acquisition unit205may recognize a characteristic (race, age or sex) of a person included in the input image data301and may acquire selectively a target information item suited for the characteristic from among the target information items stored in the storage unit204.

After the hue histogram information401of the skin area302is computed by the chromaticity computation unit203and the target information501is acquired by the target information acquisition unit205, the hue conversion unit206converts the hue of the skin area302based on the target information501. The hue conversion unit206performs a conversion process so that the hue histogram information401of the skin area302as the object of the conversion is in agreement with the target information501.

FIG. 6Ais a diagram for explaining the conversion process performed by the hue conversion unit206, and shows examples of cumulative frequency distributions of the hue histogram information401and the target information501.

A graph on the right-hand side ofFIG. 6Ais a cumulative hue histogram601in which a cumulative frequency based on the hue histogram information401of the skin area302as the object of the conversion is taken along the vertical axis. A graph on the left-hand side ofFIG. 6Ais a cumulative target histogram602in which a cumulative frequency based on the target hue histogram information as the target information501is taken along the vertical axis and the lateral axis (hue angle H) is reversed. InFIG. 6A, the hue angle H of the cumulative hue histogram601(the right-hand side half) increases in the rightward direction, and the hue angle H of the cumulative target histogram602(the left-hand side half) increases in the leftward direction. Both the graphs reach the maximum value of 1.

As shown inFIG. 6A, when a cumulative frequency of a hue angle H1in the cumulative hue histogram601is equal to g1, the hue conversion unit206is configured to determine a hue angle H1′ of the cumulative target histogram602corresponding to the cumulative frequency g1as a hue angle after conversion of the hue angle H1. Similarly, when a cumulative frequency of a hue angle H2in the cumulative hue histogram601is equal to g2, the hue conversion unit206is configured to determine a hue angle H2′ of the cumulative target histogram602corresponding to the cumulative frequency g2as a hue angle after conversion of the hue angle H2.

In this manner, the hue conversion unit206is configured to determine a maximum hue angle Hmax′ and a minimum hue angle Hmin′ in the cumulative target histogram602as a hue angle after conversion of a maximum hue angle Hmax in the cumulative hue histogram601and a hue angle after conversion of a minimum hue angle Hmin in the cumulative hue histogram601, respectively.

As described above, the hue conversion unit206is configured to determine hue angles after conversion in the cumulative target histogram602correlated to hue angles before conversion in the cumulative hue histogram601based on the correspondence of hue angles to a same cumulative frequency in the cumulative hue histogram601and the cumulative target histogram602, and generate a hue conversion table603as shown inFIG. 6B.

As shown inFIG. 6B, the lateral axis in the hue conversion table603denotes a hue angle before conversion, and the vertical axis in the hue conversion table603denotes a corresponding hue angle after conversion. The hue conversion unit206is configured to convert the hue angles H of all the pixels which constitute the skin area302in the input image data301into the hue angles H′ based on the hue conversion table603. In the following, an image in which the hue angles H of the skin area302have been converted is called “image data after hue conversion.”

In the image processing apparatus100according to the present embodiment, the hue conversion of the input image data301does not require a user to perform a complicated parameter operation and the image data of the skin area302in the input image data301may be converted to have the average color tone which does not lead to a sense of incongruity to the viewer.

<Determination of Saturation Conversion Parameter>

After the hue conversion of the skin area302is performed by the hue conversion unit206and the image data after the hue conversion is generated, the display input unit210displays the input image data301and the image data after the hue conversion on the display unit107.

FIG. 7shows an example of a designation screen which includes the input image data301, image data701after the hue conversion, image data702after saturation conversion, and a slide bar703for allowing a user to designate a suntan level of skin.

In a middle row of the screen, the slide bar703for receiving a user's designation of a suntan level of skin of a person contained in the image is displayed. In the example shown inFIG. 7, if the slide bar703is moved leftward, the person's skin color in the image continuously changes to white skin in which suntan is suppressed, and if the slide bar703is moved rightward, the person's skin color in the image continuously changes to a brown skin in which the skin is tanned well.

In the image data702after the saturation conversion displayed on the screen, the result of image conversion processing (which will be described later) which is performed according to a position of the slide bar703is reflected in real time. The user designates a desired suntan level of the skin by moving the slide bar703displayed on the screen rightward or leftward with the operation unit106, such as a mouse, while checking a final image. After an “OK” button at a lower row of the screen is clicked, the designation of the suntan level by the user is determined. To redo the designation of the suntan level, a “CANCEL” button at the lower row of the screen is clicked so that the user designation is again possible.

If the user moves the slide bar703and designates a suntan level, the reception unit207receives the designated suntan level and the parameter determination unit208determines a saturation conversion parameter based on the received suntan level. In the example shown inFIG. 7, a suntan level of skin ranging from a minimum value (−5) to a maximum value (+5) and centered on zero may be designated by operating the slide bar703. For example, the parameter determination unit208is configured to determine a saturation conversion parameter linearly based on the received suntan level so that the saturation conversion parameter is set to “1” when the received suntan level is zero, set to “0.5” when the received suntan level is the minimum value (−5), and set to “1.5” when the received suntan level is the maximum value (+5). In this embodiment, the parameter determination unit208is configured to determine a saturation conversion parameter k in accordance with a conversion formula: k=(1+0.1L) where L denotes the suntan level.

The range of the suntan level L is not limited to the range between −5 and +5 as in the present embodiment. The values of the suntan level L may be set up in an arbitrary range of numerical values with an arbitrary step size. Alternatively, the designation screen may be provided with a numeric input field which is configured so that a user may directly input a value of the suntan level L or a value of the saturation conversion parameter k. Moreover, the method of determining the saturation conversion parameter k used by the parameter determination unit208is not limited to the present embodiment. Alternatively, the saturation conversion parameter k may be set up in an arbitrary range of numeric values by using a different conversion formula.

After the saturation conversion parameter k is determined by the parameter determination unit208, the saturation conversion unit209converts the saturation of the skin area302of the input image data301based on the saturation conversion parameter k.

Using any one of some saturation conversion table generation methods which will be explained below, the saturation conversion unit209generates a saturation conversion table and converts the saturation of the skin area302of the input image data301based on the generated saturation conversion table.

FIG. 8Ais a diagram showing an example of a saturation conversion table801which is generated based on the saturation conversion parameter k by the saturation conversion unit209. In the graph indicated inFIG. 8A, the lateral axis denotes the saturation before conversion and the vertical axis denotes the saturation after conversion.

In the example ofFIG. 8A, “Save” denotes an average value of the saturations S of all the pixels which constitute the skin area302, “Smin” denotes a minimum value of the saturation S, and “Smax” denotes a maximum value of the saturation S. The saturation conversion table801is determined based on a straight line with a gradient equal to the saturation conversion parameter k by assuming that the average saturation value “Save” is a saturation invariant point. The saturation conversion table801shown inFIG. 8Ais an example of the saturation conversion table801when the saturation conversion parameter k is equal to 1.5.

As shown inFIG. 8A, when the saturation conversion parameter k is greater than 1, the saturation after conversion corresponding to a high saturation area802indicated by a dotted line inFIG. 8Abecomes greater than a maximum saturation value (255) which is an upper limit of the saturation after conversion. To avoid this, as indicated by a solid line inFIG. 8A, the saturation conversion table801is set up so that the saturation after conversion in the high saturation area converges to the maximum saturation value. Similarly, when the saturation after conversion corresponding to a low saturation area is less than a minimum saturation value (0) which is a lower limit of the saturation after conversion, the saturation conversion table801is set up (not shown) so that the saturation after conversion corresponding to the low saturation area converges to 0.

FIG. 8Bis a diagram showing an example of a saturation conversion table803which is generated based on the saturation conversion parameter k by the saturation conversion unit209. In the graph indicated inFIG. 8B, the lateral axis denotes the saturation before conversion and the vertical axis denotes the saturation after conversion.

In the example ofFIG. 8B, “Save” denotes an average value of the saturations S of all the pixels which constitute the skin area302, “Smin” denotes a minimum value of the saturation S, and “Smax” denotes a maximum value of the saturation S. The saturation conversion table803is determined based on a straight line with a gradient equal to the saturation conversion parameter k by assuming that the minimum saturation value “Smin” is a saturation invariant point. The saturation conversion table803shown inFIG. 8Bis an example of the saturation conversion table803when the saturation conversion parameter k is equal to 1.5.

As shown inFIG. 8B, when the saturation conversion parameter k is greater than 1, the saturation after conversion corresponding to a high saturation area804indicated by a dotted line inFIG. 8Bbecomes greater than the maximum saturation value (255) which is the upper limit of the saturation after conversion. To avoid this, as indicated by a solid inner inFIG. 8B, the saturation conversion table803is set up so that the saturation after conversion in the high saturation area converges to the maximum saturation value. Similarly, when the saturation after conversion corresponding to the low saturation area is less than the minimum saturation value (0) which is the lower limit of the saturation after conversion, the saturation conversion table803is set up (not shown) so that the saturation after conversion corresponding to the low saturation area converges to 0.

As described above, the saturation conversion unit209is configured to generate a saturation conversion table based on a straight line with a gradient equal to the saturation conversion parameter k by assuming that the average saturation value “Save” or the minimum saturation value “Smin” is a saturation invariant point.

Regardless of whether the saturation invariant point is the average saturation value “Save” or the minimum saturation value “Smin”, the suntan level of the skin designated by the user may be expressed properly. If the minimum saturation value “Smin” is assumed to be the saturation invariant point, it becomes possible to vary the saturation uniformly from a highlight portion to a shadow portion of the skin area302. If the average saturation value “Save” is assumed to be the saturation invariant point, it becomes possible to suppress skin color fluctuation of a portion of the skin area302whose saturation is not high like a highlight portion and not low like a shadow portion.

Furthermore, the saturation conversion unit209may be configured to generate a saturation conversion table by the following method.

Assume that “Smin” and “Smax” denote a minimum saturation value and a maximum saturation value of the saturations S of all the pixels which constitute the skin area302, “Smin′” denotes a minimum saturation target value which is a conversion target value of the minimum saturation value “Smin”, and “Smax′” denotes a maximum saturation target value which is a conversion target value of the maximum saturation value “Smax”.

When the saturation conversion parameter k is less than 1 (k<1), the saturation conversion unit209computes a minimum saturation target value “Smin′” and a maximum saturation target value “Smax′” in accordance with the following formula (1).
k<1: Smin′=k′Smin,Smax′=k·Smax  (1)

When the saturation conversion parameter k is greater than or equal to 1 (k≥1), the saturation conversion unit209computes a minimum saturation target value “Smin′” and a maximum saturation target value “Smax′” in accordance with the following formula (2).
k≥1:Smin′=Smin/k, Smax′=k·Smax  (2)

FIGS. 9A and 9Bare diagrams showing examples of saturation conversion tables901and902which are generated based on the minimum saturation target value “Smin′” and the maximum saturation target value “Smax′” which are generated based on the minimum saturation value “Smin”, the maximum saturation value “Smax”, and the above formulas (1) and (2) by the saturation conversion unit209.FIG. 9Ashows the saturation conversion table901generated when the saturation conversion parameter k is less than 1, andFIG. 9Bshows the saturation conversion table902generated when the saturation conversion parameter k is greater than or equal to 1.

As shown inFIGS. 9A and 9B, in the coordinates in which the lateral axis denotes the saturation before conversion and the vertical axis denotes the saturation after conversion, the saturation conversion unit209is configured to compute an approximate curve of a quadratic function based on four points consisting of a minimum saturation point S1(Smin, Smin′), a maximum saturation point S2(Smax, Smax′), an achromatic point S0(0, 0), and a highest saturation point5255(255,255), and generate a saturation conversion table correlated to the skin area302based on the computed approximate curve.

The saturation conversion table901shown inFIG. 9Ais an example of the saturation conversion table when the user designates the suntan level (k<1) to cause the color of the skin area302to approach white skin, and the curve of the saturation conversion table901has a downwardly convex shape.

FIG. 10Ashows a change of a saturation distribution by the saturation conversion in this case. InFIG. 10A, a solid line denotes the saturation histogram information before conversion, and a dotted line denotes the saturation histogram information after conversion. As shown inFIG. 10A, when the user designates the suntan level to cause the color of the skin area302to approach white skin (k<1), the saturation conversion lowers the overall saturation distribution.

Moreover, the saturation conversion table902shown inFIG. 9Bis an example of the saturation conversion table when the user designates the suntan level to cause the color of the skin area302to approach brown skin (k≥1), and the curve of the saturation conversion table902has an upwardly convex shape.

FIG. 10Bshows a change of a saturation, distribution by the saturation conversion in this case. InFIG. 10A, a solid line denotes the saturation histogram information before conversion, and a dotted line denotes the saturation histogram information after conversion. As shown inFIG. 10B, when the user designates the suntan level to cause the color of the skin area302to approach brown skin (k≥1), the saturation distribution greatly changes in the higher saturation direction and slightly extends in the lower saturation direction. By this saturation conversion, the saturation on the side of higher saturation may improve, the width of the overall saturation distribution may be extended, and the effect of emphasizing the contrast of the low saturation portion and the high saturation portion may be obtained.

The saturation conversion unit209is configured to convert the saturations S of all the pixels which constitute the skin area302based on the saturation conversion table generated by using any one of the above-described methods. After the saturation conversion of the skin area302is performed and the image data702after the saturation conversion is generated by the saturation conversion unit209, the display input unit210displays the image data702after the saturation conversion on the display unit107(at the upper-row right-hand portion of the screen shown inFIG. 7).

The user checks that the image data702after the saturation conversion displayed on the display unit107is a final image, and determines or adjusts again the suntan level of the skin. Hence, it is possible for the user to convert the color of the skin area302into a desired suntan condition by a simple operation of moving the slide bar703in the screen shown inFIG. 7to designate the suntan level.

<Flow of Image Processing>

FIG. 11is a flowchart for explaining an image processing process according to the first embodiment.

As shown inFIG. 11, when image data indicating a portrait image (including a person) which is an object of the image processing is input to the image processing apparatus100, the skin area extraction unit201extracts the skin area302from the input image data301at step S1. Next, at step S2, the skin color information acquisition unit202acquires the skin color information of all the pixels which constitute the extracted skin area302.

At step S3, the chromaticity computation unit203computes the chromaticity of all the pixels which constitute the skin area302based on the skin color information, and acquires the hue histogram information401and the saturation histogram information402.

At step S4, the target information acquisition unit205acquires the target hue histogram information as the target information501from the storage unit204.

At step S5, the hue conversion unit206converts the hues of all the pixels which constitute the skin area302based on the hue conversion table603which is generated based on the hue histogram information401and the target information501.

At step S6, the display input unit210is controlled to display on the display unit107the input image data301and the slide bar703for designating a suntan level, and the reception unit207receives a user's designation of the suntan level via the slide bar703.

At step S8, the saturation conversion unit209converts the saturations of all the pixels which constitute the skin area302based on the saturation conversion table which is generated based on the saturation conversion parameter.

It is desirable that the conversion of the saturations of the skin area302is performed by the saturation conversion unit209after the hues thereof are converted by the hue conversion unit206. Generally, in many cases, the color tone of the skin area302of the input image data301which is obtained with a digital camera or the like may deviate from a desired skin color due to the influences of reflected light rays from a light source and the environment thereof at a time of photographing. If the conversion of the saturations is performed without performing the conversion of the hues, the initial hues of the skin area302may remain unchanged and a sense of incongruity in the color tone of the skin area may be left.

Thus, it is desirable that the hue conversion is performed before the saturation conversion of the skin area302so that the color tone of the skin area302is converted to a desired skin color. Performing the hue conversion in advance will enable the user to easily imagine a final target image.

At step S9, the display input unit210is controlled to display, on the display unit107, the image data702after the saturation conversion on which the saturation conversion process has been performed by the saturation conversion unit209, as a final image. When the suntan level is determined by the user who has visually checked the image data702after the saturation conversion, the image processing apparatus100outputs the image data which is obtained by the image processing, and terminates the image processing process. When the user wishes to redo the adjustment of the suntan level, the image processing process starting from step S6may be performed again.

In the foregoing embodiment, each of the above-described functions of the image processing apparatus100according to the present invention may be implemented by executing the program, read from the main memory unit102(or the ROM) and loaded to the main memory unit102(or the RAM), by the processor (or the CPU) of the image processing apparatus100. The program executed by the processor of the image processing apparatus100is configured to include multiple modules including the program for implementing each of the skin area extraction unit201, the skin color information acquisition unit202, the chromaticity computation unit203, the target information acquisition unit205, the hue conversion unit206, the parameter determination unit208, the saturation conversion unit209, etc.

Moreover, the program executed by the processor of the image processing apparatus100according to the foregoing embodiment may be stored in an installable-form or executable-form file on a computer readable recording medium, such as CD-ROM, FD, CD-R, DVD, etc., and the computer readable recording medium storing the program may be offered.

Furthermore, the program executed by the processor of the image processing apparatus100according to the foregoing embodiment may be stored on a computer connected to a network, such as the Internet, and the stored program may be downloaded to another computer via the network. In addition, the program executed by the processor of the image processing apparatus100according to the foregoing embodiment may also be offered or distributed via the network, such as the Internet.

As described above, the image processing apparatus100according to the first embodiment extracts the skin area302from the input image data301which is input by the user, and converts the hue of the skin area302based on the target information so that a desired skin color may be reproduced. Moreover, the image processing apparatus100according to the first embodiment converts the saturation of the skin area302based on the suntan level of the skin designated by the user, so that the user's desired skin color may be reproduced with ease. Thus, even if the user is unfamiliar with image processing, it is possible for the user to convert a skin color of a portrait image into a desired skin color without fail by simple operation. It is possible to reproduce a desired skin color expression with ease.

The image processing apparatus100according to the foregoing embodiment may be applied to various apparatuses, such as MFPs (multifunction peripherals), copiers, printers, facsimile machines, scanners, digital cameras, PCs (personal computers), etc.

Second Embodiment

Next, a second embodiment of the present invention will be described with reference to the accompanying drawings. In the following, the composition and processing of an image processing apparatus according to the second embodiment which are essentially the same as those of the image processing apparatus100according to the first embodiment may be omitted.

FIG. 12shows a configuration of an image processing system1according to the second embodiment.

As shown inFIG. 12, the image processing system1includes MFPs (multifunction peripherals)10and20, image processing servers30and40, and an information processing terminal (e.g., a PC (personal computer))50which are interconnected via a network.

The MFPs10and20are examples of an image input apparatus. Each of the MFPs10and20is a multifunction peripheral having multiple functions including a printer function, a scanner function, a copier function, a facsimile function, etc., which are incorporated in a single housing. The MFP10generates image data from a paper medium by scanning the paper medium using the scanner function and transmits the image data to any of the image processing servers30and40and the information processing terminal50. Moreover, the MFP10receives image data and prints the image data on a printing medium. Other examples of the image input apparatus which is configured to input image data may include scanners, facsimile machines, copiers, etc.

The image processing servers30and40are implemented by computers, such as workstations, which receive the image data scanned and generated by the MFPs10and20, or the image data transmitted from the information processing terminal50, and perform various image processing functions. Each of the image processing servers30and40functions as an image processing apparatus which performs image processing on the input image data.

Alternatively, a part or all of functions provided by the image processing server30or40as the image processing apparatus may be installed in any of the MFPs10and20and the information processing terminal50. Alternatively, the number of MFPs, the number of image processing servers, and the number of information processing terminals which are included in the image processing system1and interconnected via the network may be arbitrary.

FIG. 13shows a hardware configuration of the MFP10. It should be noted that the MFP20has a hardware configuration which is the same as that of the MFP10and a description of the hardware configuration of the MFP20will be omitted.

As shown inFIG. 13, the MFP10includes a control unit11, a main memory unit12, a secondary memory unit13, a recording medium interface unit14, a network interface unit15, a reading unit16, an operation unit17, and an engine unit18, which are interconnected by a bus B.

The control unit11may include a CPU which performs control of the respective units of the MFP10and performs computation and processing of data. The control unit11may include a processor which executes a program stored in the main memory unit12, and the processor receives data from an input device or a storage unit, performs computation and processing of the data, and outputs the processed data to an output unit or a storage unit.

The main memory unit12may include a ROM (read only memory), a RAM (random access memory), etc. In the main memory unit12, an OS (operating system) as the basic software executed by the control unit11, application programs and data are stored or temporarily retained.

The secondary memory unit13may include a HDD (hard disk drive) or the like. In the secondary memory unit13, data relevant to the application programs and others are stored.

The recording medium interface unit14provides an interface between a recording medium19, such as a flash memory, and the MFP10. For example, by using a data transmission line, such as USB (universal serial bus), the recording medium interface unit14is connected to the recording medium19.

A predetermined program is stored in the recording medium19, and the recording medium19is attached to the MFP10. The predetermined program stored in the recording medium19is installed in the main memory unit12of the MFP1through the recording medium interface unit14. After the installation, the predetermined program is read from the main memory unit12and executed by the control unit11of the MFP10.

The network interface unit15provides an interface between a peripheral device and the MFP10, the peripheral device having a communication function and being connected to the MFP10via a wired or wireless network which is constructed of data transmission lines, such as a LAN (local area network) or WAN (wide area network).

The reading unit16may include a scanner device which reads an image by scanning a paper medium and acquires image data from the read image.

The operation unit17may include key switches (composed of hard keys) and a LCD (liquid crystal display) having a touch panel function including software keys of a GUI (graphical user interface). The operation unit17may include a display unit and/or an input unit which functions as a UI (user interface) for a user to perform various, setting processes when using functions of the MFP10.

The engine unit18may include a mechanical image formation unit, such as a plotter, which performs an image formation process.

FIG. 14shows a hardware configuration of the image processing server30. It should be noted that the image processing server40has a hardware configuration which is the same as that of the image processing server30and a description of the hardware configuration of the image processing server40will be omitted.

As shown inFIG. 14, the image processing server30includes a control unit31, a main memory unit32, a secondary memory unit33, a recording medium interface unit34, and a network interface unit35.

The control unit31may include a CPU which performs control of the respective units of the image processing server and performs computation and processing of data. The control unit31may include a processor which executes a program stored in the main memory unit32, and the processor receives data from an input device or a storage unit, performs computation and processing of the data, and outputs the processed data to an output unit or a storage unit.

The main memory unit32may include a ROM (read only memory), a RAM (random access memory), etc. In the main memory unit32, an OS (operating system) as the basic software executed by the control unit31, application programs, and data are stored or temporarily retained.

The secondary memory unit33may include a HDD (hard disk drive) or the like. In the secondary memory unit33, data relevant to the application programs and others are stored.

The recording medium interface unit34provides an interface between a recording medium19, such as a flash memory, and the image processing server30. For example, by using a data transmission line, such as USB (universal serial bus), the recording medium interface unit34is connected to the recording medium19.

A predetermined program is stored in the recording medium19, and the recording medium19is attached to the recording medium interface unit34of the image processing server30. The predetermined program stored in the recording medium19is installed in the main memory unit32through the recording medium interface unit34. After the installation, the predetermined program is read from the main memory unit32and executed by the control unit31of the image processing server30.

The network interface unit35provides an interface between a peripheral device and the image processing server30, the peripheral device having a communication function and connected to the image processing server30via a wired or wireless network which is constructed of data transmission lines, such as LAN (local area network) or WAN (wide area network).

In this embodiment, the image processing server30shown inFIG. 14does not include an operation unit including a keyboard or the like, nor a display unit including a LCD or the like. Alternatively, the operation unit and the display unit may be included in the image processing server30.

The information processing terminal50has a hardware configuration which is the same as that of the image processing apparatus100according to the first embodiment shown inFIG. 1.

<Functions of Image Processing System>

FIG. 15shows a functional configuration of the MFP10, the image processing server30, and the information processing terminal50in the image processing system1according to the second embodiment. A description will be given of image processing performed by the image processing system1with reference toFIG. 15.

Although not illustrated inFIG. 15, the MFP20and the image processing server40have the same functional configuration as the MFP10and the image processing server30, respectively. In the image processing which will be explained below, the MFP20and the image processing server40may be used instead of the MFP10and the image processing server30.

As shown inFIG. 15, the MFP10includes the reading unit16, a communication unit21, and the engine unit18.

The reading unit16may read an image by scanning a paper medium and acquire image data on which the image processing is to be performed. The communication unit21may transmit and receive image data between the image processing server30and the information processing terminal50. The engine unit18may print and output the image data on which the image processing is performed by the image processing server30, to a printing medium, such as a recording sheet.

The information processing terminal50includes a storage unit55, a reading unit52, a communication unit51, a display control unit53, and a display unit54.

The storage unit55may store image data, the target information501, etc. The reading unit52may read the image data and the target information501from the storage unit55. The communication unit51may transmit and receive image data between the MFP10and the image processing server30. The display control unit53may display the image data received by the communication unit51on the display unit54. The display control unit53may display the image data stored in the information processing terminal50on the display unit54. The display unit54may include any of an LCD (liquid crystal display), an organic EL (electroluminescence) display, etc. Images, operational icons, etc., are displayed on the display unit54.

The image processing server30includes a communication unit36, a skin area extraction unit37, a skin color information acquisition unit38, a chromaticity computation unit39, a hue conversion unit41, a saturation conversion unit42, a target information acquisition unit43, and a parameter determination unit44.

The communication unit36may transmit and receive image data between the MFP10and the information processing terminal50. The functions of the respective units37-39and41-44of the image processing server30other than the communication unit36are the same as those of the corresponding units of the image processing apparatus100according to the first embodiment, and a description thereof will be omitted.

In the above-described image processing system1, the user may use the reading unit16of the MFP10, acquire the image data on which the image processing is to be performed, send the image data to the image processing server30, and request the image processing server30to perform the image processing. Or the user may read the image data on which the image processing is to be performed from the storage unit55of the information processing terminal50, send the image data to the image processing server30, and request the image processing server30to perform the image processing.

In the image processing server30, the skin area extraction unit37may extract the skin area302of the input image data301. The skin color information acquisition unit38may acquire the skin color information of the skin area302. The chromaticity computation unit39may compute the chromaticity including at least the hue and the saturation based on the skin color information, and generate the distribution information of the chromaticity. The hue conversion unit41may convert the hues of the skin area302based on the target, information which is acquired by the target information acquisition unit43. The saturation conversion unit42may convert the saturations of the skin area302based on the saturation conversion parameter determined by the parameter determination unit44.

The MFP10may receive the image data on which the image processing is performed by the image processing server30, and cause the engine unit18to print the received image data. Or the information processing terminal50may receive the image data on which the image processing is performed by the image processing server30, and cause the display control unit53to display the received image data on the display unit54. Hence, the user is able to get an output image in which a desired skin color expression is reproduced by requesting the MFP10to print it or requesting the information processing terminal50to display it.

As described above, in the image processing system1according to the second embodiment, the user is able to acquire image data on which the image processing is to be performed, by means of the MFP10, and perform the image processing on the image data by means of the image processing server30or the information processing terminal50.

Alternatively, a part of the functions of the image processing server30may be incorporated in the MFP10or the information processing terminal50so that the image processing may be carried out by the image processing server30and at least one of the MFP10and the information processing terminal50.

According to the above-described embodiments, the skin area302is extracted from the input image data301input by the user and the hues of the skin area302are converted based on the target information501so that a desired skin color expression may be reproduced. Furthermore, the saturations of the skin area302are converted based on the suntan level of the skin designated by the user, so that a user's desired skin color expression may be reproduced. What has to be performed by the user is just to designate the suntan level of skin, and even if the user is unfamiliar with image processing, it is possible for the user to convert a skin color of a portrait image into a desired skin color without fail by simple operations. It is possible to reproduce a desired skin color expression with ease.

The image processing apparatus according to the invention is not limited to the above-described embodiments and various variations and modifications may be made without departing from the scope of the present invention.

The present application is based on and claims the benefit of priority of Japanese Patent Application No. 2014-025594, filed on Feb. 13, 2014, the entire contents of which are hereby incorporated by reference.

DESCRIPTION OF REFERENCE NUMERALS

PRIOR ART DOCUMENTS

Patent Documents