The image processing apparatus of the present invention is an image processing apparatus that generates gloss intensity distribution image data indicating a gloss intensity distribution based on captured image data obtained by capturing an image of a subject having concavity/convexity irradiated by a surface light source. The image processing apparatus sets an area, in which specularly reflected light from the surface light source enters a planar mirror and which is the same size as or smaller than that of an area captured by an image capturing apparatus in a case where the planar mirror is arranged at substantially the same position as that of the subject, as a measurement area of the gloss intensity distribution based on surface light source position information indicating the position of the surface light source and image capturing position information indicating the position of the image capturing apparatus that captures an image of the subject. Further, based on the surface light source position information, the image capturing position information, and pixel values of the captured image data, the gloss intensity distribution image data in the measurement area is generated.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to image processing to acquire a gloss intensity distribution of a subject from captured image data obtained by capturing an image of the subject having concavity/convexity.

Description of the Related Art

In recent years, the technique to acquire information other than color information from a subject having concavity/convexity is spreading. Information acquired from a subject having concavity/convexity is wide-ranging and includes information indicating a distance from an image capturing position to a subject, information indicating the shape of concavity/convexity of a subject, information indicating reflection characteristics of a subject and the like. In the case where information indicating reflection characteristics from a subject having concavity/convexity, it is necessary to adjust the arrangement of a light source and a light-receiving element in order to acquire a two-dimensional distribution of reflected light intensity (hereinafter, described as “gloss intensity distribution”) because the specular reflection direction changes.

The more complicated the concavity/convexity shape, the more complicatedly the specular reflection direction changes, and therefore, it was difficult to adjust the arrangement of a light source and a light-receiving element in order to acquire a high-accuracy gloss intensity distribution. Japanese Patent Laid-Open No. 2005-227197 has disclosed a reflected light measuring method of acquiring a gloss intensity distribution of a subject while reducing a measurement error accompanying a change in the specular reflection direction by irradiating a subject having a concavity/convexity shape by using a surface light source.

SUMMARY OF THE INVENTION

The image processing apparatus of the present invention is an image processing apparatus that generates gloss intensity distribution image data indicating a gloss intensity distribution based on captured image data obtained by capturing an image of a subject having concavity/convexity irradiated by a surface light source, and includes: a setting unit configured to set an area, in which specularly reflected light from the surface light source enters a planar mirror and which is the same size as or smaller than that of an area captured by an image capturing apparatus in a case where the planar mirror is arranged at substantially the same position as that of the subject, as a measurement area of the gloss intensity distribution based on surface light source position information indicating the position of the surface light source and image capturing position information indicating the position of the image capturing apparatus that captures an image of the subject; and a generation unit configured to generate the gloss intensity distribution image data in the measurement area based on the surface light source position information, the image capturing position information, and pixel values of the captured image data.

DESCRIPTION OF THE EMBODIMENTS

With the reflected light measuring method described in Japanese Patent Laid-Open No. 2005-227197, reflected light from a subject is measured by using a special optical system called a telecentric optical system, and therefore, it is difficult to increase the size of the surface light source or to arrange the surface light source in the vicinity of a subject. That is, it is not possible to widen the area of irradiation using the surface light source in order to measure a gloss intensity distribution. Because of this, a very long time is required to acquire information indicating a gloss intensity distribution from a subject having concavity/convexity.

First Embodiment

In the following, embodiments for embodying the present invention are explained with reference to the drawings. However, components described in these embodiments are merely exemplary and are not intended to limit the scope of the present invention to those.

FIG. 1is an outline diagram of an image processing system1of the present embodiment. An oil painting having concavity/convexity formed by touches of a brush at the time of attaching color materials and a gloss due to the varnish applied onto the surface of the concavity/convexity as shown inFIG. 1is taken to be a subject100. In the present embodiment, a gloss intensity distribution is acquired from the subject100such as this. In the image processing system1shown inFIG. 1, the center on the substantially flat plane of the surface of the subject (oil painting)100is taken to be O. In the three-dimensional space coordinate system with O as the origin, an image capturing apparatus201that captures an image of the object100is arranged at a point C and a surface light source202having a size of a length of 1 and a width of 1 is arranged at a point L. Further, the image capturing apparatus201is arranged so as to face toward the direction of a vector nC=(−1/√2, 0, −1/√2) indicating the direction of the optical axis. The surface light source202is arranged so as to face toward the direction of a vector nL=(0, 0, −1) indicating the direction of the plane normal line. In the present embodiment, the surface light source202includes a liquid crystal display that displays a white screen. The luminance of light emitted from the white screen of the liquid crystal display is uniform within the screen and the light is diffused light that diffuses isotropically.

FIG. 2is a hardware configuration diagram of the image processing system1in the present embodiment. The image capturing apparatus201includes a lens, an aperture stop, a shutter, an optical low-pass filter, a color filter, and a sensor such as a CMOS sensor and a CCD sensor. The image capturing apparatus201detects the amount of light of a subject, performs A/D conversion for the detected amount of light, and outputs digital data to a bus216, which is a data transfer path. The surface light source202and a flash203irradiate the subject100with light. A ROM204and a RAM205provide programs, data, a work area and the like necessary for image capturing and image processing to a CPU206. The CPU206executes programs stored in the ROM204and the RAM205by using the RAM205as a work memory and controls each function block via the bus216. By the CPU206controlling each function block, various kinds of processing, to be described later, are performed. An image capturing control unit207performs control of the image capturing system specified by the CPU206, such as focusing, shutter opening, and aperture stop adjustment. An operation unit208includes a button, a mode dial and the like and receives instructions input by a user via those. A CG generation unit209generates characters, graphics and the like by CG (Computer Graphics). As a display unit210, generally, a liquid crystal display is used widely and the display unit210displays images and characters received from the CG generation unit209, a digital signal processing unit211, and an image processing unit215, to be described later. Further, the display unit210may have a touch screen function and in this case, it is possible to handle user instructions whose input has been received via the display unit210as an input to the operation unit208. The digital signal processing unit211performs white balance processing, gamma processing, noise reduction processing and the like for the digital data received from the image capturing apparatus201and generates digital image data. A compression/decompression unit212converts digital image data generated by the digital signal processing unit211into a predetermined file format, such as jpeg and mpeg. An external interface213(hereinafter, described as “I/F”) is an interface to connect a PC and other media214(e.g., hard disk, memory card, CF card, SD card, USB memory and the like) and the image processing system1. The image processing unit215generates new image data by using the digital data received from the image capturing apparatus201or the digital image data generated by the digital signal processing unit211and outputs the generated new image data to the bus216. The components in the image processing system1exist other than those described above, but they are not the main purpose of the present embodiment, and therefore, explanation is omitted.

FIG. 3is a block diagram showing a software function configuration of the image processing unit215in the present embodiment. A processing procedure that is performed by image processing application software of the present embodiment based on instructions from the CPU206is explained with reference toFIG. 3.

A captured image data input reception unit305receives an input of data of a captured image captured by the image capturing apparatus201under the surface light source202via an input terminal301. The captured image data whose input has been received is further output to a measurement area setting unit308and a gloss intensity distribution image data generation unit311. A surface light source position information input reception unit306receives an input of data and a signal indicating the surface light source position information at the time of image capturing via an input terminal302. The surface light source position information whose input has been received is further output to the measurement area setting unit308and a correspondence information calculation unit309. An image capturing position information input reception unit307receives an input of data and a signal indicating the position information on the image capturing apparatus201via an input terminal303. The image capturing position information whose input has been received is further output to the measurement area setting unit308and the correspondence information calculation unit309.

The measurement area setting unit308sets a measurement area that is a target of measurement of a gloss intensity distribution of the subject100from the captured image data, the surface light source position information, and the image capturing position information. The measurement area information indicating the measurement area is output to the gloss intensity distribution image data generation unit311.

The correspondence information calculation unit309calculates information indicating correspondence relationship between pixel value and gloss intensity in which pixel values and reference gloss intensities are associated with each other under various image capturing conditions from the surface light source position information, the image capturing position information, and the reference gloss intensity and stores the information indicating correspondence relationship between pixel value and gloss intensity in a correspondence information holding unit310. Further, in response to a request from the gloss intensity distribution image data generation unit311, the information indicating correspondence relationship between pixel value and gloss intensity is output to the gloss intensity distribution image data generation unit311as data or a signal indicating the information. The gloss intensity distribution image data generation unit311generates gloss intensity distribution image data indicating a gloss intensity distribution from the captured image data, the measurement area information, and the information indicating correspondence relationship between pixel value and gloss intensity. The generated gloss intensity distribution image data is output from an output terminal312.

(Operation of Image Processing Unit)

Next, the processing procedure in the image processing unit215of the present embodiment is explained with reference to a flowchart inFIG. 4. The processing by the flowchart shown inFIG. 4is performed by the CPU206loading the program code stored in the ROM204onto the RAM205and executing the program code. This is also true with the processing inFIG. 5. Symbol S described hereinafter means that the step is a step in the flowchart.

At S401, the captured image data input reception unit305receives an input of data of a captured image captured by the image capturing apparatus201under the surface light source202. The captured image data whose input has been received is stored in the storage area, such as the RAM205.

At S402, the surface light source position information input reception unit306receives an input of the surface light source position information at the time of image capturing. In the image processing system1of the present embodiment, as shown inFIG. 1, at the position of the point L (−1, 0, 1), the liquid crystal display having a size of a length of 1 and a width of 1 is arranged so as to face toward the direction of nL=(1, 0, −1). The surface light source position information whose input has been received is stored in the storage area, such as the RAM205.

At S403, the image capturing position information input reception unit307receives an input of the position information on the image capturing apparatus201. In the image processing system1of the present embodiment, as shown inFIG. 1, at the position of the point C (1, 0, 1), the image capturing apparatus201is arranged so as to face toward the direction of nC=(−1/√2, 0, −1/√2). The image capturing position information whose input has been received is stored in the storage area, such as the RAM205.

At S404, the measurement area setting unit308sets a measurement area that is a target of measurement of a gloss intensity distribution of the subject100from the captured image data, the surface light source position information, and the image capturing position information. Details of the processing to set a measurement area will be described later with reference toFIG. 5. The measurement area information indicating the measurement area is stored in the storage area, such as the RAM205.

At S405, the correspondence information calculation unit309calculates information indicating correspondence relationship between pixel value and gloss intensity under the image capturing condition specified by the surface light source position information and the image capturing position information. Here, a method by which the correspondence information calculation unit309acquires information indicating correspondence relationship between pixel value and gloss intensity under various image capturing conditions is explained.

In the present embodiment, to the entire surface of an oil painting used as the subject100, varnish is applied. Because of this, it is possible to regard that the refractive index is fixed across the entire surface of the oil painting. In the present embodiment, it is assumed that, for example, a refractive index η=1.3 across the entire surface of the oil painting. The refractive index of the oil painting surface is fixed, and therefore, it is possible to consider that the reflection characteristics of the oil painting surface caused by the shine of varnish depend on the surface roughness of the oil painting surface.

In the present embodiment, in order to represent such the reflection characteristics as described above, a BRDF (Bidirectional Reflectance Distribution Function) model is used. In the BRDF model expressed by expression (1), a Cook-Torrance model representing the surface roughness by a distribution of minute plane normal lines directed in various directions is further used.

In expression (1), D is a Beckmann distribution indicating a distribution of minute plane normal lines representing the surface roughness. The Beckmann distribution D is expressed by expression (2) below.

In expression (2), m is a parameter indicating the surface roughness and indicates an average inclination of the minute planes. Here, θNHis an angle formed by a half vector H, which is a resultant vector of a light source direction vector and an image capturing direction vector, and a macro plane normal line direction vector N of the distribution of minute plane normal lines directed in various directions. A schematic diagram explaining θNH, the half vector H, and the plane normal line direction vector N is shown inFIG. 6A. In the present embodiment, the correspondence information calculation unit309sets N=(0, 0, 1) by regarding the subject100as being a substantially flat plane at the time of acquiring information indicating correspondence relationship between pixel value and gloss intensity.

As expressed by expression (3) below, the Beckmann distribution D is normalized so that 1 is obtained in the case where the Beckmann distribution D is integrated in an upper hemispheric area ΩNwith the direction of the macro plane normal line direction vector N being taken to be the direction toward the zenith.

In expression (1), F is the Fresnel term representing the reflectance in the case where light enters the interface between substances with different refractive indexes. As described previously, the subject100of the present embodiment is an oil painting to the entire surface of which, varnish is applied, and the refractive index η is regarded as being 1.3 across the entire surface of the oil painting. Because of this, under the image capturing condition of the present embodiment, the intensity of the Fresnel reflection in an angle area Ωsurface light sourceentered by the light from the surface light source202is fixed and it is possible to regard the Fresnel term F as being a fixed value F0. Here, the angle area entered by the light from the surface light source is an area defined by the angle formed by light source direction vectors from the origin O to the left and right ends A and B of the surface light source L in the case where explanation is given by takingFIG. 1as an example. Hereinafter, in the present embodiment, the angle area entered by the light from the surface light source is described as the angle area Ωsurface light source. A schematic diagram explaining a relationship by which the Fresnel term F corresponding to the angle area Ωsurface light sourcebecomes the fixed value F0under the image capturing condition of the present embodiment is shown inFIG. 6B.

In expression (1), G is a geometric attenuation term indicating attenuation of the reflected light due to self light shielding/self shading caused by the concavity/convexity portion of a minute plane. The geometric attenuation term has characteristics that the geometric attenuation term indicates a fixed value in the case where a light source direction θ is within a predetermined angle range. In the present embodiment, on the premise that the angle area Ωsurface light sourceentered by the light from the surface light source202is within the above-described predetermined angle range, it is possible to regard the geometric attenuation term G as being a fixed value G0(G0=1). A schematic diagram explaining a relationship by which the geometric attenuation term G corresponding to the angle area Ωsurface light sourcebecomes the fixed value G0(G0=1) under the image capturing condition of the present embodiment is shown inFIG. 6C.

In expression (1), ncis the vector indicating the direction of the image capturing apparatus201having received the input at S403, and in the present embodiment, nc=(−1/√2, 0, −1/√2). The vector ncsuch as this that indicates the direction of the image capturing apparatus201can be said to be one aspect of image capturing position information. As described above, in the present embodiment, the reflectance characteristics of the subject100are acquired based on the BRDF model, but any model may be used as long as the model can represent the reflectance characteristics of the subject100in accordance with the surface roughness.

As explained above, in the present embodiment, it is possible to regard the Fresnel term F, the geometric attenuation term G, the image capturing apparatus direction vector nc, and the macro plane normal line direction vector as being fixed. Consequently, it is possible to represent the pixel value for each pixel position in the data of the captured image captured under the surface light source by the value (BRDF integrated value) obtained by integrating BRDF in the angle area entered by the light from the surface light source and further, the BRDF integrated value is proportional to the value obtained by integrating the Beckmann distribution D. The value obtained by integrating the Beckmann distribution D in the angle area Ωsurface light sourceis expressed by expression (4) below.

As described above, the Beckmann distribution D is normalized so that 1 is obtained in the case where the Beckmann distribution D is integrated in the upper hemispheric area ΩNwith the direction of the macro plane normal line direction vector N being taken to be the direction toward the zenith. In the present embodiment, the surface light source is used, and therefore, the angle area Ωsurface light sourceentered by the light from the surface light source is smaller than the upper hemispheric area ΩNwithout exception. At this time, in the case where the surface smooth degree of the subject100is high, the integrated value of the Beckmann distribution D in the angle area Ωsurface light sourcebecomes close to 1. On the other hand, in the case where the surface smooth degree of the subject100is low, the integrated value of the Beckmann distribution D in the angle area Ωsurface light sourcebecomes a value smaller than that in the case where the surface smoothness degree of the subject100is high. A relationship between the integrated values of the Beckmann distribution D is expressed by expression (5) below.

In expression (5), D (θNH, mA) indicates the Beckmann distribution in the case where the surface smoothness degree of the subject100is high. On the other hand, in expression (5) D (θNH, mB) indicates the Beckmann distribution in the case where the surface smoothness degree of the subject100is low (e.g., the surface is coarse).

Here, as described above, in the present embodiment, it is possible to regard the Fresnel term F0, the geometric attenuation term G0, the cameral direction nc, and the macro plane normal line direction N as being fixed, respectively. Because of this, it is possible to represent a gloss intensity IMAXcorresponding to the pixel position of the captured image data by BRDF in the case where the Beckmann distribution D becomes the maximum. A relationship between the gloss intensity IMAXand the maximum value of the Beckmann distribution D is expressed by expression (6) below.

DMAX(θNH, m) indicates the maximum value of the Beckmann distribution D in the case where the surface roughness is m. As described previously, the maximum value DMAX(θNH, m) of the Beckmann distribution D becomes a larger value for the higher surface smoothness degree of the subject100. The maximum value DMAX(θNH, m) of the Beckmann distribution D in accordance with the surface smoothness degree of the subject100is expressed by expression (7) below.
DMAX(θNH,mB)<DMAX(θNH,mA):  expression (7)

As shown inFIG. 7, DMAX(θNH, mA) corresponds to a curve A and indicates the maximum value of the Beckmann distribution D in the case where the surface smoothness degree of the subject100is high. On the other hand, DMAX(θNH, mB) corresponds to a curve B and indicates the maximum value of the Beckmann distribution D in the case where the surface smoothness degree of the subject100is low.

In the present embodiment, the information indicating correspondence relationship between pixel value and gloss intensity under the various image capturing conditions is calculated by a simulation by the correspondence information calculation unit309and stored in advance in the correspondence information holding unit310. In the following, the calculation method of information indicating correspondence relationship between pixel value and gloss intensity by the correspondence information calculation unit309is explained.

In the present embodiment, it is possible for the correspondence information calculation unit309to calculate the information indicating correspondence relationship between pixel value and gloss intensity by using expressions (2) and (6) described above. As described previously, it is possible to regard the Fresnel term F, the geometric attenuation term G, the image capturing apparatus direction vector nc, and the macro plane normal line direction vector N as being fixed under a certain image capturing condition. At this time, in expressions (2) and (6), IMAXand m are unknown variables. The correspondence information calculation unit309inputs a reference gloss intensity into the unknown variable IMAXand calculates the value of m corresponding to each value of IMAX.

Subsequently, it is possible for the correspondence information calculation unit309to calculate the BRDF integrated value corresponding to the value of IMAXby inputting the calculated m into expressions (2) and (4). In the present embodiment, the correspondence information calculation unit309maps the BRDF integrated value corresponding to the value of IMAXto an 8-bit value (0 to 255). As described above, it is possible for the correspondence information calculation unit309to calculate the corresponding BRDF integrated value from the value (reference gloss intensity) of IMAXwhose input has been received and the correspondence information calculation unit309stores the correspondence information in the correspondence information holding unit310as the information indicating correspondence relationship between pixel value and gloss intensity.

FIG. 8AtoFIG. 8Care each a diagram showing an example in which information indicating correspondence relationship between pixel value and gloss intensity is calculated by a simulation under the image capturing condition shown inFIG. 1. InFIG. 8AtoFIG. 8C, the correspondence information calculation unit309inputs values of 0 to 0.15 to IMAXand calculates the BRDF integrated values corresponding to IMAX: 0 to 0.15, respectively. Reference numerals801to803indicate virtual samples having the gloss intensities IMAXof 0.05, 0.10, and 1.05, respectively. As explained above, it is possible for the correspondence information calculation unit309to obtain the BRDF integrated value corresponding to the value of IMAXby calculating the values of m corresponding to the gloss intensities IMAX, respectively, and based on the calculated values of m. In the examples inFIG. 8AtoFIG. 8C, the corresponding BRDF integrated values 159, 206, and 222 are calculated for the gloss intensities 0.05, 0.10, and 0.15, respectively.

The calculation of the information indicating correspondence relationship between pixel value and gloss intensity by the simulation explained inFIG. 8AtoFIG. 8Cis performed for each image capturing condition.FIG. 9AtoFIG. 9Care each a diagram showing an example of the different image capturing condition in the present embodiment andFIG. 9DtoFIG. 9Fare each a diagram showing an example of the information indicating correspondence relationship between pixel value and gloss intensity corresponding toFIG. 9AtoFIG. 9C, respectively. Among these, the image capturing condition explained inFIG. 8AtoFIG. 8Ccorresponds toFIG. 9Band the information indicating correspondence relationship between pixel value and gloss intensity explained inFIG. 8AtoFIG. 8Ccorresponds toFIG. 9E, respectively. As described above, it is possible for the correspondence information calculation unit309of the present embodiment to calculate in advance the information indicating correspondence relationship between pixel value and gloss intensity for each image capturing condition by a simulation and to store the information indicating correspondence relationship between pixel value and gloss intensity in the correspondence information holding unit310. The correspondence information calculation unit309selects the image capturing condition closest to a combination of the surface light source position information whose input has been received at S402and the image capturing position information whose input has been received as S403. Then, in response to the request from the gloss intensity distribution image data generation unit311, the correspondence information calculation unit309outputs the information indicating correspondence relationship between pixel value and gloss intensity corresponding to the selected image capturing condition.

In the present embodiment, the aspect is explained in which a desired value is input to IMAXthat is an unknown variable and the corresponding BRDF value is calculated from each value of IMAX, but it may also be possible to calculate information indicating correspondence relationship between pixel value and gloss intensity based on captured image data obtained by capturing an image of a measurement sample, such as gloss paper. In this case, by associating the pixel values of the captured image data obtained by capturing images a plurality of kinds of measurement sample whose gloss intensity is known with the corresponding gloss intensities, it is possible to acquire information indicating correspondence relationship between pixel value and gloss intensity. It is sufficient to perform the calculation of the information indicating correspondence relationship between pixel value and gloss intensity such as this for each image capturing condition. It is not necessary for the correspondence information calculation unit309to calculate the BRDF integrated values from all the gloss intensities and it may also be possible to calculate the BRDF integrated values for only part of the gloss intensities and to interpolate the BRDF integrated values by using the publicly known linear interpolation for the other BRDF integrated values.

Returning to the flowchart inFIG. 4again, at S406, the gloss intensity distribution image data generation unit311generates gloss intensity distribution image data indicating a distribution of gloss intensities from the captured image data, the measurement area information, and the information indicating correspondence relationship between pixel value and gloss intensity. In the present embodiment, the gloss intensity distribution image data generation unit311generates gloss intensity distribution image data by converting the pixel value in the measurement area of the captured image data into the gloss intensity. The generated gloss intensity distribution image data is output from the output terminal312and stored in the storage area, such as the RAM205. On the completion of the generation of the gloss intensity distribution image data (S406), the processing by this flowchart is terminated.

(Operation of Measurement Area Setting Unit)

Next, the detailed procedure of the measurement area setting processing at S404is explained with reference to the flowchart inFIG. 5. At S501, the measurement area setting unit308sets a measurement area based on the surface light source position information and the image capturing position information. In the present embodiment, the area in which the specularly reflected light of the surface light source202enters the planar mirror and which is captured by the image capturing apparatus201in the case where the planar mirror is arranged at substantially the same position as that of the subject100is set as a measurement area.

The measurement area setting unit308calculates a range P<x<Q in the x-direction and a range R<y<S, in which the specularly reflected light of the surface light source202enters the planar mirror, in the xy-coordinate system shown inFIG. 10Aand sets a rectangular area within the calculated xy range as a measurement area1001.

FIG. 10Bis a cross section diagram of the xz-plane at y=0 with the y-axis direction being taken to be a viewpoint. At the point L (−1, 0, 1), the surface light source L (surface light source202) having a size of a length of 1 and a width of 1 is arranged and the left and right ends of the surface light source L are taken to be points A and B. A mirror image obtained by reflecting the surface light source L symmetrically with respect to the x-axis around the center of the surface of a planar mirror1002is taken to be L′ and x-coordinates of intersections P and Q of straight lines connecting points A′ and B′, which are the left and right ends of the mirror image L′, and a point C, which is the center of the camera lens, and the surface of the planar mirror1002are calculated in accordance with expressions (8) and (9) below.
(0 0 1)·(tOA′+(1−t)OC)=0 (0<t<1):  expression (8)
(0 0 1)·(tOB′+(1−t)OC)=0 (0<t<1):  expression (9)

OA′, OB′, and OC are position vectors of A′, B′, and C in the case where a point O is taken to be the origin. Here, t is a parameter that takes a value between 0 and 1 and indicates a position vector on segments A′C and B′C. Under the image capturing condition shown inFIG. 10AandFIG. 10B, t=0.5 is obtained both from expression (8) and from expression (9) and P=(−0.25, 0, 0) and Q=(0.25, 0, 0) are obtained. It is possible to perform the same calculation for the range R<y<S in the y-direction and the rectangular area within the calculated xy range is set as a measurement area. The measurement area setting unit308stores measurement area data indicating the set measurement area in the storage area, such as the RAM205, and the processing returns to the flowchart inFIG. 4again.

FIG. 11Ais an image diagram of a subject having concavity/convexity.FIG. 11Ashows an image diagram in which the reflection intensity of the face area in the subject is high, the reflection intensity of the trunk area is medium, and the reflection intensity of the background is low.

FIG. 11Bis an image diagram of a gloss intensity distribution image acquired by using a point light source. The gloss intensity distribution image data indicating the gloss intensity distribution image inFIG. 11Bis 8-bit grayscale image data. In the gloss intensity distribution image data inFIG. 11B, the specularly reflected light is observed only at the center of the face area, and therefore, the pixel value of the corresponding pixel area is as high as 255 and the pixel value decreases gradually as becoming more distant from the pixel area. In the case where gloss intensity distribution image data is acquired by using a point light source, the specularly reflected light is observed only at the center of the face area, and therefore, in order to acquire a gloss intensity distribution across the two-dimensional directions of the subject, it is necessary to repeatedly measure the specularly reflected light also in areas other than the center of the face area. As above, the conventional method of measuring the specularly reflected light by using a point light source is complicated and it requires much measurement time in order to acquire gloss intensity distribution image data.

FIG. 11Cis an image diagram of a gloss intensity distribution image acquired by using a surface light source arranged in the vicinity. The gloss intensity distribution image data indicating the gloss intensity distribution image inFIG. 11Cis also 8-bit grayscale image data. In the gloss intensity distribution image data inFIG. 11C, in the face area, the pixel value is 255 indicating a high gloss intensity, in the trunk area, the pixel value is 128 indicating a medium gloss intensity, and in the background area, the pixel value is 0 indicating a low gloss intensity. As above, in the present embodiment, by causing light from light sources in a number of dictions to enter a subject by using a surface light source arranged in the vicinity, it is possible to acquire gloss intensity distribution image data in a short time.

Further, in the present embodiment, information indicating correspondence relationship between pixel value and gloss intensity on an image captured under a surface light source is calculated on the assumption that a macro plane normal line N=(0, 0, 1), but it may also be possible to calculate information indicating correspondence relationship between pixel value and gloss intensity under an image capturing condition in which the plane normal line is inclined by the concavity/convexity of a subject. Furthermore, in the present embodiment, data indicating correspondence relationship between pixel value and gloss intensity at the center of a subject is calculated and the data indicating correspondence relationship between pixel value and gloss intensity within the measurement area is fixed. In a modification example, it may also be possible to use information indicating correspondence relationship between pixel value and gloss intensity different at each point within the measurement area, or it may also be possible to covert the pixel value into a gloss intensity by using the average value thereof.

In the present embodiment, the aspect is explained in which the information indicating correspondence relationship between pixel value and gloss intensity under various image capturing conditions is calculated in advance and stored in the correspondence information holding unit310. In another embodiment, it may also be possible to calculate the information indicating correspondence relationship between pixel value and gloss intensity from the surface light source position information, the image capturing position information, and the gloss intensity each time an image of the subject100is captured.

Further, in the present embodiment, the surface light source is used whose luminance within the surface is uniform and which emits diffused light that spreads isotropically, but it may also be possible to use a surface light source having unevenness characteristics or light distribution characteristics within the surface.

Second Embodiment

In the first embodiment, explanation is given to the method of acquiring a gloss intensity distribution in a set measurement area by setting a range in which the specularly reflected light of the surface light source202enters a planar mirror in the case where the planar mirror is arranged at substantially the same position as that of the subject100as the measurement area. In the present embodiment, explanation is given to a method of efficiently acquiring a gloss intensity distribution by calculating a measurement area of the gloss intensity distribution within an angle range set by a user from a subject having concavity/convexity.

FIG. 12is a flowchart showing a processing procedure of the measurement area setting in the present embodiment. In the following, a detailed procedure of the measurement area setting at S404is explained with reference to the flowchart inFIG. 12. Explanation of the portions in common to those of the first embodiment is simplified or omitted and in the following, points unique to the present embodiment are explained mainly.

At S1201, the measurement area setting unit308sets a plane normal line range from plane normal line range information indicating an angle range of a plane normal line of a subject whose input has been received from a user. In the present embodiment, the measurement area setting unit308sets an angle range between −5° and +5° whose input has been received from a user as a plane normal line range. The measurement area setting unit308stores the acquired plane normal line range information in the storage area, such as the RAM205, as plane normal line range data.

At S1202, the measurement area setting unit308sets a measurement area from the surface light source position information, the image capturing position information, and the plane normal line range information. In the present embodiment, the area is set as a measurement area, in which the specularly reflected light of the surface light source202enters a mirror body and whose image is captured by the image capturing apparatus201in the case where the mirror body having a plane normal line in the plane normal line range between −5° and +5° is arranged at substantially the same position as that of the subject100.

Specifically, in the xy-coordinate system shown inFIG. 13A, a range P+<x<Q+in the x-direction and a range P+<y<Q+in the y-direction in which the specularly reflected light of the surface light source enters the mirror body having the plane normal line +5° inclined are calculated. Similarly, a range P−<x<Q−in the x-direction and a range R−<y<S−in the y-direction in which the specularly reflected light of the surface light source enters the mirror body having the plane normal line −5° inclined are calculated. From calculated ranges (1301,1302) in the x- and y-directions, the rectangular common area within the xy range calculated with +5° and −5° shown in the diagonally shaded portion inFIG. 13Ais set as the measurement area.

First, the range P+<x<Q+in the x-direction in the case where a mirror body1303is inclined by +5° is calculated.FIG. 13Bis a cross section diagram of the xz-plane at y=0 with the y-axis direction being taken to be a viewpoint. At the point L (−1, 0, 1), the surface light source L (surface light source202) having a size of a length of 1 and a width of 1 is arranged and the left and right ends of the surface light source L are taken to be points A and B. Further, a mirror image of the surface light source L obtained by reflecting the surface light source L symmetrically with respect to the x-axis around the center of the surface of the mirror body1303is taken to be L′ and the left and right ends of the mirror image L′ are taken to be points A′ and B′. At this time, the position L of the surface light source with the plane normal line +5° inclined as a reference is calculated in accordance with expression (10) below.)
OL+=Rot(0°,−5°)·OL:expression (10)

In expression (10), OL and OL+are position vectors of the point L and a point L+with the point O as the origin. Rot (φ, θ) is a rotation matrix expressed by expression (11) below, which rotates a three-dimensional position in the counterclockwise direction around the x-axis by φ and in the counterclockwise direction around the y-axis by θ.

Similarly, points A+and B+at the left and right ends of a surface light source L+and an image capturing position C+with the plane normal line +5° inclined as a reference are calculated. By rotating the surface light source and the image capturing position by +5° in the direction opposite to the direction of the inclination of the plane normal line so that the plane normal line coincides with the z-axis, it is possible to calculate the surface light source and the image capturing position with the plane normal line as a reference. Further, the mirror image of the surface light source L+obtained by reflecting the surface light source L+symmetrically with respect to the x-axis around the center of the surface of the mirror body1303is taken to be L+′, and the left and right ends of the mirror image L+′ are taken to be points A−′ and B+′. From the calculated surface light source and image capturing position with the plane normal line as a reference, x-coordinates of intersections P−and Q+of straight lines connecting the points A+′ and B+′ and a center point C+of the camera lens, and the surface of the mirror body1303are calculated in accordance with expressions (12) and (13) below.
(0 0 1)·(tOA+′+(1−t)OC+)=0 (0<t<1):  expression (12)
(0 0 1)·(tOB+′+(1−t)OC+)=0 (0<t<1):  expression (13)

OA+′, OB+′, and OC+are position vectors of the points A+′, B+′, and C+in the case where the point O is taken to be the origin. Here, t is a parameter that takes a value between 0 and 1 and indicates position vectors of points on segments A+′C+and B+′C+.

It is also possible to similarly calculate the range R+<y<S+in the y-direction, and the range P−<x<Q−in the x-direction and the range R−<y<S−in the y-direction in which the specularly reflected light of the surface light source enters the mirror body having a plane normal line −5° inclined. The measurement area setting unit308sets the rectangular common area within the calculated ranges in the x- and y-directions as a measurement area. The measurement area setting unit308stores measurement area data indicating the set measurement area in the storage area, such as the RAM205, and the processing returns to the flowchart inFIG. 4again.

As described above, in the image processing system1of the present embodiment, it is possible to efficiently acquire a gloss intensity distribution by calculating a measurement area of the gloss intensity distribution within the angle range set by a user from a subject having concavity/convexity. In the present embodiment, the measurement area is calculated from the range of the plane normal line of a subject set by a user, but it may also be possible to acquire the range of the plane normal line of a subject by a method other than the method that uses the setting input from a user. Further, the surface light source and the image capturing position with the plane normal line at the center of a subject as a reference are calculated and the surface light source and the image capturing position are regarded as being fixed within the measurement area, but it may also be possible to calculate the measurement area from the surface light source and the image capturing position with the plane normal line as a reference calculated for each point within the measurement area.

In the present embodiment, the area in which the specularly reflected light of the surface light source enters the mirror body and which is captured by the image capturing apparatus in the case where the mirror body is arranged at substantially the same position as that of a subject is set as a measurement area. However, the measurement area is not limited to the area in which the specularly reflected light of the surface light source enters the mirror body. For example, it may also be possible to set an area in which a difference between a combination of the pixel value and the gloss intensity at the center of a subject and a combination of the pixel value and the gloss intensity acquired under the conditions of the surface light source, the image capturing position, and the inclination of the plane normal line at each position on the subject is smaller than a predetermined threshold value as a measurement area.

In the present embodiment, the geometric attenuation term G is regarded as being the fixed value G0and it may also be possible to set a measurement area by the geometric attenuation term. For example, it may also be possible to set an area in which an error between the geometric attenuation term G due to the shade and shielding accompanying the minute concavity/concavity on a measurement sample and the fixed value G0in the case where the measurement sample is arranged at substantially the same position as that of the subject100is smaller than a predetermined threshold value as a measurement area. Further, the refractive index η on the subject surface is regarded as being equal within the surface and the Fresnel term F is regarded as being the fixed value F0and it may also be possible to set a measurement area by the Fresnel term. For example, it may also be possible to set an area in which an error between the Fresnel term F due to the Fresnel reflection on a measurement sample and the fixed value F0in the case where the measurement sample is arranged at substantially the same position as that of the subject100is smaller than a predetermined threshold value as a measurement area.

Third Embodiment

In the first and second embodiments, explanation is given to the method of acquiring a gloss intensity distribution of an image captured by the image capturing apparatus. In the present embodiment, explanation is given to a method of acquiring a gloss intensity distribution of a subject larger than the range that can be covered by the field angle of the image capturing apparatus by repeating movement of the subject by using a movable stage or the like.FIG. 14is a schematic diagram showing an aspect in which the image capturing-target area of a subject placed on a movable stage changes as the movable stage moves in the present embodiment. As shown inFIG. 14, for example, by sequentially moving the movable stage to three points with a distance Δx in between in the x-direction and to three points with a distance Δy in between in the y-direction, that is, nine points in all, it is possible to acquire gloss intensity distribution image data at each subject position Pos (i). After acquiring each piece of gloss intensity distribution image data, by performing stitch processing of the gloss intensity distribution image data at each subject position Pos (i), it is possible to acquire a gloss intensity of a subject larger than the range that can be covered by the field angle of the image capturing apparatus201.

FIG. 15is a block diagram showing a software function configuration of the image processing unit215in the present embodiment. A processing procedure performed by image processing application software of the present embodiment based on instructions from the CPU206is explained with reference toFIG. 15. Explanation of the portions in common to those of the first embodiment is simplified or omitted and in the following, points unique to the present embodiment are explained mainly.

A subject position information input reception unit1503receives an input of subject position data and a signal indicating position information on a subject at the time of image capturing via an input terminal1501. The subject position information whose input has been received is output to a stitch unit1505. An image capturing apparatus parameter input reception unit1504receives an input of data and a signal indicating image capturing apparatus parameters at the time of image capturing via an input terminal1502. The image capturing apparatus parameters whose input has been received are output to the stitch unit1505. The stitch unit1505stitches each piece of gloss intensity distribution image data from the gloss intensity distribution image data at each subject position, the subject position information, and the image capturing apparatus parameters. The stitched gloss intensity distribution image data is output from the output terminal312.

Next, a processing procedure in the image processing unit215of the present embodiment is explained with reference to the flowchart inFIG. 16. The processing by the flowchart shown inFIG. 16is performed by the CPU206loading the program codes stored in the ROM204and executing the program codes. Explanation of the portions in common to those of the first embodiment is simplified or omitted and in the following, points unique to the present embodiment are explained mainly.

At S1601, the subject position information input reception unit1503initializes a variable i indicating the order of image capturing of a subject. In the present embodiment, i is initialized to 1 (i=1).

S401to S406are the same as those of the first embodiment, and therefore, explanation is omitted. At S1602, the subject position information input reception unit1503receives an input of the position of the subject center at the time of the ith image capturing as the subject position data Pos (i). The subject position data Pos (i) whose input has been received is stored in the storage area, such as the RAM205.

At S1603, the image capturing apparatus parameter input reception unit1504receives an input of the image capturing apparatus parameters at the time of the ith image capturing. Here, the image capturing apparatus parameters of the present embodiment are parameters indicating the image capturing characteristics of the image capturing apparatus201, such as the focal length of the lens, the pixel position of the optical axis center, and the sensor pixel pitch. The image capturing apparatus parameters whose input has been received are stored in the storage area, such as the RAM205.

At S1604, the stitch unit1505determines whether the variable i satisfies i>1. In the case where i>1 is satisfied, it is determined that the gloss intensity distribution image data at a plurality of subject positions can be stitched and the processing advances to S1605. In the case where i>1 is not satisfied, it is determined that the gloss intensity distribution image data at the first subject position has been acquired and the processing advances to S1606.

At S1605, the stitch unit1505performs geometric transformation for the gloss intensity distribution image data at Pos (i) by the publicly known projection transformation processing based on the subject position data and the image capturing apparatus parameters. Further, the stitch unit1505stitches the gloss intensity distribution image data at Pos (i) for which the geometric transformation processing has been performed and the gloss intensity distribution image data at Pos (1) to Pos (i−1) having already been stitched by the publicly known α blending processing. The gloss intensity distribution image data for which the stitch processing has been performed is stored in the storage area, such as the RAM205.

At S1606, the variable i is updated. In the present embodiment, i is incremented by 1 as i=i+1.

At S1607, the stitch unit1505determines whether the variable i satisfies i>n−1. In the case where it is determined that i>n−1 is satisfied, it is determined that the gloss intensity distribution image data at all the subject positions has been stitched and this flowchart is terminated. In the case where it is determined that i>n−1 is not satisfied, the processing returns to S401again.

As explained above, in the present embodiment, it is possible to acquire a gloss intensity distribution of a subject larger than the range that can be covered by the field angle of the image capturing apparatus201at a high speed and with a high accuracy by repeating acquisition of a gloss intensity distribution and movement of the subject by using a movable stage or the like. In the present embodiment, by arranging a subject on a movable stage and moving the subject, a gloss intensity distribution of the subject larger than the range that can be covered by the field angle of the image capturing apparatus201is acquired, but it may also be possible to fix a subject and move the surface light source and the camera. Further, by using the subject position and the image capturing apparatus parameters, geometric transformation is performed for the gloss intensity distribution image data at each subject position by projection transformation, but it may also be possible to perform geometric transformation by detecting a corresponding point of each piece of gloss intensity distribution image data by using the publicly known template matching and SIFT method.

Other Embodiments

It is possible for the image processing apparatus of the present invention to efficiently acquire a gloss intensity distribution of a subject from a subject having concavity/convexity.

This application claims the benefit of Japanese Patent Application No. 2016-105514 filed May 26, 2016, which is hereby incorporated by reference wherein in its entirety.