Patent Publication Number: US-8988449-B2

Title: Image processing apparatus and image processing method

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates to an image processing apparatus and image processing method and, more particularly, to an image processing apparatus and image processing method for generating an image using ray tracing. 
     2. Description of the Related Art 
     Recently, computer graphics (CG) have been used in various fields. CG is widely used even in the design field. In this field, a technique for generating an image that is more faithful to a real object is required. 
     As a method for faithfully reproducing a real object using CG, various techniques including the following ones have been known. Examples are a method using a multi-angle spectral reflection property to simulate reflection by a paint or the like having optically complicated properties, and a method using the multi-angle spectral reflection property of an object to reproduce a highly realistic image on the monitor. Other examples are a method using a spectral refraction property to reproduce the color of reflection, and a method using multiband data captured by a multispectral camera. 
     Image generation methods in CG include a scan line method, Z-buffer method, and ray tracing method. Of these methods, the ray tracing method is suitable for generating an image faithful to a real object because it can express reflection and refraction by tracing the propagation path of light in a 3D space. However, the ray tracing method needs to repetitively trace the paths of a great number of rays. To obtain a high-quality image in a complicated scene, this method requires an enormous amount of calculation. As efficient algorithms for the ray tracing method, binary space partitioning, bounding volume, and the like are proposed. For example, as a technique using binary space partitioning, there is proposed a method of partitioning in advance a virtual space into a plurality of voxels, performing crossing detection for only objects in voxels through which rays pass, and generating an image. 
     However, the conventional methods of generating a CG image do not consider the light distribution characteristics of light sources. These methods cannot reproduce a change of a shadow which is generated in a scene depending on light sources having different light distribution characteristics such as a fluorescent light and light-emitting diode (LED) illumination. 
     A general ray tracing method handles light as wavelength-independent uniform rays. This method cannot accurately reproduce a physical phenomenon in which the behavior changes depending on the wavelength, such as refraction of light passing through an object. 
     SUMMARY OF THE INVENTION 
     In one aspect, an image processing apparatus for generating a computer graphics image, comprising: a first input section configured to input light source characteristic data representing a spectral characteristic and light distribution characteristic of at least one light source which illuminates a scene; a second input section configured to input object characteristic data representing a spectral property of at least one object contained in the scene; a trace section configured to perform ray tracing for respective rays having different wavelengths, and calculate a spectral radiance in each pixel of a computer graphics image which represents the scene based on the light source characteristic data and the object characteristic data; and a generator configured to calculate color signal values of each pixel from the spectral radiance in the each pixel, and generate the computer graphics image of the scene. 
     In another aspect, an image processing method of generating a computer graphics image, comprising the steps of: inputting light source characteristic data representing a spectral characteristic and light distribution characteristic of at least one light source which illuminates a scene; inputting object characteristic data representing a spectral property of at least one object contained in the scene; performing ray tracing for respective rays having different wavelengths to calculate a spectral radiance in each pixel of a computer graphics image which represents the scene based on the light source characteristic data and the object characteristic data; and calculating color signal values of each pixel from the spectral radiance in the each pixel to generate the computer graphics image of the scene. 
     According to these aspects, a computer graphics image which faithfully reproduces a scene can be generated by ray tracing which considers the light distribution characteristics and spectral characteristics of a light source and the spectral characteristics of an object. 
     Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a schematic view for explaining reflection and transmission of light by an object. 
         FIG. 2  is a block diagram showing the arrangement of an image processing apparatus in the first embodiment. 
         FIG. 3  is a block diagram showing the logical arrangement of the image processing apparatus in the first embodiment. 
         FIG. 4  is a flowchart showing image generation processing in the first embodiment. 
         FIGS. 5A and 5B  are schematic views for explaining the light distribution characteristics of a light source. 
         FIGS. 6A and 6B  are graphs exemplifying the spectral radiance of a light source. 
         FIG. 7  is a schematic view for explaining the reflection property and transmission property of an object. 
         FIG. 8  is a graph exemplifying visual characteristic data. 
         FIG. 9  is a flowchart showing spectral radiance calculation processing in the first embodiment. 
         FIG. 10  is a flowchart showing color signal value calculation processing in the first embodiment. 
         FIGS. 11A to 11C  are a view and graphs, respectively, exemplifying the wavelength dependence of the light distribution characteristics of a light source. 
         FIG. 12  is a flowchart showing image generation processing in the second embodiment. 
         FIG. 13  is a flowchart showing spectral radiance calculation processing in the second embodiment. 
         FIG. 14  is a graph showing a concept of a method of setting a maximum crossing count in ray tracing in the second embodiment. 
         FIG. 15  is a flowchart showing spectral radiance calculation processing in the third embodiment. 
         FIG. 16  is a graph showing a concept of a method of setting the number of rays to be cast in ray tracing in the third embodiment. 
     
    
    
     DESCRIPTION OF THE EMBODIMENTS 
     An image processing apparatus and image processing method according to embodiments of the present invention will now be described with reference to the accompanying drawings. It is to be understood that the following embodiments are not intended to limit the present invention regarding the appended claims, and that not all combinations of features set forth in the embodiments are indispensable for a solution of problems in the present invention. 
     First Embodiment 
     The first embodiment will explain processing of generating a computer graphics image (CG image) which faithfully reproduces a scene by ray tracing in consideration of the light distribution characteristics and spectral radiation characteristics of a light source and the spectral reflection property of an object. 
     The behavior of light which changes depending on the wavelength will be explained with reference to  FIG. 1 . When light strikes an object, it is split into light reflected by the surface of the object and light passing through the object. The refraction angle of the light passing through the object changes depending on a refractive index corresponding to the wavelength λ. For example, a case in which a ray having passed through a space having a spectral refractive index n 1 (λ) passes through an object  101  having a spectral refractive index n 2 (λ) using a viewpoint as a start point, as shown in  FIG. 1 , will be examined. At this time, letting φ be the incident angle of a ray with respect to the object  101 , the refraction angle θ(λ) is given by
 
θ(λ)=sin −1   [{n   1 (λ)/ n   2 (λ)} sin φ]  (1)
 
     Since the refraction angle is a function of the wavelength λ of light, the path of light to be traced by ray tracing changes depending on the wavelength λ. Hence, light needs to be handled as spectra in order to faithfully reproduce transmission of light in a given scene in which, for example, light having passed through the object  101  in  FIG. 1  further passes through an object  102  having a spectral refractive index n 3 (λ). 
     Apparatus Arrangement 
     The arrangement of an image processing apparatus in the first embodiment will be described with reference to  FIG. 2 . As shown in  FIG. 2 , the image processing apparatus in the first embodiment includes a microprocessor (CPU)  201 , main memory  202  comprising a random access memory (RAM) and a read-only memory (ROM), hard disk drive (HDD)  203 , general-purpose interface (to be referred to as I/F hereinafter)  204 , monitor  205 , and system bus  206 . The I/F  204  connects a spectrophotometer  207 , external storage device  208 , and the like to the system bus  206 . The CPU  201  executes various software programs (computer programs) stored in the HDD  203 , implementing various processes. 
     The CPU  201  loads, to a work area assigned in the main memory  202 , an application program for image processing (to be referred to as an image processing AP hereinafter) stored in the HDD  203 , and executes the image processing AP. In accordance with the image processing AP, the CPU  201  transfers, to the main memory  202  via the system bus  206 , various data stored in the HDD  203 , data input from the spectrophotometer  207  and external storage device  208  which are connected to the I/F  204 , and the like. In accordance with the image processing AP, the CPU  201  executes various calculations for data stored in the work area of the main memory  202 , displays the calculation results on the monitor  205  via the system bus  206 , and stores them in the HDD  203 . 
     Logical Arrangement 
     Image processing when the CPU  201  generates an image in accordance with the image processing AP and displays it on the monitor  205  in the above arrangement will be explained in detail. 
       FIG. 3  is a block diagram showing the logical arrangement of an image processing apparatus  301  in the first embodiment. Each building component shown in  FIG. 3  is implemented by executing the image processing AP by the CPU  201 , as described above. 
     Referring to  FIG. 3 , a light source characteristic input unit  302  inputs light source characteristic data  307  representing the light distribution characteristics and spectral radiance of at least one light source which illuminates a scene. Note that the spectral radiance of a light source will also be simply referred to as the “spectral characteristics of a light source”. An object characteristic input unit  303  inputs object characteristic data  308  representing the spectral reflection property, spectral transmission property, and spectral refractive index of at least one object which forms a scene. Note that the spectral reflection property, spectral transmission property, and spectral refractive index of an object will also be simply referred to as the “reflection property, transmission property, and refractive index of an object”. A visual characteristic input unit  304  inputs visual characteristic data  309  when observing a scene. 
     Note that the light source characteristic data  307 , object characteristic data  308 , and visual characteristic data  309  are input from the spectrophotometer  207  or external storage device  208  via the HDD  203  or I/F  204 . 
     A spectral radiance calculation unit  305  calculates a spectral radiance  310  in each pixel of the CG image of a scene based on information input by the light source characteristic input unit  302  and information input by the object characteristic input unit  303 . 
     A color signal value calculation unit  306  calculates the color signal values of each pixel in the CG image of a scene based on the spectral radiance  310  in each pixel that has been calculated by the spectral radiance calculation unit  305 , and the visual characteristic data  309  input by the visual characteristic input unit  304 . The color signal values are calculated as, for example, R, G, and B values, and output as image data  311 . The image data  311  is stored in a storage area such as the HDD  203 , and an image represented by the image data  311  is displayed on the monitor  205 . 
     Scene CG Image Generation Processing 
     Image generation processing of generating the CG image of a scene in the image processing apparatus  301  in the first embodiment will be explained with reference to the flowchart of  FIG. 4 . Note that the image generation processing shown in  FIG. 4  is executed by the logical arrangement shown in  FIG. 3  that is implemented by executing the image processing AP by the CPU  201 . 
     First, the light source characteristic input unit  302  inputs the light source characteristic data  307  (S 401 ). As described above, the light source characteristic data  307  represents the light distribution characteristics and spectral radiance of a light source which illuminates a scene. 
     The light distribution characteristics of a light source will be explained with reference to  FIGS. 5A and 5B . The light distribution characteristics represent the intensity distribution of light emitted by a light source in respective directions. The range in which one light source illuminates a scene changes depending on the difference in the directivity of the light source.  FIG. 5A  exemplifies the light distribution characteristics of a light source having low directivity such as a fluorescent light, and shows that such a light source illuminates a scene widely. In contrast,  FIG. 5B  exemplifies the light distribution characteristics of a light source having high directivity such as an LED, and shows that such a light source illuminates only part of a scene. In this manner, when the light distribution characteristics of a light source change, a shadow generated in a scene changes greatly. The light distribution characteristics of a light source are an important factor to faithfully reproduce a scene. 
     As the light distribution characteristics, data stored in the HDD  203  or the like may be input, or data obtained by measuring a light source by the spectrophotometer  207  connected to the I/F  204  may be input. The format of input data is arbitrary as long as the correspondence between the output angle of light from a light source and the intensity of light is obtained. For example, the format may be a function of the intensity of light with respect to the output angle, or a lookup table (LUT) representing the correspondence between the output angle and the intensity of light. 
     The spectral radiance will be explained with reference to  FIGS. 6A and 6B . The spectral radiance of a light source represents the distribution of luminances at respective wavelengths, and the characteristics change depending on a light source.  FIG. 6A  exemplifies the spectral radiance distribution of a fluorescent light.  FIG. 6B  exemplifies the spectral radiance distribution of an LED illumination. When the type of light source changes, the spectral radiance distribution also changes, and thus the color appearance of an object changes. Hence, the spectral radiance (spectral characteristics) of a light source is an important factor to faithfully reproduce a scene. As the spectral characteristics, similar to the light distribution characteristics, data stored in the HDD  203  or the like may be input, or data obtained by measuring a light source by the spectrophotometer  207  connected to the I/F  204  may be input. 
     The format of input data is arbitrary as long as the correspondence between a wavelength in the visible light range and the radiance is obtained. For example, the format may be a function of the radiance with respect to the wavelength of light, or an LUT representing the correspondence between the wavelength and the radiance. 
     Then, the object characteristic input unit  303  inputs the object characteristic data  308  (S 402 ). As described above, the object characteristic data  308  represents the reflection property, transmission property, and refractive index of each object contained in a scene. 
     The reflection property and transmission property of an object will be explained with reference to  FIG. 7 . In general, the reflection property of an object is represented by a bidirectional reflectance distribution function (BRDF) f r . The reflection property f r  at a position x on an object is given by
 
 f   r ( x,{right arrow over (ω)}   i ,λ r ,{right arrow over (ω)} r )={ dL   r ( x,λ   r ,{right arrow over (ω)} r )}/{ L   i ( x,λ   i ,{right arrow over (ω)} i )({right arrow over (ω)} i   ·{right arrow over (n)} ) d{right arrow over (ω)}   i }  (2)
 
where λ i  and L i  are the wavelength and intensity of light entering from the direction ω i ,
 
     λ r  and dL r  are the wavelength and intensity of light reflected in the direction ω r , and 
     n is the normal vector at the position x. 
     In general, the intensity distribution of reflected light depends on the surface shape of an object. As the object surface becomes smoother, a reflection intensity in the specular reflection direction, that is, θ i =θ r  direction becomes higher, and a reflection intensity in the diffuse reflection direction becomes lower. 
     As a method of acquiring the reflection property f r  of an object, data stored in the HDD  203  or the like may be input, or data measured by the spectrophotometer  207  connected to the I/F  204  may be input. The data format of the reflection property f r  is arbitrary as long as a reflectance corresponding to the incident angle and reflection angle of light with respect to an object is obtained. For example, the reflection property f r  may be input as a unique function as represented by equation (2). Alternatively, reflection properties may be input divisionally. For example, diffuse reflection is distributed isotropically, so a reflection property for ω i =45° and ω r =0° is applied to diffuse reflection and a reflection property for specular reflection is separately obtained from an LUT. 
     The transmission property of an object is represented by a bidirectional transmittance distribution function (BTDF) f t . The transmission property f t  at a position x on an object is given by
 
 f   t ( x,{right arrow over (ω)}   i ,λ t ,{right arrow over (ω)} t )={ dL   t ( x,λ   t ,{right arrow over (ω)} t )}/{ L   i ( x,λ   i ,{right arrow over (ω)} i )({right arrow over (ω)} i   ·{right arrow over (n)} ) d{right arrow over (ω)}   i }  (3)
 
where λ t  and dL t  are the wavelength and intensity of light transmitted in the direction ω t .
 
     In general, the intensity distribution of transmitted light depends on the refractive index. When ω t  satisfies equation (1), the intensity of transmitted light becomes maximum. 
     As a method of acquiring the transmission property f t  of an object, data stored in the HDD  203  or the like may be input, or data measured by the spectrophotometer  207  connected to the I/F  204  may be input. The data format of the transmission property f t  is arbitrary as long as a transmittance corresponding to the incident angle and refraction angle of light with respect to an object is obtained. For example, the transmission property f t  may be input as a unique function as represented by equation (3). Alternatively, transmission properties may be input divisionally. For example, a transmission property for ω i =ω r =0° is applied to a case in which ω t  is equal to the refraction angle θ in equation (1), and a transmission property for another case is separately obtained from an LUT. 
     After that, the visual characteristic input unit  304  inputs the visual characteristic data  309  (S 403 ). The visual characteristic data  309  is used when converting a spectral radiance into the color signal values of an image.  FIG. 8  shows color matching functions used in the CIEXYZ colorimetric system as an example of the visual characteristic data  309 . The color matching function represents the sensitivity of an eye to light in the visible light range. As the visual characteristic data  309 , data stored in advance in the HDD  203  or the like suffices to be input. 
     Then, the spectral radiance calculation unit  305  scans the CG image of the scene to determine a pixel (to be referred to as a pixel of interest hereinafter) subjected to calculation of a spectral radiance (S 404 ). Note that the determination order of the pixel of interest is arbitrary. 
     The spectral radiance calculation unit  305  calculates the spectral radiance  310  in the pixel of interest (S 405 ). The spectral radiance  310  is calculated using ray tracing, and details of the processing will be described later. 
     Thereafter, the color signal value calculation unit  306  calculates the color signal values of the pixel of interest (S 406 ). Details of the color signal value calculation method will be described later. 
     The CPU  201  determines whether calculation of color signal values has ended for all the pixels of the CG image of the scene (S 407 ). If the calculation has ended, the image data  311  generated by the color signal value calculation unit  306  is output, and the process ends. If the calculation has not ended, the process returns to step S 404  to determine the next pixel of interest, and continues. 
     Spectral Radiance Calculation Processing 
     The spectral radiance calculation processing (S 405 ) will be explained in detail with reference to  FIG. 9 . As described above, calculation of the spectral radiance in the first embodiment uses ray tracing. This method renders a CG image by virtually tracing, up to a light source, rays which should be received at the viewpoint from respective directions within the angle of view. More specifically, straight lines (rays) are extended from the viewpoint toward pixels to be rendered, and the presence/absence of a crossing of an object and ray is mathematically determined. Tracing of a ray generated by reflection or transmission is recursively repeated, and when the light source and ray cross each other, the tracing of the ray ends. That is, in the first embodiment, a plurality of rays are traced by recursively executing ray tracing shown in the flowchart of  FIG. 9 , and a spectral radiance in the pixel of interest is calculated from the average of spectral radiances calculated for respective rays. 
     Spectral radiance calculation processing for monochromatic light having the wavelength λ will be explained with reference to  FIG. 9 . A spectral radiance in the pixel of interest is actually obtained by executing the same processing for each wavelength. Assume that processing target wavelengths are extracted at equal intervals from the visible range. For example, 36 wavelengths are extracted as processing target wavelengths at an interval of 10 nm in a visible range of 380 to 730 nm. 
     First, a point (to be referred to as an intersection point hereinafter) at which a ray having a wavelength λ that has been cast from the start point of tracing crosses an object or light source in a scene is searched for (S 901 ). The start point of tracing is the pixel of interest in the first execution of the processing shown in  FIG. 9 , and an intersection point between a ray and an object in recursive execution of the processing shown in  FIG. 9 . Note that the intersection point search method is arbitrary. For example, crossing detection is sequentially performed for all objects and light sources within a scene to obtain an intersection point closest to the start point of tracing. 
     It is determined which of an intersection point between a light source and the ray and an intersection point between an object and the ray is the intersection point detected in step S 901  (S 902 ). If the ray crosses a light source, a spectral radiance for the wavelength λ is calculated from the intensity of the ray and the light source characteristic data  307  of the crossing light source (S 903 ). Then, ray tracing for the current wavelength ends. Since the light source characteristic data  307  represents the light distribution characteristics and spectral radiance of a light source, the spectral radiance E 0 (λ) of a ray is given by
 
 E   0 (λ)= P   0 (λ)× Q (θ)× S (λ)  (4)
 
     where P 0 (λ) is the intensity of a ray at the wavelength λ, 
     θ is the angle defined by the ray and a light source, 
     Q(θ) is the light distribution characteristics of the light source in the θ direction, and 
     S(λ) is the radiance of the light source at the wavelength λ. 
     If the ray crosses an object other than a light source, first, the object characteristic data  308  (reflection property, transmission property, and refractive index) of the object crossing the ray is acquired (S 904 ). 
     Then, the number of rays (to be referred to as the number of rays to be cast hereinafter) to be cast from the intersection point between the ray and the object is determined (S 905 ). Although the number of rays to be cast may be arbitrarily determined, an image with less noise is obtained as the number of rays to be cast increases. Processes in steps S 906  and S 907  are repeated by the number of times corresponding to the number of rays to be cast. 
     In step S 906 , the directions and intensities of rays to be cast are calculated. The directions of rays suffice to be distributed uniformly in accordance with the number of rays to be cast. The intensity P r (λ) of reflected light and the intensity P t (λ) of transmitted light at the intersection point are respectively given by
 
 P   r (λ)= P   0 (λ)× f   r ( x,ω   i ,λ,ω r )  (5)
 
 P   t (λ)= P   0 (λ)× f   t ( x,ω   i ,λ,ω t )  (6)
 
     where P 0 (λ) is the intensity of a ray before crossing an object at the wavelength λ, 
     f r  is the reflection property of the object given by equation (2), and 
     f t  is the transmission property of the object given by equation (3). 
     In step S 907 , ray tracing is performed for the ray by recursively invoking ray tracing shown in  FIG. 9 . The processing in step S 907  ends when no crossing object remains finally in recursive processing and a spectral radiance for the ray is calculated in step S 903 . In other words, the spectral radiance E 0 (λ) finally calculated according to equation (4) in the recursive processing of step S 907  becomes a spectral radiance E n (λ) for the ray. 
     If it is determined in step S 908  that the processes in steps S 906  and S 907  have ended for all rays by the number of rays to be cast, the average of the spectral radiances E n (λ) of the respective rays is calculated (S 909 ). The average spectral radiance value serves as a spectral radiance E x (λ) at the tracing target wavelength λ in the pixel of interest. More specifically, the spectral radiance E x (λ) at the tracing target wavelength λ in the pixel of interest is calculated in accordance with equation (7):
 
 E   x (λ)=(1 /N )·Σ n=0   N-1   E   n (λ)  (7)
         where N is the number of rays to be cast, and 0≦n≦N−1.       

     In this fashion, in the spectral radiance calculation processing (S 405 ), the spectral radiance E x (λ) in the pixel of interest is calculated by recursively executing ray tracing shown in  FIG. 9 . 
     Color Signal Value Calculation Processing 
     The color signal value calculation processing (S 406 ) for each pixel will be explained in detail with reference to  FIG. 10 . In the first embodiment, color signal values of the sRGB system are finally output. However, output color signal values may be color signal values of another system such as AdobeRGB. 
     First, color matching functions of, for example, the CIEXYZ colorimetric system are acquired from the visual characteristic data  309  input in step S 403  (S 1001 ). Then, the spectral radiance E x (λ) of each pixel that has been calculated in step S 405  is acquired (S 1002 ). CIEXYZ values Xout, Yout, and Zout are calculated from the acquired color matching functions and spectral radiance E x (λ) in accordance with equations (8) to (10) (S 1003 ):
 
 X out=∫ 380   730   E   x (λ)   x   (λ) dλ   (8)
 
 Y out=∫ 380   730   E   x (λ)   y   (λ) dλ   (9)
 
 Z out=∫ 380   730   E   x (λ)   z   (λ) dλ   (10)
 
     where  x (λ),  y (λ), and  z (λ) are color matching functions. 
     The color signal values of an output image are calculated (S 1004 ). Since the output color signal values comply with the sRGB system, color signal values Rout, Gout, and Bout are calculated in accordance with equations (11) and (12): 
     
       
         
           
             
               
                 
                   
                     
                       [ 
                       
                         
                           
                             
                               R 
                               Linear 
                             
                           
                         
                         
                           
                             
                               G 
                               Linear 
                             
                           
                         
                         
                           
                             
                               B 
                               Linear 
                             
                           
                         
                       
                       ] 
                     
                     = 
                     
                       
                         [ 
                         
                           
                             
                               3.2406 
                             
                             
                               
                                 - 
                                 1.5372 
                               
                             
                             
                               
                                 - 
                                 0.4986 
                               
                             
                           
                           
                             
                               
                                 - 
                                 0.9689 
                               
                             
                             
                               1.8758 
                             
                             
                               0.0415 
                             
                           
                           
                             
                               0.0557 
                             
                             
                               
                                 - 
                                 0.2040 
                               
                             
                             
                               1.0570 
                             
                           
                         
                         ] 
                       
                       ⁡ 
                       
                         [ 
                         
                           
                             
                               Xout 
                             
                           
                           
                             
                               Yout 
                             
                           
                           
                             
                               Zout 
                             
                           
                         
                         ] 
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       ( 
                       
                         
                           R 
                           Linear 
                         
                         ≤ 
                         0.0031308 
                       
                       ) 
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       Rout 
                       = 
                       
                         12.92 
                         × 
                         
                           R 
                           Linear 
                         
                       
                     
                     ; 
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   else 
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       Rout 
                       = 
                       
                         
                           1.055 
                           × 
                           
                             R 
                             Linear 
                             
                               1 
                               / 
                               2.4 
                             
                           
                         
                         - 
                         0.055 
                       
                     
                     ; 
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       ( 
                       
                         
                           G 
                           Linear 
                         
                         ≤ 
                         0.0031308 
                       
                       ) 
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       Gout 
                       = 
                       
                         12.92 
                         × 
                         
                           G 
                           Linear 
                         
                       
                     
                     ; 
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   else 
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       Gout 
                       = 
                       
                         
                           1.055 
                           × 
                           
                             G 
                             Linear 
                             
                               1 
                               / 
                               2.4 
                             
                           
                         
                         - 
                         0.055 
                       
                     
                     ; 
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     if 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       ( 
                       
                         
                           B 
                           Linear 
                         
                         ≤ 
                         0.0031308 
                       
                       ) 
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       Bout 
                       = 
                       
                         12.92 
                         × 
                         
                           B 
                           Linear 
                         
                       
                     
                     ; 
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   else 
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
             
               
                 
                   
                     Bout 
                     = 
                     
                       
                         1.055 
                         × 
                         
                           B 
                           Linear 
                           
                             1 
                             / 
                             2.4 
                           
                         
                       
                       - 
                       0.055 
                     
                   
                   ; 
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
     By the above-described image generation processing, an output image serving as the CG image of a scene is generated. 
     As described above, ray tracing in a scene is performed in consideration of the light distribution characteristics and spectral characteristics of a light source and the spectral characteristics (reflection property, transmission property, and refractive index) of an object. Accordingly, a CG image which faithfully reproduces the scene can be generated. 
     Modification of First Embodiment 
     In the first embodiment, light distribution characteristics represented by the light source characteristic data  307  are handled as the intensity distribution of light. However, the intensity distribution of light changes depending on the wavelength in some cases. For example, for an LED illumination, the light source is sometimes formed by combining an LED  11  which emits blue light, and a fluorescent substance  12  which emits yellow light, as shown in  FIG. 11A . At this time, the LED  11  emits light having high directivity, as shown in  FIG. 5B , so color nonuniformity is generated when the LED  11  is combined with the fluorescent substance  12 . Since the light distribution characteristics of a light source change depending on the wavelength, as shown in  FIGS. 11B and 11C , the spectral radiance also depends on the angle θ defined by a ray and light source. 
     It is therefore considered to input, as the light source characteristic data  307 , light distribution characteristics which change depending on the wavelength. In this case, the spectral radiance calculation processing (S 405 ) is almost the same as the processing shown in  FIG. 9  except for the final spectral radiance calculation processing in step S 903 . More specifically, when the light source has wavelength-dependent light distribution characteristics, the light distribution characteristics are represented as a function Q(λ, θ) of the wavelength λ and the angle θ defined by a ray and the light source. In this case, the spectral radiance S(λ) of the light source is obtained from the light distribution characteristic Q(λ, θ) and becomes unnecessary. Thus, the spectral radiance E 0 (λ) of a ray in step S 903  is calculated in accordance with equation (13):
 
 E   0 (λ)= P   0 (λ)× Q (λ,θ)  (13)
 
     In this way, even if the light distribution characteristics of a light source depend on the wavelength, a CG image which faithfully reproduces a scene can be generated, similar to the first embodiment. 
     Second Embodiment 
     In the first embodiment, a CG image which faithfully reproduces a scene is generated by performing ray tracing based on the light distribution characteristics and spectral characteristics of a light source and the spectral characteristics of an object. However, when ray tracing based on spectroscopy (to be referred to as spectral ray tracing hereinafter) is performed as in the first embodiment, the calculation amount increases in accordance with the number of wavelengths to be traced, decreasing the processing speed. Generally in ray tracing, the calculation amount can be suppressed to increase the processing speed by reducing the number of crossings of one ray and objects and the number of rays to be traced. However, these reductions generate noise in a generated image, degrading the image quality. 
     Also, when an efficient ray tracing method using binary space partitioning is applied to spectral ray tracing, if the number of crossings and the number of rays are reduced regardless of the wavelength of light for a higher processing speed, these reductions also generate noise. The second embodiment increases the processing speed and reduces noise by adding, to the technical feature of the first embodiment, a technical feature of controlling the calculation amount in ray tracing in accordance with the spectral characteristics of a light source and object. 
     Note that the arrangement of an image processing apparatus in the second embodiment is the same as those shown in  FIGS. 2 and 3  in the first embodiment. The same reference numerals denote the same parts, and a description thereof will not be repeated. 
     Scene CG Image Generation Processing 
     Image generation processing of generating the CG image of a scene in an image processing apparatus  301  in the second embodiment will be explained with reference to the flowchart of  FIG. 12 . Note that the image generation processing shown in  FIG. 12  is executed by the logical arrangement shown in  FIG. 3  that is implemented by executing an image processing AP by a CPU  201 . 
     First, a light source characteristic input unit  302  inputs light source characteristic data  307  (S 1401 ). The light source characteristic data  307  represents the light distribution characteristics and spectral radiance of a light source which illuminates a scene, as shown in  FIGS. 5A ,  5 B,  6 A, and  6 B. 
     Then, an object characteristic input unit  303  inputs object characteristic data  308  (S 1402 ). As shown in  FIG. 7 , the object characteristic data  308  represents the reflection property, transmission property, and refractive index of each object contained in a scene. 
     A visual characteristic input unit  304  inputs visual characteristic data  309  (S 1403 ). The visual characteristic data  309  is used when converting a spectral radiance into the color signal values of an image. The visual characteristic data  309  includes, for example, color matching functions shown in  FIG. 8 . 
     A spectral radiance calculation unit  305  scans the CG image of the scene to determine a pixel of interest subjected to calculation of a spectral radiance. Note that the determination order of the pixel of interest is arbitrary. 
     The spectral radiance calculation unit  305  initializes, to 0, a crossing count C which is a variable for counting the number of crossings of a ray traveling from the pixel of interest with objects during path tracing (S 1405 ). Subsequently, the spectral radiance calculation unit  305  initializes the maximum crossing count M(λ) serving as the upper limit of the crossing count for each wavelength λ (S 1406 ). 
     The maximum crossing count M(λ) is a threshold to, when a ray traveling from the pixel of interest crosses many objects until it reaches a light source, abort ray tracing in accordance with the crossing count C before the ray reaches the light source. A ray is desirably traced until it reaches a light source, but this requires an enormous amount of calculation. Thus, when a ray does not reach a light source by the maximum crossing count M(λ), ray tracing is aborted to reduce the calculation amount and shorten the calculation time. 
     As the initial value of the maximum crossing count M(λ), an arbitrary value of 1 or larger is set for each wavelength λ of light. If the maximum crossing count M(λ) is small, calculation can be performed quickly, but noise is generated in the image generation result. Therefore, the initial value of the maximum crossing count M(λ) is desirably set in consideration of the required calculation time and noise amount. As the initial value of the maximum crossing count M(λ), a fixed value independent of the wavelength λ of light is desirably set. 
     Thereafter, the spectral radiance calculation unit  305  calculates a spectral radiance  310  in the pixel of interest (S 1407 ). The spectral radiance  310  is calculated using ray tracing, and details of the processing will be described later. 
     A color signal value calculation unit  306  calculates the color signal values of the pixel of interest (S 1408 ). The color signal value calculation method is the same as the method described with reference to  FIG. 10 , and a description thereof will not be repeated. 
     Then, the CPU  201  determines whether calculation of color signal values has ended for all the pixels of the CG image of the scene (S 1409 ). If the calculation has ended, image data  311  generated by the color signal value calculation unit  306  is output, and the process ends. If the calculation has not ended, the process returns to step S 1404  to determine the next pixel of interest, and continues. 
     Spectral Radiance Calculation Processing 
     The spectral radiance calculation processing (S 1407 ) will be explained in detail with reference to  FIG. 13 . Even in the second embodiment, a plurality of rays are traced by recursively executing ray tracing shown in the flowchart of  FIG. 13 , and a spectral radiance in the pixel of interest is calculated from the average of spectral radiances calculated for respective rays. 
     Spectral radiance calculation processing for monochromatic light having the wavelength λ will be explained with reference to  FIG. 13 . A spectral radiance in the pixel of interest is actually obtained by executing the same processing for each wavelength, similar to the first embodiment. 
     First, a point (intersection point) at which a ray having a wavelength λ that has been cast from the start point of tracing crosses an object or light source in a scene is searched for (S 1901 ). Similar to the first embodiment, the start point of tracing is the pixel of interest in the first execution of the processing shown in  FIG. 13 , and an intersection point between a ray and an object in recursive execution of the processing shown in  FIG. 13 . Similar to the first embodiment, the intersection point search method is arbitrary. 
     It is determined which of an intersection point between a light source and the ray and an intersection point between an object and the ray is the intersection point detected in step S 1901  (S 1902 ). If the ray crosses a light source, the spectral radiance E 0 (λ) for the wavelength λ is calculated from the intensity of the ray and the light source characteristic data  307  of the crossing light source in accordance with equation (4) (S 1914 ). Then, ray tracing for the current wavelength ends. 
     If the ray crosses an object other than the light source, the count value is updated by incrementing the crossing count C of the ray by one (S 1903 ). The object characteristic data  308  (reflection property, transmission property, and refractive index) of the object crossing the ray is acquired (S 1904 ). 
     Then, the crossing count C and the maximum crossing count M(λ) of the ray at the wavelength λ are compared (S 1905 ). If the crossing count C has exceeded the maximum crossing count M(λ) (C&gt;M(λ)), the process shifts to step S 1913  in order to end ray tracing even if the ray has not reached the light source. If the crossing count C is equal to or smaller than the maximum crossing count M(λ) (C≦M(λ)), the process shifts to step S 1906  in order to newly cast a plurality of rays from the intersection point between the ray and the object and perform ray tracing for the respective rays. 
     In this manner, the second embodiment increases the processing speed by aborting ray tracing before a ray reaches a light source when it does not reaches the light source by the preset maximum crossing count M(λ). Decreasing the maximum crossing count M(λ) can increase the calculation speed, but increases noise in a generated image. To prevent this, the second embodiment sets a maximum crossing count M(λ) optimum for each wavelength of light in accordance with the reflection property and transmission property of an object first crossing a ray, the radiance of a light source, and the initial value of the maximum crossing count M(λ) that has been set in step S 1406 , details of which will be described later. 
     For C≦M(λ), the number of rays (number N of rays to be cast) to be cast from the intersection point between the ray and the object is determined (S 1906 ). The number of rays to be cast may be arbitrarily determined. However, as the number of rays to be cast increases, an image with less noise is obtained, but the calculation amount increases. Assume that the count value of the crossing count C of each ray to be newly cast and the value of the maximum crossing count M(λ) inherit the values of an original ray. Processes in steps S 1907  to S 1910  are repeated by the number of times corresponding to the number N of rays to be cast. 
     In step S 1907 , the directions and intensities of rays to be cast are calculated. The directions of rays suffice to be distributed uniformly in accordance with the number N of rays to be cast. The intensity P r (λ) of reflected light and the intensity P t (λ) of transmitted light at the intersection point are respectively given by equations (5) and (6), similar to the first embodiment. 
     In step S 1908 , it is determined whether the count value of the current crossing count C for the ray is 1. If the crossing count C is 1, that is, a crossing with the object determined in step S 1902  is the first crossing, the process shifts to step S 1909 . If the count value of the crossing count C is not 1, that is, a crossing with the object determined in step S 1902  is the second or subsequent crossing, the process shifts to step S 1910 . 
     In step S 1909 , the maximum crossing count M(λ) at the wavelength λ is calculated and updated in accordance with the spectral characteristics of the light source and those of the object. The method of setting the maximum crossing count M(λ) will be explained in detail. The maximum crossing count M(λ) is set for each of the reflected component and transmitted component of a ray with respect to an object contained in a scene. 
     For example, a ray having a wavelength at which the reflectance of an object is low greatly attenuates the radiance upon crossing the object. After this crossing, a change of the radiance upon crossing another object becomes relatively small. For this reason, a radiance at a small crossing count and a radiance at a large crossing count hardly differ from each other. 
     To the contrary, a ray having a wavelength at which the reflectance of an object is high hardly attenuates the radiance even upon crossing the object. After this crossing, a change of the radiance upon crossing another object may become relatively large. A radiance at a small crossing count and a radiance at a large crossing count may greatly differ from each other. Noise may occur unless this is considered. This characteristic also applies to a ray passing through an object. 
     From this, the second embodiment sets the maximum crossing count M(λ) to be large if the reflectance or transmittance is high for an object (to be referred to as the first crossing object hereinafter) first crossing a ray having a given wavelength, and small if it is low. The radiance of a light source is directly reflected in a radiance after a crossing, and thus is taken into consideration when setting the maximum crossing count M(λ). More specifically, as the intensity of a ray remaining after crossing the first crossing object is higher, the maximum crossing count M(λ) is set to be larger. In this way, the maximum crossing count M(λ) is set in accordance with the radiance of a light source and the spectral characteristics (reflection property or transmission property) of an object. The calculation amount can be reduced while suppressing generation of noise. 
     A concept of a method of setting the maximum crossing count M(λ) especially for a component reflected by an object will be explained with reference to  FIG. 14 . When an object crossing a ray at a given incident angle and exit angle has a spectral reflectance shown in  FIG. 14 , the maximum crossing count M(480) is set to be large at a wavelength of, for example, 480 nm at which the reflectance is high. In contrast, the maximum crossing count M(680) is set to be small at a wavelength of, for example, 680 nm at which the reflectance is low. 
     More specifically, the maximum crossing counts M r (λ) and M t (λ) of reflected light and transmitted light with respect to an object are respectively given by
 
 M   r (λ)= g{M (λ), f   r ( x,ω   i ,λ,ω r ), S (λ)}  (14)
 
 M   t (λ)= g{M (λ), f   t ( x,ω   i ,λ,ω r ), S (λ)}  (15)
 
Equations (14) and (15) represent the maximum crossing counts M r (λ) and M t (λ) as functions g of the maximum crossing count M(λ) initialized in step S 406 , the spectral characteristics f r  and f t  given by equations (2) and (3), and the spectral radiance S(λ) of a light source.
 
     Note that the function g suffices to be a function which increases the maximum crossing count M(λ) at a wavelength at which the reflectance or transmittance is high. For example, functions as represented by equations (16) and (17) can be used:
 
 g{M (λ), f   r ( x,ω   i ,λ,ω r ), S (λ)}= M (λ)× f   r ( x,ω   i ,λ,ω r )× S (λ)  (16)
 
 g{M (λ), f   t ( x,ω   i ,λ,ω t ), S (λ)}= M (λ)×√{ f   t ( x,ω   i ,λ,ω t )× S (λ)  (17)
 
     In step S 1910 , ray tracing is performed for the ray by recursively invoking ray tracing shown in  FIG. 13 . The processing in step S 1910  ends when no crossing object remains finally in recursive processing, or the crossing count C of the ray and objects exceeds the maximum crossing count M r (λ) or M t (λ), and a spectral radiance for the ray is calculated in step S 1914 . In other words, the spectral radiance E 0 (λ) finally calculated according to equation (4) in the recursive processing of step S 1910  becomes a spectral radiance E n  (λ) for the ray. 
     If it is determined in step S 1911  that the processes in steps S 1907  to S 1910  have ended for all rays by the number N of rays to be cast, the average of the spectral radiances E n (λ) of the respective rays is calculated (S 1912 ). The average spectral radiance value serves as a spectral radiance E x (λ) at the tracing target wavelength λ in the pixel of interest. 
     If the crossing count C has exceeded the maximum crossing count M(λ) in step S 1905  (C&gt;M(λ)), the direction and intensity of a ray when aborting ray tracing are calculated (S 1913 ). More specifically, the direction of a light source from an intersection point is set as the direction of the ray. The intensity P(λ) of the ray is set to 0 when another object exists on a straight line extending from the intersection point to the light source. When no other object exists, only the reflected component is considered in accordance with equation (18):
 
 P (λ)= P   0 (λ)× f   r ( x,ω   i ,λ,ω r )  (18)
 
     where P 0 (λ) is the intensity of a ray before crossing an object at the wavelength λ, and 
     f r  is the reflection property of the object given by equation (2). 
     After that, the spectral radiance E 0 (λ) for the wavelength λ is calculated according to equation (4) (S 1914 ), ending ray tracing for this wavelength. In calculation of step S 1914 , the intensity P(λ) of the ray that has been calculated in step S 1913 , and the light distribution characteristic Q(θ) and spectral radiance S(λ) of the light source are applied. 
     As described above, when performing spectral ray tracing, the crossing count is restricted for each wavelength of a ray in consideration of the radiance of a light source and the spectral characteristics of an object. Calculation is executed for only a minimum number of crossings at each wavelength. This can increase the calculation speed and reduce noise in a generated image. 
     Third Embodiment 
     In the second embodiment, the maximum crossing count is set for each wavelength of a ray in accordance with the radiance of a light source and the spectral properties of an object. The third embodiment increases the calculation speed and reduces noise in a generated image by setting the number of rays to be cast for each wavelength in accordance with the spectral properties of an object. 
     Note that the arrangement of an image processing apparatus in the third embodiment is the same as those shown in  FIGS. 2 and 3  in the first embodiment. The same reference numerals denote the same parts, and a description thereof will not be repeated. 
     Scene CG Image Generation Processing 
     Image generation processing of generating the CG image of a scene in an image processing apparatus  301  in the third embodiment is the same as the processing shown in  FIG. 12  in the second embodiment except for the spectral radiance calculation processing in step S 1407 . 
     Spectral Radiance Calculation Processing 
     Spectral radiance calculation processing (S 1407 ) in the third embodiment will be explained in detail with reference to  FIG. 15 . Even in the third embodiment, a plurality of rays are traced by recursively executing ray tracing shown in the flowchart of  FIG. 15 , and a spectral radiance in the pixel of interest is calculated from the average of spectral radiances calculated for respective rays. 
     Spectral radiance calculation processing for monochromatic light having the wavelength λ will be explained with reference to  FIG. 15 . A spectral radiance in the pixel of interest is actually obtained by executing the same processing for each wavelength, similar to the first and second embodiments. 
     In  FIG. 15 , processes except for processing (S 1201 ) of determining the number N of rays to be cast are the same as those in shown in  FIG. 13  in the second embodiment. 
     The third embodiment also uses the maximum crossing count M(λ) which is initialized in step S 1406 , similar to the second embodiment. However, the maximum crossing count M(λ) is fixed and is not updated regardless of the first crossing object. In other words, processes in steps S 1908  and S 1909  contained in the processing of  FIG. 13  are not included in the processing of  FIG. 15 . 
     Determination of the number N of rays to be cast in accordance with the spectral properties of an object in processing (S 1201 ) of determining the number N of rays to be cast in the third embodiment will be explained. 
     As described in the second embodiment, a ray having a wavelength at which the reflectance or transmittance of an object is low greatly changes the radiance upon crossing the object. A change of the radiance upon crossing another object becomes relatively small. For this reason, a radiance obtained when the number of rays to be newly cast from the intersection point is small, and a radiance obtained when it is large are hardly different. 
     In contrast, a ray having a wavelength at which the reflectance or transmittance of an object is high hardly changes the radiance even upon crossing the object. A change of the radiance upon crossing another object may become relatively large. Thus, a radiance obtained when the number of rays to be newly cast is small, and a radiance obtained when it is large may become greatly different. Noise may occur unless this is considered. 
     For each object crossing a ray having a given wavelength, the third embodiment sets the number N(λ) of rays to be cast, to be large if the reflectance or transmittance of the object is high, and small if it is low. Similar to the second embodiment, the radiance of a light source should be considered when setting the number N(λ) of rays to be cast. More specifically, as the intensity of a ray remaining after crossing an object is higher, the number N(λ) of rays to be cast is set to be larger. In this fashion, the number N(λ) of rays to be cast is set in accordance with the radiance of a light source and the spectral properties (reflection property or transmission property) of an object. The calculation amount can be reduced while suppressing generation of noise. 
     A concept of a method of setting the number N(λ) of rays to be cast especially for a component reflected by an object will be explained with reference to  FIG. 16 . When an object crossing a ray at a given incident angle and exit angle has a spectral reflectance shown in  FIG. 16 , the number N(480) of rays to be cast is set to be large at a wavelength of, for example, 480 nm at which the reflectance is high. In contrast, the number N(680) of rays to be cast is set to be small at a wavelength of, for example, 680 nm at which the reflectance is low. 
     More specifically, the numbers N r (λ) and N t (λ) of rays to be cast for reflected light and transmitted light with respect to an object are respectively given as functions h of the spectral properties f r  and f t  given by equations (2) and (3), and the spectral radiance S(λ) of a light source:
 
 N   r (λ)= h{f   r ( x,ω   i ,λ,ω r ), S (λ)}  (19)
 
 N   t (λ)= h{f   t ( x,ω   i ,λ,ω t ), S (λ)}  (20)
 
     Note that the function h suffices to be a function which increases the number N(λ) of rays to be cast at a wavelength at which the reflectance is high. For example, functions as represented by equations (21) and (22) can be used:
 
 h{f   r ( x,ω   i ,λ,ω r ), S (λ)}= R max× f   r ( x,ω   i ,λ,ω r )× S (λ)  (21)
 
 h{f   t ( x,ω   i ,λ,ω t ), S (λ)}= R max×√{ f   t ( x,ω   i ,λ,ω t )× S (λ)  (22)
 
     In equations (21) and (22), Rmax is a constant having an arbitrary value, and is the number of rays to be cast when the reflectance or transmittance is 1. As Rmax increases, an image with less noise is obtained, but the calculation amount increases. 
     In the third embodiment, new rays are cast from an intersection point by the number N(λ) of rays to be cast that is set in accordance with the radiance of a light source and the spectral properties of an object. By repetitively tracing the ray path, a radiance in the pixel of interest is calculated. 
     As described above, when performing spectral ray tracing, the number N(λ) of rays to be cast is set by taking account of the radiance of a light source and the spectral properties of an object. Calculation is executed for only a minimum number of rays at each wavelength. This can increase the calculation speed and reduce noise in a generated image. 
     Note that the processing in the third embodiment can also be executed at the same time as that in the second embodiment. The maximum crossing count M(λ) and the number N(λ) of rays to be cast suffice to be set for each wavelength of a ray in accordance with the radiance of a light source and the spectral properties of an object. 
     Modification of Second and Third Embodiments 
     In the second and third embodiments, the radiance of a light source and the reflection property and transmission property of an object have wavelength dependence, and these properties have spectral properties. However, the present invention is not limited to an application to ray tracing using spectral information, and is also applicable to ray tracing for one color having only density information, or ray tracing for three, R, G, and B colors. In this case, when the reflection property or transmission property of an object is low, the number of rays to be cast or the maximum crossing count is set to be small, increasing the processing speed. When the reflection property or transmission property of an object is high, the number of rays to be cast or the maximum crossing count is set to be large, giving priority to noise reduction rather than an increase in processing speed. 
     Other Embodiments 
     Aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment(s), and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment(s). For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (for example, computer-readable medium). 
     While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions. 
     This application claims the benefit of Japanese Patent Application No. 2011-202333, filed Sep. 15, 2011, which is hereby incorporated by reference herein in its entirety.