Abstract:
An image merging apparatus extracts noise included in a background image, and adds the noise to a CG image, thereby producing the merged image of the CG image and the background image. It can solve a problem of a conventional image merging apparatus in that although it can suppress the Mach band taking place in an edge segment, it brings about a mismatched feeling between the CG image and the background image when combining the shaded CG image and background image because it lacks a device for combining them taking account of the color difference between the CG image and the background image, or of the noise included in the background image.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention relates to an image merging apparatus and an image merging method for producing a merged image of a CG image (computer graphics image) and a background image.  
           [0003]    2. Description of Related Art  
           [0004]    [0004]FIG. 10 is a block diagram showing a configuration of a conventional shading apparatus disclosed in Japanese patent application laid-open No. 8-212384/1996. In this figure, the reference numeral  1  designates a vertex luminance extractor for calculating color or luminance at vertices of a polygon to be shaded and its neighboring polygons; and  2  designates a luminance interpolator for calculating color or luminance inside the target polygon.  
           [0005]    Next, the operation of the conventional shading apparatus will be described.  
           [0006]    Receiving the normal vectors of the vertices of the target polygon to be shaded and its neighboring polygons, the vertex luminance extractor  1  calculates the color or luminance of the vertices from the normal vectors.  
           [0007]    More specifically, assume that ΔI 1 I 2 I 3  is a target polygon in FIG. 11, then ΔI 1 I 3 I 4  becomes a neighboring polygon. The vertex luminance extractor  1  calculates the color or luminance at vertices of the target polygon ΔI 1 I 2 I 3  and the neighboring polygon ΔI 1 I 3 I 4 , or the luminance in the ΔI 1 I 2 I 3  and ΔI 1 I 3 I 4 .  
           [0008]    When the vertex luminance extractor  1  calculates the color or luminance of the vertices of the target polygon and the neighboring polygon, the luminance interpolator  2  calculates the color or luminance inside the target polygon using the calculated results.  
           [0009]    Specifically, it calculates the color or luminance I S1 , I S2  and I S3  at intersections of a scanning line  3  and the edges I 1 I 2  I 1 I 3  and I 1 I 4  of the target polygon ΔI 1 I 2 I 3  and its neighboring polygon ΔI 1 I 3 I 4 .  
           [0010]    Subsequently, using the color or luminance at the three intersections, it calculates the color or luminance I at an internal point (X,Y) of the target polygon ΔI 1 I 2 I 3  as follows: 
           I S1 =I 1 (Y S −Y 2 )/(Y 1 −Y 2 )+I 2 (Y 1 −Y S )/(Y 1 −Y 2 ) 
           I S2 =I 1 (Y S −Y 3 )/(Y 1 −Y 3 )+I 3 (Y 1 −Y S )/(Y 1 −Y 3 ) 
           I S3 =I 1 (Y S −Y 4 )/(Y 1 −Y 4 )+I 4 (Y 1 −Y S )/(Y 1 −Y 4 ) 
           I=I S1 ×Z 1 +I S2 ×Z 2 +I S3 ×Z 3   
           [0011]    where 
           Z 1 ={(X−X S2 )/(X S1 −X S2 )}×{(X−X S3 )/(X S1 −X S3 )} 
           Z 2 ={(X−X S1 )/(X S2 −X S1 )}×{(X−X S3 )/(X S2 −X S3 )} 
           Z 3 ={(X−X S1 )/(X S3 −X S1 )}×{(X−X S2 )/(X S3 −X S2 )} 
           [0012]    This method can avoid sharp changes in the gradient of the color or luminance on the edge segment I 1 I 3 , preventing the Mach band to take place on the edge segment I 1 I 3 .  
           [0013]    Although the conventional shading apparatus with the foregoing configuration can suppress the Mach band on the edge segment I 1 I 3 , it presents a problem of producing a mismatched feeling between a CG image and a background image when merging the shaded CG image with the background image. This is because the conventional apparatus lacks a means for combining them considering the color difference between the CG image and background image, or noise Contained in the background image.  
           [0014]    This will be described in more detail.  
           [0015]    A composite picture is often produced by taking a picture of an object or scene in a real world to be used as a background, and by combining it with an image produced by the CG technique. For example, a simulation is often made to see whether a building or bridge which will be built from now matches the present scene of the spot, or to confirm whether the color of a new refrigerator to be installed in a room matches the room. In such cases, although the picture of the object or scene taken includes ambient noise, the image produced by the CG has a simple color tone without noise. Besides, since the color tone of the background picture can vary depending on the weather or time it is taken, it can differ from the image produced by the CG in the color tone, making it difficult to combine the background picture with the CG image. Although the conventional shading apparatus can suppress the Mach band, a sharp change in the gradient of the color or luminance involved in the shading, the color tone of the resultant image is usually monotonous including no ambient noise. In addition, it does not consider the color difference between the two images. Thus, the color difference between the CG image and the background presents a problem when they are combined.  
         SUMMARY OF THE INVENTION  
         [0016]    The present invention is implemented to solve the foregoing problems. It is therefore an object of the present invention to provide an image merging apparatus and an image merging method capable of implementing natural merging of a CG image and a background image without bringing about any mismatched feeling.  
           [0017]    According to a first aspect of the present invention, there is provided an image merging apparatus for merging a CG (computer graphics) image and its background image to output a merged image, the image merging apparatus comprising: characteristic information output means for outputting information about a characteristic at least of the background image; and merged image producing means for producing the merged image of the CG image and the background image by adding output information of the characteristic information output means to the CG image.  
           [0018]    Here, the characteristic information output means may comprise a noise extractor for extracting noise from the background image; and the merged image producing means may comprise a noise add-on section for adding the noise extracted by the noise extractor to the CG image to produce the merged image of the CG image and the background image.  
           [0019]    The characteristic information output means may comprise a noise generator for generating noise corresponding to noise included in the background image; and the merged image producing means may comprise a noise add-on section for adding the noise generated by the noise generator to the CG image to produce the merged image of the CG image and the background image.  
           [0020]    The characteristic information output means may comprise a color difference calculating section for calculating color difference between the CG image and the background image; and the merged image producing means may comprise color difference effecting section for causing the color difference calculated by the color difference calculating section to be reflected in at least one of the CG image and the background image, thereby producing the merged image of the CG image and the background image.  
           [0021]    The noise add-on section may utilize the CG image that is being generated as an image to be processed.  
           [0022]    The color difference calculating section and the color difference effecting section may utilize the CG image that is being generated as an image to be processed.  
           [0023]    The image merging apparatus may use one of a still image and moving images as the background image.  
           [0024]    According to a second aspect of the present invention, there is provided an image merging method of merging a CG (computer graphics) image and its background image to output a merged image, the image merging method comprising the steps of: outputting information about a characteristic at least of the background image; and producing the merged image of the CG image and the background image by adding the information about the characteristic to the CG image.  
           [0025]    Here, the step of outputting information may extract noise from the background image; and the step of producing the merged image may add the noise extracted to the CG image to produce the merged image of the CG image and the background image.  
           [0026]    The step of outputting information may generate noise corresponding to noise included in the background image; and the step of producing the merged image may add the noise generated to the CG image to produce the merged image of the CG image and the background image.  
           [0027]    The step of outputting information may calculate color difference between the CG image and the background image; and the step of producing the merged image may cause the color difference calculated to be reflected in at least one of the CG image and the background image, thereby producing the merged image of the CG image and the background image.  
           [0028]    The step of producing the merged image may utilize the CG image that is being generated as an image to be processed.  
           [0029]    The image merging method may utilize the CG image that is being generated as an image to be processed.  
           [0030]    The image merging method may use one of a still image and moving images as the background image. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0031]    [0031]FIG. 1 is a block diagram showing a configuration of an embodiment 1 of the image merging apparatus in accordance with the present invention;  
         [0032]    [0032]FIG. 2 is a flowchart illustrating the image merging method of the embodiment 1;  
         [0033]    [0033]FIG. 3 is a diagram illustrating an example of a background image;  
         [0034]    [0034]FIG. 4 is a block diagram showing a configuration of an embodiment 2 of the image merging apparatus in accordance with the present invention;  
         [0035]    [0035]FIG. 5 is a block diagram showing a configuration of an embodiment 3 of the image merging apparatus in accordance with the present invention;  
         [0036]    [0036]FIG. 6 is a flowchart illustrating the image merging method of the embodiment 3;  
         [0037]    [0037]FIG. 7 is a block diagram showing a configuration of an embodiment 4 of the image merging apparatus in accordance with the present invention;  
         [0038]    [0038]FIG. 8 is a block diagram showing a configuration of an embodiment 5 of the image merging apparatus in accordance with the present invention;  
         [0039]    [0039]FIG. 9 is a diagram illustrating a method of shading a triangle that constitutes basic CG shape data;  
         [0040]    [0040]FIG. 10 is a block diagram showing a configuration of a conventional shading apparatus; and  
         [0041]    [0041]FIG. 11 is a diagram illustrating a shading method. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0042]    The invention will now be described with reference to the accompanying drawings.  
         [0043]    EMBODIMENT 1  
         [0044]    [0044]FIG. 1 is a block diagram showing a configuration of an embodiment 1 of the image merging apparatus in accordance with the present invention. In this figure, the reference numeral  11  designates a CG image generated by a computer graphics technique;  12  designates a background image, a still image such as a picture;  13  designates other background images, moving images such as video images;  14  designates a selector for selecting either the background image  12  or the background images  13 ;  15  designates a color matching section for matching color tone of the background image  12  or images  13  selected by the selector  14  with that of the CG image  11 ;  16  designates a noise extractor for extracting noise included in the background image  12  or images  13  selected by the selector  14 ;  17  designates a noise add-on section for adding the noise extracted by the noise extractor  16  to the CG image  11 , thereby outputting a merged image  18  of the CG image  11  and the background image  12  or images  13 ; and  18  designates the merged image of the CG image  11  and the background image  12  or images  13 .  
         [0045]    Next, the operation of the present embodiment 1 will be described with reference to FIG. 2 which is a flowchart illustrating the image merging method of the embodiment 1.  
         [0046]    First, the selector  14  selects either the background image  12  or images  13  (step ST 1 ). Specifically, the background image  12  consisting of a still picture and the background images  13  consisting of moving images are prepared in advance, and one of them is selected by a program, a menu selector or a switch. It is also possible to prepare one of them from the beginning, in which case the selector  14  can be removed. In the present embodiment 1, it is assumed that the selector  14  selects the background image  12  for convenience of explanation.  
         [0047]    When the selector  14  selects the background image  12 , the noise extractor  16  of the color matching section  15  extracts the noise from the background image  12  (step ST 2 ).  
         [0048]    [0048]FIG. 3 is a diagram illustrating an example of the background image  12 .  
         [0049]    As illustrated in FIG. 3, the background image  12  has an image size of M (columns) × N (rows), and the pixel value at a given point (X,Y) is I(X,Y), where X denotes the horizontal coordinate and Y denotes the vertical coordinate.  
         [0050]    As an example, it is assumed that I(X, Y) consists of RGB (red, green and blue), each of which consists of eight bits, and that least significant two bits of each eight bits represent image noise.  
         [0051]    The noise of each of the RGB components I R (X, Y), I G (X, Y) and I B (X,Y) of the I(X,Y) is expressed as follows: 
         noise of R component: I R (X/Y) % 4 
         noise of G component: I G (X/Y) % 4 
         noise of B component: I B (X/Y) % 4  (1) 
         [0052]    where % is an operator for obtaining a remainder. Thus, A % B represents the remainder when A is divided by B.  
         [0053]    Generally, when least significant b bits represent the noise, the noise component of the I(X,Y) is represented as I (X,Y) % (2 Λ   b ), where is an operator representing the power. Thus, A Λ B represents the B-th power of A.  
         [0054]    When the noise extractor  16  extracts the noise from the background image  12 , the noise add-on section  17  adds the noise to the CG image  11 , and outputs the merged image  18  of the CG image  11  and the background image  12  (step ST 3 ).  
         [0055]    This will be described in more detail.  
         [0056]    To add the noise extracted by the noise extractor  16  to the CG image  11 , it is necessary to extract a portion unrelated to the noise from the CG image  11 .  
         [0057]    Let us assume that the CG image  11  has an image size of M (columns)×N (rows) like the background image  12 , and is composed of RGB components, each of which consists of eight bits, that C(X,Y) represents the pixel value at a given point (X,Y) of the CG image  11 , and that the RGB components of the C(X,Y) are denoted as C R (X,Y), C G (X,Y) and C B (X,Y), respectively. As described above, since the least significant two bits are assumed to represent noise in the present embodiment 1, the portion unrelated too the noise can be calculated from the C(X,Y) as follows: 
         noise of R component: [C R ( X,Y )/4]×4 
         noise of G component: [C G ( X,Y )/4]×4 
         noise of B component: [C B ( X,Y )/4]×4  (2) 
         [0058]    where [A] represents an integer-valued function representing the nearest integer obtained by dropping the fractional portion of the number. Generally, the component unrelated to the nose of the C(X,Y) is represented as [C(X,Y)/(2 Λ   b )]×(2 Λ   b ) when the least significant b bits indicate the noise.  
         [0059]    Thus, the noise-unrelated component of the CG image  11  and the noise component of the background image  12  can be obtained by the foregoing calculations. Accordingly, by summing them up, the noise of the background image  12  can be added to the CG image  11 .  
         [0060]    The resultant merged image  18  produced by the noise add-on section  17  can be represented as follows:  
         [0061]    R component of merged image  18 : 
         [C R ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I R ( X,Y )%(2 Λ   b ) 
         [0062]    G component of merged image  18 : 
         [C G ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I G ( X,Y )%(2 Λ   b ) 
         [0063]    B component of merged image  18 : 
         [C B ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I B ( X,Y )%(2 Λ   b )  (3) 
         [0064]    As described above, the present embodiment 1 is configured such that it extracts the noise from the background image  12 , and adds it to the CG image  11  to produce the merged image  18  of the CG image  11  and the background image  12 . Therefore, it offers an advantage of being able to implement natural merging of the CG image  11  and the background image  12  without bringing about the mismatched feeling.  
         [0065]    EMBODIMENT 2  
         [0066]    [0066]FIG. 4 is a block diagram showing a configuration of an embodiment 2 of the image merging apparatus in accordance with the present invention. In FIG. 4, the same reference numerals designate the same or like portions to those of FIG. 1, and the description thereof is omitted here.  
         [0067]    In FIG. 4, the reference numeral  19  designates a noise generator for generating noise corresponding to the noise included in the background image  12  or images  13 .  
         [0068]    Next, the operation of the present embodiment 2 will be described.  
         [0069]    Although the foregoing embodiment 1 comprises the noise extractor  16  for extracting the noise included in the background image  12  or images  13 , it is not essential. The present embodiment 2 comprises the noise generator  19  for generating the noise corresponding to the noise included in the background image  12  or images  13 . Here, the noise generator  19  can generate the noise with or without utilizing the background image  12  or images  13 .  
         [0070]    First, a method will be described in which the noise generator  19  produces noise without using the background image  12  or images  13 .  
         [0071]    Using a function F that generates decimal fractions at random from 0.0 to 1.0, the noise generated can be defined by [F×(2 Λ   b )]. Therefore, according to equations (3), the individual components of the merged image  18  can be represented as follows:  
         [0072]    R component of merged image  18 : 
         [C R ( X,Y )/(2 Λ   b )]×(2 Λ   b )+[F×(2 Λ   b )] 
         [0073]    G component of merged image  18 : 
         [C G ( X,Y )/(2 Λ   b )]×(2 Λ   b )+[F×(2 Λ   b )] 
         [0074]    B component of merged image  18 : 
         [C B ( X,Y )/(2 Λ   b )]×(2 Λ   b )+[F×(2 Λ   b )]  (4) 
         [0075]    Since the function F generates a different value each time activated, all the noise values of the components differ from each other. To insert the same noise value intentionally, the function F is activated once, and the generated value is held to be used repeatedly. In this case, the following notations hold.  
         [0076]    R component of merged image  18 : 
         [C R ( X,Y )/(2 Λ   b )]×(2 Λ   b )+NOISE 
         [0077]    G component of merged image  18 : 
         [C G ( X,Y )/(2 Λ   b )]×(2 Λ   b )+NOISE 
         [0078]    B component of merged image  18 : 
         [C B ( X,Y )/(2 Λ   b )]×(2 Λ   b )+NOISE 
         [0079]    where NOISE=[F×(2 Λ   b )]  (5) 
         [0080]    Although this example generates the noise utilizing the random function, any functions are applicable as long as they can define noise. For example, trigonometric functions, exponential functions and the like, or the combinations thereof can be used. Alternatively, the combinations of these functions and the random function are also possible.  
         [0081]    Next, the method will be described when the noise generator  19  produces noise using the background image  12 .  
         [0082]    The noise generator  19  extracts the noise components from the individual components of the background image  12  according to equation (1), and examines the characteristics of the noise components such as the mean, variance and periodicity.  
         [0083]    Then, the noise generator  19  generates a function matching the noise characteristics by combining the random function, trigonometric functions, exponential functions and the like. The generated function basically corresponds to the pixel position of the background image  12 , and is represented as G(X,Y) An example of the noise generating functions obtained for the respective RGB components is as follows: 
         G R ( X,Y )=M R ×(Sin(X)+Sin(Y))/2 
         G G ( X,Y )=M G ×(Sin(X)+Sin(Y))/2 
         G B ( X,Y )=M B ×(Sin(X)+Sin(Y))/2  (6) 
         [0084]    where  
         [0085]    M R : mean value of noise component of I R (X,Y)  
         [0086]    M G : mean value of noise component of I G (X,Y)  
         [0087]    M B : mean value of noise component of I B (X,Y)  
         [0088]    Using these functions allows the components of the merged image  18  to be expressed as follows:  
         [0089]    R component of merged image  18 : 
         [C R (X,Y)/(2 Λ   b )]×(2 Λ   b )+G R ( X,Y ) 
         [0090]    G component of merged image  18 : 
         [C G ( X,Y )/(2 Λ   b )]×(2 Λ   b )+G G ( X,Y ) 
         [0091]    B component of merged image  18 : 
         [C B ( X,Y )/(2 Λ   b )]×(2 Λ   b )+G B ( X,Y )  (7) 
         [0092]    Although the noise generator  19  generates the noise utilizing the background image  12 , it can generate the noise in the same manner using the background images  13 , the moving images, instead of the background image  12 . In this case, however, since the background images  13  includes multiple images, an increasing number of factors that express the noise characteristics such as correlation between background images and the mean values with regard to all the images will offer a wide choice of options of the noise generating function.  
         [0093]    According to the present embodiment 2, it can generate noise without using the background image  12  or images  13 . When using the background image  12  or images  13 , it can generate the noise taking account of the noise characteristics of the background image  12  or images  13 , which allows natural merging of the CG image  11  and the background image  12  or images  13 .  
         [0094]    EMBODIMENT 3  
         [0095]    [0095]FIG. 5 is a block diagram showing a configuration of an embodiment 3 of the image merging apparatus in accordance with the present invention. In this figure, the same reference numerals designate the same or like portions to those of FIG. 1, and the description thereof is omitted here.  
         [0096]    In FIG. 5, the reference numeral  20  designates a color difference calculating section for calculating the color difference between the CG image  11  and the background image  12  or images  13  selected by the selector  14 ; and  21  designates a color difference effecting section for causing the color difference calculated by the color difference calculating section  20  to be reflected in the CG image  11 , or the background image  12  or images  13 , thereby producing the merged image  18  of the CG image  11  and the background image  12  or images  13 .  
         [0097]    Next, the operation of the present embodiment 3 will be described with reference to FIG. 6, a flowchart illustrating the image merging method in the present embodiment 3.  
         [0098]    First, the selector  14  selects either the background image  12  or the background images  13  (step ST 11 ). For convenience of explanation, the present embodiment 3 assumes that the selector  14  selects the background image  12 .  
         [0099]    When the selector  14  selects the background image  12 , the color difference calculating section  20  calculates the color difference between the CG image  11  and the background image  12  (step ST 12 ).  
         [0100]    Specifically, it calculates the mean values of the RGB components I R (X,Y), I G (X,Y) and I B (X,Y) of the background image  12  in the neighborhood of the point (X,Y), and the mean values of the RGB components C R (X,Y), C G (X,Y) and C B (X,Y) of the CG image  11  in the neighborhood of the point (X,Y), and then obtains the differences between the mean values as the color difference. 
         D R ( X,Y )=MI R ( X,Y )−MC R ( X,Y ) 
         D G ( X,Y )=MI G ( X,Y )−MC G ( X,Y ) 
         D B ( X,Y )=MI B ( X,Y )−MC B ( X,Y )  (8) 
         [0101]    where  
         [0102]    MI R ( X,Y ) : mean value of I R  in m×n neighborhood of ( X,Y )  
         [0103]    MI G ( X,Y ) : mean value of I G  in m×n neighborhood of ( X,Y )  
         [0104]    MI B ( X,Y ) : mean value of I B  in m×n neighborhood of ( X,Y )  
         [0105]    MC R ( X,Y ) : mean value of C R  in m×n neighborhood of ( X,Y )  
         [0106]    MC G ( X,Y ) : mean value of C G  in m×n neighborhood of ( X,Y )  
         [0107]    MC B ( X,Y ) : mean value of C B  in m×n neighborhood of ( X,Y )  
         [0108]    When the color difference calculating section  20  calculates the color difference, the color difference effecting section  21  causes the color difference to be reflected in the CG image  11  or the background image  12 , thereby producing the merged image  18  of the CG image  11  and the background image  12  (step ST 13 )  
         [0109]    R component of CG image  11  merged with background image  12 : 
         C R ( X,Y )+D R ( X,Y ) 
         [0110]    G component of CG image  11  merged with background image  12 : 
         C G ( X,Y )+D G ( X,Y ) 
         [0111]    B component of CG image  11  merged with background image  12 : 
         C B ( X,Y )+D B ( X,Y )  (9) 
         [0112]    R component of background image  12  merged with CG image  11 : 
         I R ( X,Y )−D R ( X,Y ) 
         [0113]    G component of background image  12  merged with CG image  11 : 
         I G ( X,Y )−D G ( X,Y ) 
         [0114]    B component of background image  12  merged with CG image  11 : 
         I B ( X,Y )−D B ( X,Y )  (10) 
         [0115]    Here, if each pixel value obtained as a result of the calculation exceeds the maximum value of the pixel values, it is set at the maximum value, whereas if it is less than the minimum value thereof it is set at the minimum value.  
         [0116]    Incidentally, it is enough for the color difference effecting section  21  to calculate one of equations (9) and (10) without calculating both of them. Usually, since the CG image  11  is adjusted to the color tone of the background image  12 , equation (9) is calculated. In contrast, when the background image  12  is adjusted to the color tone of the CG image  11 , equation (10) is calculated.  
         [0117]    Although the present embodiment 3 utilizes the background image  12 , it can also use the background images  13 , the moving images, in place of the background image  12  in the same manner. In this case, since the background images  13  include multiple pictures, the color difference can be calculated for each background image, or for a set of multiple background images.  
         [0118]    As described above, the present embodiment 3 is configured such that it calculates the color difference between the CG image  11  and the background image  12 , and causes the color difference to be reflected in the CG image  11  or the background image  12 . As a result, it can offer an advantage of being able to implement natural merging of the CG image  11  and the background image  12  without bringing about the mismatched feeling.  
         [0119]    EMBODIMENT 4  
         [0120]    Although the foregoing embodiments 1 and 2 add noise to the CG image  11 , and the foregoing embodiment 3 causes the color difference to be reflected in the CG image  11  or the background image  12 , both the noise addition processing and color difference reflecting processing can be carried out on the CG image  11  and background image  12 .  
         [0121]    More specifically, when the selector  14  selects the background image  12 , and the noise extractor  16  extracts the noise from the background image  12 , the processings according to equations (3), (8) and (9) are carried out. The RGB components of the CG image  11  merged with the background image  12  are described as follows: 
         [0122]    R component of merged CG image: 
         [C R ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I R ( X,Y ) %(2 Λ   b )+D R ( X,Y ) 
         [0123]    G component of merged CG image: 
         [C G ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I G ( X,Y ) %(2 Λ   b )+D G ( X,Y ) 
         [0124]    B component of merged CG image: 
         [C B ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I B ( X,Y ) %(2 Λ   b )+D B ( X,Y )  (11) 
         [0125]    When the noise generator  19  is used instead of the noise extractor  16 , a similar description is obtained according to equations (4)-(9). When the background images  13  consisting of the moving images are used in place of the background image  12 , the basic scheme is the same in spite of the plurality of images.  
         [0126]    According to the present embodiment 4, it not only adds the noise extracted from the background image to the CG image  11 , but also calculates the color difference between the CG image  11  and the background image  12  or the like to adjust the color tone of the CG image  11  to that of the background image  12  or the like. Thus, it can implement more natural merging of the CG image  11  with the background image  12  or the like.  
         [0127]    EMBODIMENT 5  
         [0128]    [0128]FIG. 8 is a block diagram showing a configuration of an embodiment 5 of the image merging apparatus in accordance with the present invention. In this figure, the same reference numerals designate the same or like portions to those of FIG. 7, and the description thereof is omitted here.  
         [0129]    In FIG. 8, the reference numeral  31  designates CG shape data from which the CG image is generated;  32  designates a shading section needed for generating the CG image; and  33  designates a color interpolator that performs a basic operation for generating the color of the CG image.  
         [0130]    Although the color matching section  15  is applied to the CG image  11  that has already been produced in the foregoing embodiments 1-4, it can be incorporated into a shading section  32  for generating the CG image  11  instead. Here, an example will be described in which the color matching section  15  is incorporated into the shading section  32  for generating the CG image.  
         [0131]    Next, the operation of the present embodiment 5 will be described.  
         [0132]    [0132]FIG. 9 is a diagram illustrating a shading method of a triangle, a basic element of the CG shape data  31 . In this figure, the reference numeral  34  designates a scanning line. Usually, colors at the vertices C 1 , C 2  and C 3  are calculated from the normal vectors, light source vectors and color attributes of the triangle (ambient light component, diffuse reflection light component and mirror reflection light component) at the individual vertices (for details, see, Japanese patent application laid-open No. 8-212384/1996 described as the prior art, which is incorporated here by reference).  
         [0133]    Subsequently, the colors (C S1 , C S2 ) at the intersections of the scanning line  34  and the edges C 1 C 2  and C 1 C 3  are calculated as follows by the linear interpolation between C 1  and C 2 , and C 1  and C 3 . 
         C S1 ( X,Y )={C 1 (Y−Y 2 )+C 2 (Y 1 −Y)}/(Y 1 −Y 2 ) 
         C S2 ( X,Y )={C 1 (Y−Y 3 )+C 3 (Y 1 −Y)}/(Y 1 −Y 3 )  (12) 
         [0134]    Then, the color C at a given point (X,Y) inside the triangle is obtained as follows from the C S1  and C S2 . 
         C( X,Y )={C S1 (X S2 −X)+C S2 (X−X S1 )}/(X S2 −X S1 )  (13) 
         [0135]    where the color is handled in its entirety without resolving it into the RGB components in equations (12) and (13).  
         [0136]    Here, the mean values in the m×n neighborhood of the point (X,Y) in equation (8) are calculated as follow: 
         MI R ( X,Y ): mean value of I R  in m×n neighborhood of ( X,Y ) 
         MI G ( X,Y ): mean value of I G  in m×n neighborhood of ( X,Y ) 
         MI B ( X,Y ): mean value of I B  in m×n neighborhood of ( X,Y ) 
         where m=MAX(X 1 , X 2 , X 3 )-MIN(X 1 , X 2 , X 3 ) 
         n=MAX(Y 1 , Y 2 , Y 3 )-MIN(Y 1 , Y 2 , Y 3 )  (14) 
         [0137]    where MAX is a function for obtaining the maximum value of the arguments, and MIN is a function for obtaining the minimum value of the arguments. 
         MC R ( X,Y )=(C 1R +C 2R +C 3R )/3 
         MC G ( X,Y )=(C 1G +C 2G +C 3G )/3 
         MC B ( X,Y )=(C 1B +C 2B +C 3B )/3 
         [0138]    From the foregoing results, the merged image  18  is obtained as the following expression ( 15 ), when the color matching section  15  is incorporated into the shading section  32 .  
         [0139]    R component of merged CG image: 
         [C R ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I R ( X,Y ) %(2 Λ   b )+D R ( X,Y ) 
         [0140]    G component of merged CG image: 
         [C G ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I G ( X,Y ) %(2 Λ   b )+D G ( X,Y ) 
         [0141]    B component of merged CG image: 
         [C B ( X,Y )/(2 Λ   b )]×(2 Λ   b )+I B ( X,Y ) %(2 Λ   b )+D B ( X,Y )  (15) 
         [0142]    where  
         [0143]    C R (X,Y)={C S1R (X S2 −X)+C S2R (X−X S1 )}/(X S2 −X S1 )  
         [0144]    C G (X,Y)={C S1G (X S2 −X)+C S2G (X−X S1 )}/(X S2 −X S1 )  
         [0145]    C B (X,Y)={C S1B (X S2 −X)+C S2B (X−X S1 )}/(X S2 −X S1 )  
         [0146]    C S1R (X,Y)={C 1R (Y−Y 2 )+C 2R (Y 1 −Y)}/(Y 1 −Y 2 )  
         [0147]    C S1G (X,Y)={C 1G (Y−Y 2 )+C 2G (Y 1 −Y)}/(Y 1 −Y 2 )  
         [0148]    C S1B (X,Y)={C 1B (Y−Y 2 )+C 2B (Y 1 −Y)}/(Y 1 −Y 2 )  
         [0149]    C S2R (X,Y)={C 1R (Y−Y 3 )+C 3R (Y 1 −Y)}/(Y 1 −Y 3 )  
         [0150]    C S2G (X,Y)={C 1G (Y−Y 3 )+C 3G (Y 1 −Y)}/(Y 1 −Y 3 )  
         [0151]    C S2B (X,Y)={C 1B (Y−Y 3 )+C 3B (Y 1 −Y)}/(Y 1 −Y 3 )  
         [0152]    I R (X,Y): R component of background image  12   
         [0153]    I G (X,Y): G component of background image  12   
         [0154]    I B (X,Y): B component of background image  12   
         [0155]    D R (X,Y)=MI R (X,Y)−MC R (X,Y)  
         [0156]    D G (X,Y)=MI G (X,Y)−MC G (X,Y)  
         [0157]    D B (X,Y)=MI B (X,Y)−MC B (X,Y)  
         [0158]    MI R (X,Y): mean value of I R  in m×n neighborhood of (X,Y)  
         [0159]    MI G (X,Y): mean value of I G  in m×n neighborhood of (X,Y)  
         [0160]    MI B (X,Y): mean value of I B  in m×n neighborhood of (X,Y)  
         [0161]    where m=MAX(X 1 , X 2 , X 3 )−MIN(X 1 , X 2 , X 3 )  
         [0162]    n=MAX(Y 1 , Y 2 , Y 3 )−MIN(Y 1 , Y 2 , Y 3 )  
         [0163]    MC R (X,Y)=(C 1R +C 2R +C 3R )/3  
         [0164]    MC G (X,Y)=(C 1G +C 2G +C 3G )/3  
         [0165]    MC B (X,Y)=(C 1B +C 2B +C 3B )/3  
         [0166]    As described above, the color matching section  15  can not only be applied to the CG image  11  that has already been generated, but also be incorporated into the shading section  32  for generating the CG image  11 . Thus, the present embodiment 5 can quickly implement the merged image  18  of the images such as the CG image  11  and the background image  12 , without impairing the natural feeling.