Patent Publication Number: US-2022222782-A1

Title: Image processing device, method, image reading device, and recording medium

Description:
TECHNICAL FIELD 
     The present invention relates to an image processing device, a method, an image reading device, a program, and a recording medium. 
     In particular, the present invention relates to image processing for, in an image reading device that reads an original in different wavelength bands and generates a multispectral image constituted by multiple band images, improving the S/N ratio of the multispectral image. 
     BACKGROUND ART 
     In an image reading device that obtains multiple band images by reading an original in different wavelength bands, depending on the combination of the spectrum of the light source and the spectral sensitivity of the image sensor, there is a band image of a wavelength band in which a sufficient signal amount is not obtained. For example, band images obtained by illumination using ultraviolet light sources or near-infrared light sources often have lower S/N ratios than band images of visible light bands. 
     As another example of generating a multispectral image, in imaging the ground or the like from an optical satellite or aircraft, it is widely practiced to simultaneously obtain both a high-resolution panchromatic image and a low-resolution multispectral image for the same object. 
     In a system that simultaneously obtains both a low-resolution multispectral image and a high-resolution panchromatic image, a pan-sharpening process that combines the two images to generate a high-resolution color image is performed. 
     In the pan-sharpening process, removal of noise included in the multispectral image is a problem. 
     In removing noise of band images with low resolution and low S/N ratio by using a panchromatic image with high resolution and high S/N ratio, a noise removal device described in Patent Literature 1 performs multiresolution decomposition on the panchromatic image, performs multiresolution decomposition on the band images, corrects the decomposed components of the panchromatic image by using the decomposed components of the band images, and reconstructs a panchromatic image by using the corrected decomposed components and another decomposed component. 
     CITATION LIST 
     Patent Literature 
     Patent Literature 1: International Publication No. 2015/037189 (paragraph 0024) 
     SUMMARY OF INVENTION 
     Technical Problem 
     As described above, in Patent Literature 1, noise correction is performed by a combination of a high-resolution panchromatic image and a multispectral image. In some cases, only a low-resolution multispectral image is obtained, and no high-resolution image is obtained. In a case where smoothing for noise reduction is performed with only a multispectral image, when the multispectral image includes a large amount of noise, the smoothing may destroy edges. 
     The present invention is to solve the above problem, and is intended to provide an image processing device capable of, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, improving the S/N ratio of the multispectral image without destroying edges. 
     Solution to Problem 
     An image processing device of the present invention includes: a parameter calculator to analyze a reference image of the same field of view as a plurality of band images constituting a multispectral image to calculate a local variance or an edge amount as an image feature amount, and calculate a filter parameter from the image feature amount; and a filter processor to perform an edge preserving smoothing process on the band images by using the filter parameter. 
     Advantageous Effects of Invention 
     The present invention makes it possible, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, to improve the S/N ratio of the multispectral image without destroying edges. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a block diagram illustrating an image processing device of a first embodiment of the present invention. 
         FIG. 2  is a diagram illustrating a configuration example of an image reading device including the image processing device. 
         FIG. 3  is a block diagram illustrating a configuration example of an image combiner of  FIG. 1 . 
         FIG. 4  is a diagram illustrating an example of a spectral quantum efficiency of an image sensor. 
         FIG. 5  is a block diagram illustrating an image processing device of a second embodiment of the present invention. 
         FIG. 6  is a block diagram illustrating a configuration example of an image combiner of  FIG. 5 . 
         FIGS. 7A to 7C  are diagrams illustrating examples of an image correlation calculated by a correlation calculator of  FIG. 6 . 
         FIG. 8  is a block diagram illustrating an image processing device of a third embodiment of the present invention. 
         FIG. 9  is a block diagram illustrating an image processing device of a fourth embodiment of the present invention. 
         FIG. 10  is a diagram illustrating an example of a computer that implements the functions of the image processing devices of the first to fourth embodiments, together with a light source and an image sensor. 
         FIG. 11  is a flowchart illustrating a procedure of a process when the function of the image processing device of the first or second embodiment is implemented by the computer of  FIG. 10 . 
         FIG. 12  is a flowchart illustrating a procedure of a process when the function of the image processing device of the third embodiment is implemented by the computer of  FIG. 10 . 
         FIG. 13  is a flowchart illustrating a procedure of a process when the function of the image processing device of the fourth embodiment is implemented by the computer of  FIG. 10 . 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     First Embodiment 
       FIG. 1  is a block diagram illustrating an image processing device  3  of a first embodiment of the present invention.  FIG. 2  illustrates a configuration example of an image reading device including the image processing device  3  of  FIG. 1 . 
     The image reading device illustrated in  FIG. 2  includes a light source  1  and an image sensor  2 , in addition to the image processing device  3 . 
     The light source  1  is constituted by multiple light sources each having a relatively narrow band. Hereinafter, the light sources having the relatively narrow bands will be referred to as band light sources. The multiple band light sources include, for example, a light source having a visible light wavelength band, a light source having an ultraviolet wavelength band, and a light source having a near-infrared wavelength band. The multiple band light sources are controlled to sequentially illuminate an original one by one. 
     The image sensor  2  sequentially obtains multiple band images DIN by imaging the original when the original as an object is sequentially illuminated by the above multiple band light sources. In general, the multiple band images DIN have different S/N ratios. 
     The image sensor  2  may be a one-dimensional image sensor (line sensor) or a two-dimensional image sensor (area sensor). The line sensor may be a contact image sensor. 
     The image processing device  3  combines the multiple band images DIN to generate a reference image having a higher S/N ratio than each band image, analyzes the reference image to calculate a local variance or an edge amount as an image feature amount, calculates a filter parameter from the image feature amount, and uses the filter parameter to smooth the multiple band images DIN while preserving edges. 
     The image processing device  3  includes an image combiner  31 , a parameter calculator  32 , and a filter processor  33 . 
     The image combiner  31  combines the multiple band images DIN to generate a reference image MSr having a higher S/N ratio than each band image. 
     Use of a band image having a low S/N ratio in the combination may increase the noise amount of the reference image MSr and reduce the S/N ratio. To avoid reduction in the S/N ratio of the reference image MSr, in this embodiment, the combination is performed such that a combination weight of a band image whose noise amount is greater is smaller. 
     The image combiner  31  includes a noise amount calculator  311  and a weighting combiner  312 , for example, as illustrated in  FIG. 4 . 
     The noise amount calculator  311  calculates a noise amount N i  of each of the multiple band images DIN. The calculated noise amounts N i  are each represented by a scalar, for example. 
     When the image processing device forms part of the image reading device illustrated in  FIG. 2 , a signal amount of each band image can be calculated by multiplication of the quantum efficiency of the image sensor  2  in the corresponding wavelength band and the amount of illumination by the used light source in the corresponding wavelength band. The noise amount N i  can be calculated from a reciprocal of the signal amount. 
       FIG. 4  illustrates an example of the spectral quantum efficiency of the image sensor. The image sensor has a peak near a center of a wavelength band in which the image sensor is sensitive, and the quantum efficiency decreases farther from the peak position. 
     When information on the quantum efficiency of the image sensor  2  or the amount of illumination by the light source cannot be used, the noise amount N i  of each band image can be calculated by analyzing each band image DIN. 
     Examples of the method of calculating the noise amount N i  by analyzing the image include a method of calculating a local variance of each pixel in a flat region of the image and determining an average thereof. 
     In this case, it is possible to calculate a local variance of each pixel and take, as a flat region, a region formed by pixels whose local variances are not greater than a threshold or a region in which the proportion of the pixels whose local variances are not greater than a threshold is not less than a predetermined value. 
     The local variances are determined by, for example, the same calculation as Equation (3) to be described later. 
     The weighting combiner  312  generates the reference image MSr by combining the multiple band images DIN such that the combination weight of a band image whose noise amount N i  is smaller is greater. The combination is performed by weighting and adding pixel values at the same positions of the multiple band images. Specifically, the weighting combiner  312  determines weighted averages of pixel values at the same positions of the multiple band images DIN as pixel values at the same positions of the reference image MSr. 
     The weighted average for a certain pixel (pixel of interest) is determined by, for example, the calculation represented by the following Equation (1): 
     
       
         
           
             
               
                 
                   
                     M 
                     ⁢ 
                     S 
                     ⁢ 
                     
                       r 
                       ⁡ 
                       
                         ( 
                         x 
                         ) 
                       
                     
                   
                   = 
                   
                     
                       1 
                       M 
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           i 
                           ∈ 
                           M 
                         
                       
                       ⁢ 
                       
                         
                           W 
                           i 
                         
                         · 
                         
                           
                             
                               MS 
                               i 
                             
                             ⁡ 
                             
                               ( 
                               x 
                               ) 
                             
                           
                           . 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
     In Equation (1), 
     x is a coordinate representing the position of the pixel of interest, 
     M denotes the number of the band images, 
     MS i  denotes the i-th band image of the M band images, 
     MS i (x) denotes a pixel value at the position x of the band image MS i , 
     W i  denotes a combination weight for the pixel value MS i (x) of the band image MS i , and 
     MSr(x) denotes a pixel value at the position x of the reference image MSr. 
     As the combination weight W i  for the pixel value MS i (x) of each band image MS i , a normalized value of a reciprocal of the noise amount N i  of the band image MS i  is used. The normalized value of the reciprocal of the noise amount of each band image is a value obtained by dividing the reciprocal of the noise amount of the band image by an average of the reciprocals of the noise amounts of all the band images. Thus, the combination weight W i  is represented by the following Equation (2): 
     
       
         
           
             
               
                 
                   
                     
                       W 
                       i 
                     
                     = 
                     
                       
                         1 
                         
                           N 
                           i 
                         
                       
                       
                         
                           1 
                           M 
                         
                         ⁢ 
                         
                           
                             ∑ 
                             
                               i 
                               ∈ 
                               M 
                             
                           
                           ⁢ 
                           
                             1 
                             
                               N 
                               i 
                             
                           
                         
                       
                     
                   
                   . 
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
     Instead of the above method, the weighting combiner  312  may set each weight W i  to 0 or 1 by using a preset noise amount threshold and determine the weighted averages. For example, it may set the weight W i  to 0 when the noise amount is not less than the noise amount threshold and to 1 otherwise, and determine the weighted averages by using the number of band images for which the weight W i  has been set to 1, instead of M. 
     The reference image MSr obtained by the above combination has a higher S/N ratio than each band image. 
     The parameter calculator  32  analyzes the reference image MSr to calculate, as an image feature amount, a local variance or an edge amount for each pixel of the reference image MSr, and calculates filter parameters D 32  from the image feature amounts. 
     The local variance var(x) for a position x in the reference image MSr can be determined by, for example, performing the calculation represented by the following Equation (3): 
     
       
         
           
             
               
                 
                   
                     
                       v 
                       ⁢ 
                       a 
                       ⁢ 
                       
                         r 
                         ⁡ 
                         
                           ( 
                           x 
                           ) 
                         
                       
                     
                     = 
                     
                       
                         
                           1 
                           L 
                         
                         ⁢ 
                         
                           
                             ∑ 
                             
                               y 
                               ∈ 
                               
                                 Ω 
                                 ⁡ 
                                 
                                   ( 
                                   x 
                                   ) 
                                 
                               
                             
                           
                           ⁢ 
                           
                             M 
                             ⁢ 
                             S 
                             ⁢ 
                             
                               
                                 r 
                                 ⁡ 
                                 
                                   ( 
                                   y 
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                       - 
                       
                         
                           ( 
                           
                             
                               1 
                               L 
                             
                             ⁢ 
                             
                               
                                 ∑ 
                                 
                                   y 
                                   ∈ 
                                   
                                     Ω 
                                     ⁡ 
                                     
                                       ( 
                                       x 
                                       ) 
                                     
                                   
                                 
                               
                               ⁢ 
                               
                                 M 
                                 ⁢ 
                                 S 
                                 ⁢ 
                                 
                                   r 
                                   ⁡ 
                                   
                                     ( 
                                     y 
                                     ) 
                                   
                                 
                               
                             
                           
                           ) 
                         
                         2 
                       
                     
                   
                   . 
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
     In Equation (3), 
     Ω(x) denotes a local region centered at the position x, 
     L denotes the number of pixels in the local region Ω(x), 
     y is a coordinate representing a position in the local region Ω(x), and 
     MSr(y) denotes a pixel value at the position y of the reference image MSr. 
     The edge amount can be determined by, for example, a method using a bilateral weight. For example, the edge amount bw(x) for a position x in the reference image MSr can be determined by performing the calculation represented by the following Equation (4): 
     
       
         
           
             
               
                 
                   
                     b 
                     ⁢ 
                     
                       w 
                       ⁡ 
                       
                         ( 
                         x 
                         ) 
                       
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         y 
                         ∈ 
                         
                           Ω 
                           ⁡ 
                           
                             ( 
                             x 
                             ) 
                           
                         
                       
                     
                     ⁢ 
                     
                       
                         ( 
                         
                           e 
                           ⁢ 
                           x 
                           ⁢ 
                           
                             
                               p 
                               ⁡ 
                               
                                 ( 
                                 
                                   
                                     - 
                                     
                                       1 
                                       2 
                                     
                                   
                                   ⁢ 
                                   
                                     
                                       ( 
                                       
                                         
                                           d 
                                           ⁡ 
                                           
                                             ( 
                                             
                                               y 
                                               , 
                                               x 
                                             
                                             ) 
                                           
                                         
                                         
                                           σ 
                                           1 
                                         
                                       
                                       ) 
                                     
                                     2 
                                   
                                 
                                 ) 
                               
                             
                             · 
                             
                               exp 
                               ⁡ 
                               
                                 ( 
                                 
                                   
                                     - 
                                     
                                       1 
                                       2 
                                     
                                   
                                   ⁢ 
                                   
                                     
                                       ( 
                                       
                                         
                                           
                                             M 
                                             ⁢ 
                                             S 
                                             ⁢ 
                                             
                                               r 
                                               ⁡ 
                                               
                                                 ( 
                                                 y 
                                                 ) 
                                               
                                             
                                           
                                           - 
                                           
                                             M 
                                             ⁢ 
                                             S 
                                             ⁢ 
                                             
                                               r 
                                               ⁡ 
                                               
                                                 ( 
                                                 x 
                                                 ) 
                                               
                                             
                                           
                                         
                                         
                                           σ 
                                           2 
                                         
                                       
                                       ) 
                                     
                                     2 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
     In Equation (4), 
     Ω(x) denotes a local region centered at the position x, 
     y is a coordinate representing a position in the local region Ω(x), 
     d(y,x) denotes a distance between the position indicated by the coordinate y and the position indicated by the coordinate x, 
     MSr(x) denotes a pixel value at the position x of the reference image MSr, 
     MSr(y) denotes a pixel value at the position y of the reference image MSr, and 
     σ 1  and σ 2  denote constants. The constants σ 1  and σ 2  are arbitrarily determined. 
     The parameter calculator  32  calculates and outputs the filter parameters D 32  from the calculated image feature amounts. 
     For example, it may simply output the image feature amounts as the filter parameters, or may output, as the filter parameters, reciprocals of the image feature amounts or normalized values of them. 
     The filter processor  33  uses the filter parameters D 32  to smooth the multiple band images DIN while preserving edges, and generates multiple output images DOUT. 
     The multiple output images DOUT correspond to the respective multiple band images DIN. A set of the multiple output images DOUT forms a multispectral image. 
     The filter process can be performed by using a reference smoothing filter to which the filter parameters can be input. For example, a joint bilateral filter, a guided filter, or the like can be used. 
     The filtering process using a joint bilateral filter is represented by, for example, the following Equation (5): 
     
       
         
           
             
               
                 
                   
                     D 
                     ⁢ 
                     O 
                     ⁢ 
                     U 
                     ⁢ 
                     
                       T 
                       ⁡ 
                       
                         ( 
                         x 
                         ) 
                       
                     
                   
                   = 
                   
                     
                       1 
                       
                         
                           ∑ 
                           
                             y 
                             ∈ 
                             
                               Ω 
                               ⁡ 
                               
                                 ( 
                                 x 
                                 ) 
                               
                             
                           
                         
                         ⁢ 
                         
                           
                             Ws 
                             ⁡ 
                             
                               ( 
                               
                                 x 
                                 , 
                                 y 
                                 , 
                                 
                                   σ 
                                   s 
                                 
                               
                               ) 
                             
                           
                           ⁢ 
                           W 
                           ⁢ 
                           
                             c 
                             ⁡ 
                             
                               ( 
                               
                                 x 
                                 , 
                                 y 
                                 , 
                                 
                                   σ 
                                   c 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           y 
                           ∈ 
                           
                             Ω 
                             ⁡ 
                             
                               ( 
                               x 
                               ) 
                             
                           
                         
                       
                       ⁢ 
                       
                         W 
                         ⁢ 
                         
                           s 
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                               , 
                               
                                 σ 
                                 s 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         W 
                         ⁢ 
                         
                           c 
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                               , 
                               
                                 σ 
                                 c 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           
                             DIN 
                             ⁡ 
                             
                               ( 
                               y 
                               ) 
                             
                           
                           . 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
     In Equation (5), 
     x is a coordinate representing the position of the pixel of interest, 
     Ω(x) denotes a local region centered at the position x, 
     y is a coordinate representing a position in the local region Ω(x), 
     DIN(y) denotes a pixel value at the position y of the band image DIN, and 
     DOUT(x) denotes a pixel value at the position x of the output image DOUT. 
     The range of the above local region Ω(x) need not be the same as the range of the local region Ω(x) in the above Equation (3) or (4). 
     In Equation (5), Ws is a distance weight, and determined by, for example, the following Equation (6): 
     
       
         
           
             
               
                 
                   
                     
                       W 
                       ⁢ 
                       
                         s 
                         ⁡ 
                         
                           ( 
                           
                             x 
                             , 
                             y 
                             , 
                             
                               σ 
                               c 
                             
                           
                           ) 
                         
                       
                     
                     = 
                     
                       exp 
                       ⁡ 
                       
                         ( 
                         
                           
                             - 
                             
                               1 
                               2 
                             
                           
                           ⁢ 
                           
                             
                               ( 
                               
                                 
                                   d 
                                   ⁡ 
                                   
                                     ( 
                                     
                                       y 
                                       , 
                                       s 
                                     
                                     ) 
                                   
                                 
                                 
                                   σ 
                                   s 
                                 
                               
                               ) 
                             
                             2 
                           
                         
                         ) 
                       
                     
                   
                   . 
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
     In Equation (6), 
     d(y,x) denotes a distance between the position indicated by the coordinate y and the position indicated by the coordinate x, and 
     σ s  is a parameter that determines the distance weight Ws, and denotes a variance. 
     In Equation (5), We is a pixel value weight, and determined by, for example, the following Equation (7): 
     
       
         
           
             
               
                 
                   
                     W 
                     ⁢ 
                     c 
                   
                   = 
                   
                     exp 
                     ⁡ 
                     
                       ( 
                       
                         
                           - 
                           
                             1 
                             2 
                           
                         
                         ⁢ 
                         
                           
                             ( 
                             
                               
                                 ( 
                                 
                                   
                                     M 
                                     ⁢ 
                                     S 
                                     ⁢ 
                                     
                                       r 
                                       ⁡ 
                                       
                                         ( 
                                         y 
                                         ) 
                                       
                                     
                                   
                                   - 
                                   
                                     M 
                                     ⁢ 
                                     S 
                                     ⁢ 
                                     
                                       r 
                                       ⁡ 
                                       
                                         ( 
                                         x 
                                         ) 
                                       
                                     
                                   
                                 
                                 ) 
                               
                               
                                 σ 
                                 c 
                               
                             
                             ) 
                           
                           2 
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
     In Equation (7), σ c  is a parameter that determines the pixel value weight Wc, and denotes a variance. 
     In this embodiment, var(x) given by the above Equation (3), bw(x) given by Equation (4), or a value obtained by normalizing one of them is used as the parameter σ c . 
     In the image processing device  3  of the first embodiment, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges. 
     Also, since the image combiner  31  includes the noise amount calculator  311  as illustrated in  FIG. 3 , by determining the combination weights on the basis of the noise amounts of the band images used in the combination, it is possible to generate the reference image MSr having a higher S/N ratio, and thus it is possible to obtain output images having high S/N ratios. 
     In the above example, the multiple band light sources include the light source having the visible light wavelength band, the light source having the ultraviolet wavelength band, and the light source having the near-infrared wavelength band. The set of the multiple band light sources is not limited to the above example. For example, the visible light wavelength band may be divided. For example, a light source having a red wavelength band, a light source having a green wavelength band, and a light source having a blue wavelength band may be provided instead of the light source having the visible light wavelength band. 
     In the above example, the variances or edge amounts are determined as the image feature amounts by the parameter calculator  32 , and the values or normalized values thereof are used as the parameters σ c  for determining the combination weights. The image feature amounts determined by the parameter calculator  32  may be values other than the variances or edge amounts. In any case, the image feature amounts, reciprocals thereof, or normalized values thereof may be used as parameters for determining the combination weights. 
     Second Embodiment 
       FIG. 5  illustrates an image processing device  3   b  of a second embodiment of the present invention. 
     The illustrated image processing device  3   b  is generally the same as the image processing device  3  of  FIG. 1 , but includes an image combiner  31   b  instead of the image combiner  31 . 
     The image combiner  31  of the first embodiment performs the image combination such that the combination weight of a band image whose noise amount is greater is smaller. The image combiner  31   b  of this embodiment performs the image combination by using combination weights determined on the basis of image correlations between the band images. 
     The image characteristics of each band image depend on its wavelength band, and the farther the wavelength bands of two band images are from each other, the greater the difference in image characteristics between the two band images. 
     In this embodiment, when it is intended to reduce the noise of a band image of a certain wavelength band, the reference image is generated by performing the image combination such that the combination weight for a band image whose image correlation with the band image of the target wavelength band (the image of the target wavelength band) is greater is greater. 
     The image combiner  31   b  combines the multiple band images DIN to generate a reference image MSr having a higher S/N ratio than each band image. 
     The image combiner  31   b  includes a correlation calculator  313  and a weighting combiner  312   b , for example, as illustrated in  FIG. 6 . 
     The correlation calculator  313  calculates, for each of the multiple band images DIN, an image correlation Cor i  with a band image of a target wavelength band. 
     The image correlation Cor i  between each band image and the band image of the target wavelength band can be determined on the basis of, for example, a difference in wavelength between each band image and the band image of the target wavelength. In this case, it can be considered that the smaller the above difference, the higher the image correlation. 
     In this case, the above image correlation can be represented by a function of wavelength. 
       FIGS. 7A to 7C  illustrate examples of the image correlation. In  FIGS. 7A to 7C , the horizontal axis represents the wavelength of each band image, and the vertical axis represents the image correlation. The image correlation is expressed in an arbitrary unit (a.u.). 
     λ t  on the horizontal axis denotes a center wavelength or peak wavelength (target wavelength) of the target wavelength band. 
     In the example illustrated in  FIG. 7A , the correlation is highest at the target wavelength λ t , and linearly decreases as the difference from λ t  increases. 
     In the example illustrated in  FIG. 7B , the correlation keeps a constant value in the range in which the difference from the target wavelength λ t  is not greater than a set difference threshold, and is 0 in the range in which the difference is greater than the difference threshold. 
     In the example illustrated in  FIG. 7C , the correlation varies along a normal distribution curve centered at the target wavelength λ t . 
     In the above example, the correlation calculator  313  determines, for each band image, the image correlation on the basis of the difference in wavelength from the band image of the target wavelength band. 
     Alternatively, it is possible to determine, for each band image, the image correlation on the basis of the similarity with the band image of the target wavelength band. 
     As an indicator of the similarity, a sum of absolute differences (SAD), a sum of squared differences (SSD), a normalaized cross correlation (NCC), a zero-means normalaized cross correlation (ZNCC), or the like can be used, for example. 
     SAD for each band image MS i  (denoted by the symbol SA L ) is determined by the following Equation (8): 
     
       
         
           
             
               
                 
                   
                     S 
                     ⁢ 
                     
                       A 
                       i 
                     
                   
                   = 
                   
                     
                       ∑ 
                       x 
                     
                     ⁢ 
                     
                       
                          
                         
                           
                             
                               MS 
                               i 
                             
                             ⁡ 
                             
                               ( 
                               x 
                               ) 
                             
                           
                           - 
                           
                             M 
                             ⁢ 
                             S 
                             ⁢ 
                             
                               t 
                               ⁡ 
                               
                                 ( 
                                 x 
                                 ) 
                               
                             
                           
                         
                          
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
     SSD for each band image MS i  (denoted by the symbol SS i ) is determined by the following Equation (9): 
     
       
         
           
             
               
                 
                   
                     
                       S 
                       ⁢ 
                       
                         S 
                         i 
                       
                     
                     = 
                     
                       
                         ∑ 
                         x 
                       
                       ⁢ 
                       
                         
                           ( 
                           
                             
                               M 
                               ⁢ 
                               
                                 
                                   S 
                                   i 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   x 
                                   ) 
                                 
                               
                             
                             - 
                             
                               M 
                               ⁢ 
                               S 
                               ⁢ 
                               
                                 t 
                                 ⁡ 
                                 
                                   ( 
                                   x 
                                   ) 
                                 
                               
                             
                           
                           ) 
                         
                         2 
                       
                     
                   
                   . 
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
           
         
       
     
     NCC for each band image MS i  (denoted by the symbol NC i ) is determined by the following Equation (10): 
     
       
         
           
             
               
                 
                   
                     
                       N 
                       ⁢ 
                       
                         C 
                         i 
                       
                     
                     = 
                     
                       
                         
                           ∑ 
                           x 
                         
                         ⁢ 
                         
                           
                             
                               MS 
                               i 
                             
                             ⁡ 
                             
                               ( 
                               x 
                               ) 
                             
                           
                           ⁢ 
                           M 
                           ⁢ 
                           S 
                           ⁢ 
                           
                             t 
                             ⁡ 
                             
                               ( 
                               x 
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             ∑ 
                             x 
                           
                           ⁢ 
                           
                             
                               
                                 
                                   MS 
                                   i 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   x 
                                   ) 
                                 
                               
                               2 
                             
                             ⁢ 
                             
                               
                                 ∑ 
                                 x 
                               
                               ⁢ 
                               
                                 MS 
                                 ⁢ 
                                 
                                   
                                     t 
                                     ⁡ 
                                     
                                       ( 
                                       x 
                                       ) 
                                     
                                   
                                   2 
                                 
                               
                             
                           
                         
                       
                     
                   
                   . 
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
     ZNCC for each band image MS i  (denoted by the symbol ZNC i ) is determined by the following Equation (11): 
     
       
         
           
             
               
                 
                   
                     Z 
                     ⁢ 
                     N 
                     ⁢ 
                     
                       C 
                       i 
                     
                   
                   = 
                   
                     
                       
                         
                           ∑ 
                           x 
                         
                         ⁢ 
                         
                           
                             ( 
                             
                               
                                 M 
                                 ⁢ 
                                 
                                   
                                     S 
                                     i 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     x 
                                     ) 
                                   
                                 
                               
                               - 
                               
                                 M 
                                 ⁢ 
                                 
                                   S 
                                   i 
                                 
                                 ⁢ 
                                 
                                   m 
                                   ⁡ 
                                   
                                     ( 
                                     x 
                                     ) 
                                   
                                 
                               
                             
                             ) 
                           
                           ⁢ 
                           
                             ( 
                             
                               
                                 M 
                                 ⁢ 
                                 S 
                                 ⁢ 
                                 
                                   t 
                                   ⁡ 
                                   
                                     ( 
                                     x 
                                     ) 
                                   
                                 
                               
                               - 
                               
                                 M 
                                 ⁢ 
                                 S 
                                 ⁢ 
                                 t 
                                 ⁢ 
                                 m 
                                 ⁢ 
                                 
                                   ( 
                                   x 
                                   ) 
                                 
                               
                             
                             ) 
                           
                         
                       
                       
                         
                           
                             ∑ 
                             x 
                           
                           ⁢ 
                           
                             
                               
                                 ( 
                                 
                                   
                                     M 
                                     ⁢ 
                                     
                                       
                                         S 
                                         i 
                                       
                                       ⁡ 
                                       
                                         ( 
                                         x 
                                         ) 
                                       
                                     
                                   
                                   - 
                                   
                                     M 
                                     ⁢ 
                                     
                                       S 
                                       i 
                                     
                                     ⁢ 
                                     
                                       m 
                                       ⁡ 
                                       
                                         ( 
                                         x 
                                         ) 
                                       
                                     
                                   
                                 
                                 ) 
                               
                               2 
                             
                             ⁢ 
                             
                               
                                 ∑ 
                                 x 
                               
                               ⁢ 
                               
                                 
                                   ( 
                                   
                                     
                                       MS 
                                       ⁢ 
                                       
                                         t 
                                         ⁡ 
                                         
                                           ( 
                                           x 
                                           ) 
                                         
                                       
                                     
                                     - 
                                     
                                       M 
                                       ⁢ 
                                       S 
                                       ⁢ 
                                       t 
                                       ⁢ 
                                       
                                         m 
                                         ⁡ 
                                         
                                           ( 
                                           x 
                                           ) 
                                         
                                       
                                     
                                   
                                   ) 
                                 
                                 2 
                               
                             
                           
                         
                       
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
     In Equations (8) to (11), 
     x is a coordinate representing the position of the pixel of interest, 
     MS i (x) denotes a pixel value at the position x of the band image MS i , 
     MSt(x) denotes a pixel value at the position x of the band image MSt of the target wavelength band, 
     MS i m denotes an average of the pixel values MS i (x) of all the pixels in the image MS i , and 
     MStm denotes an average of the pixel values MSt(x) of all the pixels in the image MSt. 
     The higher the similarity, the smaller the values of SAD and SSD. The higher the similarity, the greater the values of NCC and ZNCC. 
     SAD and SSD are appropriate when it is intended to make a comparison in terms of image characteristics including intensity (brightness, luminance), and NCC and ZNCC, each of which provides a value independent of the image intensities (the magnitudes of the pixel values) due to normalization, are appropriate when it is intended to extract spectral reflection characteristics of the object in a specific wavelength band. 
     The weighting combiner  312   b  generates the reference image MSr by combining the multiple band images DIN such that the combination weight of a band image whose image correlation Cor i  is greater is greater. The generation is performed by weighting and adding pixel values at the same positions of the multiple band images. Specifically, the weighting combiner  312   b  determines weighted averages of pixel values at the same positions of the multiple band images DIN as pixel values at the same positions of the reference image MSr. 
     The weighted average for a certain pixel (pixel of interest) is determined by, for example, the calculation represented by the above Equation (1). 
     However, as the combination weight W i  for the pixel value MS i (x) of each band image MS i , a normalized value of the image correlation Cor i  of the band image MS i  is used. The normalized value of the image correlation of each band image is a value obtained by dividing the image correlation of the band image by an average of the image correlations of all the band images. Thus, the combination weight W i  is represented by the following Equation (12): 
     
       
         
           
             
               
                 
                   
                     W 
                     i 
                   
                   = 
                   
                     
                       
                         C 
                         ⁢ 
                         o 
                         ⁢ 
                         
                           r 
                           i 
                         
                       
                       
                         
                           1 
                           M 
                         
                         ⁢ 
                         
                           
                             ∑ 
                             
                               i 
                               ∈ 
                               M 
                             
                           
                           ⁢ 
                           
                             Cor 
                             i 
                           
                         
                       
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
     Although in the above example, the similarity of an image is determined for the entire image, it is possible to determine the similarity for each pixel and determine the image correlation for each pixel. 
     In this case, in Equation (8) to (11), the similarity is determined by using pixels in a local region centered at the pixel of interest, and the image correlation for the pixel of interest is determined from the determined similarity. 
     Also in the image processing device  3   b  of the second embodiment, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges. 
     Also, since the image combiner  31  includes the correlation calculator  313  as illustrated in  FIG. 6 , by determining the combination weights on the basis of the image correlations between the band images used in the combination and the band image of the target wavelength band, it is possible to generate an output image with less collapse in image characteristics. 
     For the first embodiment, as another example of the weighting combination, there has been described a method of setting each weight W i  to 0 or 1 by using a preset noise amount threshold and determining the weighted averages. The same modification can be made to the second embodiment. 
     Specifically, it is possible to set each weight W i  to 0 or 1 by using a preset correlation threshold and determine the weighted averages. For example, it is possible to set the weight W i  to 1 when the image correlation is not less than the image correlation threshold and to 0 otherwise, and determine the weighted averages by using the number of band images for which the weight W i  has been set to 1, instead of M. 
     Third Embodiment 
       FIG. 8  illustrates an image processing device  3   c  of a third embodiment of the present invention. 
     The illustrated image processing device  3   c  is generally the same as the image processing device  3  of  FIG. 1 , but does not include the image combiner  31  of  FIG. 1  and includes a parameter calculator  32   c  instead of the parameter calculator  32  of  FIG. 1 . 
     Also, in addition to the band images DIN of  FIG. 1 , a white image DIW is input to the image processing device  3   c.    
     The band images DIN are the same as the band images of  FIG. 1 . 
     The white image DIW is an image having a band including all the bands of the multiple band images DIN, and is used as a reference image. The band of the white image DIW is preferably wider than a band obtained by summing all the bands of the multiple band images DIN. 
     The parameter calculator  32   c  analyzes the white image DIW, and for each pixel of the white image DIW, calculates a local variance or an edge amount as an image feature amount and calculates a filter parameter D 32   c  from the image feature amount. 
     The content of the process in the parameter calculator  32   c  is the same as that of the process in the parameter calculator  32  of  FIG. 1 . It differs in determining the filter parameters by using the white image DIW as the reference image. 
     The filter processor  33  uses the filter parameters D 32   c  to smooth the multiple band images DIN while preserving edges, and generates multiple output images DOUT. 
     The content of the process in the filter processor  33  is the same as that of the process in the filter processor  33  of  FIG. 1 . 
     In the image processing device  3   c  of the third embodiment, it is possible to obtain the white image having the band including all the bands of the multiple band images and use it as the reference image. Thus, even in a case in which the number of wavelength bands of the used light source is small, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges. 
     Fourth Embodiment 
       FIG. 9  illustrates an image processing device  3   d  of a fourth embodiment of the present invention. 
     The illustrated image processing device  3   d  is generally the same as the image processing device  3   c  of  FIG. 8 , but includes a bandpass filter group  34 . Also, the multiple band images DIN are not input to the image processing device  3   d , and only the white image DIW is input to the image processing device  3   d.    
     The bandpass filter group  34  receives as an input the white image and includes multiple bandpass filters that pass different wavelength bands, and multiple band images D 34  of different wavelength bands are output from the bandpass filter group  34 . 
     In the above first to third embodiments, the multiple band images DIN are sequentially obtained by performing imaging when the original is sequentially illuminated by the multiple band light sources having different wavelength bands. On the other hand, in the fourth embodiment, the multiple band images DIN are generated by using the multiple bandpass filters that pass the different wavelength bands of the white image of a wide band. 
     The filter processor  33  uses the filter parameters D 32   c  to smooth the multiple band images D 34  while preserving edges, and generates multiple output images DOUT. 
     The fourth embodiment provides the same advantages as the first to third embodiments. 
     In the above first to fourth embodiments, the image processing device forms part of the image reading device. However, image processing devices of the present invention can be used for purposes other than image reading. For example, they can also be used in imaging the ground or the like from an optical satellite or aircraft. 
     Each of the image processing devices  3 ,  3   b ,  3   c , and  3   d  described in the first to fourth embodiments may be partially or wholly formed by processing circuitry. 
     For example, the functions of the respective portions of the image processing device may be implemented by respective separate processing circuits, or the functions of the portions may be implemented by a single processing circuit. 
     The processing circuitry may be implemented by hardware, or by software or a programmed computer. 
     It is possible that a part of the functions of the respective portions of the image processing device is implemented by hardware and another part is implemented by software. 
       FIG. 10  illustrates an example of a configuration when all the functions of the image processing device  3 ,  3   b ,  3   c , or  3   d  of the above respective embodiments are implemented by a computer  9  including a single processor, together with the light source  1  and image sensor  2 . 
     In the illustrated example, the computer  9  includes a processor  91  and a memory  92 . 
     A program for implementing the functions of the respective portions of the image processing device is stored in the memory  92 . 
     The processor  91  uses, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, a digital signal processor (DSP), or the like. 
     The memory  92  uses, for example, a semiconductor memory, such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a magnetic disk, an optical disk, a magnetic optical disk, or the like. 
     The processor  91  implements the function of the image processing device by executing the program stored in the memory  92 . 
     A procedure when a process in the image processing device of the first or second embodiment is performed by the computer illustrated in  FIG. 10  will be described with reference to  FIG. 11 . 
     In step ST 11 , the multiple band images DIN are received. 
     In step ST 12 , the reference image MSr is generated by combining the multiple band images DIN. This process is performed as described for the image combiner  31  or  31   b.    
     In step ST 13 , the local variances or edge amounts are calculated as the image feature amounts by analyzing the reference image MSr, and the filter parameters D 32  are calculated from the image feature amounts. This process is performed as described for the parameter calculator  32 . 
     In step ST 14 , smoothing the multiple band images DIN while preserving edges is performed by using the filter parameters D 32 . This process is performed as described for the filter processor  33 . 
     A procedure when a process in the image processing device of the third embodiment is performed by the computer illustrated in  FIG. 10  will be described with reference to  FIG. 12 . 
     In  FIG. 12 , processes that are the same as those in  FIG. 11  are indicated by the same symbols. 
     In step ST 11 , the multiple band images DIN are received. 
     In step ST 22 , the white image DIW is received. 
     In step ST 13   c , the local variances or edge amounts are calculated as the image feature amounts by analyzing the white image DIW, and the filter parameters D 32   c  are calculated from the image feature amounts. This process is performed as described for the parameter calculator  32   c.    
     In step ST 14 , smoothing the multiple band images DIN while preserving edges is performed by using the filter parameters D 32   c . This process is performed as described for the filter processor  33 . 
     A procedure when a process in the image processing device of the fourth embodiment is performed by the computer illustrated in  FIG. 10  will be described with reference to  FIG. 13 . 
     In  FIG. 13 , processes that are the same as those in  FIG. 11 or 12  are indicated by the same symbols. 
     In step ST 22 , the white image DIW is received. 
     In step ST 13   c , the local variances or edge amounts are calculated as the image feature amounts by analyzing the white image DIW, and the filter parameters D 32   c  are calculated from the image feature amounts. This process is performed as described for the parameter calculator  32   c.    
     In step ST 33 , the bandpass filtering that passes the different wavelength bands is performed on the white image DIW, so that the multiple band images D 34  are generated. This process is performed as described for the bandpass filter group  34 . 
     The process of step ST 33  can be performed in parallel with the process of step ST 13   c.    
     In step ST 14 , smoothing the multiple band images D 34  while preserving edges is performed by using the filter parameters D 32   c . This process is performed as described for the filter processor  33 . 
     Although image processing devices of the present invention have been described above, the image processing methods implemented by the above image processing devices also form part of the present invention. Also, programs for causing computers to execute processes of the above image processing devices or image processing methods, and computer-readable recording media, e.g., non-transitory recording media, storing the programs also form part of the present invention. 
     Although embodiments of the present invention have been described, the present invention is not limited to these embodiments. 
     REFERENCE SIGNS LIST 
       1  light source,  2  image sensor,  3 ,  3   b ,  3   c ,  3   d  image processing device,  31  image combiner,  32 ,  32   c  parameter calculator,  33  filter processor,  34  bandpass filter group,  311  noise amount calculator,  312 ,  312   b  weighting combiner,  313  correlation calculator.