# Patent

## Title
Method and apparatus for image signal processing

## Abstract

Image signal processing method and apparatus having a first filter storage portion in which first filters are correlatively stored; a second filter storage portion in which second filters are correlatively stored; a first filter selection portion for selecting a first filter based on the power spectrum of the input image; a second filter selection portion for selecting a second filter based on the SIN (signal-to-noise ratio) of the input image; a third filter creation portion for creating a third filter by summing up the first and second filters; and a convolutional processing portion for convolving the input image using the created third filter.

## Background
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method and apparatus for signal processing.
2. Description of Related Art
Techniques for performing image processing such as edge enhancement and blurring using a convolution filter such as a Laplacian filter or Gaussian filter are known from the past (see, for example, JP-A-2009-79949).
However, related art techniques do not have any filter that acts precisely on any arbitrary input image. Consequently, it has been required to use a filter whose parameters have been adjusted by assuming input images. Furthermore, a wavelet algorithm that expands an input image into image elements of plural resolutions is known. The input image is expanded into a series of image elements which are half as many as there are pixels. Therefore, the resolutions differ greatly. This presents the problem that it is impossible to finely adjust the image quality. Another problem with wavelet analysis is that an exorbitantly long time is required to process a large image because the input image is expanded.
SUMMARY OF THE INVENTION
In view of the foregoing problems, the present invention has been made. According to some embodiments of the invention, it is possible to offer an image processing method and apparatus capable of acting on and processing various input signals in a short time.
(1) A method for signal processing associated with the present invention starts with calculating a power spectrum of an input signal. A power spectrum approximate to the calculated power spectrum is selected from a plurality of power spectra stored in a first filter storage portion. A first filter corresponding to the selected power spectrum is selected from a plurality of first filters stored in the first filter storage portion. This processing step may be hereinafter referred to as the first filter selection step. Then, the S/N (signal-to-noise ratio) of the input signal is calculated. An S/N approximate to the calculated S/N is selected from a plurality of S/Ns stored in a second filter storage portion. A second filter corresponding to the selected S/N is selected from a plurality of second filters stored in the second filter storage portion. This processing step may be hereinafter referred to as the second filter selection step. The selected first and second filters are summed up to create a third filter. This processing step may be hereinafter referred to as the third filter creation step. The input signal is convolved using the created third filter. This processing step may be hereinafter referred to as the convolution step.
According to the present invention, a first filter selected based on the power spectrum of an input signal is added to a second filter selected based on the S/N of the input signal, thus a third filter is created. The input signal is convolved using the third filter. Consequently, the input signal can be processed using the filter adapted for the input signal in a short time.
(2) In a method for signal processing associated with the present invention, the first filters stored in the first filter storage portion have been created based on sample input signals by a filter creation step and are correlated with power spectra of the sample input signals. The second filters stored in the second filter storage portion have been created by the filter creation step and are correlated with S/Ns of the sample input signals. During the filter creation step, a filter of an arbitrary distribution function G0 is expanded into a series of n filter elements (where n is an arbitrary number) each of which is the difference between two distribution functions. The coefficients of the filter elements are set at will. The filter elements are multiplied by their respective coefficients and summed up. Thus, the first filter that is a function of the number of the filter elements n and of the coefficients of the filter elements is created. The distribution function Gn of the nth order is obtained when the filter is expanded into the arbitrary number n of filter elements. The distribution function Gn and the distribution function G0 are combined with arbitrary weight coefficients to create second filters. The value of the number n, the coefficients of the filter elements, and the weight coefficients may be set based on the sample input signals.
According to the present invention, the first filter adapted for the sample input signal can be created by setting the value of the number n and the coefficients of the filter elements based on the sample input signal. Furthermore, the second filter adapted for the sample input signal can be created by setting the value of the number n and the weight coefficient based on the sample input signal. Plural first filters and plural second filters are created based on sample input signals. A first filter selected from the created first filters based on a power spectrum of an input signal and a second filter selected from the created second filters based on the S/N of the input signal are summed up to create a third filter. The input signal is convolved using the third filter. Consequently, signal processing can be performed in a short time using the filter adapted for the input signal.
(3) In the signal processing method associated with the present invention, during the filter creation step, the value of the number n, the coefficients of the filter elements, and the weight coefficients may be set to maximize the S/Ns of the signals obtained by convolving the sample input signals using the created first and second filters.
According to the present invention, the first and second filters adapted for the sample input signal can be created.
(4) In the method of signal processing associated with the present invention, each of the distribution functions may be a Gaussian function.
Furthermore, the distribution function Gn of the nth order may be a Gaussian function. The distribution function G0 may be a function approximate to a delta function.
(5) An apparatus for signal processing associated with the present invention has: a first filter storage portion in which first filters created based on sample input signals and power spectra of the sample input signals are correlatively stored; a second filter storage portion in which second filters created based on sample input signals and S/Ns of the sample input signals are correlatively stored; a first filter selection portion for calculating a power spectrum of an input signal, selecting a power spectrum approximate to the calculated power spectrum from the power spectra stored in the first filter storage portion, and selecting a first filter corresponding to the selected power spectrum from the first filters stored in the first filter storage portion; a second filter selection portion for calculating an S/N of the input signal, selecting an S/N approximate to the calculated S/N from the S/Ns stored in the second filter storage portion, and selecting a second filter corresponding to the selected S/N from the second filters stored in the second filter storage portion; a third filter creation portion for creating a third filter by summing up the selected first and second filters; and a convolutional processing portion for convolving the input signal using created third filter.
According to the present invention, a first filter selected based on the power spectrum of the input signal and a second filter selected based on the S/N of the input signal are summed up to create a third filter. The input signal is convolved using the third filter. Consequently, signal processing can be performed in a short time using a filter adapted for the input signal.
(6) The apparatus for signal processing associated with the present invention may further include a filter creation portion for creating the first and second filters based on the sample input signals and calculating power spectra and S/Ns of the sample input signals. The filter creation portion may expand a filter of an arbitrary distribution function G0 into a series of n filter elements (where n is an arbitrary number) each of which is the difference between two distribution functions, set coefficients of the filter elements at will, multiply the filter elements by the coefficients, respectively, sum up the multiplied filter elements to create a first filter that is a function of the number of the filter elements n and of the coefficients of the filter elements, obtain a distribution function Gn of the nth order when the filter is expanded into the n filter elements, combine together the distribution function Gn and the distribution function G0 with an arbitrary weight coefficient to obtain a second filter, and set the value of the number n, the coefficients of the filter elements, and the weight coefficients based on the sample input signals.
According to the present invention, first filters adapted for sample input signals can be created by setting the value of the number n and the coefficients of the filter elements based on the sample input signals. Second filters adapted for sample input signals can be created by setting the value of the number n and the weight coefficient based on the sample input signals. A first filter is selected according to a power spectrum of an input signal from the first filters created based on the sample input signals. A second filter is selected according to an S/N of the input signal from the second filters created on the basis of the sample input signals. The selected first and second filters are summed up to create a third filter. The input signal is convolved using the third filter. Consequently, signal processing can be performed in a short time using the filter adapted for the input signal.
(7) In the apparatus for signal processing associated with the present invention, the filter creation portion may set the value of the number n, the coefficients of the filter elements, and the weight coefficients to maximize the S/Ns of the signals obtained by convolving the sample input signals using the created first and second filters.
According to the present invention, the first and second filters adapted for sample input signals can be created.
(8) In the apparatus for signal processing associated with the present invention, each of the distribution functions may be a Gaussian function.
Furthermore, the distribution function Gn of the nth order may be a Gaussian function. The distribution function G0 may be a function approximate to a delta function.



BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a functional block diagram of an image processing apparatus (one example of signal processing apparatus) associated with one embodiment of the present invention.
FIG. 2 is a detailed functional block diagram of the filter creation portion of the apparatus shown in FIG. 1.
FIG. 3 shows examples of filter element obtained by expansion.
FIG. 4 is a graph illustrating the coefficients of the filter elements.
FIG. 5 shows one example of second filter.
FIG. 6 shows one example of intermediate filter.
FIG. 7 is a flowchart illustrating one example of a sequence of operations performed by one embodiment of the invention.
FIG. 8 is a flowchart illustrating one example of a sequence of operations performed by one embodiment of the invention.
FIGS. 9A and 9B are images illustrating one example of image processing method according to the present invention.



DESCRIPTION OF THE PREFERRED EMBODIMENTS
The preferred embodiments of the present invention are hereinafter described in detail with reference to the drawings. It is to be understood that embodiments described below do not unduly restrict the contents of the invention delineated by the appended claims and that all the configurations described below are not always essential constituent components of the present invention.
1. Configuration
FIG. 1 is a functional block diagram of an image processing apparatus associated with one embodiment of the present invention, the image processing apparatus being one example of signal processing apparatus. Note that some of the constituent elements of the image processing apparatus of the present embodiment shown in FIG. 1 may be omitted.
The image processing apparatus, generally indicated by reference numeral 1, of the present embodiment is an apparatus for performing image processing to vary the image quality (resolution) of an input SEM image that is represented by an input signal and taken by a scanning electron microscope (SEM). The apparatus is configured including a filter creation portion 10, a first filter storage portion 20, a second filter storage portion 22, a first filter selection portion 30, a second filter selection portion 32, and a third filter creation portion 50, and a convolutional processing portion 40. The functions of the filter creation portion 10, the first filter selection portion 30, the second filter selection portion 32, the third filter creation portion 50, and the convolutional processing portion 40 can be accomplished by various processors (such as CPU and DSP) or computer programs. The functions of the first filter storage portion 20 and the second filter storage portion 22 can be realized by a RAM, a hard disk, or the like.
The filter creation portion 10 creates first and second filters based on a sample image represented by a sample input signal, calculates a power spectrum and the signal-to-noise ratio (S/N) of the sample image, outputs the created first filter and the calculated power spectrum to the first filter storage portion 20, and outputs the created second filter and the calculated S/N to the second filter storage portion 22.
The filter creation portion 10 expands a filter that is an arbitrary distribution function G0 (e.g., a function approximate to a delta function) into a series of n filter elements (where n is an arbitrary number) each of which is the difference between two distribution functions (e.g., two-dimensional Gaussian functions), sets the coefficients of the filter elements at will, multiplies the filter elements by the coefficients, adds up the multiplied filter elements to create a first filter that is a function of the number of the filter elements n and the coefficients of the filter elements, and sets the number of the filter elements n and the coefficients of the filter elements based on a sample image. The filter creation portion 10 combines the distribution function Gn of the nth order obtained when the filter is expanded into the n filter elements and the distribution function G0 with an arbitrary weight coefficient to create a second filter. The filter creation portion sets the value of the number n and the weight coefficient based on the sample input signal.
The filter creation portion 10 sets the value of the number n, the coefficients of the filter elements, and the weight coefficient to maximize the S/N of the signal obtained by convolving the sample image using the created first and second filters.
First filters created based on sample images by the filter creation portion 10 and power spectra of the sample images are correlatively stored in the first filter storage portion 20. Second filters created based on the sample images by the filter creation portion 10 and S/Ns of the sample images are correlatively stored in the second filter storage portion 22. Preferably, a number of (e.g., about 100) sets of first filters created from sample images and respective power spectra of the sample images are stored in the first filter storage portion 20. Preferably, a number of (e.g., about 100) sets of second filters created from sample images and respective S/Ns of the sample images are stored in the second filter storage portion 22. The sample images can be SEM images taken from various samples and images of natural objects and artificial objects taken by a digital camera. In the present invention, multiple sets of such first filters and power spectra are previously stored in the first filter storage portion 20. Multiple sets of such second filters and S/Ns are previously stored in the second filter storage portion 22.
The first filter selection portion 30 calculates a power spectrum of an input image (represented by input signals) to be processed, selects a power spectrum approximate to the calculated power spectrum from power spectra stored in the first filter storage portion 20, selects a first filter corresponding to the selected power spectrum from the first filters stored in the first filter storage portion 20, and outputs the selected first filter to the third filter creation portion 50.
The second filter selection portion 32 calculates the S/N of the input image represented by the input signal, selects an S/N approximate to the calculated S/N from the S/Ns stored in the second filter storage portion 22, selects a second filter corresponding to the selected S/N from the second filters stored in the second filter storage portion 22, and outputs the selected second filter to the third filter creation portion 50.
The third filter creation portion 50 adds up the first and second filters selected in this way to create a third filter and outputs the created third filter to the convolutional processing portion 40.
The convolutional processing portion 40 convolves the input image using the third filter generated by the third filter creation portion 50. Furthermore, the convolutional processing portion 40 may convolve an input image, whose image size (spatial size) has been expanded by an image size expansion portion (not shown), using a selected filter. A spatial size restoration portion (not shown) may restore the spatial size of the output image convolved by the convolutional processing portion 40 to the spatial size of the input image.
2. Technique of Present Embodiment
The technique of the present embodiment is next described by referring to drawings.
2-1. Selection of First and Second Filters and Convolution
The image quality of an input image to be processed can be varied by convolving the input image using a filter of a distribution function. The convolutional processing portion 40 of the present embodiment performs convolutional processing as given by:

Iout=F0Iin  (1)

where Iin indicates an input image, Iout indicates an output image, and F0 indicates a third filter generated by the third filter creation portion 50. Eq. (1) can be rewritten as follows:





  
    
      
        
          
            I
            out
          
          ⁡
          
            (
            
              x
              ,
              y
            
            )
          
        
        =
        
          
            
              ∫
              ∫
            
            
              X
              ,
              
                Y
                ∈
                R
              
            
          
          ⁢
          
            
              I
              
                i
                ⁢
                
                  
                
                ⁢
                n
              
            
            ⁡
            
              (
              
                
                  x
                  -
                  X
                
                ,
                
                  y
                  -
                  Y
                
              
              )
            
          
          ×
          
            
              F
              0
            
            ⁡
            
              (
              
                X
                ,
                Y
              
              )
            
          
          ⁢
          
            d
            X
          
          ⁢
          
            d
            Y
          
        
      
    
    
      
        (
        2
        )
      
    
  




where x and y indicate the X- and Y-coordinates, respectively, of the input image, and X and Y indicate the X- and Y-coordinates, respectively, of the third filter. Usually, the input image is represented by discrete values and so Eq. (2) can be rewritten as follows:





  
    
      
        
          
            I
            out
          
          ⁡
          
            (
            
              nx
              ,
              ny
            
            )
          
        
        =
        
          
            ∑
            NX
          
          ⁢
          
            
          
          ⁢
          
            
              ∑
              NY
            
            ⁢
            
              
            
            ⁢
            
              
                
                  I
                  
                    i
                    ⁢
                    
                      
                    
                    ⁢
                    n
                  
                
                ⁡
                
                  (
                  
                    
                      nx
                      -
                      NX
                    
                    ,
                    
                      ny
                      -
                      NY
                    
                  
                  )
                
              
              ×
              
                
                  F
                  0
                
                ⁡
                
                  (
                  
                    NX
                    ,
                    NY
                  
                  )
                
              
            
          
        
      
    
    
      
        (
        3
        )
      
    
  




where nx and ny indicate the numbers of pixels of the input image in the X- and Y-directions, respectively, and NX and NY indicate the numbers of pixels of the third filter in the X- and Y-directions, respectively.

The first filter selection portion 30 of the present embodiment operates to perform a discrete Fourier transform of the discrete input image Iin {ij} using:




  
    
      
        
          F
          pq
        
        =
        
          
            ∑
            
              j
              =
              1
            
            ny
          
          ⁢
          
            
          
          ⁢
          
            
              ∑
              
                i
                =
                1
              
              nx
            
            ⁢
            
              
            
            ⁢
            
              
                I
                
                  i
                  ⁢
                  
                    
                  
                  ⁢
                  n
                
              
              ⁢
              
                {
                ij
                }
              
              ⁢
              exp
              ⁢
              
                {
                
                  
                    -
                    J
                  
                  ⁢
                  
                    
                  
                  ⁢
                  2
                  ⁢
                  
                    
                  
                  ⁢
                  
                    π
                    ⁡
                    
                      (
                      
                        
                          
                            p
                            ×
                            
                              (
                              
                                i
                                +
                                1
                              
                              )
                            
                          
                          
                            n
                            ⁢
                            
                              
                            
                            ⁢
                            x
                          
                        
                        +
                        
                          
                            q
                            ×
                            
                              (
                              
                                j
                                -
                                1
                              
                              )
                            
                          
                          ny
                        
                      
                      )
                    
                  
                
                }
              
            
          
        
      
    
    
      
        (
        4
        )
      
    
  




where J=√{square root over (−1)}.

Then, the first filter selection portion 30 calculates the power spectrum PSin(p,q) of the input image using:

PSin(p,q)=√{square root over (Re(Fpq)2+Im(Fpq)2)}{square root over (Re(Fpq)2+Im(Fpq)2)}  (5)

The first filter selection portion 30 computes the error, err, between the power spectrum PSin(p,q) of the input image and each power spectrum PS(p,q) stored in the first filter storage portion 20 using the equation:




  
    
      
        err
        =
        
          
            
              ∑
              
                q
                =
                1
              
              ny
            
            ⁢
            
              
            
            ⁢
            
              
                ∑
                
                  p
                  =
                  1
                
                nx
              
              ⁢
              
                
              
              ⁢
              
                
                  (
                  
                    
                      
                        PS
                        
                          i
                          ⁢
                          
                            
                          
                          ⁢
                          n
                        
                      
                      ⁢
                      
                        {
                        
                          p
                          ,
                          q
                        
                        }
                      
                    
                    -
                    
                      PS
                      ⁢
                      
                        {
                        
                          p
                          ,
                          q
                        
                        }
                      
                    
                  
                  )
                
                2
              
            
          
        
      
    
    
      
        (
        6
        )
      
    
  




The first filter selection portion 30 then selects one of the power spectra stored in the first filter storage portion 20 which minimizes the error, err, with the power spectrum of the input image, selects one of the first filters Fex stored in the first filter storage portion 20 which corresponds to the selected power spectrum, and outputs the selected spectrum and first filter to the third filter creation portion 50.
The second filter selection portion 32 of the present embodiment calculates the signal-to-noise ratio, SNin, of the input image Iin using the formula:




  
    
      
        
          SN
          
            i
            ⁢
            
              
            
            ⁢
            n
          
        
        =
        
          
            μ
            signal
          
          
            σ
            noise
          
        
      
    
    
      
        (
        7
        )
      
    
  




where μsignal is the average of the input signal Iin and σnoise is the standard deviation of the input signal Iin as given by:





  
    
      
        
          μ
          signal
        
        =
        
          
            
              ∑
              
                q
                =
                1
              
              ny
            
            ⁢
            
              
            
            ⁢
            
              
                ∑
                
                  p
                  =
                  1
                
                nx
              
              ⁢
              
                
              
              ⁢
              
                
                  I
                  
                    i
                    ⁢
                    
                      
                    
                    ⁢
                    n
                  
                
                ⁡
                
                  (
                  
                    p
                    ,
                    q
                  
                  )
                
              
            
          
          
            nx
            ×
            ny
          
        
      
    
    
      
        (
        8
        )
      
    
  
  
    
      
        
          σ
          noise
          2
        
        =
        
          
            1
            
              nx
              ×
              ny
            
          
          ⁢
          
            
              ∑
              
                q
                =
                1
              
              ny
            
            ⁢
            
              
            
            ⁢
            
              
                ∑
                
                  p
                  =
                  1
                
                nx
              
              ⁢
              
                
              
              ⁢
              
                
                  (
                  
                    
                      
                        I
                        
                          i
                          ⁢
                          
                            
                          
                          ⁢
                          n
                        
                      
                      ⁡
                      
                        (
                        
                          p
                          ,
                          q
                        
                        )
                      
                    
                    -
                    
                      μ
                      signal
                    
                  
                  )
                
                2
              
            
          
        
      
    
    
      
        (
        9
        )
      
    
  




The second filter selection portion 32 then calculates the error, err, between the signal-to-noise ratio, SNin, of the input image and each signal-to-noise, SN, stored in the second filter storage portion 22 using:

err=(SNin−SN)2  (10)

The second filter selection portion 32 selects the S/N which minimizes the error, err, with the S/N of the input image from S/Ns stored in the second filter storage portion 22, selects the second filter Fex which corresponds to the selected S/N from the second filters stored in the second filter storage portion 22, and outputs the selected S/N and second filter Fex to the third filter creation portion 50.
The third filter creation portion 50 adds up the selected first filter Fex and second filter Fex to generate a third filter F0, and outputs the generated third filter F0 to the convolutional processing portion 40.
According to the present embodiment, a filter optimal for an input image can be obtained in a short processing time without repeating a processing operation for generating a filter adapted for the input image by selecting a first filter corresponding to a power spectrum most approximate to the power spectrum of the input image to be processed and selecting a second filter corresponding to an S/N most approximate to the S/N of the input image.
2-2. Creation of First and Second Filters
FIG. 2 is a detailed functional block diagram of the filter creation portion 10 of the present embodiment.
The filter creation portion 10 includes a filter expansion portion 11, a filter element storage portion 13, a coefficient creation portion 14, a filter reconfiguration portion 15, a convolution portion 16, an image quality evaluation portion 17, a power spectrum calculating portion 18, and an S/N calculating portion 19.
The filter expansion portion 11 expands a filter of an arbitrary distribution function G0 into a series of n filter elements (where n is an arbitrary number) each of which is the difference between two distribution functions.
Let Gk be a two-dimensional distribution function (where k=0, 1, 2, . . . , n (integer)). It is assumed that as the integer k increases, the standard deviation σk of the distribution function Gk increases. It is here assumed that the distribution function Gk is a distribution function of the kth order. For example, if the distribution function Gk is a two-dimensional Gaussian function, the distribution function Gk and its standard deviation σk are given by:




  
    
      
        
          
            G
            k
          
          ⁡
          
            (
            
              x
              ,
              y
            
            )
          
        
        =
        
          
            1
            
              2
              ⁢
              
                πσ
                k
              
            
          
          ⁢
          
            exp
            (
            
              -
              
                
                  
                    x
                    2
                  
                  +
                  
                    y
                    2
                  
                
                
                  2
                  ⁢
                  
                    σ
                    k
                  
                
              
            
            )
          
        
      
    
    
      
        (
        11
        )
      
    
  
  
    
      
        
          
            σ
            k
          
          =
          
            
              
                (
                
                  k
                  -
                  1
                
                )
              
              ⁢
              Δ
              ⁢
              
                
              
              ⁢
              σ
            
            +
            
              
                σ
                k
              
              ⁢
              
                
              
              ⁢
              
                (
                
                  k
                  ≥
                  1
                
                )
              
            
          
        
        ⁢
        
          

        
        ⁢
        
          
            σ
            k
          
          =
          
            
              σ
              0
            
            ≅
            
              0
              ⁢
              
                
              
              ⁢
              
                (
                
                  k
                  =
                  0
                
                )
              
            
          
        
      
    
    
      
        (
        12
        )
      
    
  




If k=0 as given by Eq. (12), G0 is approximate to a delta function (i.e., the dispersion σ0 of G0 is approximate to 0).
In the present embodiment, the distribution function G0 approximate to the delta function is expanded into the sum of the distribution function Gn of the nth order and a series of n filter elements each of which is a distribution function being the difference between two distribution functions. In this example, the distribution function G0 is expanded as follows:

G0=Gn+(G0−Gn)  (13)

Expanding the second term of the right side of Eq. (13) similarly into n terms gives rise to:




  
    
      
        
          
            
              
                G
                0
              
              =
              
              ⁢
              
                
                  G
                  n
                
                +
                
                  (
                  
                    
                      G
                      0
                    
                    -
                    
                      G
                      n
                    
                  
                  )
                
              
            
          
        
        
          
            
              =
              
              ⁢
              
                
                  G
                  n
                
                +
                
                  (
                  
                    
                      G
                      0
                    
                    -
                    
                      G
                      1
                    
                  
                  )
                
                +
              
            
          
        
        
          
            
              
              ⁢
              
                
                  (
                  
                    
                      G
                      1
                    
                    -
                    
                      G
                      2
                    
                  
                  )
                
                +
                
                  
                    (
                    
                      
                        G
                        2
                      
                      -
                      
                        G
                        3
                      
                    
                    )
                  
                  ⁢
                  
                    
                  
                  ⁢
                  ...
                
                +
              
            
          
        
        
          
            
              
              ⁢
              
                (
                
                  
                    G
                    
                      n
                      -
                      1
                    
                  
                  -
                  
                    G
                    n
                  
                
                )
              
            
          
        
      
    
    
      
        (
        14
        )
      
    
  




Assuming that the kth expansion term is a filter element Mk of the kth order and that the equation Mk=Gk-1−Gk holds, Eq. (14) can be rewritten as:




  
    
      
        
          G
          0
        
        =
        
          
            G
            n
          
          +
          
            
              ∑
              
                k
                =
                1
              
              n
            
            ⁢
            
              
            
            ⁢
            
              M
              k
            
          
        
      
    
    
      
        (
        15
        )
      
    
  




As can be seen from Eq. (15), the distribution function G0 has been expanded into the distribution function Gn of the nth order and n filter elements Mk (k=1 to n) each of which is the difference between the distribution function Gk-1 of the (k−1)th order and the distribution function Gk of the kth order. As described later, the distribution function Gn of the nth order is used when the second filter that is a noise filter is created. The n filter elements Mk are used when the first filter being an edge enhancement filter is created.
Examples of the filter elements Mk are shown in FIG. 3, where Δσ=0.5, σ1=1.5, and k=1, 3, and 5 in Eq. (12).
As shown in FIG. 3, each filter element Mk is a differential filter that emphasizes the edges of the input image in the kth order. As the value of k of each filter element Mk is increased, its dispersion increases. Therefore, a filter element Mk having a small value of k acts as an edge enhancement filter acting on higher-frequency components of the input image. On the other hand, a filter element Mk having a greater value of k acts as an edge enhancement filter acting on lower-frequency components of the input image.
The filter element storage portion 13 shown in FIG. 2 stores the distribution function G0, the nth-order distribution function Gn generated by the filter expansion portion 11, and the n filter elements Mk.
The coefficient creation portion 14 creates coefficients ωk by which the n filter elements Mk stored in the filter element storage portion 13 are respectively multiplied, for example, using the following formula:

 ωk=λn-k  (16)

where λ is an arbitrary real number and preferably assumes a positive value from approximately 0.5 to 0.9. In the present embodiment, λ is a real number less than unity. Therefore, a filter element Mk (a filter element acting on low-frequency components of the image) having a large value of k has a large value of coefficient ωk as shown in FIG. 4. On the other hand, a filter element Mk (a filter element acting on high-frequency components of the image) having a small value of k has a small value of coefficient ωk. That is, a filter element Mk acting on higher-frequency components has the coefficient ωk whose value is attenuated to a greater extent, because power spectra of input images generally have large amounts of low frequency components. This can improve the resolution of the image while maintaining the apparent naturalness.

As shown in FIG. 4, when the value of λ is varied, the coefficient ωk is attenuated to a different degree. For example, as the value of λ is increased, the coefficient ωk is less attenuated, and the coefficient ωk of the filter element Mk acting on higher-frequency components of the image can be increased. Conversely, as the value of λ is reduced, the coefficient ωk is more attenuated, and the coefficient ωk of the filter element Mk acting on higher-frequency components of the image can be reduced. That is, by varying the value of λ, the weight of each filter element Mk acting on each frequency component of the image can be varied. FIG. 4 shows an example in which n=10.
Where one wants to perform image processing such as edge enhancement on certain frequency components of an input image, the coefficients ωk of filter elements Mk acting on these frequency components are set to nonzero real numbers, and the coefficients ωk of the other filter elements Mk are set to 0.
The filter reconfiguration portion 15 shown in FIG. 2 reconfigures a first filter Fe(n,λ) that is a function of the number of filter elements n and of λ featuring the coefficients ωk, based on the n filter elements Mk (k=1 to n) stored in the filter element storage portion 13 and on the coefficients ωk (k=1 to n) generated by the coefficient creation portion 14, using the following equation:




  
    
      
        
          
            F
            e
          
          ⁡
          
            (
            
              n
              ,
              λ
            
            )
          
        
        =
        
          
            ∑
            
              k
              =
              1
            
            n
          
          ⁢
          
            
          
          ⁢
          
            
              π
              k
            
            ×
            
              Norm
              ⁡
              
                (
                
                  M
                  k
                
                )
              
            
          
        
      
    
    
      
        (
        17
        )
      
    
  




where Norm ( ) indicates normalization.

The first filter Fe(n,λ) having various characteristics depending on the values of λ and n can be created by adding up n filter elements Mk (k=1 to n) multiplied by the coefficients ωk based on λ so as to reconfigure a first filter that is an edge enhancement filter in this way. For example, as shown in FIG. 4, it is possible to create the first filter Fe(n,λ) having filter elements Mk which act on higher-frequency components and on which greater weights are attached by increasing the value of λ. Furthermore, the first filter Fe(n,λ) acting on lower-frequency components can be created by increasing the value of the number n.
The filter reconfiguration portion 15 reconfigures a second filter Fc(n,α), based on the distribution function G0 approximate to a delta function and on the distribution function Gn of the nth order, the functions being stored in the filter element storage portion 13, by the use of the following equation:

Fc(n,α)=(1−α)×Norm(G0)+Norm(Gn)  (18)

where α is a weight coefficient having a value from 0 to 1. The second filter Fc(n,α) having different characteristics according to the values of n and a can be created by combining together the distribution function G0 and the nth-order distribution function Gn using the arbitrary weight coefficient α so as to reconfigure the second filter being a noise filter in this way.

An example of the second filter Fc(n,α) is shown in FIG. 5. As shown in FIG. 5, in this second filter Fc(n,α), high intensity portions are distributed around the center. Low intensity portions are distributed in long tail portions. This filter acts as a noise filter (blur filter) for removing noise from the input image.
As shown in FIG. 5, the level of filtering of the second filter Fc(n,α) shows a distribution function G0 that approximates a delta function more closely as the value of the weight coefficient α is reduced. The distribution more closely approaches the nth-order distribution function Gn that is a Gaussian function as the value of the weight coefficient α is increased. That is, the noise filtering action of the second filter Fc(n,α) can be weakened by reducing the value of the weight coefficient α, and vice versa. Furthermore, since the dispersion of the distribution function Gn increases with increasing the value of n, the second filter Fc(n,α) can be so designed as to act on higher-frequency components of the input image by reducing the value of n. The second filter Fc(n,α) can be designed such that it acts on the low-frequency components of the input image by increasing the value of n.
In the technique of the present embodiment, filter elements having a small size compared with the input image (e.g., 9×9 filter elements) are arranged and expanded to reconfigure each of the first and second filters. Therefore, the processing time can be much shortened compared with a wavelet technique in which the input image itself is expanded.
The filter reconfiguration portion 15 adds up the first filter Fe(n,λ) and second filter Fc(n,α) to create an intermediate filter {tilde over (M)}(n,λ,α) for evaluation, using the following equation:

{tilde over (M)}(n,λ,α)=Fe(n,λ)+Fc(n,α)  (19)

An example of the intermediate filter {tilde over (M)}(n,λ,α) is shown in FIG. 6. The third filter F0 created by the third filter creation portion 50 is similar to the intermediate filter {tilde over (M)}(n,λ,α). As shown in FIG. 6, the intermediate filter {tilde over (M)}(n,λ,α) acts as an edge enhancement filter for emphasizing the edges of the input image and also as a noise filter for removing noise from the input image. Furthermore, as shown in FIG. 6, the intermediate filter {tilde over (M)}(n,λ,α) acts more as an edge enhancement filter with reducing the value of the weight coefficient α. The intermediate filter acts more as a noise filter with increasing the value of the weight coefficient α.
The convolution portion 16 shown in FIG. 2 creates an image IC by convolving the sample image Iin using the intermediate filter {tilde over (M)}(n,λ,α) as given by the following formula, the intermediate filter {tilde over (M)}(n,λ,α) being created by the filter reconfiguration portion 15.

Ic={tilde over (M)}(n,λ,α)Iin  (20)

The image quality evaluation portion 17 similarly calculates the S/N of the image Ic created by the convolution portion 16 using Eq. (7).
The convolution portion 16 creates the images by convolving a sample image using the intermediate filter {tilde over (M)}(n,λ,α) about all the combinations of n, λ, and α of the intermediate filter {tilde over (M)}(n,λ,α). The image quality evaluation portion 17 calculates the S/Ns of the created images Ic and finds a combination of n, λ, and a that gives a maximum value of S/N using the following equation. The first filter Fe(n,λ) having a combination of n and λ that gives a maximum S/N is output as the first filter Fc optimal for the sample image to the first filter storage portion 20 (see FIG. 1). The second filter Fc(n,α) having a combination of n and α that gives a maximum S/N is output as the second filter Fc optimal for the sample image to the second filter storage portion 22 (see FIG. 1).

SN→arg·max{SN|∀n,∀λ,∀α}  (21)

The power spectrum calculating portion 18 calculates a power spectrum, PS, of a sample spectrum similarly to the case using Eq. (5) and outputs the calculated spectrum to the first filter storage portion 20. The optimum first filter Fe created based on the sample image and the power spectrum PS of the sample image are correlatively stored in the first filter storage portion 20.
The S/N calculating portion 19 calculates the S/N, SN, of a sample image similarly to the case where Eq. (7) is used and outputs the result to the second filter storage portion 22. An optimum second filter Fc created based on the sample image and S/N, SN, of the sample image are correlatively stored in the second filter storage portion 22.
The processing consisting of creating the first filter Fe and second filter Fc from the sample image and calculating the power spectrum PS and S/N of the sample image in this way is carried out for plural different sample images.
According to the present embodiment, it is possible to create the first filters Fe and second filters Fc which are optimal for the sample images among the first filters Fc(n,λ) and second filters Fc(n,α) having various characteristics according to the values of n, λ, and α.
First filters (edge enhancement filters) adapted for input images having power spectra approximate to power spectra of sample images can be easily selected by storing first filters Fe optimal for the sample images correlatively with the power spectra of the sample images. Furthermore, second filters (noise filters) adapted for input images having S/Ns approximate to S/Ns of sample images can be easily selected by storing second filters Fc optimal for the sample images correlatively with the S/Ns of the sample images. In addition, image processing can be performed by the use of a filter which is optimal for an input image and which acts as an edge enhancement filter and as a noise filter by convolving the input image using a third filter created by summing up the selected first and second filters.
For example, in response to an input image having a large amount of high-frequency components, a first filter having a large value of λ and a small value of n is selected as an optimum first filter. As a result, a third filter more effectively acting on high-frequency components is created as a filter optimal for the input image. On the other hand, in response to an input image having a large amount of low-frequency components, a first filter having a small value of λ and a large value of n is selected as an optimum first filter. As a result, a third filter more effectively acting on low-frequency components is created as a filter optimal for the input image.
In response to an input image having a high S/N, a second filter having a small value of α is selected as an optimum second filter. Consequently, a third filter (see FIG. 6) acting mainly as an edge enhancement filter is created as a filter optimal for the input image. On the other hand, in response to an input image having a low S/N, a second filter having a large value of α is selected as an optimum second filter. As a result, a third filter (see FIG. 6) acting mostly as a noise filter is created as a filter optimal for the input image.
3. Processing
One example of the processing performed in the present embodiment is next described by referring to the flowcharts of FIGS. 7 and 8. FIG. 7 is a flowchart illustrating examples of a filter selection process and convolutional processing.
First, the first filter selection portion 30 calculates a power spectrum of an input image Iin using Eq. (5) (step S10). Then, the selection portion 30 calculates the error, err, between the power spectrum of the input image Iin and each power spectrum stored in the first filter storage portion 20 using Eq. (6) (step S12). The selection portion selects the first filter Fex corresponding to the power spectrum that minimizes the error, err, with the power spectrum of the input image Iin from the power spectra stored in the first filter storage portion 20 and reads the selected filter from the storage portion 20 (step S14).
Then, the second filter selection portion 32 calculates the S/N of the input image Iin using Eq. (7) (step S16). The selection portion 32 then computes the error, err, between the S/N of the input image Iin and each S/N stored in the second filter storage portion 22 using Eq. (10) (step S18). The selection portion selects a second filter Fex corresponding to the S/N that minimizes the error, err, with the S/N of the input image Iin from S/Ns stored in the second filter storage portion 22, and reads the selected filter from the second filter storage portion 22 (step S20).
The third filter creation portion 50 then adds up the first filter Fex selected in step S14 and the second filter Fex selected in step S20 to create a third filter Fo (step S22).
The convolutional processing portion 40 convolves the input image Iin using the third filter Fo created in step S22 to generate an output image Iout (step S24).
FIG. 8 is a flowchart illustrating one example of processing performed by the filter creation portion 10.
First, the filter creation portion 10 expands a filter G0 that is a distribution function approximate to a delta function into a distribution function Gn of the nth order and n filter elements Mk each of which is the difference between a distribution function Gk-1 of the (k−1)th order and a distribution function Gk of the kth order using Eq. (15) (step S30).
Then, n and λ are set to 1 and 0.5, respectively. α is set to Δα (0<Δα<1; for example, Δα=1) (step S32). The coefficients ωk of the filter elements Mk (k=1 to n) are computed using Eq. (16) (step S34).
Then, the first filter Fe(n,λ) is reconfigured by adding up together the n filter elements Mk multiplied by the coefficients ωk using Eq. (17) (step S36). Then, the second filter Fc(n,α) is reconfigured by combining together the nth-order distribution function Gn and the distribution function G0 with the weight coefficient α using Eq. (18) (step S38).
Then, the first filter Fe(n,λ) and the second filter Fc(n,α) are added up to create the intermediate filter {tilde over (M)}(n,λ,α) using Eq. (19) (step S40).
Then, the sample image is convolved using the intermediate filter {tilde over (M)}(n,λ,α) through Eq. (20) to generate the image IC (step S42). The S/N of the generated image Ic is calculated using Eq. (7) (step S44).
Then, a decision is made as to whether the value of n has reached a given maximum value, nmax (step S46). If not so, the value of n is incremented by 1 (step S48) and then control proceeds to step S34. The processing subroutine starting with step S34 is repeatedly carried out until the value of n reaches the maximum value, nmax.
If the decision at step S46 is that the value of n has reached the maximum value, nmax, a decision is made as to whether the value of λ has reached 0.9 (step S50). If the decision is NO, the value of λ is incremented by Δλ (step S52) and control goes to step S34. The processing subroutine starting with step S34 is repeatedly performed until the value of λ reaches 0.9.
If the decision at step S50 is YES (i.e., the value of λ has reached 0.9), a decision is made as to whether the value of α has reached a given maximum value, αmax (e.g., αmax=1) (step S54). If the decision at step S54 is NO, the value of α is incremented by Δα (step S56) and control proceeds to step S34. Then, the processing subroutine starting with step S34 is repeatedly performed until the value of αreaches αmax.
If the decision at step S54 is YES (i.e., the value of α has reached αmax (that is, the S/N of the image Ic has been calculated for all the combinations of n, λ, and α)), the first filter Fe(n,λ) having a combination of n and λ, giving a maximum S/N is taken as an optimum first filter Fe and output to the first filter storage portion 20. A second filter Fc(n,α) having a combination of n and α that gives a maximum S/N is taken as an optimum second filter Fc and output to the second filter storage portion 22 (step S58).
Power spectra of sample images are then calculated using Eq. (5) and output to the first filter storage portion 20. The S/Ns of the sample images are calculated using Eq. (7) and output to the second filter storage portion 22 (step S60).
4. Example of Image Processing
FIG. 9A shows one example of SEM image. FIG. 9B shows an output image obtained as a result of image processing of the input image that is the SEM image shown in FIG. 9A by the technique of the present embodiment. As shown in FIG. 9B, the technique of the present embodiment permits appropriate edge enhancement and noise removal of the input image. Consequently, the image quality (resolution) of the input image can be improved.
It is to be understood that the present invention is not restricted to the above embodiment but rather various changes and modifications may be made. The present invention embraces configurations substantially identical (e.g., in function, method, and results or in purpose and advantageous effects) with the configurations described in the preferred embodiment of the invention. Furthermore, the invention embraces configurations described in the embodiment and including portions which have non-essential portions replaced. In addition, the invention embraces configurations which produce the same advantageous effects as those produced by the configurations described in the preferred embodiment or which can achieve the same objects as the configurations described in the preferred embodiment. Further, the invention embraces configurations which are similar to the configurations described in the preferred embodiment except that well-known techniques have been added.
For instance, in the above embodiment, the present invention is applied to a case where image processing is applied to two-dimensional images. The invention can also be applied to signal processing of one-dimensional signal (such as audio signal).
Having thus described my invention with the detail and particularity required by the Patent Laws, what is desired protected by Letters Patent is set forth in the following claims.

## Claims

1. A method for signal processing, comprising the steps of:
calculating a power spectrum of an input image signal;
selecting a power spectrum approximate to the calculated power spectrum from a plurality of power spectra stored in a first filter storage portion;
selecting a first filter corresponding to the selected power spectrum from a plurality of first filters stored in the first filter storage portion;
calculating a S/N (signal-to-noise ratio) of the input signal;
selecting a S/N approximate to the calculated S/N from a plurality of S/Ns stored in a second filter storage portion;
selecting a second filter corresponding to the selected S/N from a plurality of second filters stored in the second filter storage portion;
creating a third filter by adding up the selected first and second filters; and
convolving the input signal using the created third filter.

2. A method for signal processing as set forth in claim 1,
wherein the first filters stored in said first filter storage portion have been created by a step of creating filters based on sample input signals and are correlated with power spectra of the sample input signals,
wherein the second filters stored in said second filter storage portion have been created by said step of creating filters and are correlated with S/Ns of the sample input signals, and
wherein during said step of creating filters, a filter of an arbitrary distribution function G0 is expanded into a series of n filter elements, where n is an arbitrary number, each of which is the difference between two distribution functions, coefficients of the filter elements are set at will, the filter elements are multiplied by the coefficients, respectively, and summed up to create the first filters being a function of n, where n is the number of the filter elements, and of the coefficients of the filter elements, the distribution function Gn of the nth order obtained when expanding the filter into the n filter elements and the distribution function G0 are combined together with arbitrary weight coefficients to create the second filters, and the value of n, the coefficients of the filter elements, and the weight coefficients are set based on the sample input signals.

3. A method for signal processing as set forth in claim 2, wherein during said step of creating filters, the value of number n, the coefficients of the filter elements, and the weight coefficients are set to maximize the S/Ns of the signals obtained by convolving the sample input signals using the created first and second filters.

4. A method for signal processing as set forth in any one of claims 2 and 3, wherein each of said distribution functions is a Gaussian function.

5. An apparatus for signal processing, comprising:
a first filter storage portion in which first filters created based on sample input signals and power spectra of the sample input signals are correlatively stored;
a second filter storage portion in which second filters created based on sample input signals and S/Ns (signal-to-noise ratio) of the sample input signals are correlatively stored;
a first filter selection portion for calculating a power spectrum of an input signal, selecting a power spectrum approximate to the calculated power spectrum from the power spectra stored in the first filter storage portion, and selecting a first filter corresponding to the selected power spectrum from the first filters stored in the first filter storage portion;
a second filter selection portion for calculating a S/N of the input signal, selecting a S/N approximate to the calculated S/N from the S/Ns stored in the second filter storage portion, and selecting a second filter corresponding to the selected S/N from the second filters stored in the second filter storage portion;
a third filter creation portion for creating a third filter by summing up the selected first and second filters; and
a convolutional processing portion for convolving the input signal using the created third filter.

6. An apparatus for signal processing as set forth in claim 5,
wherein there is further provided a filter creation portion for creating the first and second filters based on the sample input signals and calculating power spectra and S/Ns of the sample input signals, and
wherein said filter creation portion expands a filter of an arbitrary distribution function G0 into a series of n filter elements, where n is an arbitrary number, each of which is the difference between two distribution functions, sets coefficients of the filter elements at will, multiplies the filter elements by the coefficients, respectively, and sums up the multiplied filter elements to create the first filter that is a function of the number of the filter elements n and of the coefficients of the filter elements, combines together a distribution function Gn of the nth order obtained when expanding the filter into the n filter elements and the distribution function G0 with an arbitrary weight coefficient to create the second filter, and sets the value of n, the coefficients of the filter elements, and the weight coefficient based on the sample input signals.

7. An apparatus for signal processing as set forth in claim 6, wherein said filter creation portion sets the value of the number n, the coefficients of the filter elements, and the weight coefficients to maximize the S/N of the signals obtained by convolving the sample input signals using the created first and second filters.

8. An apparatus for signal processing as set forth in any one of claims 6 and 7, wherein each of said distribution functions is a Gaussian function.