Abstract:
A method for correcting an image from defects and filtering from Gaussian noise corrects each pixel of the image when it is considered defective and filters it from Gaussian noise in one-pass. The one-pass improves the speed for performing the correcting and filtering. The drawbacks associated with choosing incompatible defect correction and filtering operations are overcome.

Description:
FIELD OF THE INVENTION  
       [0001]     The invention relates to image processing, and more precisely, to a method for determining a defective pixel in an image, a method for correcting the defective pixel in the image and a method for filtering a noisy pixel in the image, and a corresponding relative hardware implementation of these methods.  
       BACKGROUND OF THE INVENTION  
       [0002]     Digital cameras generate an array of pixels that represents an image acquired by the cameras. A very diffused format for an array of pixels is the Bayer mosaic pattern layout, which is shown in  FIG. 1 . In a Bayer pattern, each pixel contains information that is relative to only one color component, for instance, red, green or blue. Typically, the Bayer pattern includes a green pixel in every other space, and in each row, either a blue or a red pixel occupies the remaining spaces.  
         [0003]     As shown in  FIG. 1 , there are rows that alternate green G and red pixels R, and rows that alternate green G and blue pixels B. Therefore, a Bayer image is substantially a mosaic of red, green and blue pixels, where there are twice as many green pixels as red or blue pixels. This array can accurately represent an image because the human eye is more sensitive to the green data than either the red or blue.  
         [0004]     To better show the features of the invention, the ensuing description will be referred to Bayer images, but the same considerations apply to other kinds of images, such as black and white images or any other type of image detected by a color filter sensor of a camera.  
         [0005]     CMOS image sensors generate images that could be affected by different kinds of noise. The main causes of badly acquired pictures are Gaussian noise, defective pixels and white tailed noise.  
         [0006]     With Gaussian noise the assumption of AWGN (additive white gaussian noise) is quite common and it normally refers to the noise randomly covering the image data, usually generated by many factors such as dark currents, high gain values etc. With defective pixels spikes, dead pixels and generally flawed pixels are located at the same locations of the image sensor and they are always set to flawed values. In particular, dead pixels are set to values close to zero, while spikes are set to near saturation or to saturation values.  
         [0007]     White tailed noise, this is a particular kind of noise, visible in particular under low light conditions. It causes unusually bright impulsive pixels (spikes) located at random positions of the image. The difference with the above mentioned spiked or dead pixels is due to the fact that it is not possible to build a map to track them because they always appear in random locations. Furthermore, spikes generated by this noise can be close to each other. This makes the correction process of these pixels particularly tough as more than one spike could be present in the filter mask at the same time.  
         [0008]     In this description, pixels corrupted by white tailed noise, spiked and dead pixels will be referred as defective pixels. Consequently, this attributes to this expression a broader meaning than that commonly attributed to it in literature.  
         [0009]     Noise level typically increases in low light conditions because the image signal has to be amplified. This is normally achieved by increasing gains in the imaging pipeline that strengthen the signal so as to yield an acceptable picture. Obviously, the gain values are applied to the image signal and to the superimposed noise as well. Hence, one cannot boost the image signal without pushing up the noise level too.  
         [0010]     In digital cameras a color filter is placed on top of the imager. Thus each pixel is sensitive to one color component only. A color reconstruction algorithm interpolates the missing information at each location and reconstructs the full RGB image.  
         [0011]     Noisy Bayer data appears as shown in  FIG. 2 . The intensities of pixels deviate from their correct value by some constant drawn from a Gaussian distribution. Furthermore, a certain number of pixels have completely flawed values. These pixels are called outliers. They usually manifest as impulsive noise spread over the Bayer data and appear as bright white or dark black points.  
         [0012]     Especially under low light conditions, spikes can be close to each other, as shown in  FIG. 3 . Arrangements of noisy pixels such as the one shown in  FIG. 3  are particularly tough to remove. One of the two spikes could be preserved and considered in the final weighted average for removing Gaussian noise. Furthermore, dead pixels could also be present at random locations.  
         [0013]     Dead pixels may also appear at random locations of the Bayer data array. A pixel whose flawed value is not set to near zero or near saturation values is treated as corrupted by Gaussian noise or as normal non-extreme outlier by a Duncan filter [5].  
         [0014]     Techniques have been developed to reduce the effects of noise to generate pleasant images in every lighting condition. Typically, the noise superimposed all over the image signal is assumed to be Gaussian distributed. This has enabled the realization of powerful filters that enhance image quality significantly [4], [6]. Unfortunately noisy pixels exist for which the assumption of Gaussian noise is not valid at all. An example of non-Gaussian noise is represented by the defective dead or spiked pixels in the image.  
         [0015]     These defective pixels are removed by considering a map denoting their position and treating them accordingly using a defect correction algorithm. A Gaussian noise reduction filter and a defect correction algorithm efficiently remove noise. Thus, one approach for getting rid of both Gaussian noise and defective pixels consists in treating the two noises separately using two different cascaded filters. Two processing modes are possible: 
        i) scanning the image array twice, removing white tailed noise together with defective pixels, and Gaussian noise at each pass; and     ii) scanning the image once, processing each pixel by a different filter for removing white tailed noise together with defective pixels, and Gaussian noise.        
 
         [0018]     A drawback of scanning twice the image is the slowness of the correction process of the image, while treating a scanned pixel with two filters is faster, but the filters should be adequately chosen for preventing the generation of artifacts or deletion of details of the image. Therefore, a fast method for correcting defective pixels and filtering Gaussian noise that does not generate artifacts and preserves details is needed.  
       SUMMARY OF THE INVENTION  
       [0019]     The invention provides a method for deciding if there is a defective pixel in a set of pixels of a digital image or in a sequence of digital images that is dead, including spiked pixels even if they are generated by white tailed noise.  
         [0020]     The method comprises choosing a set of pixels, with each pixel having two closest neighbor pixels belonging to the set; calculating the absolute value of differences of intensity of pairs of neighbor pixels; and excluding the presence of any defective pixel when no more than one of the absolute values is larger than the pre-established threshold, otherwise deciding whether or not there is a defective pixel in function of the comparison.  
         [0021]     Preferably, no pixel is identified as defective if more than two of the above differences out of eight are larger than the threshold, in order to preserve texture portions of the image.  
         [0022]     In one aspect of the invention, it is possible to identify the defective pixel and to correct it, or it is possible to correct a pair of pixels containing the defective pixel without locating exactly which one is defective. In the first case, the defective pixel is located by identifying the pixels for which the absolute values of both the differences between its intensity and the intensity of one and the other neighbor pixel of the set surpass the pre-established threshold.  
         [0023]     In the other case, corresponding to a preferred embodiment of the invention, the largest absolute value of the above mentioned differences and the pair of pixels used for calculating it are identified. Then the correct intensities of this pair of pixels are evaluated with the median intensity among the intensities of the two pixels of the pair and of a pixel neighbor to them.  
         [0024]     According to another aspect of the invention, the intensity of an identified defective pixel may be corrected by substituting its intensity with the intensity of one of the neighbor pixels of the set or with a combination thereof.  
         [0025]     Each pixel of an image of a sequence of images may be corrected and filtered from Gaussian noise by selecting pixels neighbor to it; checking whether there is or not a defective pixel among the neighbor pixel by calculating the above mentioned differences, and evaluating its correct intensity as previously explained; checking whether the central pixel is defective and eventually correcting its intensity, generating a corrected pixel; and filtering from Gaussian noise the corrected pixel by substituting its intensity with a weighted sum of the intensities of the neighbor pixel and the pixel itself.  
         [0026]     With this method, each pixel is corrected from eventual defects and filtered from Gaussian noise in a single scanning. Moreover, the method is very straightforward and there are no problems of compatibility between the defect correction and the Gaussian noise filtering.  
         [0027]     A hardware device for performing a Gaussian noise filtering is also disclosed. The methods may be easily implemented with a software computer program. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0028]     The various aspects and advantages of the invention will be even more evident through a detailed description referring to the attached drawings, wherein:  
         [0029]      FIG. 1  is a sample array of pixels detected by a Bayer color filter according to the prior art;  
         [0030]      FIG. 2  is a sample noisy Bayer data according to the prior art;  
         [0031]      FIG. 3  is a magnified view of a portion of  FIG. 2  showing two adjacent impulsive defective pixels;  
         [0032]      FIG. 4  shows a typical 3×3 selection window according to the present invention;  
         [0033]      FIG. 5  shows a diamond shaped selection window for green pixels of a Bayer image according to the present invention;  
         [0034]      FIGS. 6   a  and  6   b  show rectangular windows for red and blue pixels of a Bayer image according to the present invention;  
         [0035]      FIG. 7  shows how to calculate intensity differences between neighbor pixels of a Bayer image selected by a diamond shaped window according to the present invention;  
         [0036]      FIG. 8  is a sample intensity distribution of pixels of  FIG. 7  with two spiked pixels;  
         [0037]      FIG. 9  is a sample intensity distribution of pixels of  FIG. 7  with a dead pixel;  
         [0038]      FIG. 10  shows how to assign weights to pixels according to the present invention;  
         [0039]      FIG. 11  is a diagram of intensities of a spiked central pixel of a selection window surrounded by non-spiked peripheral pixels according to the present invention;  
         [0040]      FIG. 12  is a diagram of intensities of a dead central pixel of a selection window surrounded by non-dead peripheral pixels according to the present invention;  
         [0041]      FIG. 13  shows a diamond shaped selection window with two adjacent dead pixels according to the present invention;  
         [0042]      FIG. 14  is a diagram of intensities of pixels of  FIG. 13 ;  
         [0043]      FIG. 15  is a diagram of the intensities of pixels of  FIG. 13  after the dead pixel of the peripheral pixels has been corrected;  
         [0044]      FIG. 16  is a diagram of the intensities of pixels of  FIG. 13  after both the peripheral and central dead pixels have been corrected;  
         [0045]      FIG. 17  shows a diamond shaped selection window with two adjacent spiked pixels according to the present invention;  
         [0046]      FIG. 18  is a diagram of intensities of pixels of  FIG. 17 ;  
         [0047]      FIG. 19  is a diagram of the intensities of pixels of  FIG. 17  after the spiked pixel of the peripheral pixels has been corrected;  
         [0048]      FIG. 20  is a diagram of the intensities of pixels of  FIG. 17  after both the peripheral and central spiked pixels have been corrected;  
         [0049]      FIG. 21  is a flow-chart of a preferred embodiment of a method according to the invention for correcting defective pixels;  
         [0050]      FIG. 22  compares results of a recursive version of a method for correcting defective pixels with a non-recursive version thereof according to the present invention;  
         [0051]      FIGS. 23   a  and  24   a  show two noisy Bayer images according to the present invention;  
         [0052]      FIGS. 23   b  and  24   b  respectively show filtered versions of the images of  FIGS. 23   a  and  24   a  with the method of the invention for correcting defective pixels;  
         [0053]      FIG. 25  compares portions of the images of  FIGS. 24   a  and  24   b;    
         [0054]      FIG. 26  shows a sample embodiment of a method for filtering noise according to the present invention; and  
         [0055]      FIG. 27  shows a hardware device for implementing a method for filtering noise according to the present invention. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0056]     A selection window commonly used for filtering noise corrupting pixels of an image is the rectangular 3×3 selection window illustrated in  FIG. 4 . In order to include 9 valid pixels of a same color (green, red or blue) of a Bayer image, the dimensions of the 3×3 window have to be enlarged to 5×5 and two selection windows are considered: a diamond shaped mask for green pixels G as shown in  FIG. 5 , and a rectangular mask for red R and blue B pixels as shown in  FIGS. 6   a  and  6   b , respectively.  
         [0057]     In the ensuing description of the different embodiments of the invention, reference will be made to a set of nine green pixels selected with a diamond shaped window over a static Bayer image, though the same considerations apply also in case of a rectangular selection window for selecting red or blue pixels, or in case the selected pixels of the set are chosen among different images of a sequence of images.  
         [0058]     To summarize, the selection window selects a set of nine pixels of the same color of a Bayer image, one of which is located at the center of the window and will be referred as a central pixel, while the remaining eight pixels will be referred as peripheral pixels.  
         [0059]     To remove the noise described above, the algorithm for correcting pixels are designed to take into account the possibility that the central pixel, and probably one of the peripheral pixels, are outliers.  
         [0060]     As usually done in image filtering processes, the image array is scanned in a top-down manner starting from the top leftmost pixel. Depending on the color of the pixels to be processed, the appropriate selection window (rectangular or diamond shaped) is chosen having the pixel to be processed as a central pixel. In so doing, the pixels of the image borders cannot be processed by the selection window, thus they are copied in an output buffer without being processed.  
         [0061]     Let us suppose that defective green pixels of an image are sought. A set of pixels to be processed is selected by a diamond shaped window, such as shown in  FIG. 7 . In the set of selected pixels, there are eight peripheral pixels of the selection window G 0 , . . . , G 7 , and a central pixel G c . Moreover, each peripheral pixel has two neighbor peripheral pixels.  
         [0062]     The first step to be performed is to identify whether there is or not a defective pixel among the peripheral pixels of the selected set. Differences related to the peripheral pixels are computed: 
    D 0 =|G 0 -G 1 |; D 1 =|G 1 -G 2 |; D 2 =|G 2 -G 3 |; D 3 =|G 3 -G 4 |;     D 4 =|G 4 -G 5 |; D 5 =|G 5 -G 6 |; D 6 =|G 6 -G 7 |; D 7 =|G 7 -G 0 |.    
 
         [0065]     The order in which these differences are calculated has no relevance, thus a clockwise elaboration is equivalent to a counter-clockwise one. Moreover, the computation of the differences may start from any peripheral pixel.  
         [0066]     Each difference D i  is compared with a threshold that preferably is a multiple Kσ of the estimated noise standard deviation σ. The constant value K has to be determined according to the range of differences that can be considered to be safe. Tests showed that a suitable value is K=4 for 8 bit-precision Bayer data, though the value K may be different for satisfying other specific requirements.  
         [0067]     When comparing these differences with the threshold Kσ, the number of differences that are less than or equal to Kσ is counted. If six differences out of eight are smaller than the threshold and the two differences larger than Kσ are consecutive, then among the peripheral pixels G 0 , . . . , G 7  is likely an outlier. In fact, the presence of an outlier affects two consecutive differences out of the eight computed ones that are the differences between the intensity of the outlier and the intensities of one and the other neighbor peripheral pixel.  
         [0068]     According to the invention, a pixel is identified as defective when the differences between its intensity and the intensity of one and the other neighbor peripheral pixel surpass the threshold Kσ.  
         [0069]     A very straightforward way of identifying such a defective pixel will now be described. Let D i  be the maximum difference. As D i =|G i −G i+1 |, then D i  is univocally associated to two pixels, namely G i  and G i+1 . If the outlier is a spike, the defective pixel is the pixel that has the maximum intensity between the intensities of the pixels G i , and G i+1 .  
         [0070]      FIG. 8  shows an example in which two spikes appear in the same working window. The computed differences are:  
                                                                           D 0     D 1     D 2     D 3     D 4     D 5     D 6     D 7                             0   124   128   7   3   10   5   5                        
         [0071]     It is evident that two (D 1  and D 2 ) of the above differences are significantly greater than the others. D 1  and D 2  indicate that a pixel is likely to be defective among in G 1 , G 2  or G 3 .  
         [0072]     The maximum difference D 2  indicates that the pixels to be considered are G 2  and G 3 . One of them is probably a spike and has to be removed. According to an embodiment of the invention, the defective pixel is the pixel having the largest intensity max(G 2 , G 3 ), i.e., max(254, 106); the spike is G 2 =254.  
         [0073]     At this point, the correct intensity of the spike is to be estimated. Reasonable choices are: 
        i) the intensity of G 2  is the mean intensity between G 1  and G 3 ;     ii) the intensity of G 2  is the intensity of G 1 ; and     iii) the intensity of G 2  is the intensity of G 3 .        
 
         [0077]     The option i) is discarded because it requires a division, which is computationally heavy. The options ii) and iii) are identical under a computational point of view. Thus, for sake of example, option iii) is chosen, though similar considerations apply for option ii).  
         [0078]     The above described steps are very straightforward to be implemented and very effective if the defective peripheral pixel is a spiked pixel. When the defective peripheral pixel is a dead pixel, the method steps previously described need further refinements. This situation is considered below by way of an example.  
         [0079]     Let us consider the sample working window of  FIG. 9 . The differences D 0 , . . . , D 7  are:  
                                                                           D 0     D 1     D 2     D 3     D 4     D 5     D 6     D 7                             0   130   106   7   3   10   5   5                      
 
         [0080]     The two consecutive differences larger than the threshold Kσ of interest are D 1  and D 2 , and D 1  is the maximum difference. Unfortunately, D 1  is not a spike but it is a dead pixel. By using the previous method steps, the intensity of G 2 , would be estimated with the intensity of G 2 , but this is not correct because this would “kill” the pixel G 1  too.  
         [0081]     According to an embodiment of the method of the invention, a very straightforward way of estimating the correct intensity of a peripheral defective pixel includes considering the two pixels G i  and G i+1  that generate the maximum difference D i  and a neighbor pixel G i−1 . Then the median value of the intensity of these three pixels is considered to be the correct intensity of both pixels that generate the maximum difference.  
         [0082]     In the previous example the pixels that generate the maximum difference D 1  are the pixels G 1  and G 2 . According to the invention, the correct intensities of the pixels G 1 and G   2  are the median intensity of the pixels G 0 , G 1 , and G 2 , that is 130.  
         [0083]     By taking the median value, the intensity of the dead pixel will never be chosen as a suitable value. It is worth noticing that evaluating the correct intensity of the two peripheral pixels that generate the maximum difference with the median intensity among the two pixels and another neighbor peripheral pixel is an effective expedient even for peripheral spiked pixels. Indeed, it is easy to understand that even the intensity of a spiked pixel will never be chosen as a median intensity.  
         [0084]     Another key aspect to consider is that edges of objects represented in the image are not deleted because the previous modifications to the peripheral pixels of the chosen set of pixels are applied only if two consecutive differences (that are two differences involving a same peripheral pixel) out of eight are larger than the threshold Kσ. According to an embodiment of the invention, if less than 6 differences out of 8 are smaller than the threshold Kσ, then the area is considered as texture and no pixel is identified as defective.  
         [0085]     After an eventual peripheral dead or spiked pixel has been identified and its correct intensity has been evaluated, the central pixel P, which is the pixel to be corrected and filtered from Gaussian noise, is processed. According to the invention, the intensity of the central pixel is corrected and filtered from Gaussian noise as a function of the correct intensities of the peripheral pixels. That is, the intensities of the peripheral pixels not recognized as defective and the estimated correct intensity of an eventually present defective peripheral pixel.  
         [0086]     Three alternatives are possible: 1) the central pixel P is correct; 2) the central pixel P is affected by Gaussian noise; and 3) the central pixel P is an outlier (dead or spike). To better understand the method of the invention for correcting the central pixel P and for filtering Gaussian noise from all the pixels of the set, let us refer to the particular case of  FIG. 10 .  
         [0087]     According to a preferred embodiment, three intervals centered on the intensity P of the central pixel and on the noise increased P+σ and decreased P−σ version thereof are considered, wherein σ is the standard deviation of the noise corrupting the image. The noise biased intensities P−σ and P+σ are considered because the central pixel can be corrupted by Gaussian noise.  
         [0088]     As an alternative, the noise increased P+σ and decreased P−σ version of the central pixel P are obtained by increasing or reducing, respectively, the intensity of the central pixel by a fraction or a multiple of the standard deviation σ, or only the central pixel P and the noise increased P+σ or decreased version P−σ thereof can be considered.  
         [0089]     Preferably, three intervals of increasing wideness 2*Th 1 , 2*Th 2  and 2*Th 3  are chosen and their values are set to 2σ, 4σ and 6σ, respectively. For sake of clarity, in  FIG. 10  only the intervals 2*Th 3  for P−σ and P are represented with a dash and dot line. Of course, there can be even more or less than three intervals and they can be assigned a value that is a fraction or another multiple of the standard deviation  
         [0090]     Let P be the intensity of the central pixel P of the selection window and P 0 , . . . , P 7  the intensities of the peripheral pixels; the following values are computed for i=0, . . . , 7:  
         d   i     =          P   -     P   i                
         dm   i     =            (     P   -   σ     )     -     P   i                
         dp   i     =            (     P   +   σ     )     -     P   i                
       and   ⁢     :         
         DC   =       ∑     i   =   0     7     ⁢     d   i         ;     DM   =       ∑     i   =   0     7     ⁢     dm   i         ;     DP   =       ∑     i   =   0     7     ⁢       dp   i     .             
 
         [0091]     The minimum sum of differences among DC, DM and DP is computed: MinDiff=min(DC,DM,DP). Then: If MinDiff is DC, the intensity P is considered. If MinDiff is DM, the intensity P−σ is considered. If MinDiff is DP, the intensity P+σ is considered.  
         [0092]     In the example of  FIG. 10 , the intensity P+σ is considered. A problem arises when the central pixel P is an outlier. In fact, if P is a dead pixel belonging to a bright area or if it is a spike belonging to a dark area, then even the interval 2*Th 3  might not contain any pixel at all. A sample situation of a central spiked pixel is shown in  FIG. 11 , wherein the eight peripheral pixels are represented with a green circle, while the central pixel and its noise increased or decreased versions are represented with a white circle with a red border.  
         [0093]     In the case of a dead central pixel P located in a non-black area, sometimes even if P+σ is considered no pixels is included in the widest interval 2*Th 3 . This situation is shown by way of an example in  FIG. 12 .  
         [0094]     By referring to  FIGS. 11 and 12 , the following observations hold. If P is a spike, then the intensity P−σ will be considered as the one closest to the majority of the other pixels, because P−σ is closer to the other pixels if compared to P&#39;s and P+σ&#39;s distance. A similar consideration holds in the case in which P is a dead central pixel; in this case P+σ is closer to the majority of the other pixels.  
         [0095]     This suggests the way in which an outlier occurring in the central position could be identified. Each time the interval centered on P−σ (P+σ) is selected, then P−σ (P+σ) cannot be automatically identified as an outlier because DM or DP could be the minimum without P being an outlier. Hence, the criteria for extreme outliers identification is reinforced with further constraints as follows: 
        i) if DM is chosen and P−σ is greater (brighter) than the maximum intensity among peripheral pixels, then the central pixel is a spike to be corrected;     ii) if DM is chosen but P−σ is smaller (darker) than or equal to the maximum intensity among peripheral pixels, then the central pixel is not recognized as an outlier;     iii) if DP is chosen and P+σ is smaller (darker) than the minimum intensity among peripheral pixels, then the central pixel is a dead pixel to be corrected;     iv) if DP is chosen and P+σ is greater (brighter) than the minimum intensity among peripheral pixels, then the central pixel is not recognized as an outlier; and     v) if DC is chosen then P is supposed to be affected by Gaussian noise.        
 
         [0101]     When the central pixel is judged to be an outlier, it must be substituted accurately; an improper change of the central pixel P may very likely generate strange artifacts in the image, especially along edges of represented objects.  
         [0102]     As previously stated, the central pixel is recognized as an outlier in cases i) and iii); its substitution could be coarsely implemented using the two following rules: 
        a) if the central pixel in the selection window is identified as a spike, changing its value using the maximum intensity (brightest) among peripheral pixels; and     b) if the central pixel in the selection window is identified as a dead one, changing its value using the minimum intensity (darkest) among peripheral pixels.        
 
         [0105]     This choice is fast and produces good results, but it is not the best approach because some artifacts could arise in the image. The best approach that has been determined after many tests, comprises performing the following steps: 
        a) if the central pixel in the selection window is identified as a spike, changing its value using the average of all the peripheral pixels; and     b) if the central pixel in the selection window is identified as a dead one, changing its value using a weighted average of the four darkest peripheral pixels. Specifically, if P 0 , P 1 , P 2  and P 3  are the darkest peripheral pixels in ascending order (i.e., P 0  is the darkest and P 3  is the brightest of the darkest pixels), then changing the central pixel by substituting the following value: (V 0 *P 0 +V 1 *P 1 +V 2 *P 2 +V 3 *P 3 )/(V 0 +V 1 +V 2 +V 3 ), where V 0 =8, V 1 =4, V 2 =2, V 3 =2 are one of the possible choices for the weighting coefficients.        
 
         [0108]     Once the central pixel P has been corrected because it was a spike or a dead pixel, it is filtered from Gaussian noise, as explained below. Depending on which value between DC, DM and DP is the minimum, the values d i , dm i  or dp i  are chosen. Then peripheral pixels in the interval of amplitude 2*Th 1  are assigned a weight W 1  (high similarity), e.g., W 1 =8; peripheral pixels comprised in the interval of amplitude 2*Th 2  but not in the interval of amplitude 2*Th 1  are assigned a weight W 2  (medium similarity), e.g., W 2 =2; peripheral pixels comprised in the interval of amplitude 2*Th 3  but not in the interval of amplitude 2*Th 2  are assigned a weight W 3  (low similarity), e.g., W 3 =1; and peripheral pixels outside the largest interval 2*Th 3  are considered non-similar to P and are assigned a null weight.  
         [0109]     The intensity of the noise-free version of the central pixel P is obtained as a weighted sum of the intensity of the central pixel weighted with the largest weight W 1 , with the intensities of the peripheral pixels weighted with the weights previously assigned to them.  
         [0110]     The above method steps for correcting and filtering the central pixel P from noise are repeated for each pixel of an image to be filtered but the border pixels, which are simply copied in an output buffer, generate a defect-free and noise-free image.  
         [0111]     With the method of the invention, each pixel of the image is processed for correcting it and filtering Gaussian noise in a single pass. Therefore, it is not necessary to scan twice the image for correcting defects and filtering Gaussian noise, but these two operations are performed with a single scanning on each pixel without generating artifacts or having problems of compatibility between the defects correction and the noise filtering operations.  
         [0112]      FIG. 13  shows pixels selected with a diamond shaped selection window, two of which are dead pixels. According to the method of the invention for identifying defective pixels, the peripheral dead pixel P 4  is identified, as schematically shown in  FIG. 14 . Then, as shown in  FIG. 15 , the correct intensity of the peripheral pixel P 4  is estimated with the median value of the intensity of pixels P 3 , P 4  and P 5 . The central pixel is identified as a dead pixel and its intensity is corrected with the average of intensities of its neighbor pixels.  
         [0113]     Similarly, FIGS.  18  to  20  describe method steps of the invention for correcting a central defective spiked pixel in the example of  FIG. 17 . A flow-chart of a preferred embodiment of the method of the invention is depicted in  FIG. 21 . Substantially, the following steps are performed: 
        scanning a pixel P to be processed;     choosing a set of pixels neighbor to it P 0 , . . . , P 7 ;     processing the peripheral pixels to seek defective pixels. If a defective pixel is found, estimating its intensity with the median intensity among the intensities of the neighbor pixels and the intensity of the pixel itself;     considering the central pixel P, which is the pixel to be corrected and filtered from noise, and its noise increased and decreased versions. Determining whether the central pixel is defective or not and eventually correcting its intensity; and     finally, filtering from Gaussian noise the central pixel weighing the corrected intensities of the peripheral pixels with the weights W 1 , . . . , W 3  determined as shown above referring to the diagram of  FIG. 10 .        
 
         [0119]     The values of the weights may vary depending on the specific filtering strength required. This method of the invention may be carried out in different alternative versions. In general, the pixels of an image to be processed are stored in an input memory buffer and the pixels of the processed image are stored in an output memory buffer. Each pixel is processed by selecting the pixels itself and pixels of the image neighbor to it.  
         [0120]     According to a first version, referred to as a “recursive embodiment”, each processed pixel is stored in the output buffer and is also overwritten in the input buffer over the corresponding non-processed pixel of the original image. According to a second version, referred to as “conditionally-recursive embodiment”, the processed pixel is overwritten also in the input buffer only if the corresponding non-processed pixel has been identified as a defective pixel, otherwise it is simply stored in the output buffer.  
         [0121]     Of course, it is possible to carry out the method of the invention in a “non-recursive” way, that is, without overwriting pixels in the input buffer even if they are recognized as defective.  
         [0122]     A drawback of the recursive embodiment is that the edges of objects represented in the processed image result are less sharp than in an image processed with the conditionally-recursive embodiment. In contrast, the recursive embodiment seems to be more efficient in removing white tailed noise than the conditionally-recursive embodiment.  
         [0123]      FIG. 22  shows a noisy image and two corresponding images filtered with the recursive and conditionally-recursive embodiment of the method of the invention. Specifically, the left column shows a homogeneous area were three multiple instances of defective pixels are present (triplets in this case). The right column shows the results of the filter without recursion; two spikes are left after filtering. The central column demonstrates that with the recursive embodiment it is possible to remove all the spikes but loosing some sharpness of the edges of the letter “L” and the number “1”.  
         [0124]     To assess the filtering procedure, a very noisy and defective Bayer pattern was fed in input to the proposed filtering algorithm.  FIG. 23  shows the results in the case of Gaussian noise and single and double spiked pixels.  
         [0125]      FIGS. 23   a ,  23   b ,  24   a ,  24   b  and  25  show the results of the method of the invention for correcting defective pixels and noise filtering. It is evident that defective pixels are removed and details are preserved as much as possible.  
         [0126]     The methods of the invention are preferably performed with a software computer program. A hardware device for filtering a pixel from Gaussian noise is shown in  FIG. 27  and is described in detail in the ensuing paragraph.  
         [0127]     The invention also provides a novel method of filtering noise from a set of pixels of images of a sequence of images. Method of filtering noise from digital images of a image sequence are described in published European patent applications EP 1,100,260 and EP 1,394,742 which are assigned to the current assignee of the invention and are also incorporated herein by reference in their entirety.  
         [0128]     According to the methods described in the cited applications, a working window of pixels to be filtered is chosen by selecting a noisy pixel to be filtered P of a current picture of a sequence of pictures and a set of spatially or temporally neighbor pixels of the same or of another picture of the sequence, respectively. The absolute value of the difference between the intensities of the noisy pixel P, of a noise reduced version thereof P−σ and of a noise increased version thereof P+σ with each of the neighbor pixels are calculated.  
         [0129]     Each of these differences is compared with different thresholds, for example, a small threshold Th 1 , a medium threshold Th 2  and a large threshold Th 3 . Depending on this comparison, a respective weight W 1 , . . . , W 3  or a null weight is assigned to each neighbor pixel. Then first, second and third weighted sums of intensities of the pixel P, P−σ or P+σ, respectively, and of the neighbor pixels are calculated using the weights assigned to them, and the largest of these sums is chosen. Finally, the filtered intensity of the pixel P is determined as the ratio between the largest sum and the sum of the weights used for calculating this sum.  
         [0130]     The methods disclosed in the cited prior applications are time consuming because all the above mentioned differences must be compared with all the thresholds.  
         [0131]     According to a preferred embodiment of the method of the invention for filtering noise from a selected set of pixels of an image, a first DC, second DM and third DP sum of the absolute values of the differences between the intensities of the pixel P, P−σ or P+σ, respectively, with the intensities of the neighbor pixels are calculated, and the smallest of these sums is chosen. Then only the absolute values of the differences used for obtaining the smallest sum are compared with the thresholds Th 1 , Th 2  and Th 3  in order to assign weights to each pixel.  
         [0132]     In so doing, the number of comparisons with a threshold is strongly reduced with respect to the algorithm disclosed in the cited prior applications because these comparisons are performed after the intensity P, P−σ or P+σ has been chosen, thus the method of the invention for filtering noise is surely faster than the methods of the prior application.  
         [0133]     This method of filtering noise is effective even if the pixel P and its neighbor pixels do not belong to the same image, but to consecutive images of an image sequence. It may be applied to any number of selected neighbor pixels of a noisy pixel P, and the intensity of the noise increased and decreased versions of the pixel P may be obtained by summing or subtracting a fraction or a multiple of the standard deviation σ of the noise corrupting the pixels of the set.  
         [0134]     A sample embodiment of this method is given considering seventeen neighbor pixels P 0 , . . . , P 16  of a pixel P to be filtered from noise, chosen among pixels belonging to the same image of the pixel P and to images that precede or follow in a sequence of images.  
         [0135]      FIG. 26  is a sample diagram of the differences of intensity 
   d   i   =|P−P   i |   dm   i =|( P −σ)−P i |   dp   i =|( P +σ)− P   i | 
         [0136]     According to the method of the invention, the sums  
         DC   =       ∑     i   =   0     16     ⁢     d   i         ;     DM   =       ∑     i   =   0     16     ⁢     dm   i         ;     DP   =       ∑     i   =   0     16     ⁢     dp   i         ;       
 
 are calculated and the smallest of them is identified, which in the example of  FIG. 26  is DC. Thus, the differences d i =|P−P i | are compared with certain thresholds Th 1 , Th 2  and Th 3  and a weight W 1 , W 2 , W 3  or 0 is assigned to each neighbor pixel P i  depending on which is the smallest threshold larger than the difference d i . 
 
         [0137]     It is worth remarking that this method of filtering noise is effective even with a number of thresholds larger or smaller than three.  
         [0138]     Finally, a noise filtered version of the pixel P is obtained by correcting the intensity of the pixel P with the weighted sum of intensities of the neighbor pixels, each with the respective weight determined at the previous step, and of the noisy pixel P itself.  
         [0139]     Preferably, the intensity of the noisy pixel P is weighted with the largest available weight.  
         [0140]     Preferred values of the weights are: 
        W 1 =8; W 2 =2; W 3 =1. 
 
 while preferred values of the thresholds are: 
    Th 1 =σ; Th 2 =2*σ; Th 3 =3*σ.        
 
         [0143]     As an alternative, it is possible to select even more than three thresholds and assign them other values, but this approach makes the method computationally heavier without producing noticeable improvements in the quality of the final image.  
         [0144]     A hardware device for calculating the intensity value of a noise filtered pixel is shown in  FIG. 27 .  
         [0145]     A memory buffer tWW stores the intensities of spatially or temporally neighbor pixels of a noisy pixel P to be filtered. A computing unit (not shown in the FIG.) calculates which of the sums DC, DM or DP is the smallest one, and stores in a second memory buffer dWW the differences related to this smallest sum, which in the sample case of  FIG. 27  are the differences d i , using the values stored in the buffer tWW.  
         [0146]     A noise estimation block evaluates the standard deviation σ of the noise level corrupting the selected pixels and generates a certain number of thresholds, that in the sample embodiment of  FIG. 7  are the three thresholds Th 1 , Th 2  and Th 3 . An output computation circuit compares each of these differences with the thresholds, identifies for each difference the smallest threshold larger than the considered difference and generates the intensity of the noise filtered pixel P as a weighted sum of the intensities of the noisy pixel P and of its neighbor pixels P i .  
         [0147]     This method is preferably implemented with a software computer program performing the method steps previously described when run on a computer.  
       REFERENCES  
       [0000]    
       
          [1] B. E. Bayer, “Color Imaging Array”, U.S. Pat. No. 3,971,065.  
          [2] A. Bosco, M. Mancuso, S. Battiato, G. Spampinato, “Adaptive Temporal Filtering For CFA Video Sequences”, Proceedings of IEEE Acivs (Advanced Concepts for Intelligent Vision Systems) 2002.  
          [3] A. Bosco, M. Mancuso, S. Battiato, G. Spampinato “Temporal Noise Reduction Of Bayer Matrixed Video Data”, Proceedings of IEEE ICME (International Conference on Multimedia and Expo) 2002.  
          [4] A. Bosco, M. Mancuso, “Adaptive Filtering For Image Denoising”, in IEEE Proceedings of ICCE 2001, pp. 208-209, June 2001.  
          [5] A. M. Borneo, L. Salinari, D. Sirtori, “An Innovative Adaptive Noise Reduction Filter for Moving Pictures Based on Modified Duncan Range Test”, ST-Journal of System Research, Vol. 1, No. 1, 2000.  
          [6] G. Cortellazzo, G. A. Mian, R. Parolari, “Statistical Characteristics of Granular Camera Noise”, IEEE Transactions On Circuits and Systems For Video Technology, Vol. 4, No. 6, December 1994.  
          [7] A. Bosco, K. Findlater, S. Battiato, A. Castorina, “A Noise Reduction Filter for Full-Frame Data Imaging Devices”, Transactions on Consumer Electronics, Vol. 49, No. 3, August 2003.  
          [8] A. M. Borneo, L. Salinari, “Spatio-temporal filtering method for noise reduction during pre-processing of picture sequences in video encoders”, European Patent Application EP 1,100,260.  
          [9] A. Bosco, S. Battiato, “Method for filtering the noise of a digital image sequence”, European Patent Application EP 1,394,742.