Abstract:
The disclosure relates to a device and a method for estimating the noise level of a video signal representing image sequences. The device comprises a movement compensated interpolation means, a recursive filter designed to receive on a first input the output of the recursive filter movement compensated by the movement compensated interpolation means and on a second input the video signal, means of calculating the difference between the video signal and the output of the recursive filter movement compensated by the movement compensated interpolation means. In accordance with the principles of the invention, the device comprises means of estimating the noise level of the video signal according to the difference and a maximum threshold delimiting this difference.

Description:
[0001]     The present invention relates to a device and a method for estimating the noise level of a video signal, and a noise reduction system and an encoding system including such a device.  
       BACKGROUND OF THE INVENTION  
       [0002]     Noise reduction is an important factor in pre-encoding video preprocessing. It is based on recursive time domain filtering, for which the recursion rate is adjusted according to the noise level present in the sequence of images to be processed.  
         [0003]     Estimation of the noise level is therefore important for obtaining a filtering system which adapts automatically to the sequence of incoming images.  
         [0004]     In known methods, in the total absence of movement and because of its time-random nature, the noise can be detected by establishing the pixel-by-pixel difference between two consecutive images. The average of the differences in an image then constitutes a representation of the noise level.  
         [0005]     When there is movement, the noise level can be estimated by calculating the difference between the current image and the preceding image reconstructed to match the current image using the vector fields supplied by the movement estimator.  
         [0006]     However, such methods present numerous drawbacks: 
        the movement vector can produce estimation errors, provoking a poor reconstruction of the image and therefore an overestimation of the noise level, since there is movement in the image sequence,     the movement vector and therefore the differences between images (Displaced Frame Difference DFD), are inconsistent all around the perimeter of the image, which also falsifies the estimation of the noise level, 
            the estimation of the noise level is based on the random nature of the noise, which is not always reliable.    
               
 
       BRIEF SUMMARY OF THE INVENTION  
       [0010]     The present invention relates to a robust noise estimating device using the pixel-by-pixel difference between the source image and the filtered image, delayed by one image and movement compensated in order to obtain a good estimation of the noise, by considerably reducing the negative effects explained above.  
         [0011]     To this end, the invention proposes a device for estimating the noise level of a video signal representing image sequences comprising 
        movement-compensated interpolation means,     a recursive filter designed to receive on a first input the output of the recursive filter movement compensated by the movement compensated interpolation means and on a second input the video signal,     means of calculating the difference between the video signal and the output of the recursive filter movement compensated by the interpolation means.        
 
         [0015]     According to the invention, the device comprises means of estimating the noise level of the video signal according to said difference and a maximum threshold delimiting this difference.  
         [0016]     This device can make the estimation of the noise level in a video sequence robust and improve the performance of a noise reducer to which it would be linked.  
         [0017]     Preferably, the means of estimating the noise level comprise means of comparing, for each pixel of an image in the video signal, the difference between said pixel value and said pixel value after movement compensated recursive filtering, at said threshold (Sc).  
         [0018]     This can allow for a better estimation of noise level for each pixel and so avoid an overall estimation which would generate blurring effects on the frame.  
         [0019]     Advantageously, the device comprises 
        means of defining for each image a working window composed of a number of pixels centred on the image,     means of accumulating, for each pixel in the working window for which the difference between its value and its value after movement compensated recursive filtering is less than said threshold, said difference,     means of calculating an average noise level (σ avg ) by dividing the differences accumulated previously by the number of accumulated differences.        
 
         [0023]     Such a working window allows for a better estimation of the noise level by achieving independence from potential movement vector computation errors around the edges of the image.  
         [0024]     According to a preferred embodiment, the device comprises: 
        means of measuring the inter-image and intra-frame correlations of each image,     means of calculating an overall noise level for each frame according to the average noise level and the inter-image and intra-frame correlations,     means of calculating a local noise level for each pixel of said frame according to the overall noise level of said frame and the luminance of said pixel.        
 
         [0028]     Advantageously, said threshold is linked to filtering parameters of said recursive filter.  
         [0029]     The invention also relates to a method for estimating the noise level of a video signal representing image sequences comprising the following steps: 
        movement compensated interpolation,     recursive filtering designed to receive on a first input of a recursive filter the output of said recursive filter movement compensated by the interpolation means and on a second input of the recursive filter the video signal,     calculation of the difference between the video signal and the output of the recursive filter movement compensated by the interpolation means.        
 
         [0033]     According to the invention, the method comprises a step for estimating the noise level of the video signal according to said difference and a maximum threshold delimiting this difference. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0034]     The invention will be better understood and illustrated using examples of advantageous embodiments and implementations, by no means limiting, with reference to the figures appended, in which:  
         [0035]      FIG. 1  represents a noise reduction device coupled to a noise estimation device according to an embodiment of the invention,  
         [0036]      FIG. 2  represents an embodiment of a recursive filter such as can be used in the noise reduction device. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0037]     A noise estimation device as shown in  FIG. 1  can be used to obtain a robust estimation of the noise on a video sequence  10  e(x,y,t). A good estimation of the noise can be used to improve considerably the performance of a noise reducer. To this end, the noise reducer comprises a recursive filter  1 . An example of an efficient recursive filter that can be used is illustrated in  FIG. 2 .  
         [0038]     To obtain efficient operation, the noise reduction methods involving a recursive filter consider the very strong time correlation of the images of a video sequence. Consequently, the concepts of movement and displacement are important for perfecting an effective noise reduction. This is why the recursive filter  1  is coupled to movement compensated interpolation means  2 .  
         [0039]     The device  3  for estimating the noise level of the incoming video sequence  10  supplies the estimated noise level  11  designated σ to the recursive filter  1 .  
         [0040]     The noise estimation device  3  comprises comparison means  4  receiving as input a threshold Sc which can be linked to the filtering severity of the recursive filter  1 .  
         [0041]     For example, if it is assumed that there is a five-position filtering adjustment cursor (cursor_position), Sc can be assigned the following value: 
    Sc=16+cursor_position, 16 being an arbitrarily set value.    
 
         [0043]     The comparison means  4  also receive as input the output of a differentiator  9 . The differentiator  9  calculates the difference between the incoming video signal  10  and the movement compensated filtered output  11  r(x,y,t).  
         [0044]     The differentiator  9  calculates a pixel-by-pixel difference between the two signals.  
         [0045]     The comparison means  4  compare the difference received from the differentiator  9  with a threshold value Sc.  
         [0046]     If ε=e(x,y,t)−r(x,y,t)&lt;Sc then the comparison means  4  transmit ε to validating means  5 .  
         [0047]     Otherwise, if ε=e(x,y,t)-r(x,y,t)≧Sc then the comparison means  4  transmit the value “0” to the validating means  5 .  
         [0048]     The comparison means  4  can thus be used to eliminate the significant value errors that are unrelated to the noise to be eliminated.  
         [0049]     The validating means are used to work only on a window of the image to improve the robustness of the device. Since the video signal represents an image, the movement vectors often present imperfections on the pixels located at the edge of the image. It is therefore advantageous to work on a window of the image located in the middle of the image so that the pixels located around the edges can be disregarded.  
         [0050]     An accumulation module  6  accumulates all the differences retained by the validating module  5 .  
         [0051]     A module  7  calculates an average noise level σ avg  estimated firstly by dividing the sum of the differences accumulated by the module  6  by the number of accumulated differences. The result gives Avgdif:  
       Avgdif   =       ∑   accumulateddifferences     numberofaccumulateddifferences         
 
         [0052]     Then, this first noise level obtained is adapted to the target application and is made dependent on the filtering severity.  
         σ   avg     =         avgdif   2     ×   cursor_position       avgdif   +   cursor_position           
 
         [0053]     It can then be seen that the more the cursor value increases, the more the estimated noise level increases and the more severe the recursive filter becomes.  
         [0054]     Advantageously, in this embodiment σ avg  is limited between 0.5 and 15.5.  
         [0055]     The value of σ avg  changes according to the frames, but the variation from one frame to the next is limited to +0.5 or minus  2 . It is therefore possible to reduce the value of σ avg  rapidly but it is more difficult to increase it rapidly to avoid an oscillation effect.  
         [0056]     The module  7  therefore produces at the output an estimated noise level transmitted to an overall weighting module  8   a.    
         [0057]     To correct the average noise level σ avg  produced by the module  7 , the latter is weighted by the module  8   a  which then generates an overall noise level σ glob . This is particularly advantageous for video signals with strong movement and/or high definition.  
         [0058]     The module  8   a  provides an overall weighting at image level.  
         [0059]     The inter-image correlation (corrinter) is equal to the sum of all the pixel-by-pixel differences between the current frame T and the preceding frame T-2, divided by the number of pixels in the frame.  
         [0060]     The intra-frame correlation (corrintra) is equal to the sum of all the pixel-by-pixel differences between the current line of the frame and the preceding line of the frame divided by the number of pixels in the frame.  
         [0061]     Therefore σ avg  is weighted according to the inter-image and intra-image correlations measured on the image sequence currently being processed by the noise reduction device  1  according to the following formula:  
         σ   glob     =       σ   avg     K         
 
         [0062]     The coefficient K can be calculated, by way of illustration, in the following way:  
       corr   =         corr   intra     +     corr   inter       2         
 
         [0063]     If the value of Corr is located between two thresholds S 1  and S 2 , then K varies linearly between the values “1” and “2”.  
         [0064]     The thresholds S 1  and S 2  are chosen relative to two typical video sequences:  
         [0065]     Firstly, a simple video sequence without significant movement and with low definition: the value of “Corr” calculated on this sequence gives the value of S 1 .  
         [0066]     Secondly, a complex video sequence with strong movement and high definition: the value of “Corr” calculated on this sequence gives the value of S 2 .  
         [0067]     Consider the following illustration: 
    if Corr&lt;S 1 , then K=1;     if Corr&gt;S 2 , then K=2    
 
         [0070]     Which gives: 
        for a simple sequence: σ glob =σ avg .     for a complex sequence σ glob =σ avg/2 .     for a video sequence located between these two extremes, σ glob =σ glob  weighted by the value of “corr”.        
 
         [0074]     The local weighting module  8   b  receives as input σ glob  and a luminance threshold, luminance_threshold, to establish a local weighting for each pixel. In practice, by exploiting a property of the human visual system concerning the low sensitivity of the eye to degradations in low luminance areas, it is advantageous to establish a weighting of the noise level locally at the level of each pixel.  
         [0075]     If the luminance value of the current pixel is less than a threshold, luminance_threshold, the estimated noise level associated with this pixel is raised to a higher value.  
         [0076]     By way of illustration, the noise level estimated on the frame σ glob  can, for example, be weighted locally for each pixel by 1.25 to obtain the local noise level a of said pixel.  
         [0077]      FIG. 2  represents an exemplary recursive filter which can be used in the noise reduction device as described in  FIG. 1 .  
         [0078]     A noise reducing filter  1  as represented in  FIG. 2  can be used to correct the imperfections linked to the movement compensation in a movement compensated recursive filtering device.  
         [0079]     Such a device can be used to separate the errors due to the movement compensation and the real noise.  
         [0080]     The device  1  comprises a recursive filtering module  13  and a multiplexer  21 .  
         [0081]     The noise contained in certain video sequences can reduce the effectiveness of a video encoder, of MPEG type for example, resulting in poor viewing quality despite a reasonable encoding rate.  
         [0082]     The recursive filter  13  preferably has a granularity of the order of the pixel.  
         [0083]     The video signal  10  is representative of an image and is composed of a certain number of pixels. Each image is encoded in interlaced mode and is composed of two frames. Each pixel of the image is filtered by the device  13 .  
         [0084]     The video signal  10  is received on an input of the subtractor  19 . The subtractor  19  receives on its second input the output of the multiplexer  21 , m(x,y,t).  
         [0085]     The recursive filter  13  receives at the input an estimation of the frame noise level  12  sigma (σ) of the frame established by a noise estimator as represented in  FIG. 1 .  
         [0086]     The noise level (σ) is used to calculate a recursion coefficient σ.  
         [0087]     The recursion coefficient σ weights the effectiveness of the filtering according to the noise level a in the frame and the error  18  calculated between the video signal e(x,y,t) at the input and the output m(x,y,t) of the multiplexer  21 , or the output of the differentiator  19 .  
       α   =         ɛ     4   ×   σ       ⁢           ⁢   and   ⁢           ⁢   α     ∈     [     0.25   ;   1     ]           
 
         [0088]     The coefficient α is transmitted to the multiplier  20 .  
         [0089]     The following is therefore obtained at the output of the multiplier  20 : 
 
α×ε=α( e ( x,y,t )− m ( x,y,t )) 
 
         [0090]     At the output of the adder  15 , f(x,y,t) is obtained, where: 
 
 f ( x,y,t )=α× e ( x,y,t )+(1−α)× m ( x,y,t ) 
 
         [0091]     The signal f(x,y,t) is then transmitted to the movement compensated interpolation means  2  after having undergone a delay of one frame f(x,y,t−1). The interpolation means  2  generate from f(x,y,t−1) and from the displacement vector (dx,dy) of the current frame, the movement compensated output r(x,y,t).  
         [0092]     The movement compensated output r(x,y,t) is then transmitted to a differentiator  22 . The differentiator  22  also receives as input the video signal e(x,y,t). The differentiator  22  produces the signal g(x,y,t) as output: 
 
 g ( x,y,t )= e ( x,y,t )− r ( x,y,t ) 
 
         [0093]     The comparator  23  represented in  FIG. 1  receives as input g(x,y,t) and a noise level threshold Se.  
         [0094]     The threshold Se depends on the final application to which the noise reduction device according to the invention can be linked. It represents a compromise between the effectiveness of the noise reducer and the appearance of artefacts in the noise reducing filter output.  
         [0095]     In a first embodiment, the threshold Se can be linked linearly to a filtering severity degree.  
         [0096]     For example, if it is assumed that there is a five-position filtering adjustment cursor (cursor-position), the following value can be assigned to Se:  
         [0097]     Se=16+cursor_position, 16 being a value set arbitrarily.  
         [0098]     In a second embodiment, Se can be adapted to the noise level σ and so vary it for each frame.  
         [0099]     According to a third embodiment, the threshold can be modified dynamically and locally and adapted to characteristic areas of the image being processed. Thus, for the movement areas, the risk of artefacts increases and it is preferable to lower the value of the threshold Se. For the low movement areas, the threshold value can be higher.  
         [0100]     The multiplexer  21  directs towards an input of the recursive filter the video signal e(x,y,t) or the movement compensated output r(x,y,t) depending on the threshold Se.  
         [0101]     The multiplexer can be used to sort between the real noise and the compensation errors.  
         [0102]     The comparator  23  generates a control signal to the multiplexer  21 .  
         [0103]     If g(x,y,t)&gt;Se, then the control signal sent to the multiplexer  21  tells the multiplexer  21  to send the video signal e(x,y,t) to the recursive filter  13 .  
         [0104]     If g(x,y,t)&lt;Se, then the control signal sent to the multiplexer  21  tells the multiplexer  21  to send the movement compensated output r(x,y,t) to the recursive filter  13 .