Abstract:
The disclosure relates to a device and to a method for noise reduction of a video signal. The device is comprised of the following components. A motion-compensated interpolation means, a recursive filter intended to receive the output of the recursive filter motion-compensated by the interpolation means at a first input, and the video signal at a second input; means for calculating the difference between the video signal and the output of the recursive filter motion-compensated by the interpolation means; wherein the device includes means for providing the first input of the recursive filter either with the video signal if the difference is greater than a predetermined noise level threshold, or the motion-compensated output of the recursive filter, if the difference is less than the said predetermined noise level threshold.

Description:
[0001]     The present invention relates to a device and to a method for noise level reduction.  
       BACKGROUND OF THE INVENTION  
       [0002]     Noise reduction is an essential part of video pre-processing prior to encoding. It is based on recursive time filtering.  
         [0003]     The noise reduction techniques are generally carried out on digital video images in the form of a matrix of samples; each sample is composed of a luminance signal and, in the case of a colour signal, a chrominance signal.  
         [0004]     Even today, the acquisition of video image sequences is still broadly carried out in an analogue form so that, once they have been acquired and optionally transmitted and stored in analogue formats, the images contain a substantial noise component. Once they have been digitized, these images are often also subjected to storage/editing operations which in turn introduce noise, this time of a digital nature. Lastly, a sequence of images undergoes a succession of transformations which lead to spatiotemporal noise of a highly random nature.  
         [0005]     In order to obtain good results, the noise reduction methods which use recursive filtering address the very high temporal correlation of the images in a video sequence. The ideas of motion and displacement are therefore important with a view to developing effective noise reduction.  
         [0006]     “Displacement” is intended to mean an object&#39;s change of position in a scene, this change of position being localized and specific to this object. “Motion” is intended to mean all the displacements of objects in a video sequence, taken together.  
         [0007]     The motion is conventionally detected either by simple image-to-image differencing, or by using a motion estimator.  
         [0008]     When a motion estimator is used, the displacements are accounted for by taking image differences at separate times, as well as by moving spatially through the frames. These displacements are represented by motion vector fields relating to pixels (motion estimation per pixel) or to blocks (motion estimation in blocs). This provides motion-compensated image differences, referred to as DFDs (Displacement Frame Differences), for pixels or for blocks.  
         [0009]     A motion estimator has imperfections, however, which may lead to output defects of the recursive filter, and which are propagated through time by the principle of recursion. Examples of these imperfections are the problems of object tracking, one object being masked by another, and the appearance of a new object.  
         [0010]     One solution envisaged in order to overcome these drawbacks is to apply the recursive filter only to fixed regions or regions with little motion. The noise reducer is then a motion-adapted noise reducer, instead of a motion-compensated noise reducer. A major drawback of such a method is that the noise is only removed from sequences without motion or with little motion, or from regions of images, but, if there is noise in a sequence, then the noise will be present throughout the sequence. Such a noise reducer would not therefore be very effective.  
       BRIEF SUMMARY OF THE INVENTION  
       [0011]     The invention relates to a device and a method for noise reduction which evaluate the noise in the image and the errors due to the motion compensation.  
         [0012]     To that end, the invention relates to a device for noise reduction of a video signal, comprising: 
        motion-compensated interpolation means,     a recursive filter intended to receive the output of the recursive filter motion-compensated by the interpolation means at a first input, and the video signal at a second input,     means for calculating the difference between the video signal and the output of the recursive filter motion-compensated by the interpolation means.        
 
         [0016]     According to the invention, the device has: 
        means for providing the first input of the recursive filter either with the video signal, if the difference is greater than a predetermined noise level threshold, or the motion-compensated output of the recursive filter, if the difference is less than the said predetermined noise level threshold.        
 
         [0018]     The invention thus makes it possible to separate the errors due to the motion compensation and the actual noise. The recursion loop is interrupted when the difference between the video signal and the motion-compensated output of the recursive filter is greater than a threshold, which makes it possible to avoid propagating the errors due to the motion compensation by breaking the recursion loop. It is therefore possible to carry out very severe filtering without producing artefacts at the output of the filter, which may occur owing to strong motion-compensation errors in the event of poor motion estimation.  
         [0019]     According to a preferred embodiment, the device has: 
        means for estimating the noise level of the video signal,     means for calculating the said predetermined noise level threshold.        
 
         [0022]     According to a preferred embodiment, the means for calculating the difference between the video signal and the output of the recursive filter motion-compensated by the interpolation means calculate the said difference for each pixel of the image carried by the video signal.  
         [0023]     Advantageously, the recursive filter has one or two filtering cells.  
         [0024]     According to a preferred embodiment, the said threshold is related to the filtering parameters of the said recursive filter.  
         [0025]     This makes it possible to manually adjust the threshold, and may therefore allow a user to adjust the threshold according to the application.  
         [0026]     According to a preferred embodiment, the said threshold depends on the noise level of each frame.  
         [0027]     In this way, with the noise level capable of varying from one frame to another, the threshold can be modified and recalculated for each frame.  
         [0028]     According to a preferred embodiment, the said threshold can be modified dynamically and locally by adapting it to characteristic regions of each image.  
         [0029]     This makes it possible to adapt to frames which have an irregular noise level and therefore to locally limit the artefact risks within a given frame, while maintaining a high level of filtering.  
         [0030]     The invention also relates to a method for noise reduction of a video signal, comprising the following steps: 
        motion-compensated interpolation,     recursive filtering intended to receive the motion-compensated output of the recursive filter at a first input and the video signal at a second input,     calculation of the difference between the video signal and the motion-compensated output of the recursive filter.        
 
         [0034]     According to the invention, the method includes: 
        a step of providing the first input of the recursive filter either with the video signal, if the difference is greater than a predetermined noise level threshold, or the motion-compensated output of the recursive filter, if the difference is less than the said predetermined noise level threshold.        
 
         [0036]     The invention also relates to a program product comprising program code instructions which are capable of carrying out the method according to the invention when the program is run on a computer.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0037]     The invention will be understood more clearly and illustrated by means of preferred exemplary embodiments, which do not imply any limitation, with reference to the appended figures in which:  
         [0038]      FIG. 1  represents a preferred embodiment of the invention, based on a recursive filter which has one filtering cell;  
         [0039]      FIG. 2  represents a preferred embodiment of the invention, based on a recursive filter which has two filtering cells. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0040]     A noise reduction device  1  as represented in  FIG. 1  can allow to correct the imperfections associated with the motion compensation in a motion-compensated recursive filtering device.  
         [0041]     Such a device makes it possible to separate the errors due to the motion compensation and the actual noise.  
         [0042]     The device  1  has a recursive filtering module  3  and a motion-compensated interpolation module  2 .  
         [0043]     The noise contained in some video sequences can reduce the effectiveness of a video encoder, for example of the MPEG type, leading to a poor visual quality of the decoded image even if there is a reasonable encoding rate.  
         [0044]     The noise reduction device  1  is based on recursive filtering carried out by the recursive filter  3  and the motion-interpolation means  2 .  
         [0045]     The recursive filter  3  preferably has a granularity of the order of one pixel.  
         [0046]     The video signal  12  represents an image made up of a certain number of pixels. Each image is encoded in an interleaved mode and is made up of two frames. Each pixel of the image is filtered by the device  3 .  
         [0047]     The video signal  12  is received at one input of the subtractor  7 . The subtractor  7  receives the output m(x,y,t) of the multiplexer  8  at its second input.  
         [0048]     The input of the recursive filter  3  receives an estimate sigma ((σ) of the noise level  13 , created for each frame by a noise level estimator (not shown in  FIG. 1 ).  
         [0049]     The noise level (σ) is used in order to calculate a recusive coefficient α.  
         [0050]     The recusive coefficient α weights the effectiveness of the filtering as a function of the noise level σ in the frame and the calculated error  14  between the input video signal e(x,y,t) and the output m(x,y,t) of the multiplexer  8 , that is to say the output of the differentiator  7 .  
       α   =         ɛ     4   ×   σ       ⁢           ⁢   and   ⁢           ⁢   α     ∈     [     0.25   ;   1     ]           
 
         [0051]     The coefficient α is sent to a multiplier  6 , which receives the output ε of the differentiator  7  at its other input.  
         [0052]     The following is therefore obtained at the output of the multiplier  6 :
 
α×ε=α( e ( x,y,t )− m ( x,y,t ))
 
         [0053]     The output of the multiplier  6  is connected to the input of an adder  10 . The other input of the adder  10  is connected to the output of the multiplexer  8 . f(x,y,t) is obtained at the output of the adder  10 :
 
 f ( x,y,t )=α× e ( x,y,t )+(1−α)× m ( x,y,t )
 
         [0054]     The signal f(x,y,t) is then sent to the motion-compensated interpolation means  2 , after having experienced a delay of one frame f(x,y,t−1). The interpolation means  2  generate the motion-compensated output r(x,y,t) from f(x,y,t−1) and the estimated displacement vector (dx,dy) between the current frame and the previous frame.  
         [0055]     The motion-compensated output r(x,y,t) is then sent to a differentiator  9 . The differentiator  9  also receives the video signal e(x,y,t) as input. The differentiator  9  produces the signal g(x,y,t) at its output.
 
 g ( x,y,t )= e ( x,y,t )− r ( x,y,t )
 
         [0056]     A comparator  11  receives g(x,y,t) and a noise level threshold Sc as input.  
         [0057]     The threshold Sc depends on the final application with which the noise reduction device according to the invention may be associated. It represents a compromise between the effectiveness of the noise reducer and the appearance of artefacts at the output of the noise reducer. In a first embodiment, the threshold Sc may be linearly related to a degree of filtering severity.  
         [0058]     If it is assumed a cursor is available for filtering adjustment with 5 positions (cursor_position), for example, then the following value may be assigned to Sc:
 
 Sc= 16+cursor_position, 16 being an arbitrarily fixed value.
 
         [0059]     Sc may be adapted to the noise level σ in a second value embodiment, so that it can be varied for each frame.  
         [0060]     According to a third embodiment, the threshold can be modified dynamically and locally by adapting it to characteristic regions of each image being processed. For instance, the risk of artefacts increases for the regions with motion, and it is preferable to lower the value of the threshold. The value of the threshold may be higher for regions with little movement.  
         [0061]     The multiplexer  8  provides an input of the recursive filter with the video signal e(x,y,t) or the motion-compensated output r(x,y,t), depending on the result of the comparison performed by the comparator  11 .  
         [0062]     The multiplexer makes it possible to carry out sorting between the actual noise and the compensation errors.  
         [0063]     The comparator  11  generates a control signal for the multiplexer  8 .  
         [0064]     If g(x,y,t)&gt;Sc, then the control signal sent to the multiplexer  8  tells the multiplexer  8  to transmit the video signal e(x,y,t) to the recursive filter  3 .  
         [0065]     If g(x,y,t)&lt;Sc, then the control signal sent to the multiplexer  8  tells the multiplexer  8  to transmit the motion-compensated output r(x,y,t) to the recursive filter  3 .  
         [0066]      FIG. 2  represents another embodiment of the invention, applied to a recursive filter comprising two cells.  
         [0067]     Like the device represented in  FIG. 1 , the device represented in  FIG. 2  makes it possible to avoid the detrimental effects of the imperfections associated with the motion compensation in a motion-compensated recursive filtering device.  
         [0068]     A recursive filter having two recursive filtering cells is more effective than a recursive filter including a single cell.  
         [0069]     The noise reduction device as represented in  FIG. 2  has two recursive filters  29  and  39 , respectively coupled to two motion-interpolation modules  28  and  38 .  
         [0070]     The recursive filters  29  and  39  preferably have a granularity of the order of one pixel.  
         [0071]     The video signal  18  represents an image made up of a certain number of pixels. In the preferred embodiment, each image is encoded in an interleaved mode and is made up of two frames, although it could also be encoded in a progressive mode. Each pixel of the incoming image is filtered by the recursive filter  29  and by the recursive filter  39 . The video signal is received at one input of a subtractor  23  and at one input of a subtractor  33 . The subtractor  23  receives the output of a multiplexer  22  at its second input, and the subtractor  33  receives the output of a multiplexer  32  at its second input.  
         [0072]     The inputs of the recursive filters  29  and  39  receive an estimate delta ( 5 ) of the noise level, created for each frame by a noise level estimator (not shown in  FIG. 2 ). The noise level δ is used in order to calculate a recusive coefficient α 1  of the recursion filter  29  and a recusive coefficient α 2  of the recursive filter  39 .  
         [0073]     The recusive coefficients α 1  and α 2  weight the effectiveness of the filtering as a function of the noise level δ in the frame and, respectively, the calculated error ε 1  or ε 2  between the input video signal e(x,y,t) and the output m 1 (x,y,t) of the multiplexer  22  or the output m 2 (x,y,t) of the multiplexer  32 , that is to say respectively the output of the differentiators  23  or  33 .  
       α1   =         ɛ1     4   ×   δ       ⁢           ⁢   and   ⁢           ⁢   α1     ∈     [     0.25   ;   1     ]           
 
         [0074]     The coefficient α 1  is sent to a multiplier  24 , which receives the output ε 1  of the differentior  23  at its other input.  
         [0075]     The following is therefore obtained at the output of the multiplier  24 :
 
α 1 ×ε 1 =α 1 ( e ( x,y,t )− m   1 ( x,y,t ))
 
         [0076]     The output of the multiplier  24  is connected to the input of an adder  26 . The other input of the adder  26  is connected to the output of the multiplexer  22 . f 1 (x,y,t) is obtained at the output of the adder  26 .
 
 f   1 ( x,y,t )=α 1 × e ( x,y,t )+(1−α 1 )× m   1 ( x,y,t )
 
  
       α2   =         ɛ2     4   ×   δ       ⁢           ⁢   and   ⁢           ⁢   α2     ∈     [     0.25   ;   1     ]           
 
         [0077]     The coefficient α 2  is sent to a multiplier  34 , which receives the output ε 2  of the differentior  33  at its other input.  
         [0078]     The following is therefore obtained at the output of the multiplier  34 :
 
α 2 ×ε 2 =α 2 ( e ( x,y,t )− m   2 ( x,y,t ))
 
         [0079]     The output of the multiplier  34  is connected to the input of an adder  36 . The other input of the adder  36  is connected to the output of the multiplexer  32 . f 2 (x,y,t) is obtained at the output of the adder  36 .
 
 f   2 ( x,y,t )=α 2 × e ( x,y,t )+(1−α 2 )× m   2 ( x,y.t )
 
         [0080]     f 1 (x,y,t) and f 2 (x,y,t) are then sent to an adder  39 , which provides the filtered signal f′(x,y,t) at its output.
 
 f′ ( x,y,t )=[ f   1 ( x,y,t )+ f   2 ( x,y,t )]/2
 
         [0081]     The signal f′(x,y,t) is then sent to a frame delay module  27  and to an image delay module  37 .  
         [0082]     The signal f′(x,y,t−1) is thus obtained at the output of the frame delay module  27 , and f′(x,y,t−2) is obtained at the output of the image delay module  37 .  
         [0083]     The interpolation means  27  generate the motion-compensated output r 1 (x,y,t) of the current frame from estimated motion vectors between the current frame and the previous frame, and from f′(x,y,t−1).  
         [0084]     The interpolation means  37  generate the motion-compensated output r 2 (x,y,t) of the current frame from estimated motion vectors between the current frame and the previous frame, and from f′(x,y,t−2).  
         [0085]     The motion-compensated output r 1 (x,y,t) is sent to a differentiator  21 . The differentiator  21  also receives the video signal e(x,y,t) as input. The differentiator  21  produces the signal g 1 (x,y,t) at its output.
 
 g   1 ( x,y,t )= e ( x,y,t )− r   1 ( x,y,t )
 
         [0086]     The comparator  20  receives g 1 (x,y,t) and a noise level threshold Sc 1  as input.  
         [0087]     A multiplexer  22  provides an input of the recursive filter with the video signal e(x,y,t) or the motion-compensated output r 1 (x,y,t), depending on the result of the comparison performed by the comparator  20 .  
         [0088]     The multiplexer  22  makes it possible to carry out sorting between the actual noise and the compensation errors  
         [0089]     A comparator  20  generates a control signal for the multiplexer  8 .  
         [0090]     If g 1 (x,y,t)&gt;Sc, then the control signal sent to the multiplexer  22  tells the multiplexer  22  to transmit the video signal e(x,y,t) to the recursive filter  29 .  
         [0091]     If g 1 (x,y,t)&lt;Sc, then the control signal sent to the multiplexer  22  tells the multiplexer  22  to transmit the motion-compensated output r 1 (x,y,t) to the recursive filter  29 .  
         [0092]     The motion-compensated output r 2 (x,y,t) is sent to a differentiator  31 . The differentiator  31  also receives the video signal e(x,y,t) as input. The differentiator  31  produces the signal g 2 (x,y,t) at its output.
 
 g   2 ( x,y,t )= e ( x,y,t )− r   2 ( x,y,t )
 
         [0093]     A comparator  30  receives g 2 (x,y,t) and a noise level threshold Sc 2  as input.  
         [0094]     The two noise level thresholds Sc 1  and Sc 2  may be independent.  
         [0095]     A multiplexer  32  provides an input of the recursive filter with the video signal e(x,y,t) or the motion-compensated output r 2 (x,y,t), depending on the result of the comparison performed by the comparator  30 .  
         [0096]     The multiplexer  32  makes it possible to carry out sorting between the actual noise and the compensation errors.  
         [0097]     The comparator  30  generates a control signal for the multiplexer  32 .  
         [0098]     If g 2 (x,y,t)&gt;Sc 2 , then the control signal sent to the multiplexer  32  tells the multiplexer  32  to transmit the video signal e(x,y,t) to the recursive filter  39 .  
         [0099]     If g 2 (x,y,t)&lt;Sc 2 , then the control signal sent to the multiplexer  32  tells the multiplexer  32  to transmit the motion-compensated output r 2 (x,y,t) to the recursive filter  39 .  
         [0100]     The thresholds Sc 1  and Sc 2  may be fixed, variable or dependent on the filtering severity, as indicated above for the threshold Sc.