Abstract:
In accordance with the teachings described herein, systems and methods are provided for enhancing the image quality of video. One or more filters may be used to filter a received video signal to generate a vertical filtered output, a horizontal filtered output and a diagonal filtered output. One or more amplification components may be used to apply separate gains to each of the vertical, horizontal and diagonal filtered outputs to generate a vertical enhancement signal, a horizontal enhancement signal and a diagonal enhancement signal. One or more summation components may be used to combine the vertical, horizontal and diagonal enhancement signals with the video signal to generate an enhanced video signal.

Description:
FIELD 
     The technology described in this patent document relates generally to the fields of image processing, video and graphics. More particularly, the patent document describes video enhancement systems and methods. 
     BACKGROUND AND SUMMARY 
     The image quality of many video sources is low and not subjectively sharp. This can be the result of various factors, such as poor initial recording, video compression low-pass filtering, and/or other post processing. In accordance with the teachings described herein, systems and methods are provided for enhancing the image quality of video. One or more filters may be used to filter a received video signal to generate a vertical filtered output, a horizontal filtered output and a diagonal filtered output. One or more amplification components may be used to apply separate gains to each of the vertical, horizontal and diagonal filtered outputs to generate a vertical enhancement signal, a horizontal enhancement signal and a diagonal enhancement signal. One or more summation components may be used to combine the vertical, horizontal and diagonal enhancement signals with the video signal to generate an enhanced video signal. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of an example video enhancement system. 
         FIG. 2  is a block diagram of an example video enhancement system having red, green and blue channels. 
         FIG. 3  is a block diagram of an example video enhancement system having an enhancement gain control element. 
         FIG. 4  is a block diagram of an example enhancement gain control element. 
         FIG. 5  is a block diagram of an example video enhancement system having an overshoot control element. 
         FIG. 6  is a block diagram of an example video enhancement system with optional RGB or grayscale (Y) processing. 
         FIG. 7  is a block diagram of another example video enhancement system. 
         FIG. 8  is a block diagram of an example high pass filter and noise thresholding component. 
         FIG. 9  is a block diagram of an example enhancement gain control component. 
     
    
    
     DETAILED DESCRIPTION 
       FIG. 1  is a block diagram of an example video enhancement system  10 . The system  10  includes one or more filters  12 , one or more amplification components  14 , and one or more summation components  16 . The filters  12  receive a video signal  18  and filter the video signal to generate a vertical filtered output (HPF_V), a horizontal filtered output (HPF_H) and a diagonal filtered output (HPF_D). The filters  12  may, for example, include a bank of directional high-pass filters. The amplification components  14  apply separate gains (kV, kH and kD) to each of the vertical, horizontal and diagonal filtered outputs to generate a vertical enhancement signal, a horizontal enhancement signal and a diagonal enhancement signal, respectively. The vertical, horizontal and diagonal gains (kV, kH and kD) may be determined based on one or more user inputs, and control the amount of amplification/enhancement that is applied to the filtered outputs (HPF_V, HPF_H and HPF_D). The vertical, horizontal and diagonal enhancement signals are then combined with the video signal  18  by one or more summation components  16  to generate an enhanced video signal  20 . 
       FIG. 2  is a block diagram of an example video enhancement system  30  having red (R), green (G) and blue (B) channels  32 ,  34 ,  36  that respectively receive red (R)  38 , green (G)  40  and blue (B)  42  components of a video signal. Each channel (R, G and B)  32 ,  34 ,  36  filters the received component signal (R, G or B) to generate vertical, horizontal and diagonal filtered outputs (HPF_V, HPF_H and HPF_D), and applies separate gains (kV, kH and kD) to each of the vertical, horizontal and diagonal filtered outputs. As illustrated, all three channels (R, G and B) may receive the same vertical, horizontal and diagonal gain control signals (kV, kH and kD), which are generated based on one or more user inputs. The enhanced vertical, horizontal and diagonal signals for each channel are combined with the respective R, G and B component signals  38 ,  40 ,  42  to generate an enhanced R channel video signal (DE R )  44 , an enhanced G channel video signal (DE G )  46  and an enhanced B channel video signal (DE B )  48 . 
       FIG. 3  is a block diagram of an example video enhancement system  50  having an enhancement gain control element  52 . The enhancement gain control element  52  generates the gain control signals (kV, kH and kD) as a function of the R, G and B video components  54  and edge enhancement and texture enhancement level inputs  56 ,  58 . The enhancement gain control element  52  performs a contrast calculation to classify portions (e.g., pixels) of the video image  54  as either edge portions or texture portions. For example, a pixel having a high contrast may be classified as an edge pixel, and a pixel having a low contrast may be classified as a texture pixel. The edge enhancement level  56  and texture enhancement level  58  inputs enable a user to specify the amount of gain applied to edge portions of the video image  54  independent of the amount of gain applied to the texture portions of the video image  54 . 
     Only the R channel  60  is shown in  FIG. 3  for simplicity. It should be understood, however, that the gain control signals (kV, kH and kD) generated by the enhancement gain control element  52  may also be input to G and B channels, for example as illustrated in  FIG. 2 . 
       FIG. 4  is a block diagram of an example enhancement gain control (EGC) element  70 . The EGC element  70  includes an RGB to grayscale (Y) converter  72 , a mean absolute measure (MAM) calculation element  74 , an edge look-up table (LUT)  76  and a texture look-up table (LUT)  78 . Also included are two amplification components  76 ,  78  and a summation component  84 . The RGB to Y converter  72  converts an RGB video input  86  into a simplified grayscale signal, which is input to the MAM calculation element  74 . The MAM calculation element  74  calculates a contrast measure for each pixel of the grayscale signal, and the edge and texture LUTs  76 ,  78  are used to classify the pixels as either edge pixels or texture pixels based on the MAM calculation. For example, the edge LUT  76  may specify that pixels having MAM measures above a predetermined value are edge pixels, and the texture LUT  78  may specify that pixels having MAM measures below a predetermined value are texture pixels. The parameters used by the LUTs  76 ,  78  may, for example, be determined through experimentation and stored in a memory device, may be programmable by a user, or may be both stored in memory and programmable. 
     The edge enhancement amplification component  80  receives three edge enhancement gain settings: a vertical setting, a horizontal setting and a diagonal setting. The edge enhancement gain settings are applied to the edge pixels to generate vertical, horizontal and vertical edge enhanced signals  88 . Similarly, the texture enhancement amplification component  82  receives vertical, horizontal and diagonal texture enhancement gain settings, which are applied to the texture pixels to generate vertical, horizontal and vertical texture enhanced signals  90 . The edge and texture enhanced signals  88 ,  90  are combined with the summation component  84  to generate the vertical, horizontal and diagonal gain control signals (kV, kH and kD). 
       FIG. 5  is a block diagram of an example video enhancement system  100  having an overshoot control element  102 . Only the R channel is shown in  FIG. 5  for simplicity. In this example, the enhanced video signal  104  is input to the overshoot control element  102 , which controls the amount of allowable overshoot or undershoot based on edge, texture and edge/texture parameters  106 ,  108 ,  110 . The overshoot control operates to filter any portions of the enhanced video signal that are in excess of threshold values that are calculated based on the edge, texture and/or edge/texture parameter settings  106 ,  108 ,  110 . As an example,  FIG. 5  includes signal traces  112 - 116  at five points in the system to demonstrate how the overshoot control element  102  may operate. 
       FIG. 6  is a block diagram of an example video enhancement system  130  with optional RGB or Y processing. Only the R channel is shown in  FIG. 6  for simplicity. In this example, the channel processor  132  receives both a grayscale (Y) signal  134  and an R, G or B component signal  136  as inputs. The Y signal may, for example, be provided by an enhancement gain control element, as shown in  FIG. 4 . The option to select the grayscale image (Y) for processing may be desired because of concerns that detail enhancement on RGB may cause color shifts on certain types of images. 
     In operation, one of the Y signal  134  or the R, G or B component signal  136  is selected for processing using a multiplexer  138  that is controlled by a user input  140 . The selected Y or RGB signal is filtered with one or more high pass filters  142  to generate vertical, horizontal and diagonal filtered outputs (HPF_V, HPF_H and HPF_D), and separate gains (kV, kH and kD) are applied to each of the vertical, horizontal and diagonal filtered outputs by one or more amplification elements  144 . The enhanced vertical, horizontal and diagonal signals are combined with the R, G or B component signal  136  by one or more summation components  146  to generate an enhanced channel video signal (DE R )  148 . 
       FIG. 7  is a block diagram of another example video enhancement system. The system includes three RGB channel processors  210  and an enhancement gain control element  220 . Only a single RGB channel processor  210  is shown in  FIG. 7  for simplicity. However, the enhancement gain control element  220  may also be coupled to two additional RGB channel processors. 
     The three channel processors  210  each receive one of an R, G or B video component  222 , which is input to a high pass filter and noise thresholding element  224 . The high pass filter and noise thresholding element  224  also receives a grayscale (Y) image  226  from the enhancement gain control element  220  and noise threshold  228  and RGB or Y selection  230  inputs. The RGB or Y selection input  230  selects one of the video component input  222  or the grayscale (Y) image  226  for processing. The selected image (RGB or Y) is high pass filtered and thresholded to reject noise to generate vertical, horizontal and diagonal filtered outputs (HPF_V, HPF_H and HPF_D). A more detailed diagram of an example high pass filter and noise thresholding element  224  is described below with reference to  FIG. 8 . 
     Separate gains are applied to each of the vertical, horizontal and diagonal filtered outputs (HPF_V, HPF_H and HPF_D) by one or more amplification components  232  based on gain control signals (kV, kH, kD) received from the enhancement gain control element  220 . The enhancement gain control element  220  generates the gain control signals (kV, kH, kD) as a function of vertical, horizontal and diagonal edge and texture enhancement inputs  234 ,  236 . A more detailed diagram of an example enhancement gain control element  220  is described below with reference to  FIG. 9 . 
     The vertical, horizontal and diagonal enhancement signals are bit shifted  238  and combined by a first summation element  240 . The combined enhancement signal is clipped  242  to limit its bit width and is combined with the R, G or B component video signal  222  by a second summation element  244 . The enhanced video signal (InEnh) is input to an overshoot control element  246 , which controls the amount of allowable overshoot or undershoot based on edge, texture and edge/texture parameters to generate the enhanced component video signal (DEOut). 
     The overshoot control element  246  receives edge overshoot  248  and texture overshoot  250  user settings and a texture v. edge parameter  252 , which are used to specify how much overshoot/undershoot is allowed compared to the local maximums and local minimums. In addition, the local maximums and local minimums are calculated for each pixel in the R, G or B component signal  222 . From this data, the maximum and minimum overshoot/undershoot may be determined, for example using the following algorithm. 
     
       
         
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 if (MAM &gt;= TextureOrEdgeThres) 
               
               
                   
                 { 
               
               
                   
                  maxvalue = min(1023, max(local_window) + EdgeOSLevel) 
               
               
                   
                  minvalue = max(0, min(local_window) − EdgeOSLevel) 
               
               
                   
                 } 
               
               
                   
                 else 
               
               
                   
                 { 
               
               
                   
                  maxvalue = min(1023, max(local_window) + TextureOSLevel) 
               
               
                   
                  minvalue = max(0, min(local_window) − TextureOSLevel) 
               
               
                   
                 } 
               
               
                   
                   
               
             
          
         
       
     
     In the above algorithm, “TextureOrEdgeThres” is the texture v. edge threshold parameter  252 , “EdgeOSLevel” is the edge overshoot input  248 , “TextureOSLevel” is the texture overshoot input  250 , max(local_window) is the local maximum and min(local_window) is the local minimum. The enhanced component video signal (DEOut) may be generated by clipping the enhanced video signal (InEnh) between the “maxvalue” and “minvalue” parameters calculated using the above algorithm. 
     The texture v. edge threshold parameter  252  may, for example, have a default value of 20, but may be programmable for flexibility. The overshoot control element  246  could also be used to simulate LTI/CTI by setting the edge and texture overshoot inputs  248 ,  250  near zero. 
       FIG. 8  is a block diagram of an example high pass filter and noise thresholding component  224 . The component  224  includes a 2:1 multiplexer  260 , four 3×3 directional high pass filters (HPFs)  262 - 265 , and four noise filters  266 - 269 . The multiplexer  262  selects either a grayscale (Y) or RGB component signal for processing, as described above. The four 3×3 directional HPFs extract horizontal (0°), vertical (90°) and diagonal (45° and 135°) detail information from the selected RGB or Y signal. Example filter matrices for the horizontal (h 0 ), vertical (h 90 ) and vertical (h 45  and  h135 ) filters are set forth below. 
     
       
         
           
             
               
                 h 
                 0 
               
               = 
               
                 
                   
                     
                       [ 
                       
                         
                           
                             0 
                           
                           
                             
                               - 
                               1 
                             
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             2 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             
                               - 
                               1 
                             
                           
                           
                             0 
                           
                         
                       
                       ] 
                     
                     / 
                     2 
                   
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   
                     h 
                     90 
                   
                 
                 = 
                 
                   
                     [ 
                     
                       
                         
                           0 
                         
                         
                           0 
                         
                         
                           0 
                         
                       
                       
                         
                           
                             - 
                             1 
                           
                         
                         
                           2 
                         
                         
                           
                             - 
                             1 
                           
                         
                       
                       
                         
                           0 
                         
                         
                           0 
                         
                         
                           0 
                         
                       
                     
                     ] 
                   
                   / 
                   2 
                 
               
             
             ⁢ 
             
                 
             
           
         
       
       
         
           
             
               
                 h 
                 45 
               
               = 
               
                 
                   
                     
                       [ 
                       
                         
                           
                             
                               - 
                               1 
                             
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             2 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               - 
                               1 
                             
                           
                         
                       
                       ] 
                     
                     / 
                     2 
                   
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   
                     h 
                     135 
                   
                 
                 = 
                 
                   
                     [ 
                     
                       
                         
                           0 
                         
                         
                           0 
                         
                         
                           
                             - 
                             1 
                           
                         
                       
                       
                         
                           0 
                         
                         
                           2 
                         
                         
                           0 
                         
                       
                       
                         
                           
                             - 
                             1 
                           
                         
                         
                           0 
                         
                         
                           0 
                         
                       
                     
                     ] 
                   
                   / 
                   2 
                 
               
             
             ⁢ 
             
                 
             
           
         
       
     
     All of the HPF coefficients in the above matrices are a power of 2 so that implementation of the multiplication may be performed with simple bit shifts. In addition, all of the filters are designed to be high-pass in one direction and all-pass in the orthogonal direction. It should be understood, however, that other filter designs could also be used. 
     The HPF outputs are filtered by the noise thresholding components  266 - 269  to reject noise. The noise rejection threshold operation sets to zero any HPF output having a magnitude that is less than a user-defined noise threshold  270 . If the HPF output is above the noise threshold  270 , then the HPF magnitude is left unchanged. 
     After thresholding, the directional HPFs are divided by 4 (e.g., shifted by 2) to generate the horizontal, vertical and diagonal filtered outputs (HPFOutH, HPFOutV and HPFOutD). In addition, the outputs are clipped to between 31 and −31 to reduce the implementation size of the downstream components. 
       FIG. 9  is a block diagram of an example enhancement gain control component  220 . The enhancement gain control component  220  generates the gains (kV, kH, kD) that are used to perform the amplification/enhancement of the HPF outputs. All three R, G and B channels use the same k values for enhancement, and thus the enhancement gain control component  220  need only be implemented once for all three RGB channels. The image enhancement function performed by the enhancement gain control component  220  is an adaptive scheme that depends on six user enhancement gain settings (3 edge enhancement settings and 3 texture enhancement settings) and a mean absolute measure (MAM) calculation. 
     In operation, the enhancement gain control component  220  receives the three RGB component video signals and converts the component images to a grayscale (Y) image using an RGB to Y converter  280 . The Y image may, for example, be calculated as follows. 
     
       
         
           
             Y 
             = 
             
               
                 
                   5 
                   ⁢ 
                   G 
                 
                 + 
                 B 
                 + 
                 
                   2 
                   ⁢ 
                   R 
                 
               
               8 
             
           
         
       
     
     The grayscale image (Y) is bit shifted and input to a mean absolute measure (MAM) component  282 , which performs the following calculation. 
     
       
         
           
             MAM 
             = 
             
               
                 ∑ 
                 
                   i 
                   , 
                   
                     j 
                     ∈ 
                     
                       [ 
                       
                         1 
                         , 
                         3 
                       
                       ] 
                     
                   
                 
               
               ⁢ 
               
                  
                 
                   
                     Y 
                     ⁡ 
                     
                       ( 
                       
                         i 
                         , 
                         j 
                       
                       ) 
                     
                   
                   - 
                   
                     Y 
                     _ 
                   
                 
                  
               
             
           
         
       
     
     In the above equation, “  Y ” is the weighted mean of the current Y input window. A weighted mean is used as opposed to a pure mean to reduce implementation complexity. The weighted mean,  Y , may be calculated using the weight mask: 
     
       
         
           
             W 
             = 
             
               
                 [ 
                 
                   
                     
                       1 
                     
                     
                       2 
                     
                     
                       1 
                     
                   
                   
                     
                       2 
                     
                     
                       4 
                     
                     
                       2 
                     
                   
                   
                     
                       1 
                     
                     
                       2 
                     
                     
                       1 
                     
                   
                 
                 ] 
               
               / 
               16. 
             
           
         
       
     
     The calculated MAM value is bit shifted and clipped to limit bit widths and is input to an edge look-up table (LUT)  284  and a texture look-up table (LUT)  286 , which distinguish edge pixels from texture pixels. The edge and texture LUTs  284 ,  286  may, for example, be programmed by a host CPU, and may include values that are derived through experimentation to achieve an acceptable level of detail enhancement. Example default values for the edge and texture LUTs  284 ,  286  are as follows. 
     
       
         
               
             
               
               
               
             
           
               
                   
               
               
                 Edge Enhancement LUT 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                   
                 &lt;5 
                 0 
               
               
                   
                 &lt;10 
                 1 
               
               
                   
                 &lt;20 
                 1 
               
               
                   
                 &lt;35 
                 8 
               
               
                   
                 &lt;50 
                 11 
               
               
                   
                 &lt;65 
                 10 
               
               
                   
                 &lt;85 
                 9 
               
               
                   
                 &lt;95 
                 6 
               
               
                   
                 &gt;=95 
                 2 
               
               
                   
                   
               
             
          
         
       
     
     
       
         
               
             
               
               
               
             
           
               
                   
               
               
                 Texture Enhancement LUT 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                   
                 &lt;5 
                 1 
               
               
                   
                 &lt;10 
                 4 
               
               
                   
                 &lt;20 
                 4 
               
               
                   
                 &lt;35 
                 0 
               
               
                   
                 &lt;50 
                 0 
               
               
                   
                 &lt;65 
                 0 
               
               
                   
                 &lt;85 
                 0 
               
               
                   
                 &lt;95 
                 0 
               
               
                   
                 &gt;=95 
                 0 
               
               
                   
                   
               
             
          
         
       
     
     The outputs of the LUTs  284 ,  286  are multiplied by the respective edge and texture enhancement gain settings by multiplication components  288 ,  290 . The output of the multiplication components  288 ,  290  is bit shifted and combined by one or more summation components  292  to generate the enhancement gain values (kV, kH, kD). 
     This written description uses examples to disclose the invention, including the best mode, and also to enable a person skilled in the art to make and use the invention. The patentable scope of the invention may include other examples that occur to those skilled in the art.