Abstract:
Elements of the present invention relate to systems and methods for generating, modifying and applying backlight array driving values.

Description:
RELATED REFERENCES  
       [0001]     This application claims the benefit of U.S. Provisional Patent Application No. 60/940,378, entitled “Methods and Systems for Motion Adaptive Backlight Driving for LCD Displays with Area Adaptive Backlight,” filed on May 25, 2007; this application is also a continuation-in-part of U.S. patent application Ser. No. 10/966, 258, entitled “Adaptive Flicker and Motion Blur Control,” filed on Oct. 15, 2004; this application is also a continuation-in-part of U.S. patent application Ser. No. 11/219,888, entitled “Black Point Insertion,” filed on Sep. 6, 2005; and this application is also a continuation-in-part of U.S. patent application Ser. No. 11/157,231, entitled “Image Display Device with Reduced Flickering and Blur,” filed on June 20. All applications listed in this section are hereby incorporated herein by reference. 
     
    
     FIELD OF THE INVENTION  
       [0002]     Embodiments of the present invention comprise methods and systems for generating, modifying and applying backlight driving values for an LED backlight array.  
       BACKGROUND  
       [0003]     Some displays, such as LCD displays, have backlight arrays with individual elements that can be individually addressed and modulated. The displayed image characteristics can be improved by systematically addressing backlight array elements.  
       SUMMARY  
       [0004]     Some embodiments of the present invention comprise methods and systems for generating, modifying and applying backlight driving values for an LED backlight array.  
         [0005]     The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention taken in conjunction with the accompanying drawings.  
     
    
     BRIEF DESCRIPTION OF THE SEVERAL DRAWINGS  
       [0006]      FIG. 1  is a diagram showing a typical LCD display with an LED backlight array;  
         [0007]      FIG. 2  is a chart showing motion adaptive LED backlight driving;  
         [0008]      FIG. 3  is a graph showing an exemplary tone mapping;  
         [0009]      FIG. 4  is an image illustrating an exemplary LED point spread function;  
         [0010]      FIG. 5  is a chart showing an exemplary method for deriving LED driving values;  
         [0011]      FIG. 6  is a diagram showing an exemplary error diffusion method;  
         [0012]      FIG. 7  is a graph showing an exemplary inverse gamma correction;  
         [0013]      FIG. 8  is a diagram showing how a blank signal is fed to drivers in an LED array;  
         [0014]      FIG. 9  is a diagram showing synchronized timing for backlight flashing;  
         [0015]      FIG. 10  is a diagram showing pulse width modulated pulses in LED driving; and  
         [0016]      FIG. 11  is a graph showing an exemplary LCD inverse gamma correction.  
     
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS  
       [0017]     Embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The figures listed above are expressly incorporated as part of this detailed description.  
         [0018]     It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the methods and systems of the present invention is not intended to limit the scope of the invention but it is merely representative of the presently preferred embodiments of the invention.  
         [0019]     Elements of embodiments of the present invention may be embodied in hardware, firmware and/or software. While exemplary embodiments revealed herein may only describe one of these forms, it is to be understood that one skilled in the art would be able to effectuate these elements in any of these forms while resting within the scope of the present invention.  
         [0020]     In a high dynamic range (HDR) display, comprising an LCD using an LED backlight, an algorithm may be used to convert the input image into a low resolution LED image, for modulating the backlight LED, and a high resolution LCD image. To achieve high contrast and save power, the backlight should contain as much contrast as possible. The higher contrast backlight image combined with the high resolution LCD image can produce much higher dynamic range image than a display using prior art methods. However, one issue with a high contrast backlight is motion-induced flickering. As a moving object crosses the LED boundaries, there is an abrupt change in the backlight: In this process, some LEDs reduce their light output and some increase their output; which causes the corresponding LCD to change rapidly to compensate for this abrupt change in the backlight. Due to the timing difference between the LED driving and LCD driving, or an error in compensation, fluctuation in the display output may occur causing noticeable flickering along the moving objects. The current solution is to use infinite impulse response (IIR) filtering to smooth the temporal transition, however, this is not accurate and also may cause highlight clipping.  
         [0021]     An LCD has limited dynamic range due the extinction ratio of polarizers and imperfections in the LC material. In order to display high-dynamic-range images, a low resolution LED backlight system may be used to modulate the light that feeds into the LCD. By the combination of modulated LED backlight and LCD, a very high dynamic range (HDR) display can be achieved. For cost reasons, the LED typically has a much lower spatial resolution than the LCD. Due to the lower resolution LED, the HDR display, based on this technology, can not display high dynamic pattern of high spatial resolution. But, it can display an image with both very bright areas (&gt;2000 cd/m 2 ) and very dark areas (&lt;0.5 cd/m 2 ) simultaneously. Because the human eye has limited dynamic range in a local area, this is not a significant problem in normal use. And, with visual masking, the eye can hardly perceive the limited dynamic range of high spatial frequency content.  
         [0022]     Another problem with modulated-LED-backlight LCDs is flickering along the motion trajectory, i.e. the fluctuation of display output. This can be due to the mismatch in LCD and LED temporal response as well as errors in the LED point spread function (PSF). Some embodiments may comprise temporal low-pass filtering to reduce the flickering artifact, but this is not accurate and may also cause highlight clipping. In embodiments of the present invention, a motion adaptive LED driving algorithm may be used. A motion map may be derived from motion detection. In some embodiments, the LED driving value may also be dependent on the motion status. In a motion region, an LED driving value may be derived such that the contrast of the resulting backlight is reduced. The reduced contrast also reduces a perceived flickering effect in the motion trajectory.  
         [0023]     Some embodiments of the present invention may be described with reference to  FIG. 1 , which shows a schematic of an HDR display with an LED layer  2 , comprising individual LEDs  8  in an array, as a backlight for an LCD layer  6 . The light from the array of LEDs  2  passes through a diffusion layer  4  and illuminates the LCD layer  6 .  
         [0024]     In some embodiments, the backlight image is given by 
 
 bl ( x, y )=LED( i, j )* psf ( x, y )   (1) 
 
 where LED(i,j) is the LED output level of each individual LED in the backlight array, psf(x,y) is the point spread function of the diffusion layer and * denotes a convolution operation. The backlight image may be further modulated by the LCD. 
 
         [0025]     The displayed image is the product of the LED backlight and the transmittance of the LCD: T LCD (x,y). 
 
 img ( x, y )= bl ( x, y ) T   LCD ( x, y )=(led( i, j )* psf ( x, y )) T   LCD ( x, y )   (2) 
 
 By combining the LED and LCD, the dynamic range of the display is the product of the dynamic range of LED and LCD. For simplicity, in some embodiments, we use a normalized LCD and LED output between 0 and 1. 
 
         [0026]     Some exemplary embodiments of the present invention may be described with reference to  FIG. 2 , which shows a flowchart for an algorithm to convert an input image into a low-resolution LED backlight image and a high-resolution LCD image. The LCD resolution is m x n pixels with its range from 0 to 1, with 0 representing black and 1 representing the maximum transmittance. The LED resolution is M×N with M&lt;m and N&lt;n. We assume that the input image has the same resolution as the LCD image. If the input image is a different resolution, a scaling or cropping step may be used to convert the input image to the LCD image resolution. In some embodiments, the input image may be normalized 10 to values between 0 and 1.  
         [0027]     In these embodiments, the image may be low-pass filtered and sub-sampled  12  to an intermediate resolution. In some embodiments, the intermediate resolution will be a multiple of the LED array size (aM×aN). In an exemplary embodiment, the intermediate resolution may be 8 times the LED resolution (8M×8N). The extra resolution may be used to detect motion and to preserve the specular highlight. The maximum of the intermediate resolution image forms the Blockmax image (M×N )  14 . This Blockmax image may be formed by taking the maximum value in the intermediate resolution image (aM×sN) corresponding to each block to form an M×N image. A Blockmean image  16  may also be created by taking the mean of each block used for the Blockmax image.  
         [0028]     In some embodiments, the Blockmean image  16  may then be tone mapped  20 . In some embodiments, tone mapping may be accomplished with a 1D LUT, such as is shown in  FIG. 3 . In these embodiments, the tone mapping curve may comprise a dark offset  50  and expansion nonlinearity  52  to make the backlight at dark region slightly higher. This may serve to reduce the visibility of dark noise and compression artifacts. The maximum of the tone-mapped Blockmean image and the Blockmax image is generated  18  used as the target backlight value, LED 1 . These embodiments take into account the local maximum thereby preserving the specular highlight. LED 1  is the target backlight level and its size is the same as the number of active backlight elements (M×N).  
         [0029]     Flickering in the form of intensity fluctuation can be observed when an object moves cross LED boundaries. This object movement can cause an abrupt change in LED driving values. Theoretically, the change in backlight can be compensated by the LCD. But due to timing differences between the LED and the LCD, and mismatch in the PSF used in calculating the compensation and the actual PSF of the LED, there is typically some small intensity variation. This intensity variation might not be noticeable when the eye is not tracking the object motion, but when the eye is tracking the object motion, this small intensity change can become a periodic fluctuation. The frequency of the fluctuation is the product of video frame rate and object motion speed in terms of LED blocks per frame. If an object moves across an LED block in 8 video frames and the video frame rate is 60 Hz, the flickering frequency is 60 hz*0.125=7.5 Hz. This is about the peak of human visual sensitivity to flickering and it can result in a very annoying artifact.  
         [0030]     To reduce this motion flickering, a motion adaptive algorithm may be used to reduce the sudden LED change when an object moves across the LED grids. Motion detection may be used to divide a video image into two classes: a motion region and a still region. In the motion region, the backlight contrast is reduced so that there is no sudden change in LED driving value. In the still region, the backlight contrast is preserved to improve the contrast ratio and reduce power consumption.  
         [0031]     Motion detection may be performed on the subsampled image at aM×aN resolution. The value at a current frame may be compared to the corresponding block in the previous frame. If the difference is greater than a threshold, then the backlight block that contains this block may be classified as a motion block. In an exemplary embodiment, each backlight block contains 8×8 sub-elements. In some exemplary embodiments, the process of motion detection may be performed as follows:  
         [0032]     For each frame, 
        1. calculate the average of each sub-element in the input image for the current frame,     2. if the difference between the average in this frame and the sub-element average of the previous frame is greater than a threshold (e.g., 5% of total range, in an exemplary embodiment), then the backlight block that contains the sub-element is classified as a motion block. In this manner a first motion map may be formed.     3. Perform a morphological dilation operation or other image process technique on the first motion map (change the still blocks neighboring a motion block to motion blocks) to form a second enlarged motion map.     4. For each backlight block, the motion status map is updated based on the motion detection results: 
            if it is a motion block, 
 
 m Map t ( i, j )=min(4,  m Map t-1 ( i, j )+1); 
    else (still block) 
 
 m Map t ( i, j )=max(0,  m Map t-1 ( i, j )−1); 
   
               
 
         [0039]     The LED driving value is given by  
                 LED   2     ⁡     (     i   ,   j     )       =         (     1   -     mMap   4       )     ⁢       LED   1     ⁡     (     i   ,   j     )         +       mMap   4     ⁢       LED   max     ⁡     (     i   ,   j     )                   (   3   )             
 
 where LED max  is the local max of LEDs in a window that centers on the current LED. One example is a 3×3 window. Another example is a 5×5 window. 
 
         [0040]     In some embodiments, motion estimation may be used. In these embodiments, the window may be aligned with a motion vector. In some embodiments, the window may be one-dimensional and aligned with the direction of the motion vector. This approach reduces the window size and preserves the contrast in the non-motion direction, but the computation of a motion vector is much more complex than simple motion detection. In some embodiments, the motion vector values may be used to create the enlarged motion map. In some embodiments, the motion vector values may be normalized to a value between 0 and 1. In some embodiments, any motion vector value above 0 may be assigned a value of 1. The motion status map may then be created as described above and the LED driving values may be calculated according to equation 4, however, LEDmax would be determined with a ID window aligned with the motion vector.  
         [0041]     Since the PSF of the LED is larger than the LED spacing to provide a more uniform backlight image, there is considerable crosstalk between the LED elements that are located close together.  FIG. 4  shows a typical LED PSF where the black lines  55  within the central circle of illumination indicate the borders between LED array elements. From  FIG. 4 , it is apparent that the PSF extends beyond the border of the LED element.  
         [0042]     Because of the PSF of the LEDs, any LED has contribution from each of its neighboring LEDs. Although Equation 2 can be used to calculate the backlight, given an LED driving signal, deriving the LED driving signal to achieve a target backlight image is an inverse problem. This is an ill-posed de-convolution problem. In one approach, a convolution kernel is used to derive the LED driving signal as shown in Equation 3. The crosstalk correction kernel coefficients (c 1  and c 2 ) are negative to compensate for the crosstalk from neighboring LEDs.  
             crosstalk   =                c   2           c   1           c   2               c   1           c   0           c   1               c   2           c   1           c   2                        (   4   )             
 
         [0043]     The crosstalk correction matrix does reduce the crosstalk effect from its immediate neighbors, but the resulting backlight image is still inaccurate with a too-low contrast. Another problem is that it produces many out of range driving values that have to be truncated and can result in more errors.  
         [0044]     Since the LCD output can not be more than 1, the LED driving value must be derived so that backlight is larger than target luminance, e.g., 
 
led( i, j ):{led( i, j )* psf ( x, y )≧1( x, y )}  (5) 
 
 In Equation 5,“:” is used to denote the constraint to achieve the desired LED values of the function in the curly bracket. Because of the limited contrast ratio (CR), due to leakage, LCD(x,y) can no longer reach 0. The solution is that when a target value is smaller than LCD leakage, the led value may be reduced to reproduce the dark luminance. 
 
led( i, j ):{led( i, j )           psf ( x, y )&lt;1( x, y )· CR}   (6) 
 
         [0045]     In some embodiments, another goal may be a reduction in power consumption so that the total LED output is reduced or minimized.  
               led   ⁡     (     i   ,   j     )       ⁢     :     ⁢     {     min   ⁢       ∑     i   ,   j       ⁢     led   ⁡     (     i   ,   j     )           }             (   7   )             
 
         [0046]     Flickering may be due to the non-stationary response of the LED combined with the mismatch between the LCD and LED. The mismatch can be either spatial or temporal. Flickering can be reduced or minimized by reducing the total led output fluctuation between frames.  
               led   (     i   ,   j     )     ⁢     :     ⁢     {     min   (       ∑     i   ,   j       ⁢     [         led   t     ⁡     (     i   ,   j     )       -       led     t   -   1       ⁡     (       i   -       v   x     ⁢   t       ,     j   -       v   t     ⁢   t         )         ]       )     }             (   8   )             
 
 where v x  and v y  are the motion speed in term of LED blocks. Combining Equations 5 and 8 yields Equation 9 below.  
               led   ⁡     (     i   ,   j     )       ⁢     :     ⁢     {               led   ⁡     (     i   ,   j     )       *     psf   ⁡     (     x   ,   y     )         ≥     I   ⁡     (     x   ,   y     )                       led   ⁡     (     i   ,   j     )       *     psf   ⁡     (     x   ,   y     )         &lt;       I   ⁡     (     x   ,   y     )       ·   CR                 min   ⁢       ∑     i   ,   j       ⁢     led   ⁡     (     i   ,   j     )                     min   ⁡     (       ∑     i   ,   j       ⁢     [         led   t     ⁡     (     i   ,   j     )       -       led     t   -   1       ⁡     (       i   -       v   x     ⁢   t       ,     j   -       v   t     ⁢   t         )         ]       )             }             (   9   )             
 
         [0047]     In some embodiments, the algorithm to derive the backlight values that satisfy Eq. 8 comprises the following steps: 
        1. A single pass routine to derive the LED driving values with a constraint that led &gt;0.     2. Post-processing: for those LED with driving value more than 1 (maximum), threshold to 1 and then using anisotropic error diffusion to distribute the error to its neighboring LEDs        
 
         [0050]     Finding an LED driving value from a target value is an ill-posed problem that requires an iterative algorithm, which is difficult to implement in hardware. The method, of some embodiments of the present invention, can be implemented as a single pass method. These embodiments may be described with reference to  FIG. 5 . In these embodiments, LED driving values are determined for a new frame  60 . These values may be determined using  62  the difference between the target backlight (BL) and previous backlight (BL i-1 ). This difference may be scaled by a scale factor that may, in some embodiments, range from 0.5 to 2 times the inverse of the sum of the PSF. Previous backlight values may be extracted from a buffer  64 . The new driving value (LED i ) is the sum of the previous LED driving value (Led i-1 ) and the scaled difference. The new backlight may be estimated  66  by the convolution of the new LED driving value and the PSF  68  of the LED.  
         [0051]     In some embodiments, the derived LED value  67  from the single pass algorithm can be less than 0 and greater than 1. Since the LED can only be driven between 0 (minimum) and 1 (maximum), these values may be truncated to 0 or 1. Truncation to 0 still satisfies Eq. 4, but truncation to 1 does not. This truncation causes a shortfall in backlight illumination. In some embodiments, this shortfall may be compensated by increasing the driving value of neighboring LEDs. In some embodiments, this may be performed by error diffusion methods. An exemplary error diffusion method is illustrated in  FIG. 6 .  
         [0052]     In some embodiments, a post processing algorithm may be used to diffuse this error as follows: 
        1. For these led i,j &gt;1     2. tmpVal=led i,j −1;     3. set led i,j =1;     4. Sort the 4 neighboring LEDs to ascending order     5. If(max−min&lt;min(diffThd, tmpVal/2) 
            All the neighbor LEDs are increased by tmpVal/2 else     They are increased by errWeight*tmpVal*2. 
 
 where ErrWeight is the array for error diffusion coefficients based on the rank order. In an exemplary embodiment, errWeight=[0.75 0.5 0.5 0.25], where the largest coefficient is for the neighboring LED with the lowest driving value, and the smallest coefficient is for the neighboring LED with the highest driving value. 
   
               
 
         [0060]     In some situations, the LED output may be non-linear with respect to the driving value, and, if the driving value is an integer, inverse gamma correction and quantization may be performed to determine the LED driving value.  FIG. 7  illustrates an exemplary process of inverse gamma correction for LED values wherein normalized LED output values  70  are converted, via a tonescale curve  72 , to driving values  74 .  
         [0061]     LED driving is commonly done with pulse width modulation (PWM), where the LED driving current is fixed and its duration or “on” time determines the light output. This pulse width driving at a 60 Hz frame rate can cause flickering. Therefore, two PWM pulses are typically used in prior art methods. This doubles the backlight refresh rate so that flickering is reduced or eliminated. However, the use of two PWM pulses may cause motion blur at higher duty-cycles or ghosting (double edges) at lower duty-cycles. To reduce both flickering and motion blur, motion adaptive LED driving may be used.  FIG. 8  illustrates an arrangement for LED drivers  80  and LED backlight elements  82  in a display  84 .  
         [0062]     To compensate for the time difference between LCD driving from top to bottom, a BLANK signal is used to synchronize PWM driving with the LCD driving. These embodiments may be further illustrated with reference to  FIG. 9 . In these embodiments, the BLANK signal shifts to the right according to the vertical position. There are two “on” pulses  92  and  93  in the BLANK signal to trigger the two PWM pulses. VBR n    94  and VBR n+1    95  are two vertical blanking retracing (VBR) signals, which define an LCD frame time  96 . For each LCD frame, there are two LED PWM pulses  92  and  93 . The time between the two PWM pulses (T offset2 −T offset1 )  91  is exactly half of the LCD frame time  96 . T offset1    90  and T offset2    91  are adjusted based on the BLANK signal to synchronize with the LCD driving. For shorter duty cycles (i.e., duty cycle less than 100%). T offset1    90  and T offset2    91  should be shifted to the right so that PWM “on” occurs at the flat part of the LCD temporal response curve.  
         [0063]     The use of two PWM pulses in one LCD enables motion adaptive backlight flashing. If there is no detected motion, the two PWM pulses may have the same width, but may be offset in time by half of an LCD frame time. If the LCD frame rate is 60 Hz, the perceived image is actually 120 Hz, thereby eliminating the perception of flickering. If motion is detected, PWM pulse  1   92  may be reduced or eliminated, while the width of PWM pulse  2   93  is increased to maintain the overall brightness. Elimination of PWM pulse  1   92  may significantly reduce the temporal aperture thereby reducing motion blur.  
         [0064]      FIG. 10  shows the PWM pulses in LED driving. Assume the LED intensity is I {0,1} and duty cycle is λ {0,100%}, the PWM “on” time in terms of fraction of LCD frame time is given by  
                 Δ   ⁢           ⁢     T   ⁡     (     i   ,   j     )         =     λ   ⁢           ⁢     I   ⁡     (     i   ,   j     )           ⁢     
     ⁢       Δ   ⁢           ⁢       T   2     ⁡     (     i   ,   j     )         =       (     1   +       mMap   ⁡     (     i   ,   j     )       4       )     ⁢       Δ   ⁢           ⁢     T   ⁡     (     i   ,   j     )         2         ⁢     
     ⁢       Δ   ⁢           ⁢     T   1       =       Δ   ⁢           ⁢   T     -     Δ   ⁢           ⁢     T   2                   (   10   )               
         [0065]     In some embodiments, the next step is to predict the backlight image from the LED. The LED image may be upsampled to the LCD resolution (m×n) and convolved with the PSF of the LED.  
         [0066]     The LCD transmittance may be determined using Equation 10. 
 
 T   LCD ( x, y )= img ( x, y )/ bl ( x, y )   (10) 
 
         [0067]     In some embodiments, inverse gamma correction may also be performed to correct the nonlinear response of the LCD. In these embodiments, a normalized LCD transmittance value  100  may be mapped with a tonescale curve  102  to an LCD driving value  104 .  
         [0068]     The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalence of the features shown and described or portions thereof.