Abstract:
This invention presents a YUV to RGB conversion method which preserves high precision of luminance information in an original YUV image signal when converting it to RGB signal. The method can be used to convert the original YUV signal to arbitrary quantization levels in RGB space. In addition, this invention presents methods of pre-quantization and re-quantization as to compensate conventional YUV to RGB color conversion.

Description:
FIELD OF THE INVENTION  
       [0001]     The present invention relates generally to video image processing and in particular to color conversion and quantization within the YUV (Y represents luminance, U a first color difference, and V a second color difference) and the RGB (Red, Green, Blue) color space.  
       BACKGROUND OF THE INVENTION  
       [0002]     As the display technology advances, the RGB color space is commonly used in virtually every computer system as well as television, video etc. The RGB color space is easy to implement although it is non-linear with visual perception. In order to be displayed on RGB displays, images and videos coded in YUV space are converted into RGB signals (usually the 24-bit RGB true color signals).  
         [0003]     Research has shown that the human eye is capable of distinguishing around 10 million different colors under optimal viewing conditions. The traditional true color display with 8 bit precision on each channel can display about 16.7 million colors, which is more than 10 million. However, the true color display is still not sufficient for human vision perception. This is mainly because of the non-uniformity, in the sense of human perception, of the RGB space. The non-uniformity leads to the result that in some regions of the RGB color space the different colors are perceived the same and in some other regions the colors jump by more than one JND (just noticeable difference). Therefore on the one hand, to reduce the color differences of neighboring quantization levels, higher bit-depth displays, including the higher bit processing chips and drivers, are becoming a trend in the display industry. On the other hand, with the existing color display that cannot achieve continuous perceptual colors, image processing methods are expected to produce perceptually better images.  
         [0004]     The human visual system is much more sensitive to luminance than to chrominance. Research has shown that the human eye can distinguish roughly 463 different grayscales, which is about 9 bits, while it can only distinguish about 150 different hues and 50 different saturation levels. The traditional 24-bit true color display can display only less than 255 different shades of grayscales (considering some of them are indistinguishable), which is insufficient for human eyes. In addition, as the display is made brighter and brighter, the banding artifacts become more obvious between neighboring shades of gray scales in 24-bit displays.  
         [0005]     In regards to converting YUV signals to RGB signals, the fact that the human vision is much more sensitive to luminance than to chrominance, makes it possible to manipulate the converted RGB color signals to preserve luminance while keeping the difference in the chrominance within a tolerable range.  
       BRIEF SUMMARY OF THE INVENTION  
       [0006]     In one embodiment the present invention provides a method for color conversion from the YUV color space to the RGB color space, where the accuracy of converted RGB signals is constrained by the bit-depth (i.e., number of bits that can be used to present the signal). Such a color conversion method essentially preserves luminance information of the original color signal compared to the conventional color conversion methods that use simple rounding after a double precision transformation. By selecting the proper RGB values, the luminance difference between the input signal and output signal can essentially be minimized. However, the minimization of the luminance may result in a large difference in the chrominance (color) components between the input and output signals. Therefore, to maintain the color difference within a tolerable range, color conversion is formulated and solved as a constrained minimization problem.  
         [0007]     As such, an example method of video image color conversion according to the present invention, comprises the steps of: receiving an image signal in YUV color space having a first precision; converting the image signal into a quantized RGB color space having a second precision; such that the luminance of the image signal is essentially preserved in the quantized RGB signal. The luminance difference between the signals before and after conversion is essentially minimized by trading off the chrominance differences. Minimizing the luminance difference includes the steps of constraining chrominance errors while essentially minimizing the luminance difference.  
         [0008]     In another example, the conversion step further includes the steps of: determining a set of possible RGB signal values such that the difference in chrominance before and after conversion is constrained; and selecting an RGB signal value from the above possible RGB signal values that essentially minimizes the difference in luminance before and after the conversion.  
         [0009]     In addition, because conventional color conversion from YUV to RGB color space is very commonly used in various hardware and chips, to achieve the same luminance preserving result for color conversion, in another aspect the present invention provides a pre-quantization method for YUV signals and a re-quantization method for RGB signals, to compensate the conventional conversion methods.  
         [0010]     In another aspect the present invention provides a converter and video systems that implement the above example methods.  
         [0011]     Other embodiments, features and advantages of the present invention will be apparent from the following specification taken in conjunction with the following drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0012]      FIG. 1  shows an example functional block diagram of an example color space conversion from high precision YUV to high or low precision RGB according to an embodiment of the present invention.  
         [0013]      FIG. 2  shows a block diagram of a conventional color conversion method from YUV space to RGB space, where simple rounding is used after double precision rgb is obtained.  
         [0014]      FIG. 3  shows a block diagram of a conventional color conversion method from RGB space to YUV space, where simple rounding is used after double precision yuv is obtained.  
         [0015]      FIG. 4 ( a ) shows an example functional block diagram of pre-quantization of high precision YUV color to a same precision YUV color, according to an embodiment of the present invention.  
         [0016]      FIG. 4 ( b ) shows an example functional block diagram of a system, where pre-quantization of the system in  FIG. 4 ( a ) is used in front of a hardware system that immediately thereafter perform conventional color conversion.  
         [0017]      FIG. 4 ( c ) shows that to achieve luminance preserving RGB value in the output of the system in  FIG. 4 ( b ), the two consecutive conventional color conversions should be equivalent to an identity transformation.  
         [0018]      FIG. 5 ( a ) shows an example block diagram of re-quantization of an RGB color to a same precision RGB color, according to an embodiment of the present invention.  
         [0019]      FIG. 5 ( b ) shows an example function block diagram of a system, where re-quantization in  FIG. 5 ( a ) is used immediately after a system that contains a conventional color conversion before output.  
         [0020]      FIG. 5 ( c ) shows that to achieve luminance preserving RGB value in the output of the system in  FIG. 5 ( b ), the two consecutive conventional color conversions should be equivalent to an identity transformation. 
     
    
       [0021]     In the drawings, like references indicate like elements.  
       DETAILED DESCRIPTION OF THE INVENTION  
       [0022]     In one embodiment, the present invention provides a luminance preserving color conversion method and system which can be utilized to convert arbitrary quantized or un-quantized YUV signals to arbitrary quantized RGB signals. In the following description, in representing a color component, upper case letters [R, G, B] and [Y, U, V] are used to indicate lower precision (e.g., 8-bit values) and lower case letters [r, g, b] and [y, u, v] are used to indicate the higher precision values, more bit or double precision. Further in the drawings, “L” on an input or output line indicates low precision, “H” on an input or output line indicates high precision, and “S” on an input or output line indicates same precision.  
         [0023]      FIG. 1  shows a functional block diagram of an example system  100  for color space conversion from high precision YUV to high or low precision RGB, according to an embodiment of the present invention. An input YUV color signal is first converted to double precision rgb value in a conversion block  110 , wherein the rgb value is used to determine a set of RGB values in a block  120 , wherein one of the RGB values which essentially minimizes the luminance difference is selected in the selection block  130  as output of the system  100 . In order to simplify the description, in this example, a 24-bit RGB signal is used as the system output.  
         [0024]     Conversion between YUV and RGB can take many different formats depending on the standards used. The present invention is not dependent on any standard. An example of YUV to RGB conversion is provided in relation (1) below:  
                   [         y           u           v         ]     =       M   ·     [         r           g           b         ]       =       [           M   1               M   2               M   3           ]     ·     [         r           g           b         ]           ,     
     ⁢   where     ⁢     
     ⁢         M   1     =     [         0.299       0.587       0.114         ]       ,     
     ⁢       M   2     =     [           -   0.169           -   0.332         0.500         ]       ,     
     ⁢       M   3     =     [         0.500         -   0.419           -   0.081           ]       ,             (   1   )             
 
         [0025]     where M=[M 1 , M 2 , M 3 ]T is the conversion matrix, r, g, b and y are in the range [0, 1] and u and v are in the range [−0.5, 0.5].  
         [0026]     The corresponding inverse conversion from YUV to RGB is provided in relation (2) below:  
               [         r           g           b         ]     =         M     -   1       ·     [         y           u           v         ]       =       [         1       0       1.407           1         -   0.345           -   0.717             1       1.779       0         ]     ·     [         y           u           v         ]                 (   2   )             
 
         [0027]     In the example YUV to RGB conversation herein, given a high precision [y, u, v] T  signal, first the corresponding high precision RGB values [r, g, b] T  are obtained using relation (2) above. The resulting high precision [r, g, b] T  is to be quantized to 8-bit RGB colors for display. The common practice has been to simply round the high precision [r, g, b] T  to 8-bit [R, G, B] T . However, this causes quantization errors in RGB color space, which can be transformed back to equivalent quantization errors in YUV color space.  
         [0028]     As mentioned, the human eye is much more sensitive in luminance than in chrominance, and the quantization errors from simple rounding in RGB color space is perceptually non-uniform for luminance and chrominance components. To obtain a high quality 24-bit RGB image according to an embodiment of the present invention, the quantization error in luminance components is reduced and preferably essentially minimized by trading off the accuracy in chrominance components. In other words, a set of 8-bit RGB triples are found such that the quantization error in luminance components is essentially minimized. However, this minimization of luminance components is achieved at the expense of increasing the quantization errors in chrominance (color) components which will also cause color difference. Therefore, to keep the color difference within a tolerable range, the chrominance errors are constrained while minimizing luminance quantization errors.  
         [0029]     Accordingly, the quantization is formulated as a constrained minimization problem. The quantization errors in luminance are minimized while ensuring that the difference between chrominance (colors) before and after quantization is constrained, such as according to example relation (3) below:  
               [         R           G           B         ]     =     arg   ⁢           ⁢       min         [     R   ,   G   ,   B     ]     T     ∈     D   ⁡     (     y   ,   u   ,   v     )           ⁢            y   -       M   1     ·          [         R           G           B         ]              ,                   (   3   )             
 
         [0030]     where D(y, u, v) is a range that for colors [R, G, B] T εD, the chrominance difference between [R, G, B] T  and [y, u, v] T  is tolerable (wherein x=arg min f(x) means x minimizes f(x)).  
         [0031]     If the setup parameters of the display (i.e., the system white point, chromaticity coordinates of R, G and B, and the gamma law) are available, the colors can be converted into device independent color space, e.g., CIE Lab color space, and CIE 1994 color difference model used to calculate the color difference between the high precision [r, g, b] T  color and the quantized [R, G, B] T  colors. The CIE 1994 color difference model is based on CIELAB and attempted to derive local distortions of the CIELAB to make the color difference more uniform in the color space. CIELAB color space is directly based on CIE XYZ, which is device independent and at the root of all colorimetry. CIE94 color difference DE*94 is defined on CIE L*a*b* color space as follows:  
         [0032]     For two colors C 1 =[L1, a1, b1]T and C 2 =[L2, a2, b2]T in L*a*b* color space, the CIE94 color difference DE* 94 is as example relation (4) below (wherein L, a and b are coordinates in L*a*b* color space):  
                   DE   *   94   ⁢     (       C   1     ,     C   2       )       =           (     dl     kl   ·   sl       )     2     +       (       d   ⁢           ⁢   c       kc   ·   sc       )     2     +       (     dh     kh   ·   sh       )     2           ,     
     ⁢   where     ⁢     
     ⁢       dl   =       L   1     -     L   2         ,     
     ⁢     dh   =     {                 de   2     -     dl   2     -     d   ⁢           ⁢     c   2                   if   ⁢           ⁢     de   2       &gt;       dl   2     +     d   ⁢           ⁢     c   2                   0       else         ,     
     ⁢     de   =           (       L   1     -     L   2       )     2     +       (       a   1     -     a   2       )     2     +       (       b   1     -     b   2       )     2           ,     
     ⁢       d   ⁢           ⁢   c     =         c   2     -     c   1       ⁢     
     ⁢           =           a   1   2     +     b   1   2         -         a   2   2     +     b   2   2               ,     
     ⁢       and   ⁢     
     ⁢   kl     =     kc   =     kh   =   1         ,     
     ⁢     sl   =   1     ,     
     ⁢     sc   =     1   +     0.045   ·   c         ,     
     ⁢     sh   =     1   +     0.015   ·   c         ,     
     ⁢       where   ⁢     
     ⁢   c     =           c   1     ·     c   2         .                       (   4   )             
 
         [0033]     With the help of the CIE 1994 color difference model, the quantization in relation (3) above can be further formulated as minimizing the difference between the real luminance value y and the luminance value represented by 8-bit [R, G, B] T , with the constraint that the color difference between them, expressed in DE*94, is smaller than a Threshold as shown by example in relation (5) below:  
                 [         R           G           B         ]     =     arg   ⁢           ⁢       min       R   ,   G   ,     B   ∈   Z           DE   *   94     &lt;   Threshold         ⁢          y   =       M   1     ·     [         R           G           B         ]                    ,           (   5   )             
 
         [0034]     In order to reduce the computation needed for this minimization for real-time TV signal conversion, a simpler format is provided herein below.  
         [0035]     The main idea is same as above, however DE* 94 is not calculated. Instead, a small range is defined as: 
 
 [R, G, B] T  └r┘≦R≦┌r┐, └g┘≦G≦┌g┐ and └b┘≦B≦┌b┐, where └·┘ is the nearest quantization level that is less than or equal to and ┌·┐ is the nearest quantization level that is less than or equal to ·. In other words, [R,G,B] can take values only at the eight vertices of the unit cube that contains high precision value [r,g,b] T . Then, the minimization is according to example relation (6) below:  
               [         R           G           B         ]     =     arg   ⁢           ⁢       min       R   ,   G   ,     B   ∈   Z             ⌊   r   ⌋     ≤   R   ≤     ⌈   r   ⌉             ⌊   g   ⌋     ≤   G   ≤     ⌈   g   ⌉           ⌊   b   ⌋     ≤   B   ≤     ⌈   b   ⌉               ⁢            y   -       M   1     ·     [         R           G           B         ]              .                 (   6   )             
 
         [0036]     This minimization problem can be solved by an exhaustive search, wherein the resulting signals from the quantization method contain color values that have higher precision on luminance value than conventional methods. The error in luminance value (Y) of an individual pixel is then contained in a smaller range than the conventional direct rounding quantization. Considering the 24-bit RGB quantization example, then: 
 
Δ RGB=[R−└r┘, G−└g┘, B−└b┘]   T , 
 
 such that the difference in luminance is: 
 
Δ Y=M   1   *ΔRGB.  
 
         [0037]     Table. I summarizes the illuminance difference Δy between the 8 different possible quantized RGB colors and colors [└r┘, └g┘, └b┘] T :  
                                                                                                     TABLE I                           LUMINANCE DIFFERENCES FOR EACH POSSIBLE       QUANTIZATION RGB VALUE COMPARED WITH [└r┘, └g┘, └b┘] T .                ΔRGB                [0, 0,       [0, 1,                   [1, 1,           0]   [0, 0, 1]   0]   [0, 1, 1]   [1, 0, 0]   [1, 0, 1]   [1, 1, 0]   1]                        Δy   0   0.114   0.587   0.701   0.299   0.413   0.886   1                  
 
         [0038]     Because the quantization process minimizes the luminance difference Δy, the achievable luminance steps δy are much smaller than can be represented by 8-bit Y value. The values δy in Table I are sorted and δy values are calculated and shown in Table II below:  
                                                                                                     TABLE II                           REACHABLE LUMINANCE STEPS.                Index.                0   1   2   3   4   5   6   7                        Δy   0   0.114   0.299   0.413   0.587   0.701   0.886   1       δy       0.114   0.185   0.114   0.174   0.114   0.185   0.114                  
 
         [0039]     As can be seem in Table II, the biggest luminance step δy is 0.185. Its equivalent bit-depth of luminance value Y is increased by δd bit, where:  
         δ   ⁢           ⁢   d     =         log   0.5     ⁢   0.185     ⁢     
     ⁢           =     2.43   ⁢     bit   .             
 
         [0040]     Therefore, the quantized 8-bit RGB values preserve luminance precision as high as 8+2.43=10.43 bit.  
         [0041]     Using the above example method according to the present invention, a YUV signal can be converted to an RGB signal, preserving high precision luminance values while keeping small color differences. The high precision luminance is achieved at the expense of the precision of chrominance, which is less sensitive in human eyes.  
         [0042]     Because the conventional color conversion from YUV color space to RGB color space is very commonly used in various hardware and chips, to achieve the same luminance preserving results for the existing conventional YUV to RGB conversion methods, the present invention further provides a pre-quantization method for the input YUV signals to the conventional YUV to RGB conversion, and a requantization method for the output RGB signals from the conventional YUV to RGB conversion. The pre-quantization for YUV signal can be performed before the input YUV signal to the existing chips if the conventional color conversion is performed immediately after the input. Alternatively, the re-quantization of RGB signal can be performed after the output RGB signal of the existing chips if the conventional color conversion is performed immediately before output.  
         [0043]     Conventional YUV to RGB color conversion comprises of a transformation and a rounding process.  FIG. 2  shows a block diagram of a conventional system  200  for color conversion from YUV space to RGB space, where simple rounding is used after double precision rgb is obtained. Further,  FIG. 3  shows a block diagram of a conventional system  300  for color conversion from RGB space to YUV space, where simple rounding is used after double precision yuv is obtained.  
         [0044]      FIG. 4 ( a ) shows a functional block diagram of a system  400  for pre-quantization of high precision YUV color to a same precision YUV color which includes a luminance preserving conversion block  410  in addition to a conventional RGB to YUV color conversion block  200  that performs a transformation process and a rounding process. Further,  FIG. 4 ( b ) shows a functional block diagram of an example system  450  wherein the pre-quantization block  400  of  FIG. 4 ( a ) is used with an existing system  455  that includes a conventional YUV to RGB color conversion block  300  ( FIG. 3 ).  
         [0045]     Referring back to  FIG. 4 ( a ), the input YUV signal is first converted to RGB signal with the same precision as the output in  FIG. 4 ( b ), using the luminance preserving conversion block  410 , wherein in this example, block  410  implements the YUV to RGB conversion luminance preserving function of the block  100  in  FIG. 1 . The RGB output from block  410  in  FIG. 4 ( a ) is then converted to a YUV signal using the conventional color conversion block  200 . For an existing hardware or chip that takes a quantized YUV signal as input, and performs conventional YUV to RGB color conversion immediately thereafter, a pre-quantization block  400  according to an embodiment of the present invention is added before the conventional conversion block  300  ( FIG. 4 ( b )) in that hardware such that the RGB signals converted after block  300  are the same as if the hardware is using the luminance preserving conversion from YUV to RGB according to the present invention. To achieve luminance preserving RGB values in the output of system  450  in  FIG. 4 ( b ), the two consecutive conventional color conversion blocks  200  and  300  should be equivalent to an identity transformation, as shown by example in  FIG. 4 ( c ).  
         [0046]     As such, first, according to  FIG. 4 ( a ), the input YUV signal is converted to an RGB signal using the example luminance preserving conversion block  410  according to the present invention, and then converted back to YUV signal using the conventional color conversion block  200 .  FIG. 4 ( b ) illustrates the entire procedure, including the pre-processing block  400  and the conventional color conversion  300  in the existing hardware  455 . In order to make the generated RGB signal exactly the same as the resulting RGB signal after the luminance preserving conversion within the pre-processing block  400 , the two sequentially connected conventional color conversions blocks  200 ,  300 , back and forth between RGB and YUV color space should be equivalent to an identity transformation as shown in  FIG. 4 ( c ).  
         [0047]     Because of the limited precision of the signal, considering the conversion matrix between RGB and YUV color space, as in relation (2) above, in one example the basis of YUV space: [{right arrow over (y)},{right arrow over (u)},{right arrow over (v)}] T  has coordinates in RGB space as:  
           y   →     =     [         1           1           1         ]       ,       u   →     =     [         0             -   0.345             1.779         ]       ,       v   →     =       [         1.407             -   0.717             0         ]     .           
 
 Their norms can be calculated as: 
 
∥{right arrow over (y)}∥=1.732, ∥{right arrow over (u)}∥=1.812, ∥{right arrow over (v)}∥=1.5792 
 
 with the three angles among the basis vectors as 117, 104 and 95 degrees. The YUV color space is much coarser than the RGB space if the same bit-depth of data are used to present color. Therefore, in order to make the two sequentially connected conventional color converters  200 ,  300  shown on the left side of  FIG. 4 ( c ) equivalent to the unity operation shown on the right side of  FIG. 4 ( c ), higher precision YUV values than RGB values are used. Generally, 1-bit more precision makes the YUV color space finer than the RGB color space. 
 
         [0048]     Similarly, for an existing hardware or chip that processes the color signal in YUV space and outputs RGB colors using a conventional color conversion method, the output RGB color can be re-quantized so that the resulting RGB color after re-quantization preserves high precision luminance from the YUV signal in the hardware.  FIG. 5 ( a ) shows a block diagram of an example system  500  according to the present invention for re-quantization of RGB color to a same precision RGB color. The input RGB color is first converted to lower or same precision YUV using a conventional color conversion block  300  and then the resulting YUV color is converted back to RGB color using the conversion block  410  which in this example implements the YUV to RGB conversion luminance preserving function of the block  100  in  FIG. 1 .  
         [0049]      FIG. 5 ( b ) shows a block diagram of an example system  550  according to the present invention wherein the re-quantization block  500  of  FIG. 5 ( a ) is used immediately after the conventional YUV to RGB conversion block  200  in the hardware. As such, the example in  FIG. 5 ( a ) shows the post-processing step of RGB re-quantization according to the present invention, and the example in  FIG. 5 ( b ) shows the entire procedure including the conventional conversion block  200  in the hardware, and the post-processing step in block  500  according to the present invention. Similar to  FIG. 4 ( c ), the example in  FIG. 5 ( c ) illustrates the equivalence of the two sequentially connected conventional color conversion blocks  200 ,  300 , and the identity transformation. In this case, because the RGB space is finer than YUV space, higher or same precision RGB is acceptable.  
         [0050]     While this invention is susceptible of embodiments in many different forms, there are shown in the drawings and will herein be described in detail, preferred embodiments of the invention with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the broad aspects of the invention to the embodiments illustrated. The aforementioned example architectures above according to the present invention, can be implemented in many ways, such as program instructions for execution by a processor, as logic circuits, as ASIC, as firmware, etc., as is known to those skilled in the art. Therefore, the present invention is not limited to the example embodiments described herein.  
         [0051]     The present invention has been described in considerable detail with reference to certain preferred versions thereof; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.