Abstract:
Presented herein are system(s), method(s), and apparatus for improving the display of compressed video data. In one embodiment, there is presented a method for encoding video data. The method comprises estimating a displayed intensity of a video source displayed on a predetermined display device; estimating a displayed intensity of the reconstruction of the video source displayed on a predetermined display device; and correcting the video source as a function of the estimated displayed intensity of the video source on the predetermined display device and the estimated displayed intensity of the reconstruction of the video source on the predetermined display device.

Description:
RELATED APPLICATIONS 
     This application claims priority to “Systems, Methods, and Apparatus for Improving Display of Compressed Video Data”, Provisional Application for U.S. Patent Ser. No. 60/681,269, filed May 16, 2005, which is incorporated herein by reference for all purposes. 
    
    
     FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
     [Not Applicable] 
     MICROFICHE/COPYRIGHT REFERENCE 
     [Not Applicable] 
     BACKGROUND OF THE INVENTION 
     Video compression is used to compress video data for storage on devices with limited memory and transmission over communication channels with limited bandwidth. A number of video compression standards, such as MPEG-2, VC-1, and AVC compress the video data by, among other things, removing or deemphasizing higher frequency components. The higher frequency components have been observed to be less perceptible to the human eye. 
     However, when the compressed video data is decompressed and displayed on a display device, the video data has been observed to produce a different intensity compared to the original video data. The intensity difference is the result of the nonlinear response of system. The eye takes things with high spatial frequency and averages their intensity. When a nonlinear response is involved, this averaging changes the perceived value. For example, if a system has a response of y=x 2 , with the values of x, 1, 2, and 3, the output is 1, 4, and 9. If the input is low pass filtered and changed to 2, 2, and 2, the output is 4, 4, 4. In the original case, the average of the output value is 14/3 In the low pass filtered case, it is 4. 
     Display devices, such as CRTs, have non-linear characteristics between pixel value and brightness. To compensate for this, the displayed video data is gamma corrected. However, even where the decompressed video is perceptually similar to the original video, its response to gamma correction often produces a different intensity compared to the original video data. 
     Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings. 
     BRIEF SUMMARY OF THE INVENTION 
     Aspects of the present invention may be found in a system, method, and/or apparatus for improving the display of compressed video data, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims. 
     These and other advantages and novel features of the present invention, as well as illustrated embodiments thereof will be more fully understood from the following description and drawings. 
    
    
     
       BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS 
         FIG. 1  is a block diagram of an exemplary circuit in accordance with an embodiment of the present invention; 
         FIG. 2  is a flow diagram for encoding video data in accordance with an embodiment of the present invention; 
         FIG. 3A  is a block diagram describing spatial prediction; 
         FIG. 3B  is a block diagram describing motion prediction; 
         FIG. 3C  is a block diagram describing transformation and quantization; 
         FIG. 4  is a block diagram of an exemplary circuit in accordance with an embodiment of the present invention; 
         FIG. 5  is a block diagram of an exemplary circuit in accordance with an embodiment of the present invention; and 
         FIG. 6  is a flow diagram for encoding video data in accordance with an embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
     Referring now to  FIG. 1 , there is illustrated a block diagram describing an exemplary circuit  100  for encoding video data in accordance with an embodiment of the present invention. 
     The circuit  100  comprises a first circuit  110  and a second circuit  200 . The first circuit  110  estimates the display intensity for a video source, when the video source is displayed on a predetermined display device and estimates a displayed intensity of the reconstruction of the video source displayed on a predetermined display device. 
     It is noted that the video source may be compressed prior to transmission over a communication media. Reconstructed video data can be video data that results from compressing and then decompressed the video source. 
     The display intensity can be estimated in a variety of ways. In certain embodiments of the present invention, the display intensity can be measured by applying a transformation to the video source and the reconstructed video data that models the predetermined display device. 
     The display intensity of the reconstructed video source can be estimated in a variety of ways. In certain embodiments of the present invention, the video source can be compressed in accordance with an encoding standard and then decompressed. A transformation modeling the display device can be applied to the decompressed video source to estimate the display intensity of the reconstructed pixels. 
     The second circuit  200  corrects the video source as a function of the estimated displayed intensity of the video source on the predetermined display device and the estimated displayed intensity of the reconstruction of the video source on the predetermined display device. 
     In certain embodiments of the present invention, the video source can be corrected as a function of the difference between the estimated displayed intensity of the video source on the predetermined display device and the estimated displayed intensity of the reconstruction of the video source on the predetermined display device. In certain embodiments of the present invention, the difference can comprise the average difference. In accordance with certain embodiments of the present invention, the resulting corrected video source can then be compressed and transmitted over a communication media. 
     Referring now to  FIG. 2 , there is illustrated a flow diagram for encoding video data in accordance with an embodiment of the present invention. At  205 , the intensity of the video source displayed on a predetermined display device is estimated. At  210 , the intensity of a reconstruction of the video source displayed on the predetermined display device is measured. At  215 , the video source is corrected as a function of the estimated displayed intensity of the video source on the predetermined display device and the estimated displayed intensity of the reconstruction of the video source on the predetermined display device. 
     As noted above, there are a variety of different compression standards for compressing video data. Certain embodiments of the present invention can be incorporated in the context of a compression standard. In certain embodiments of the present invention, the video source can be compressed in accordance with a standard. The reconstruction of the compressed video data can be the resulting video data that is generated by decompressing the compressed video data and the display intensity of the reconstructed data can be estimated. The video source can be corrected according to the display intensity of the video source and the reconstructed video data. The corrected video source can then be compressed and transmitted in accordance with the compression standard. An exemplary compression standard will now be described followed by exemplary embodiments of the present invention. 
     Advanced Video Coding (also known as H.264 and MPEG-4, Part 10) generally provides for the compression of video data by dividing video pictures into fixed size blocks, known as macroblocks  320 . The macroblocks  320  can then be further divided into smaller partitions  430  with varying dimensions. 
     The partitions  430  can then be encoded, by selecting a method of prediction and then encoding what is known as a prediction error. AVC provides two types of predictors, temporal and spatial. The temporal prediction uses a motion vector to identify a same size block in another picture and the spatial predictor generates a prediction using one of a number of algorithms that transform surrounding pixel values into a prediction. Note that the data coded includes the information needed to specify the type of prediction, for example, which reference frame, partition size, spatial prediction mode etc. 
     The reference pixels can either comprise pixels from the same picture or a different picture. Where the reference block is from the same picture, the partition  430  is spatially predicted. Where the reference block is from another picture, the partition  430  is temporally predicted. 
     Spatial Prediction 
     Referring now to  FIG. 3A , there is illustrated a block diagram describing spatially encoded macroblocks  320 . Spatial prediction, also referred to as intra prediction, is used by H.264 and involves prediction of pixels from neighboring pixels. Prediction pixels are generated from the neighboring pixels in any one of a variety of ways. 
     The difference between the actual pixels of the partition  430  and the prediction pixels P generated from the neighboring pixels is known as the prediction error E. The prediction error E is calculated and encoded. 
     Temporal Prediction 
     Referring now to  FIG. 3B , there is illustrated a block diagram describing temporally prediction. With temporal prediction, partitions  430  are predicted by finding a partition of the same size and shape in a previously encoded reference frame. Additionally, the predicted pixels P can be interpolated from pixels in the frame or field, with as much as ¼ pixel resolution in each direction. A macroblock  320  is encoded as the combination of data that specifies the derivation of the reference pixels P and the prediction errors E representing its partitions  430 . The process of searching for the similar block of predicted pixels P in pictures is known as motion estimation. 
     The similar block of pixels is known as the predicted block P. The difference between the block  430  and the predicted block P is known as the prediction error E. The prediction error E is calculated and encoded, along with an identification of the predicted block P. The predicted blocks P are identified by motion vectors MV and the reference frame they came from. Motion vectors MV describe the spatial displacement between the block  430  and the predicted block P. 
     Transformation, Quantization, and Scanning 
     Referring now to  FIG. 3C , there is illustrated a block diagram describing the encoding of the prediction error E. With both spatial prediction and temporal prediction, the macroblock  320  is represented by a prediction error E. The prediction error E is a two-dimensional grid of pixel values for the luma Y, chroma red Cr, and chroma blue Cb components with the same dimensions as the macroblock  320 , like the macroblock. 
     In certain embodiments of the present invention, the display intensity of a macroblock  320  of the video source and the display intensity of the corresponding macroblock in the reconstructed video source can be estimated. The difference of the display intensity of the macroblock of the video source and the corresponding macroblock in the reconstructed video source can be added to the prediction error E, thereby resulting in a corrected prediction error. 
     A transformation transforms the prediction errors E to the frequency domain. In H.264, the blocks can be 4×4, or 8×8. The foregoing results in sets of frequency coefficients f 00  . . . f mn , with the same dimensions as the block size. The sets of frequency coefficients are then quantized, resulting in sets  440  of quantized frequency coefficients, F 00  . . . F mn . 
     Quantization is a lossy compression technique where the amount of information that is lost depends on the quantization parameters. The information loss is a tradeoff for greater compression. In general, the greater the information loss, the greater the compression, but, also, the greater the likelihood of perceptual differences between the encoded video data, and the original video data. 
     The pictures  115  are encoded as the portions  120  forming them. The video sequence is encoded as the frames forming it. The encoded video sequence is known as a video elementary stream. 
     Referring now to  FIG. 4 , there is illustrated a block diagram of an exemplary circuit for encoding video data in accordance with an embodiment of the present invention. The circuit comprises a first compressor  405 , a decompressor  410 , a display intensity transformation circuit  415 , a subtractor  420 , an adder  425 , and a second compressor  432 . 
     The first compressor  405  compresses the video source. The decompressor  410  decompresses the video source, thereby resulting in a reconstructed video source. The display intensity transformation circuit  415  measures the display intensity for the video source and the reconstructed video source. The subtractor  420  measures the difference of the display intensity for the video source and the reconstructed video source. The adder  425  adds the difference of the display intensity to the video source, thereby resulting a corrected video source. The second compressor  430  compresses the corrected video source. In certain embodiments, a low pass filter  440  filters the difference. 
     Referring now to  FIG. 5 , there is illustrated a block diagram of another exemplary circuit in accordance with an embodiment of the present invention. The circuit comprises a predictor circuit  505 , first transformer/quantizer  510 , an inverse quantizer/transformer  515 , a motion compensator  518 , a display intensity transformer  520 , a subtractor  525 , an adder  530 , and a second transformer/quantizer  535 . 
     The predictor  505  receives actual blocks from the video source for either spatial or motion prediction and outputs a prediction error E and an identification of the reference pixels. It is noted that the reference pixels can be identified by a motion vector (where the case is motion prediction). The transformer/quantizer  510  transforms and quantizes the prediction error E. The inverse quantizer/transformer  515  inverse quantizes and transforms, providing a reconstructed prediction error E′. 
     Display intensity transformer  520  determines the display intensity for the reconstructed prediction error E′ and the original prediction error E. The subtractor  525  provides the difference between the display intensity for the prediction error E and the reconstructed prediction error E′. Adder  530  adds the difference to the original prediction error E′, thereby resulting in a corrected prediction error. The second transformer/quantizer  535  transforms and inverse quantizes the corrected prediction error. 
     Referring now to  FIG. 6 , there is illustrated a flow diagram for encoding video data in accordance with an embodiment of the present invention. At  605 , a portion of a picture from the video source is predicted, either temporally or spatially, thereby resulting in a prediction error. At  610 , the prediction error is transformed and quantized, thereby resulting in a compressed portion of the video source. 
     At  615 , the compressed video source is decompressed, thereby resulting in a reconstructed portion of the video source. A transfer function that models the predetermined display device is applied to portion of the video source at  620  and the reconstructed video source at  625  to estimate the display intensity of the portion of the video source and the reconstructed portion of the video source. 
     At  630 , the difference between the estimated display intensity of the portion of the video source and the reconstructed portion of the video source is taken, and added to the prediction error at  635 , thereby resulting in a corrected prediction error. At  640 , lossy compression is applied to the prediction error. 
     The embodiments described herein may be implemented as software, a board level product, as a single chip, application specific integrated circuit (ASIC), or with varying levels of the decoder system integrated with other portions of the system as separate components. 
     The degree of integration of the decoder system will primarily be determined by the speed and cost considerations. Because of the sophisticated nature of modern processor, it is possible to utilize a commercially available processor, which may be implemented external to an ASIC implementation. 
     If the processor is available as an ASIC core or logic block, then the commercially available processor can be implemented as part of an ASIC device wherein certain functions can be implemented in firmware. Alternatively, the functions can be implemented as hardware accelerator units controlled by the processor. 
     While the present invention has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present invention. 
     Additionally, many modifications may be made to adapt a particular situation or material to the teachings of the present invention without departing from its scope. For example, although the invention has been described with a particular emphasis on H.264 encoded video data, the invention can be applied to a video data encoded with a wide variety of standards. 
     Therefore, it is intended that the present invention not be limited to the particular embodiment disclosed, but that the present invention will include all embodiments falling within the scope of the appended claims.