Abstract:
A method to predict visual quality of a DCT (discrete cosine transform) based compressed image or video stream without referring to its source. When applied to an MPEG video stream, the method is based on (1) an estimation of quantization errors using MPEG quantization scales and statistics of the inverse quantized DCT coefficients, (2) a blind estimation of the 8×8 and 16×16 blocking effect, and (3) an adaptive combination of the quantization error estimation and the blocking effect estimation using the MPEG motion vector information. The method may be used in many applications, such as network video servers, switches and multiplexers for automatic quality monitoring and control of video services, video encoders, decoders, transcoders, and statistical multiplexers for picture quality optimization.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention relates generally to a method and system for objective quality assessment of DCT (discrete cosine transform) based compressed image or video streams, and more particularly pertains to a method and system for objective quality assessment of compressed images of MPEG (moving picture experts group) video streams without the need for the original or reference video source. The method utilizes only the information embedded in the compressed video stream, which consists a sequence of coded frames, to assess the picture visual quality. The inventive method consists of the steps of quantization error estimation, blocking effect estimation, and adaptive combining of these estimates with the motion information if available for each coded frame.  
           [0003]    2. Discussion of the Prior Art  
           [0004]    Most objective image and video quality measurement approaches require the original image or video source as a reference, which is often not available in practical network image and video communication applications.  
           [0005]    In the past several decades, great efforts have been made to develop objective image and video quality measurement approaches that aim to predict perceived image and video quality automatically. The most widely used method is the peak signal-to-noise ratio (PSNR). Other well known approaches include the Sarnoff just-noticeable-difference (JND) model [J. Lubin, “The use of psychophysical data and models in the analysis of display system performance”, in A. B. Watson (ed.),  Digital Images and Human Vision , pp. 163-178, MIT press, 1993] and the NASA digital video quality (DVQ) model [A. B. Watson, J. Hu, J. F. McGowan III, and J. B. Mulligan, “Design and performance of a digital video quality metric”,  Proceedings of SPIE, Human Vision, Visual Processing, and Digital Display IX , San Jose, 2001]. A novel quality measurement algorithm has also been developed [Z. Wang and L. Lu, “Objective image and video quality assessment,”  Technical Report, IBM T J. Watson Research Center , August 2001] which has demonstrated advantages in better correlating with subjective measurement over other approaches in experiments on the video quality experts group (VQEG) Phase I test data set [VQEG, “Final report from the video quality experts group on the validation of objective models of video quality assessment,” http://www.vqeg.org/, March 2000]. These objective quality measurement approaches require the original source sequence as a reference. However, in many practical image and video service applications, especially network image and video communications, the reference source is often not available.  
           [0006]    It would be useful to develop single-ended or “blind” video quality measurement algorithms, where access to the reference video sequence is not required.  
           [0007]    Little has been done in designing blind video quality assessment methods in the literature. In [M. Knee, “A robust, efficient and accurate single-ended picture quality measure for MPEG-2”, presented at VQEG meeting  2001 ], a single-ended approach was proposed for quality measurement of MPEG-2 video. This method, known as Picture Appraisal Rating (PAR), assumes a distribution of the DCT coefficient and attempts to predict PSNR using an “activity” parameter and the quantization scale parameters, which are available from the MPEG bitstream. The author claimed that their measurement results have a good correlation with PSNR without using the reference video sequence. Software, hardware and embedded system products that implement this method are for sale by the Snell &amp; Wilcox Company (http://www.snellwilcox.com) The PAR approach has several disadvantages:  
           [0008]    1) It tries to estimate PSNR, which has been widely criticized for not correlating well with perceived video quality. A better benchmarking technique is needed to assess the method and also to improve the method.  
           [0009]    2)/Blocking artifact has proved to be one of the most annoying artifacts in Discrete Cosine Transform (DCT) based image and video coding algorithms. However, it is not considered as a distinct factor in PAR. As a result, PAR cannot properly capture and measure the blocking effect, which is similar to PSNR.  
           [0010]    3) There is more information than just quantization scales available from the MPEG bitstream, which is not considered by the PAR method. The statistics of the inverse quantized coefficients and the motion information given by the motion vectors would also be useful for the design of no-reference video quality metrics.  
         SUMMARY OF THE INVENTION  
         [0011]    Accordingly, it is a primary object of the present invention to provide a method and system for objective quality assessment of DCT based compressed image and video streams.  
           [0012]    The present invention provides a method of automatic measurement of compressed image and video quality without the need for the original or reference image and video source. The method utilizes only the information embedded in the compressed image or video stream, which consists of a sequence of coded frames, to assess the picture visual quality. The inventive method consists of the steps of quantization error estimation, blocking effect estimation, and adaptive combining of these estimates with the motion information if available for each coded frame. The estimation of the quantization errors uses the information of the quantization scales and statistics of the inverse quantized DCT coefficients in that coded frame. The blocking effect is estimated based on the block sizes used in the signal processing for compression, for example, 8×8, 8×16, 16×16, etc. The quantization error estimation and the blocking effect estimation are adaptively combined linearly or non-linearly and using motion information if available.  
           [0013]    The present invention provides a new method to predict visual quality of DCT based compressed image and video without referring to its source. The method, when applied to MPEG video, is based on (1) an estimation of quantization errors using MPEG quantization scales and statistics of the inverse quantized DCT coefficients, (2) a blind estimation of the 8×8 and 16×16 blocking effect, and (3) an adaptive combination of the quantization error estimation and the blocking effect estimation using the MPEG motion vector information. The method correlates well with a recently developed full-reference video quality index [Z. Wang and L. Lu, “Objective image and video quality assessment,”  Technical Report, IBM T. J. Watson Research Center , August 2001], which has shown its advantages over many state-of-the-art objective video quality assessment methods. The subject invention may be used in many applications, such as network video servers, switches and multiplexers for automatic quality monitoring and control of video services. It may also be used in video encoders, decoders, transcoders, and statistical multiplexers for picture quality optimization. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0014]    The foregoing objects and advantages of the present invention for a method and system for objective quality assessment of image and video streams may be more readily understood by one skilled in the art with reference being had to the following detailed description of several embodiments thereof, taken in conjunction with the accompanying drawings wherein:  
         [0015]    [0015]FIG. 1 is a high level block diagram of the method of the present invention for a no-reference quality measurement of a compressed video stream.  
         [0016]    [0016]FIG. 2 is a high level block diagram of the method of the present invention for a no-reference quality measurement of a compressed image stream.  
         [0017]    [0017]FIG. 3 is a logic flow diagram of the quantization error estimation pursuant to the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0018]    [0018]FIG. 1 is a high level block diagram of the method of the present invention for a no-reference quality measurement of a compressed video stream, such as an MPEG video stream or other DCT based compressed video streams such as H.261, H.263 and H.264 standard coded video streams. The input to the system is simply a compressed video bitstream  10 . No reference sequence is needed. The output quality index value  12  can be evaluated and reported on either a frame basis or a sequence basic, depending on the application.  
         [0019]    First, the input compressed video bitstream  10  is partially decoded by a video decoder  13  (such as an MPEG decoder for an MPEG video bitstream) to obtain (1) the inverse quantized DCT coefficients, (2) the quantization scale and (3) the motion vector for each block.  
         [0020]    Second, the quantization error (which is the error due to quantization of values, e.g. a 3.2 value on a scale of 0 to 10 is quantized to a 3.0 value) is estimated at  14 . If the probability distribution of the DCT coefficients is known, then the quantization error can be exactly calculated using the quantization scales. In MPEG decoding, this distribution is not known and has to be estimated. In [M. Knee, “A robust, efficient and accurate single-ended picture quality measure for MPEG-2”, presented at VQEG meeting  2001 ], the author assumed a fixed form of distribution (e.g. the Laplacian distribution) such that the problem is simplified to estimating one parameter. However, this assumption is not guaranteed to give a precise and reliable approximation of the true distribution. In the present method, histogram statistics are applied to the inverse quantized DCT coefficients, which are available from the video decoder, to obtain a histogram. The histogram is then used to estimate the distribution on a piece-wise basis (which is different from [M. Knee, “A robust, efficient and accurate single-ended picture quality measure for MPEG-2”, presented at VQEG meeting  2001 ]).  
         [0021]    For a given DCT coefficient, if the inverse quantized value is L and the quantization scale is q, then the quantization error is estimated as  
       E   =         ∫     L   -     q   /   2         L   +     q   /   2                       x   -   L          2                     p        (   x   )               x             ∫     L   -     q   /   2         L   +     q   /   2                p        (   x   )                          x                                 
 
         [0022]    where p(x) is the estimated probability distribution of the DCT coefficient. The quantization errors of all the DCT coefficients are then averaged together to obtain an estimate of the overall quantization error of the frame.  
         [0023]    The blocking effect or block artifact is a form of the noise or distortion introduced in block based DCT coding due to excessive quantization and appears as one or more small blocks in a decoded image or video frame. The blocking effect is one of the most annoying distortions, especially at low bit rate coding using block based DCT coding algorithms and is a very important picture quality factor in MPEG coding. The blocking effect is estimated at  15 . The estimation of the blocking effect can be performed in either the spatial domain with a fully decoded frame [Z. Wang, A. C. Bovik, and B. L. Evans, “Blind measurement of blocking artifact in images,”  IEEE International Conference on Image Processing , vol. 3, pp. 981-984, September 2000] or in the DCT domain without full decoding [A. C. Bovik, and S. Liu, “DCT-domain blind measurement of blocking artifacts in DCT-coded images,”  IEEE International Conference on Acoustic, Speech, and Signal Processing , vol. 3, pp. 1725-1728, May 2001]. In a preferred embodiment, the evaluation of the blocking effect is performed in the DCT domain without full decoding. An alternative embodiment can perform evaluation of the blocking effect with a fully decoded frame. However, the alternative approach requires a Fast Fourier Transform (FFT) on the image signal, which is computationally complicated. The present invention uses a simplified method. First, take a row f(n), n=1, 2, . . . from the image, and take the absolute difference of adjacent samples: d(n)=|f(n)−f(n−1)|. A blocky signal d(n) will have periodic peaks. The periodic peaks are detected, and then the relative magnitude between the peaks in d(n) and their adjacent samples of d(n) are used as an estimate of blocking effect. The same process is performed on all or a proportion of all the rows (depending on allowed computation power) and columns. The blockiness values are then averaged to obtain the overall blocking effect measure of the frame.  
         [0024]    The human visual sensitivity to errors changes with motion. In MPEG the motion information can be simply evaluated at  16  by using the motion vectors extracted from the MPEG bitstream. In a preferred embodiment, only the magnitude of the motion vectors is calculated and used by the algorithm of the present invention, but could be extended to consider other aspects of the motion vectors.  
         [0025]    Finally, the quantization error estimation is adaptively combined with the blocking effect estimation at  18 . In the preferred embodiment, only a simple linear combination of these two factors is used and normalized to generate a single overall quality measure or index of the frame at  12 , although a more sophisticated combination, such a non-linear combination, can also be used. When available, the motion information is used in some special cases as a sign of adjustment. The reason to use motion information is based upon the fact what when large motion exists, the human eyes become less sensitive to the errors caused by blurring. In MPEG, this usually corresponds to the cases where the blocking effect is small. Therefore, if the frame exhibits severe quantization error but its blocking effect is small, and at the same time the motion is large, then the measure is adjusted to obtain a better quality index value at  12 .  
         [0026]    The method described above is for the evaluation of a single MPEG frame. The frame quality values can also be averaged to provide a quality measurement of a group of frames or pictures or even an entire video sequence.  
         [0027]    [0027]FIG. 2 is a high level block diagram of the method of the present invention for a no-reference quality measurement of a compressed image stream  20  such as a JPEG coded image stream. The input to the system is simply a compressed image bitstream  20 . No reference sequence is needed.  
         [0028]    First, the input compressed image bitstream  20  is partially decoded by an image decoder  23  to obtain (1) the inverse quantized DCT coefficients, (2) the quantization scale and (3) motion vectors for each frame, if available.  
         [0029]    Second, the quantization error is estimated at  24 . If the probability distribution of the DCT coefficients is known, then the quantization error can be exactly calculated using the quantization scales. In JPEG decoding, this distribution is not known and has to be estimated, similar to  14  of FIG. 1.  
         [0030]    The blocking effects is evaluated at  25 , similar to  15  of FIG. 1.  
         [0031]    Finally, the quantization error estimation is combined with the blocking effect estimation at  28 . This is not an adaptive combination as in FIG. 1 as motion evaluation is not a factor in a compressed image stream. In the preferred embodiment, only a simple linear combination of these two factors is used and normalized to generate a single overall quality measure or index of the frame at  22 , although a more sophisticated combination, such as a non-linear combination, can also be used.  
         [0032]    [0032]FIG. 3 is a logic flow diagram of the quantization error estimation at  14  or  24 . First obtain the inverse quantized DCT coefficients of a video frame or an image frame at  30 . Next use the inverse quantized DCT coefficients to generate histogram statistics at  31 . The histogram statistics are then used to estimate the distribution of the DCT coefficients on a piece-wise basis at  32 . Next, obtain the quantization error of each of the DCT coefficients at  33 , and average the quantization errors of all inverse quantized DCT coefficients at  34 , which are then used to estimate the overall quantization error of the video frame or image frame at  35 .  
         [0033]    While several embodiments and variations of the present invention for a method and system for objective quality assessment of video streams are described in detail herein, it should be apparent that the disclosure and teachings of the present invention will suggest many alternative designs to those skilled in the art.