Abstract:
A method and apparatus for decoding video data that is encoded by bit rate control to keep a bandwidth of a bit stream in a predetermined range are provided. The encoded video data is decoded, picture quality estimation information is estimated, the estimation including information about a bit stream size of a decoded video frame, the number of fragmented macro blocks included in the video frame, and the number of skip-mode macro blocks included in the video frame, the picture quality of the video data is determined based on a correlation between picture quality estimation information and SNR, and the video data is output by applying a picture quality improving algorithm to the video data, if the determined picture quality is lower than a predetermined level.

Description:
CLAIM OF PRIORITY  
       [0001]    This application claims the benefit of the earlier filing date, pursuant to 35 USC 119, to that patent application filed in the Korean Patent Office on Dec. 20, 2007 and afforded serial number 10-2007-0134331, the contents of which are incorporated by reference herein. 
       BACKGROUND OF THE INVENTION  
       [0002]    1. Field of the Invention 
         [0003]    The present invention relates to a video decoding and more particularly, to a method for decoding data encoded by bit rate control. 
         [0004]    2. Description of the Related Art 
         [0005]    A conventional video encoder uses bit rate control to keep the bandwidth of a bit stream within a predetermined range. The bit rate control scheme keeps the bandwidth of the bit stream constant by controlling at least a quantization value that determines picture quality. However, the change of the quantization value leads to fluctuation of the picture quality, which is the main cause of picture flickering and low-quality video. 
         [0006]    To reduce the bit rate control-incurred change of video quality in the video encoder, a video decoder should be aware of the level of picture quality. For this purpose, the Peak Signal to Noise Ratio (PSNR) of an original video is calculated during encoding and is provided to the video decoder on a separate channel. Thus, this conventional video encoding/decoding technology requires an additional device for performing a PSNR calculation and an additional channel to deliver information about the picture quality level to provide the picture quality level information to the video decoder. 
       SUMMARY OF THE INVENTION  
       [0007]    An aspect of exemplary embodiments of the present invention is to provide a method for decoding encoded video data of a bit stream format to high-quality video data without separately generating picture quality information. 
         [0008]    Moreover, an aspect of exemplary embodiments of the present invention provides a decoding method for estimating a picture quality level based on information acquired during decoding video data and filtering the decoded video data according to the estimated picture quality level. 
         [0009]    In accordance with an aspect of exemplary embodiments of the present invention, there is provided a method for decoding video data that is encoded by bit rate control, in which the encoded video data is decoded, picture quality estimation information including information about a bit stream size of a decoded video frame, the number of fragmented macro blocks included in the video frame, and the number of skip-mode macro blocks included in the video frame is estimated, the picture quality of the video data is determined based on a correlation between picture quality estimation information and Signal-to-Noise Ratio (SNR), and the video data is output by applying a picture quality improving algorithm to the video data, if the determined picture quality is lower than a predetermined level. 
         [0010]    The SNR can be PSNR. The SNR is estimated by the below equation and it is preferable to determine a picture quality of the video data based on the estimated SNR. 
         [0000]      Cost(SIZE, SKIP,4 MV)=ƒ(SIZE)·α+ƒ(SKIP)·β+ƒ(4 MV)·γ       where α, β and γ are weights and   ƒ(x) is a standardization function defined as         
         [0000]    
       
         
           
             
               f 
                
               
                 ( 
                 x 
                 ) 
               
             
             = 
             
               
                 X 
                 - 
                 μ 
               
               μ 
             
           
         
       
       
         
           
             where X is a target value and u is an accumulated average. 
           
         
       
     
         [0014]    To determine the picture quality, an SNR decrease can be calculated. The picture quality improving algorithm can be a PSNR improving algorithm. 
         [0015]    In accordance with another aspect of exemplary embodiments of the present invention, there is provided an apparatus for decoding video data that is encoded by bit rate control, in which a decoder decodes the encoded video data, a picture quality detector estimates picture quality estimation information including information about a bit stream size of a decoded video frame, the number of fragmented macro blocks included in the video frame, and the number of skip-mode macro blocks included in the video frame, and measures the picture quality of the video data based on a correlation between picture quality estimation information and SNR, a picture quality improvement filter improves the picture quality of the video data received from the decoder, and a selection switch switches the video data selectively to one of a picture quality improvement filter and an external device according to the picture quality of the video data. 
         [0016]    It is preferred that the picture quality detector measures the picture quality of the video data by calculating a decrease in an SNR estimated using the picture quality estimation information. The picture quality improvement filter can be a noise reduction filter. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS  
         [0017]    The above features and advantages of certain exemplary embodiments of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: 
           [0018]      FIG. 1  is a block diagram of a video decoder according to an exemplary embodiment of the present invention; 
           [0019]      FIG. 2  illustrates an exemplary video frame decoded by the video decoder according to the exemplary embodiment of the present invention; 
           [0020]      FIG. 3  is a graph illustrating PSNRs improvement by use of a picture quality improvement filter in the video decoder according to the exemplary embodiment of the present invention; 
           [0021]      FIG. 4  is a flowchart illustrating a video decoding method according to an exemplary embodiment of the present invention; 
           [0022]      FIG. 5 . illustrates a condition of an exemplary embodiment of the present invention; 
           [0023]      FIG. 6  illustrates a result estimated under the condition of  FIG. 5 ; and 
           [0024]      FIG. 7  illustrates a coefficient between the PSNR and the greatest power signal that is calculated by using the result of  FIG. 6 . 
       
    
    
       [0025]    Throughout the drawings, the same drawing reference numerals will be understood to refer to the same elements, features and structures. 
       DETAILED DESCRIPTION OF THE INVENTION  
       [0026]    The matters defined in the description such as a detailed construction and elements are provided to assist in a comprehensive understanding of exemplary embodiments of the invention. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness. 
         [0027]      FIG. 1  is a block diagram of a video decoder according to an exemplary embodiment of the present invention. 
         [0028]    Referring to  FIG. 1 , the video decoder according to the exemplary embodiment of the present invention includes a Moving Picture Experts Group-4/Advanced Video Coding (MPEG-4/AVC) decoder  10 , a picture quality detector  20 , a selection switch  30 , and a picture quality improvement filter  40 . 
         [0029]    The MPEG-AVC decoder  10  receives data encoded in a bit stream format by an MPEG-4/AVC encoder (not shown) and outputs color difference signal values for each pixel of the received video data. Also, the MPEG-4/AV decoder  10  provides information about the bit stream size, type and the fragmentation state of each macro block acquired during decoding the encoded data to the picture quality detector  20 . 
         [0030]    The picture quality detector  20  acquires picture quality estimation information based on the information received on a macro block basis from the MPEG-4/AVC decoder  10 . The picture quality estimation information includes information about the bit stream byte size of a current received video frame, the number of skip-mode macro blocks included in the video frame, and the number of fragmented macro blocks. The picture quality detector  20  then estimates the Signal-to-Noise Ratio (SNR) of the received video based on the correlation between the SNR and the picture quality estimation information. Preferably, the SNR can be a PSNR. 
         [0031]      FIG. 2  illustrates an exemplary video frame decoded by the video decoder according to the exemplary embodiment of the present invention. 
         [0032]    Referring to  FIG. 2 , reference character “a” denotes skip-mode macro blocks and reference character “b” denotes 4-fragment macro blocks. The picture quality detector  20  counts the number of the skip-mode macro blocks “a” in the video frame, SKIP and counts the number of the 4-fragment macro blocks “b”, 4 MV.Preferably, the picture quality detector  20  calculates a reduction level in a SNR based on the video data that standardizes the correlation between the SNR and the picture quality estimation information. A cost of the SNR of the video image of a current frame is calculated by the below equation (1), using the calculated reduction level. 
         [0000]      Cost(SIZE,SKIP,4 MV)=ƒ(SIZE)·α+ƒ(SKIP)·β+ƒ(4 MV)·γ  (1) 
         [0000]    where  60  , β and γ are weighting factors
       ƒ(x) is a standardization function defined as       
 
         [0000]    
       
         
           
             
               f 
                
               
                 ( 
                 x 
                 ) 
               
             
             = 
             
               
                 X 
                 - 
                 μ 
               
               μ 
             
           
         
       
       
         
           
             
               
                 where X is a target value and μ is an accumulated average. 
               
             
           
         
       
     
         [0035]    In one aspect, α, β and γ can be set to −0.5, 0.1, and −0.4, respectively, and X represents the bit stream byte size of the video frame (SIZE), the number of the skip-mode macro blocks in the video frame (SKIP), or the number of the 4-fragment macro blocks (4 MV), and μ is the accumulated average of X. For instance, if 4 MV=10, 30 and 40 respectively in first, second and third video frames, X=40 for 4 MV and μ is the accumulated average of the first and second video frames, i.e. 20 in the third video frame. 
         [0036]    While the picture quality detector  20  calculates a cost of an SNR of the video data by using the above equation (1) and determines a picture quality level of the video data based on the cost of the SNR in the exemplary embodiment of the present invention, to which the present invention is not limited, the picture quality level can be determined in other ways according to the correlation between SNR and picture quality estimation information extracted during decoding. 
         [0037]    For example, the SNRs and picture quality estimation information of a plurality of videos are measured, the consistency and regularity of the SNRs and the picture quality estimation information are analyzed, and the correlation between SNR and picture quality information is placed accordingly. 
         [0038]    The selection switch  30 , which includes a single input port and a plurality of output ports, selectively switches the input port to one of the output ports. The output ports of the selection switch  30  are connected to an external output device (not shown) and the picture quality improvement filter  40 , respectively. The selection is made among the output ports of the selection switch  30  based on the SNR estimated by the picture quality detector  20 . In other words, the selection switch  30  compares the estimated SNR with the first predetermined threshold. If the estimated SNR is less than the first threshold, the selection switch  30  switches the input port to the output port connected to the picture quality improvement filter  40 . If the estimated SNR is equal to or larger than the first threshold, the selection switch  30  switches the input port to the output port connected to the external output device. Given the first threshold of 40, if the average of SNRs received from the picture quality detector  20  is less than 40, the selection switch  30  connects the input port connected to the MPEG-4/AV decoder  10  to the output port connected to an input path  31  to the picture quality improvement filter  40 . If the average SNR is equal to or larger than 40, the selection switch  30  connects the input port connected to the MPEG-4/AV decoder  10  to the output port connected to an input path  35  to the external output device. 
         [0039]    The picture quality improvement filter  40  receives video data as the color difference signal values for each pixel from the MPEG-4/AVC decoder  10  and cancels noise from the video data. 
         [0040]    For instance, the picture quality improvement filter  40  filters each pixel of the video frame and calculates the difference between the filtered value and the original signal value. If the difference is less than the second predetermined threshold, the picture quality improvement filter  40  cancels noise, determining that the pixel is noise. If the difference is larger than the second threshold, the picture quality improvement filter  40  recovers the original decoded value by adding the difference to the filtered value, considering that the pixel is not noise but an edge. Further, the picture quality improvement filter  40  compensates for a filtering-caused decrease in the value of the edge. 
         [0041]    In accordance with the exemplary embodiment of the present invention, the picture quality improvement filter  40  filters each pixel in the video frame and detects and cancels noise in the video data using the difference between the filtered value and the original signal value, to which the present invention is not limited. Yet, the picture quality improvement filter  40  suffices as far as it improves the picture quality impaired during encoding. For example, the picture quality improvement filter  40  is enough if it can improve PSNR as illustrated in  FIG. 3 . In  FIG. 3 , a dotted line indicates a PSNR before improvement and a sold line indicates a PSNR improved by the picture quality improvement filter as shown in an enlargement area A. 
         [0042]    Now a description will be made of an operation of the video decoder according to an exemplary embodiment of the present invention. 
         [0043]    Upon receipt of data encoded in a bit stream format by a MPEG-4/AVC encoder, the MPEG-4/AVC decoder  10  recovers color difference signal values of pixels included in each video frame in an MPEG-4/AVC decoding scheme. During the decoding, the MPEG-4/AVC decoder  10  provides information about the bit stream size, type, and fragmentation state of each macro block, extracted from the received data, to the picture quality detector  20 . 
         [0044]    The picture quality detector  20  acquires picture quality estimation information based on the information received on a macro block basis and estimates the PSNR of the video frame by equation (1). 
         [0045]    The selection switch  30  compares the estimated PSNR received from the picture quality detector  20  with the first predetermined threshold and selectively switches the input port connected to the MPEG-4/AVC decoder  10  to an output port connected to an external output device or a picture quality improvement filter  40 . To be more specific, for the first threshold of 35.6, the selection switch  30  switches the input port to the output port connected to the picture quality improvement filter  40 , if the estimated PSNR is less than the first threshold of 36.5 and switches the input port to the output port connected to the external output device, if the estimated PSNR is equal to or larger than the first threshold of 36.5. 
         [0046]    When the MPEG-4/AVC decoder  10  is connected to the picture quality improvement filter  40 , the picture quality improvement filter  40  receives video data from the MPEG-4/AVC decoder  10 , filters each pixel of the video frame, and calculates the difference between the filtered value and the original signal value of the pixel. Then if the difference is less than the second predetermined threshold, the picture quality improvement filter  40  determines that the pixel is noise and cancels the noise. If the difference is larger than the second threshold, the picture quality improvement filter  40  determines that the pixel is an edge, not noise, and recovers the original decoded value by adding the difference to the filtered value. Further, Further, the picture quality improvement filter  40  compensates for a filtering-caused decrease in the value of the edge. 
         [0047]    In accordance with the exemplary embodiment of the present invention, the picture quality improvement filter  40  determines the picture quality of the received video data by estimating the SNR of the video data, to which the present invention is not limited. The picture quality level can be one of stepwise values matching to predetermined SNR ranges. For example, picture quality levels 1, 2, 3, 4, 5 and 6 can be set respectively for an SNR range from 0 to below 10, an SNR range from 10 to below 20, an SNR range from 20 to below 30, an SNR range from 30 to below 40, an SNR range from 40 to below 50, and an SNR range from 50 to below 60. In addition, the first threshold of the selection switch  30  is an SNR value that can be one of the SNR ranges. 
         [0048]      FIG. 4  is a flowchart illustrating a video decoding method according to an exemplary embodiment of the present invention. It is assumed that video data encoded by an MPEG-4/AVC encoder is received through a communication device. 
         [0049]    Referring to  FIG. 4 , upon receipt of the encoded data in step  10 , the video decoder recovers color difference signal values of pixels included in each video frame in an MPEG-4/AVC decoding scheme in step  11 . 
         [0050]    In step  13 , the video decoder extracts information about the bit stream size, type, and fragmentation state of each macro block from the video data during the decoding and acquires picture quality estimation information based on the information. The picture quality estimation information includes information about the size of the video frame, the number of skip-mode macro blocks included in the video frame, and the number of fragmented macro blocks included in the video frame. 
         [0051]    The video decoder estimates the SNR of the video frame based on the correlation between picture quality estimation information and SNR in step  15 . For example, the PSNR of the video frame is estimated by 
         [0000]      Cost(SIZE,SKIP,4 MV)=ƒ(SIZE)·α+ƒ(SKIP)·β+ƒ(4 MV)·γ  (2)       where α, β and γ are weights;   ƒ(x) is a standardization function defined as         
         [0000]    
       
         
           
             
               f 
                
               
                 ( 
                 x 
                 ) 
               
             
             = 
             
               
                 X 
                 - 
                 μ 
               
               μ 
             
           
         
       
       
         
           
             
               
                 where X is a target value and μ is an accumulated average. 
               
             
           
         
       
     
         [0055]    That is, X is the bit stream byte size of the video frame (SIZE), the number of the skip-mode macro blocks in the video frame (SKIP), or the number of the 4-fragment macro blocks (4 MV), and μ is the accumulated average of X. For instance, if 4 MV=10, 30 and 40 respectively in first, second and third video frames, X=40 for 4 MV and μ is the accumulated average of the first and second video frames, i.e. 20 in the third video frame. 
         [0056]    In step  17 , the video decoder compares the estimated PSNR with the first predetermined threshold (e.g. 36.5). If the estimated PSNR is less than the first threshold of 36.5, the video decoder performs step  19 . If the estimated PSNR is equal to or larger than the first threshold of 36.5, the video decoder jumps to step  25 . 
         [0057]    In step  19 , the picture quality improvement filter  40  of the video decoder filters each pixel of the video frame and compares the difference between the filtered value and the original value of the pixel with the second predetermined threshold, for noise cancellation. If the difference is less than the second threshold, the picture quality improvement filter  40  performs step  23 . In the step  23 , the picture quality improvement filter  40  determines that the pixel is noise and cancels the noise. If the difference is larger than the second threshold, the picture quality improvement filter  40  performs step  21 . In the step  21 , the picture quality improvement filter  40  determines that the pixel is an edge, not noise, and outputs the original decoded value by adding the difference to the filtered value. 
         [0058]    In step  25 , the video decoder outputs the video data decoded in step  11  or the video data compensated in step  23  to an external output device such as a display. 
         [0059]    While in accordance with the exemplary embodiment of the present invention, the picture quality of the received video data is determined by estimating the PSNR or other measure of quality, of the video data. The picture quality level can be determined as one of stepwise values matching to predetermined SNR ranges. For example, picture quality levels 1, 2, 3, 4, 5 and 6 can be set respectively for an SNR range from 0 to below 10, an SNR range from 10 to below 20, an SNR range from 20 to below 30, an SNR range from 30 to below 40, an SNR range from 40 to below 50, and an SNR range from 50 to below 60. In addition, the threshold of step  17  is an SNR value, which can be one of the SNR ranges. 
         [0060]    While video data is encoded and decoded based on an MPEG-4/AVC standard in the exemplary embodiments of the present invention, to which the present invention is not limited, it is to be clearly understood that the present invention is also applicable to various video encoding/decoding schemes such as H.263 and H.264. 
         [0061]    Also, while the picture quality level of video data is determined by equation (2) in step  15 , to which the present invention is not limited, the determination can be made in many ways based on the correlation between SNR and picture quality information extracted during decoding the video data. For example, the SNRs and picture quality estimation information of a plurality of videos are measured, the consistency and regularity of the SNRs and the picture quality estimation information are analyzed, and the correlation between SNR and picture quality information is placed accordingly. How the correlation between SNR and picture quality estimation information is set will be described, using experiment examples. 
         [0062]    The video encoding and decoding experiments of the present invention are carried out for typical seven videos (Container, Crew, Foremen, Harbor, Mobile, Soccer, and Stefan) under the conditions of image sizes, frame rates, total frame numbers, data rates, and I-frame intervals set as illustrated in  FIG. 5 . 
         [0063]    Referring to  FIG. 5 , an image size of QVGA (320×240 p) is set for Experiment  1 , Experiment  2  and Experiment  3  and an image size of CIF (352×288 p) is set for Experiment  4  to Experiment  7 . The frame rates, total frame numbers, data rates, and I-frame intervals of Experiment  1  to Experiment  7  are the same, 30 fps, 300 frames, 384 kbps, and 30 frames. 
         [0064]    The videos are encoded in MPEG-4. Then the PSNRs of each video are measured and the average AVG and standard deviation STDDEV of the PSNRs are listed in  FIG. 6 . Also, for 300 video frames in each video type, the byte sizes of bit streams SIZE, the numbers of skip-mode macro blocks SKIP, the numbers of intra-mode macro blocks INTRA, average motion vector powers MV_PWR, and the numbers of 4-fragment macro blocks 4 MV are measured and their averages are listed in  FIG. 6 . 
         [0065]    Correlation coefficients between PSNR and SIZE, SKIP, INTRA, MV_PWR, and 4 MV illustrated in  FIG. 6  are computed by equation (3) and listed in  FIG. 7 . 
         [0000]    
       
         
           
             
               Correlation 
                
               
                 ( 
                 
                   X 
                   , 
                   Y 
                 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 
                   
                     ( 
                     
                       x 
                       - 
                       
                         x 
                         _ 
                       
                     
                     ) 
                   
                    
                   
                     ( 
                     
                       y 
                       - 
                       
                         y 
                         _ 
                       
                     
                     ) 
                   
                 
               
               
                 
                   ∑ 
                   
                     
                       
                         ( 
                         
                           x 
                           - 
                           
                             x 
                             _ 
                           
                         
                         ) 
                       
                       2 
                     
                      
                     
                       
                         ( 
                         
                           y 
                           - 
                           
                             y 
                             _ 
                           
                         
                         ) 
                       
                       2 
                     
                   
                 
               
             
           
         
       
       
         
           
             X 
             = 
             
               
                 AVG 
                  
                 
                   ( 
                   
                     array 
                      
                     
                         
                     
                      
                     1 
                   
                   ) 
                 
               
                
               
                 : 
               
                
               PSNR 
             
           
         
       
       
         
           
             
               y 
               = 
               
                 
                   AVG 
                    
                   
                     ( 
                     
                       array 
                        
                       
                           
                       
                        
                       2 
                     
                     ) 
                   
                 
                  
                 
                   : 
                 
                  
                 SIZE 
               
             
             , 
             SKIP 
             , 
             INTRA 
             , 
             MV_PWR 
             , 
             
               4 
                
               
                   
               
                
               MV 
             
           
         
       
     
         [0066]    As noted from Figure xxz PSNR increases with SIZE and with 4 MV whereas PSNR decreases with SKIP. Based on these experiment results, the correlation between PSNR and picture quality estimation information can be set as equation (4), which can be standardized in different ways; 
         [0000]    
       
         
           
             
               
                 
                   
                     1 
                     PSNR 
                   
                   ∝ 
                   
                     
                       ( 
                       
                         1 
                         SIZE 
                       
                       ) 
                     
                     · 
                     
                       ( 
                       SKIP 
                       ) 
                     
                     · 
                     
                       ( 
                       
                         1 
                         
                           4 
                            
                           
                               
                           
                            
                           MV 
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0067]    The above-described methods according to the present invention can be realized in hardware or as software or computer code that can be stored in a recording medium such as a CD ROM, an RAM, a floppy disk, a hard disk, or a magneto-optical disk or downloaded over a network, so that the methods described herein can be rendered in such software using a general purpose computer, or a special processor or in programmable or dedicated hardware, such as an ASIC or FPGA. As would be understood in the art, the computer, the processor or the programmable hardware include memory components, e.g., RAM, ROM, Flash, etc. that may store or receive software or computer code that when accessed and executed by the computer, processor or hardware implement the processing methods described herein. 
         [0068]    As is apparent from the above description, the present invention advantageously estimate the picture quality of video data based on information acquired during decoding encoded video data without additional information representing the picture quality level (e.g. a PSNR). Further, compensation of the video data according to the estimated picture quality results in a high-quality video. 
         [0069]    While the invention has been shown and described with reference to certain exemplary embodiments of the present invention thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims and their equivalents.