Patent Publication Number: US-2013243084-A1

Title: Bit rate regulation module and method for regulating bit rate

Description:
TECHNICAL FIELD 
     The present invention generally relates to devices and methods for regulating bit rate for encoding video frames. 
     BACKGROUND 
     The approaches described in this section could be pursued, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section. Furthermore, all embodiments are not necessarily intended to solve all or even any of the problems brought forward in this section. 
     In a block-based video encoder, following for instance the H.264 recommendation, one of the main control parameters for the bit rate is the quantization parameter (QP) or quantization step (Qstep), which may be chosen at various levels. In a video sequence level, a video is encoded with a constant quality level, using a single quantization step for the whole sequence. In a video frame level, the quantization step varies from one frame to the other. And, in a macro-block level, the quantization step varies from one macro-block to the other, a macro-block comprising for example 16*16 pixels. 
     A frame of a video is encoded according to a frame type, which can be Intra-frame (I), Predicted-frame (P) or Bidirectional-frame (B). See for example the H.264 standard. 
     A frame of I frame type can be decoded independently of any other frame. 
     The P frame type improves compression by exploiting the temporal redundancy in a video. P frames encode only the difference in image from a reference frame (e.g. an I picture frame or a P picture frame or a B picture frame) immediately preceding it in coding order. This reference frame is also called an anchor frame. 
     The B frame type is quite similar to P frame type, except it can make predictions by predicting from two anchor frames at the same time. 
     A goal of a bit rate regulation module is to control the Qstep parameter during encoding, in order to enforce a required bit-rate and meet buffer constraints. Given a target bit budget for a frame and the frame type (I/P/B) of the next frame to be encoded, the regulation finds an appropriate quantization step using a bit rate model. 
     For example, one possible bit rate model for MPEG-2 is: 
       Qstep*Pbits=χ frame     —     type  
 
     Where Pbits is the number of frame bits, and χ frame     —     type  denotes the complexity of a frame, frame_type being the frame type, either I, P or B. χ frame     —     type  depends on the sequence, the standard and the frame type (I or P or B) but is assumed to remain constant for frames of the same type between scene cuts. 
     For the intra rate model (I frames), there is a good correlation between the spatial activity of the frame (for instance the sum of absolute transform (e.g. Discrete Fourier Transform, Z-transform, Hadamard Transform, Discrete Hartley Transform Discrete Cosine Transform, Discrete Sine Transform, Modified Discrete Cosine Transform, or other suitable transform) coefficients computed on the luminance channel of the frame), quantization steps and bit consumptions. It allows to infer the complexity of the frame, by expanding it as: 
       Qstep*Pbits=α.activity
 
     where α is a constant that depends on the video encoder and video standard, the activity can be measured during a preprocessing step on the frame. 
     Given a bit target Pbits target  for the frame, the regulation module can use the rate model above to derive the right quantization step for the encoding: 
       Qstep=α.activity/Pbits target  
 
     For the inter rate model (P or B frames), right after a scene-cut (i.e. when the video content changes from one scene to the other), the complexity of the frame χ frame     —     type  is unknown. For the first P or B frame after a scene-cut, a default complexity χ frame     —     type  is used. Right after encoding, once the number of bits for the frame is known, the complexity can be updated as: 
       χ frame     —     type =Qstep*Pbits
 
     For the next frame of the same type, the regulation module can use the formula below to determine the right quantization step from the target bits: 
       Qstep=χ frame     —     type /Pbits target  
 
     One problem with the solution above is that for temporally predicted frames such as P or B frames, which are predicted from previously encoded frames (called reference frames), the equation Qstep*Pbits=χ frame     —     type  is valid only in stationary mode, i.e. when the reference frame(s) was(were) already encoded with the same quantization step. 
     When the bit budget changes from one frame to the other, or if the rate model did not lead to the correct quantization step computation for the previous frame because the measured complexity was not valid, the regulation will have problems to find the correct value and stabilize the quantization step for the sequence, causing variations in the video quality. 
     Thus, there is a need for improved methods and devices for bit rate regulation. 
     SUMMARY 
     To address these needs, a first aspect of the present invention relates to a bit rate regulation module comprising:
         a selection block configured to select a frame type to be used to encode a current frame of a video,   a determination block configured to determine a target number of bits to be used to encode the current frame, and
           a computation block configured to compute a quantization step to be used to encode the current frame by using a rate model taking into account a quantization step of a previously encoded reference frame and by modelling rate variations due to a quantization step change between the previously encoded reference frame and the current frame. One object of this bit rate regulation module is to predict the number of bits for encoding a frame by a quantization step. Such prediction may be achieved even if the quantization step is not the same one as in the reference frame. Such prediction may be achieved even for a temporally predicted frame such as P or B picture. Thus, the prediction result may be improved. A better prediction typically allows the bit rate regulation module to converge more quickly to a stable quantization step size value, which in turn offers a more stable, and thus subjectively better, video quality.   
               

     In some embodiments of the invention, the bit rate regulation module may be configured to select the frame type among Intra-frame, Predicted-frame and Bidirectional-frame. 
     In some embodiments of the invention, the bit rate regulation module may be configured to select the frame type according to a predefined pattern. 
     In some embodiments of the invention, for predicted frames such as P or B frames, the bit rate regulation module may be configured to model rate variations by decomposing a number of bits required for encoding the current frame into a first number of bits required if the previous reference frame had the same quantization step as the current frame, and a second number of bits corresponding to a change of quantization step between the previous reference frame and the current frame. 
     The bit rate regulation module may further be configured to define the first number of bits as the ratio of a complexity of the current frame to the quantization step of the current frame. 
     The bit rate regulation module may further be configured to define the second number of bits as the difference of the first number of bits and a number of bits associated with the previous reference frame. 
     The bit rate regulation module may further be configured to define the number of bits as a function of the complexity, of the quantization step of the current frame, and of the quantization step of the previous reference frame. 
     The bit rate regulation module may further be configured to compute the quantization step of the current frame as a function of the complexity, of the quantization step of the previous reference frame, and of the target number of bits. 
     In some embodiments of the invention, the bit rate regulation module may further be configured to update the rate model once the current frame has been encoded. 
     The bit rate regulation module may further be configured to update the rate model by updating a complexity of the current frame by removing a number of bits, corresponding to a change of quantization step between the previous reference frame and the current frame, from an actual number of bits produced for the frame. 
     A second aspect relates to a video encoder comprising:
         a reception module configured to receive a current frame of a video,   a bit rate regulation module according to the first aspect, and   an encoding module configured to encode the current frame by using the quantization step computed by the bit rate regulation module.       

     A third aspect relates to an electronic device comprising a video encoder according to the second aspect. 
     A fourth aspect relates to a method for regulating bit rate for encoding a video comprising the steps of:
         determining a target number of bits to be used to encode a current frame of the video,   computing a quantization step to be used to encode the current frame by using a rate model taking into account a quantization step of a previously encoded reference frame and by modelling rate variations due to a quantization step change between the previously encoded reference frame and the current frame.       

     The method may comprise a step of selecting a frame type to be used to encode the current frame. 
     The method may further comprise a step of encoding the current frame by using the quantization step computed. 
     The modelling may comprise an operation of decomposing a number of bits required for encoding the current frame into a first number of bits required if the previous reference frame had the same quantization step as the current frame, and a second number of bits corresponding to a change of quantization step between the previous reference frame and the current frame. 
     A fifth aspect relates to a computer program product comprising one or more stored sequences of instructions that are accessible to a processor and which, when executed by the processor, cause the processor to carry out the steps of the method according to the fourth aspect. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS  
       The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which: 
         FIG. 1  is a schematic block diagram of a video encoder according to an embodiment of the invention; 
         FIG. 2  is a flow chart showing steps of a method for encoding a frame according to an embodiment of the invention; 
         FIG. 3  is a chart which shows curves giving the quantization parameter versus the frame number for rate models according to the state of the art and to some embodiments of the invention; 
         FIG. 4  is a chart which shows curves giving the number of bits versus the frame number for rate models according to the state of the art and to some embodiments of the invention; and 
         FIG. 5  is a schematic block diagram of a bit rate regulation module of the video encoder of  FIG. 1 . 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     Embodiments of the invention deal with the problem of improving the quality of a video by improving the bit rate regulation during encoding. Embodiments of the invention may, in particular, be applicable for temporally predicted frames such as Predicted-frame (P) or Bidirectional-frame (B). 
       FIG. 1  shows a video encoder  1  comprising a reception module  2 , a bit rate regulation module  3 , and an encoding module  4 . 
     The reception module  2  is configured to receive a picture or frame of the video. 
     Referring to  FIG. 5 , the bit rate regulation module  3  comprises a selection block  31  configured to select a frame type to be used to encode the frame received. The frame type can, for example, be Intra-frame (I), Predicted-frame (P) or Bidirectional-frame (B). 
     The bit rate regulation module  3  further comprises a determination block  32  configured to determine, by using the frame type selected, a target number of bits for encoding the frame. 
     The bit rate regulation module  3  further comprises a computation block  33  configured to compute, by using the target number of bits, a quantization step Qstep. As explained below, the quantization step Qstep is computed by using a rate model, which takes into account a quantization step Qstep ref  of a previous reference frame and models rate variations due to a quantization step change. 
     The encoding module  4  is configured to encode the frame by using the quantization step Qstep computed by the bit rate regulation module  3 . 
     Referring to  FIG. 2  we are describing below steps of a method for encoding a frame according to an embodiment. The method of  FIG. 2  may, according to some embodiments be performed by the video encoder of  FIG. 1 . 
     In step S 1 , the bit rate regulation module  3  selects a frame type to be used to encode a frame. The frame may have been received by the reception module  2  of the video encoder  1 . 
     The frame type can be Intra-frame (I), Predicted-frame (P) or Bidirectional-frame (B). The selection may, for example, be done according to a predefined pattern. For instance, the I frame type may be selected for a first frame of the video, then the P frame type may be selected for a second frame of the video, then the B frame type may be selected for a third frame of the video, etc. 
     In step S 2 , the bit rate regulation module  3  uses the frame type selected in step S 1  to determine a target number of bits for encoding the frame. 
     The determination may be performed by any suitable method currently known in the prior art or later to be developed. 
     For example, if the regulation module works on the basis of a bit rate window, and has a target number of bits over this window WindowTargetBits, the target number of bits for a P frame can be given by 
       Pbits target =(WindowTargetBits*WeightP)/(NumberI*WeightI+NumberP*WeightP+NumberB*WeightB) 
     where NumberI, NumberP and NumberB are respectively the number of I, P and B frames in the bit rate window, and WeightI, WeightP and WeightB are respectively the weights for I, P and B frames. Such weights can be arbitrarily defined a priori, for instance with WeightI=6, WeightP=2 and WeightB=1, to reflect the fact that empirically, for a given Qstep, the P frame type requires twice the number of bits of the B frame type, and the I frame type requires three times the number of bits of the P frame type. This formula simply balances the target bit budget between the frames that will be encoded within the sliding window according to the encoded frame types. It should be noted that alternatively, step S 2  can be done using any suitable method of the prior art or developed in the future. 
     In step S 3 , the bit rate regulation module  3  uses the target number of bits determined in step S 2  to compute—based on the rate model—a quantization step Qstep that will match the target number of bits. 
     The quantization step Qstep for predicted frames such as P or B frames is computed by using a rate model in which the number of produced bits is decomposed in two terms: 
       Pbits=Pbits stationary mode +Pbits change    
     where Pbits refers to the number of bits required for the frame to be encoded, Pbits stationary mode  refers to the number of bits required if the reference frame for encoding had the same quantization step Qstep as the current frame, and Pbits change  refers to the number of bits coming from a change of quantization step between the reference frame and the current frame. 
     For a given quantization step Qstep, we have: 
       Pbits stationary mode =χ frame type /Qstep
 
     where χ frame type  denotes the complexity of the frame and depends on the sequence, the standard and the frame type (I, P or B). 
     When the reference frame has been encoded with a quantization step Qstep ref , the difference in bits between the current frame and the reference frame is given by the difference of their stationary mode bits: 
       Pbits change =Pbits stationary mode −Pbits stationary mode,ref  
 
     Furthermore, the number of bits of stationary mode for the reference frame is equal to: 
       Pbits stationary mode,ref =χ frame     —     type /Qstep ref ,
 
     As a consequence, we have: 
       Pbits change =χ frame     —     type /Qstep−χ frame     —     type /Qstep ref  
 
     When adding Pbits stationary mode  and Pbits change , we finally get: 
       Pbits=2*χ frame     —     type /Qstep−χ frame     —     type /Qstep ref  
 
     Using this rate model, the bit rate regulation module  3  can then find the appropriate quantization step Qstep for the current frame given the target number of bits Pbits target , the complexity χ frame     —     type and the Qstep ref  of the reference frame: 
       Qstep=2*χ frame     —     type /(Pbits target +χ frame     —     type /Qstep ref )
 
     In step S 4 , the encoding module  4  encodes the frame by using the quantization step Qstep computed in step S 3 . 
     In step S 5 , the bit rate regulation module  3  updates the rate model given the number of produced bits for the frame. In other words, once the frame has been coded, the complexity χ frame     —     type  is updated as if the frame had been encoded in the stationary mode, by removing the predicted Pbits change  from the actual number of bits Pbits produced for the frame, as: 
       χ frame     —     type =(Pbits−Pbits change )*Qstep
 
     After step S 5 , the video encoder  1  is ready to encode the next frame of the video, as symbolized by the arrow A next frame . 
     One advantage of the rate model described above is that it allows to predict the number of bits for encoding a frame by a quantization step Qstep even if it is not the same one as in the reference frame. A better prediction allows the bit rate regulation module  3  to converge more quickly to a stable quantization step size value, which in turn offers a more stable, and thus subjectively better, video quality. 
     When the video encoder  1  works in low-delay mode, for example for video telephony, it also allows to encode more frames. If the bit rate regulation module  3  does not find the right quantization step Qstep for the given target number of bits, either the frame cannot be encoded in low delay and is skipped by the encoder, or it arrives out of delay at the encoder. Embodiment of the invention decreases the probability of this problem. 
       FIG. 3  shows a curve C 1  giving the quantization parameter QP versus the frame number for a rate model according to the state of the art, and a curve C 2  giving the quantization parameter QP versus the frame number for the rate model according to an embodiment of the invention. As it can be seen, the quantization parameter QP, which indicates the actual quantization step Qstep in the bitstream, found by the regulation module according to the state of the art oscillates from one frame to the other. This oscillation causes unstable numbers of bits used for each frame, as represented in  FIG. 4  showing a curve C 10  giving the number of bits versus the frame number according to the state of the art and a curve C 20  giving the number of bits versus the frame number according to an embodiment of the invention. 
     One other advantage of the method described above is that it also allows to keep a simple linear rate model, avoiding more complex quadratic ones, such as the VM8 proposed for MPEG-4. 
     Embodiments of the invention may be implemented in hardware or software or a combination thereof. For example, embodiments of the inventions may be implemented in an integrated circuit such as an application-specific integrated circuit (ASIC). 
     Embodiments of the invention may be performed by a computer program product comprising one or more stored sequences of instructions. Sequences of instructions are accessible to a processor and, when executed by the processor, cause the processor to carry out steps of the method for encoding a video. The computer program product may be, for example, a CD-ROM. 
     While there has been illustrated and described what are presently considered to be the preferred embodiments of the present invention, it will be understood by those skilled in the art that various other modifications may be made, and equivalents may be substituted, without departing from the true scope of the present invention. Additionally, many modifications may be made to adapt a particular situation to the teachings of the present invention without departing from the central inventive concept described herein. Furthermore, an embodiment of the present invention may not include all of the features described above. Therefore, it is intended that the present invention not be limited to the particular embodiments disclosed, but that the invention include all embodiments falling within the scope of the invention as broadly defined above. 
     Expressions such as “comprise”, “include”, “incorporate”, “contain”, “is” and “have” are to be construed in a non-exclusive manner when interpreting the description and its associated claims, namely construed to allow for other items or components which are not explicitly defined also to be present. Reference to the singular is also to be construed in be a reference to the plural and vice versa. 
     A person skilled in the art will readily appreciate that various parameters disclosed in the description may be modified and that various embodiments disclosed may be combined without departing from the scope of the invention.