Abstract:
A digital video encoder and method for its operation are disclosed. This encoder comprises a frame-layer rate controller that bases a rate-distortion-optimized target bit assignment T j  on an energy estimate for frame j, an average energy estimate for a group of frames, and the desired overall bit rate. If the encoder uses motion compensation, motion bits used for frame j and an average motion bits estimate for a group of frames may also be used by the frame-layer rate controller. Preferably, the desired delay in the system affects how average estimates are computed by the encoder. The frame-layer rate controller of the video encoder above may also utilize buffer protection logic during target bit assignment. This logic adjusts an initial target bit assignment T j  based on the transmission bit rate and desired delay. A primary benefit of the frame-layer rate controller disclosed herein is its basis in a rate-distortion optimal solution to the frame-layer rate control problem. Additional benefits are provided by the explicit consideration of buffer delay during selection of a target bit assignment: rate control robustness allows one controller to be used in a range of delay situations (or even a varying delay situation); and buffer underflow/overflow can be largely eliminated, even in very low delay implementations.

Description:
This application claim benefit to provisional application No. 60/067,692 Dec. 1, 1997. 
    
    
     FIELD OF THE INVENTION 
     This invention pertains generally to digital video encoders, and more particularly to frame-adaptive bit rate control of such encoders. 
     BACKGROUND OF THE INVENTION 
     Uncompressed video requires a relatively high transmission bandwidth. Almost all human-viewable video sequences, however, contain a large amount of redundant and/or visually unimportant information. Digital video allows the use of complex algorithms that remove redundant and relatively unimportant information from a digital video bitstream. With this information removed, video transmission bandwidth may be reduced to acceptable levels. A system that implements video compression algorithms is known as a digital video encoder. 
     FIG. 1 shows a digital video encoder  36  that employs motion compensation to reduce bandwidth. An image sequence  20 , consisting of M frames F 1 , F 2 , . . . F j , . . . F M , provides the input to encoder  36 . Motion compensator  24  is often crucial for effective bandwidth reduction. Motion compensator  24  produces a prediction frame {tilde over (F)} j  for each input frame F j  by comparing regions of F j  to regions from previously encoded frames for similarity. To be more concrete, given a region in frame F j , the encoder searches for regions in previously encoded frames that are good matches for such region and combines these good matches to form the region&#39;s prediction. Thus {tilde over (F)} j  represents an approximation of F j  based on combinations and/or translations of regions of previously encoded frames. This approximated image can be encoded by merely sending the combination and/or translation instructions, or motion vectors MV j , for frame j. 
     Before an image F j  is encoded, approximation {tilde over (F)} j  is subtracted from it in image summer  22 . The remaining motion-compensated frame, i.e., the difference image F j −{tilde over (F)} j , represents the portion of the input image F j  frame that cannot be easily predicted from previous frames. 
     The motion-compensated frame is compressed to a target size of approximately T bits by image encoder  26 . The motion vectors are likewise compressed in motion encoder  28 . When the image sequence is to be transmitted directly to a viewer through channel or interface  34  without intermediate storage (e.g., storage to a disk or other storage media), a buffer  30  is often used to allow the “bursty” encoder outputs to be smoothly transmitted as bitstream  32 . 
     At the receiving end of interface  34 , bitstream  32  is received by a video decoder  38 . Video decoder  38  often places bitstream  32  in a decoder buffer  47  and then parses bitstream  32  to a motion decoder  42  and an image decoder  40 . Image summer  44  combines the output of decoders  40  and  42  to produce an output image sequence  46 . Depending on whether image encoder  26  is lossless or lossy, output image sequence  46  may or may not be an exact representation of image sequence  20 . 
     In a digital video encoder  36 , and more specifically its embedded image coder  26 , it is often preferable to vary the target size T from image frame to image frame. A process that varies T is known as a “frame-layer rate control.” A large body of work on frame-layer rate control has been reported in the patent and academic literature [1-14, Appendix A]. Typically, all of these methods decide the target number of bits for a frame using some formula that depends on the energy in that frame, the number of bits used for encoding previous frames, and the current fullness of the encoder buffer  30  (or that of the decoder buffer  47 , which is equivalent since both buffers are related). 
     For example, the frame-layer rate control described in [1] uses a formula for the frame target that depends on the energy of the pixels in the frame and the number of bits used in a previous frame with similar energy. In [13], the energy and previous bits are combined in a different formula to select the target and in [14] such formula depends on the product of quantization values and bits used for a previous frame of the same type. In [3], the frame target depends on a formula that increases in inverse proportion to the fullness of the encoder buffer. Finally, the method in [12] assigns a fixed target number of bits per frame (equal to the channel rate divided by the frame rate) and skips frames when the encoder buffer is close to buffer overflow. 
     SUMMARY OF THE INVENTION 
     Even though all frame-layer rate control techniques in the prior art measure similar parameters to determine the target number of bits for a frame [1-14], the formula or method chosen for combining these measurements is the key for an effective bit allocation. Typically, these formulas are ad hoc and are not optimized in a rate-distortion sense and, as a result, they do not minimize image distortion (i.e., maximize image quality) for the available bit rate. Additionally, the desired communication delay, which increases with the size of the encoder buffer, is not taken directly into account when deciding the frame target. At low delay, the latter produces large fluctuations of the fullness level in the encoder buffer that lead to undesired buffer overflow, underflow, and frame skipping. 
     The present invention provides a frame-layer rate control mechanism that is based on a rate-distortion optimization. And the present invention further teaches modifications to this basic mechanism that allow a digital video encoder rate controller to respond to differences in communication delay. In addition to the benefits of the rate-distortion optimal solution provided by the basic mechanism, these modifications add robustness to a rate controller, such that one controller can be used in a range of delay situations (or even a varying delay situation). 
     A digital video encoder is disclosed herein. This encoder comprises a frame-layer rate controller that bases a target bit assignment T j  on an energy estimate for frame j, an average energy estimate for a group of frames, and the desired overall bit rate. If the encoder uses motion compensation, motion bits used for frame j and an average motion bits estimate for a group of frames may also be used by the frame-layer rate controller. Preferably, the desired delay in the system affects how average estimates are computed by the encoder. 
     The rate controller of the video encoder above may also use buffer protection logic during target bit assignment. This logic corrects an initial target bit assignment T j , based on the transmission bit rate, desired delay, and current buffer fullness. 
     In a further aspect of the invention, methods for combining the parameters and estimates described above are also disclosed. For example, an initial target bit assignment can be computed by multiplying the desired overall average bit assignment by the ratio of the frame energy estimate to the average energy estimate. Preferably, the average energy estimate is calculated by filtering frame energy estimates with a filter having a time constant dependent on the desired system delay. 
    
    
     BRIEF DESCRIPTION OF THE DRAWING 
     The invention may be best understood by reading the disclosure with reference to the drawing, wherein: 
     FIG. 1 illustrates a block diagram of a prior art digital video encoder; 
     FIG. 2 shows a corresponding block diagram of a prior art digital video decoder; 
     FIG. 3 illustrates the concept of a block coder; 
     FIGS. 4-6 show suggested delay-adaptive relationships for several parameters of frame-adaptive bit rate control; 
     FIG. 7 shows a digital video encoder embodiment of the present invention; and 
     FIG. 8 shows the improvement in image quality in terms of PSNR (peak signal-to-noise ratio) obtained using frame layer rate control according to the present invention, in comparison to the frame layer rate control in [14]. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     A typical block-based image coder will be used to describe the following preferred embodiments. The present invention can, however, be used for assigning the target number of bits per frame within any image or video coder. 
     In block-based image coding, the image to be encoded is decomposed into image blocks of the same size, typically 16×16 pixels per block, and each block is encoded with bits using some coding technique, for example a DCT-based coder. FIG. 3 illustrates the process of encoding the ith block in the jth frame of a video sequence having M frames. 
     In FIG. 3, B i,j  is the number of bits produced when encoding the ith image block in the jth frame and D i,j  is the loss in quality or distortion introduced in that block. In many types of block coders, the distortion is controlled by the user with one or several parameters and is typically measured by computing the mean squared error (MSE) between the original and the encoded block. For example, in H.263 and MPEG encoders, the distortion at each image block is controlled by a parameter called the block&#39;s quantization step and the MSE distortion is proportional to the square of that parameter. Encoding a block with higher distortion (i.e., lower quality) produces fewer bits for the block. In other words, B i,j  decreases with larger D i,j . The present invention uses the following mathematical model for the relationship between B i,j  and D i,j . 
     Bit-Distortion Model 
     U.S. patent application Ser. No. 09/008,137, filed Jan. 16, 1998, (now U.S. Pat. No. 6,111,991) by J. Ribas-Corbera and S. Lei, and incorporated herein by reference, discloses an analytical study of typical block-based coders, including formulas for the number of bits produced for a block (B i,j ) and the MSE distortion in the block (D i,j ), as a function of several parameters. A combination of these formulas expresses a direct relationship between B i,j  and D i,j :                  B     i   ,   j       =           AK   j     12              α     i   ,   j     2          σ     i   ,   j     2         D     i   ,   j           +     A                   C   j           ,           (   1   )                                
     where A is the number of pixels in an image block (e.g., in MPEG and H.263 A=16 2  pixels), K j  is a parameter that depends on the encoder&#39;s coding efficiency and frame pixels&#39; distribution (K j  will not be needed), C j  is the rate (in bits per pixel) of the motion and syntax in the jth frame, and σ i,j   2  is the variance or energy of the pixels in the block, i.e.,                  σ     i   ,   j     2     =       1   A            ∑     l   =   1     A                       (         P     i   ,   j            (   l   )       -       P   _       i   ,   j         )     2           ,           (   2   )                                
     where A is the number of pixels in a block, P i,j (l) is the value of the l-th pixel in the block, and {overscore (P)} i,j  is the average of the pixel values in the block,                  P   _       i   ,   j       =       1   A            ∑     l   =   1     A                         P     i   ,   j            (   l   )       .                 (   3   )                                
     For color images, the P i,j (l)&#39;s are the values of the luminance and chrominance components at the respective pixels. Finally, α i,j   2  is a distortion weight chosen by the encoder designer (as described in U.S. patent application Ser. No. 09/008,137) and typically takes values between 1 and σ i,j   2 . 
     Equation (1) indicates that the number of bits B i,j  produced when encoding the ith block (in the jth frame) is inversely proportional to the block&#39;s distortion D i,j . 
     The Rate-Distortion Optimization Problem 
     Let N be the number of image blocks in a frame (with A pixels per block). Then, the total number of image blocks in the video sequence is MN, since there are M frames. We define the overall MSE distortion D as the average of the distortions for all the blocks, i.e.,                D   =         1   MN            ∑     j   =   1     M                       ∑     I   =   1     N                     D     i   ,   j             =       1   MN            ∑     j   =   1     M                       ∑     i   =   1     N                         AK   j     12          α     i   ,   j     2            σ     i   ,   j     2       (       B     i   ,   j       -     A                   C   j         )                   ,           (   4   )                                
     using the relationship between B i,j  and D i,j  from (1). Now, let T be the total number of bits available for the M frames and T j  the number of bits for the jth frame. T is the sum of the bits for the frames or, equivalently, the sum of the bits for all the blocks, i.e.,              T   =         ∑     i   =   1     N                     T   j       =       ∑     j   =   1     M                       ∑     i   =   1     N                       B     i   ,   j       .                   (   5   )                                
     Our objective is to find the values of the optimal target number of bits for each frame in the sequence, T 1 *, T 2 *, . . . , T M *, that minimize the distortion in (4) subject to the bit budget constraint in (5), i.e.,                  T   i   *     ,     T   2   *     ,   …              ,       T   M   *     =       arg                 min           T   1     ,     T   2     ,   …              ,     T   M         T   =       ∑     j   =   1     M                     T   j                      
          D   =         arg                 min           T   1     ,     T   2     ,   …              ,     T   M         T   =         ∑     j   =   1     M                     T   j       =       ∑     j   =   1     M                       ∑     i   =   1     N                     B     i   ,   j                          ∑     j   =   1     M                       ∑     i   =   1     N                         AK   j     12                α     i   ,   j     2          σ     i   ,   j     2         (       B     i   ,   j       -     A                   C   j         )       .                       (   9   )                                
     Since Equation (6) involves the minimization of a convex function on a convex set, a unique solution to Equation (6) may be found using the method of Lagrange multipliers:                T   i   *     ,     T   2   *     ,   …   ,                  T   M   *     =                    argmin       T   1     ,     T   2     ,   …              ,     T   M                ∑     j   =   1     M                       ∑     i   =   1     N                         AK   j     12          α     i   ,   j     2            σ     i   ,   j     2       (       B     i   ,   j       -     A                   C   j         )               +     λ        (     T   -       ∑     j   =   1     M                       ∑     i   =   1     N                     B     i   ,   j             )           ,           (   7   )                                
     where λ is the Lagrange multiplier. We solve the minimization problem in Equation (6) by setting partial derivatives of Equation (7) to zero to obtain the following expression for the optimum target number of bits for a frame:                  T   j   *     =           S   j       S   ^              (     T   -     AN          ∑     m   =   1     M                     C   m           )     M       +     ANC   j         ,           (   8   )                                
     where S j  is the following weighted sum of the standard deviations in the jth frame,                  S   j     =       ∑     i   =   1     N                       α     i   ,   j            σ     i   ,   j             ,           (   9   )                                
     which can be interpreted as the energy in the jth frame, and Ŝ is the average of all the frame energies,                S   ^     =       1   M            ∑     i   =   1     M                       S   j     .                 (   10   )                                
     The formula in Equation (8) is the basis in the invention for assigning a target number of bits to frames of a video sequence. Equation (8) makes sense intuitively. If all the frames have approximately the same energy and motion-syntax overhead, i.e., 
     
       
           S   1   ≈S   2   ≈ . . . ≈S   j   ≈ . . . ≈S   M , 
       
     
     
       
           C   1   ≈C   2   ≈ . . . ≈C   j   ≈ . . . ≈C   M , 
       
     
     then T j * in Equation (8) is approximately T/M, and hence each frame is assigned the same number of bits (recall that T and M are the total number of bits and frames, respectively, for the video sequence). If different frames have different energy and motion activity, those with higher energy and motion (i.e., larger S j  and C j ) will be assigned more bits. 
     In order to compute the value of Ŝ in Equation (10), the energy in each frame of the M frame-sequence must be computed. This computation would introduce large delays in the encoding process, since it would require the encoder to wait for M frames before encoding the first one (Ŝ would also be inaccurate without recursion, as frame energy for a frame depends on the target bit rate used for predictive frames). These drawbacks are often not acceptable; preferably then, the encoder estimates Ŝ with an averaging technique (described in the next section). Similarly, the sum of the C j &#39;s in Equation (8) cannot be known in advance and will also preferably be estimated. 
     Frame Target Selection 
     Practical implementations of Equation (8) can take a variety of forms. For example, in MPEG2-type applications Equation (8) could be used for selecting the bit targets in groups of frames (i.e., patterns of I, P, and B frames). In the following embodiment, low-delay type applications, where all frames are predicted from the previous ones, and information for future frames is not available, are considered. The goal in this type of application is to select an adequate target number of bits for the current frame to be encoded. 
     Based on Equation (8), the following low-delay formula can be used for assigning a target number of bits T 0  for the current frame:                T   o     =     {                 S     S   ^            (     B   -     V   ^       )       +   V     ;           B   &gt;     V   ^                     S     S   ^          B     ;         otherwise                   (   11   )                                
     The parameters B, S, {circumflex over (V)} and V in Equation (11) correspond to T/M, S j , ANΣ m C m /M, and ANC j  in Equation (8), respectively. The formula in Equation (11) for B&gt;{circumflex over (V)} is the same as that in Equation (8). However, {circumflex over (V)} is estimated and may be inaccurate at times—when B&lt;{circumflex over (V)}, the estimate for {circumflex over (V)} is too large. In that case, a second estimate S{circumflex over (V)}/Ŝ=V reduces Equation (8) to the simple formula at the bottom of Equation (11). The values of the parameters in Equation (11) can be computed as follows: 
     B—Average Number of Hits Per Frame 
     B is the number of bits accepted by the communication channel during a frame interval. Specifically:                B   =     R   F       ,           (   12   )                                
     where R is the channel or target bit rate (in bits per second) and F is the frame rate of the video source (in frames per second). For example, 10 encoded frames per second of digital video at 64 Kilobits per second results in B=6400 bits. 
     S—Energy of the Current Frame 
     In this example a frame to be encoded is decomposed into N blocks of a fixed size, e.g., 16×16 pixels per block. S is the sum of the empirical standard deviations of the pixels in the blocks and can be obtained as described in Equation (9). Preferably, α i,j  is set to σ i,j , resulting in the simplification          S   j     =       ∑     i   =   1     N                       σ     i   ,   j     2     .                              
     Ŝ—Average Frame Energy 
     Ŝ is the average energy of the frames in the video sequence as defined in Equation (10). But, as explained earlier, in practice it is often impractical to look ahead to find the value of the energies of future frames and hence the value of Ŝ needs to be estimated. At the first frame, Ŝ=S can be used as the estimate. In the next frames, the value of Ŝ can be updated using a weighted average, e.g., a first-order filtered version of S: 
     
       
           Ŝ=aŜ+ (1− a ) S,   (13) 
       
     
     where the weight parameter a is a function of the desired maximum delay d (in seconds) introduced by the encoder buffer; for example:              a   =     {               0.9   0.5        d     ;           d   &lt;   0.5               0.9   ;         otherwise                   (   14   )                                
     In this example, when the delay d is below 0.5 seconds, the value of a decreases linearly with d. The formula for a is plotted in FIG.  4 . 
     Typically, the buffer delay d is directly proportional to the size (in bits) of the encoder buffer, Buff, defined herein as Buff=d R. 
     V—Motion and Syntax Bit Overhead 
     V is the number of bits spent in motion and syntax for the current frame. This number can be directly measured in some encoders, or can be estimated by the number of motion and syntax bits spent in the previous frame. 
     {circumflex over (V)}—Average Motion and Syntax Bit Overhead 
     {circumflex over (V)} is the average number of bits per frame used for motion vectors and codec syntax in the video sequence,                  V   ^     =       1   M            ∑     m   =   1     M                     V   m           ,           (   15   )                                
     with M the total number of frames and V m  the number of motion and syntax bits used for the m-th frame. However, as in the case of Ŝ, in practice it is often impractical to look ahead to find the value of future V m &#39;s—hence, the value of {circumflex over (V)} needs to be estimated. At the first frame, {circumflex over (V)}=V can be used as the estimate. In succeeding frames, the value of {circumflex over (V)} can be updated using the same type of weighted average used for Ŝ in Equation (13): 
     
       
           {circumflex over (V)}=a{circumflex over (V)}+ (1− a ) V,   (16) 
       
     
     where the weight parameter a is a function of the desired buffer delay d (in seconds) in the communication system, as described above. 
     Frame Target Selection for Very Low Delay 
     The frame target selection system described above reduces the range of values taken by T 0  when the user desires lower communication delay. This is because the value of a becomes smaller for lower delay, and hence the values of the estimates Ŝ Equation (13), and {circumflex over (V)}, Equation (15) become closer to S and V, respectively. Actually, when the buffer delay d is close to 0, a is also close to 0, and as a result Ŝ≈S and {circumflex over (V)}≈V. Consequently, with very low d the frame target Equation (11) is approximately constant and equal to the average channel bits per frame,                    T   o     ≈   B     =     R   F       ,           (   17   )                                
     which is the best strategy for frame-layer rate control at very low delay. This is because the number of bits per frame matches those taken by the channel and hence few or no bits are accumulated in the buffer. As a result, the buffer size and delay can be very small. 
     Buffer Protection 
     The objective of a buffer protection subsystem is to maintain the number of bits in the buffer close to a desired level. If the buffer fullness is too high or low, the buffer may overflow or underflow, respectively. Also, even if a target number of bits would not overflow or underflow the buffer, typical block-layer rate control schemes do not achieve the target exactly; thus a danger of overflow or underflow still exists. Two separate buffer protection levels, disclosed below, can help protect the buffer from such problems. These levels can be used together (for maximum protection) or independently. 
     Level 1 
     In this level, a factor Δ, which depends on the fullness of the encoder buffer, corrects the target number of bits per frame. Specifically, Δ adds to the initial frame target T 0 , forming the target T 1 , with level-1 protection, 
     
       
           T   1   =T   0 +Δ,  (18) 
       
     
     where              Δ   =     {               -     W   F       ;           W   &gt;   G                   G   -   W       F   ′       ;         otherwise         ,               (   19   )                                
     and where: 
     W—Current number of bits in the encoder buffer 
     F—Frame rate of the video sequence in frames per second (e.g., 30 fps) 
     G—Desired number of bits in the encoder buffer. 
     The goal of Equation (19) is to decrease the frame target if the current number of bits in the buffer, W, is above the goal G, or to decrease it otherwise. The value of G can be selected in terms of the buffer delay d and the buffer size Buff as follows:              G   =     {               0.5                 Buff     ;           d   &gt;   0.5                 d                 Buff     ;           0.1   &lt;   d   ≤   0.5                 0.1                 Buff     ;           d   ≤   0.1           .               (   20   )                                
     The formula for G indicates that for delays larger than 0.5 seconds, the rate control should aim to keep the buffer fullness at 50 percent, but at lower delays the fullness should be lower (down to 10 percent). This is because when the delay is very small, the buffer size is also very small (recall that Buff=d R), and keeping the buffer fullness to 50 percent may easily produce buffer overflow. The formula for G in Equation (20) is illustrated in FIG.  5 . 
     F′—A parameter that depends on the frame rate F and the delay d, as follows:                F   ′     =     {             F   ;           d   &gt;   0.5                       F   -   1     0.4          (     d   -   0.1     )       +   1     ;           0.1   &lt;   d   ≤   0.5               1   ;           d   ≤   0.1           .               (   21   )                                
     The value of F′ is set using Equation (21), which is illustrated in FIG. 6, throughout the video sequence. The rationale in Equation (21) is that when the buffer delay d (or buffer size) is small, since G in Equation (20) is also small, the buffer fullness will be low and there is danger of buffer underflow. To prevent underflow, F′ can be made smaller for lower d, which in turn will make the target increase Δ in Equation (19) larger. 
     Level 2 
     The buffer protection at this level is also done before encoding the current frame. This level assumes that T 1  bits will be spent for encoding that frame and checks whether that may produce buffer overflow or underflow. To do this, the encoder computes the number of bits expected to be in the buffer after encoding the current frame as follows: 
     
       
           W ′=max( W+T   1   −B,  0),  (22) 
       
     
     where W is the current number of bits in the buffer, T 1  is the target number of bits for the current frame (with level-1 protection), and B is the number of bits accepted by the communication channel per frame interval. If the current frame occupies T 1  bits, the number of bits or fullness of the buffer after encoding this frame will be the current fullness W plus T 1  and minus the B bits sent through the channel. If B is larger than W+T 1  there is buffer underflow, because the channel accepts more bits than those available in the buffer. In this situation, the max operator in Equation (22) sets the content of the buffer to zero. 
     To prevent buffer overflow, H Buff is selected to be the desired maximum number of bits in the encoder buffer (the value of H is typically between 0.85 and 0.95—hence, H Buff is 85 to 95 percent of the buffer size). If the expected number of bits in the buffer W′, Equation (22) is larger than the desired maximum H Buff, the buffer is almost full and the frame target is decreased to prevent buffer overflow. Conversely, if W′ is zero, there will be buffer underflow and the target is increased. This process can be described as follows: 
     
       
         if ( W′&gt;H  Buff)  T   2   =H  Buff− W+B;   
       
     
     
       
         else if ( W′= 0)  T   2   =−W+B;   
       
     
     
       
         else  T   2   =T   1 .  (23) 
       
     
     where T 2  is the new target number of bits per frame (with level-2 protection) and H is the buffer threshold that can be set to a fixed value, or found using either of the two formulas below:                  a   )                   H     =     {         0.95         R   &gt;   112000                   0.1   0.102        R     +     85.7   102             10000   &lt;   R   ≤   112000             0.85         R   ≤   10000                     (   24   )                     b   )                   H     =     1   -     p   Fd         ,           (   25   )                                
     where R, F, and d are the bit rate, frame rate, and delay, respectively, and p is a number between 0 and 1 (e.g., 0.1). 
     The rationale behind Equations (24) and (25) is that the buffer threshold H can be larger with larger buffer size Buff, where recall that Buff=R d. H increases with larger R in Equation (24) and larger d in Equation (25). Equation (24) was designed using empirical experiments. Equation (25) was derived by isolating the value of H from the following constraint: 
     
       
         Buff− H  Buff= p R/F,   
       
     
     
       
         (1− H ) R d=p R/F,   (26) 
       
     
     in which Buff−H Buff is the safety number of bits kept above the desired maximum buffer fullness H Buff, and p R/F is a percentage of the average number of bits per frame. For example, if p=0.1, Equation (25) would select the value of H that would keep 10 percent of the average bits per frame as the safety margin. 
     Encoder Block Diagram 
     Frame target selection and buffer protection as described above can be implemented in conjunction with any image or video encoder. For generality, FIG. 7 shows a video encoder  60  that includes an image encoder  62 , a motion encoder  56 , and a buffer  68 . The motion compensator  24  and summer  22  of FIG. 1 are not shown to aid clarity, but such would typically exist in encoder  60 . Neither motion compensation nor buffering need be present in a specific implementation in order for the invention to be used. 
     In operation, motion-compensated frames are routed to energy estimator  70 . Energy estimator  70  implements, e.g., Equation (9) and routes its output S j  both to frame-layer rate controller  66  and average energy estimator  72 . Average energy estimator implements, e.g., Equations (13) and (14), and routes its output Ŝ to frame-layer rate controller  66 . In a basic system, frame-layer rate controller  66  ratios these values and multiplies the result by the desired average bits per frame B to produce target bit rate T j . Image encoder  62  then uses T j  to adjust its compression to the desired target. 
     In a system employing motion compensation (and more generally, prediction), prediction bit totals can be supplied to frame-layer rate controller  66 . Motion encoder  56  sends its count of motion bits for the current frame, C j , both to frame-layer rate controller  66  and average motion bits estimator  58 . Average motion bits estimator  58 , e.g., implementing Equations (15) and (16), also sends its output Ĉ to frame-layer rate controller  66 . Frame-layer rate controller uses these additional variables, e.g., to form an estimate for T j  as described in equation (11). Note that T j  includes C j ; image encoder  62  must comprehend that these C j  bits are not available to it for coding the motion-compensated image. 
     In a system employing buffer protection, buffer  68  can supply feedback on its current fullness level W to frame-layer rate controller  66 . Controller  66  can use, e.g., Level 1 and/or Level 2 buffer protection to adjust T j . 
     Particular estimates in each estimation block of encoder  60  will generally be application-dependent, although the delay adaptive implementations described herein are preferred. 
     Experimental Results 
     The frame target selection and buffer protection of the present invention were implemented in the frame layer of a rate control technique that encoded frames of real video sequences using an MPEG∝codec. This codec also used the block and data selection layers as described in co-pending U.S. patent application Ser. No. 09/008,137 (now U.S. Pat. No. 6,111,991) incorporated above, and Ser. No. 09/052,882, filed Mar. 31, 1998, by J. Ribas-Corbera and S. Lei, also incorporated herein by reference. The methods described in these two applications were used as block and data selection layers, respectively. The frame layer included the frame target selection system and Level 2 buffer protection (Level 1 buffer protection was also implemented and tested in a different codec). Equation (24) was used to select the value of H. The performance of this method (referred to as SLA rate control) was compared to VM7 rate control, which is a well-known rate control method from Sarnoff Corporation [13]. 
     Table 1 describes the name of the video sequences, format, bit rates and frame rates used in each experiment, and assigns a name to each experiment. All video sequences contained 300 frames and were downsampled at the encoding frame rate specified in the experiments. Overall, 1400 frames were encoded. In the tests, the fullness of the encoder buffer was simulated. Rate control was tested with two different buffers: 
     Buffer 1: a buffer of size 0.125 times the bit rate of each sequence, Buff=0.125*R, which corresponds to a buffer delay of d=0.125 seconds. This buffer size is often used for low delay video communications. 
     Buffer 2: a buffer of size 0.05 times the bit rate, Buff=0.050*R, which corresponds to a delay d=0.050. This is a very small buffer size that is not typically used in practice, but was selected here in order to explore the performance of the SLA rate control method in such a challenging case. 
     Tables 2 and 3 show the percentage of times that the buffers overflow and underflow, respectively, with SLA and VM7 rate control. Buffer overflow is an important problem that forces the encoder to slow down or drop image blocks—preferably, overflow should rarely occur. On the other hand, buffer underflow is less critical, but should also be avoided if possible since it wastes channel bandwidth. 
     Table 2 shows the percentage of times that the buffer overflows (with respect to the number of frames encoded) when Buffer 1 and Buffer 2 are used with SLA and VM7 rate control. For Buffer 1, SLA rate control controls the bit rate so well that the buffer practically never overflows. In fact, overflow occurred only once in “fmn112_c” (and only a few bits overflowed), which would have barely affected the encoder. For the challenging Buffer 2, the SLA rate control method suffers overflow in only 4 of the 1400 encoded frames—again, only a few bits overflowed in each case. This is particularly impressive, since it is very difficult to control the bit rate fluctuations within such a small buffer. In order to appreciate the benefits of the SLA procedure, observe that overflow is much more common in both buffers with VM7 rate control. Further, with VM7 it was found that the number of bits that overflow is sometimes as large as twice the size of the buffer. 
     Table 3 shows the percentages of buffer underflow. Observe that SLA rate control underflows the buffers occasionally, but much less often than VM7 rate control. In particular, VM7 rate control underflows the buffer above 80 percent of the time in most of the experiments for Buffer 2. 
     Overall, SLA rate control is very robust for controlling the bit rate with small buffer sizes and overflows and underflows the buffer much less often than VM7&#39;s. 
     
       
         
               
             
               
               
               
               
               
             
               
               
               
               
               
             
           
               
                 TABLE 1 
               
             
             
               
                   
               
               
                 Description of the video sequences, formats, bit rates and frame 
               
               
                 rates used in the experiments. Each video sequence contained 300 
               
               
                 frames at 30 frames per second and was downsampled at the encoding 
               
               
                 frame rate. The left-most column shows the name given to each of 
               
               
                 the experiments. 
               
             
          
           
               
                   
                 Video Seq. 
                   
                 Bit Rate 
                 Frames/ 
               
               
                 Experiment 
                 Name 
                 Format 
                 R (Kbps) 
                 Second 
               
               
                   
               
             
          
           
               
                 coast112_c 
                 “coastguard” 
                 CIF 
                 112 
                 15 
               
               
                 coast112 
                 “coastguard” 
                 QCIF 
                 112 
                 10 
               
               
                 coast48 
                 “coastguard” 
                 QCIF 
                 48 
                 10 
               
               
                 cont24 
                 “container” 
                 QCIF 
                 24 
                 10 
               
               
                 fmn112_c 
                 “foreman” 
                 CIF 
                 112 
                 15 
               
               
                 fmn112 
                 “foreman” 
                 QCIF 
                 112 
                 10 
               
               
                 fmn48 
                 “foreman” 
                 QCIF 
                 48 
                 10 
               
               
                 mad10 
                 “mother &amp;daughter” 
                 QCIF 
                 10 
                 7.5 
               
               
                 mad24 
                 “mother &amp;daughter” 
                 QCIF 
                 24 
                 10 
               
               
                 news112_c 
                 “news” 
                 CIF 
                 112 
                 15 
               
               
                 news112 
                 “news” 
                 QCIF 
                 112 
                 10 
               
               
                 news48_c 
                 “news” 
                 CIF 
                 48 
                 7.5 
               
               
                 sil24 
                 “silent” 
                 QCIF 
                 24 
                 10 
               
               
                   
               
             
          
         
       
     
     
       
         
               
             
               
               
               
             
               
               
               
               
               
             
               
               
               
               
               
             
               
               
               
               
               
             
           
               
                 TABLE 2 
               
             
             
               
                   
               
               
                 Percentage of times that buffer overflow occurs in an MPEG4 codec 
               
               
                 using SLA rate control (SLA) and VM7&#39;s (VM7) [13]. 
               
             
          
           
               
                   
                 Percentage of Buffer Overflow 
                   
               
             
          
           
               
                   
                 Buffer 1 
                   
                 Buffer 2 
                   
               
             
          
           
               
                 Name 
                 SLA (%) 
                 VM7 (%) 
                 SLA (%) 
                 VM7 (%) 
               
               
                   
               
             
          
           
               
                 coast112_c 
                 0 
                 6.6 
                 0 
                 7.3 
               
               
                 coast112 
                 0 
                 0 
                 0 
                 0 
               
               
                 coast48 
                 0 
                 0 
                 0 
                 0 
               
               
                 cont24 
                 0 
                 1.0 
                 1.0 
                 0 
               
               
                 fmn112_c 
                 0.6 
                 16.0 
                 0.6 
                 24.0 
               
               
                 fmn112 
                 0 
                 0 
                 1.0 
                 0 
               
               
                 fmn48 
                 0 
                 0 
                 0 
                 5.3 
               
               
                 mad10 
                 0 
                 12.0 
                 0 
                 14.6 
               
               
                 mad24 
                 0 
                 0 
                 0 
                 0 
               
               
                 news112_c 
                 0 
                 0 
                 0 
                 0 
               
               
                 news112 
                 0 
                 0 
                 0 
                 0 
               
               
                 news48_c 
                 0 
                 0 
                 0 
                 2.6 
               
               
                 sil24 
                 0 
                 1.0 
                 0 
                 0 
               
               
                   
               
             
          
         
       
     
     
       
         
               
             
               
               
               
             
               
               
               
               
               
             
               
               
               
               
               
             
               
               
               
               
               
             
           
               
                 TABLE 3 
               
             
             
               
                   
               
               
                 Percentage of times that buffer underflow occurs in an MPEG4 
               
               
                 codec using SLA rate control (SLA) and VM7&#39;s (VM7) [13]. 
               
             
          
           
               
                   
                 Percentage of Buffer Underflow 
                   
               
             
          
           
               
                   
                 Buffer 1 
                   
                 Buffer 2 
                   
               
             
          
           
               
                 Name 
                 SLA (%) 
                 VM7 (%) 
                 SLA (%) 
                 VM7 (%) 
               
               
                   
               
             
          
           
               
                 coast112_c 
                 0 
                 0 
                 0 
                 55.3 
               
               
                 coast112 
                 0 
                 14.0 
                 0 
                 99.0 
               
               
                 coast48 
                 0 
                 14.0 
                 0 
                 99.0 
               
               
                 cont24 
                 0 
                 20.0 
                 1.0 
                 98.0 
               
               
                 fmn112_c 
                 0 
                 0 
                 0 
                 46.0 
               
               
                 fmn112 
                 0 
                 20.0 
                 0 
                 99.0 
               
               
                 fmn48 
                 0 
                 20.0 
                 0 
                 89.0 
               
               
                 mad10 
                 0 
                 37.3 
                 0 
                 57.3 
               
               
                 mad24 
                 0 
                 41.3 
                 3.0 
                 96.0 
               
               
                 news112_c 
                 0 
                 0 
                 0 
                 93.3 
               
               
                 news112 
                 0 
                 26.0 
                 0 
                 99.0 
               
               
                 news48_c 
                 0 
                 68.0 
                 0 
                 85.3 
               
               
                 sil24 
                 0 
                 19.0 
                 9.0 
                 87.0 
               
               
                   
               
             
          
         
       
     
     Alternative Implementations 
     Several variations on the basic invention are described below. 
     One-pass 
     If the computation of the σ k &#39;s used for finding S j  in Equation (9) cannot be done in advance, a good estimate for Sj such as the value of S from the previous video frame may be substituted. 
     Low-complexity Estimation of σ i    
     In order to further reduce computational complexity, the formula for the standard deviation in Equation (2) may be approximated as            σ     i   ,   j       =       1   A            ∑     l   =   1     A                     abs        (         P     i   ,   j            (   l   )       -       P   _       i   ,   j         )             ,                          
     where abs(x) is the absolute value of x. In video coding, the mean value of pixels in motion-compensated blocks is usually zero and hence this formula may be further simplified by setting {overscore (P)} i,j =0. 
     Encoding Intra and Inter Blocks 
     If some of the blocks to be encoded are of class intra and some are inter, as described in U.S. patent application Ser. No. 09/008,137, the performance of the frame target selection method can be improved by dividing the standard deviation of the intra blocks by a factor {square root over (γ)}. Specifically, after computing the value for the σ k &#39;s in Step 1, set:          σ   k     =     {               σ   k       γ             if                 k                 th                 block                 is                 intra               σ   k         otherwise         .                              
     The factor γ is          γ   =       K   P       K   I         ,                          
     where K I  and K P  are the averages of the K&#39;s from Equation (1) measured for the intra and inter blocks, respectively. The value of γ can be estimated and updated during encoding. In practice, a constant γ=3 works quite well. 
     Encoding a Group of Pictures (GOP) 
     In MPEG and H.263+codecs, there are different types of frames: 
     I frames, whose pixel values are encoded by themselves (or intracoded) without using a prediction from other frames in the video sequence. 
     P frames, whose pixels are predicted from a previous frame in the video sequence. 
     B frames, whose pixels are bi-directionally predicted using previous and future frames in the sequence. 
     A simple variation of equation (8) can be used to assign a target number of bits for each type of frame. For example, consider the typical case where the sequence or pattern of frame types is: 
     
       
           I,B, . . . ,B,P,B, . . . ,B,P,B, . . . ,B,P,B, . . . ,B,P, . . .    (27) 
       
     
     The set of frames “B, . . . ,B,P” is repeated periodically after the first I frame. In MPEG, such a set (or any other that repeats periodically) is called a group of pictures or GOP. 
     GOP Type B, . . . ,B,P 
     Let N B  be the number of B frames in the GOP above. The target number of bits for the P picture in that GOP, T P , and the target for each of the B frames, T B , can be computed from Equation (8) as follows:                  T   P     =           S   P       S   ^              (     T   -     ANMC   PB       )         N   B     +   1         +     ANC   P         ,           (   28   )                   T   B     =           S   B       S   ^              (     T   -     ANMC   PB       )         N   B     +   1         +     ANC   B         ,           (   29   )                                
     where the parameters in Equations (28) and (29) are equivalent to those in Equation (8). Specifically: 
     S P  is the energy, Equation (9), for the P frame in the current GOP, which can either be measured or estimated as the average of the energies of P frames in previous GOP&#39;s. 
     S B  is the average of the energies, Equation (9), for the B frames in the current GOP, which again can either be measured or estimated as the average of B frame energy in previous GOP&#39;s. 
     S is the (estimated) average of the energy in the GOP, i.e.,          S   ^     =           S   P     +       N   B          S   B           1   +     N   B         .                            
     C P  and C B  are the motion and syntax rate (in bits per pixel) for the P and B frames, respectively, as defined in Equation (1), and they can also either be measured or estimated. C PB  is their average in the GOP, i.e.,          C   PB     =           C   P     +       N   B          C   B           1   +     N   B         .                            
     Finally, T, M, and N are the total number of bits (for the GOP), frames (in the GOP), and macroblocks in a frame. 
     Equations (28) and (29) may be rearranged in an implementation. For example, after some lengthy but straightforward manipulations, equations (28) and (29) can be rewritten as follows:                  T   B     =       T   -     AN        (       C   P     -     β                   C   B         )           β   +     N   B           ,           (   30   )                                T   P   =T−N   B   T   B ,  (31) 
     where        β   =         S   P       S   B       .                            
     Alternative Formulas for Estimating β 
     The value off β is the ratio S P /S B . In Equations (30) and (31), β is a key parameter that has strong influence on how many bits are assigned to P and B frames. For example, if β is close to 1, the number of bits for each frame in the GOP is similar, i.e., T P ≈T B , and if β is larger than 1, then T P  is also larger than T B . Since β is a key parameter, its value must be estimated carefully. Two methods for estimating β are presented here: 
     Method 1: 
     Set β equal to S P /S B , where S P  is the average of the energies in the M previously encoded P frames and S B  is the average of the energies in the N previously encoded B frames. If available, the energy of the frame that is being encoded could be used as well in the respective average. 
     Method 2: 
     After a given frame is encoded, update the value of β as follows:                β   =       b                   β   prev       +       (     1   -   b     )        F          S   P       S   B             ,           (   32   )                                
     where here β prev  is the previous value of β, and S P  and S B  are the energies of the last encoded P and B frames, respectively. F and are b filter or weight parameters. Typically F takes values between 1 and 5, and b between 0.7 and 0.9. 
     When F is 1, experiments show that the image quality (in terms of MSE) is similar for P and B frames. However, in some cases, the average image quality can be increased by allowing some frames to have better quality than others. For example, increasing F would result in a larger β, which in turn would increase both T P  and the quality of the P frame. This would appear to reduce the quality of the B frames, since T B  would be smaller. But since the P frame has higher quality and is used to predict the B&#39;s by motion compensation, the quality of the B frames could also increase. Using empirical tests, the value of F that optimizes MSE performance decreases with bit rate. This effect can be modeled with the following formula:                F   =       c     Bpp                ψ         +   d       ,           (   33   )                                
     where Bpp is the rate in bits per pixel for the given video sequence, and c, d, and ψ are constant parameters whose value can be set for a given video codec. For example, in an H.263+ encoder, we empirically found that c=1.35, d=−0.4, and ψ=0.5, are good choices. 
     We implemented a frame layer rate control based on Equations (30), (31), (32) and (33), and Thomson&#39;s frame layer from [14] in a video encoder compliant to the H.263+ standard. The method in [14] is known as TM5 rate control and is a very popular technique used as a benchmark for comparisons in the rate control literature. For a fair comparison, the rate control at the block layer was the same for both cases. We turned on the advanced prediction, extended motion range, and true B frame options (annexes D, F, and O) in the H.263+ encoder, and selected GOPs with either one (N B =1) or two (N B =2) B frames. The tests are described in Table 4 and the results are in Table 5. FIG. 8 plots the peak signal-to-noise ratio or PSNR per video frame for one of the tests. The PSNR is the typical measure of image quality used in the image and video compression literature and is given in dB&#39;s, and a PSNR improvement of 1 dB corresponds to bit rate savings of about 20 percent. Observe that our new frame layer improves the average PSNR up to 1.38 dB with respect to TM5&#39;s. Our superior performance is because our frame target formulas are based on a rate-distortion optimization. 
     
       
         
               
             
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 4 
               
             
             
               
                   
               
               
                 Description of the experiments: names assigned to each experiment, 
               
               
                 original video sequences, GOP pattern of frames, and target bit rates. 
               
               
                 The spatial resolution of the video frames was QCIF and the frame rate 
               
               
                 was of 15 frames per second. The bit rate achieved was very close to 
               
               
                 the target in all cases. A total of five seconds were encoded for each 
               
               
                 video sequence. 
               
             
          
           
               
                   
                   
                 GOP pattern of 
                 Bit Rate 
               
               
                 Test Name 
                 Video Sequence 
                 Frames 
                 (Kbps) 
               
               
                   
               
             
          
           
               
                 Fmn112_a 
                 “foreman” 
                 IBPBP . . . 
                 112 
               
               
                 Fmn112_b 
                 “foreman” 
                 IBBPBBP . . . 
                 112 
               
               
                 Fmn64 
                 “foreman” 
                 IBPBP . . . 
                 64 
               
               
                 Hall24 
                 “hall” 
                 IBPBP . . . 
                 24 
               
               
                 Hall48 
                 “hall” 
                 IBPBP . . . 
                 48 
               
               
                 Hall64_a 
                 “hall” 
                 IBPBP . . . 
                 64 
               
               
                 Hall64_b 
                 “hall” 
                 IBBPBBP . . . 
                 64 
               
               
                 mad24 
                 “mother &amp; daughter” 
                 IBPBP . . . 
                 24 
               
               
                 mad48 
                 “mother &amp; daughter” 
                 IBPBP . . . 
                 48 
               
               
                 mad64_a 
                 “mother &amp; daughter” 
                 IBPBP . . . 
                 64 
               
               
                 mad64_b 
                 “mother &amp; daughter” 
                 IBBPBBP . . . 
                 64 
               
               
                 Sil112_a 
                 “silent” 
                 IBPBP . . . 
                 112 
               
               
                 Sil112_b 
                 “silent” 
                 IBBPBBP . . . 
                 112 
               
               
                 Sil48 
                 “silent” 
                 IBPBP . . . 
                 48 
               
               
                   
               
             
          
         
       
     
     
       
         
               
               
               
             
               
               
               
               
               
               
               
               
             
           
               
                   
                 TABLE 5 
               
             
             
               
                   
                   
               
               
                   
                 TM5 Frame Layer 
                 New Frame Layer 
               
             
          
           
               
                 Test 
                 PSNR P 
                 PSNR B 
                 PSNR 
                 PSRN P 
                 PSNR B 
                 PSNR 
                 GAIN 
               
               
                   
               
               
                 Fmn112 a 
                 35.44 
                 34.63 
                 35.04 
                 35.50 
                 34.58 
                 35.04 
                 +0.00 
               
               
                 Fmn112 b 
                 35.41 
                 34.62 
                 34.88 
                 35.62 
                 34.60 
                 34.94 
                 +0.06 
               
               
                 Fmn64 
                 32.94 
                 32.35 
                 32.65 
                 33.02 
                 32.34 
                 32.68 
                 +0.03 
               
               
                 Hall24 
                 32.92 
                 32.46 
                 32.69 
                 33.16 
                 32.64 
                 32.90 
                 +0.21 
               
               
                 Hall48 
                 36.01 
                 35.40 
                 35.70 
                 36.64 
                 35.37 
                 36.00 
                 +0.30 
               
               
                 Hall64 a 
                 37.13 
                 36.78 
                 36.96 
                 37.99 
                 36.58 
                 37.29 
                 +0.33 
               
               
                 Hall64 b 
                 37.02 
                 36.67 
                 36.79 
                 37.94 
                 36.73 
                 37.14 
                 +0.35 
               
               
                 Mad24 
                 34.69 
                 34.45 
                 34.57 
                 34.69 
                 34.48 
                 34.59 
                 +0.02 
               
               
                 Mad48 
                 37.03 
                 36.87 
                 36.95 
                 37.37 
                 36.95 
                 37.16 
                 +0.21 
               
               
                 Mad64 a 
                 37.72 
                 37.71 
                 37.71 
                 38.57 
                 37.98 
                 38.28 
                 +0.57 
               
               
                 Mad64 b 
                 36.95 
                 37.30 
                 37.19 
                 38.77 
                 37.98 
                 38.24 
                 +1.05 
               
               
                 Sil112 a 
                 36.83 
                 37.07 
                 36.95 
                 38.74 
                 37.93 
                 38.33 
                 +1.38 
               
               
                 Sil112 b 
                 37.35 
                 37.49 
                 37.44 
                 38.67 
                 37.99 
                 38.21 
                 +0.77 
               
               
                 Sil48 
                 33.52 
                 33.05 
                 33.28 
                 33.80 
                 33.19 
                 33.50 
                 +0.22 
               
               
                   
               
               
                 Results of the average PSNR (PSNR) obtained when using our new frame-layer rate control and TM5&#39;s [14] for the P (PSNR P) and B (PSNR B) frames. The last column (GAIN) shows the improvement in average PSNR with our new method.  
               
             
          
         
       
     
     General GOP Type 
     Observe that equations similar to Equations (28) and (29) can be derived from Equation (8) for any GOP frame pattern. For example, consider a GOP where the number of I, P, and B frames is respectively N I , N P , and N B . Then the target number of bits for each frame type is:                  T   I     =           S   I       S   ^              (     T   -     ANMC   IPB       )         N   I     +     N   P     +     N   B           +     ANC   I         ,           (   34   )                   T   P     =           S   P       S   ^              (     T   -     ANMC   IPB       )         N   I     +     N   P     +     N   B           +     ANC   P         ,           (   35   )                   T   B     =           S   B       S   ^              (     T   -     ANMC   IPB       )         N   I     +     N   P     +     N   B           +     ANC   B         ,           (   36   )                                
     which, once again, after some manipulations, can be re-written as follows:                  T   I     =         (     T   -     ANMC   IPB       )         N   I     +       N   P       β   IP       +       N   B         β   IP          β   PB             +     ANC   I         ,           (   37   )                   T   P     =         (     T   -     ANMC   IPB       )           N   I          β   IP       +     N   P     +       N   B       β   PB           +     ANC   P         ,           (   38   )                 T   B     =         (     T   -     ANMC   IPB       )           N   I          β   IP       +       N   P          β   PB       +     N   B         +       ANC   B     .               (   39   )                                
     The parameters in Equations (37), (38), and (39) have the same meaning as in previous formulas. For example, {C I , C P , C B } are the respective C&#39;s for I, P, and B frames, and:                C   IPB     =             N   I          C   I       +       N   P          C   P       +       N   B          C   B             N   I     +     N   P     +     N   B         .             (   40   )                                
     The values of the β&#39;s can be estimated with either Method 1 or 2 above. For example,                  β   IP     =         b   IP          β     IP   ,   prev         +       (     1   -     b   IP       )          F   IP            S   I       S   P             ,           (   41   )                   β   PB     =         b   PB          β     PB   ,   prev         +       (     1   -     b   PB       )          F   PB            S   P       S   B             ,           (   42   )                                
     where {b IP , F IP } and {b PB , F PB } are the respective estimator parameters, as in Equation (32). 
     After reading this disclosure, one of ordinary skill in the art will recognize that many advantageous modifications to the disclosed embodiments are enabled by the concepts taught herein. In particular, estimation of the parameters used in the calculation of a frame target bit allocation is typically a design choice, and practically will be based on encoder structure and block timing. Such obvious modifications are intended to fall within the scope of the invention as claimed.