896

IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 7, JULY 2015

An Efficient Frame-Content Based Intra Frame Rate
Control for High Efficiency Video Coding
Miaohui Wang, Student Member, IEEE, King Ngi Ngan, Fellow, IEEE, and Hongliang Li, Senior Member, IEEE

Abstract—Rate control plays an important role in the rapid
development of high-fidelity video services. As the High Efficiency
Video Coding (HEVC) standard has been finalized, many rate
control algorithms are being developed to promote its commercial
use. The HEVC encoder adopts a new R-lambda based rate
control model to reduce the bit estimation error. However, theR-lambda model fails to consider the frame-content complexitythat ultimately degrades the performance of the bit rate control.
In this letter, a gradient based R-lambda (GRL) model is proposed
for the intra frame rate control, where the gradient can effectively
measure the frame-content complexity and enhance the performance of the traditional R-lambda method. In addition, a newcoding tree unit (CTU) level bit allocation method is developed.
The simulation results show that the proposed GRL method can
reduce the bit estimation error and improve the video quality in
HEVC all intra frame coding.
Index Terms—Frame-content complexity, HEVC, lagrange multiplier, perceptual coding, rate control.

I. INTRODUCTIONW

ITH the rapid development of high-fidelity video services in recent years, how to efficiently compress video
data has been considered by many organizations and companies.
To meet this demand, ITU-T Q.6/SG16 (VCEG) and ISO/IEC
JTC1/SC29/WG11 (MPEG) have established the Joint Collaborative Team on Video Coding (JCT-VC) to develop a new videocoding standard–High Efficiency Video Coding (HEVC) [1]. As
the HEVC has adopted many advanced encoding tools, it cansignificantly improve the compression performance of high definition (HD) videos and save half of the bit rate compared to the
H.264/Advanced Video Coding (AVC) [2] for the same perceptual video quality.
Rate control plays a significant role in transmitting
high-quality video data via the communication channel. If
the channel bandwidth is smaller than the output bit rate of theencoder, the encoded bits will accumulate in the encoder buffer.

Manuscript received May 26, 2014; revised August 18, 2014; accepted
November 25, 2014. Date of publication December 04, 2014; date of current
version December 08, 2014. This work was supported in part by a grant fromthe Research Grants Council of the Hong Kong SAR, China, under Project
CUHK 415712. The associate editor coordinating the review of this manuscript
and approving it for publication was Prof. Marco Mattavelli.
M. Wang and K. N. Ngan are with the Department of Electronic Engineering,
The Chinese University of Hong Kong, Hong Kong (e-mail: mhwang@ee.cuhk.
edu.hk; knngan@ee.cuhk.edu.hk).
H. Li is with School of Electronic Engineering, University of Electronic Science and Technology of China, Chengdu 610051, China (e-mail: hlli@uestc.
edu.cn).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LSP.2014.2377032

When the buffer size of these accumulated bits is too large, theencoder needs to skip some encoding frames to alleviate thebuffer delay and avoid the buffer overflow. On the contrary, if
the channel bandwidth is larger than the output bit rate of theencoder, this means that some channel bandwidth is wasted,
and the encoder buffer may be underflow. Thus, the goal of rate
control is to achieve a balance between the compressed video
quality and the channel bandwidth to meet specific applications.
Intra frame rate control is more important in HEVC thanin H.264/AVC. As the motion compensated prediction (MCP)
is greatly improved in HEVC, the bit rate saving for MCP
frames (i.e., B or P frames) is higher than for the intra-codedframes. Consequently, intra frame bits of the HEVC occupy
more channel bandwidth than those of the H.264/AVC in acompressed video stream. In addition, initial quantization parameter (QP) of the first frame (i.e., intra-coded frame) greatly
affects the channel bandwidth. On the one hand, a high-quality
intra frame needs much more bandwidth than the MCP frames
in a group of pictures (GOP), which may cause the buffer
overflow and frame-skip on the encoder side. On the other
hand, a low-quality intra frame that is used as a reference frame
for the MCP frames degrades the video quality in the current
GOP and may waste the channel bandwidth. Thus, intra frame
rate control algorithms [3]–[6] have been considered in boththe H.264/AVC and HEVC encoders.
In this letter, we propose a new gradient based R-lambda(GRL) model for the HEVC intra frame rate control, where
the gradient is used to measure the frame-content complexity.
In addition, a novel bit allocation method is developed for thecoding tree unit (CTU) rate control. To the best of our knowledge, this is the first experimental study in which the relationship between rate-gradient and lambda is modeled and a newcontent complexity based bit allocation is developed for the
HEVC intra frame rate control. Simulation results show that in
HEVC, the proposed GRL method can significantly improve
the performance of the intra frame rate control compared tostate-of-the-art methods.
The rest of the letter is organized as follows. We briefly
introduce the related works in Section II. The proposed GRL
method is addressed in Section III. The simulation results arepresented in Section IV while the concluding remarks are given
in Section V.
II. RELATED RATE CONTROL MODELS IN HEVC
Recently, many rate control models are being considered in
the HEVC encoder. In this section, we give a brief review ofdifferent rate models to facilitate the study of the rate control
algorithms. In HEVC, rate models can be roughly classified
into three categories: quadratic model, -domain model andR-lambda model.

1070-9908 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

WANG et al.: EFFICIENT FRAME-CONTENT BASED INTRA FRAME RATE CONTROLIn [7], [8], the quadratic method has been extended to theearly version of the HEVC reference software. However, twolimitations of the quadratic model need to be further considered.
Firstly, the quadratic model only determines the encoding bitsof the transform coefficients but ignores the overhead bits. Secondly, the quadratic model contains a well-known “chicken andegg” dilemma [4] in the computation of quantization step.
In [9], the -domain method has been studied in HEVC,
where is the percentage of zeros among quantized coefficients. The experiments in [9] show a little improvement incomparison with the quadratic method on the HEVC reference software. However, it should be noted that the -domain
method is developed for the codec with a fixed transform block
size, which is not compatible with the quad-tree based coding
structure in HEVC, especially when the transform block size
can be varying fromto.
In modern video codecs (i.e., H.264/AVC and HEVC), theLagrange multiplier (i.e., lambda) not only explicitly affects thebits of a block but also implicitly affects the determination of its
encoding modes for rate distortion optimization (RDO) scheme.
Thus, the Lagrange multiplier has been considered as an important factor for the rate control methods [10], [11] in H.264/AVC.
Recently, Li et al. [12] developed a new R-lambda method for
the HEVC codec.
Compared with the quadratic and -domain models, theR-lambda model considers the overall bit rate including boththe transform coefficient bits and the overhead bits. In addition,
parameters in the R-lambda model can be easily computed. Dueto its high efficiency and low complexity, the R-lambda model
has been adopted as the rate control method for the HEVC
codec by the JCT-VC. However, the model does not consider
the frame complexity, which should enhance the performanceof the R-lambda based rate control methods in HEVC. Thus,in this letter, the frame-content complexity is studied andincorporated into an improved R-lambda model.
III. PROPOSED RATE CONTROL METHOD
In this section, we introduce the proposed intra frame rate
control method for the HEVC encoder, including a new framecontent complexity based R-lambda model and a new CTU level
bit allocation method.
A. Modeling the Relationship Between Rate-Gradient andLambda for the HEVC Intra Frame Coding
In general, the output bits of the encoded frames in a video
sequence have varying content from each other. The reason isthat different frames have different encoding complexities. To
reduce the bit estimation error, the frame-content complexity
measure is incorporated into the proposed method for the HEVC
intra frame coding.
The gradient based frame-content complexity measure has
been considered in the H.264/AVC intra frame rate control [4],[5]. It is reported that the gradient per pixel (GPP) of the intracoded frames has an approximated linear relationship with theirencoded bits. The GPP is defined by
(1)

897

Fig. 1. Modeling the rate-gradient and lambda for intra frame coding. Four QPs) are used in the experiments.
(i.e.,

whereand
are the height and width of one CTU or one
is the pixel value at position ( ).
frame, andSince the HEVC utilizes similar encoding tools as the
H.264/AVC in intra frame coding, it is believed that the GPP
can also be used to measure content complexity of the HEVC
intra-coded frames. However, the HEVC employs more flexible partitions, prediction directions and transform sizes thanthe H.264/AVC in intra frame coding. Thus, the relationshipbetween rate-gradient and lambda should be modeled in HEVC
intra frame coding.
Fig. 1 shows the relationship between rate-gradient andlambda in HEVC. In the experiment, the encoder is configured
the same as in Section IV. There are a total of 120 frames
encoded with the All-intra Main configuration. The average
results of BPP (i.e., bits per pixel), GPP and of four QPs (i.e.,) are used to model the relationship between
rate-gradient and lambda. It is observed that in HEVC intra
frame coding, a hyperbolic function can adequately fit the data.
(2)
where
and
are the model parameters. Other video sequences also present similar characteristics. In the proposed
GRL method, equation (2) is used to compute content basedthat is used to compute a corresponding QP.
As mentioned above, the initial QP greatly affects the performance of many rate control algorithms. Given the model parameters, the traditional R-lambda method computes a fixed initial
QP for different video sequences. Obviously, the fixed initial QP
inevitably degrades the performance of the R-lambda method,
because different frames have varying content complexities aswell as varying bits. On the contrary, our method can obtain aframe-content based and the initial QP for different video sequences based on their frame gradients. One can see that our
method can obtain a more accurate initial QP than conventional
R-lambda methods according to the experiments in Section IV.
B. Bit Allocation
A typical rate control algorithm usually consists of two
steps: allocating the target bits and achieving the allocatedtarget budget. As mentioned above, the target bit rate can be898IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 7, JULY 2015

achieved by three rate models in HEVC. Based on the same rate
model, different bit allocation methods can yield quite distinct
results. Consequently, bit allocation is an important componentof a rate control algorithm. The proposed method follows aclassical three-level bit allocation structure: GOP level, framelevel and CTU level.
GOP Level Bit Allocation: Given a target bit rateandthe frame rate , the target bits allocated to the current GOP is

(3)
where
is the target bits for the current GOP,
is the) that have alreadybits of the number of pictures (i.e.,been encoded,
is the number of pictures in the current
GOP, andis the smooth window [13].
Frame Level Bit Allocation: Since in all intra frame coding,, the target bits allocated to the current picture isFig. 2. Comparison of the updated between the JCT-VC M0257 [14] andthe proposed method. In the experiments, the target bit rate is obtained by the(left) and(right), respectively.
anchor HEVC encoder at

C. Model Parameter Update
After encoding one CTU or one frame, we use the actual en,and.
coded bits to update the values of(4)
CTU Level Bit Allocation: When the frame levelis allocated, the proposed CTU level bit allocation method isbased on the GPP of each CTU in the current frame.
(5)is the th CTU gradient,is the allocated bits
whereis the number of CTUs in the curfor the th CTU, andrent frame.
The CTU level bit allocation is updated as(8)
is the actual bits of an encoded CTU or frame, andwhere
is a scaling factor that reduces the speed of changes, and in
our experiments it is set to 0.125. Note that the larger the scaling
factor is, the larger variation of is obtained.
Fig. 2 shows four results of the updated versus the frame
number in both the proposed method and the JCT-VC M0257[14] that is proposed for the intra frame rate control in HEVC.
It can be seen that our method can attain a smaller fluctuationthat indicates a smaller QP fluctuation and a more stable buffer
status in comparison with the JCT-VC M0257.
D. Summary of the Proposed Algorithm

(6)
where
is the allocated bits for the current (i.e., th)CTU,
is the remaining bits used to encode the rest ofis the actual bits of the th
CTUs in the current frame,
CTU andis the weight for the current CTU definedby(7)
In traditional R-lambda methods, bit-budget of a CTU is allocated according to the mean absolute difference (MAD) of the
CTU at the same position in the previous decoded frame. Obviously, MAD of the previous CTU may be quite different fromthe current one, and hence the MAD value used as a weight performs poorly in allocating the bit-budget. Instead of the MAD
weight, we propose to use the gradient of the current CTU toallocate the bit-budget, because the gradient can be obtained directly from the current CTU, and it has an approximated linear
relationship to the bit rate in intra frame coding. In the experiments (see Section IV), one can see that the gradient method canachieve a more accurate bit estimation than the MAD method.

The proposed GRL algorithm is as follows.
1) Bit allocation according to equations (3) (7);computation for each CTU according to equation (2);2)3) QP computation for each CTU by, where and are two constants;
4) Encoding and recording related encoding parameters;5) Parameter update according to equation (8);6) Going to Step 1 until the end of encoding.
IV. SIMULATION RESULTS
To evaluate performance of the proposed GRL method,
we have implemented it on the HEVC reference software
HM10.0 [15] configured with the common test conditions assuggested in [16]. The All-intra Main configuration is used in
the experiments, where the RDQ and RDQTS are disabled.
When the rate control is enabled, a target bit rate is assigned toeach frame. The target bit-budget assigned to each frame is theaverage of the bit rates obtained by the anchor HEVC with four
. In the simulation, we have performed
QPs=
four methods with the same configuration.
• HM10.0: the original HM10.0 without rate control.
• JCT-VC K0103: the original HM10.0 with the default rate
control.

WANG et al.: EFFICIENT FRAME-CONTENT BASED INTRA FRAME RATE CONTROL

899

TABLE ISIMULATION RESULTS FOR THE PERFORMANCE COMPARISON IN HEVC RATE CONTROL

Fig. 3. Bits comparison versus frame number for ParkScene.

Fig. 4. Buffer fullness comparison versus frame number for ParkScene.

• JCT-VC M0257: the original HM10.0 with the default intra
frame rate control.
• Proposed method: the proposed GRL method.
In the experiments, the normalized root mean square error(NRMSE) in equation (9) is the measure for bit estimation accuracy of each method, and the standard deviation of PSNR is
the measure of video quality variation.

(9)is the average
where is the number of encoded frames,is the number of actual bits andis theof actual bits,
target bits in the th frame. Note that a smaller NRMSE meansthat a better bit estimation result is achieved.

Fig. 3 shows the bit estimation comparison of five methods). It is observed
for the sequence “ParkScene” (that at the frame level, our method can more accurately achieve
the allocated target bit budget than other methods as shown by
the smooth bit rate variation over the frames. Since our GRL
method considers the frame-content complexity, a better initial
QP is obtained in comparison with other methods, which guarantees that the encoder can quickly approach to a given channel
bandwidth.
Fig. 4 shows the buffer fullness comparison of five algorithms for the sequence “ParkScene”. We can see that the buffer
fullness variation of our method is much smaller than othermethods. In this case, we know that the encoder buffer is operating at a safer level (i.e., not underflow or overflow) whenemploying the proposed GRL method as compared to othermethods.
More detailed numerical experiments are tabulated in Table I.
From this table, we can observe that the proposed GRL method
is able to achieve a more accurate bit-budget estimation. The
NRMSE of the GRL method ranges from 0.24% to 1.73%. Compared with the JCT-VC M0257 [14] whose NRMSE ranges
from 3.35% to 6.23%, our GRL method shows a significant improvement. The smaller bit-budget estimation errors result in asmoother buffer fullness variation and hence more consistent
video quality over the video frames (see PSNR variation). This
has increased the perceptual quality of the reconstructed frames.
In addition, it is worth noting that both the GRL method, and the
traditional methods do not increase the decoding complexity at
all.
V. CONCLUSION
In this letter, a frame-content based rate control method is
proposed for the HEVC intra frame coding. The frame-content complexity is measured by its gradient, which has been
incorporated into an improved R-lambda model. In addition, a
new bits-budget allocation method is developed for intra frame
coding at the CTU level. Experimental results show that the proposed method can significantly reduce the bit estimation error
for the HEVC intra frame rate control, and hence more consistent video quality over the video frames is obtained in comparison with conventional methods.

900IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 7, JULY 2015

REFERENCES[1] G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand, “Overview of thehigh efficiency video coding (HEVC) standard,” IEEE Trans. Circuits
Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, 2012.
[2] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview
of the h. 264/avc video coding standard,” IEEE Trans. Circuits Syst.
Video Technol., vol. 13, no. 7, pp. 560–576, 2003.
[3] J. Lee, I. Shin, and H. Park, “Adaptive intra-frame assignmentand bit-rate estimation for variable GOP length in H. 264,”
IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 10, pp.
1271–1279, 2006.
[4] X. Jing, L.-P. Chau, and W.-C. Siu, “Frame complexity-based ratequantization model for H. 264/AVC intraframe rate control,” IEEE
Signal Process. Lett., vol. 15, pp. 373–376, 2008.
[5] B. Yan and M. Wang, “Adaptive distortion-based intra-rate estimationfor H. 264/AVC rate control,” IEEE Signal Process. Lett., vol. 16, no.
3, pp. 145–148, 2009.
[6] Y. Zhou, L. Tian, and X. Ning, “Intra frame constant rate control
scheme for high efficiency video coding,” in IEEE Int. Conf.
Computing, Networking and Communications (ICNC), 2013, pp.
648–652.
[7] H. Choi, J. Nam, J. Yoo, D. Sim, and I. Bajić, “Rate control based on
unified RQ model for HEVC,” JCT-VC H0213, 2012.

[8] H. Choi, J. Yoo, J. Nam, D. Sim, and I. Bajic, “Pixel-wise unified ratequantization model for multi-level rate control,” IEEE J. Sel. Topics
Signal Process., vol. 7, no. 6, pp. 1112–1123, Dec. 2013.
[9] X. Liang, Q. Wang, Y. Zhou, B. Luo, and A. Men, “A novel rq model
based rate control scheme in HEVC,” Vis. Commun. Image Process.,
pp. 1–6, 2013.
[10] M. Jiang and N. Ling, “On lagrange multiplier and quantizer adjustment for H.264 frame-layer video rate control,” IEEE Trans. Circuits
Syst. Video Technol., vol. 16, no. 5, pp. 663–669, 2006.
[11] M. Wang and B. Yan, “Lagrangian multiplier based joint three-layer
rate control for H. 264/AVC,” IEEE Signal Process. Lett., vol. 16, no.
8, pp. 679–682, 2009.
[12] B. Li, H. Li, L. Li, and J. Zhang, “Rate control by R-lambda model for
HEVC,” JCT-VC K0103, 2012.
[13] L. Xu, D. Zhao, X. Ji, L. Deng, S. Kwong, and W. Gao, “Window-levelrate control for smooth picture quality and smooth buffer occupancy,”
IEEE Trans. Image Process., vol. 20, no. 3, pp. 723–734, Mar. 2011.
[14] X. Wang and M. Karczewicz, “Intra frame rate control based on
SATD,” JCT-VC M0257, Incheon, 2013.
[15] “HEVC reference software,” [Online]. Available: http://hevc.kw.
bbc.co.uk/trac/browser/jctvc-hm/tags 2012[16] F. Bossen, “Common test conditions and software reference configurations,” Joint Collaborative Team on Video Coding, JCT-VC G1200,
Nov. 2011.