IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

2147

Fast HEVC Inter CU Decision Basedon Latent SAD Estimation
Jian Xiong, Hongliang Li, Senior Member, IEEE, Fanman Meng, Member, IEEE,
Qingbo Wu, and King Ngi Ngan, Fellow, IEEE

Abstract—The emerging high efﬁciency video coding(HEVC) standard has improved compression performance
signiﬁcantly in comparison with H.264/AVC. However, more
intensive computational complexity has been introduced byadopting a number of new coding tools. In this paper, a fastinter CU decision is proposed based on the latent sum ofabsolute differences (SAD) estimation. Firstly, a two-layer motion
estimation (ME) method is designed to take advantage of thelatent SAD cost. The new ME method can obtain the SAD costs
for both the upper CU and its sub-CUs. Secondly, a concept of
motion compensation rate-distortion (R-D) cost is deﬁned, and an
exponential model is proposed to express the relationship between
the motion compensation R-D cost and the SAD cost. Then, afast CU decision approach is designed based on the exponential
model. The fast CU decision is implemented by comparing aderived threshold with the SAD cost difference between the upperand sub SAD costs. Experimental results show that the proposed
algorithm achieves an average of 52% and 58.4% reductions of
the coding time at the cost of 1.61% and 2% bit-rate increases
under the low delay and random access conditions, respectively.
Index Terms—H.264, high efﬁciency video coding (HEVC), interprediction, motion estimation, video coding.

I. INTRODUCTIONR

ECENTLY, with the rapid development of network communication and multimedia technologies, video contents
are presented to be High Deﬁnition (HD) and Ultra HD. The
resolutions of the HD videos are larger and the visual quality
is better, when compared with the standard deﬁnition videos.
Meanwhile, higher requirements are put forward to the videocoding technologies. To meet the requirements, International
Telecommunication Union-Telecom (ITU-T) and International
Organization for Standardization/International Electrotechnical

Manuscript received March 30, 2015; revised August 03, 2015 and September
18, 2015; accepted October 01, 2015. Date of publication October 14, 2015;
date of current version November 13, 2015. This work was supported in part by
the National Natural Science Foundation of China under Grant 61525102 and61271289, and in part by the Program for Science and Technology Innovative
Research Team for Young Scholars in Sichuan Province, China under Grant
2014TD0006. The associate editor coordinating the review of this manuscript
and approving it for publication was Prof. Adrian Munteanu.
J. Xiong, H. Li, F. Meng, and Q. Wu are with the School of Electronic
Engineering, University of Electronic Science and Technology of China,
Chengdu 611731, China (e-mail: jxiong219@gmail.com; hlli@uestc.edu.cn;
fmmeng@uestc.edu.cn; wqb.uestc@gmail.com).
K. N. Ngan is with the Department of Electronic Engineering, The Chinese
University of Hong Kong, Hong Kong (e-mail: knngan@ee.cuhk.edu.hk).
Color versions of one or more of the ﬁgures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identiﬁer 10.1109/TMM.2015.2491018

Commission (ISO/IEC) jointly developed a new generation ofvideo coding standard, namely High Efficiency Video Coding(HEVC).
HEVC provides signiﬁcant improvement on the compression
ratio in comparison with H.264/AVC [1]. Part of the improvement comes from the newly adopted quad-tree structure
based coding tree unit (CTU) [2], which brings a ﬂexibility ofmultiple sizes of coding blocks. The coding blocks are named
as coding unit (CU). Each CU can be recursively split into
four squared sub-CUs. For applications with high performancerequirements, Rate-Distortion (R-D) optimization is checkedin a recursive manner for all the CU sizes to obtain the optimalmode. The quad-tree coding structure can improve the coding
performance signiﬁcantly by the recursive RD optimization.
Thus, it also brings intensive computational complexities. Reducing the coding computational complexity effectively is very
important for applying HEVC on the real-time applications.
Since the process of coding a CU includes almost all the modules in the HEVC, the CU decision occupies most of coding timein HEVC. Many studies aim to design the fast CU decision algorithms for HEVC intra prediction [3]–[6] and inter prediction[7]–[9] in order to reduce the coding time. This work focuseson the fast inter CU decision. According to the employed features, the inter CU decision methods can be divided into threeclasses, including encoding parameters based, neighboring CU
based, and R-D cost based methods.
The ﬁrst class speeds up the CU decision by referring the encoding parameters, such as motion vector (MV), coded block
ﬂag (CBF), differential MV, and parameters of sample adaptive
offset (SAO) [10]–[17]. For example, Ahn et al. proposed a fast
CU decision algorithm based on the spatio-temporal encoding
parameters [10], [11]. This method used the SAO parameters tomeasure the CU complexity, and used the MV, CBF and Transform Unit (TU) sizes to measure the motion complexity. A fast
CU decision was implemented by evaluating the CU complexity
and motion complexity, respectively. Correa et al. presented adata mining based fast CU decision method [12], [13]. The information of skip ﬂags, merge ﬂags, and R-D costs were extracted to generate a decision tree. Then, the CU selection wasdetermined by the decision tree. Shen et al. suggested a fast
Bayesian theory based CU decision algorithm [14]. They employed the sum of absolute transform differences (SATD), MV,
and R-D costs as the features. The CUs were selected based on
the Bayesian decision rule. In addition, Pan et al. developed amerge mode detection algorithm based on zero-block detection
and motion estimation [15]. Kim et al. employed the differential MV and CBF to detect the skip mode [16]. Chiang et al.

1520-9210 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

2148

applied a zero-block detection using the differential MV [17].
These works efﬁciently reduced the encoding complexity. However, the encoding parameters are extracted in the middle of thecoding process so that they are insufﬁcient to reﬂect the motion
compensation performance exactly.
The second class employs the depth information of the neighboring CUs to select the size of the current CU [18]–[24]. The
motivation is that the depths of the neighboring CUs tend to be
the same. For example, Shen et al. proposed a CU depth range
estimation in the fast CU decision algorithm [18]. It employed
the depths of the spatio-temporal neighboring CUs to estimate
the depth range of the current CU. Correa et al. also suggested
to estimate the depth of CUs using the depth of spatio-temporalneighboring CUs [19] In order to reduce the depth range, Zhang
et al. applied the depth correlation between the spatio-temporaladjacent CTUs and the current CTUs [21]. With the purposeof skipping R-D optimization in both the frame and CU levels,
Leng et al. employed the depth information of the neighboring
CUs and the co-located CUs of the previous frame [22]. Recently, a Markov Random Field (MRF) based fast CU decisionalgorithm was proposed in [23], [24]. The MRF can efﬁciently
employ the neighboring information in CU decision. In thesemethods, whether the optimal modes can be selected mainly depends on that the neighboring CUs are encoded with the samemodes. However, around the boundary of moving objects, theneighboring CUs tend to be encoded with different modes [25].
The third class is R-D cost based fast CU decision methods[26]–[29]. Lee et al. proposed an R-D cost fast inter CU decision method [26]. They employed the temporary R-D cost aftercoding the skip mode and
PU mode. The Baysianrule on a given false ratio was proposed for the fast CU decision.
Vanne et al. analysed the distribution of the PU modes and therelation between the modes [27]. They proposed the conditionsof coding the symmetric motion partition (SMP) modes andasymmetric motion partition (AMP) modes. In order to reducethe complexity of the RD optimization, Correa et al. developed
a fast CU decision algorithm by comparing the R-D cost witha threshold [28]. In addition, Shen et al. suggested a fast interCU selection algorithm using the temporary R-D cost [29]. It is
known that R-D cost is the criterion in R-D optimization of CU
decision. Thus, the R-D cost based methods can exactly selectthe optimal modes. However, the time savings are limited since
the computation of R-D costs is essentially time-consuming.
In the recent works, the common point is to propose features
for early estimating the motion compensation performance. The
motion compensation performance is generally measured by
the sum of absolute differences (SAD) cost in motion estimation (ME). As we know, ME is an important module in theblock-based hybrid video coding framework. In this module,
the SAD between the original CU and the reference block is
calculated to search the most matching block. The target blockwith the smallest SAD cost is employed as the prediction CU forcoding the residuals. In conventional ME methods, the SAD iscalculated only for the current CU. Actually, since the SAD calculation is a summation operation, the SAD costs of its sub-CUs
are latent variables in the SAD calculation of the current CU.
Here, “latent variables” are deﬁned as variables that are inferredIEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

in the process of calculating the other variables. The latent SAD
costs can efﬁciently reﬂect the motion compensation performance of the sub-CUs. Thus, this study tries to employ the latent
SAD estimation for the fast CU decision. The main contributions are listed as follows.
1) We design a two-layer based ME method. The new ME
method can obtain both the SAD cost of the current CU
and the latent SAD costs of its sub-CUs.
2) A concept of motion compensation R-D cost is deﬁned todescribe the R-D cost related with the motion compensation. Particularly, we propose an exponential model to express the relationship between the motion compensation
R-D cost and the SAD cost.
3) We propose a fast CU decision based on the latent SAD
estimation. A threshold is derived based on the proposed
exponential model. The fast CU decision is implemented
by comparing the derived threshold with the SAD costdifference.
The rest of the paper is organized as follows. An overview of
HEVC mode decision is presented in Section II. The two-layer
based ME method is given in Section III. The analysis of motion compensation cost model is provided in Section IV. The
proposed two-layer based fast CU decision method is proposed
in Section V. Experiments are provided in Section V-C to validate the efﬁciency of the proposed method. Finally, we draw
some concluding remarks in Section VII.
II. OVERVIEW OF HEVC CU PREDICTION
Like H.264/AVC, HEVC [30]–[32] is still a block-based hybrid video coding standard. First, the video frames are divided
into sequences of Coding Tree Units (CTUs). CTU is in a quadtree coding structure, and the nodes of the quad-tree are called
as CUs. CU is the concept of the basic coding region, including
a luma coding block and two chroma coding blocks. Each CU
can be recursively divided into four squared sub-CUs. In HEVC
reference software, the maximum and the minimum of the CU
sizes areand, respectively. Fig. 1(a) shows anexample of a CTU which is recursively divided into sub-CUs,
and forms a quad-tree coding structure. CUs with the sizes,
can be recursively divided into four
sub-CUs. In addition, as the smallest CU,
CUs cannot
be split any more.
Accordingly, prediction unit (PU) is introduced in HEVC. PU
is the basic unit in the prediction. As show in Fig. 1(b), there are
8 PU modes in total. In inter prediction, the optimum mode ischosen as the one which encoded with the minimal Rate-Distortion (R-D) cost. The R-D cost is given by(1)
where denotes the R-D cost. The term denotes the reconstruction distortion which is calculated as the sum of squared
differences (SSD) between the original CU and the reconstructed CU. The term denotes as the number of the coding
bits. The parameter is the Lagrangian multiplier. This is thewell-known Rate-Distortion optimization (RDO) technique.

XIONG et al.: FAST HEVC INTER CU DECISION BASED ON LATENT SAD ESTIMATION

2149

Fig. 2. Conventional motion estimation and two-layer motion estimation
methods.
Fig. 1. HEVC coding structures. (a) The example of a CTU. (b) The PU partition modes.

RDO is a time-consuming operation since it performs transform, quantization, entropy coding, inverse quantization andinverse transform.
We called a CU encoded asmanner if it is encoded
without dividing into sub-CUs. Otherwise, if a CU is encoded as
dividing into sub-CUs, we called it encoded asmanner. For
each CU (not includingCUs), R-D costs are calculated astheandmanners, respectively. The optimum split
ﬂag is determined as the one encoded with the minimal R-D
cost. It is expressed by
split
where
symbols
as the

and
and

(2)

denotes the optimal splitting. The
denote the R-D costs of CU encodedmanner, respectively.

III. TWO-LAYER MOTION ESTIMATION
A. Conventional Motion Estimation
ME is an important module in the block-based video coding
framework. Two ME algorithms are provided in the HEVC
reference software, such as the full search method and the
TZSearch method. The goal of both of them is to search themost matching block in the search regions of the reference
frames. The target block is employed as the prediction CU
for coding the residuals. The top part of the Fig. 2 shows theconventional ME algorithm. The current coded frame and its
reference frame are shown on the right and left, respectively.
The dashed box in the reference frame denotes the search regionof the current CU. In this region, it can search a reference CU,
denoted as. The matching criterion is SAD cost, which is
calculated as
(3)

where
denotes the SAD cost. The term
denotes theestimated bits of the ME. anddenotes the SAD between
the original CU and the reference block.
B. Two-Layer Based Motion Estimation With Latent SAD
In the conventional ME method, SAD cost is calculated only
for the current CU. We called it as Upper SAD cost. On the contrary, we called the SAD costs of its sub-CUs as Sub SAD costs.
However, the SAD calculation is a summation operation, i.e.,
the upper SAD cost is the sum of the sub SAD costs for eachsearch point. Thus, the sub SAD costs are latent variables in thecalculation of the upper SAD cost. Furthermore, as the matching
criterion in the ME methods, SAD cost can reﬂect the motion
compensation performance efﬁciently, i.e., the upper and sub
SAD costs can reﬂect the motion compensation of the CU encoded as the unsplit and split manners, respectively. Therefore,
we can employ both of the upper SAD cost and the latent sub
SAD costs in the fast CU decision.
In order to obtain both the upper SAD cost and the latent sub
SAD costs, we design a two-layer based ME methods. As shownin the bottom part of Fig. 2, the two-layer ME method also needs
to search a reference blockfor the current CU. However,the difference is that the new ME method needs to search thereference blocks of the sub-CUs. The references of the sub-CUs
are denoted as,,and, respectively.
The two-layer ME method can be easily implemented. Since
the sub SAD costs are latent variables in the calculation of theupper SAD cost, only a small amount of additions, shifts andlogical operations are needed. More speciﬁcally, before calculating the SAD of the current CU at each search point, it ﬁrst
calculates the SADs of its sub-CUs one-by-one. The summationof the sub SADs is equal to the upper SAD at each search point.
Then, update the minimal upper SAD cost and the minimal sub
SAD costs, respectively. After performing the ME method, we
can obtain the optimal upper SAD cost and the sub SAD costs.
The optimal upper SAD cost is denoted as, which can reﬂect the motion compensation effect of the current CU encoded

2150

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

as the unsplit manner. The summation of the optimal sub SAD
costs is denoted as. It can reﬂect the motion compensation effect of the sub-CUs encoded as the split manner. This
study tries to employ the SAD costs
andin thefast CU decision.
IV. MOTION COMPENSATION COST MODEL
In this section, we proposed a motion compensation R-D costmodel to express the relationship between the SAD cost andmotion compensation R-D cost. The existing rate and distortion
models are introduced ﬁrst. Then, a concept of motion compensation cost is presented, and an exponential model is proposed
to express the motion compensation R-D cost.
A. Rate-Distortion Models
In [33], the authors have studied the R-D cost models, which
include the rate model and the distortion model. The rate has
been divided into two parts, the header bits and the coefﬁcient
bits, which are expressed by(4)
denotes the header bits, and the term
dewhere the term
notes the coefﬁcient bits. Furthermore, the header bits are modeled to have a good linear relationship with the numbers of the
CU and PU partitions, which is given by(5)
anddenote the number of the CU and PU, rewhere
spectively. However, the coefﬁcient bits are modeled as a function of the SATD and Quantization Parameter (QP) [33](6)
where is the frame number anddenotes the SATD
of the th frame. The parameter denotes the current frame
number, anddenotes the generated source bits of the latest
previous frame. The symbols
anddenote the QPsof the current frame and the latest previous frame, respectively.
The symbol
is a constant. Thus, the coefﬁcient bit numberof the current frame is estimated by the SATD of the previous
frames, the bits number of the latest previous frame, as well as
the QPs of the current frame and the latest previous frame.
The distortion model is also proposed as a function of the
SATD and QP, which is an exponential formula, shown as(7)where , and are the model parameters. Therefore, with thetrained parameters, the distortion can be estimated by the SATD
and QP.
B. The Model of the Motion Compensation Cost
In order to employ SAD in the fast CU decision, we try toestablish the relationship between SAD and the rate-distortion
model. Combining (1) and (4), the R-D cost can be rewritten as(8)

Fig. 3. Relationship between the motion compensation cost and.

From (6) and (7), it can be observed that the distortion andthe coefﬁcient bits
are both modeled as the functions of the
SATD and QP. Thus, it is reasonable to combine the distortionand the coefﬁcient bits cost into a new cost, which is deﬁned as
motion compensation cost. The symbol
is used to denotethe motion compensation cost, i.e.,. The motion compensation cost means the cost related with the motion
compensation. It has a strong relationship with the motion cost(such as SATD and SAD) and QP. Then, (8) can be rewritten as(9)
On one hand, the motion compensation cost has a strong relationship with the motion cost and QP. On the other hand, theSAD costs of both the upper CU and its sub-CUs can be obtainedin the two-layer based ME method. In order to employ the SAD
costs in the fast CU decision, we analyse the relationship between the motion compensation cost, the SAD cost and QP. In[33], the parameter was set to 1.5. Thus, we study on the statistical data ofand, where. Fig. 3 shows
the statistical data for P-slices and B-slices, respectively. It is interesting that there is a strong exponential relationship between
and, which can be expressed by(10)
This formula is called as the motion compensation cost model(MCCM). The symbols , and are the model parameters.
Although both of the distortion model in [33] and the proposed
MCCM are in the exponential forms, MCCM is different fromthe distortion model. Since the values
andare different fromandrespectively, the model parameters
and are different with the parameters and .
In addition, we evaluate the ﬁtting goodness of MCCM.
Table I shows the R-square values of MCCM for the P-slice
and B-slice, respectively. It can be observed that the average
R-square values are above 0.997 for both the P-slice andB-slice. That is, MCCM has high accuracy on modeling theXIONG et al.: FAST HEVC INTER CU DECISION BASED ON LATENT SAD ESTIMATION

2151

where

TABLE IFITTING GOODNESS (R-SQUARE) OF THE
MOTION COMPENSATION COST MODEL, i.e.,

is the corresponding motion compensation cost of.

B. The Proposed Fast CU Decision

relationship between the motion compensation cost and SAD
cost.
V. FAST CU DECISIONIn this section, a fast CU decision algorithm is proposed basedon the motion compensation model. First, an analysis of CU
decision is presented, and a threshold is derived to compare withthe difference of the SAD costs in the fast CU decision. Then,the details of the proposed fast CU decision are provided.
A. Analysis of CU Decision
From (9), the differences between the R-D costs
can be calculated as

and
(11)

where
and
are the motion compensation costs for the
CU coded as the unsplit and split manners, respectively. The
terms
and
denote the header bits for the CU coded as theunsplit and split manners, respectively.
Since QP is a ﬁx value in the CU decision, (10) can be considered as the function ofwith the variation SAD cost. Performing the Taylor expansion at a point, and ignoring
the high order terms, we obtain

There are three steps in the proposed fast CU decision, i.e.,
the fast skip mode detection, the fast split mode detection, andthe fast unsplit determination.
1) Fast Skip Mode Detection: The skip mode is an important
coding scheme in HEVC. Especially in the static regions, mostof the CUs tend to be encoded with the skip mode. Furthermore,
the skip mode is generally coded before the other modes. If the
optimal mode is the skip mode, the other modes can be earlyterminated. Thus, it is necessary to detect the skip mode in thefast CU decision.
In this study, a fast skip mode detection method is proposedusing the R-D costs. The R-D cost of a CU encoded with theskip mode is called as the skip R-D cost. When the temporary
optimal mode is the skip mode after performing the skip andmerge modes, the skip R-D cost is used to determine whether theskip mode is the ﬁnal optimal mode. Speciﬁcally, the skip R-D
costs are recorded for the last 10 CUs which are encoded withthe skip mode as the optimal mode. A threshold can be estimatedby these 10 R-D costs. A trimmed mean estimator [34]–[36] isused to calculate the threshold. Speciﬁcally, after discarding the
maximum and minimum R-D costs in the samples, it computethe mean of the remaining R-D costs. The trimmed mean is lesssensitive to outliers than the mean, such that it is a reasonable estimate of central tendency or mean for many statistical models.1
It is expressed by

(16)
where

(12)
for the points
This formula is a linear approximation ofaround.
After the two-layer based ME, the SAD costs of the upper CU
and its sub-CUs can be obtained, which are denoted asand, respectively. From (10), the motion compensation
costs
can be estimated with the SAD cost, andcan be estimated with. Then, (11) can be rewritten as(13)Then, (2) can be rewritten as

where
The symbol(14)is the number of the sample number, i.e.,.
. denote the R-D costs of the last 10 skip
CUs. If the skip R-D cost is smaller than
, the current
CU is determined to encode with the skip mode, i.e., the othermodes are early terminated. Otherwise, the other modes should
be coded continuously.
2) Fast Split Mode Detection: In the fast skip mode detection,
if the skip mode is not determined as the ﬁnal optimal mode, theother modes should be coded continuously. The
N
N mode will be coded ﬁrst. There will be a ME in coding
this mode. As described in the above section, the two-layer motion estimation will replace the conventional ME. The new ME
method can obtain both the SAD cost of the upper CU and the latent SAD costs of its sub-CUs, which are denoted asand, respectively. Givenand, by estimating
the value of the parameters
,
,
, and
, the split
ﬂag can be determined as (14) and (15).
Since (12) is the linear approximation of
for the points
near, it is reasonable to set
as the average ofand. It is expressed bydenotes a threshold. It can be rewritten as

(17)
(15)

1[Online]. Available: https://en.wikipedia.org/wiki/Truncated_mean

2152

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

Furthermore,is estimated using the previous encoded
CUs. The R-D costs of the last 10 previous CUs are recorded.
For the previous CU whose SAD cost is the closest to,
the corresponding R-D cost is set as the prediction R-D cost,. Then,is estimated as
minus the cost of header
bits. It is given by(18)
Then, (15) can be rewritten as(19)
The remain estimated parameters are the header bits
and
. The previous encoded CU is also used in the estimation. It
records header bit numbers of the last 10 encoded CUs. Assume
is larger than, it is easy to prove that, if
gets themaximum andgets the minimum, will get the maximum.
It is expressed by(20). The terms
where is the sample number, i.e.,anddenote the th header bits for the depth and,respectively. From (5), header bits have the linear relationshipwith the number of CU and PU. The number of CU and PU
will be increased if it is encoded as the split manner. Then, theestimation ofis as 4 times of.
Combining (19) and (20), we can obtain the maximum estimation of . Finally, with the difference between
and, the split ﬂag can be early detected by the estimated .
If, the coding of current
N N
mode will be terminated, and the current CU should be encodedas split manner. Otherwise, continue coding the other modes
normally.
3) Fast Unsplit Mode Determination: In the fast split mode
detection, if, continue coding the othermodes on the current depth. After that, it may get a new R-D
cost, denoted as, and a new header bits, denoted as , i.e.,. For simplicity, the new estimation ofis . The
details of the fast unsplit mode determination are as follows.
1) After coding all the modes on the current depth, if the temporary optimal mode is the skip mode, it will be determined
to be encoded as unsplit manner.
2) Otherwise, updatewith the new R-D cost
, i.e.,, and calculate the new thresholdusing the new parameters,,
as in (15). Meantime, calculate the average of the R-D costs of the previous
unsplit CUs. It is expressed by(21)
where is the th R-D cost of the previous CUs which is
encoded as the unsplit manner. If
and, the current CU is determined to be
encoded as unsplit manner.

Fig. 4. Flowchart of the proposed fast CU Decision.

3) Otherwise, continue coding the current CU as the split
manner.
C. Flowchart
The ﬂowchart of the proposed fast CU decision is shown in
Fig. 4. The main steps are as follows.
1) First, code the skip and merge mode and obtain the R-D
cost. Calculate
as in (16). If the temporary optimal mode is the skip mode, and, the current
CU is determined to be encoded as the skip mode, and go
to step 5. Otherwise, go to step 2.
2) Perform the two-layer ME of theN
N mode,and obtain the SAD costs
and. Estimate
the header bitsand
, and obtain the threshold . If
, the current CU is determined to be
encoded as the split manner. Otherwise, continue coding
the other modes on the current depth and go to step 3.
3) If the temporary optimal mode is the skip mode, the current
CU is determined to be encoded as the unsplit manner, andgo to step 5. Otherwise, go to step 4.
4) Update using the new R-D cost
and the new header
as in (21), if
bits. Calculateand, the current CU is determined to be
encoded as the unsplit manner, and go to step 5. Otherwise,code it with the split manner.
5) Finish coding the current CU on the current depth.

XIONG et al.: FAST HEVC INTER CU DECISION BASED ON LATENT SAD ESTIMATIONTABLE IIFITTED VALUES OF THE PARAMETER IN
THE MOTION COMPENSATION COST MODEL

VI. EXPERIMENTAL RESULTS
A. Test Conditions
In this section, the performance of the proposed fast CU decision is evaluated in terms of the difference of the computation time, the change of the Bjontegaard Delta (BD) bit-rate(BDBR) and Peak Signal to Noise Ratio (PSNR) [37]. The proposed method is integrated on the HEVC reference software,
HM9.0.2 The simulation was performed on a PC with an Intel(R) 2.30 GHz processor, 32 Gb RAM.
In the experiment, 17 sequences were tested, and Quantization Parameters (QP) values were set to 22, 27, 32,
and 37 in order to cover a broad range of qualities andbit-rate. The experiment settings include the low delay (“encoder_lowdelay_P_main”) and random access (Random_Access_Main). The time savings were calculated as
(22)
where
denotes the overall encoding time of the proposed fast algorithm and,denotes the total encoding
time of reference method. Since most of the CUs tend to be encoded with the large sizes, such as
and
, the
proposed method is implemented only on depth 0 and 1.
The model parameter in MCCM is employed to estimate thethreshold . Table II shows the ﬁtted values of the parameter .
It can be observed that the average ﬁtted values are 1.473 and1.481 for the P and B slices, respectively. Furthermore, it showsthat most of the ﬁtted values are close to 1.5. Thus, is set to
1.5 in the experiments.
In order to validate the effectiveness of the proposed method,
we compared it with the state-of-the-art fast algorithms, including the Shen’s algorithms [18], [29], the Vanne’s algorithm[27], the Ahn’s algorithm [11] and our previous works [23],[25]. The Shen’s algorithms are labeled as CUD [18] andSTMD [29], respectively. Our previous works are labeled as
PMD [25] and MRF [23], respectively. Since the methods [11],[18], [27], [29] were developed in different versions (HM12.0,
HM2.0, HM8.0 and HM10.0, respectively) of HM reference
software, we have carefully implemented them into HM9.0
reference software for fair comparison.
B. Results of the Low Delay SettingFirst, we evaluated the performance of the proposed methodon the low delay setting. The 8th column of Table III shows
the performance of the proposed algorithm compared with the2[Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn HEVCSoftware/tags/HM-9.0rc2/

2153

reference software. The test sequences are with different resolutions and motion intensity. From the results, it can be observed that the proposed fast CU decision algorithm can save
52% coding time of the reference software on an average. In
other words, the proposed algorithm takes less than half of thecoding time consumed by the reference software. Furthermore,
the average BD bit-rate only increases 1.61%. In particular, forthe sequence Vidyo1, the time saving reaches up to 74.6% with
only 0.49% bit-rate increment. Fig. 5 shows the R-D curves of
6 sequences. It shows that the proposed fast CU decision algorithm can achieve a similar R-D performance as that of thereference software no matter for the low bit-rates and the highbit-rates. To sum up, the results show that the proposed algorithm can signiﬁcantly reduce the coding time with only 1.61%
bit-rate increase.
The 2nd column of Table III shows the results of the PMD
method [25]. It can be observed that the average BD bit-rate increment of the PMD method is larger than that of the proposed
method (the former is 1.89% and the latter is 1.61%). However,the proposed method can further reduce 10% coding time withrespect to the PMD method. Therefore, the proposed method
signiﬁcantly reduce more coding time with less performanceloss than the PMD method, i.e., the proposed method outperforms the PMD method.
In Table III, the 3rd column shows the results of the CUD
method [18]. From the comparison results, it can be observedthat the average BD bit-rate increment of the proposed algorithm is almost equal to that of the CUD method (The former is
1.61% and the latter is 1.60%). However, the proposed method
can further reduce about 13% of the coding time with respect
to the CUD method. In other words, the proposed algorithm reduce the coding time far more than the CUD method with almost the same coding performance degradation. Some of theresults (such as Kimono, BasketballDrive) show that the proposed method achieve slightly more bit-rate increments thanthat of the CUD method. However, the proposed algorithm canachieve signiﬁcantly more time savings than that of the CUD
method. For example, 23% of the coding time can be further reduced with only 1.45% bit-rate increase for sequence Kimono. It
can be concluded that the proposed algorithm can obtain a better
trade-off in terms of the time saving and the performance degradation. The proposed algorithm outperforms the CUD method.
The 4th column of Table III shows the results of the MRF
method [23]. It can be observed that the average time saving of
the proposed algorithm is slightly less than that of the method[23]. However, the bit-rate increment of the proposed algorithm
is less than that of the method [23] (the former is 1.61%, and thelatter is 2.19%). It shows that the proposed fast CU algorithm
is competitive with the method [23]. However, on one hand, theproposed method is designed with only simple operations, suchas few additions and shifts. On the other hand, there are only
small changes of the two-layer ME with respect to the conventional ME. Thus, the design style of the proposed method is very
suitable for chip development.
The 5th, 6th, 7th columns of Table III show the results of the
Ahn’s algorithm [11], the Vanne’s algorithm [27], the STMD
method [29], respectively. As shown in Table III, the average
BD bit-rate increments of these methods [11], [27], [29] are2154

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

TABLE IIIPERFORMANCE COMPARISON OF PROPOSED METHOD WITH RECENT WORKS (LOW DELAY)TABLE IVPERFORMANCE COMPARISON OF PROPOSED METHOD WITH RECENT WORKS (RANDOM ACCESS)

slightly less than that of the proposed method. However, the proposed method signiﬁcantly reduce more coding time (11%, 12%
and 10%, respectively) than these methods [11], [27], [29]. The
above experimental results indicate that the proposed method issuperior or comparable to the three state-of-the-art fast CU decision methods on the low delay setting.
C. Results of the Random Access Setting
We also evaluate the proposed algorithm on the random access setting. The 8th column of Table IV shows the results ofthe proposed method when compared with the reference software. It can be observed that the proposed method can reduce
58.4% of the coding time with only 2% BD bit-rate increment.
The results indicate that the proposed method can speed up the
CU decision effectively.
The 2nd column of Table IV shows the results of the PMD
method [25]. As shown in Table IV, the PMD method reduce average 43.2% of the total coding time with 1.89% BD bit-rate increment. The proposed method reduce more (15%) coding timethan PMD with only 0.11% BD bit-rate increment. It can be concluded that the proposed method signiﬁcantly outperforms the
PMD method on the random access setting.
The 3rd column of Table IV shows the results of the CUD
method. The CUD method can reduce 43% of the coding timewith 2.97% BD bit-rate increment in average. The proposed
method can further reduce about 15% of the total coding timewith respect to the CUD method. Furthermore, the average bitrate increment of the proposed method is less than that of the
CUD method. It can be concluded that the proposed method signiﬁcantly outperforms CUD in terms of both the time saving andcoding performance degradation on this setting.
The 4th column of Table IV shows the results of the MRFbased method [23]. It can be observed that, the time saving of
the proposed method is slightly smaller than that of the method[23] (the former is 58.4%, the latter is 63.2%). However, thebit-rate increment of the proposed method is signiﬁcantly lower
than that of the method [23] (The former is 2%, the latter is
2.78%). On this setting, the proposed method can obtain a better

XIONG et al.: FAST HEVC INTER CU DECISION BASED ON LATENT SAD ESTIMATION

2155

Fig. 5. R-D performance of the proposed fast CU decision.

trade-off between the time saving and coding performance loss
than the method [23].
The 5th, 6th, 7th columns of Table IV show the results of the
Ahn’s algorithm [11], the Vanne’s algorithm [27], the STMD
method [29], respectively. Furthermore, the performance comparisons of the random access setting are similar to that of the
low delay setting. On this setting, it can be observed that the BD
bit-rate increment of the methods [11], [27], [29] are slightly lessthan that of the proposed method, and the SDs are smaller thanthat of the proposed method. However, the proposed method

saves more coding time (13%, 13% and 10%, respectively) thanthese methods [11], [27], [29]. It can be concluded that the proposed method is superior and comparable to the state-of-the-art
algorithms on this setting.
We also calculate the standard deviations (SD) of the BD
bit-rate increments and time savings. As shown in Table III, andTable IV, for the proposed method, the SDs of bit-rate increment
are only 0.81% and 1.06% under the low delay and random access settings, respectively. It can be observed that the proposed
method is robust to different video contents. On one hand, the2156IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

TABLE VPERFORMANCE OF THE SUB-ALGORITHMS (LOW DELAY)TABLE VIPERFORMANCE OF THE SUB-ALGORITHMS (RANDOM ACCESS)

bit-rate increment SDs of the proposed method are close to thatof the PMD and MRF-based methods, and they are smaller thanthat of the CUD method. That is, the proposed method achieves
comparable or superior robustness as the PMD, MRF, and CUD
methods. On the other hand, the bit-rate increment SDs of the
proposed method are larger than that of the Ahn’s algorithm[11], the Vanne’s algorithm [27], the STMD method [29]. However, the proposed method results in signiﬁcantly more time savings. Furthermore, it can be observed that the SDs of the timesavings are around 10% for all the comparative methods. It indicate that the time savings are mainly inﬂuenced by the video
contents. When encoding the simple videos, more time is likely
to be saved. On the contrast, less time is likely to be saved forthe complex videos.
D. Contributions of the Sub-Algorithms
As introduced in the above section, the proposed method includes three mode detection algorithms, i.e., fast skip mode detection (FSkip), fast split mode detection (FSplit), and fast unsplit detection (FUnsplit). In order to investigate the performance contribution of each part, we have evaluated the BD

bit-rates and time savings of these sub-algorithms. Experimental
results of the low delay and random access settings are shownin Tables V and VI, respectively.
As shown in Table V, FSkip yields about average 28.1%
reduction of the total coding time with only 0.33% BD-rateincrement. Especially for the sequence Vidyo1, FSkip greatly
reduces about 55.1% coding time with only 0.24% BD bit-rateincrement. On the other hand, FUnsplit saves about 39.7%
coding time with only 0.91% BD bit-rate increment. The
sequence Vidyo1 also achieves the best performance in all
the tested sequences. That is, the sequence Vidyo1 achieves
the most time saving for both the FSkip and FUnsplit detections. The reason is that, the sequence Vidyo1 is with staticbackground and smooth motion. It makes that CUs in thissequence are more likely to be encoded as the skip mode andthe unsplit manner. The results of the other sequences also
validate that FSkip and FUnsplit reduce more coding time forthe sequences with smooth motion than that of the complex
sequences. Furthermore, FSplit results in about 12.6% timesaving with average 0.59% BD bit-rate increment. It can beobserved that, for Vidyo1, FSplit saves almost the least coding

XIONG et al.: FAST HEVC INTER CU DECISION BASED ON LATENT SAD ESTIMATION

2157

TABLE VII
TIME SAVINGS OF DIFFERENT VIDEO RESOLUTIONSTABLE VIIIRESULTS OF TIME SAVINGS ON DIFFERENT QPS

time. However, for the sequences with high intensity motion(such as, RaceHorsesC, PartyScene and PeopleOnstreet),
FSplit saves signiﬁcantly more coding time than the sequenceswith smooth motion (such as, Vidyo1, Vidyo3, and BQSquare).
It can be considered that FSplit is complementary with FUnsplit. The overall scheme which combines these sub-algorithms
together yields average 52% time saving with only 1.61% BD
bit-rate loss, i.e., the overall scheme saves the most coding
time. Results in Table VI show that the performance of random
access setting is similar with that of the low delay setting. It
is observed that all the sub-algorithms reduce the total coding
time effectively with acceptable BD bit-rate increments.
E. Time Saving Statistics and Analyses
We also investigate the performance of the proposed method
tested with different parameters, such as QP, and video resolutions. Table VII shows the time saving of the proposed method
performed on different resolutions. From the results, we can seethat the average time saving of class E reaches up to 72.75%,
which is signiﬁcantly larger than that of the other classes. Thatis because the videos in class E are particular. Backgrounds ofthese video are static, and the foreground are with small motions. For these videos, most of the CUs are encoded as the unsplit manner, or the skip mode. Furthermore, the time savings ofthe other classes are around 50%. It indicates that the proposed
algorithm is robust to a wide range of video resolutions.
Table VIII shows the time savings of different QPs on the twosettings, respectively. When QP is set to 37, the time saving is
66.64%. When QP is set to 22, the time saving is 44.75%. It
can be observed that the time savings of high QPs are larger
than that of the low QPs. When a video is encoded with thehigh QP, the CUs tend to be encoded as the unsplit manner,and most of the CUs encoded as the skip mode. It indicatesthat the proposed method is more efﬁcient for the low bit-rate
applications. However, even for the QP 22, almost half of thecoding time is reduced. It can be concluded that the proposed
method is robust to a wide range of bit-rates.
F. Results of the AVC Tested Sequences
In order to validate that the proposed algorithm is generally
applicable to different contents, the AVC sequences with sizes
QCIF and CIF are tested. Results are shown in Table IX. It

TABLE IXPERFORMANCE OF THE PROPOSED METHOD FOR THE AVC TESTED SEQUENCES

can be observed that the proposed algorithm reduces average
42.8% and 48.0% of the total coding time, when the average
BD bit-rate increments are only 1.80% and 1.69% for the low
delay and random access settings, respectively. Especially forthe sequence Akiyo, the proposed method saves up to 65.4%
and 70.7% coding time with only 0.16% and 0.94% BD bit-rateincrements for the low delay and random access settings, respectively. The results indicate that the proposed algorithm isalso applicable to the QCIF and CIF sequences.
G. Performance of Applying the Full Search ME
The TZsearch ME method is applied in this work. Some of thesearch points are skipped in the TZsearch ME. In this case, SAD
calculation of the sub-CU only covers the points that the CU
ME reaches. However, the sub-CUs ME may search on different
points. In order to investigate the differences, we tested the proposed scheme with the full search ME on the low delay setting.
Experimental results are shown in Table X. It can be observed
that, when applying the full search ME, the average BD bit-rateincrement is slightly less than that of applying the TZsearch ME(The former is 1.64%, and the latter is 1.84%). However, bothof the coding performance losses are signiﬁcantly small, i.e., theproposed algorithm is suitable to both of the two ME schemes.
In addition, we investigate the encoding time of different
schemes. As shown in Table X,
anddenote the encoding time consumed by the HM reference software when applying the full search ME and TZsearch ME, respectively. The
symbolsanddenote the encoding time ofthe proposed fast CU decision algorithm when applying thefull search ME and TZ search ME, respectively. The relative
time savings of applying the two ME schemes are denoted asand, respectively. The relative timesaving of applying the full search ME is larger than that ofapplying the TZsearch ME (The former is about 60.6%, andthe latter is about 44.0%), i.e., more modes are early detectedor skipped when applying the full search ME. It can be concluded that the more accurate SAD calculation of the sub-CUs
will lead to more time saving. However, when comparing with

2158

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 17, NO. 12, DECEMBER 2015

TABLE XPERFORMANCE OF APPLYING THE FULL SEARCH ME AND THE TZSEARCH ME, the average time saving of applying the TZsearch ME
is signiﬁcantly larger than that of applying the full search ME(The former is about 96.6%, and the latter is about 60.6%). The
reason is that the full search ME drastically increase the computational complexity. Therefore, it is better to implement the
proposed fast CU decision algorithm on the encoder with thefast ME methods, such as the TZsearch ME. The future work is
improving the accuracy of the sub-CU SAD calculation.
VII. CONCLUSION
In this paper, a two-layer ME based fast CU decision method
has been proposed. This method employs the latent SAD estimation for the fast CU decision. First, we design a two-layer based
ME method. The new ME method obtains the SAD costs forboth the upper CU and its sub-CUs. The upper SAD cost can reﬂect the motion compensation effect of the upper CU, as well as
the sub SAD costs can reﬂect the motion compensation effect of
the sub-CUs. Then, we deﬁne the concept of motion compensation R-D cost and its exponential expressing model. This model
can express the relationship between the motion compensation
R-D cost and the SAD cost. Based on the exponential model, we
can derive a threshold. The fast CU decision is converted to the
problem of estimating the threshold. The split ﬂags can be early
determined by comparing the estimated threshold with the SAD
cost difference. Experimental results show that the proposed algorithm signiﬁcantly reduce average 52% and 58.4% encoding
time with only 1.61% and 2% bit-rate increases for the low delay
and random access settings, respectively.
REFERENCES[1] T. Wiegand, G. Sullivan, G. Bjontegaard, and A. Luthra, “Overview
of the H.264/AVC video coding standard,” IEEE Trans. Circuits Syst.
Video Technol., vol. 13, no. 7, pp. 560–576, Jul. 2003.
[2] I.-K. Kim, K. McCann, K. Sugimoto, B. Bross, and W.-J. Han,
“Hm7: High efﬁciency video coding (HEVC) test model 7 encoder
description,” in Proc. 9th JCT-VC Meeting, Apr. 2012, pp. 6–11, Art.
ID JCTVC-I1002.
[3] L. Shen, Z. Zhang, and P. An, “Fast CU size decision and mode decision algorithm for HEVC intra coding,” IEEE Trans. Consumer Electron., vol. 59, no. 1, pp. 207–213, Feb. 2013.
[4] L. Shen, Z. Zhang, and Z. Liu, “Effective CU size decision for
HEVC intracoding,” IEEE Trans. Image Process., vol. 23, no. 10, pp.
4232–4241, Oct. 2014.
[5] J. Xiong and H. Li, “Fast and efﬁcient prediction unit size selectionfor HEVC intra prediction,” in Proc. Int. Symp. Intell. Signal Process.
Commun. Syst., Nov. 2012, pp. 366–369.
[6] J. Xiong, “Fast coding unit selection method for high efﬁciency videocoding intra prediction,” Opt. Eng., vol. 52, no. 7, 2013, Art. ID
071504.

[7] H. L. Tan, F. Liu, Y. H. Tan, and C. Yeo, “On fast coding tree blockand mode decision for high-efﬁciency video coding (HEVC),” inProc. IEEE Int. Confe. Acoust., Speech Signal Process., Mar. 2012,
pp. 825–828.
[8] Y. Zhang, S. Kwong, X. Wang, H. Yuan, Z. Pan, and L. Xu, “Machinelearning-based coding unit depth decisions for ﬂexible complexity allocation in high efﬁciency video coding,” IEEE Trans. Image Process.,
vol. 24, no. 7, pp. 2225–2238, Jul. 2015.
[9] L. Shen, P. An, Z. Zhang, Q. Hu, and Z. Chen, “A 3D-HEVC fast modedecision algorithm for real-time applications,” ACM Trans. Multimedia
Comput., Commun., Appl., vol. 11, no. 3, pp. 34–56, Jan. 2015.
[10] S. Ahn, M. Kim, and S. Park, “Fast decision of CU partitioning basedon SAO parameter, motion and PU/TU split information for HEVC,”
in Proc. Picture Coding Symp., Dec. 2013, pp. 113–116.
[11] S. Ahn, B. Lee, and M. Kim, “A novel fast CU encoding scheme basedon spatio-temporal encoding parameters for HEVC inter coding,” IEEE
Trans. Circuits Syst. Video Technol., vol. 25, no. 3, pp. 422–435, Mar.
2015.
[12] G. Correa, P. Assuncao, L. Agostini, and L. da Silva Cruz, “FastHEVC encoding decisions using data mining,” IEEE Trans. Circuits
Syst. Video Technol., vol. 25, no. 4, pp. 660–673, Apr. 2015.
[13] G. Correa, P. Assuncao, L. Agostini, and L. da Silva Cruz, “A method
for early-splitting of HEVC inter blocks based on decision trees,” inProc. 22nd Eur. Signal Process. Conf., Sep. 2014, pp. 276–280.
[14] X. Shen, L. Yu, and J. Chen, “Fast coding unit size selection for HEVC
based on Bayesian decision rule,” in Proc. Picture Coding Symp., May
2012, pp. 453–456.
[15] Z. Pan, S. Kwong, M.-T. Sun, and J. Lei, “Early merge mode decision based on motion estimation and hierarchical depth correlation for
HEVC,” IEEE Trans. Broadcast., vol. 60, no. 2, pp. 405–412, Jun.
2014.
[16] J. Kim, J. Yang, K. Won, and B. Jeon, “Early determination of modedecision for HEVC,” in Proc. Picture Coding Symp., May 2012, pp.
449–452.
[17] P.-T. Chiang and T. S. Chang, “Fast zero block detection and early CU
termination for HEVC video coding,” in Proc. IEEE Int. Symp. Circuits
Syst., May 2013, pp. 1640–1643.
[18] L. Shen, Z. Liu, X. Zhang, W. Zhao, and Z. Zhang, “An effective CU
size decision method for HEVC encoders,” IEEE Trans. Multimedia,
vol. 15, no. 2, pp. 465–470, Feb. 2013.
[19] G. Correa, P. Assuncao, L. Agostini, and L. Cruz, “Coding tree depth
estimation for complexity reduction of HEVC,” in Proc. Data Compression Conf., Mar. 2013, pp. 43–52.
[20] J.-H. Lee, C.-S. Park, B.-G. Kim, D.-S. Jun, S.-H. Jung, and J. S. Choi,“Novel fast PU decision algorithm for the HEVC video standard,” inProc. 20th IEEE Int. Conf. Image Process., Sep. 2013, pp. 1982–1985.
[21] Y. Zhang, H. Wang, and Z. Li, “Fast coding unit depth decision algorithm for interframe coding in HEVC,” in Proc. Data Compression
Conf., Mar. 2013, pp. 53–62.
[22] J. Leng, L. Sun, T. Ikenaga, and S. Sakaida, “Content based hierarchicalfast coding unit decision algorithm for HEVC,” in Proc. Int. Conf. Multimedia Signal Process., May 2011, vol. 1, pp. 56–59.
[23] J. Xiong, H. Li, F. Meng, S. Zhu, Q. Wu, and B. Zeng, “MRF-based fast
HEVC inter CU decision with the variance of absolute differences,”
IEEE Trans. Multimedia, vol. 16, no. 8, pp. 2141–2153, Dec. 2014.
[24] J. Xiong, H. Li, F. Meng, B. Zeng, S. Zhu, and Q. Wu, “Fast and efﬁcient inter CU decision for high efﬁciency video coding,” in Proc.
IEEE Int. Conf. Image Process., Oct. 2014, pp. 3715–3719.

XIONG et al.: FAST HEVC INTER CU DECISION BASED ON LATENT SAD ESTIMATION[25] J. Xiong, H. Li, Q. Wu, and F. Meng, “A fast HEVC inter CU selection method based on pyramid motion divergence,” IEEE Trans. Multimedia, vol. 16, no. 2, pp. 559–564, Feb. 2014.
[26] J. Lee, S. Kim, K. Lim, and S. Lee, “A fast CU size decision algorithmfor HEVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 3,
pp. 411–421, Mar. 2015.
[27] J. Vanne, M. Viitanen, and T. Hamalainen, “Efﬁcient mode decision
schemes for HEVC inter prediction,” IEEE Trans. Circuits Syst. Video
Technol., vol. 24, no. 9, pp. 1579–1593, Sep. 2014.
[28] M. Cassa, M. Naccari, and F. Pereira, “Fast rate distortion optimization
for the emerging HEVC standard,” in Proc. Picture Coding Symp., May
2012, pp. 493–496.
[29] L. Shen, Z. Zhang, and Z. Liu, “Adaptive inter-mode decision for
HEVC jointly utilizing inter-level and spatiotemporal correlations,”
IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 10, pp.
1709–1722, Oct. 2014.
[30] G. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand, “Overview of thehigh efﬁciency video coding (HEVC) standard,” IEEE Trans. Circuits
Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, Dec. 2012.
[31] K. Ugur et al., “High performance, low complexity video coding
and the emerging HEVC standard,” IEEE Trans. Circuits Syst. Video
Technol., vol. 20, no. 12, pp. 1688–1697, Dec. 2010.
[32] W.-J. Han et al., “Improved video compression efﬁciency throughﬂexible unit representation and corresponding extension of coding
tools,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 12, pp.
1709–1720, Dec. 2010.
[33] S. Ma, J. Si, and S. Wang, “A study on the rate distortion modelingfor high efﬁciency video coding,” in Proc. 19th IEEE Int. Conf. Image
Process., Sep. 2012, pp. 181–184.
[34] T. J. Rothenberg, F. M. Fisher, and C. B. Tilanus, “A note on estimationfrom a Cauchy sample,” J. Amer. Statist. Assoc., vol. 59, no. 306, pp.
460–463, 1964.
[35] D. Bloch, “A note on the estimation of the location parameter of the
Cauchy distribution,” J. Amer. Statist. Assoc., vol. 61, no. 315, pp.
852–855, 1966.
[36] P. J. Rousseeuw and S. Verboven, “Robust estimation in very small
samples,” Comput. Statist. Data Anal., vol. 40, no. 4, pp. 741–758,
2002.
[37] Calculation of Average PSNR Differences Between RD Curves, ITU-T
SC16/Q6, VCEG-M33, Apr. 2001.

Jian Xiong received the B.Sc. degree in computerscience and technology from Anhui University of
Finance and Economics, Bengbu, China, in 2007,
the M.Sc degree in computer application technology
from Xihua University, Chengdu, China, in 2010,
and the Ph.D. degree in single and information
processing from the University of Electronic Scienceand Technology of China, Chengdu, China, in 2015.
From February 2014 to May 2014, he was a
Research Assistant with the Image and Video
Processing Laboratory, The Chinese University ofHong Kong, Hong Kong. Since September 2015, he has been a Lecturer with
the College of Telecommunications and Information Engineering, Nanjing
University of Posts and Telecommunications, Nanjing, China. His current
research interests include image and video coding, quality evaluation, andperceptual video coding and processing.

Hongliang Li (SM’12) received the Ph.D. degree inelectronics and information engineering from Xian
Jiaotong University, Xi’an, China, in 2005.
From 2005 to 2006, he was a Research Associatewith the Visual Signal Processing and Communication Laboratory (VSPC), Chinese University of Hong
Kong (CUHK), Hong Kong. From 2006 to 2008, he
was a Postdoctoral Fellow with the VSPC, CUHK.
He is currently a Professor with the School of Electronic Engineering, University of Electronic Scienceand Technology of China, Chengdu, China. He has
authored or coauthored numerous technical articles in international journals andconferences. He is a co-editor of the book Video Segmentation and Its Applications (Springer, 2011). His research interests include image segmentation, object

2159

detection, image and video coding, visual attention, and multimedia communication system.
Dr. Li is a Member of the Editorial Board of the Journal on Visual Communications and Image Representation, and the Area Editor of Signal Processing:Image Communication. He served as a Technical Program Co-Chair in ISPACS
2009, General Co-Chair of the ISPACS 2010, Publicity Co-chair of the IEEE
VCIP 2013, Local Chair of the IEEE ICME 2014, and TPC Member in a numberof international conferences, such as ICME 2013, ICME 2012, ISCAS 2013,
PCM 2007, PCM 2009, and VCIP 2010. He serves as a Technical Program
Co-Chair for IEEE VCIP 2016. He was selected as the New Century Excellent Talent in University, Chinese Ministry of Education, China, in 2008.

Fanman Meng (S’11–M’13) received the Ph.D. degree in single and information processing from the
University of Electronic Science and Technology ofChina (UESTC), Chengdu, China, in 2014.
From July 2013 to July 2014, he was a Research
Assistant with the Division of Visual and Interactive
Computing, Nanyang Technological University, Singapore. He is currently an Associate Professor with
the School of Electronic Engineering, University
of Electronic Science and Technology of China,
Chengdu, China. He has authored or coauthorednumerous technical articles in international journals and conferences. His
research interests include image segmentation and object detection.
Prof. Meng is a Member of the IEEE Circuits and Systems Society. He was
the recipient of the Best Student Paper Honorable Mention Award at the 12th
Asian Conference on Computer Vision 2014 and the Top 10% Paper Award at
the IEEE International Conference on Image Processing 2014.

Qingbo Wu received the B.E. degree in educationof applied electronic technology from Hebei Normal
University, Shijiazhuang, China, in 2009, and is currently working toward the Ph.D. degree in electronicengineering at the University of Electronic Scienceand Technology of China, Chengdu, China.
From February 2014 to May 2014, he was a
Research Assistant with the Image and Video Processing Laboratory, The Chinese University of Hong
Kong, Hong Kong. He is currently a Visiting Studentwith the Department of Electrical & Computer Engineering, University of Waterloo, Waterloo, ON, Canada. His research interests
include image/video coding, quality evaluation, and perceptual modeling andprocessing.

King Ngi Ngan (F’00) received the Ph.D. degree inelectrical engineering from Loughborough University, Loughborough, U.K., in 1982.
He is currently a Chair Professor with the Department of Electronic Engineering, Chinese Universityof Hong Kong, Hong Kong. He was previously
a Full Professor with the Nanyang Technological
University, Singapore, and the University of Western
Australia, Crawley, WA, Australia. He has been
appointed Chair Professor with the University ofElectronic Science and Technology, Chengdu, China,
under the National Thousand Talents Program, since 2012. He holds honorary
and visiting professorships at numerous universities in China, Australia, andSoutheast Asia. He has authored or coauthored 3 books, 7 edited volumes, andover 370 refereed technical papers, and has edited 9 special issues in journals.
He holds 15 patents in the areas of image/video coding and communications.
Prof. Ngan is a Fellow of the IET and IEAust, and was an IEEE Distinguished Lecturer from 2006 to 2007. He served as Associate Editor of the IEEE
TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, theJournal on Visual Communications and Image Representation, the EURASIP
Journal of Signal Processing: Image Communication, and the Journal ofApplied Signal Processing. He was Chair or Co-Chair for a number of prestigious international conferences on image and video processing, including the2010 IEEE International Conference on Image Processing, and served on theAdvisory and Technical Committees of numerous professional organizations.