2490

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

Blind Image Quality Assessment Based on
Rank-Order Regularized Regression
Qingbo Wu , Member, IEEE, Hongliang Li, Senior Member, IEEE, Zhou Wang, Fellow, IEEE,
Fanman Meng, Member, IEEE, Bing Luo, Wei Li, and King N. Ngan, Fellow, IEEE

Abstract—Blind image quality assessment (BIQA) aims toestimate the subjective quality of a query image without accessto the reference image. Existing learning-based methods typicallytrain a regression function by minimizing the average error
between subjective opinion scores and model predictions. However,minimizing average error does not necessarily lead to correct
quality rank-orders between the test images, which is a highly
desirable property of image quality models. In this paper, we
propose a novel rank-order regularized regression model toaddress this problem. The key idea is to introduce a pairwise rankorder constraint into the maximum margin regression framework,
aiming to better preserve the correct perceptual preference. To thebest of our knowledge, this is the first attempt to incorporate rankorder constraints into margin-based quality regression model. By
combing with a new local spatial structure feature, we achieve
highly consistent quality prediction with human perception.
Experimental results show that the proposed method outperforms
many state-of-the-art BIQA metrics on popular publicly available
IQA databases (i.e., LIVE-II, TID2013, VCL@FER, LIVEMD,
and ChallengeDB).
Index Terms—Image quality assessment, rank-order regularized
regression.

I. INTRODUCTION
ITH the rapid development of multimedia and networking technologies and services, the volume of digital
image data has been growing explosively in recent years. With
the wide spread of intelligent mobile devices (smartphones,
tablets, etc.), it has become incredibly convenient for common
consumers to acquire, edit and share images in their daily

W

Manuscript received September 21, 2016; revised February 12, 2017; accepted April 22, 2017. Date of publication May 2, 2017; date of current version
October 13, 2017. This work was supported in part by the National Natural Science Foundation of China under Grant 61601102, Grant 61525102, and Grant
61502084, and in part by the Natural Sciences and Engineering Research Council of Canada. The associate editor coordinating the review of this manuscript
and approving it for publication was Dr. Lingfen Sun. (Corresponding author:
Qingbo Wu.)
Q. Wu, H. Li, F. Meng, B. Luo, and W. Li are with the School of Electric Engineering, University of Electronic Science and Technology of China,
Chengdu 611731, China (e-mail: wqb.uestc@gmail.com; hlli@uestc.edu.cn;
fmmeng@uestc.edu.cn; mathild1987@163.com; weili.cv@gmail.com).
Z. Wang is with the Department of Electrical and Computer Engineering,
University of Waterloo, Waterloo, ON N2L 3G1, Canada (e-mail: Z.Wang@
ece.uwaterloo.ca).
K. N. Ngan is with the Department of Electronic Engineering, The Chinese
University of Hong Kong, Hong Kong, China, and also with the School ofElectronic Engineering, University of Electronic Science and Technology ofChina, Chengdu 611731, China (e-mail: knngan@ee.cuhk.edu.hk).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TMM.2017.2700206

lives. However, various annoying distortions may degrade thesubjective quality of an image during its acquisition, storage
and transmission processes. Recently, perceptual image quality
evaluation has received great attention due to its important rolein the acquisition, compression, transmission, enhancement anddisplay of images [1]–[6]. In many applications, the reference
image and distortion information are unavailable, and thus thedevelopment of general purpose blind image quality assessment(BIQA) methods becomes highly desirable.
In practice, a BIQA model may be required to answer two
fundamental questions: 1) How good the quality of an image
is, i.e., the absolute perceptual quality; 2) Whether one image
has better quality than another, i.e., the relative quality between
images. To date, many BIQA algorithms have been proposed,
but most of them focus on estimating the absolute perceptual
quality, without specific mechanisms to maintain the correct
rank-orders between images.
Existing general purpose BIQA approaches maybe roughlyclassified into four categories. The first class of methods typically train a regression function to map quality-sensitive features
to the subjective quality scores. In [7], Mittal et al. introduced
the mean subtracted contrast normalized (MSCN) coefficients todescribe the natural scene statistics (NSS) in the spatial domain,
and then support vector regression (SVR) is employed to generate the predicted quality score. In [8], Moorthy et al. developed
NSS features in the wavelet domain and the subjective quality is generated with two steps, i.e, distortion identification anddistortion-specific regression. Gao et al. [9] further enriched the
NSS features in the wavelet domain and multiple kernel learning (MKL) is introduced to improve the SVR performance. The
second class of approaches try to estimate the subjective image
quality by pooling the human opinion scores from labeled samples. In [10], He et al. designed a sparse representation based
method to determine the weights assigned to each differential
mean opinion score (DMOS). In [11], Ye et al. discussed aclustering based weighting method. In [12], a KNN based labeltransfer model was explored for fusing multiple image quality
labels. The third class of methods predict the perceptual image quality by directly measuring the similarity between thequery image and the pristine image in the feature space. In [13],Mittal et al. utilized the probabilistic latent semantic analysis(pLSA) to learn the latent topics. For a query image, its estimated loadings across all topics are compared with all pristine
images, whose average dot product is used as the predicted
quality. In [14], Mittal et al. further developed a multivariate

1520-9210 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

WU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSION

2491

Fig. 1. Diagram of rank-order regularized regression. The absolute opinion score constraint drives the regression function to improve the prediction accuracy ofeach image. The perceptual rank-order constraint refines the regression function to improve the consistency of the quality rank-order of each pair of images.

Gaussian (MVG) model to describe the statistics of MSCN coefficients. The predicted image quality is represented as the
distance of the MVG features between the query and pristine
images. The fourth class of approaches arise with the booming
deep neural network, which aims at end-to-end optimization of
the feature extractor and quality regressor simultaneously. In[15], Kang et al. explored a deep convolutional neural network(CNN) based method, where the deep learner is fed by the local patches extracted from the human-rated images. Similarly,
Bosse et al. [16] utilized a popular CNN architecture in [17],and fine tuned it to adapt to the BIQA task.
Among the aforementioned algorithms, the first category of
methods show the superiority in producing high prediction accuracy for perceptual image quality. Although different regression
models [9], [18], [19] are explored for BIQA, they share one
common feature. That is, only the absolute opinion score isutilized in training the perceptual quality regressor, and the relative perceptual rank-orders are not imposed. A mean prediction
error based cost function is insensitive to mistaken rank-orders.
For example, suppose there are two human opinion scores a andb in the training set (a > b). Each one has two alternative predictions with the same absolute error x, i.e., â = a ± x, b̂ = b ± x,
and a − x < b + x. When the predictions are (a − x, b + x),
it is clear that the rank-order is wrong. Apparently, one would
prefer the predictions to be (a − x, b − x), (a + x, b + x)or (a + x, b − x), which maintain the correct rank-order.
However, existing methods are ignorant to this, because all
combinations share the same mean prediction error x. Therefore, it is meaningful to adopt a robust regression model, which
could enforce the correct perceptual rank-orders between theimages.
In this paper, we propose a general purpose BIQA algorithm,
which contains two main contributions:
1) We develop a novel Rank-order Regularized Regression(R3 ) model. Its diagram is shown in Fig. 1, which includes two types of constraints. On the one hand, theabsolute opinion score constraint attempts to associateeach single image with an accurate quality score. On theother hand, the perceptual rank-order constraint tries topreserve the correct perception rank-order for each pairof images. In the proposed algorithm, we convert thispairwise constrained regression problem into a quadratic

programming problem [20], [21] and use the interior-point
algorithm [22], [23] to solve it.
2) A new image feature is introduced by computing thejoint statistics of the spatial intensity variations and the
distribution of the neighboring pixels. Both the multiscale and Lab color perception properties are explored
to enhance the discriminative power of this quality-aware
feature.
The proposed regression model and image feature lead tosignificantly improved quality prediction accuracy of the proposed method against state-of-the-art BIQA metrics on the
LIVE-II, TID2013, VCL@FER, LIVEMD and ChallengeDB
databases.
The rest of this paper is organized as follows. Section II briefly
reviews the SVR model and relevant rank learning methods. The
proposed algorithm is presented in Section III. The experimental
results are discussed in Section IV. Finally, Section V concludes
this paper.
II. RELATED WORKIn existing BIQA methods, SVR [19], [24] is one of the mostwidely used regression tools for bridging the gap between image
features and the subjective image quality. Let L = {(xi , yi )}denote the training set, which consists of n samples. xi ∈ Rd
is the input feature vector of the ith sample, and yi ∈ R isthe corresponding subjective quality label. SVR aims to learn
a function f (xi ), which maps xi to yi as closely as possible.
f (xi ) is typically a linear function, i.e.,f (xi ) = wT Φ(xi ) + b(1)

where w and b denote the linear weight vector and the bias, respectively, and Φ(·) represents the feature space transformation.
The parameters w and b can be learned by solving the following optimization problem:
1min ∗ w2 + C(ξi + ξi∗ )w ,ξ i ,ξ i 2
is.t. yi − wT Φ(xi ) − b ≤  + ξi , ∀iwT Φ(xi ) + b − yi ≤  + ξi∗ , ∀i
ξi ≥ 0, ξi∗ ≥ 0

(2)

2492

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

where  denotes the tolerated precision, ξi and ξi∗ denote thetraining loss.
It is noted that the above model does not explicitly penalize rank-order mistakes, which may limit it from producing
high rank-order correlations with the human perception. Fortunately, a wealth of studies about rank learning provide us withmany useful methodologies to address this problem. In [25],Herbrich et al. discussed a margin based framework for learning the ranking function, where a pairwise rank constraint is utilized. In [26], Burges et al. introduced the probabilistic ranking
function, which can be learned by a gradient descent method. In[27], Hüllermeier et al. further decomposed the ranking problem
into an ensemble of binary classification problems to accountfor the pairwise preferences.
Following the pairwise comparison framework, some interesting attempts were explored for BIQA recently. In [28], Chenet al. used the rank SVM to evaluate the perceptual rank-order
between two enhanced images. In [29], Yan et al. employed
the multiple additive regression trees to predict the perceptual
rank-order for the color images enhanced with different parameters. In [30], Reibman et al. proposed a probabilistic predictor
to measure the confidence of the pairwise preceptual preference
between two images. Although these rank learning methods
cannot be directly used for estimating the absolute perceptual
quality of a single image, their rank constraints inspire us to extend SVR to a more general rank-order regularized regression
model.
III. PROPOSED APPROACH
A. Rank-Order Regularized Regression (R3 )In the proposed regression model, we incorporate a pairwise
rank constraint into (2), and rewrite the primal problem as



1
2
∗
w
+
C
ξ
+
C
ξ
+
C
ηij
min
1
i
2
3
i
w ,ξ i ,ξ i∗ ,η i j 2
iiij
s.t. yi − wT Φ(xi ) − b ≤  + ξi , ∀iwT Φ(xi ) + b − yi ≤  + ξi∗ , ∀iwT Φ(xi ) − wT Φ(xj ) ≥ ε − ηij , ∀(i, j) ∈ P
ξi ≥ 0, ξi∗ ≥ 0, ηij ≥ 0(3)where P = {(i, j) : yi > yj + Δ} and Δ is a perceptual difference threshold which eliminates the pairwise training samples
of indistinguishable differences in subjective quality scores. C1 ,C2 and C3 are parameters to maintain a trade-off between minimizing the prediction error and preserving the pairwise rankorders.
To illustrate the roles of different constraints, a simulated example is shown in Fig. 2, where five data samples are included
in all plots, and a linear model is used to predict the vertical axis
y from the horizontal axis x. Two drastically different prediction
models are obtained (top and bottom), with model parameters(w1 , b1 ) and (w2 , b2 ), respectively. The left plots illustrate thepenalties of prediction error, where the prediction error terms ξand ξˆ are similar to each other. Therefore, the two linear mod-Fig. 2. Comparison of the regression performances under different model
parameters w and b. The left plots show the prediction error with respect to theabsolute opinion score. The right plots show the predicted pairwise rank-order.
The x-coordinate represents the observation data and the y-coordinate indicates
the target variable. The red bounding boxes highlight two samples whose ranks
are correctly predicted in (a) and mistaken in (b).

els perform similarly. However, the predicted ranks are very
different. For example, comparing the samples i and j in theright plots, since yi is significantly higher than yj , one would
expect i to be ranked higher than j. Obviously, such an expected
rank-order is well preserved by the top linear model, but not thebottom one, where wT (xi − xj ) < 0. Since our pairwise rankorder constraint requires wT (xi − xj ) ≥ ε − ηij , the mistaken
ranks would increase ηij and avoid selecting the parameters w2
and b2 . This demonstrates that an additional rank-order constraint can greatly help guide the regression training process toselect a better model.
To extend the linear prediction function, we further explore
its kernel version, as in [19], [25]. Moreover, the constrained
optimization problem of (3) can be converted to a unconstrained
optimization problem by introducing Lagrangian loss functions
L=




1
w2 + C1
ξi + C2
ξi∗ + C3
ηij
2
i
i
ij
+



αi (yi − wT Φ(xi ) − b −  − ξi )

i

+


i

+



αi (wT Φ(xi ) + b − yi −  − ξi∗ )



αij
wT (Φ(xi ) − Φ(xj )) − ε + ηij

(4)

ij

are the Lagrange multipliers.
where αi , αi and αij
According to the Karush-Kuhn-Tucker (KKT) conditions[24], the solution that minimizes the primal problem of (4)is given by solving the saddle point, which corresponds to vanishing the partial derivatives of L w.r.t. the primal variables

WU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSION

2493(w, b, ξi , ξi∗ , ηij ). Meanwhile, the constraints and the Lagrange
multipliers should satisfy the following requirements
1) Primal feasibility:yi − wT Φ(xi ) − b ≤  + ξi , ∀iwT Φ(xi ) + b − yi ≤  + ξi∗ , ∀iwT Φ(xi ) − wT Φ(xj ) ≥ ε − ηij , ∀(i, j) ∈ P.

(5)

By replacing the prediction function (10) using (13), we reduce the number of parameters from n + m to n, which significantly simplifies the solution of the R3 problem. In addition, as
discussed in [32], the penalty for w could also be achieved bysuppressing the coefficient vector α = [α1 , . . . , αn ]T . Accordingly, the objective function in (3) can be rewritten as



min
α2 + C1
ξi + C2
ξi∗ + C3ηij∗α ,ξ i ,ξ i ,η i j2) Dual feasibility:
αi , αi , αij≥ 0, ∀ i and (i, j) ∈ P.

(6)

is.t. yi −nαi (yi − wT Φ(xi ) − b −  − ξi ) = 0αi (wT Φ(xi ) + b − yi −  − ξi∗ ) = 0wT (Φ(xi ) − Φ(xj )) − ε + ηij = 0.
αij

i

(7)

ij

(8)
Given ∂L/∂w = 0, we can deduce the w by





Φ(xi ) − Φ(xj ) .
w=(αi − αi )Φ(xi ) −
αij

(9)

ij

Replacing αi − αi with βi , we obtain the kernel version of
the prediction function by plugging (9) into (1)





K(xi , x) − K(xj , x) + b
f (x) =βi K(xi , x) −
αij
ij

(10)
where K(·, ·) is the kernel function.
Let n denote the number of the training images, and m thenumber of all their pairwise combinations, where m = n (n2−1) .
From (10), we can find that the maximum number of the support
vectors could be up to n + m. Since m rapidly increases with n,the complexities of both the training and testing processes of theregression model would greatly increase, making it intractable.
To address this issue, simplification is necessary.
Inspired by the “Representer Theorem” in [31], we knowthat our loss function in (4) corresponds to a special case of
the arbitrary cost function c which satisfies c (X × R2 )n →R ∪ ∞. Let h denote a strictly monotonically increasing realvalued function on [0, ∞), and F the function set∞
αi K(xi , x),
F = f ∈ X |f (x) =αi ∈ R, xi ∈ X , f  < ∞ .
i=1

(11)

Then any f ∈ F minimizing the regularized risk function
c (x1 , y1 , f (x1 ), . . . , (xn , yn , f (xn )) + h(f )(12)would admit a representation form off (x) =

n

i=1

ij

αu K(xu , xi ) ≤  + ξi , ∀iαi K(xi , x).

αu K(xu , xi ) − yi ≤  + ξi∗ , ∀iu =1Particularly, the derivative on w is represented by





∂L/∂w = w+
Φ(xi ) − Φ(xj ) .
(αi − αi)Φ(xi ) +
αij

i

i

u =1

3) Complementary slackness:

i

n


(13)

n




αu K(xu , xi ) − K(xu , xj ) ≥ ε − ηij ,u =1∀(i, j) ∈ P
ξi ≥ 0, ξi∗ ≥ 0, ηij ≥ 0.

(14)Here, the optimization for (14) is a quadratic programming (QP)
problem, which can be solved by the interior-point algorithm[23]. Furthermore, inspired by the sample selection scheme inlarge-scale classification [33], [34], we introduce a k-nearestneighbor (KNN) based method to speed up the solving of (14)by limiting the size of the pairwise training samples. Particularly,
it follows three steps to train the regressor, i.e.,1) Compute the chi-square distances [35] of the features between the test image and all training samples, based on
which we find the KNN of the test image.
2) Use the KNN to construct the training set. Then, the parameter α of (13) is learned from the 2K prediction error constraints and K (K2 −1) − l pairwise rank-order constraints as in (14), where l is the number of the pairwise
training samples that do not pass the perceptual difference
threshold condition in P.
3) Use the learned regression function to predict the subjective quality score for the test image.
In this way, the number of the support vectors is limited tobe less than or equal to K, which corresponds to the training
samples located within the soft-margin in (14).
After building the R3 model, we consider the differences ofthe deviations caused by different distortion types and estimate
the image quality with two steps, which are similar to [8], [36].
Firstly, an SVM [37] classifier is used to identify the distortion
type of a test image. Then, a distortion-specific regressor isused to predict its perceptual quality, where the training subset
is identified to share the same distortion type with the test image
by the SVM classifier.
B. Local Spatial Structure Feature
The structural information of an image is contained in both thespatial intensity variations and the dependencies between neighboring pixels. Existing NSS features work well in describing thestatistical characteristic of the spatial intensity variations. The

2494

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

property of the spatial neighboring interactions has also been exploited. For example, Mittal et al. [7] investigated the productsof pairwise neighboring pixels in four directions. An asymmetric generalized Gaussian distribution (AGGD) model is used todescribe their distribution. In [9], pairwise mutual information
between 17 neighboring coefficients and one central coefficientis computed in the wavelet domain.
For local spatial structure (LSS), a complete description
should contain the joint statistics between all neighboring pixels,
which cannot be fully achieved from pairwise relationships only.
To overcome this limitation, many local binary pattern (LBP)
based methods [12], [38]–[40] were proposed. In comparison
with pairwise neighboring statistics, LBP captures the relative
relationships between the central pixel and all of its neighbors.
Inspired by the success of LBP, we focus on extracting image features based on LSS. Because LBP misses the magnitude
information of the spatial contrast, we incorporate a more comprehensive joint statistic of LSS which is referred to as thecompleted LBP (CLBP) [35]. Particularly, CLBP utilizes threecomponents to describe the local image structure, i.e., the central
pixel’s intensity, and the sign and the magnitude of the neighboring pixel’s local difference. The three components are firstencoded into binary patterns, which are denoted by CLBP C,
CLBP S and CLBP M , respectively. The joint statistics ofthese encoded patterns are used as the image features.
According to [35], the code of the central intensity’s binary
pattern is given by
1, x ≥ ct(x, c) =(15)
CLBP CP ,R = t(gc , cI ),0, x < cwhere gc denotes the gray level of the center pixel. The subscript
P denotes the number of neighboring pixels and R is the radiusbetween the center pixel and its neighbors. cI is a threshold set
as the average intensity of all the pixels in an image.
The encoding of the local difference’s sign can be expressedas P −1p=0 t(gp − gc , 0), if U (LBPP ,R ≤ 2)
CLBP SP ,R =P + 1,
otherwise(16)
U (LBPP ,R ) = |t(gP −1 − gc , 0) − t(g0 − gc , 0)|
+

P
−1


|t(gi − gc , 0) − t(gi−1 − gc , 0)| (17)i=1where U (LBPP ,R ) denotes the discontinuities of the binarypresentation in a circular shape of neighboring pixels [41].
The code of the local difference’s magnitude mp is definedas
CLBP MP ,R =P
−1
t(mp , c)2p

(18)

p=0

where c denotes the threshold, which is set as the average valueof all mp ’s in an image.

The image feature sP ,R can be interpreted as the joint distribution of all the three binary patterns, i.e.,sP ,R = hist3(CLBP CP ,R , CLBP SP ,R , CLBP MP ,R )(19)
where hist3(·, ·, ·) is the 3D histogram operator.
Let nC (P ), nS (P ) and nM (P ) denote the number of thebinary patterns for the central intensity, sign and magnitudeof the neighboring pixels’ local differences, respectively. Let
D(sP ,R ) denote the dimension of the CLBP feature, whichequals to nC (P ) × nS (P ) × nM (P ). According to [35], thevalues of D(sP ,R ) are given by⎧⎨ 2 × 10 × 10, P = 8(20)
D(sP ,R ) = 2 × 18 × 18, P = 16⎩
2 × 26 × 26, P = 24.
Inspired by the hierarchical property of visual perception[42]–[44], we extract the multi-scale CLBP feature under threeparameter settings for (P, R), i.e., (8, 1), (16, 2) and (24, 3).
In addition, we extract the CLBP features from all three color
channels of a color image, which is implemented in the perceptually uniform Lab color space [45]. In Fig. 3, we show thediagram of the multi-scale feature extraction process for the L
component. Let S L , S a and S b denote the multi-scale featuresin the three color components, respectively, which are expressedas
S L = [sL8,1 , sL16,2 , sL24,3 ],S a = [sa8,1 , sa16,2 , sa24,3 ],S b = [sb8,1 , sb16,2 , sb24,3 ].

(21)

Finally, the proposed multi-scale and multi-channel LSS feature set S is given by
S = [S L , S a , S b ].

(22)

IV. EXPERIMENTAL RESULTS
A. Procedures
To evaluate the performance of the proposed algorithms,
we conduct the experiments on three publicly available
benchmark databases, i.e., LIVE-II [46], TID2013 [47] andVCL@FER [48].
1) LIVE-II: The LIVE-II IQA database is composed of 29
original images and 779 distorted versions of them under
5 distortion types, which include 169 JPEG2000 (JP2K)
compression images, 175 JPEG compression images, 145
additive Gaussian White Noise (WN) images, 145 Gaussian Blur images, and 145 Fast Fading (FF) images. Eachdistorted image is associated with a Differential Mean
Opinion Score (DMOS), which ranges from 0 to 100 anda larger value indicates worse perceptual quality.
2) TID2013: The TID2013 IQA database includes 25
reference images and 3000 distorted versions of themunder 17 distortion types, which are additive Gaussian
WN, additive noise in color components (ANC), spatiallycorrelated noise (SCN), masked noise (MN), highfrequency noise (HFN), impulse noise (IN), quantization

WU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSION

2495

Fig. 3. Diagram of the multiscale CLBP feature extraction. Step 1: Generate three encoded maps, i.e., CLBP_S/M/C, for the local contrast’s sign, magnitudeand central intensity, respectively. Step 2: Compute the 3D joint histogram for CLBP_S/M/C, where the x/y/z-axis correspond to the values of three encoded maps
and a warmer color denotes a higher frequency of occurrence. Step 3: Reshape the 3D histogram into the 1D vector sL
P , R and concatenate the multiscale vectors.

noise (QN), Gaussian Blur, image denoising (ID), JPEG
compression, JP2K compression, JPEG transmission
errors, JPEG2000 transmission errors (JPEGTE), Non
eccentricity pattern noise (NEPN), Local block-wise
distortions of different intensity (LBDDI), Mean shift(MS) and Contrast change (CC). Each distorted image
is assigned a Mean Opinion Score (MOS), which ranges
from 0 to 9 and a larger value indicates better subjective
quality.
3) VCL@FER: The VCL@FER IQA database consists of 23reference images and 552 distorted versions of them under
4 distortion types, which include additive Gaussian WN,
Gaussian Blur, JP2K and JPEG. Each distorted image has
a MOS value, which ranges from 0 to 100.
Following the same criterion in [7], [8], [36], [49], we divide the LIVE-II database into non-overlapped training andtesting sets. Particularly, we randomly select 23 of the 29
reference images and their associated distorted images toconstruct the training set. The rest of the images are used
for testing. We conduct the random splitting evaluation 100
times. The median values of the indices across 100 trials arereported.
Following the approach used by the video quality experts
group (VQEG) [50], the objective quality scores are first mapped
to the subjective scores via a four-parameter logistic function,
whose coefficients are solved via the iterative least squares estimation [51]. Two common measures are then employed toevaluate the performance of different BIQA algorithms, which
include the Pearson’s linear correlation coefficient (PLCC) andthe Spearman’s rank-order correlation coefficient (SROCC) between the predicted quality metric Q and the ground-truth humanperception (e.g, DMOS or MOS).

B. Implementation Details
In implementing the regression, the YALMIP toolbox [52]with the Gurobi solver [53] is used to model and solve the optimization problem in (14). The polynomial kernel is employed inthis experiment. We set the perceptual difference threshold Δ to
10, which is obtained by rounding off the mean standard deviation of all DMOS in LIVE-II database. The model parameters
C1 ∼ C3 are determined by cross validation.
To better understand the impact of R3 , we implement two
versions of the proposed regression method. The first is a partial version (2) that does not involve the rank-order constraint.
The second is the full version (3) that includes the rank-orderregularization.
C. Parameter Selection for K
In this section, we investigate how the prediction accuracy
and complexity vary with the parameter K in the KNN method.
Particularly, we implement the proposed BIQA algorithm withdifferent K on the LIVE-II database, where K ranges from 5
to 40 with an interval of 5. The training and testing sets are
obtained by randomly splitting the LIVE-II database 100 times,
as discussed in Section IV-A. The median SROCC value is used
to evaluate the prediction accuracy and the median running time
is used to measure the complexity. The results are reported inFig. 4.
As shown in Fig. 4(a), the SROCC performances of the proposed method with and without R3 monotonically go up as K
increases from 5 to 40, but the increase becomes slower when
K is larger than 20. When K is larger than 30, SROCC starts toconverge. By contrast, for both the proposed method with andwithout R3 , the running time is always monotonically increas-

2496

Fig. 4. Median SROCC and running time in seconds versus the parameter
K as testing on the LIVE-II database. (a) SROCC variation. (b) Running time
variation.

ing. In particular, the running time of the proposed method with
R3 grows much faster due to the increased cost in computing
the pairwise rank-order term. To achieve a balance between accuracy and complexity, we set the parameter K to 30 in the restof the paper, and there are at most 30 support vectors needed forthe quality regressor.
D. Prediction Accuracy and Monotonicity
A good BIQA metric should satisfy two requirements: 1) Predicts perceptual quality accurately with small prediction error,
which can be measured by PLCC; 2) Predicts perceptual quality
with high monotonicity, which can be evaluated by SROCC.
In this section, we conduct such performance evaluation on the
LIVE-II database. Both the proposed method with and without
R3 are compared with classical full-reference (FR) IQA metrics(PSNR, SSIM [54]), and state-of-the-art BIQA metrics (BIQIIEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

Fig. 5. Scatter plots of the predicted quality index Q versus DMOS for thetest set with median SROCC. The x-axis is the predicted quality index Q and
the y-axis is the DMOS value. The red line represents the ideal linear correlated
line. (a) Proposed without R 3 . (c) Proposed w/R 3 .

[36], DIIVINE [8], BLINDS-II [49], BRISQUE [7], NIQE [14]and QAC [55]).
In Fig. 5, we show the scatter plots of the proposed methods
for the test set with median SROCC. It can be seen that thepredicted image quality scores show a nearly linear relationshipwith respect to DMOS, where most samples compactly gather
around the diagonal line, i.e., “Q = DMOS”. It demonstrates thatthe model predictions are highly consistent with human perception. By comparing Fig. 5(a) and 5(b), we can observe that thestrongest impact of R3 is on the “outlier” image samples, which
create strong rank-order penalties that led to significant changes
in the behavior of the regressor. As a result, these samples are
more tightly clustered towards the diagonal line when the proposed R3 approach is applied. To quantify the impact of R3
in Fig. 5, we further compute the number of incorrectly ranked
image pairs, which is represented by Poutlier = Nij δi,j . Par-WU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSION

2497

TABLE IMEDIAN PLCC ACROSS 100 TRAIN-TEST
TRIALS ON THE LIVE-II IQA DATABASE
Distortion

JP2K

Metric

Type

PSNR
SSIM
BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
OAC
Proposed w/o R3
Proposed w/ R3

FR
FR
Blind
Blind
Blind
Blind
Blind
Blind
Blind
Blind

0 896
0.937
0 750
0 922
0 963
0 923
0.926
0 865
0.938
0.965

JPEG

0 860
0.928
0 630
0 921
0.979
0 974
0.952
0 944
0.945
0 974

WN

0 986
0.970
0 968
0 988
0 985
0 985
0.976
0 9 18
0.982
0.989

Blur

0 783
0.874
0 800
0 923
0 948
0 951
0.943
0 911
0.957
0.972

TABLE III
STANDARD DEVIATIONS IN PERFORMANCE OF THE PROPOSED METHODWITHOUT AND WITH R 3 ACROSS 100 TRIALS ON THE LIVE-II DATABASE
FF

0 890
0.943
0 722
0 888
0 864
0 903
0.879
0 825
0.917
0.948

All

0 872
0.945
0 740
0 917
0 923
0 942
0.905
0 863
0.943
0.966

TABLE IIMEDIAN SROCC ACROSS 100 TRAIN-TEST
TRIALS ON THE LIVE-II IQA DATABASE
Distortion
Metric

Type

PSNR
SSIM
BIQI
DIIVINE
BLINDS-II
BRISQUENIOE
OAC
ProDosed w/o R3
Proposed w/ R3

FR
FR
Blind
Blind
Blind
Blind
Blind
Blind
Blind
Blind

JP2K
JPEG
WN
Blur
FF
All

JPEG

WN

Blur

FF

All

0 890
0.932
0 736
0 913
0.951
0 914
0.919
0 862
0.928
0.952

0 841
0.903
0 591
0 910
0.942
0.965
0.942
0 936
0.928
0.954

0 985
0.963
0 958
0 984
0.978
0 979
0.972
0 951
0.981
0.984

0 782
0.894
0 778
0 921
0.944
0 951
0.933
0 913
0.965
0.968

0 890
0.941
0 700
0 863
0.862
0 877
0.864
0 823
0.907
0.923

0 876
0.947
0 726
0 916
0.920
0 940
0.909
0 868
0.941
0.960

ticularly, N is the total number of all possible combinations forimage pairs in the test set, and δi,j denote the counting function,
which is given by
1, if (DM OSi − DM OSj ) · (Qi − Qj ) < 0
δi,j =(23)
0, otherwise
where DM OSi and Qi denote the ground-truth and the predicted quality for the ith image, respectively. Corresponding to
Fig. 5, the Poutlier for the proposed method w/ and w/o R3 are
1476 and 1789, respectively. That is, the proposed R3 model
could significantly reduce the “outlier” image samples, whose
rank-orders are mistaken.
The quantitative comparisons of different IQA metrics arepresented in Tables I and II, which list the median PLCC andSROCC results, respectively. For clarity, the optimal BIQA results in terms of PLCC and SROCC are highlighted in boldface
for each distortion type. It can be seen that for each distortion type, the proposed methods with and without R3 always
achieve the top three PLCC or SROCC results. In particular,
the proposed method with R3 performs the best on almost allindividual distortion types other than JPEG, for which its performance is very close to the best result obtained by the BRISQUE
metric.

Proposed w/R3

PLCC

SROCC

PLCC

SROCC

0.007
0.008
0.004
0.012
0.013
0.005

0.009
0.015
0.006
0.017
0.025
0.005

0.010
0.008
0.004
0.016
0.022
0.008

0.012
0.014
0.005
0.021
0.027
0.007

TABLE IVMEDIAN RMSE OF THE PROPOSED METHOD WITHOUT AND WITH
3
R ACROSS 100 TRIALS ON THE LIVE-II DATABASE

Proposed w/o R3
Proposed w/ R3

JP2K

Proposed w/o R3

JP2K

JPEG

WN

Blur

FF

All

9.416
6.924

8.448
5.924

5.274
4.183

7.170
5.377

9.291
7.514

8.255
6.202

A challenging task of general purpose BIQA is to accurately
predict the subjective image quality across different distortion
types. On this criterion, the proposed method performs the beston the entire database, which contains all test samples across all
five distortion types. As shown in Tables I and II, both versionsof the proposed methods outperform the other BIQA metrics.
Specifically, the proposed method with R3 achieves the highest
PLCC and SROCC results. Since the rank-order constraint better
preserves the perceptual preference, no matter for each distortion type or the entire database test, the proposed method with
R3 always achieves higher PLCC and SROCC results than thatwithout R3 . In Table III, we show the standard deviations across
100 train-test trials. It can be seen that the standard deviations
are all very small, which demonstrates that the performance ofthe proposed method is robust to variations of the training sets.
In addition to the PLCC and SROCC indexes, the median root
mean square error (RMSE) results of the proposed method w/
and w/o R3 are also reported in Table IV. We can find that the
RMSE of the regression model w/ R3 is smaller than that w/o R3
across all distortion types and the whole test set. It validates thatthe correct rank-order information is also beneficial for better
approximating the human perception of image quality.
To verify that the superiority of the proposed method is statistically significant, we further perform the one-sided t-test [56]on the SROCC results which are generated from 100 train-test
trials. The results are reported in Table V. It can be seen that the
proposed method with R3 is statistically better than all the other
BIQA algorithms. This is consistent with the median SROCC
comparison result in Tables I and II.
E. Classification Accuracy
In a two-step method, distortion-type classification plays
an important role in assigning the correct distortion-specificregression function to the test image. To verify the superiorityof our LSS feature in identifying different distortion types, weinvestigate the median classification accuracy across 100 trials

2498

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

TABLE VSTATISTICAL SIGNIFICANCE ON RELATIVE PERFORMANCE OF BIQA METHODS. A VALUE OF “1”/“0”/“−1” INDICATES
THE ROW ALGORITHM IS STATISTICALLY SUPERIOR/EQUIVALENT/INFERIOR TO THE COLUMN ALGORITHM
BIQI

DIIVINE

BLINDS-II

BRISQUE

NIQE

QAC

Proposed w/o R3

Proposed w/R3

0
1
1
1
1
1
1
1

−1
0
−1
1
−1
−1
1
1

−1
1
0
1
−1
−1
1
1

−1
−1
−1
0
−1
−1
1
1

−1
1
1
1
0
−1
1
1

−1
1
1
1
1
0
1
1

−1
−1
−1
−1
−1
−1
0
1

−1
−1
−1
−1
−1
−1
−1
0

BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
QAC
Proposed w/o R3
Proposed w/ R3

TABLE VI
MEDIAN CLASSIFICATION ACCURACY (%) ACROSS 100
TRAIN-TEST TRIALS ON THE LIVE-II IQA DATABASEDIIVINELSSTABLE VII
SROCC BETWEEN THE PREDICTED QUALITY SCORE AND MOS
ON THE “KNOWN” SUBSETS OF THE TID2013 DATABASE

JP2K

JPEG

WN

Blur

FF

All

Metric

Type

JP2K

JPEG

WN

Blur

All

80.00
91.55

81.10
100.00100.00100.00

90.00
96.67

73.30
80.00

83.75
92.90

PSNR
SSIM
BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
QAC
Proposed w/o R3
Proposed w/ R3

FR
FR
Blind
Blind
Blind
Blind
Blind
Blind
Blind
Blind

0.891
0.949
0.852
0.854
0.901
0.909
0.898
0.894
0.870
0.905

0.919
0.932
0.891
0.668
0.859
0.889
0.864
0.871
0.872
0.890

0.944
0.874
0.807
0.872
0.661
0.851
0.816
0.754
0.809
0.840

0.967
0.970
0.839
0.830
0.870
0.881
0.815
0.885
0.869
0.919

0.924
0.921
0.839
0.796
0.793
0.882
0.812
0.855
0.857
0.886

in Table VI, where DIIVINE based classification is also listed
for comparison. It can be seen that the proposed LSS featurebased classification achieves higher accuracies for all distortion
types. The mostly occurred classification errors are between the
JP2K and FF datasets. This is likely because the FF images werecreated in two steps: JP2K compression followed by simulated
random fast fading channel distortion. As a result, the artifacts
in the FF set are very similar to those in JP2K, especially when
the fast fading errors are minor. Consequently, such classification errors do not lead to strong degradations in the overall
performance of the proposed algorithm.
F. Cross Database Verification
To verify the proposed methods across databases, we follow
the approach in [7], [8], [49]. In particular, we use the entire
LIVE-II database as the training set, and then test the proposed
method on the “known” subset of TID2013 and VCL@FER
databases which include four common distortion types, i.e.,
JP2K, JPEG, WN and Blur.
Tables VII and VIII show the detailed SROCC results between the predicted quality score and MOS on TID2013 and
VCL@FER databases, respectively. For comparison, two classic FR-IQA metrics (i.e., PSNR, SSIM) and six state-of-the-art
BIQA algorithms (i.e., BIQI, DIIVINE, BLINDS-II, BRISQUE,
NIQE and QAC) are also listed. The best SROCC results among
all BIQA metrics are highlighted by boldface.
In the TID2013 database, the proposed method with R3
achieves the best SROCC result for the distortion type of Blur,and the second best for JP2K and JPEG. For the case of all test
samples across four distortion types, it outperforms all the other
BIQA metrics. Similarly, on the VCL@FER database, for JP2K,
JPEG and Blur, the proposed method with R3 obtains the best
SROCC performance. When tested on the entire VCL@FER
database, it is again superior to all the other BIQA algorithms.

TABLE VIIISROCC BETWEEN THE PREDICTED QUALITY SCORE
AND MOS ON THE VCL@FER DATABASE
Metric

Type

JP2K

JPEG

WN

Blur

All

PSNR
SSIM
BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
QAC
Proposed w/o R3
Proposed w/ R3

FR
FR
Blind
Blind
Blind
Blind
Blind
Blind
Blind
Blind

0.852
0.935
0.573
0.752
0.765
0.744
0.800
0.757
0.798
0.810

0.604
0.924
0.583
0.518
0.733
0.711
0.746
0.594
0.785
0.838

0.974
0.914
0.703
0.913
0.894
0.823
0.848
0.882
0.891
0.831

0.779
0.905
0.640
0.850
0.853
0.901
0.904
0.899
0.913
0.915

0.825
0.911
0.617
0.727
0.810
0.812
0.811
0.769
0.805
0.827

In addition, similar to the results in LIVE-II database, the proposed method with R3 performs better than that without R3 inthis cross-database investigation. It verifies that our R3 modelis robust to the visual content variation across different IQA
databases.
In order to show the superiority of the R3 method in ranking
the quality of images, we further build a challenging test set
to compare different BIQA algorithms. Particularly, all human
rated test samples are selected from the VLC@FER database,which has no visual content overlapping with the training
set (i.e., LIVE-II database). For each reference image in the
VLC@FER database, we implement all possible pairwise comparisons between its associated distorted images under4 distor
tion types and 6 degradation levels, which produces 24
2 , i.e.,WU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSIONTABLE IX
PLCC AND SROCC BETWEEN THE PREDICTED QUALITY SCORE AND
MOS ON THE CHALLENGING SUBSET OF VCL@FER DATABASE

2499

TABLE XIMEDIAN SROCC ACROSS 100 TRAIN-TEST
TRIALS ON THE CHALLENGEDB DATABASE

Metric

PLCC

SROCC

Metric

PLCC

SROCC

BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
QAC
Proposed w/o R3
Proposed w/ R3

0.375
0.495
0.578
0.554
0.556
0.451
0.699
0.717

0.363
0.477
0.572
0.566
0.574
0.437
0.700
0.721

BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
QAC
Proposed w/o R3
Proposed w/ R3

0.540
0.556
0.576
0.625
0.498
0.298
0.625
0.642

0.519
0.527
0.542
0.597
0.489
0.318
0.612
0.631

TABLE XMEDIAN SROCC ACROSS 100 TRAIN-TEST
TRIALS ON THE LIVEMD DATABASE

BIQIDIIVINEBLIINDS-II
BRISQUE
NQIE
QAC
Proposed w/o R3
Proposed w/ R3

Part 1

Part 2

All

0.845
0.828
0.884
0.915
0.835
0.714
0.915
0.913

0.793
0.853
0.887
0.876
0.846
0.790
0.895
0.929

0.768
0.822
0.866
0.874
0.833
0.731
0.890
0.911

276 pairs of images. If more than half of the six representative
BIQA metrics (i.e., BIQI, DIIVINE, BLINDS-II, BRISQUE,
NIQE and QAC) mistake the perceptual rank-order of a pair of
aforementioned images, this pair of images are collected into
our challenging test set. Then, we compare the performance ofdifferent BIQA algorithms in ranking the quality of these selected images. The PLCC and SROCC results are reported inTable IX. It can be seen that the proposed methods outperform
all the other methods in terms of both PLCC and SROCC bylarge margins.
G. Multiply Distortion Verification
In the real-world application, the natural images usually undergo multiply distortions during their acquisition and postprocessing process. To verify that the proposed method works wellin this scenario, we further evaluate the performance of R3 on
the multiply distortion database–LIVEMD [57]. More specifically, the LIVEMD contains 15 pristine images and their 450
distorted versions, which are divided into two parts according
to their distortion types. The Parts 1 and 2 are created by adding
JPEG and noise to the blurred images, respectively.
Following the criteria in Section IV-A, we implement the verification based on 100 train-test trials on LIVEMD. The training
set includes 360 distorted images that are generated from 12
randomly selected reference images, and the other 90 distorted
images are used for testing. For comparison, the six state-ofthe-art BIQA algorithms [7], [8], [14], [36], [49], [55] are also
involved in this verification. The median SROCC results havebeen reported in Table X, where the highest value are highlighted
by bold face in each column. It is seen that the quality predictedby R3 model is highly consistent with the human perception on
the multiply distortion images, whose SROCC is up to 0.911 on
the entire test set of LIVEMD. In addition, similar to the resultson the single distortion databases, the proposed method is alsosuperior to all the other BIQA metrics in this investigation.
H. Authentic Distortions Verification
In addition to the graded simulated distortions discussed inprevious sections, we further investigate the performance of the
proposed method in coping with diverse authentic image distortions, which is implemented on the ChallengeDB database[58]. More specifically, there are totally 1162 authentically distorted images in [58], which are captured from diverse mobile
devices.
The test criteria still follows the instruction in Section IV-A,
and the 100 rounds of random train-test trials are implemented
for all BIQA algorithms [7], [8], [14], [36], [49], [55] in comparison. In each train-test split, there are 930 images used fortraining and the rest 232 images are used for testing. The median
SROCC results are reported in Table XI, and the highest value
are highlighted by bold face for clarity. Similar to the resultson the simulated distortions, the proposed method still achieves
the best prediction performance with respect to all the other
BIQA metrics. Meanwhile, the proposed method w/ R3 is still
consistently superior to the method w/o R3 .
I. Cross Feature Verification
To verify the proposed method across feature spaces, we compare the performance between the regression models w/o andw/ R3 constraint by feeding them with six popular quality-aware
image features [7], [8], [14], [36], [49], [55]. More specifically,
we follow the train-test splitting criterion in Section IV-A and
implement the test on LIVE-II database. Let Swo and Sw denote the median SROCC produced from the regression models
w/o and w/ R3 constraint, respectively. The difference valuedR 3 , which is equal to Sw − Swo , is used to evaluate the contribution of R3 model. A positive dR 3 means that the R3 constraint improves the quality prediction accuracy and vice versa.
The experimental results have been shown in Fig. 6, where
the x-axis indicates different features fed to two regression
models and y-axis is the SROCC improvement brought by R3
constraint.

2500

Fig. 6.

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

SROCC improvement brought by R 3 across different features.

TABLE XIIPREDICTION PERFORMANCE OF THE PROPOSED METHODWITH AND WITHOUT R 3 UNDER DIFFERENT KERNELS
Kemel

Polynomial
RBF

Proposed w/o R3

Proposed w/R3

PLCC

SROCC

PLCC

SROCC

0.943
0.943

0.941
0.940

0.966
0.964

0.960
0.957

It is seen that the proposed R3 constraint could consistently
improve the performance of regression model across all six
image features, where dR 3 is always positive as shown in Fig. 6.
In our investigation, the maximum SROCC improvement of R3
is up to 0.052, which is achieved for the BIQI feature. When thestate-of-the-art BRISQUE feature is fed to R3 model, we could
still increase the SROCC by 0.02 with respect to a classic SVRmodel without rank-order constraint.
J. Influence of Different Kernel
Through extensive experiments on five popular databases(i.e., LIVE-II, TID2013, VCLFER, LIVEMD and ChallengeDB), it is validated that the rank-order regularization plays
a positive role in improving the prediction performance towards
BIQA, where the proposed method w/ R3 is consistently superior to the one w/o R3 . To further verify the necessary of R3
in a quality regression model, we compare its influence withthe popular radial basis function (RBF) kernel, which is widelyused to better explore the training data. More specifically, we reimplement the consistency experiment in Section IV-D, and the
RBF kernel is applied to the proposed regression models. The
results are reported in Table XII. It is seen that the non-linear
RBF kernel does not bring PLCC/SROCC improvements withrespect to the polynomial kernel for both the regression models w/ and w/o R3 . As discussed in [59], [60], the kernel type
and parameter selection is usually based on application-domain
knowledge, which does not necessarily lead to a performance
improvement by using one particular kernel. In contrast, when

Fig. 7. SROCC performance comparison for the LSS features extracted fromdifferent scales and color spaces. (a) Regression model without R 3 constraint.
(b) Regression model with R 3 constraint.

we add R3 in the regression model, its PLCC/SROCC performance would consistently outperform the model w/o R3 , nomatter which kernel is used.
K. Contributions of LSS Feature
The proposed LSS feature is extended from classic texture descriptor CLBP [35] by introducing two visual perception clues,i.e., multi-scale and Lab color space convention. To verify thatthese two clues are beneficial in improving the quality prediction accuracy, we investigate the SROCC performance changes
by combining each of the two clues with the CLBP. Following the train-test splitting criterion in Section IV-A, we conduct
the investigation on the LIVE-II database. Both the regression
models w/o and w/ R3 constraint are tested on different combinations. The detailed results have been reported in Fig. 7, where
the x-axis indicates different scales of CLBP and Multi-scalecorresponds to the feature vector of concatenating CLBP8,1 ,
CLBP16,2 and CLBP24,3 . In the legend, we show two color
spaces which are used for extracting the image feature, where
the Gray is initially used in [35] and Lab is employed in our LSSWU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSIONTABLE XIIIMEDIAN SROCC COMPARISON BETWEEN THE
PIXEL-WISE AND BLOCK-WISE LSS FEATURE

2501

TABLE XIVRUNNING TIME (SECONDS) COMPARISON OF BIQA METRICS
MetricPixel-wise
Block-wise 3 × 3
Block-wise 5 × 5
Block-wise 7 × 7

JP2K

JPEG

WN

Blur

FF

All

0.952
0.937
0.894
0.827

0.954
0.939
0.897
0.867

0.984
0.976
0.956
0.930

0.968
0.946
0.911
0.850

0.923
0.892
0.776
0.733

0.960
0.939
0.891
0.841

feature. The y-axis is the median SROCC produced by different
combinations.
It is clear that both the multi-scale and Lab color space convention boost the SROCC performance of BIQA task. As shownin Fig. 7(a), the Multi-scale feature outperforms all the othersingle-scale CLBP features as combing with the regression
model w/o R3 constraint. In the Gray color space, its maximum SROCC improvement is more than 0.05 with respect to
CLBP16,2 . In the Lab color space, the maximum SROCC improvement of Multi-scale feature is close to 0.02 in comparison
with CLBP8,1 . Similarly, when we test on the regression modelw/ R3 constraint, the Multi-scale feature still consistently outperforms the single-scale CLBP features on both of the color
spaces as shown in Fig. 7(b). In addition, the Lab color space
exhibits superior performance in estimating image quality withrespect to the Gray in this investigation. As shown in Fig. 7(a)and (b), the median SROCC of the Lab features is higher thanthe Gray features across all scales. For the regression model
w/o R3 constraint, the maximum SROCC gain between Lab
and Gray features is achieved by CLBP16,2 which could be up
to 0.059. As testing on the regression model w/ R3 constraint,
we can also achieve 0.05 SROCC improvement by extracting
CLBP24,3 from the Lab color space in comparison with the
Gray.
L. Investigation on Block-Wise LSS Feature
The pixel-wise LSS feature describes the spatial complexity
in a rather small scale, where only the central and neighboring
pixels are involved in the comparison. It is interesting to investigate how a larger center and neighbor regions would affectthe discriminating power of the proposed LSS feature. Morespecifically, we analyze the performance of a block-wise LSS
feature, whose center and neighbors are represented by the mean
intensity values of a d × d block whose locations are same withthe settings of CLBP as shown in Fig. 3. Three scales of blocks
are tested in this investigation, which include 3 × 3, 5 × 5, and7 × 7. Similar to Section IV-D, we implement 100 rounds ofrandom splitting trials on the LIVE-II database, where the sizesof training and testing sets are 80% and 20%, respectively. The
median SROCC results are reported in Table XIII. It is seenthat the block-wise LSS is inferior to the pixel-wise LSS feature
for the BIQA task. Meanwhile, a larger block size would bringmore performance degradation. The possible reason could befound from previous texture descriptor related literatures. As
discussed in [61], [62], the block-wise or region based differential operation is widely used for developing robust texture

BIQI
DIIVINE
BLINDS-II
BRISQUE
NIQE
QAC
Proposed w/o R3
Proposed w/ R

Running Time
1.198
54.232217.038
0.565
1.214
0.407
8.694
8.931

descriptors, which aims to reduce its sensitivity to the noise or
distortion. This intention is contrary to the purpose of designing
quality or distortion degree relevant image feature, and could
reduce its performance in the BIQA task.
M. Computational Complexity
To evaluate computational complexity, we compare different BIQA metrics on their running time. The test is performed
on a machine with Intel Core 2 processor of speed 2.0 GHz,
6 GB RAM and Windows 7 64-bit. The BIQA models are implemented using MATLAB2013a.
The running time is computed by evaluating a 512 × 768
color image selected from the LIVE-II database. The detailed
results have been shown in Table XIV . It can be seen thatthe complexity of the proposed methods is moderate in comparison with DIIVINE and BLINDS-II. Meanwhile, the CLBP
feature extraction takes up most computations in the proposed
method, whose running time is 8.125 seconds for each image. It
should be noted that the running time of the proposed methods
is obtained from unoptimized MATLAB code. Since the computation of CLBP is completely independent across scales andcolor channels, a parallel operation can efficiently speed up the
LSS feature extraction process.
In addition, as shown in Table XIV, the proposed methodwith R3 is only a little slower than that without R3 . It indicatesthat the added rank-order constraint term does not significantly
increase the computational cost of the regression model.
N. Discussion
In the proposed R3 model, the parameter Δ provides additional flexibility to manage the pairwise samples which are
involved in the rank-order constraint. More specifically, a larger
threshold Δ would eliminate more pairwise training samples,whose perceptual differences are considered to be indistinguishable. Not surprisingly, the performance of R3 model would converge to the same value with respect to the classic SVR as Δ
removing all rank-order constraints. To investigate this impact,
we repeat the train-test experiments on LIVE-II and change theparameter Δ from 0 to 100 at the interval of 10. The median
SROCC results are shown in Fig. 8. In addition, the running
time is also reported in Fig. 9.
We can find that the median SROCC would gradually decline
as Δ increasing. Since more pairwise samples are eliminated2502

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017training samples for measuring the absolute opinion score andpairwise perceptual preference, respectively.
To facilitate the pairwise comparison, a modified Siamese
network would be more suitable for optimizing the loss functionin (24). In our future work, some more sophisticated networkarchitecture and regularization terms will be further studied
for interpreting the complex multi-target property of humanperception towards image quality. Meanwhile, more perceptual
properties, such as, masking effect and visual saliency would
be studied for building quality-aware features. A more robust
general regressor without regard to rank-order information
would be explored as well.
Fig. 8.

Median SROCC variation with respect to the parameter Δ.

V. CONCLUSION

Fig. 9.

Running time variation of R 3 with respect to the parameter Δ.

by higher Δ, the correcting effect from rank-order constraint
is reduced, which results in the performance degradation of R3
model. When Δ increases to 80, all rank-order constraints are
removed and the performance of R3 becomes converging to
the value of regression model w/o R3 . Meanwhile, as shown in
Fig. 9, the similar downtrend in terms of running time can befounded for R3 model, where the decrease of pairwise samples
reduces the complexity of solving the problem in (14).
In addition, another interesting extension of our R3 model
is to integrate it into a deep neural network. To adapt to theback-propagation computation, we can rewrite the constrained
optimization task in (3) to a derivable unconstrained loss function, i.e.,1 (yi − ω ∗ §i )2 +
L=
Na i
λ·

1 
max{0, ε + ω ∗ (§i − §j )},
Np ij

(24)

∀(i, j) ∈ P
where ω is the kernel parameter for the last fully connected
layer, ∗ denotes the convolution operator, λ is a weight parameterto balance the regression targets between the absolute opinion
score estimation and pairwise rank-order preservation. §i and§j are the outputs from previous convolution layer for the ith
and jth image, respectively. Na and Np are the total number ofIn this paper, we propose a novel general purpose BIQA algorithm, where the main novelty is in the rank-order regularized
regression (R3 ) model and the local spatial structure (LSS) feature. Unlike the conventional methods which focus on minimizing the prediction error for labeled training samples only, our R3
model also tries to preserve perceptual rank-orders. By computing the joint distribution of the encoded central intensity, and thesign and magnitude of contrast, we propose a CLBP based LSS
feature that is able to capture the spatial intensity variations andthe neighboring pixel dependencies simultaneously. Extensive
experiments show that for the LIVE-II, TID2013, VCL@FER,
LIVEMD and ChallengeDB databases, the proposed methodwith R3 performs highly consistent with human perception ofimage quality and shows superior performance over state-ofthe-art BIQA algorithms.
REFERENCES[1] Z. Wang, “Applications of objective image quality assessment methods [applications corner],” IEEE Signal Process. Mag., vol. 28, no. 6,
pp. 137–142, Nov. 2011.
[2] F. Zhang, L. Ma, S. Li, and K. N. Ngan, “Practical image quality metric applied to image coding,” IEEE Trans. Multimedia, vol. 13, no. 4,
pp. 615–624, Aug. 2011.
[3] K. Gu et al., “Saliency-guided quality assessment of screen content
images,” IEEE Trans. Multimedia, vol. 18, no. 6, pp. 1098–1110,
Jun. 2016.
[4] W. Zhou and L. Yu, “Binocular responses for no-reference 3D image quality assessment,” IEEE Trans. Multimedia, vol. 18, no. 6,
pp. 1077–1084, Jun. 2016.
[5] Q. Wu, Z. Wang, and H. Li, “A highly efficient method for blind image
quality assessment,” in Proc. IEEE Int. Conf. Image Process., Sep. 2015,
pp. 339–343.
[6] Q. Wu, H. Li, F. Meng, K. N. Ngan, and S. Zhu, “No reference image quality assessment metric via multi-domain structural informationand piecewise regression,” J. Vis. Commun. Image Represent., vol. 32,
pp. 205–216, 2015.
[7] A. Mittal, A. Moorthy, and A. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Trans. Image Process., vol. 21, no. 12,
pp. 4695–4708, Dec. 2012.
[8] A. K. Moorthy and A. C. Bovik, “Blind image quality assessment: From
natural scene statistics to perceptual quality,” IEEE Trans. Image Process.,
vol. 20, no. 12, pp. 3350–3364, Dec. 2011.
[9] X. Gao, F. Gao, D. Tao, and X. Li, “Universal blind image quality assessment metrics via natural scene statistics and multiple kernel learning,”
IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 12, pp. 2013–2026,
Dec. 2013.
[10] L. He, D. Tao, X. Li, and X. Gao, “Sparse representation for blind image
quality assessment,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog.,
Jun. 2012, pp. 1146–1153.

WU et al.: BIQA BASED ON RANK-ORDER REGULARIZED REGRESSION[11] P. Ye and D. Doermann, “No-reference image quality assessment using visual codebooks,” IEEE Trans. Image Process., vol. 21, no. 7,
pp. 3129–3138, Jul. 2012.
[12] Q. Wu et al., “Blind image quality assessment based on multichannel feature fusion and label transfer,” IEEE Trans. Circuits Syst. Video Technol.,vol. 26, no. 3, pp. 425–440, Mar. 2016.
[13] A. Mittal, G. Muralidhar, J. Ghosh, and A. Bovik, “Blind image quality
assessment without human training using latent quality factors,” IEEE
Signal Process. Lett., vol. 19, no. 2, pp. 75–78, Feb. 2012.
[14] A. Mittal, R. Soundararajan, and A. Bovik, “Making a “completely blind”
image quality analyzer,” IEEE Signal Process. Lett., vol. 20, no. 3,
pp. 209–212, Mar. 2013.
[15] L. Kang, P. Ye, Y. Li, and D. Doermann, “Convolutional neural networks
for no-reference image quality assessment,” in Proc. IEEE Conf. Comput.
Vis. Pattern Recog., Jun. 2014, pp. 1733–1740.
[16] S. Bosse, D. Maniry, T. Wiegand, and W. Samek, “A deep neural network
for image quality assessment,” in Proc. IEEE Int. Conf. Image Process.,
Sep. 2016, pp. 3773–3777.
[17] K. Simonyan and A. Zisserman, “Very deep convolutional networks
for large-scale image recognition,” CoRR, 2014. [Online]. Available:
http://arxiv.org/abs/1409.1556[18] C. Li, A. Bovik, and X. Wu, “Blind image quality assessment using ageneral regression neural network,” IEEE Trans. Neural Netw., vol. 22,
no. 5, pp. 793–799, May 2011.
[19] V. N. Vapnik, The Nature of Statistical Learning Theory. New York, NY,
USA: Springer-Verlag, 1995.
[20] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.:
Cambridge Univ. Press, 2009.
[21] M. Frank and P. Wolfe, “An algorithm for quadratic programming,” Nav.
Res. Logist. Quart., vol. 3, no. 1–2, pp. 95–110, 1956.
[22] R. J. Vanderbei, “Loqo: An interior point code for quadratic programming,” Optim. Methods Softw., vol. 11, no. 1–4, pp. 451–484, 1999.
[23] Y. Ye, Interior Point Algorithms: Theory and Analysis. Hoboken, NJ,
USA: Wiley, 2011, vol. 44.
[24] A. J. Smola and B. Schölkopf, “A tutorial on support vector regression,”
Statist. Comput., vol. 14, no. 3, pp. 199–222, 2004.
[25] R. Herbrich, T. Graepel, and K. Obermayer, Large Margin Rank Boundaries for Ordinal Regression. Cambridge, MA, USA: MIT Press, Jan.
2000, ch. 7, pp. 115–132.
[26] C. Burges et al., “Learning to rank using gradient descent,” in Proc. Int.
Conf. Mach. Learn., 2005, pp. 89–96.
[27] E. Hüllermeier, J. Fürnkranz, W. Cheng, and K. Brinker, “Label ranking by learning pairwise preferences,” Artif. Intell., vol. 172, no. 16,
pp. 1897–1916, 2008.
[28] Z. Chen, T. Jiang, and Y. Tian, “Quality assessment for comparing image enhancement algorithms,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recog., Jun. 2014, pp. 3003–3010.
[29] J. Yan, S. Lin, S. B. Kang, and X. Tang, “A learning-to-rank approach
for image color enhancement,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2014, pp. 2987–2994.
[30] A. Reibman, K. Shirley, and C. Tian, “A probabilistic pairwise-preference
predictor for image quality,” in Proc. IEEE Int. Conf. Image Process.,
Sep. 2013, pp. 413–417.
[31] B. Schölkopf, R. Herbrich, and A. J. Smola, “A generalized representer
theorem,” in Proc. 14th Annu. Conf. Comput. Learn. Theory, Jan. 2001,
pp. 416–426.
[32] O. L. Mangasarian, “Generalized support vector machines,” in Advancesin Large Margin Classifiers. Cambridge, MA, USA: MIT Press, 1998,
pp. 135–146.
[33] E.-H. Han, G. Karypis, and V. Kumar, “Text categorization using weightadjusted k-nearest neighbor classification,” in Proc. Adv. Knowl. Discovery
Data Mining, 2001, vol. 2035, pp. 53–65.
[34] H. Zhang, A. Berg, M. Maire, and J. Malik, “SVM-KNN: Discriminative
nearest neighbor classification for visual category recognition,” in Proc.
IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Jun. 2006,
vol. 2, pp. 2126–2136.
[35] Z. Guo, D. Zhang, and D. Zhang, “A completed modeling of local binary
pattern operator for texture classification,” IEEE Trans. Image Process.,
vol. 19, no. 6, pp. 1657–1663, Jun. 2010.
[36] A. Moorthy and A. Bovik, “A two-step framework for constructing
blind image quality indices,” IEEE Signal Process. Lett., vol. 17, no. 5,
pp. 513–516, May 2010.
[37] C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, pp. 27:1–27:27,
May 2011.

2503[38] J. Wu, W. Lin, and G. Shi, “Image quality assessment with degradation on
spatial structure,” IEEE Signal Process. Lett., vol. 21, no. 4, pp. 437–440,
Apr. 2014.
[39] M. Zhang, J. Xie, X. Zhou, and H. Fujita, “No reference image quality
assessment based on local binary pattern statistics,” in Proc. Vis. Commun.
Image Process., Nov. 2013, pp. 1–6.
[40] M. Panić, D. Ćulibrk, S. Sladojević, and V. Crnojević, “Local binary
patterns and neural networks for no-reference image and video quality assessment,” in Proc. Eng. Appl. Neural Netw., 2013, pp. 388–
395.
[41] T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scaleand rotation invariant texture classification with local binary patterns,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971–987,
Jul. 2002.
[42] V. Bruce, P. R. Green, and M. A. Georgeson, Visual Perception: Physiology, Psychology, & Ecology. New York, NY, USA: Psychology Press,
2003.
[43] S. Hochstein and M. Ahissar, “View from the top: Hierarchies and reverse
hierarchies in the visual system,” Neuron, vol. 36, no. 5, pp. 791–804,
2002.
[44] R. P. Rao and D. H. Ballard, “Predictive coding in the visual cortex: Afunctional interpretation of some extra-classical receptive-field effects,”
Nature Neurosci., vol. 2, no. 1, pp. 79–87, 1999.
[45] A. Ford and A. Roberts, “Colour space conversions,” Westminster University, London, U.K., vol. 1998, pp. 1–31, 1998.
[46] H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik, “LIVE Image
Quality Assessment Database release 2,” Lab. Image Video Eng., Univ.
Texas Austin, Austin, TX, USA, Accessed on: 2005. [Online]. Available:
http://live.ece.utexas.edu/research/quality[47] N. Ponomarenko et al., “A new color image database TID2013: Innovations and results,” in Proc. Adv. Concepts Intell. Vis. Syst., 2013,vol. 8192, pp. 402–413.
[48] A. Zaric et al., “VCL@FER image quality assessment database,” in Proc.
ELMAR, Sep. 2011, pp. 105–110.
[49] M. Saad, A. C. Bovik, and C. Charrier, “Blind image quality assessment:
A natural scene statistics approach in the DCT domain,” IEEE Trans.
Image Process., vol. 21, no. 8, pp. 3339–3352, Aug. 2012.
[50] Video Quality Experts Group, “Final report from the Video Quality
Experts Group on the validation of objective models of video quality
assessment, Phase I (FR_TV),” 2000. [Online]. Available: http://www.
vqeg.org/[51] K. Ma et al., “Group mad competition—A new methodology to compare
objective image quality models,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recog., Jun. 2016, pp. 1664–1673.
[52] J. Löfberg, “YALMIP: A toolbox for modeling and optimization in MATLAB,” in Proc. IEEE Int. Symp. Comput. Aided Control Syst. Des.,
Sep. 2004, pp. 284–289.
[53] Gurobi Optimizer Reference Manual, Gurobi Optimization, Inc., Houston,
TX, USA, 2014. [Online]. Available: http://www.gurobi.com[54] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image
Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
[55] W. Xue, L. Zhang, and X. Mou, “Learning without human scores for blind
image quality assessment,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recog., Jun. 2013, pp. 995–1002.
[56] D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical
Procedures. Boca Raton, FL, USA: CRC Press, 2003.
[57] D. Jayaraman, A. Mittal, A. K. Moorthy, and A. C. Bovik, “Objective
quality assessment of multiply distorted images,” in Proc. 46th Asilomar
Conf. Signals, Syst. Comput., Nov. 2012, pp. 1693–1697.
[58] D. Ghadiyaram and A. C. Bovik, “Massive online crowdsourced studyof subjective and objective picture quality,” IEEE Trans. Image Process.,
vol. 25, no. 1, pp. 372–387, Jan. 2016.
[59] O. Chapelle and V. Vapnik, “Model selection for support vector machines,”
in Proc. Adv. Neural Inf. Process. Syst., 1999, pp. 230–236.
[60] B. Schölkopf, C. J. C. Burges, and A. J. Smola, Advances in Kernel
Methods: Support Vector Learning. Cambridge, MA, USA: MIT Press,
1999.
[61] T. Song, H. Li, F. Meng, Q. Wu, B. Luo, B. Zeng, and M. Gabbouj, “Noise-robust texture description using local contrast patterns via
global measures,” IEEE Signal Process. Lett., vol. 21, no. 1, pp. 93–96,
Jan. 2014.
[62] F. M. Khellah, “Texture classification using dominant neighborhood structure,” IEEE Trans. Image Process., vol. 20, no. 11, pp. 3270–3279,
Nov. 2011.

2504

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 19, NO. 11, NOVEMBER 2017

Qingbo Wu (S’12–M’15) received the B.E. degreein education of applied electronic technology from
Hebei Normal University, Shijiazhuang, China, in
2009, and the Ph.D. degree in signal and information
processing from the University of Electronic Scienceand Technology of China, Chengdu, China, in 2015.
From February 2014 to May 2014, he was a Research Assistant with the Image and Video Processing
Laboratory, Chinese University of Hong Kong, Hong
Kong, China. From October 2014 to October 2015,
he was a Visiting Scholar with the Image and Vision
Computing Laboratory, University of Waterloo, Waterloo, ON, Canada. He iscurrently a Lecturer with the School of Electronic Engineering, University ofElectronic Science and Technology of China. His research interests include image/video coding, quality evaluation, and perceptual modeling and processing.

Fanman Meng (S’12–M’13) received the Ph.D. degree in signal and information processing from the
University of Electronic Science and Technology ofChina, Chengdu, China, in 2014.
From July 2013 to July 2014, he joined the Division of Visual and Interactive Computing, Nanyang
Technological University, Singapore, as a Research
Assistant. He is currently an Associate Professor
with the School of Electronic Engineering, University of Electronic Science and Technology of China,
Chengdu, China. He has authored or coauthored numerous technical articles in well-known international journals and conferences.
His research interests include image segmentation and object detection.
Prof. Meng is a Member of IEEE CAS society. He was the recipient of the“Best Student Paper Honorable Mention Award” for the 12th Asian Conference
on Computer Vision (2014), Singapore, and the “Top 10% Paper Award” in theIEEE International Conference on Image Processing (2014), Paris, France.

Hongliang Li (M’06–SM’11) received the Ph.D. degree in electronics and information engineering fromXi’an Jiaotong University, Xi’an, China, in 2005.
From 2005 to 2006, he was with the Visual Signal Processing and Communication Laboratory, Chinese University of Hong Kong (CUHK), Hong Kong,
China, as a Research Associate, where he was a Postdoctoral Fellow from 2006 to 2008. He is currently aProfessor with the School of Electronic Engineering,
University of Electronic Science and Technology ofChina, Chengdu, China. He has authored or coauthored numerous technical articles in international journals and conferences. He
is the co-editor of Video Segmentation and Its Applications (Springer). He wasinvolved in many professional activities. His research interests include image
segmentation, object detection, image and video coding, visual attention, andmultimedia communication system.
Prof. Li is a Member of the Editorial Board of the Journal on Visual Communications and Image Representation, and the Area Editor of Signal Processing:Image Communication, Elsevier Science. He was a Technical Program Co-Chairof ISPACS 2009, General Co-Chair of the ISPACS 2010, Publicity Co-Chair ofIEEE VCIP 2013, Local Chair of the IEEE ICME 2014, and TPC Member ina number of international conferences, e.g., ICME 2013, ICME 2012, ISCAS
2013, PCM 2007, PCM 2009, and VCIP 2010. He is currently a Technical Program Co-Chair of the IEEE VCIP 2016.

Bing Luo received the B.Sc. degree in communication engineering from The Second Artillery Command College, Wuhan, China, in 2009, the M.Sc.
degree in computer application technology from
Xihua University, Chengdu, China, in 2012, and iscurrently working toward the Ph.D. degree in electronic engineering at the University of Electronic
Science and Technology of China, Chengdu, China,
supervised by Prof. Hongliang Li.
His research interests include image and video
segmentation and machine learning.

Zhou Wang (S’99–A’02–M’03–SM’12–F’14) received the Ph.D. degree from The University of Texas
at Austin, Austin, TX, USA, in 2001.
He is currently a Professor with the Department ofElectrical and Computer Engineering, University ofWaterloo, Waterloo, ON, Canada. He has more than
100 publications with over 30 000 citations (Google
Scholar). His research interests include image processing, coding, and quality assessment, computational vision and pattern analysis, multimedia communications, and biomedical signal processing.
Prof. Wang is a Fellow of the Canadian Academy of Engineering. He is a
Senior Area Editor of the IEEE TRANSACTIONS ON IMAGE PROCESSING (2015–
present), and an Associate Editor of the IEEE TRANSACTIONS ON CIRCUITS AND
SYSTEMS FOR VIDEO TECHNOLOGY (2016–present). He was a Member of theIEEE Multimedia Signal Processing Technical Committee (2013–2015), an Associate Editor of the IEEE TRANSACTIONS ON IMAGE PROCESSING (2009–2014),
Pattern Recognition (2006–present) and the IEEE SIGNAL PROCESSING LETTERS(2006–2010), and a Guest Editor of the IEEE JOURNAL OF SELECTED TOPICS
IN SIGNAL PROCESSING (2013–2014 and 2007–2009). He was the recipient of
the 2017 IEEE Signal Processing Society Sustained Impact Paper Award, 2015
Primetime Engineering Emmy Award, 2014 NSERC E.W.R. Steacie Memorial
Fellowship Award, 2013 IEEE Signal Processing Magazine Best Paper Award,
2009 IEEE Signal Processing Society Best Paper Award, 2009 Ontario Early
Researcher Award, and ICIP 2008 IBM Best Student Paper Award (as a seniorauthor).

Wei Li received the B.Sc. degree in electrical and information engineering from Henan Polytechnic University, Jiaozuo, China, in 2011, and is currently
working toward the Ph.D. degree in the Intelligent
Visual Information Processing and Communication
Laboratory, University of Electronic Science andTechnology of China, Chengdu, China.
His research interests include image recognition,
object detection, and machine learning.

King N. Ngan (S’79–M’82–SM’91–F’00) received
the Ph.D. degree in electrical engineering from theLoughborough University, Loughborough, U.K., in
1982.
He is currently a Chair Professor with the Department of Electronic Engineering, Chinese Universityof Hong Kong, Hong Kong, China. He was previously a Full Professor with the Nanyang Technological University, Singapore, and the University ofWestern Australia, Perth, WA, Australia. Since 2012,he has been the Chair Professor with the University
of Electronic Science and Technology, Chengdu, China, under the National
Thousand Talents Program. He holds honorary and visiting professorships ofnumerous universities in China, Australia, and South East Asia. He has published extensively including three authored books, seven edited volumes, over
400 refereed technical papers, and edited nine special issues in journals. In
addition, he holds 15 patents in the areas of image/video coding and communications.
Prof. Ngan is a Fellow of IET (U.K.), IEAust (Australia), and an IEEE Distinguished Lecturer (2006–2007). He was an Associate Editor of the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, the Journal on
Visual Communications and Image Representation, the EURASIP Journal of
Signal Processing: Image Communication, and the Journal of Applied Signal
Processing. He chaired and co-chaired a number of international conferences on
image and video processing including the 2010 IEEE International Conference
on Image Processing, and served on the advisory and technical committees ofnumerous professional organizations.