Signal Processing: Image Communication 28 (2013) 884–902

Contents lists available at ScienceDirect

Signal Processing: Image Communication
journal homepage: www.elsevier.com/locate/image

Reduced-reference image quality assessment in reorganized
DCT domain
Lin Ma n, Songnan Li, King Ngi Ngan
Department of Electronic Engineering, The Chinese University of Hong Kong, Shatin, NT., Hong Konga r t i c l e in f oabstract

Available online 23 August 2012

In this paper, a novel reduced-reference (RR) image quality assessment (IQA) is
proposed by depicting the subband statistical characteristics in the reorganized discretecosine transform (RDCT) domain. First, the block-based DCT coefﬁcients are reorganized
into a three-level coefﬁcient tree, resulting in ten RDCT subbands. For the intra RDCT
subband characteristic, the coefﬁcient distribution of each RDCT subband is modeled by
the generalized Gaussian density (GGD) function. The city-block distance (CBD) is
employed to measure the modeling error between the actual distribution and the ﬁtted
GGD curve. For the inter RDCT subband characteristic, the mutual information (MI) is
utilized to depict the dependencies between coefﬁcient pairs in related RDCT subbands.
Moreover, a frequency ratio descriptor (FRD) calculated in the RDCT domain is proposed
to depict how the image energy distributes among different frequency components. The
FRD values computed from both the reference and distorted images are jointly
considered to derive a novel mutual masking strategy for simulating the texture
masking property of the human visual system (HVS). By considering the GGD modelingof intra RDCT subband, MI of inter RDCT subbands, and FRD of the image, the proposed
RR IQA is developed. Experimental results demonstrate that a small number of RR
features is sufﬁcient to represent the reference image for the perceptual quality
analysis. The proposed method can outperform the state-of-the-art RR IQAs, and eventhe full-reference (FR) PSNR and SSIM.
& 2012 Elsevier B.V. All rights reserved.

Keywords:
Image quality assessment (IQA)
Reduced-reference (RR)
Human visual system (HVS)
Reorganized discrete cosine transform(RDCT)

1. Introduction
Image perceptual quality measurement plays a keyrole in the image processing and communication [1,2]such as image compression, printing, displaying, restoration, enhancement, transmission, and so on. Human eyes
are the ultimate receivers of the images. Therefore, themost reliable way for assessing the image perceptual
quality is the subjective testing method [3,4]. However,these subjective testing methods require many observers
to participate in the experiments and provide theirnCorresponding author. Tel.: þ 852 39438255.
E-mail addresses: lma@ee.cuhk.edu.hk (L. Ma),
snli@ee.cuhk.edu.hk (S. Li), knngan@ee.cuhk.edu.hk (K.N. Ngan).
0923-5965/$ - see front matter & 2012 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.image.2012.08.001

personal opinions on the image/video perceptual quality,
which make them very time consuming and expensive.
Therefore, the subjective testing method is not suitable
for the practical image applications, such as guiding theimage compression, optimization for the image denoising,and so on. The image quality metrics are in demand,
which can automatically evaluate the image quality andguide the image processing applications.
According to the availability of the reference image, thequality metrics can be roughly categorized into three classes[5]: full-reference (FR) [7–12], no-reference (NR) [13–19], andreduced-reference (RR) [20–36,58–62,68,69]. In order toevaluate the perceptual quality of the distorted image, theFR metrics require the whole reference image, which isassumed to be artifact free and of perfect quality. These
metrics can only be applied to the applications where theL. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

reference image is available, such as image compression [10],watermarking [12,55], and so on. The simplest FR metrics are
the mean squared error (MSE) and the corresponding peaksignal-to-noise ratio (PSNR). MSE and PSNR are widelyadopted, because of their clear physical meanings andfriendliness for optimization. However, MSE and PSNR only
consider the differences in the pixel level, which are notrelated to the perception property of the human visualsystem (HVS) [6–33]. They are not reliable for evaluating oreven controlling the perceptual quality of the distorted image
during the processing stages. Therefore, the structure similarity (SSIM) index [7] is developed by depicting the structural distortions rather than the pixel absolute difference,
which is more sensitive to the HVS. In [10], the orientation
sensitivity and saliency property of the HVS are incorporated
to reﬁne the SSIM distortion map, which can ensure a better
performance. Chandler et al. [8] proposed the visual signalto-noise ratio (VSNR) to evaluate the perceptual quality of the
distorted image in the wavelet domain. The disadvantage ofthese IQAs is the complexity, especially for optimization in
the practical image processing applications, such as image
coding and watermarking. In [10], Zhang et al. simply
considered the contrast sensitivity function and texture
masking effect of the HVS to develop a practical image
quality metric, which has been proved to be effective for
guiding perceptual-based image compression. Subsequently,
this practical quality metric is employed for image watermarking [12].
In many real-world applications, we cannot access thereference image for the quality evaluation, such as image
denoising, restoration, etc., where only the distorted image isavailable for analysis. Therefore, the NR IQAs [14–19] are thus
needed to evaluate and control the perceptual quality of theprocessed image. Many researchers employ the behaviors ofspeciﬁc distortions for the NR quality assessment, such as theblocking artifact of JPEG coded images, ringing artifact of the
JPEG 2000 coded images, and so on. As JPEG 2000 employs
the wavelet transform to compress the image, the wavelet
statistical model is utilized to capture the compression
distortion [14]. Liang et al. [15] combined the sharpness,blurring, and ringing measurements together to depict theperceptual quality of the JPEG 2000 coded image. The
distribution of the DCT coefﬁcient after quantization ismodeled in [16] to predict the PSNR value of the JPEG coded
image. Furthermore, Ferzli et al. [18] did the psychophysicalexperiment to test the blurring tolerance ability of the HVS,
based on which the just-noticeable blur (JNB) model isdeveloped. These methods employ the behaviors of speciﬁc
distortions to predict the degradation level. Therefore, if a
new distortion is introduced, these methods can hardly
evaluate the perceptual quality of the distorted image. In
order to compromise between the FR and NR IQAs, RR IQAs
are developed. It is expected that the RR methods caneffectively evaluate the image perceptual quality based on alimited number of features extracted from the reference
image. Only a small number of bits is required for representing the extracted features, which can be efﬁciently encoded
and transmitted for the quality analysis. Consequently, it will
be very useful for the quality monitoring during the imagetransmission and communication. The image perceptual
quality can be easily analyzed by referring to the extracted

885

features from the reference image. Therefore, a better qualityof user experience can be further provided for the consumers.
For designing an effective RR quality metric, we needto consider not only its performance but also its RR data
rate for representing the extracted features. First, theextracted features should be sensitive to a variety ofimage distortions and relevant to the HVS perception of
the image quality. Second, the RR data rates should not belarge, as the extracted features need to be embedded ortransmitted to the receiver side for the quality analysis.
For a larger RR data rate, one may include more information about the reference image. Then a good performancecan be obtained. However, it will introduce a heavy
burden to the RR feature transmission. The FR IQA canbe regarded as an extreme case of RR IQA, with the RR
data rate is the whole reference image. For a smaller RR
data rate, only a little information of the reference image
is available for quality analysis. Therefore, the performance is hard to be ensured. The NR IQA is anotherextreme case of RR IQA, with no information from thereference image. Therefore, how to balance the RR data
rate and the performance is the essential for the RR
quality metric development.
In [20–23], the distortions introduced are modeled toderive the RR quality metrics. Wolf et al. [20,22] proposed to
extract a set of spatial and temporal features for measuring
the distortions, which occur in the standard video compression and communication environment. The features that areassociated with the blurring, blocking, and frame differences
are extracted in [21] to depict the compression artifacts
introduced by MPEG-2. The ratio between the parent coefﬁcient (second DCT coefﬁcient) and the child coefﬁcient (thethird and fourth DCT coefﬁcients) is employed to measure
the perceptual quality of the MPEG-2 coded video sequence[23]. The amplitude and phase information of the harmonics[60] generated by blocking artifacts are employed to estimate
the blockiness of the MPEG-2 coded video sequences. TheseRR quality metrics are designed for some speciﬁc distortions,which cannot be effectively applied to the other images ofdifferent distortions. Therefore, a general RR IQA for evaluating the image perceptual quality of different distortions isrequired.
As the human eyes are the ultimate receivers of theimage, the HVS properties need to be considered for designing an effective RR IQA. Le Callet et al. [24] employed a neuralnetwork to train and evaluate the perceptual qualities ofvideo sequences based on the perceptual-related features
extracted from the video frames. In [25,26], the perceptualfeatures motivated from the computational models of lowlevel vision are extracted as the reduced descriptors to
represent the image perceptual quality. The merits from thecontourlet transform, the contrast sensitivity function, andWeber’s law of just noticeable difference are incorporated toderive an RR IQA [27], which are employed for evaluating theperceptual qualities of the JPEG and JPEG 2000 coded images.
Recently, an RR IQA [28] for wireless imaging is developed byconsidering different structural information that is observedin the distortion model of wireless link. The structural
information from the viewing area is trained for the HVS
perception. Moreover, as the HVS is sensitive to the degradation around the edges, the RR video quality metric proposed

886

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

in [58] mainly measures the edge degradations. The edge
degradation is computed by measuring the mean squared
error of the edge pixels. Therefore, this method is named asedge PSNR (EPSNR). In [59], the authors employed discriminative local harmonic strength with motion consideration toevaluate the distorted video quality, which we name as
RR-LHS. The gradient information of each frame is employedfor harmonic and discriminative analysis.
Recently, the statistical modeling of the image signal has
been investigated for the image perceptual quality assessment for both RR IQAs [29,36,61,62,68,69] and NR IQAs
[14,17,19]. In [32], the divisive normalization is employed
to depict the coefﬁcient distributions of the wavelet subbands. The distribution difference between the reference anddistorted images is used to depict the image perceptual
quality, which we name as RR-DNT. In [61,62], the developed
RR image quality metric RR-SSIM extracted the statisticalfeatures from a multi-scale, multi-orientation divisive normalization transform. By following the philosophy in theconstruction of SSIM, a distortion measurement is developed
to estimate the SSIM index of the distorted image. In [68], thestatistics of image gradient magnitude are modeled by the
Weibull distribution to develop an RR image quality
metric, which is named as RR-Weibull. Also the statistics ofthe edge [69] are utilized for developing the RR IQA, which
we name as RR-Edge. In [34], the authors measure thedifferences between the entropies of wavelet coefﬁcientsof the reference and distorted image to quantify the image
information change, which can indicate the image perceptual quality. In [36], the color distribution changes of animage as a consequence of the distortions are employedfor depicting the perceptual quality, where the color correlogram is extracted as the RR feature. Wang et al. [29,30]
proposed a wavelet-domain natural image statistic
metric (WNISM), which models the marginal probability
distribution of the wavelet coefﬁcients of a natural
image by the generalized Gaussian density (GGD)
function. The Kullback–Leibler distance (KLD) is used to
depict the distribution difference. Although WNISM canachieve good performances on image quality assessment,
some limitations can still be ﬁgured out. First, KLD isasymmetric [39], which is not suitable for the quality
analysis, as demonstrated in [33]. The perceptual quality
distance from one image to another should be identical nomatter how it is measured. Second, as revealed in [32],although WNISM can work quite well on individual distortion types, its performance degrades signiﬁcantly when
image of different distortion types are evaluated together.
In order to overcome the aforementioned limitations, the RR
quality metric in [33] is proposed by employing the reorganized discrete cosine transform (RDCT) based image representation. The DCT coefﬁcients are ﬁrst reorganized into
several representative subbands, whose distributions aremodeled by the GGD. The city-block distance (CBD) is utilized
to capture the image perceptual quality. However, although abetter performance can be achieved as shown in [33], only
the identical nature of the coefﬁcient distribution in the same
RDCT subband is utilized. As to be demonstrated in thefollowing section, the correlations between related RDCT
subbands also exist. However, the RR IQAs [29,33] do notconsider the inter RDCT subband relationships. In order todesign an effective RR IQA, both the intra and inter RDCT
subband statistical characteristics need to be considered.
As aforementioned, the statistical properties of the image
signals which are sensitive to the introduced distortions canbe employed to develop IQA methods. DCT has been widelyadopted and employed for image/video compression,
deblocking, denoising, and so on. Therefore, if we can extract
several HVS-sensitive features based on the DCT coefﬁcients,
they can be utilized as the RR feature for quality analysis oreven guiding the image processing. As shown in [33], thecoefﬁcient distribution of the RDCT subband can be accurately modeled by GGD, which is useful for designing the RR
IQA. Moreover, the inter RDCT subband relationship needs tobe further depicted and modeled. This statistical property
will be affected by the introduced distortions. Furthermore,as the image has been represented in the RDCT domain, therelationship between different frequency components should
be considered and depicted. First, the energy variations indifferent frequencies can somewhat represent the distortionlevel. Second, the energy distributions over different frequencies can help to model the HVS masking properties. As aresult, we will manipulate the DCT coefﬁcients in the RDCT
domain from the three aspects discussed above to evaluate
the perceptual qualities of the distorted images.
In this paper, a novel RR IQA is developed by depicting theintra and inter subband statistical characteristics in the RDCT
domain. It is shown that after performing DCT the statisticaldependencies between the DCT subbands still exist. Applying
the reorganization strategy, the intra RDCT subband statistical characteristic, speciﬁcally the identical natural of thecoefﬁcient distribution within the RDCT subband, is exploitedby the GGD modeling. The inter RDCT subband dependency
is captured by the mutual information (MI) between the DCT
coefﬁcient pair in corresponding RDCT subbands, such as
parent–child pair coefﬁcient, brother–child pair coefﬁcient,and cousin–child pair coefﬁcient. Furthermore, a frequency
ratio descriptor (FRD) computed in the RDCT domain is
employed to measure the energy distribution among different frequency components. It can be further utilized tosimulate the HVS texture masking property. By considering
the intra RDCT subband GGD modeling, inter RDCT subband
MI values, and the image FRD value, an effective RR IQA isdeveloped.
This paper is organized as follows. The relationship ofintra and inter RDCT subbands is presented in Section 2.
The detailed algorithms, including the feature extractionin the sender side and quality analysis in the receiver side,
will be introduced in Section 3. Section 4 will demonstrate the performance comparisons. Finally, the conclusion will be given in Section 5.
2. Relationship analysis of intra and inter RDCT
subbands
Since the HVS is more sensitive to luminance thanchrominance [63], the proposed image quality metric andthe other compared ones work with luminance only. Color
inputs will be converted to gray scale before further
analysis. The coefﬁcients obtained by the block-based
DCT present high correlations, which can be employedfor depicting the image degradation level. In [23], afterL. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

the 8-tap DCT, the second DCT coefﬁcient and the third/
fourth DCT coefﬁcient are related to each other as theparent and child bands of the wavelet transform. In [16],the Laplace probability density function (pdf) is employed
to model the coefﬁcient distribution of each DCT subband.
The ﬁtted Laplace pdf parameter l of one DCT subband
can be linearly predicted by the l values of the neighboring upper and left DCT subbands. Therefore, although DCT
has decomposed the spatial image block into different
frequency components, the relationship between related
DCT subbands still exists. In order to utilize the identicalnature of the neighboring coefﬁcient distributions, thereorganization strategy [33,37,38] is employed to compose the block-based DCT coefﬁcients into a three-leveltree structure, as demonstrated in Fig. 1.
First, the 8  8 DCT is performed block by block on theimage. For each 8  8 DCT block, the DCT coefﬁcients aredecomposed into ten subbands, as shown in Fig. 1(a). For
the subbands 0, 1, 2, and 3, each subband only contains
one DCT coefﬁcient. For the subbands 4, 5, and 6, eachsubband contains a 2  2 DCT coefﬁcient matrix. For thesubbands 7, 8, and 9, each subband contains a 4  4 DCT
coefﬁcient matrix. After the decomposition, the same887

subbands of all the 8  8 DCT blocks are grouped andorganized together according to their corresponding positions, as shown in Fig. 1(b). In this manner, the blockbased DCT coefﬁcients are reorganized into a three-level
coefﬁcient tree. In Fig. 1, Sn denotes the grouped subbandof all the DCT coefﬁcients lying on the position denoted byn. For example, S7 is the reorganized subband by grouping
the 4  4 DCT coefﬁcient matrix lying on the position 7 of
all the 8  8 DCT blocks. An example of the reorganizationof the LENA DCT coefﬁcient image is illustrated in Fig. 1.
The 8  8 DCT representation is obtained by applying thenon-overlapped 8  8 block based DCT, as shown in
Fig. 1(c). The reorganized DCT (RDCT) representation isshown in Fig. 1(d). It can be observed that the RDCT representation appears like a wavelet representation [52], i.e.,
exhibiting structural similarities between subbands, andcoefﬁcient magnitude decaying toward high-frequency
subbands. Moreover, the RDCT representation is more
efﬁcient for the RR quality metric design than the wavelet
representation, such as the steerable pyramid [64,65],
even though the wavelet directly have an access to theoriented subbands. As demonstrated in [33], the generalized Gaussian density (GGD) can more accurately depict

Fig. 1. Reorganization strategy of DCT coefﬁcients. (a) one 8  8 DCT block with ten subband decomposition; (b) the reorganized DCT image
representation taken as a three-level coefﬁcient tree; (c) 8  8 DCT representation of LENA image; (d) the RDCT representation of LENA image.
(For better visualization, the DC components are rescaled to integers between 0 and 255, while the AC coefﬁcients are obtained by 255  (5  9AC9).)

888

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

the coefﬁcient distributions of the RDCT subbands thanthose of the steerable pyramid ones. Furthermore, theexperimental results in [33] have clearly demonstrated
that the RDCT representation is more effective than thesteerable pyramid to develop the RR IQA.
The statistical relationships between RDCT coefﬁcients are
examined in the following ways. First, consider the parent–child coefﬁcient pair representing the information at adjacentscale subbands of the same orientation (e.g., S4 and S7). Eachparent coefﬁcient in the subband S4 corresponds to four child
coefﬁcients in the subband S7, as illustrated in Fig. 2. In order
to exploit the underlying statistics, the joint histogram of thecoefﬁcient pair (parent, child) is built, which is gathered overthe spatial extent of the image. Fig. 3(a) shows the conditional histogram h(child9parent), which is simply calculated
by counting the child coefﬁcients in the subband S7 conditioned on the coarser-scale subband S4. Several important
aspects can be observed from the conditioned histogram.
These coefﬁcients are approximately second-order decorrelated, as the value of the child coefﬁcient is always zero whenthe values of parent coefﬁcients are not large enough. Moreover, the standard deviation of the child coefﬁcients highly
depends on the value of the parent coefﬁcient. The larger theparent coefﬁcient value, the larger the standard deviation of
the child coefﬁcients tends to be, as illustrated by the blue
curve in Fig. 3. In [42,43], it has been demonstrated that themean and the standard deviation curves of the conditional
histogram can be well ﬁtted by a Student’s t model of acluster of coefﬁcients. Furthermore, although they are decorrelated, the statistical dependency can still be observed
between the child and parent coefﬁcients. These dependencies also exist in the wavelet coefﬁcient pairs [40], whichcannot be eliminated by the linear transformations. This
statistical dependency can be more clearly observed byconverting the coefﬁcient value into the log-domain asshown in Fig. 3(b). The left part of the conditional histogramh(log2(child)9log2(parent)) concentrates on a nearly horizontal
line (shown by the green curve), which means that the value

S0

S1

S2

S3

parentS5

S4

S6

cousin
S8

upper
left
child
S7

brother
S9

Fig. 2. Statistical correlation between inter RDCT subbands. Each parent
coefﬁcient in the coarser scale RDCT subband corresponds to four child
coefﬁcients in the ﬁner scale subband. Each child coefﬁcient corresponds
to one cousin/brother coefﬁcient in the same scale subbands of different
orientations.

of log2(child) is independent of log2(parent) in this area.
Actually, natural images are composed of smooth regions
which are delimited by edge discontinuities. After performing
DCT, most of the image energy is compacted to the lowfrequency components, which results in a small amount ofenergy in the high-frequency components. Therefore, thechild coefﬁcient values in the ﬁner RDCT subband tend to besmall, especially when the parent coefﬁcient values are notlarge enough. The right part of the conditional histogram in
log-domain presents a nearly linear correlation. It implies
that the conditional expectation e(log2(child)9log2(parent)) isapproximately proportional to log2(parent).
Fig. 3(c) and (e) show the histograms of the child
coefﬁcient conditioned on the brother and cousin coefﬁcient, respectively. Compared with the conditional histogram in Fig. 3(a), the child coefﬁcient values varysigniﬁcantly, which do not present a close scattering
around the zero value. When the cousin or brothercoefﬁcient value becomes larger, the child value ﬂuctuates more dramatically, which can be observed by thestandard deviation values (the blue curve of each ﬁgure).
The brightness of Fig. 3 corresponds to the probability.
The brighter the area, the larger the corresponding probability is. Compared with Fig. 3(a), the brightness ofFig. 3(c) and (e) is not so signiﬁcant. It means that thechild coefﬁcient value depends on the cousin/brothercoefﬁcient less than the parent coefﬁcient. Furthermore,it can be observed that Fig. 3(e) is brighter than Fig. 3(c).
And the standard deviation curves of Fig. 3(e) appear to be
more regular than those of Fig. 3(c). The observations
show that the dependency relationship between child–cousin coefﬁcient pair is closer than that between child–
brother coefﬁcient pair. After converting the histograms into
log-domain, as shown in Fig. 3(d) and (f), the correlations
appear much looser. Although the mean value in the logdomain concentrates approximately on a line, the child
coefﬁcient values are of great differences. The child coefﬁcients do not present a concentrated distribution (with larger
standard deviation values), which makes the probability ofeach coefﬁcient value to be very small. Therefore, the brightness of Fig. 3(d) and (f) can hardly be detected. Although therelationship between child and brother/cousin appears to bemuch looser than that between child and parent, it isadmitted that the dependency does exist however in a very
complex way, which is very hard to depict.
In order to provide a more accurate description about
the relationship between RDCT subbands, the mutual
information (MI) is employed to describe the dependencies between the child and its condition parent, brother,cousin, upper, and left coefﬁcients, as illustrated in Fig. 2.
As introduced in [39], MI admits the direct data compression and classiﬁcation interpretations. Let X and Y be two
random variables (or vectors) having a joint pdf p(x,y).
The MI between X and Y is deﬁned as:
Z Zpðx,yÞ
IðX; YÞ ¼
dxdy
pðx,yÞlog
pðxÞpðyÞx ypðx,yÞ
9EXY log¼ Dðpðx,yÞ99pðxÞpðyÞÞ
ð1ÞpðxÞpðyÞ
where D(99) is the relative entropy between two distributions, known as the KLD. The MI I(X;Y) indicates how

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

889

Fig. 3. Conditional histogram for the coefﬁcients of the RDCT subbands from the BOAT image. Brightness corresponds to the probability. Each column has
been individually rescaled for a better visualization. (a) histogram of the child coefﬁcient conditioned on the parent coefﬁcient; (b) log-domain
representation of (a); (c) histogram of the child coefﬁcient conditioned on the brother coefﬁcient; (d) log-domain representation of (c); (e) histogram of
the child coefﬁcient conditioned on the cousin coefﬁcient; (f) log-domain representation of (e). The green curve corresponds to E(child9condition), and The
blue curves correspond to Eðchild9conditionÞ 8 stdðchild9conditionÞ, where the condition of each ﬁgure is the parent, cousin, and brother, respectively.

much information Y conveys about X. Therefore, the larger
the MI value, the more information is shared by X and Y.
Hence, the statistical correlation between X and Y isstronger.

The MI values between the RDCT subbands are illustrated in Table 1. We have provided the MI values of theinter RDCT subbands, such as parent–child S4 and S7,brother–child S9 and S7, and cousin–child S8 and S7, and890

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

Table 1
Mutual information between the RDCT subbands.
Subband orientation

Inter RDCT subband

Horizontal

Parent–child (S4 and S7)
0.5496
Parent–child (S5 and S8)
0.5091
Parent–child (S6 and S9)
0.2974

VerticalDiagonal

Intra RDCT subband
Brother–child (S9 and S7)
0.2739Brother–child (S9 and S8)
0.2685Brother–child (S7 and S9)
0.2739Cousin–child (S8 and S7)
0.2908
Cousin–child (S7 and S8)0.2908
Cousin–child (S8 and S9)0.2685Upper-child (S7)0.3892Upper-child (S8)0.3672Upper-child (S9)0.2165Left-Child(S7)0.3918
Left-child (S8)0.3508
Left-child (S9)0.2095

Fig. 4. RR feature extraction in the sender side.

the intra RDCT subbands, such as upper-child and leftchild. In order to provide a more convincing result, each entry
gives the average MI value over all the reference images fromthe LIVE image quality assessment database [50]. Someinteresting ﬁndings can be observed. First, no matter what
the subband orientation is, the MI value of parent–child is thelargest. It means that the parent coefﬁcients in coarser
subband affect the child coefﬁcients in the ﬁner subband
most, which presents the same property as the wavelet
transform. These dependencies have been successfully utilized for the image compression [37,38]. Second, the parent–child MI value of diagonal RDCT subband is much smaller
than those of horizontal and vertical ones. The reason is thatnatural images present much more horizontal and verticalinformation than the diagonal one. Therefore, most of the
DCT coefﬁcients in the diagonal subbands tend to be zero.
Furthermore, the MI values somewhat match the HVS
property, namely the oblique effect [51], that is, the HVS is
more sensitive to the horizontal and vertical frequency
components, compared with the diagonal ones. Thirdly, forthe horizontal and vertical RDCT subbands, the MI values of
intra RDCT subbands are larger than those of brother–childand cousin–child inter RDCT subbands. Therefore, the relationship between neighboring DCT coefﬁcients also exists.
This relationship has been further employed for image
compression [38] and image quality metric [16,33]. In [16],the authors employ the neighboring DCT subband relationship to improve the modeling accuracy of the DCT coefﬁcient
distribution. The adjacent DCT coefﬁcients are reorganized
into several representative subbands for designing the RR IQA[33]. Finally, the dependencies between cousin–child andbrother–child RDCT subbands can be observed. Although DCT
has decomposed the spatial image content into different
components with different orientations and frequencies, thedependencies cannot be removed by the linear transformations. Therefore, the correlations between inter RDCT subbands can be exploited for image processing researches, suchas compression [40,41], and so on.
3. The proposed reduced-reference image quality metric
As discussed above, the RR IQAs aim at evaluating theimage perceptual quality based on some RR features
extracted from the reference image. In order to design
an effective RR IQA, the features extracted should besensitive to the distortions, related to the HVS perception
property, and efﬁcient for representation. Therefore, theRR features are critical to the RR IQA performances. Basedon the analysis in the above section, the dependencies of
intra and inter RDCT subbands do exist, which can bedepicted and quantiﬁed in the receiver side. And thesedependencies are expected to be sensitive to the distortions, which can be utilized as the RR features for thequality analysis in the receiver side.
3.1. RR feature extraction in the sender side
Fig. 4 provides the framework of extracting the RR
features from the reference image. As shown in Table 1, theintra RDCT subband correlation, which has been proved to beL. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

sensitive to the image distortion, can be utilized for the RR
IQA [33]. The identical nature of the coefﬁcient distributions
between adjacent DCT subbands in the same RDCT subband
is exploited for extracting the RR features. As the coefﬁcient
distribution of each RDCT subband is highly kurtotic [29], thegeneralized Gaussian density (GGD) is employed to model its
distribution:pa, b ðxÞ ¼bb  eð9x9=aÞ ,2aG b1

ð2Þ

where b 40 and a are two parameters of the GGD function.
G is the Gamma function given by:
Z 1GðxÞ ¼
t x1 et dt:ð3Þ0Here a models the width of the PDF peak (standard deviation), while b is inversely proportional to the decreasing rateof the peak. a and b are also referred to as the scale andshape parameters, respectively. As demonstrated in [33],GGD is demonstrated to be suitable for modeling the highly
kurtotic coefﬁcient distribution in the RDCT domain, which
results in a better performance. In order to further improvethe modeling accuracy, another parameter except ða, bÞ isemployed which is named as the city-block distance (CBD)[27,33]:dCBD ðp,pa, b Þ ¼

hLX

9pðiÞpa, b 9ðiÞ,

ð4Þ

i¼1

where p is the histogram distribution of the actual RDCT
subband, pa,b is the ﬁtted GGD curve, and hL is the total
number of the histogram bins. Compared with KLD, CBD issymmetrical, which is reasonable for evaluating image perceptual quality [33]. According to the oblique effect [51] of
the HVS, human eyes present similar sensitive values to thehorizontal and vertical information, while less sensitive to the
diagonal information. Therefore, in order to reduce the RR
data rates, only three horizontal RDCT subbands, speciﬁcally
S1, S4, and S7, are employed for GGD modeling and CBD
calculation to extract the RR features.
Referring to Table 1, the dependencies between inter
RDCT subbands also exist. MI as deﬁned in (1) is
employed to capture the corresponding dependencies,
which can be further expressed as:
IðX; YÞ ¼ hðXÞhðX9YÞ
¼ EX ðlog 2 pðxÞÞEX,Y ðlog 2 pðx9yÞÞ

ð5Þ

where h(X) and h(X9Y) denote the entropy of X and X
conditioned on Y, respectively. As shown in (5), we canobserve that the MI is symmetric and non-negative. If X
and Y are independent, the MI is equal to zero. While if X
is a function of Y, I(X;Y)¼N. Actually, the MI I(X;Y)
indicates how much information Y conveys about X.
It admits a well-know data compression interpretation:coding X to a precision DX costs h(X) log2(DX) bits, basedon the assumption that DX is sufﬁciently small. If Y is
known, by considering the same encoding precision DX,
the total bits cost for encoding X is equal to h(X)
log2(DX)  I(X;Y) bits [39]. Therefore, the total saving bits
by introducing Y is I(X;Y).

891

The MI value is introduced in the sender side todescribe the essential relationship between inter RDCT
subbands, which is changed by the introduced distortion.
As we employed three horizontal RDCT subbands (S1, S4,
and S7) for GGD modeling and CBD calculation to depict
the intra RDCT relationship, the MI values between thesehorizontal RDCT subbands and other related ones arecomputed as the RR features to depict the inter RDCT
dependencies. These RR features include two MI values to
depict the parent–child correlation between the RDCT
subband pairs (S1, S4) and (S4, S7), three MI values to
depict the cousin–child correlation between the RDCT
subband pairs (S2, S1), (S5, S4), and (S8, S7), and three MI
values to depict the brother–child correlation between
the RDCT subband pairs, (S3, S1), (S6, S4), and (S9, S7).
Therefore, there are 8 MI values in total extracted tocapture the inter RDCT subband dependencies.
Furthermore, in order to accurately depict the reference image characteristic, an image-level feature, speciﬁcally the frequency ratio descriptor (FRD), is proposed byconsidering the HVS properties. For our RR feature extraction, after performing 8  8 DCT, the coefﬁcients arereorganized into several RDCT subbands, as illustrated inFig. 1. The frequency oij of the (i,j)th subband for each8  8 DCT block can be obtained by [46]:qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
1
oij ¼ 2N
ði=yx Þ2 þ ðj=yy Þ2
ð6Þ
 @D 
yD ¼ 2  arctan 2l, ðD ¼ x,yÞ
where N is the dimension of the DCT block (in this paper,N ¼8), yx and yy are the horizontal and vertical visualangels of a pixel. l is the viewing distance and @ stands forthe display width/length of a pixel on the monitor.
According to the international standard ITU-R BT.500-11[47], the ratio of viewing distance to picture height should
be a ﬁxed number between 3 and 6. Moreover, for most of
the displays, pixel aspect ratio (PAR) is equal to 1. It
means that the horizontal and vertical visual angles
ðyx , yy Þ are identical:

yx ¼ yy ¼ 2  arctan

1,
2  Rvd  Hpic

ð7Þ

where Rvd is the ratio of viewing distance to pictureheight. Hpic is the number of pixels in picture height.
The frequency values obtained by (6) and the spatial
contrast sensitivity function (CSF) values [67] of the8  8 DCT subbands are illustrated in Fig. 5. It can beobserved that the adjacent DCT subbands present similarfrequency and CSF values. The lower the frequency
component, the larger the CSF value is. After the reorganization process introduced in Section 2, the CSF values of
the RDCT subbands S0 , S1 , S2 , and S3 are larger than 0.5,
which are the most sensitive components to the HVS. By
checking the frequency oij value, we can ﬁnd that thefrequency values of these RDCT subbands are smaller than
5. Therefore, these RDCT subbands S0 , S1 , S2 , and S3(denoted by the red box) are simply regarded as the
low-frequency (LF) components. For the RDCT subbands
S4 , S5 , and S6, the CSF values (except the one of o33) arelarger than 0.2 and smaller than 0.5. And the frequency
values (except o33) are larger than 5 and smaller than 12.

892

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

Fig. 5. Frequency oij and the spatial contrast sensitivity function (CSF) value of each DCT subband. (a) frequency oij value and (b) spatial CSF value.

These RDCT subbands S4 , S5 , and S6 (indicated by the bluebox) are viewed as the medium-frequency (MF) components with medium sensitivity values. The rest of the
RDCT subbands S7 , S8 , and S9 (denoted by the green box)
present the lowest sensitivity values and the highest
frequency values larger than 12, which are regarded as
the high-frequency (HF) components.
The introduced distortion will not only change thehistogram distribution in each RDCT subband and dependencies between adjacent RDCT subbands, but also alter
frequency components of the image. For example, if JPEG
is utilized to code the reference image, the blocking andringing artifacts will appear which result from the frequency coefﬁcient truncation. As the quantization steps of
the HF components are higher than the LF ones, the HF
components will be degraded more seriously than LF
ones. In this paper, the image-level FRD is proposed byconsidering the ratio information between the LF, MF, and
HF components. The FRD can be efﬁciently computed in
the RDCT domain, which is deﬁned as:
FRD ¼

M value þ Hvalue,
Lvalueonly the 3 horizontal subbands are included, which
results in 9 parameters. For the inter RDCT subband
relationship, 8 MI values in total are introduced to depict
the parent–child, cousin–child, and brother–child relationships. For the frequency distribution, only one parameter named as FRD is extracted from the reference
image. Therefore, the proposed RR method employs
9þ 8þ1¼18 parameters to represent the reference image.
By comparing them with the ones extracted from thedistorted image, the perceptual quality can be analyzed.

3.2. Visual quality analysis in the receiver side
In the receiver side, we need to compare the extracted
features to analyze the perceptual quality of the distorted
image. For the parameters extracted from intra RDCT
subbands, CBD is employed to depict the distance
between the reference image and the distorted one:dCBD ðp,pd Þ ¼ð8Þ

where Lvalue, Hvalue, and Mvalue represent the sums of theabsolute DCT coefﬁcient values in the LF (S0 , S1 , S2 , and S3),
MF (S4 , S5 , and S6) and HF (S7 , S8 , and S9) RDCT subbands,
respectively. The FRD can help to capture the frequency
proportion changes caused by the distortions. Furthermore,
the larger the value of FRD, the more energy the MF and HF
components possess. It means that the DCT block is more
likely to contain texture information. For the plain block, theenergy mostly concentrates in the LF components. For theedge block, there will be only a small number of DCT
coefﬁcients in the HF group. Consequently, the texture block
will present higher FRD. As discussed in the JND models[44,45,67], the texture block can tolerate more distortions
than the plain and edge block, which is interpreted as the
texture masking property of the HVS. Therefore, the proposed
FRD can be employed to simulate the texture masking
property for the derivation of the ﬁnal image quality metric.
As discussed above, there are total 3 parameters{a,b,dCBD(p,pa,b)} to depict the histogram distribution ofeach RDCT subband. Considering the HVS oblique effect,hLX9pðiÞpd ðiÞ9,

ð9Þ

i¼1

where p is the coefﬁcient distribution of the reference
image, and pd belongs to the corresponding distorted one.
However, as the reference image is not available in thereceiver side, the distance is approximated by:
dCBD ðp,pd Þ9dCBD ðp,pa, b ÞdCBD ðpa, b ,pd Þ,ð10Þ

where dCBD(p,pa,b) is the third parameter introduced in
the sender side. Therefore, in the receiver side, only
dCBD(pa,b,pd) needs to be calculated. Their difference will
be recorded to represent the statistical feature distance of
the intra RDCT subband.
For the inter RDCT subband, the differences between
the corresponding MI values of the adjacent RDCT subbands are calculated:dMI ðSm ,Sn Þ9IðSm ,Sn Þ^IðSm ,Sn Þð11Þ

where I(Sm,Sn) is the MI of the RDCT subband Sm and Sn
in the reference image, and ^IðSm ,Sn Þ is the MI of Sm and Sn
in the distorted image.
For the image frequency, as the distortion will degrade
the HF, MF, and LF components differently, the FRD distanceL. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

can effectively represent the frequency component changes:
FL ¼ 9FRDori FRDdis 9,ð12Þ

where FRDori is the original feature, FRDdis is calculated fromthe distorted image, and FL denotes the frequency information change. As discussed before, FRD can represent how
much texture information the image contains. Therefore, it
can help to simulate the texture masking property of the
HVS. Furthermore, as discussed in [48,56], for the content of
the original image and the artifacts, one’s presence will affectthe visibility of the other. Therefore, in this paper, a novel
mutual masking strategy is proposed by considering the FRD
values of both the original and distorted image:8
FL>< FL þ FRDori , FRDori o FRDdis
FLv ¼
,
ð13Þ
FL
>
: FL þ FRDdis , FRDori ZFRDdis
where FLv is the ﬁnal HVS-related features to depict thefrequency information change. FL in the denominator isemployed to scale FLv into the range [0,1]. When an image
containing texture information is smoothed by the distortions, such as JPEG compression and blur, the detailed texture
information cannot be perceived by the HVS. Therefore, novisual masking effect should occur. Also if a smooth image isdistorted to be highly textured by the distortion, suchas additive Gaussian noise and fast-fading in the LIVE
database [50], only the noise can be perceived from thedegraded image. In this case, there should be no visualmasking effect either. This phenomenon is named as themutual masking [57]. In [56], the mutual masking effect isdetermined by the minimum value of the thresholds calculated from the original and distorted image. In this paper, as
the computed FRD value can depict the texture informationof the image, we employ (13) to depict the mutual masking
effect of the HVS perception, where the smaller value of
FRDori and FRDdis is employed to model the masking effect. In
this way, only the image is highly textured in both thereference and distorted images (large FRDori and FRDdis
values) can produce a signiﬁcant masking effect. In othercases, an insigniﬁcant masking effect will be introduced, as
expressed in (13).
Now we have obtained the CBD values of the intra
RDCT subbands, MI difference values of the inter RDCT
subbands, and the FLV value depicting the image frequency information change. How to combine them
together for developing an effective RR IQA needs to beconsidered. In this paper, a simple linear combination
method is employed to obtain the ﬁnal quality values:X
Q ¼ par1 
dCBD ðpsub ,psubd Þ þ par 2


X

sub

dMI ðSm ,Sn Þ þpar 3  FLv ,

ð14Þ

ðm,nÞ

where (par1, par2, par3) are the three weighting parameters to be determined, Q is the perceptual quality index
of the distorted image. First, we sum together the CBDvalues of intra RDCT subbands, and MI differences of inter
RDCT subbands, respectively. Their sum values and theFLv value are further linearly combined together according to (14). In order to ﬁnd the optimal parameters (par1,
par2, par3), the genetic algorithm [49] is employed to train893

them on several distorted images. In this paper, fourreference images and their corresponding distorted
images in the LIVE image quality assessment database [50]are employed to obtain the three parameter values. The
selected four reference images for parameterization are‘rapids’, ‘paintedhouse’, ‘plane’, and ‘building2’. The correlation
between the difference mean opinion score (DMOS) values
and the calculated Q values in (14) of the training images ismaximized to determine the optimized parameters (par1,
par2, par3). As there are only 3 parameters to be determined,
the number of the genes is equal to 3. Each gene uses 8-bit
binary representation. Each gene will be divided by 255 toconstrain them within [01.0]. The generation gap is set as 0.9,
which means that only 30–30  0.9¼3 best ﬁtted genes will
be propagated to the successive generation. Therefore, 27
new genes will be produced at each generation. The crossover for creating new genes is a single-point with probability
0.7. And the mutation for creating new genes is withprobability 0.0014. The number of generations is set to 100.
The initial population is created randomly and uniformly
distributed. The ﬁtness assignment is based on ranking
instead of raw performance. Selection method is stochastic
universal sampling. Reinsertion is ﬁtness-based (instead of
uniform random). After performing the genetic algorithm, theparameterization result is par1 ¼ 0.4883, par2 ¼0.0313, andpar3 ¼ 0.6719. Furthermore, as in [29,33], a logarithm process
is employed to scale the perceptual quality index Q:
Q fin ¼ log 10 1 þ

Q
,
D0

ð15Þ

where Qﬁn is the ﬁnal obtained quality score, D0 is utilized for
scaling the distortion measure to avoid the variation of Q
being too small. It just helps to depict the perceptual quality
index clearly, which will not inﬂuence the performance of the
proposed RR IQA. In this paper, D0 is set as 0.0001 forsimplicity.
4. Performances
In this section, the performances of different IQAs arecompared to demonstrate the efﬁciency of the proposed
RR IQA for evaluating the image perceptual quality.
Subsequently, the statistical signiﬁcance of our proposed
method is discussed. Finally, we will discuss the effectiveness of each component of the proposed RR IQA,
speciﬁcally, the intra RDCT CBD, the inter RDCT MI
differences, and the FLv value of the image.
4.1. Performance of the proposed RR IQA
We compare the performance of our proposed RR IQA
with the representative RR image quality metric: WNISM[29,30], recently developed RR IQA [33], RR-LHS [59], EPSNR[58], RR-DNT [32], RR-SSIM [61,62] , RR-Weibull [68], RREdge [69], and the FR metrics: PSNR, and SSIM [7]. The LIVE
image database [50] (excluding the distorted images generated from the four training reference images), the IRCCyN/
IVC image database [4], the MICT image database [66], andthe CSIQ image database [70] are employed to compare theperformances of these metrics. The most prevailing distortion types have been considered in these image data-

894

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

bases, including JPEG, JPEG 2000, blur, white Gaussian noise(WGN), and fast fading (FF). Each distorted image isassigned a subjective score, speciﬁcally the DMOS or MOS
value, which is obtained from the subjective viewing tests
where many observers participated and provided theiropinions on the perceptual quality of each distorted image.
Therefore, it can be regarded as the ground truth forevaluating the metric performances. We follow the performance evaluation procedure introduced in the video quality
experts group (VQEG) HDTV test [53] and that in [54]. Let xj
represent the perceptual quality index of the jth distorted
image obtained by the corresponding IQA. The ﬁve parameter {b1,b2,b3,b4,b5} monotonic logistic function is
employed to map xj into Vj:
V j ¼ b1  0:51
þ b4  x j þ b5 :1þ eb2 ðxj b3 Þ

ð16Þ

The corresponding ﬁve parameters are determined byminimizing the sum squared differences between the
mapped objective scores Vj and the subjective DMOS or
MOS values. In order to evaluate the performances, threestatistical measurements are employed. Linear correlation
coefﬁcient (LCC) deﬁned as:Pn
i ¼ 1 ðxi xÞðyi yÞ
qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
ﬃLCCðX,YÞ ¼ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
ð17Þ
Pn
Pn
2
2
i ¼ 1 ðxi xÞ
i ¼ 1 ðyi yÞ ,measures the correlation. xi and yi are the sample values,
while x and y are the corresponding mean value. The Spearman rank-order correlation coefﬁcient (SROCC) evaluates theprediction monotonicity, which is deﬁned as:
P 2
6 di
SROCCðX,YÞ ¼ 1:
ð18Þ
nðn2 1Þ
di is the difference between the ith image’s rank in subjective
and the perceptual quality index. n denotes the number of
the total samples. The root mean square prediction error(RMSE) of the ﬁtting process is also introduced to evaluate
the efﬁciencies, which is deﬁned as:vﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
u n
u1 XRMSEðX,YÞ ¼ t
ð19Þðx y Þ2 :ni¼1 i i
xi is the subjective value, such as MOS or DMOS value, while
yi is the objective value generated by each metric after the
mapping process deﬁned in (16). Therefore, RMSE measuresthe difference between the subjective value and the algorithm score after nonlinear regression [54]. After the mapping
process, the values obtained from the quality metrics will bein the same range as the subjective quality values. Therefore,
the RMSE values will be affected by the ranges of thesubjective values. According to the deﬁnitions, larger valuesof LCC and SROCC mean that the objective and subjectivescores correlate better, that is to say, a better performance of
the IQA. And smaller RMSE values indicate smaller errors
between the two groups of scores, therefore a better
performance.
The performances of different IQAs over different
image subjective quality databases are illustrated inTable 2, where the RR data rate of each IQA is also
illustrated. It can be observed that the proposed method
can outperform the other RR and FR metrics on the LIVE[50] and MICT [66] image databases, with larger LCC/SROCC and smaller RMSE value. While for the IRCCyN/IVC[4] image database, only the metrics RR-LHS [59] and RRSSIM [61,62] can generate better performances. For CSIQ
image database [70], only the metric RR-SSIM [61,62]
outperforms the proposed RR metric. However, these two
RR metrics require a much larger bit rate to represent the
RR features than the proposed RR metric. From Table 2,experimental results demonstrate that PSNR performs
badly, although it requires the whole reference image
for perceptual quality analysis. The reason is that PSNR
only measures the pixel absolute differences, which does
not take the HVS property into consideration. For SSIM[7], the structural distortions are measured rather than
the absolute pixel value differences, which are sensitive tothe HVS perception. Therefore, SSIM demonstrates a
better performance than PSNR. However, SSIM also utilizes the whole reference image for quality analysis,
which will introduce a heavy burden for the RR featuretransmission. For EPSNR [58], in order to reduce the bits
to represent the location, the reference and distorted
images are ﬁrst cropped to 614  454, where only thecentral parts are kept. Therefore, as shown in [58], 19 bits
are required to encode the location, while 8 bits areneeded to represent the pixel value. In our comparisons,
10 pixels together with their locations are employed as
the RR features, which require 270 bits in total for
representation. The performances of EPSNR over the threedatabases seem to be the worst. Although the HVS is
sensitive to the edges, 10 edge points are not sufﬁcient toaccurately represent the image perceptual quality. If moreedge pixels are included, the performance will be better.
In that case, a heavy burden for transmitting the RR
features will be introduced. For RR-Weibull [68] and RREdge [69], as the authors only provide the performance
results on the LIVE image database, their performances
results on the IRCCyN/IVC, MICT, and CSIQ image databases are not available for comparison. RR-Weibull [68]extracted 6 scalar parameters from each source image todepict the statistics of the image gradient magnitude.
It can generate a better performance than WNISM andEPSNR. However, as the number of the RR features is verysmall, which may not be sufﬁcient to depict the information of the source image, the performance is not goodenough, compared with other RR image metrics. RR-Edge[69] further incorporated more RR features to depict thestatistics of the edge. In total, 12 RR features are extracted
from the source image, which generates a better performance compared with RR-Weibull. However, it is still notgood enough. Since RR-LHS [59] considers the motion
information to design the RR video quality metric, in ourcomparisons only the discriminative local harmonicstrength in the spatial domain is employed for the RR
image quality assessment. Therefore, about 320 elementsof each image are extracted as the RR features. If 8 bits are
employed to encode each element, the RR data rate is
2560 bits. It is a high burden for the RR data transmission.
For RR-DNT [32] and RR-SSIM [61,62], the performance
results on the four image databases are illustrated in [62].
All the 779 distorted images in the LIVE image databaseare employed to demonstrate the performances of RR-

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

895

Table 2
Performance comparisons of different IQAs over different image subjective quality databases. PSNR, SSIM, WNISM [29,30], EPSNR [58], RR-LHS [59], RR
IQA [33], and the proposed method are evaluated on all of these four databases. For RR-Weibull [68] and RR-Edge [69], as the authors only provide theperformance results on the LIVE image database, their performances on the IRCCyN/IVC, MICT and CSIQ image databases are not available for comparison.
For RR-DNT [32] and RR-SSIM [61,62], the performance results on the four image databases are illustrated in [62]. However, as the proposed method
utilized four reference images and the corresponding distorted images for training the parameters, it is not fair to compare RR-DNT and RR-SSIM with theproposed method on the LIVE database. Therefore, only the performance comparisons on IRCCyN/IVC, MICT, and CSIQ image databases are illustrated.
(‘‘—’’ means that the IQA is an FR metric, where the RR feature number is the pixel number of the image, and the RR data rate is also viewed as the whole
image. ‘‘n’’ means that the RR IQA only calculates the number of the features, while the number of the bits for representing the RR parameters cannot beprovided.).
Database

Method

LCC

SROCC

RMSE

RR feature number

RR data rate

LIVE [50]PSNR
SSIM [7]WNISM [29,30]
EPSNR [58]
RR-LHS [59]
RR-Weibull [68]
RR-Edge [69]RR IQA [33]The proposed

0.87590.90410.75850.65710.88090.85670.86130.89840.93090.88130.91120.77090.62570.88310.86500.89080.89560.927913.15711.65317.771
20.55912.90914.47514.256
11.975
9.965

–
–
18
30
320
6
12
18
18

–
–
162 bits
270 bits
2560 bits

PSNR
SSIM [7]WNISM [29,30]
EPSNR [58]
RR-LHS [59]RR-DNT [32]
RR-SSIM [61] [62]RR IQA [33]The proposed

0.70370.77580.45250.39470.80780.63160.81770.70570.77120.67910.77780.40940.39580.82030.60990.8156
0.72670.7649

0.866
0.769
1.087
1.119
0.718
0.94460.7014
0.863
0.776

–
–
18
30
320
48
36
18
18

–
–
162 bits
270 bits
2560 bits

PSNR
SSIM [7]WNISM [29,30]
EPSNR [58]
RR-LHS [59]RR-DNT [32]
RR-SSIM [61] [62]RR IQA [33]The proposed

0.61540.71740.65680.4016
0.76230.67330.80510.64570.82820.57480.78700.6446
0.40590.76440.65210.80030.69410.8317

0.987
0.872
0.944
1.146
0.810
0.92530.7423
0.956
0.701

–
–
18
30
320
48
36
18
18

–
–
162 bits
270 bits
2560 bits

PSNR
SSIM [7]WNISM [29,30]
EPSNR [58]
RR-DNT [32]
RR-SSIM [61] [62]RR IQA [33]The proposed

0.79990.81500.73020.62280.70090.84260.81350.81970.80490.83680.75050.65370.70270.85270.80480.8044

0.158
0.152
0.179
0.205
0.18720.1413
0.153
0.150

–
–
18
30
48
36
18
18

–
–
162 bits
270 bits

IRCCyN/IVC [4]MICT [66]

CSIQ [70]

DNT and RR-SSIM. However, as the proposed method
utilized four reference images and the corresponding
distorted images for training the parameters, it is not fairto compare RR-DNT and RR-SSIM with the proposed
method on the LIVE database. Therefore, only the performance comparisons on IRCCyN/IVC, MICT, and CSIQ image
databases are illustrated in Table 2. RR-DNT [32] employs
the divisive normalization to depict the coefﬁcient distributions of the wavelet subbands. The distributiondifference between the reference and distorted images isused to depict the image perceptual quality. However, a
training process is utilized to determine the 5 parameters
in RR-DNT. And the performances of RR-DNT seem to besensitive to these parameters. That is the reason why RRDNT performs very well over the LIVE image database[32], while performs poorly over the IRCCyN/IVC, MICT,n

96 bits
162 bits
153 bitsnn

162 bits
153 bitsnn

162 bits
153 bitsnn

162 bits
153 bits

and CSIQ image databases. RR-SSIM [61,62] extracted thestatistical features from a multi-scale, multi-orientation
divisive normalization transform. By following the philosophy in the construction of SSIM, a distortion measurement is developed to estimate the SSIM index of the
distorted image. As a linear relationship between the RRSSIM and SSIM has been discovered, the performances of
RR-SSIM are good, which is comparable with the proposed
method. RR-SSIM outperforms the proposed metric on the
IRCCyN/IVC and CSIQ databases, while its performance isworse than the proposed one on the MICT database.
However, RR-SSIM extracted 36 RR features to represent
the source image, which is twice of that extracted by the
proposed RR metric. WNISM [29,30] is proposed in the
wavelet domain by depicting the marginal probability
distribution of each wavelet subband. The steerable

896

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

pyramid is ﬁrst employed to decompose the image into
several wavelet subbands, whose coefﬁcient distributions
are modeled by GGD. As demonstrated in [33], GGD canmore accurately model the coefﬁcient distribution of
RDCT subband than that of the steerable pyramid. Moreover, KLD is utilized in WNISM to depict the histogram
distribution distance. However, KLD is asymmetric, which
is not suitable for image quality evaluation, because thevisual quality distance from one image to another should be
identical no matter how it is measured. Those are the reasons
why WNISM performs badly over the four image quality
databases, as illustrated in Table 2. For the RR data rate, eachwavelet subband needs 3 parameters to describe its distribution, which requires 8þ8þ8þ3¼27 bits for representing
these parameters. In total 6 wavelet subbands are considered
to construct WNISM, which results in 27  6¼162 bits toencode all the RR features of the reference image. In order tohandle the drawbacks of WNISM, the RR IQA [33] employed
the DCT reorganization strategy and the CBD to depict thehistogram distribution difference. As demonstrated in [33]and Table 2, a better performance over LIVE and CSIQ image
database can be achieved. However, it performs poorly over
the other two image databases. The reason is that only
the intra RDCT subband correlation is exploited in [33],but the inter RDCT subband dependencies and the frequency
distribution over the image are not considered. Same as
WNISM, 162 bits are needed for representing the RR features
extracted for the RR IQA [33]. For the proposed method, theinter RDCT subband correlation is depicted by the MI values,
and the frequency distribution is captured by FRD, which canfurther simulate the HVS texture masking property. As thethree horizontal RDCT subbands are employed for depicting
the intra RDCT subband distribution, (8þ8þ8þ3)  3¼
81 bits are needed to represent the GGD modeling parameters. For the MI and FRD values, 8-bit representation is
employed. Therefore, 8  9¼72 bits are needed for representing all the MI and FRD values. In total, 81þ72¼153 bits
are required to encode all the RR features extracted for the
proposed method. The scatter-plots of different IQAs over theLIVE image database are illustrated in Fig. 6. And the scatterplots of the proposed RR metric on the four image subjective
quality database are illustrated in Fig. 7. It can be observedthat the points of the proposed method scatter more closely
to the ﬁtted line, compared with other IQAs. It means that theDMOS or MOS values correlate better with the perceptual
quality values obtained by the proposed RR IQA.
Furthermore, we test the proposed RR IQA over individual distortion types from the LIVE image database,which are illustrated in Table 3. It can be observed that
PSNR performs well over JPEG 2000 and WGN images,
especially for WGN images. However, for the JPEG, Blur,and FF noise images, PSNR performs poorly. EPSNR only
employs several edge pixels to measure the corresponding PSNR. Therefore, it presents a performance similar to
PSNR. For the WGN images, EPSNR demonstrates a very
good performance. It means that the perceptual qualities
of WGN images correlate closely with the absolute pixel
value differences, in contrast to other noise images. RRLHS demonstrates good performances on the JPEG 2000,
JPEG, WGN, and FF noise images. However, its performance over the Blur noise images is very poor, even a verylarge number of RR features has been employed. It means

Fig. 6. Scatter plots of the DMOS values versus model predictions on the LIVE image quality assessment database. Each sample point represents one test
image. (a) PNSR; (b) SSIM [7]; (c) WNISM [32]; (d) RR IQA [33]; (e) EPSNR [58] and (f) RR-LHS [59].

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

897

Fig. 7. Scatter plots of the DMOS or MOS values versus model predictions on the three image subjective quality databases. Each sample point represents
one test image. (a) LIVE image database; (b) IRCCyN/IVC image database; (c) MICT image database and (d) CSIQ image database.

Table 3
Performances of different IQAs over individual distortion types on the LIVE image database.

PSNR

WNISM [29,30]

EPSNR [58]

RR-LHS [59]

RR-Weibull [68]

RR-Edge [69]RR IQA [33]The proposed method

LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE

JPEG 2000

JPEG

WGN

Blur

FF

0.90780.9042
10.5460.92700.92119.434
0.67730.681618.5000.88610.879211.654
0.9422
0.94157.912
0.94040.94068.592
0.83350.8242
13.8930.89830.8912
11.0510.89420.885314.4060.86290.853916.2580.64890.640024.482
0.97610.9557
6.995
0.94930.9402
10.1150.93830.9408
11.1280.93630.923911.2990.95280.95209.766

0.98570.98504.711
0.87910.857213.3460.97000.96706.776
0.93450.98469.970
0.97710.97495.954
0.88150.865413.2240.90560.8907
11.8790.92750.909310.4710.7856
0.789411.2140.92340.92906.955
0.48900.308615.8070.60510.625014.4270.94710.94045.817
0.91520.90837.302
0.90620.9202
7.662
0.94590.95255.880

0.88800.8897
12.8980.94220.93509.399
0.61290.561222.1670.85690.8575
14.4620.92340.926110.741
0.94210.93299.400
0.93420.914810.0080.94370.92049.277

898

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

that the discriminative local harmonic strength is notsuitable for depicting the perceptual quality of theBlur noise images. For WNISM, RR-Weibull, and RR-Edge,
the experimental results over the individual distortion types are very good. However, their performances
degrade signiﬁcantly when images with different types of
distortions are tested together, as shown in Table 2. As
revealed by the previous literature [32], it is also the maindrawback of WNISM. In order to handle the drawback, RR
IQA [33] is proposed. The experimental results demonstrate that RR IQA [33] outperforms WNISM except for the
JPEG 2000 distortion. Actually, JPEG 2000 employed thewavelet transform for compression. Therefore, the steerable pyramid employed in WNISM is more suitable for
depicting the coefﬁcient distribution than the DCT. For the
proposed method, as new RR features, speciﬁcally theinter RDCT subband MI and image FRD FLv, have been
introduced, the performance over the JPEG 2000 noise
images has been greatly improved. Therefore, the proposed method can not only perform very well overindividual distortion types, but also provide a good
performance across different distortion types. It meansthat it performs more robustly for evaluating image visualquality. Furthermore, the proposed metric maintains asmaller RR data rate, compared with RR IQA [33], WNISM,
and RR-LHS. The improvements have demonstrated thatthe inter RDCT subband dependencies and the image FLv
value are helpful for designing an effective RR IQA.
It reﬂects that the MI differences and FLv value can help
to depict the levels of the introduced distortions. Therefore, for the proposed RR IQA, the RR features for depicting the vertical RDCT subbands [33] are excluded to savesome bit rates for the inter RDCT subband MI values andimage FRD value.
As illustrated in Tables 2 and 3, the effectiveness of our
proposed RR quality metric has been clearly demonstrated compared with the other RR metrics or even FR
metrics in terms of both performance and required RR
data rate. The computational complexities of RR feature
extraction and comparison need to be further evaluated.
The processing complexity in the sender side is different
from that in the receiver side. In the sender side, as
introduced in Section 3.1, 8  8 block-based DCT is ﬁrst
performed on the source image. After the reorganization
strategy, the DCT subbands are grouped into several

representative RDCT subbands. The DCT coefﬁcient distribution of each RDCT subband is modeled by the GGD.
MI is employed to depict the relationship between different RDCT subbands. Based on the RDCT subband, theimage FRD FLv is calculated. We implement the RR feature
extraction in Matlab. During our implementation, we do
not perform any optimizations. A speed test is performedon our PC with a 3.0 GHz Quad CPU and 1.0 GB memory.
For each source image of LIVE image database, it only
requires 2.94 s on average to extract the RR features. In
the receiver side, as illustrated in Section 3.2, the 8  8block-based DCT and reorganization strategy is also
performed. But the ﬁtting process of the GGD does notneed to be performed. Only the histogram of each RDCT
subband is constructed. And MI values between RDCT
subbands, and the image FRD FLv are calculated. Therefore, the computation is faster. The speed test is performed on the same PC, which indicates that only 1.93 s
per image on average is needed for the image quality
analysis. If further optimization is applied, it is believedthat the quality analysis in the receiver side can perform
even faster.

4.2. Statistical signiﬁcanceTo assess the statistical signiﬁcance of the performance difference between two metrics, F-test is conducted on the prediction residuals between the metric
outputs (after nonlinear mapping) and the subjective
ratings. The residuals are supposed to be Gaussian.
Smaller residual variance implies more accurate prediction. Let F denote the ratio between the residual variancesof two different metrics (with the larger variance as thenumerator). If F is larger than Fcritical which is calculated
based on the number of residuals and a given conﬁdence
level, then the difference between the two metrics are
considered to be signiﬁcant at the speciﬁed conﬁdence
level. Table 4 lists the residual variance of each metric on
the four subjective image databases. Notably due to thedifferences in employed subjective scales, the residual
variance varies a lot across different image databases. The
Fcritical with 95% conﬁdence is also shown in Table 4 foreach database. In Table 5, the proposed metric is comparedwith the other metrics regarding the statistical signiﬁcance.

Table 4
Residual variances of the IQAs on the four image subjective quality databases.

PSNR
SSIM [7]RR-LHS [59]EPSNR [58]
RR-Weibull [68]
RR-Edge [69]WNSIM [29,30]
RR IQA [33]The proposed

LIVE (672 images)
Fcritical ¼1.1355

IRCCyN/IVC (185images) Fcritical ¼1.275

MICT (168 images)
Fcritical ¼1.291

CSIQ (866 images)
Fcritical ¼1.1185

173.3645
136.0017
166.8887
423.3052
209.8357
203.5310
333.7304
159.3776
99.62360.75340.59420.5186
1.2599––1.18690.74850.60490.98040.76470.66001.3213––0.89580.91860.49480.02490.0232–0.0422––0.03220.02260.0233

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

899

Table 5
Performance comparisons regarding the statistical signiﬁcance. In each entry, the symbol ‘‘1’’, ‘‘0’’ or ‘‘ ¼ ’’ means that on the image database the proposed
RR metric is statistically (with 95% conﬁdence) better, worse or indistinguishable in comparison to its competitor. ‘‘n’’ means that the comparison cannot
be performed due to the unavailable result data.

PSNR
SSIM [7]RR-LHS [59]EPSNR [58]
RR-Weibull [68]
RR-Edge [69]WNSIM [29,30]
RR IQA [33]LIVE (672 images)
Fcritical ¼ 1.1355

IRCCyN/IVC (185images) Fcritical ¼1.275

MICT (168 images)
Fcritical ¼ 1.291

CSIQ (866 images)
Fcritical ¼ 1.1185

1
1
1
1
1
1
1
1

¼
¼
¼
1

1
1
1
1

¼
¼
1

n
n

n
n

n
n

1
¼

1
1

1
¼

In each entry, the symbol ‘‘1’’, ‘‘0’’, or ‘‘¼’’ means that on theimage databases indicated by the ﬁrst row of the table, theproposed metric is statistically (with 95% conﬁdence) better,worse, or indistinguishable, respectively, when comparedwith its competitors indicated by the ﬁrst column. ‘‘n’’ meansthat the comparison cannot be performed due to the unavailable result data. For the RR metrics RR-Weibull and RR-Edge,
the metric outputs of the distorted images on the IRCCyN/
IVC, MICT, and CSIQ image databases are not available. Also
we cannot ﬁt the RR-LHS outputs to the obtained DMOS
value using (16) for the CSIQ image database. Therefore, wecannot compare the statistical signiﬁcances of these metrics
with the proposed method on the corresponding image
databases. By referring to the other entry values shown in
Table 5, it can be observed that the proposed metric outperforms most of its competitors statistically. Although its
performances on IRCCyN/IVC, and CSIQ image databases
seem to be equivalent to other IQAs, overall it demonstrates
better performances on the other two image databases.
4.3. Performance analysis of each component
As we have mentioned before, the intra RDCT subband
correlation, the inter RDCT subband dependency, and theimage frequency distribution are utilized to design the RR
IQA. In this section, we will try to ﬁgure out the contribution of each component to the ﬁnal performance.
Table 6 illustrates the individual performance of eachcomponent of the proposed RR metric over the image
databases. For the CBD values of intra RDCT subbands,
only three horizontal RDCT subbands are considered.
Therefore, as 3 parameters are required to depict the
coefﬁcient distribution, 3  3¼9 parameters are extracted
for the RR features of intra RDCT subband correlation. It
can be observed that the performance is comparable with
the RR IQA in [33]. Actually, it matches the HVS obliqueeffect, which means the HVS presents similar sensitivity
to the horizontal and vertical information. Therefore, byconsidering only the horizontal ones, the visual quality of the
distorted image can be accurately depicted. For the MI
difference of inter RDCT subbands, 8 MI values are employed
to depict the parent–child, cousin–child, and brother–childdependencies. The performance outperforms the WNISM,
while it only requires a far smaller number of RR featuresnTable 6
Performance of each component of the proposed RR metric.

LIVE [50]

IRCCyN/IVC [4]MICT [66]

LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE

CBD

MI

FLv

0.89830.894311.9810.70040.69060.870
0.67130.68060.927

0.7746
0.769717.2480.6956
0.66560.875
0.47870.44351.099

0.87700.880913.1060.73240.72270.830
0.62590.71200.976(8 parameters vs. 18 parameters of WNISM). Additionally, it
can be observed that the MI differences perform worse thanthe CBD values. The reason is that the correlations between
inter RDCT subbands have been essentially ensured by thelinear transformations. Therefore, compared with the coefﬁcient distribution in each RDCT subband, the MI values
between different subbands vary less signiﬁcantly, thus
cannot effectively depict the image distortions. However,
the introduced distortion in the image will affect the MI
values between RDCT subbands. Therefore, it is necessary toincorporate the inter RDCT subband dependencies in designing the RR IQA, which plays a less but nevertheless animportant role in image quality assessment. For the FLv of
the image, the performance is very good. Even with only one
parameter FRD extracted from the reference image, theperformance is comparable with the RR IQA [33] and PSNR,
and even better than WNISM, RR-Weibull, and RR-Edge.
Therefore, if we want to further reduce the RR data rate,
we can extract the FRD only and transmit it to the receiverside for perceptual quality analysis. It only requires 8 bits torepresent the FRD of the reference image. The good performance may attribute to two reasons. First, the distortions
introduced will signiﬁcantly change the frequency distribution of the image. The larger the FRD changes, the higher the
distortion level. For example, the more compression is
introduced for JPEG coded image, the more HF and MF
components are discarded, compared with the LF ones. The
FRD differences as in (12) will become larger, which indicates

900

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902

Table 7
Performances of the combinations of different components of the
proposed metric.

LIVE [50]

IRCCyN/IVC [4]MICT [66]

LCC
SROCC
RMSE
LCC
SROCC
RMSE
LCC
SROCC
RMSE

CBDþ MICBDþ FL v

FLv þMI0.91000.905011.3090.71510.70590.852
0.67640.68740.922

0.92060.919410.6520.76320.75500.787
0.81260.81860.729

0.91140.9127
11.2200.73420.72350.827
0.72080.7132
0.867

Table 8Performances of the combinations of different components of the
proposed metric over individual distortion type.
JPEG 2000 JPEG
CBDþ MICBDþ FLv

FLv þMI

LCC
0.8461
SROCC 0.8400
RMSE 13.403
LCC
0.8908SROCC 0.8821
RMSE 11.426
LCC
0.9315
SROCC 0.9223
RMSE
9.149

WGN

0.9347 0.9141
0.9241 0.9010
11.438 11.357
0.9556 0.9150
0.9540 0.8988
9.479 11.296
0.9586 0.9294
0.9597 0.9144
9.160
10.331Blur

FF

0.9245 0.9394
0.9331 0.9177
6.907
9.617
0.9320 0.9383
0.9406 0.9170
6.568
9.703
0.9652 0.9288
0.9666 0.9268
4.736 10.395

worse perceptual quality. Therefore, the FRD difference candepict the distortion level. Second, the mutual masking
strategy is employed as formulated in (13). As discussed in[48], for the content of the original image and the artifacts,
one’s presence will affect the visibility of the other. Therefore,
by using mutual masking, the texture masking effect of the
HVS can be more accurately simulated.
In order to further demonstrate the contribution ofeach component of the proposed RR metric, different
combinations of these components are evaluated on theimage databases, as well as over each individual distortion type. The experimental results are illustrated inTables 7 and 8. It can be observed that the proposed
method can outperform all of these different combinations. It means that each component of our proposed RR
metric does contribute to the ﬁnal performance. Comparing the three combinations, we can see that CBD þFLv canensure the best performance. It is also consistent with theperformances illustrated in Table 7, where CBD and FLv
perform better than MI. However, CBDþFLv is still not as
good as the proposed RR metric. Therefore, the MI is still anecessary component that contributes to the performance
improvement of the proposed RR metric. In this case, if avery small RR data rate is required, we can only extract
FLv and transmit it to the receiver side for perceptual
quality analysis. With the increasing of the required RR
data rate, we can further transmit the CBD RR features tothe receiver side. Finally, if the RR data rate is sufﬁcient,

all the three components, speciﬁcally the FLv, CBD, andMI, will be extracted and transmitted to the receiver side
for a better performance.

5. Conclusion
In this paper, we propose a novel RR IQA by considering the intra and inter subband correlations in the RDCT
domain. The CBD and MI values are ﬁrst employed to
depict the intra and inter RDCT relationships, respectively.
The FRD calculated in RDCT domain depicts the frequency
distribution of the images, which can be further employed
to simulate the HVS texture masking effect in a mutual
masking way. Combining the CBD values, MI differences,and FRD value together, an effective RR IQA is developed.
Evaluations on several image quality databases demonstrate that the proposed method outperforms the stateof-the-art RR metrics and even FR metrics PSNR and SSIM.
It means that the proposed metric correlates well with thehuman perception of the image quality. Meanwhile, onlya small number of RR features are extracted. Furthermore,
we can extend the proposed RR image quality metric to
the RR video quality metric. The video sequences are
encoded by the block-based DCT, such as MPEG-2
and H.264. Since the proposed metric is based on blockbased DCT, it can easily capture the distortion due tothe compression. Hence, the proposed RR image can beeasily applied to the video sequence and employed formonitoring video perceptual quality during thetransmission.

Acknowledgement
This work was supported in part by a grant from the
Research Grants Council of Hong Kong, under Project
415712. The authors are grateful to Prof. Xuanqin Mou
and Mr. Wufeng Xue for providing their results forexperiment comparisons. Thank Prof. Irwan Prasetya
Gunawan and Mr. Kai Zeng for their valuable discussions.
References[1] W. Lin, C.-C.J. Kuo, Perceptual visual quality metrics: a survey,
Journal of Visual Communication and Image Representation 22 (4)(2011) 297–312.
[2] Z. Wang, H.R. Sheihk, A.C. Bovik, Objective video quality assessment, in:B. Furht, O. Marqure (Eds.), The Handbook of Video Databases: Design
and Application, CRC Press, 2003, pp. 1041–1078. Sept.
[3] G. Zhai, J. Cai, W. Lin, X. Yang, W. Zhang, M. Etoh, Cross-dimensional
perceptual quality assessment for low bit-rate videos, IEEE Transactions on Multimedia 10 (7) (2008) 1316–1324.
[4] P. Le Callet, F. Autrusseau. Subjective Quality Assessment IRCCyN/IVC
database. [Online]. Available: /http://www.irccyn.ec-nantes.fr/ivcdb/S.
[5] Z. Wang, A.C. Bovik, Modern Image Quality Assessment, Morgan &Claypool, New Youk, 2006.
[6] B. Girod, What’s Wrong With Mean-squared Error, in Digital Images
and Human Vision, MIT Press, Cambridge, MA, 1993 207-220.
[7] Z. Wang, A.C. Bovik, H.R. Sheihk, E.P. Simoncelli, Image quality
assessment: from error visibility to structural similarity, IEEE
Transactions on Image Processing 13 (4) (2004) 600–612.
[8] D.M. Chandler, S.S. Hemami, VSNR: a wavelet-based visual signalto-noise ratio for natural images, IEEE Transactions on Image
Processing 16 (9) (2007) 2284–2298.

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902[9] K. Yang, C.C. Guest, K.E. Maleh, P.K. Das., Perceptual temporal
quality metric for compressed video, IEEE Transactions on Multimedia 9 (7) (2007) 1528–1535.
[10] L. Ma, S. Li, K.N. Ngan, Visual horizontal effect for image
quality assessment, IEEE Signal Processing Letters 17 (7) (2010)
627–630.
[11] F. Zhang, L. Ma, S. Li, K.N. Ngan, Practical image quality metricapplied to image coding, IEEE Transactions on Multimedia 13 (4)(2001) 615–624.
[12] F. Zhang, W. Liu, W. Lin, K.N. Ngan, Spread spectrum image
watermarking based on perceptual quality metric, IEEE Transactions on Image Processing 20 (11) (2011) 3207–3218.
[13] S.S. Hemami, A.R. Reibman, No-reference image and video quality
estimation: applications and human-motivated design, Signal Processing: Image Communication 25 (7) (2010) 469–481.
[14] H.R. Sheikh, A.C. Bovik, L. Cormack, No-reference quality assessment using nature scene statistics: JPEG 2000, IEEE Transactions on
Image Processing 14 (11) (2005) 1918–1927.
[15] L. Liang, S. Wang, J. Chen, S. Ma, D. Zhao, W. Gao, No-reference
perceptual image quality metric using gradient proﬁles for JPEG
2000, Signal Processing: Image Communication 25 (7) (2010)
502–516.
[16] T. Brandao, M.P. Queluz, No-reference image quality assessment basedon DCT domain statistics, Signal Processing 88 (4) (2008) 822–833.
[17] Z. Wang, H. R. Sheikh, and A. C. Bovik, No-reference perceptual
quality assessment of JPEG compressed images, Proc. IEEE International Conference on Image Processing Sept. 2002.
[18] R. Ferzli, L.J. Karam, A no-reference objective image sharpness
metric based on the notion of just noticeable blur (JNB), IEEE
Transactions on Image Processing 18 (4) (2009).
[19] Z. Wang, A.C. Bovik, Reduced- and no-reference image quality
assessment: the natural scene statistic model approach, IEEE Signal
Processing Magazine 28 (2011) 29–40.
[20] S. Wolf, M. H. Pinson, Low bandwidth reduced reference video quality
monitoring system, Proc. International Workshop on Video Processing
and Quality Metrics for Consumer Electronics, Scottsdale, AZ, 2005.
[21] P. Le Callet, C. V. Gaudin, and D. Barba, Continuous quality
assessment of MPEG2 video with reduced reference, Proc. International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, Jan. 2005.
[22] S. Wolf, M.H. Pinson, Spatio-temporal distortion metrics for inservice quality monitoring of any digital video system, Proceedingsof SPIE 3845 (1999) 266–277.
[23] S. Yang, Reduced reference MPEG-2 picture quality measure based onratio of DCT coefﬁcients, Electronics Letters 47 (6) (2011) 382–383.
[24] P. Le Callet, C.V. Gaudin, D. Barba, A convolutional neural network
approach for objective video quality assessment, IEEE Transactions
on Neural Networks 17 (5) (2006) 1316–1327.
[25] M. Carnec, P. Le Callet, D. Barba, An image quality assessment
method based on perception of structural information, Proc. IEEE
International Conference on Image Processing 3 (2003) 185–188.
[26] M. Carnec, P. Le Callet, D. Barba, Visual features for image quality
assessment with reduced reference, Proc. International Conference
on Image Processing 1 (2005) 421–424.
[27] D. Tao, X. Li, W. Lu, X. Gao, Reduced-reference IQA in contourlet
domain, IEEE Transactions on Systems, Man, and Cybernetics Part
B: Cybernetics 39 (6) (2009) 1623–1627.
[28] U. Engelke, M. Kusuma, H.J. Zepernick, M. Caldera, Reducedreference metric design for objective perceptual quality assessment
in wireless imaging, Signal Processing: Image Communication 24(7) (2009) 525–547.
[29] Z. Wang, G. Wu, H.R. Sheikh, E.P. Simoncelli, E. Yang, A.C. Bovik,
Quality-aware images, IEEE Transactions on Image Processing 15(6) (2006) 1680–1689.
[30] Z. Wang, E.P. Simoncelli, Reduced-reference image quality assessment using a wavelet-domain natural image statistic model, Proc.
SPIE, Human Vision and Electronic Imaging (2005).
[31] L. Ma, S. Li, and K. N. Ngan, ’’Reduced-reference video quality assessment of compressed video sequences’’, IEEE Transactions on Circuits
and Systems for Video Technology, accepted.
[32] Q. Li, Z. Wang, Reduced-reference image quality assessment using
divisive normalization-based image representation, IEEE Journal of
Selected Topics in Signal Processing 3 (2) (2009) 201–211.
[33] L. Ma, S. Li, F. Zhang, K.N. Ngan, Reduced-reference image quality
assessment using reorganized DCT-based image representation,
IEEE Transactions on Multimedia 13 (4) (2011) 824–829.
[34] R. Soundararajan, A. C. Bovik, RRED indices: reduced reference
entropic differencing framework for image quality assessment,

901

Proc. International Conference on Acoustics, Speech, and Signal Processing, 2011.
[35] L. Ma, S. Li, K. N. Ngan, Reduced-reference image quality assessment via
intra- and inter-subband statistical characteristics in reorganized DCT
domain, Proc. Asia Paciﬁc Signal and Information Processing Association
Annual Summit and Conference, Oct., 2011.
[36] J.A. Redi, P. Gastaldo, I. Heynderickx, R. Zunino, Color distribution
information for reduced-reference assessment of perceived image
quality, IEEE Transactions on Circuits and Systems for Video
Technology 20 (12) (2010) 1757–1769.
[37] Z. Xiong, K. Ramchandran, M.T. Orchard, Y.Q. Zhang, A comparative
study of DCT- and wavelet-based image coding, IEEE Transactions on
Circuits and Systems for Video Technology 9 (5) (1999) 692–695.
[38] D. Zhao, W. Gao, Y.K. Chan, Morphological representation of DCT
coefﬁcients for image compression, IEEE Transactions on Circuits
and Systems for Video Technology 12 (9) (2002) 819–823.
[39] T.M. Cover, J.A. Thomas, Element of Information Theory, Wiley,
New York, 1991.
[40] R.W. Buccigrossi, E.P. Simoncelli, Image compression via joint
statistical characterization in the wavelet domain, IEEE Transactions on Image Processing 8 (12) (1999) 1688–1701.
[41] J. Liu, P. Moulin, Information-theoretic analysis of interscale andintrascale dependencies between image wavelet coefﬁcients, IEEE
Transactions on Image Processing 10 (11) (2001) 1647–1658.
[42] S. Lyu, E.P. Simoncelli, Nonlinear image representation using
divisive normalization, IEEE Conference on Computer Vision and
Pattern Recognition (CVPR) (2008).
[43] S. Lyu, Divisive normalization: justiﬁcation and effectiveness as
efﬁcient coding transform, Advances in Neural Information Processing Systems (NIPS) (2010).
[44] X. Zhang, W. Lin, P. Xue, Improved estimation for just-noticeablevisual distortion, Signal Processing 85 (2005) 795–808.
[45] X. Zhang, W. Lin, P. Xue, Just-noticeable difference estimation withpixels in images, Journal of Visual Communication and Image
Representation 19 (2008) 30–41.
[46] Z. Wei, K.N. Ngan, Spatio-temporal just noticeable distortion proﬁlefor grey scale image/video in DCT domain, IEEE Transactions on
Circuits and Systems for Video Technology 19 (3) (2009) 337–346.
[47] ITU-R Recommendation BT.500-11, Methodology for the subjective
assessment of the quality of television pictures, International
Telecommunications Union, Tech. Rep., 2002.
[48] S. Li, F. Zhang, L. Ma, K.N. Ngan, Image quality assessment byseparately evaluating detail losses and additive impairments, IEEE
Transactions on Multimedia 13 (5) (2011) 935–949.
[49] Genetic Algorithm Toolbox. [Online]. Available:/http://www.shef.
ac.uk/acse/research/ecrg/gat.htmlS.
[50] H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik, LIVE Image
Quality Assessment Database. [Online]. Available: /http://live.ece.
utexas.edu/research/qualityS.
[51] B. Li, M.R. Peterson, R.D. Freeman, Oblique effect: a neural basis in
the visual cortex, Journal of Neurophysiology (2003) 204–217.
[52] M.N. Do, M. Vetterli, Wavelet-based texture retrieval using generalized Gaussian density and Kullback–Leibler distance, IEEE Transactions on Image Processing 11 (2) (2002) 146–158.
[53] VQEG. Final report from the video quality experts group on thevalidation of objective models of video quality assessment, 2000.
[Online]. Available: /http://www.vqeg.orgS.
[54] H.R. Sheikh, M.F. Sabir, A.C. Bovik, A statistical evaluation of recent
full reference image quality assessment algorithms, IEEE Transactions on Image Processing 15 (11) (2006) 3440–3451.
[55] Y. Niu, M. Kyan, L. Ma, A. Beghdadi, and S. Krishnan, A visualsaliency modulated just noticeable distortion proﬁle for image
watermarking, European Signal Processing Conference, 2011.
[56] A.P. Bradley, A wavelet visible difference predictor, IEEE Transactions on Image Processing 8 (5) (1999) 717–730.
[57] S. Daly, The visible difference predictor: an algorithm for theassessment of image ﬁdelity, in: A.B. Watson (Ed.), Digital Images
and Human Vision, MIT Press, Cambridge, MA, 1993, pp. 179–206.
[58] ITU-T Recommendation J.246, Perceptual visual quality measurement
techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference, Aug. 2008.
[Online] Available: /http://www.itu.int/rec/T-REC-J.246/enS.
[59] I.P. Gunawan, M. Ghanbari, Reduced-reference video quality
assessment using discriminative local harmonic strength withmotion consideration, IEEE Transactions on Circuits and Systems
for Video Technology 18 (1) (2010) 71–83.
[60] K.T. Tan, M. Ghanbari, Blockiness detection for MPEG2-coded video,
IEEE Signal Processing Letters 7 (8) (2000) 213–215.

902

L. Ma et al. / Signal Processing: Image Communication 28 (2013) 884–902[61] A. Rehman, Z. Wang, Reduced-reference SSIM estimation, in Proc. IEEE
International Conference on Image Processing, Sept. 26-29, 2010.
[62] A. Rehman, Z. Wang, Reduced-reference image quality assessment
by structural similarity estimation, IEEE Transactions on Image
Processing., accepted, to appear 2012.
[63] D. Austin, Image compression: seeing whats not there, Feature Column,
American Mathematical Society, [Online]. Available: /http://www.ams.
org/samplings/feature-column/fcarc-image-compressionS.
[64] E.P. Simoncelli, W.T. Freeman, E.H. Adelson, D.J. Heeger, Shiftable
multi-scale transforms, IEEE Transactions on Information Theory
38 (2) (1992) 587–607.
[65] E.P. Simoncelli, W.T. Freeman, The steerable pyramid: a ﬂexible
architecture for multi-scale derivative computation, IEEE International Conference on Image Processing (1995) 444–447.

[66] Y. Horita, K. Shibata, Y. Kawayoke, and Z. M. Parvez, MICT Image
Quality Evaluation Database. [Online]. Available: /http://mict.eng.
u-toyama.ac.jp/mictdb.htmlS.
[67] L. Ma, K.N. Ngan, F. Zhang, S. Li, Adaptive block-size transform
based just-noticeable difference model for images/videos, Signal
Processing: Image Communication 26 (3) (2011) 162–174.
[68] W. Xue, X. Mou, Reduced reference image quality assessment basedon weibull statistics, Proc. International Workshop on Quality ofMultimedia Experience (QoMEX), Jun. 2010.
[69] M. Zhang, W. Xue, X. Mou, Reduced reference image quality assessment based on statistics of edge, Proceedings of SPIE 7876 (2011)
787611.
[70] E.C. Larson, D.M. Chandler, Most apparent distortion: full-reference
image quality assessment and the role of strategy, Journal of
Electronic Imaging 19 (1) (2010).