Bull. Hiroshima Inst. Tech. Research Vol. 56（2022）35-48

ArticleVisual Quality Balancing between Enhanced High-resolution and Ordinary
JPEG-decoded Images in Steganographic Image Compression System

Isao FURUKAWA ＊ and Junji SUZUKI ＊＊（Received Nov. 5, 2021）

Abstract
The steganographic image compression system provides a coding method that is compatible
with standard JPEG, but allows only users who know that the information of higher frequency
components is embedded to obtain enhanced high-resolution images at the same time. Until now,
the main focus has been on improving the quality of high-resolution images by embedding as many
codes as possible in lower frequency DCT coefficients. However, this conventional method leads togreater degradation in JPEG-decoded low-resolution images, and it is not appropriate to prioritize
high-resolution images over low-resolution images because of the inherent duality of providable
resolution images by the system. Therefore, in this study, we employ Watson’s model to evaluate thedistortion caused by embedding enhancement codes to lower frequency, and extend the perceptualmodel to higher frequency by extrapolating DCT frequency sensitivity values to calculate visual
distortion of higher frequency DCT coefficients when they are discarded due to a constraint of
payload. In addition, we propose a visual quality balancing (VQB) algorithm for low- and highresolution images by successively comparing their perceptual distances during enhancement code
embedding. Finally, we demonstrate the effectiveness of the proposed algorithm through simulation
experiments on standard test images with 4K resolution.
Key Words: steganography, image enhancement, JPEG compression, Watson’s model, JND

have investigated a steganographic image coding

1. INTRODUCTION

system in which the secret information is replaced

In image communications, a variety of resolution

by enhancement codes of high-resolution frequency

images are used depending on various uses ofcomponents of the image and only the user who

images, and the coding methods to achieve thisknows the secret can recover the high-resolution

goal are called multi-resolution image coding.

image from the embedded codes [2]-[4].

Such encoding systems a re also referred to

The conventional steganographic image coding

scalable coding, and JPEG2000 [1] is an example

system focuses on improving the quality ofof international standards for multi-resolution still

high-resolution images by embedding as many

images. On the other hand, there is a steganographic

enhancement codes created from higher frequency

technique that embeds irrelevant information inquantized DCT coefficients as possible within thethe image and conveys it as a secret message. We

payload derived from lower frequency quantized＊＊＊ epartment of Computer Science, Faculty of Applied Information Science, Hiroshima Institute of Technology
D
School of Information Science and Technology, Aichi Prefectural University (retired)― 35 ―

Isao FURUKAWA and Junji SUZUKI

DCT coefficients. However, since the system canprovide both enhanced high-resolution image andordinary JPEG -decoded low-resolution image,

2. BASIC SYSTEM CONFIGURALATION
2. 1 CODING SCHEME

it is also important to determine the number of

Figure 1 shows a basic configuration of theenhancement codes so that the visual quality ofsteganographic image coding system in which theboth resolution images is balanced. For this purpose,enhancement codes obtained from higher frequency

it is necessary to develop a method to adaptively

DCT coefficients are embedded in lower frequency

determine the maximum embeddable enhancement

DCT coefficients, so that both enhanced high-code length in order to balance both distortions inresolution and JPEG-decoded low-resolution images

terms of a human visual system by comparing theircan be obtained in the same decoder. After applying

visual impacts caused by embedding codes to lower

16×16 DCT to an input image, 16×16 except 8×8

frequency DCT coefficients and discarding higher

quantized DCT coefficients are run-length Huffman

frequency DCT coefficients. In this paper, we review

encoded to create enhancement codes, and thesesuch problems of the conventional method in thecodes are embedded in the LSBs of lower frequency

first place, and intend to solve them using Watson’squantized DCT coefficients. Then, the payload, which

visual model together with its extension to higher

corresponds to the upper limit of the embeddable

frequency DCT coefficients, and also examine thecode length, is imposed as a constraint in order noteffectiveness of the proposed method on standard

to excessively degrade the quality of the recovered

4K images.

image. The subsequent encoding procedure after theThis paper is organized as follows. The

enhancement code embedding is exactly the samenext section reviews the basic configuration ofas the standard JPEG algorithm. On the other hand,the steganographic image coding system withat the decoder, ordinary users can apply the JPEG

enhancement code embedding, and shows the

decoding algorithm to obtain a low-resolution image

requirements to balance the visual quality ofwith half the vertical and horizontal resolutions

enhanced high-resolution images and JPEG-decodedof the input image. In addition, special users who

low-resolution images. The procedure to evaluate

know that the secret information is embedded in thethe visual distortion using Watson’s model is

LSBs of lower frequency quantized DCT coefficients

also described in the section. Section 3 shows an

can extract the enhancement codes and perform

example of the distortion when the coding scheme

run-length Huffman decoding on them in order tois applied to a simple one-dimensional/four-point

recover the high-resolution image with the same

DCT/IDCT system. Moreover, actual 4K images

resolution as the input image.

are used as input to the conventional system, andThe conventional coding scheme [3] embeds as

several comments are made about these results.

many enhancement codes as possible within the

Section 4 discusses a relationship between the visual

payload of lower frequency DCT coefficients so as

distortion of lower frequency DCT coefficients andto make the quality of enhanced image high enough.

the enhancement code length, and extends theThus, in this case, the quality of enhanced high-frequency sensitivity to higher DCT coefficients

resolution image could be higher than that of JPEG-

in order to estimate the distortion caused bydecoded low-resolution image. Now that the quality

discarding these coefficients. Section 5 proposes aof high-resolution image is inversely related to thatmethod to control the amount of enhancement codesof low-resolution image under the constraint of theby successively comparing the visual distortions

embeddable enhancement code length, there is afrom lower and higher frequency DCT coefficients,

tradeoff between aforementioned two resolution

and verify the effectiveness of the method using

images. The most straightforward way to reduce theactual 4K images. Section 6 gives conclusions andquality degradation of JPEG-decoded low-resolution

future work.

image is to keep the payload relatively low. However,uniformly reducing the bit depth in all DCT blocks― 36 ―

D
wa
Dwa

where
where d(id(icontrast
contrast sese

Payload
Payloadlimit
limit

LSB
Encode 16×16 higher
Encode
DCT coefficients
Enhancement embedding
codes
PayloadCompute payload ofTarget
8×8 DCT coefficients 8×8 quantized DCT coefficientsTarget

16×16 DCT

Conventional
Conventionalmethod
methodHigh-resolution
input image

VisualVisualdistortion
distortionDD

Visual Quality Balancing between Enhanced High-resolution and Ordinary JPEG-decoded Images in Steganographic Image Compression Systemd ((ii,,d

Codestream

As
As aa resul
resul
single perpersingleWe
We conconcalculatincalculatin(a) Encoder
Steganographic decoding
Codestream

Decodebeforebefore quaqua

Enhancement code length CEcode length
CE
Recover Enhancement
High-resolution
16×16
16×16
coefficients
output
IDCT
Fig. 2 Enhancement code length vs. visual image
distortion.

LSB extraction andrun-length decoding

Fig. 2Standard JPEG decoding

but
but before
beforeQF1
QF1 andand Q
Q

Enhancement code length vs. visual distortion.

Recover
Low-resolution
8×8
bit
uniformly
two
distortionsare
bit depth
depth
uniformly when
when thetheIDCT
two kinds
kinds ofof output
distortions
are mixed
mixed
8×8 coefficients
image
in
a
singleimage.
in a single image.

coefficien
coefficien

enhancem
enhancem
quantized
quantized(b) Decoder

When capturing the distortions that occur in higher and lowerWhencapturing the
that
occur
in higher and lower
Fig. 1 Steganographic encoder/decoder configuration withenhancement
codedistortions
embeddingandextraction.

also has an effect of reducing the number of nonzero higher frequency components, so that it isnecessary to develop an enhancement code decision
technique that balances the visual distortions in both
frequency domains. In addition, since the distortion
occurring in lower frequency is mainly blockiness(i.e., square-shaped distortion) while the distortion
occurring in higher frequency is mainly blurring(i.e., indistinct contour), it is difficult to balance theimpacts of the distortions by simply reducing the bit
depth uniformly when the two kinds of distortions
are mixed in a single image.

Watson’s
Watson’s
where
where SSo cc

ofresolution
images,andimagesthetwotargetregionlocatedin decoded
the centerof aretheoftothesethese
tworesolution
images,and thethedecodedimagesareapparently
inappropriate
forthepurposeofthecodingsystem.
figure,where
thevisualqualityofbothimagesapparently inappropriate for the purpose of the coding system.istake their visual effects into account. Figure 2

In this study, we use Watson’s model [5] forshows a conceptual relationship between the

2.2
EVALUATIONVISUALDISTORTION
of OF
visualdistortion.
The procedure for
2.2evaluations
EVALUATIONOF
VISUALDISTORTIONIn
thisstudy,
weuse
Watson's
model
[5]
for
evaluations
of
model
the
steganographic
Inincorporating
this study, we use thisWatson'smodelinto[5] forevaluationsof visualvisualD in lower frequency increases, while D in higher

visualvisual dist
dist
block
block cau
cau

schemeareindicated
the
just
the
codingTherefore,
a method
to beschemeare system.
indicated byby
the dasheddashed line
line
just left
left totoneeds
the payload
payloadlimit.
However,
thisresults
inanimbalance
betweenthequalitydeveloped
enhancement
lengthlimit.
However,tothisrestrict
results inthe
an imbalance
betweencode
the qualityof2.
the2figure,figure,where thethe visualvisualqualityof bothboth DISTORTION
images isis balanced.
balanced.
EVALUATIONOF
VISUALofthewherequalityofimagesenhancement code embedding. As CE increases,

length mu
mulengthvalue ininvalue

frequency
decreases,sotherelationshipexistsfrequency
decreases,images,
so thatthat andthe tradeoff
tradeoff
relationshipexiststwo resolution
the decoded
imagesarebetweenthem.
Thevisualdistortions
with
theconventional
betweenthem. Thevisual distortions
the conventional
apparently
inappropriate
for withthe purposeof thehigher and lower frequencies, it is important tovisual distortions D of both frequencies by theenhancem
enhancem
QF2 asas w
w
QF2

frequencies
code
embedding.
Asfrequencies
bythetoenhancement
enhancement
code
embedding.
As CE
CEline just byleftthethe payload
limit.
However,thisincreases,
D
in
lower
frequency
increases,
while
D
in
higherincreases,
frequency
increases,
D inofhigher
results D
in inanlower
imbalance
betweenthewhile
qualitythese

When capturing the distortions that occur infrequency quantized DCT coefficients and the

PL
PL is
is thethehigherhigher fre
fre

enhancement
code length
lengthCE Theobtained
fromdistortions
higher frequency
frequency
enhancementcode
CEobtainedfromhigherexists between
them.
visualwithquantized
DCT coefficientscoefficientsand arethe visualvisualdistortions
D ofofdashed
both
quantized
DCT
and
the
distortionsDboththe conventional
schemeindicatedby theTherefore,
method needs
needs toto be
be developed
developed toto restrict
restrict thethebalanced.
Therefore,
aa methodenhancement code
code length
length toto thethe target
target region
region locatedlocated inin thethe center
centerenhancement

enhancement code length CE obtained from higher

QF1, andand
QF1,

frequencies,
frequencies, it
it is
is important
important toto taketake their
their visualvisual effects
effects into
into account.
account.
Figure2showsaconceptual
relationshipbetweenthe
Figure2showsaconceptual
relationshipbetweenthefrequency decreases, so that the tradeoff relationshipocoefficien
coefficien
The
The propro

above
above isis ss
fromfrom 16×
16×

and DCT
DCT
andcoefficien
coefficien

frequency
frequency
table
table isis giv
giv

distortion.
for
model
coding The
for procedure
calculating
the visualthis
distortion
distortion.
The
procedure
for incorporating
incorporating
thismodel into
intoisthetheassteganographic
codingforcalculatingthevisualdistortion
is
(i,
j)
and
S
(i,
j)
be
the
quantized
follows.
Letgsteganographic coding
for calculatingasee the visual distortion is ascoefficien
coefficien
CE
CE withwith ttfollows.
Let(i, j)j) andand SSafter(i, j)j) bebe
the quantized
quantized
DCT
coefficients
DCT Letcoefficientsembeddingtheenhancementee(i,follows.
ggee(i,the
DCT
coefficientsafter
embeddingtheenhancement
codes
and
inversecodes
and their
quantized
coefficients,
after
embeddingthe inverse
enhancement
codes DCT
and their
their
inversequantized
DCT coefficients,
coefficients,
respectively.
In j)addition,let(i, j)
j)respectively.
In addition,
let So(i,be a letreferencequantized
DCT
respectively.
In
addition,
SSo(i,

lower
lower freq
freqoSo (0,(0, 0)0)  S
tt L ((ii,, jj )) 
 ttbb ((ii,, jj ))  oS
L0,0 S0,0 



,,



0.7

14

onventional method

(3)
(3)

Payload limit

by thesensitivity
contrastthresholdsensitivity
contrastt(i, j):threshold t(i, j):― 37 ―d (i, j ) Se (i, j )  So (i, j ).
t (i, j )(4)

As a result, the individual errors of Equation (4) are pooled into asingle perceptual distance Dwat(So, Se) of Equation (3).

depends oo
depends
natural ququnatural(2)(2)(2)

wherej) theis theof DCT
coefficientswhered(i,d(i,j) iserrorerrorof DCT
coefficients
divideddivided
by theFig. 2 Enhancement code length vs. visual distortion.

lower
lower and
andenhancem
enhancem(1)(1)(1)

 7 7
4
Dwat ( So , Se )    d (i, j )  , 
j 0i 0

Enhancement code length CE

Visual distortion D

0.649
0.649

0.3
0.7
tt ((ii,, jj ))  max
max tt LL ((ii,, jj ),), SSoo ((ii,, jj )) tt LL ((ii,, jj )) 0.3 ,, 

Bull. Hiroshima Inst. Tech. Research Vol.56 (2022)
Target3.
3. EXAM
EXAM
As
As explain
explainwith
the next
next withthree-step
calculations.
distance
the calculations.
next three-step calculations.
withthethree-step
Payload limit

Conventional method

Visual distortion D

beDCT
reference
DCT coefficientscoefficientsfor thethe distortion
distortion
evaluation.
coefficientsfor the distortion
evaluation.
Then,beaa reference
DCT
forevaluation.
Then,
Watson'smodel
model isis usedused
to evaluate
evaluate
the perceptual
perceptual
distance
Watson’sused
to evaluate
the perceptual
Then,
Watson's
model
is
to
the
distance

33higher fre
frehigher

14

 7 7
4
Dwat ( So , Se )    d (i, j )  ,j 0i 0


(3)where d(i, j) is the error of DCT coefficients divided
by theIsao FURUKAWA
and Junji SUZUKI
contrast sensitivity threshold t(i, j):d (i, j ) Se (i, j )  So (i, j ).
t (i, j )DCT coefficients are lower frequency coefficients,(4)(4)and DCT coefficients excluding them are higher

a result,
the individual
errors of(4)Equation
AsAs
a result,
the individual
errors of Equation
are pooled (4)intoareafrequency coefficients. The perceptual distance isof(3).DCT coefficients So(i, j) as the reference for
WeEquationconsider two

the DCT frequency sensitivity table is given only

referencefor calculating
the Sperceptual
distance:
beforequantization,
or DCT coefficientsq(i, j) after quantization

coefficients are used to create the enhancement codes

coefficientsSq(i,j) afterlower
quantization
beforebitQF1and QF2 used
to quantize
and higher but
frequency
DCT

determined from the lower frequency coefficients.

pooledinto adistancesingleDperceptual
distance D (So, Se)
singleperceptualwat(So, Se) of Equation (3). wat

calculated for lower frequency coefficients, because

We consider
two DCT
coefficients
So(i, j)Sbas
calculating
the perceptual
distance:
DCT coefficients(i, thej)

to 8×8 DCT components. The higher frequency

.

Sb(i,
j)parameters
before quantization,
DCT
butDCT
beforecoefficients
bit-embedding.
The
are the qualityor
factorsof length CE with the constraint of the payload PL

mixed

embedding.
The parameters
are thefactors
coefficients,
respectively,
and the bit depthnLSB quality
up to which
the

lower

frequencycoefficients,
respectively,
and thequantized
DCTDCT
coefficientsare obtainedby the quality
factorbit

QF1 and code
QF2bitsusedto embedded.
quantize The
lower
higherenhancement
can belowerandfrequency

3. EXAMPLES OF DISTORTION

depth
nLSB
up toPLwhich
the enhancement
bits
QF1,
and thepayloadis determinedby the bit depth code
nLSB, i.e.,count.
theuency

f boths CEhigherexiststionalyload

ualityes arestem.

As explained above, there is a tradeoff between

besum
embedded.
Theoflower
frequency
PLcan
is theof the number
embeddable
bits forquantized
non-zerothe distortions of lower and higher frequency

factor QF1,
is quality
determinedenhancement
code andlengththe
CE ispayload
determinedPL
by thefactorcode length. The optimization of this coding

numberembeddable
for non-zerohigherlengthmust beofequal
to or less thanbits
the payload.
Substituting
eachcharacteristics, and the natural question is how

the distortion
enhancement
code
lengthDCT
CEcoefficientsis determinedthevisualof lowerfrequency
in oneby
DCT

frequency DCT coefficients respectively. As athe enhancementmust
be equal
Watson’smodel, i.e., the codepooledlength
perceptual
distance
Dwat(So, toSe),orof simple one-dimensional/four-point DCT/IDCT

the above three-step calculations, we can obtain thecoefficients are used.

when the conventional coding scheme is applied to

DCTfrequency
coefficientsareobtainedby Moreover,the quality
higherquantized
DCT
coefficients.
the

DCT coefficients with respect to the enhancementby asthedepthnLSB , CE
i.e.,
PLthe
is enhancement
the sum oftheQF2
wellbitas theinequality
≤ PL:
code

system depends on the image content and visualfrequency
quantized
coefficients.
value
in the above
three-stepDCT
calculations,
we can Moreover,obtain theto allocate the distortion to the lower and higher

qualityfactorQF2
as well ascodethe inequality
CE ≤ onPL:
block
causedby theenhancement
embedding based

preliminary study, we will first consider a case

lessS thanthe payload. Substituting each value inwhere
o can be either Sb or Sq according to which reference DCT

system, and then demonstrate the above tradeoff

visualdistortion of lower frequency DCT coefficients
The procedure for evaluating the perceptual distance describedct theactual 4K images.

in one DCT block caused by the enhancement code

centerabove is shown in Figure 3, where 8×8 DCT coefficients picked

nced.

from 16×16 DCT coefficients are lower frequency coefficients,

3. 1 E XAMPLE FOR 1-D/4-POINT DCT/IDCT

embedding based on Watson’s model, i.e., the pooled
perceptual distance D

SYSTEM(S , S ), where S can be

wat
o
e
o
and DCT coefficients excluding
them
are higher frequency

either S or S according to which reference DCT

C onsider 1-D/4 -point D C T/ I D C T system ,qcoefficients. b The perceptual
distance is calculated for lower

coefficients are used.

where DCT coefficients are denoted by S(0), ...,

visualfrequency coefficients, because the DCT frequency sensitivity

to thetable is given only to 8×8 DCT components. The higher frequency

is ascoefficients are used to create the enhancement codes of length

cients

CE with the constraint of the payload PL determined from thenverselower frequency coefficients.

Lower frequency coefficients

ation.

3. EXAMPLES OF DISTORTION

Sb(i, j)(0 ≤ i, j ≤ 7)

stanceAs explained above, there
is a tradeoff
Qualityfactor between the distortions of

The procedure for evaluating the perceptual

S(3), and the reconstructed signals after IDCT are

distance described above is shown in Figure 3,

denoted by s(0), ..., s(3). If we apply this system

where 8×8 DCT coefficients picked from 16×16

to the steganographic image coding system, S(0)o(i, j)

8×8
DCT
16×16 DCT
excluding 8×8

Sb(i, j)(0 ≤ i, j ≤ 15 excluding 0 ≤ i & j ≤ 7)

lower and higher frequency QF1
DCT coefficients with respect to the

Quality factor
QF2

gq(i,ofj) this coding systemenhancement code length. The optimization

Payload PL

Quantization

Bit depth nLSB

1)Higher frequency coefficientsdepends on the image content and visual characteristics, and theEnhancement codes (length CE ≤ PL)

Quantization
gq(i, j)Run-length encoding

Embedding to LSBs

natural question is how to allocate the distortion
to the lower andg (i, j)2)e3higher frequency DCT coefficients
respectively. As a preliminary
Dequantization
So(i, j) = Sq(i, j)
So(i, j) = Sb(i, j)

Fig. 3Distorted DCT Se(i, j)
Reference DCT So(i, j)

Watson’s model

Perceptual distance
Dwat(So, Se)

Procedure for evaluating perceptual distance of 8×8 DCT coefficients with Watson’s model.
― 38 ―

the above tradeoff when the conventional coding scheme isappropriate positive number, and assume that the existence of

applied to actual 4K images.

S(2) and S(3) is determined according to the following rules.
m If 0 ≤ a ≤ 1/2 (the lower frequency distortion is relatively

Visual Quality Balancing between Enhanced High-resolution and Ordinary JPEG-decoded Images in Steganographic Image Compression System

3.1 EXAMPLE FOR 1-D/4-POINT DCT/IDCT SYSTEM

small), then S(2) exists and S(3) is missing.
mConsider 1-D/4-point
DCT/IDCT S(1)system,where DCT
corresponds
to DC coefficient,corresponds

 If 1/2 < a (the lower frequency distortion is relatively large),

after IDCTandare denotedby s(0),..., s(3).
If we applyto
thisbesignals
embedded,S(2) andS(3)correspond

Applying this model to the 1-D/4-point DCT/IDCT system, thethen both S(2) and S(3) exist.

correspondto higher
frequency value.
DCT coefficientsto be run-length
and
it retains
the original
The bit-embedding
to the rules
this codingofsystem,
isHuffmanappliedencoded.
only toAccording
S(1), resulting
in of
distortionthe2

2 Only2S(2) 2exists 22

 2 22 2 2 2
2

2  2 1
 2  2  1 e  a 

. (6)
2


2  2  1 e
3
1 e  h3 (m)  2  2 

m=5

2 2

m=6

frequency
DCT coefficientsto be embedded,
and S(2)S(3)
codingsystem,no bit-embedding
is applied
toand
S(0),

2

m=7


 s (0) 




(1)
s
1

 
 s (2)  4  s (3) corresponds
to DC coefficient,S(1) corresponds
to a uniquelower
Huffmanencoded.
Accordingto the rulesof thism=4

reconstructed signals s(0), ..., s(3) can be expressed asm=8system frequency
to the steganographic
image tocodingsystem, S(0)higher
DCT coefficientsbe run-length

m=3

Rms distortion for whole signals of s(i)denotedby S(0), ..., DCT
S(3), andthe reconstructedtocoefficientsa uniquearelower
frequencycoefficientstoIn other words, the lower frequency distortion a is added to S(1),value.
The bit-embedding
is applied only theyto S(1),arehigherfrequency components,
areresulting
eitherin

and the existence of S(3) is determined by the following function

m=9

no bit-embedding
applied to S(0),Since
and it retains
the originallower
frequencyiscomponents.
S(2) andS(3)distortionorof absent;
the lower whenfrequency
components.
SincetoS(2)and S(3)presentS(3)in addition
S(2)ish3(m).

components
The(blurring
tradeoff is that
whenand S(3)bothlikely istosmall.
exist
will
be S(2)
small),
theareenhancement codes are embedded in lower simulation
frequency experiments
DCT[4]. Coicients 8×8
Higherfrequencycoefficients
fficients
Higherfrequencycoefficientsare higher
frequency
components,
they arecaused
either present
0,
8×8
if 0  a  1 2m
unlikelytobe present,
thedistortionby theor(7)h3 (m)  
DCT
m
DCT
1,
if
1
2

a
absent;
when
S(3)
in
additiontoS(2)isunlikelytobepresent,theAdditionalerrorabit-embedding to lower frequency
isSbS(i,(i,j)j) components
Bull. Hiroshima Inst. Tech. Research Vol.56 (2022)b
16×16
16×16DCT
DCT
Let so(x)be 4theDistortion
reconstructedsignal after
IDCT
that nodistortion
causedbythebit-embedding
tolowerfrequency
Fig.
of recovered
signalin assuming
1-D/4-point
(0
≤
i,
j
≤
15
excluding
0
≤
i
&
j
≤
7)
small.
when S(2)(0 ≤isi, jthat≤ 15 excluding
0 ≤ andi & j ≤S(3)7) areexcluding
8×8
excluding
8×8The tradeoff
DCT/IDCT system.

coefficients,
andthealsoreconstructedthat both the higher
frequency
DCT AC coeffic
non-zeroLet so (x) be
signalafter quantized
IDCT
m=3

coefficients
exist (i.e., no
encoding
processisfrequency
components
when thassuming S(2)thatandno S(3)enhancement
codes
are embedded

applied),let se(x) beDCT
the reconstructedsignalobtainedby (c) and (d) c
isand
used.
Conditions
in lowerandfrequency
coefficients,
also
that
m=4

IDCT
bit-embedding.
Since
Watson's
2-D
bitmodeldepth foris reduced
to 2 and 1both after
the thehigherfrequency
DCT
coefficients
S(2)
m=5

Rms distortion for whole signals of s(i)Quality
Qualityfactor
factorQuantization
both likely to
exist(blurring
willQuantization
be small),the distortionof lower
distortion
ofQF2
lower
frequency
components
is large
QF2
g
(i,
j)
j) be large).
frequency components
is large (blockinesswill
qgq(i,
will be large).
itdepthdepthnLSB
nLSB (blockiness
PayloadPL
PayloadPL
We willthis thisrelationshipusing
a simplefor theRun-length
encodingWe
willevaluate
evaluate
relationshipusing modela simple
Run-length
encoding
Enhancementcodes(lengthCE≤
PL)
Enhancementmodel
codes(lengthCE frequency≤ PL) S(i). Assume
frequencythat the S(i).
DCT Assume
coefficientsfor components
thecomponents

decrease
exponentiallywith frequency.
This
is a reasonable
DCT
not available
here, theprocesserrorfactors,
betweenthem
andisQF2, are the sa
and coefficients
S(3) existis(i.e.,
no encoding
is QF1applied),thatthe DCT
coefficientsdecrease
exponentially
Only S(2) exists

0, ...,
3.DCT
Furthermore,
the Watson’s
distortion
caused by the bittual
coefficientsmodel.
ptualdistancedistanceofof8×8
8×8DCT
coefficientswith
withWatson’smodel.

e oneone-

nstratemonstrateemehemeisis

EM
EM

e DCT
DCT

tructedstructedply
pplythisthis, S(0)m,S(0)euelowerlower

ndandS(3)S(3)-lengthn-lengthystem,
system,original
originalting inin
ultingndandS(3)S(3)sentesentoror

ent,
sent,thethequencyequency(3)S(3)areareembedding in lower frequency DCT coefficients S(1)Furthermore,
bit-embedding
ininlower
Furthermore,thedistortion
causedbythe
thebit-embedding
lower
isdenotedthe
bydistortiona. Let causedmbebyanappropriate
positivefrequency
S(1)isisdenotedbybya.a.ofLet
mmbebeananfrequencyDCT
DCTcoefficientscoefficients
S(1)thedenotedLet
number,andassumethatexistence
S(2)
and

m=7

The thresholdlower frequency
distortion
a,
which
Table
1 Simulation pa
IDCT
after thefor
bit-embedding.
Since
Watson’s
model

Conditions
determines
the existenceof the higher
frequencycoefficientfor 2-D DCT
coefficientsis notavailable
here,S(3),the(a)
Maximum SNR
iserrorvariedbetween
for integerthem
values isof m= 1, 2, ..., 9.
4 shows
theevaluatedbyFigure
the rootmean(b)More non-zero AC coefsquare (rms) for whole signals of s(i). (c)Bit depth is reduced to4(d)
Bit depth is reduced to

m=8

m=9

example,makewhen dealing with actual images. For example,
Perceptual
Perceptualdistancedistance
Distorted
Distorted DCT
DCTSeS(i,(i,j)j)  ieS(i )  e ,  Watson’s(5)(5)
Watson’smodel
model
DD
(S(S
, eS)e)Reference
j)j)
wat
o,oS
ReferenceDCT
DCTSoS(i,(i,watowhere ee isis thethe baselogarithm
and i and= 0, i...,= 3.
wherebaseofofnatural
naturallogarithm

m=6

evaluatedtheberootthemeansquare (rms) forsignal
whole signalsof s(i).
and let byse(x)reconstructedobtainedbyassumption
to makewith actual
images. toForwithfrequency.
Thiswhenis adealing
reasonableassumption

The threshold for lower frequency distortiona, which determines the existence of the higher

enhancement codes and the SNR
appropriate
positiveandofappropriate
positivenumber,number,andassume
assumethatthetheexistence
existence
S(3)is determinedaccordingto thethat
followingrules.
m = 1, signal
2, ..., in9.1-D/4-point
Figure 4 shows the evaluation
Fig.
4 of
Distortionofof recovered
images based on conditions (a)S(2)accordingrules.
DCT/IDCT
S(2)
and
S(3)
determinedaccordingtothe
thefollowing
followingrules. is system.results of the rms of distortions by varying m. The・I fand
0 S(3)≤ aisisdetermined≤
1/2m (thelower tofrequency
distortion
mm
table, RH code stands for run IfIfrelatively
00≤≤aa≤≤1/2
(the
lower
frequency
1/2(thethenlowerfrequency
distortion
relatively
small),
S(2)exists distortionandS(3) isisisrelatively
missing.
rms error has discontinuities at a = 1/2m because
enhancement code. The rate of R
m S(2) exists and S(3) is missing.
small),
thenevaluation results
rms ofthedistortions
varying m. The
rms
small),
thenS(3) frequency
is missing. distortion・I f 1/2<S(2)a exists(the andlower
is of the only
higherbyfrequency
coefficientS(2) exists formmof
RH codes to that of the JPEm IfIfrelatively
1/2
lower
frequency
relatively
large),
bothdistortion
S(2) andisisS(3)exist.
a, whileboth
error
haslarge),
discontinuitiessmaller
at a = 1/2
because
onlyS(2)the andhigherS(3) exist for larger
1/2 <<aa(the(thelowerthenfrequency
distortion
relatively
large),
statistical values of enhancemen
then
S(3)
coefficient S(2)existsexample,
for smallerthea, while
both S(2)thenboth
bothS(2)S(2)andS(3)exist.
exist.
Applyingthisandmodel
to the 1-D/4-point frequency
DCT/IDCT
a. Fortransitionof the rms error withblocks containing no enhancem
Applying
system,
the
S(3)
exist
for larger increasing
a. For example,
transition of
the rms
Applyingthis
thismodel
modeltotothe
the1-D/4-point
1-D/4-point
DCT/IDCT
system,thesystem,thereconstructedsignalsDCT/IDCT
s(0), ...,and
s(3)
can
be
thetheadditionalerror
a for m = 7 is shown
calculation. The quality of highreconstructedsignals
asas with increasing theinreconstructedas
signalss(0),s(0),...,...,s(3)s(3)cancanbebeexpressed
expressed
error
additional
errorasa for= 7 isshown
in an arrow. For m =expressed
the figure
am
solid
line
with
4
SNR in the table, and the percep
42
2 2  2 222
2 2  2 2 
a =an1/2= For
0.0625,corresponds
to the caseas a solid line4,witharrow.
m = which4, a = 1/2= s(0)   1the
1 figures (0)    2
forlow-resolution  




1
e
a

2 2  2 2   2 2   2 2  2 2  1 e0.0625,
where
17% ofthe17%
bit-embedding
distortion is images will bwhich corresponds
to theaboutcase whereaboutof the a   (6)s (1)  1 1 2 2 s (1)
 
.
 
(6)
.
2 2
(6)

 s(2)e s (2)  4 4 
2 2  2 2   1 1ebit-embedding
forforS(1)For m ≤ 4, assuming
distortionincluded
is included
S(1)== 1/e
1/e  0.368.
0.368. For      2 2   2 2  2 2   2 2
  3 3
1
e
h
(
m
)

s
(3)
  s (3)   Table 2 Simulation  1 e 3h3 (m) 
2
2
2
2
2
2




that
the
higherfrequencycoefficientS(3) exists atm ≤ 4, assuming that the higher frequency coefficient S(3) exists
2  2  2  2  2 2(a) u01
In
words,
thedistortion
isis
In
other
words,
the frequency
lower
frequencyaS(1),
a, theasymptotic
where
ataddedlargertotoa,S(1),
theiserrors arelarger
asymptotic
for errors
the caseare
where
only S(2) for the case
Inotherotherwords,
thelowerlower
frequency
distortionaadistortion
added
Conditions
(a)
45.83
and
isisdeterminedfunction
addedtoS(1), ofandtheexistenceofS(3)isexists
determinedexistscase whereS(3)
RH andMean
andthetheexistence
existenceofS(3)
S(3)determinedby
bythe
thefollowing
following
functionandthe case whereonly
bothS(2)S(2) andS(3)andexist.theTherefore,
it is both S(2)
218
codes
Max
hby
the following function h3(m).
exist. at
Therefore,
it iswithobviousthat the error
increases
obvious that the error increases
a constant ratea regardless
3h(m).
3(m).
4(bits)
Min
185,600
0,
ata
constant
rate
with
a
regardlessofwhether
S(3)
Total
RH
(bytes)ifif 00aa1122m m
of whetherS(3)ispresent
or
not,
when
the
bit-embedding
causes0,(7)(7)
h3h(3m) )

(7)
(m
9.47
Rate of RH (%)
ifif1122m maapresent
not, when
the bit-embedding
causes an1,an error of 20% or moreis
relativeto theormagnitudeof S(1).
1,
29.24
SNR (dB)Let
be the reconstructed signal after IDCT assuming that noerror of 20% or more relative to the magnitude of S(1).
Letsos(x)o(x) be the reconstructed signal after IDCT assuming that no(b) u04_
enhancement
DCT
3.2 EXAMPLE
enhancement codes
codes areare embedded
embedded inin lower
lower frequency
frequency
DCT FOR 4K IMAGES

foflower
lower

― 39
―
coefficients,
coefficients, andand also
also thatthat both
both thethe higher
higher frequency
frequency
DCT
Next, weDCT
show
simulation
results when the conventional.

coefficientsprocess
steganographic
coefficientsS(2)S(2)andandS(3)S(3)exist
exist(i.e.,(i.e.,nonoencoding
encoding
processisiscoding scheme is applied to actual 4K images. In
applied),thisobtained
case, theby
bitapplied),and
andletletses(x)(x)bebethe
thereconstructedreconstructedsignal
signalobtainedby depth n changes from 0 up to 6 depending on

lfor
forthetheTable 2 shows the statisticfrequencycoefficient
S(3), is varied for integer values
Additional
error aConditions
Mean
RH
codes
Max
(bits)
Min
Total RH (bytes)

(a)
27.49
226
4
111,335

Isao FURUKAWA and Junji SUZUKI

3. 2 EXAMPLE FOR 4K IMAGES

Table 2

Next, we show simulation results when the

Simulation results for 4K images(a) u01_Books

Conditions
(a)
Mean
45.83
RH
codes
Max
218
(bits)
Min
4
Total RH (bytes) 185,600
Rate of RH (%)
9.47
SNR (dB)
29.24

conventional steganographic coding scheme isapplied to actual 4K images. In this case, the bit
depth nLSB changes from 0 up to 6 depending on
the magnitude of the DCT coefficients. Two 4K
images of u01_Books and u04_Kimono [7] are used
as input. The simulation compares and evaluates thereconstructed images in case that the quality factor

(b)
71.48
216
4
289,505
14.62
28.07

(c)
30.37
116
4
122,992
6.27
28.82

(d)
19.17
62
4
77,619
3.95
28.29

(c)
18.57
116
4
75,200
4.92
30.71

(d)
12.52
63
4
50,6883.32
30.04(b) u04_Kimono

Conditions
(a)
Mean
27.49
RH
codes
Max
226
(bits)
Min
4
Total RH (bytes) 111,335
Rate of RH (%)
7.29
SNR (dB)
31.05

QF2 = 95 or 70 and the bit depth nLSB = 6, and also
in case that QF2 = 70 and nLSB = 2 or 1.
Table 1 summarizes the parameter combinationsof QF2 and nLSB corresponding to the four conditions(a)-(d) for the simulation. The quality factor for
lower frequency DCT coefficient quantization is

(b)
44.88
225
4
181,768
11.83
30.06

always fixed to QF1 = 95. Condition (a) corresponds

Figure 5 shows the high-resolution images

to the situation that yields maximum SNR in theobtained by using the parameter sets of conditions

simulation experiments [4]. Condition (b) is for(a)-(d) for the 4K image u01_Books. The upper

creating more non-zero quantized AC coefficientsleft coordinate of the cropped partial images iswith emphasis on higher frequency components(1500, 1000), and the number of horizontal andwhen the same bit depth as condition (a) is used.

vertical pixels is (240, 180). Some comments on theConditions (c) and (d) correspond to the case where

result are as follows.

the bit depth is reduced to 2 and 1, respectively,・I mage (a) shows the highest quality based onwhen the quality factors, QF1 and QF2, are the

SN R among various parameter settings insame as in condition (a).

the conventional method that uses the same
parameters for all DCT blocks.

Table 1

(a)
(b)
(c)
(d)

・I mage (b) has more non-zero higher frequency

Simulation parameters for 4K images.

Conditions
QF1
Maximum SNR
More non-zero AC coefficients95
Bit depth is reduced to 2 in (a)Bit depth is reduced to 1 in (a)

QF2
70
95
70
70

nLSB
6
6
2
1

quantized DCT coefficients than image (a). In thiscase, the payload of lower frequency quantized
DCT coefficients is the same as in condition (a),so the allowable enhancement code length reaches
the payload at an early stage during higher

Table 2 shows the statistics of the amount of

frequency coefficient scanning, and the otherembedded enhancement codes and the SNR of the

non-zero coefficients after the truncation point

enhanced high-resolution images based on conditions

are discarded. As a result, only relatively lower(a)-(d) for the two 4K images. In this table, RH

components in the higher frequency domain arecode stands for run-length Huffman code, i.e., thepreserved, and the image appearance of (b) inenhancement code. The rate of RH means the ratio

terms of resolution is inferior to that of (a).

of the amount of RH codes to that of the JPEG

・In images (c) and (d), the payload is smaller thancodestream. For the various statistical values ofimages (a) and (b) due to the smaller bit depth

enhancement codes per block, the number of blocks

constraint. Therefore, the allowable enhancement

containing no enhancement codes are excluded fromcode length is also reduced, and the image looks likethe calculation. The quality of high-resolution images

it has undergone low-pass filtering, but instead, theis evaluated by SNR in the table, and the perceptual

blockiness is alleviated in comparison with the image

distance by Watson’s model for low-resolution images(a). The effect of this distortion tradeoff is, of course,

will be evaluated in 4.1.

more emphasized in the image (d) than in the image (c).
― 40 ―

Visual Quality Balancing between Enhanced High-resolution and Ordinary JPEG-decoded Images in Steganographic Image Compression System(a) QF 2 = 70, nLSB = 6(b) QF 2 = 95, nLSB = 6(c) QF 2 = 70, nLSB = 2(d) QF 2 = 70, nLSB = 1

Fig. 5Enhanced high-resolution images with varying QF2 and nLSB for u01_Books (QF1 = 95).

・Even though the number of enhancement codes

the enhancement codes corresponding to the DCT

embedded in the image (b) is larger than in thecoefficients are embedded. On the other hand, forimages (a) and (c), it does not contribute to SNR

blocks in relatively flat regions of the image where

improvement. In other words, excessively raising

the higher frequency DCT coefficients are all zeros,

the quantization quality factor for higher frequency

only a 4-bit EOB code is embedded if the payload is

DCT coefficients can have negative effects on the

4 or more, and no codes are embedded if the payload

visual quality of the reconstructed image.

is less than 4. This embedding of a single EOB code
cannot be avoided due to the termination of run-

4. EXTENDING FREQUENCY SENSITIVITY

length Huffman codes, and it should be consideredseparately from other cases of the bit-embedding.

4. 1 E MBEDDED CODES VS. PERCEPTUAL

Figure 6 shows, for DCT blocks where code

DISTANCES

embedding has been performed, the histogram

In this section, the conceptual relationshipof the occurrence frequency of the perceptual

between the enhancement code length and the

distance values for DCT blocks that have only

visual distortion of the lower frequency DCT

EOB as an enhancement code. Figure 7 shows, as

coefficients shown in Fig. 2 is examined for actual

examples for the image u01_Books and the quality

4K images, u01_Books and u04_Kimono. The bit

factor (QF1, QF 2) = (95, 70), the 3-D histogram

depth is set to n LSB = 6 and the combination of

for the occurrence frequency of the combination

quantization quality factors is (QF1, QF2) = (95,of enhancement code length and the perceptual70) and (70, 70). We also consider two reference

distance for blocks whose enhancement codes are

signals for calculating the perceptual distancenot EOB code only. The contour plot shows the same

as explained in 2.2: the DCT coefficients after

occurrence frequency of in the form of bird’s-eye

quantization and before bit-embedding, as wellview. When the reference signal for evaluating theas the DCT coefficients before quantization. For

perceptual distance is So = Sb, both the quantization

DCT blocks where higher frequency coefficients

distortion and the bit-embedding distortion exist,

are not all zeros and the payload is greater than 4,

resulting in a distribution with a larger perceptual― 41 ―

Isao FURUKAWA and Junji SUZUKI

300
250

Frequency

200
150
8

100
6

50
0

4
50

100

Perceptual distance

2
150

200

250

(a) Dwat(Sq, Se)(a) u01_Books

300

Frequency

250
200
150
8

100
6

50
0

Perceptual distance(b) u04_Kimono

4
50

100

2
150

200

250

(b) Dwat(Sb, Se)

Fig. 6 Histogram of perceptual distance for DCT blocks
with only EOB code.

Fig. 7 Histogram and its contour of perceptual distance for
u01_Books using conventional method with (QF1, QF2) = (95, 70).
(Except DCT blocks having only EOB code)

distance than when the reference signal is So = Sq.
This shows that the perceptual distance is almost
directly promotional to the enhancement code

distance can be quantitatively expressed using

length. In addition, although the payload is smaller

Watson’s model from the frequency thresholds of 8

in the case of QF1 = 70 than in QF1 = 95, which

×8 DCT coefficients, though similar thresholds foris omitted here, the distribution moves toward thehigher frequency coefficients are required, since ourlarger value of perceptual distance even thoughimage compression scheme deals with 16×16 DCT

the enhancement code length becomes smaller

coefficients. Therefore, it is necessary to compare

accordingly.

the visual distortion caused by discarding higher
frequency DCT coefficients with that caused by

4. 2 A P P R OX I M AT I O N O F F R E Q U E N C Y

embedding enhancement codes in lower frequency

SENSITIVITY

DCT coefficients, and consider to what extent it is

The frequency sensitivity table [5, 6] for 8×8

optimal to embed the codes within the payload from

DCT coefficients are obtained by expressing thethe view point of overall image quality. Of course,

threshold as a quadratic function of the frequency

the visual distortion caused by bit-embedding inboth in their logarithmic form. The perceptual

lower frequency and coefficient-discarding in higher― 42 ―

VisualQuality Balancing
betweenEnhanced
High-resolution and Ordinary JPEG-decoded Images in Steganographic Image Compression System
Bull.
HiroshimaInst. Tech.
Research
Vol.56 (2022)

coefficients
caused byenhancement
frequencywith
is thatdifferent:
theembedding
former isblockinesscodesandthe next
enhancementcode
the LSBs offorlower
frequency
considertheinprocedurebalancing
theDCTvisualinthe
lowerfrequency
DCT coefficients,
and considerimage
to what inextentlatteris blurring.
The enhanced
thecoefficients,
and caused
consider byhowtheto determine
the allowabledistortion
steganographic
image

itdecoderis optimal contains
to embed thecodesdifferent
within the kinds
payloadoffrom
the view
thesedistortions

enhancement
length In
thatotherbalanceswords,
total visual
coding code
scheme.
we distortion.
compare thepointof overall
image soquality.
course, the visual
at thesame time,
thatOfa subjective
imagedistortion
quality

Assumethe i-th non-zero
is S[i] athroughvisual that
distortion
caused coefficientby discarding
certaincausedby bit-embedding
lower
frequencythe
andappropriate
coefficientevaluation
is essentialin to
determinescanning
higherfrequencyDCT
quantized
DCT coefficients,
andcausedthehigherfrequencycoefficientwith that

discarding
higher frequency
balancinginbetween
them. is different: the former isenhancement
code length
correspondingcode
to thatnon-zeroby embedding
the enhancementin theLSBs ofblockinessthe latter isblurring. Thethresholds
enhanced image
theThe andfrequency
sensitivity
for in8×8

coefficient
CL[i]. Moreover,that theandvisualdistortionhow
lower isfrequency
DCT suppose
coefficients,
considerdecoderthese different
kinds
of distortions
at theonsame
DCT contains
coefficients
described
above
are basedanby discarding
the non-zerocoefficientenhancement
is S[i]/zxy. Here,code
zxy is length
theto determine
the allowable

time,
so that a subjective
image
qualityevaluation frequencies,
is essential toapproximation
model
using
quadratic

DCTthatfrequency
sensitivity
given by Equation (8), andbalances
total threshold
visual distortion.

determine
the appropriatebalancing
betweenin
them.
and since
the various
parameters
the model areit can beAssume
considered thatas a JND
of thevisualdistortion.coefficient
Furthermore, isthei-th
non-zero

The frequencyfrom
sensitivity
thresholds
for 8×8 DCT
coefficientsdeterminedsubjective
evaluation
experiments

let DS[i]be the perceptual
distancecausedby embedding
thescanning
higherfrequency
quantized
wat[i]throughdescribed
are ubased
an approximation
using
in t he above
literatre , tonhey
ca n be regmodela rdedascodesof length
CL[i] inandthe theLSBs
of 8×8 quantized
DCT
coefficients,
enhancement
codeDCT
lengthquadraticfrequencies,
and since
variousparametersin theapproximations
ratherthantheexact
values.
Therefore,

coefficients.
Since thetorun-length
Huffmancoefficientcoding of higher
correspondingthat non-zero
is CL[i].

model
are determinedfrom subjective
evaluationfor
experimentsinwe extend
the sensitivity
thresholds
8×8 DCTfrequency
DCT coefficientscodes
every time by
Moreover,supposedeterminesthat theentropy
visualdistortion

theliterature, they
canthosebe regarded
as approximations
rather thancoefficients
intofor 16×16
DCT coefficientsbya non-zero
AC coefficient
appears, coefficient
if we were to is
encodeS[i]discarding
the non-zero
S[i]/zHere,
xy. andexact
values. Therefore,
extendthe sensitivity
thresholds for
extrapolatingthem wewithleast-squareapproximation.

embedcorresponding
codes intosensitivity
lower frequency
quantizedz xy theis the
DCT frequency
thresholdgiven

8×8 DCT
coefficients
into frequency
those for 16×16
DCT coefficientsbyAssumethat thesensitivity
threshold

DCTbycoefficients,
thenandthisit canprocedurecreates additional
Equation (8),be considered
as a JND

forwithintegercoordinates(x, y), 0 ≤ x, y ≤z xy is giventhem
extrapolating
least-squareapproximation.

perceptual
Dwat[i]-DwatFurthermore,[i-1], where Dwatlet[i-1]Disof thedistancevisual ofdistortion.
[i] bewatthethattwo-dimensional
the frequency sensitivity
threshold
zxy is given forto
7,Assumeon theplane
corresponding

the perceptual
distancecaused byfor
embedding
perceptual
distance causedby the bit-embedding
one previous thethe frequency
8×8
DCT.
We further
integercoordinates components(x, y), 0 ≤ x, yof≤ 7,on thetwo-dimensional

non-zeroS[i-1].
hand,
S[i] quantized
are leftcodescoefficientof length
CL[i]Oninthe
theotherLSBs
of if8×8

assumethat the tothresholdz xy isapproximated
using
plane
corresponding
the frequency
components
of 8×8 DCT.

DCTandcoefficients.
Since
the run-length
Huffmanuncodeddiscarded, thenthe lower
frequency distortionWefurther assume
thatdistancethe threshold
zxy theis approximatedusing2-Daa functionof thefromorigin of thecodingof higher
frequency
remains
unchanged,
but instead
additionalDCT
highercoefficientsfrequency

plane, ofwhen
they arefunction
the distancefromexpressed
the origin ofin
thethe2-D logarithmic
plane, when

visualdistortion ofentropy
S[i]/zxy is codes
generated.
determines
every time a non-zero AC

form:
they
are expressed in the logarithmic form:

log z xy a  b log x 2  y 2  c log x 2  y 2

coefficient
if wecode
were
to we
encodeS[i]Fordeterminingappears,
the enhancement
length,considerthatand ,
2embed the corresponding
codesifinto
lowerdistortion
frequency
the bit-embedding
is allowed to continue,
the visual

(8)
(8)

quantized
DCTenhancement
coefficients,
this thanprocedurecausedby embedding
codesthenis smaller
that

where
the naturalb, andwhere
log logis theisnatural
logarithm,logarithm,
and a, b, andandc area,arbitrary

creates
additional
perceptual
distanceTherefore,of D wat [i]causedby discarding
those non-zero
AC coefficients.

c are arbitrary
constants.
These constants
canthebeconstants.
These constants
can be determinedby minimizing

D wat[i-1], where D wat[i-1] is the perceptual distanceStart

determinedby minimizingthesumof squaredsumof squared differences
between theactuallogarithmof zxycaused by the bit-embedding for one previous non-the approximated(8), andwe
0.16924,b = -0.52246,value
and c in= Equation
0.744195. Notethat theare left uncoded and discarded, then the lower

differences
betweenactual logarithmz xy aandandthe approximatedvaluethein Equation(8), and we ofobtain=zero coefficient S[i-1].i On← 0the other hand, if S[i]obtain a = procedure
0.16924, bis =c = 0.744195.
approximation
not-0.52246,
applied to andz00, because
it is afrequency distortion remains unchanged, but instead

Note thatapproximation
procedure
not theappliedspecialvalue thefor DC
component andis excludedisfrom
bit-

EncodeS[i] and determine
CL[i],
additional higher
frequency
visual distortionof S[i]/z xy

and is excluded
from theof bit-embedding.
Thus,
wethresholds
for DCT coefficients
16×16 size for integer
valuesFor determining the enhancement code length,

Compute Dhigh = S[i]/zxy

to z 00, because
is acanspecialfor DC component
embedding.
Thus, itweobtainvalue
the frequency
sensitivity

is generated.

obtainfrequency
sensitivity
thresholds
for
ofcan
0 ≤ x,y ≤ 15. theThe rms
error between
8×8 frequency
sensitivity

that PL＜ΣCL[i]?
the bit-embeddingYesis allowed
i we← i +consider
1

0 ≤ x, y ≤ 15. The rms error between 8×8 frequency

embedding enhancement codes is smaller thanthresholdmatrix[5] and the result of
5.sensitivity
BALANCING
OF VISUALDISTORTION

that causedby discarding
those
non-zero AC
Embed CL[i]bits to lower DCT
coefficients,

Now that we have an approximation of the DCT frequency

visual distortions from lower and higher frequencies

DCT coefficients
16×16
size
for integer
values ofthresholdmatrix [5] andofthe
result of
Equation(8) is 1.27.

to continue, if the visual distortion caused by
No

Dlow =theDwat[i]– Dwat[i1]condition forcoefficients.Compute
Therefore,balancingEquation(8) is 1.27.CODE CREATION
5.1
ENHANCEMENT

is that ifYes
the enhancementcodesof length CL[i]Dlow < VC·Dhigh?

sensitivity
threshold forOF
16×16
size, weDISTORTION
next consider the
5. BALANCINGVISUALprocedure for balancing the visual distortion caused by thecan be embedded within the payload PL, thenNowthethat
we distortion
have an caused
approximationof theDCT
comparevisualby discarding
a certain

Code termination
bit-embedding continues
as long as the followingNo

5. 1 ENHANCEMENT
steganographic
image codingCODE
scheme.CREATIONIn other words, wethe procedure of enhancement code creation andfrequency
sensitivity
threshold
16×16
size, wehigherfrequency
DCT coefficientwith thatfor
causedby embedding

Fig. 8 Visualquality balancing (VQB) algorithm.
inequality
is satisfied,8― 43 ―

the balancing condition for visual distortions from lower andresolution image tends to decrease. The results in the figure show

higher frequencies is that if the enhancement codes of lengththat the tradeoff effect saturates at about VC = 5.

CL[i] can be embedded within the payload PL, then the procedure

Figure 11 shows the average enhancement code length per

Isao FURUKAWA
Junji SUZUKI
against VC values. Although the smallest enhancement codeof enhancement code creation and bit-embedding
continues as and block

long as the following inequality is satisfied,Dwat [i]  Dwat [i  1]  VC  S[i] z xy , length is 4 bits (i.e., EOB code) in a block if the bit-embedding

code length is significantly reduced, and the visual(9) (9)occurs, there are cases where the average is smaller than 4

distance in the lower frequency is also reduced

where VC is a constant to adjust the impacts on visual quality

because the blocks with no enhancement codes are also counted

where VC is a constant to adjust the impacts on

accordingly. Although omitted here, the visualvisual quality when both blockiness and blurring

quality balancing has a small impact on the results

exist. The flowchart of this visual quality balancing

for QF1 = 70, indicating that the enhancement code

(VQB) algorithm is shown in Figure 8. If thelength constraint incorporated to balance the visual

distortion associated with the bit-embedding in lower

distortion has a greater effect in proportion to thefrequency DCT coefficients is greater than thatsize of the payload.

when both blockiness and blurring exist. The flowchart of thisas the number of DCT blocks. It can be seen that the enhancement
code length increases steeply in the range of VC from 1 to aboutvisual quality balancing (VQB) algorithm is shown in Figure 8. If

4, and tends to saturate at about VC = 5. This confirms the SNR

the distortion associated with the bit-embedding in lower
frequency DCT coefficients is greater than that with discarding

results in Figure 10.

higher frequency DCT coefficients (i.e., Dwat[i]-Dwat[i-1] >The saturated SNR values in Figure 10 are shown in Table 3

VC·S[i]/zxy), or if the amount of enhancement codes exceeds the(indicated by VQB) in comparison with those obtained by thewith discarding higher frequency DCT coefficients

Next, the SNR values of enhanced high-resolution

payload (i.e., PL < Σ CL[i]), then this process is ended, and(i.e., Dwat[i]-Dwat[i-1] > VC·S[i]/z xy), or if the amount ofand JPEG-decoded low-resolution images against VC

enhancement codes exceeds the payload (i.e., PL <values are shown in Figure 10 in order to examine

Σ CL[i]), then this process is ended, and followed by

the160interaction effects when blockiness and blurringfollowed by the code termination process [3].

5.2 PERFORMANCE EVALUATION FOR 4K IMAGES

the code termination process [3].

are mixed. As V C becomes large, more high140

First, we obtain a 3-D histogram for the occurrence frequency of

frequency coefficients are retained because more120

the combination of enhancement code length and perceptual

5. 2 P ERFORMANCE EVALUATION FOR 4K

visual distortion in higher frequency is taken into

distance for DCT blocks similar to Figure 7 by applying the VQB

100

IMAGESaccount. This leads to the resolution enhancement

algorithm described above. The histogram for blocks containing

First , we obtain a 3 -D histog ram for the80
effect,but on the other hand, the blockinessoccurrence frequency of the combination of60
increases
as a result of the distortion tradeoff.
8

only EOB code is omitted here because it is almost the same as
the results in Figure 6. The reason is that the balancing algorithm

enhancement code length and perceptual distance40
In otherwords, as VC increases, the SNR ofthe6

is not applied to blocks containing only EOB code, and also thefor DCT blocks similar to Figure 7 by applying theenhancedhigh-resolution image tends to improve,20
4

number of such blocks is almost the same before and after the

VQB algorithm described above. The histogram

while that of the JPEG-decoded low-resolution image
2

0

code termination process. Figure 9 shows the 3-D histogram forfor blocks containing only EOB code is omitted

50
tends to decrease.
results in the figure show
100 The
150
200

DCT blocks whose enhancement code is not a single EOB for

here because it is almost the same as the results inthat the tradeoff effect saturates at 250
about VC = 5.
(a) D (S , S )image u01_Books, and the contour plot of the frequency diagram.

Figure 6. The reason is that the balancing algorithm

wat qeFigure 11 shows theaverageenhancement code

is not applied to blocks containing only EOB code,

length per block against VC values. Although theand also the number of such blocks is almost thesmallest enhancement code length is 4 bits (i.e.,same before and after the code termination process.

160 code) in a block if the bit-embedding occurs,
EOB

Comparing this result with that of Figure 7, we can see that theenhancement code length is significantly reduced, and the visual
distance in the lower frequency is also reduced accordingly.

Although omitted here, the visual quality balancing has a small
Figure9 shows the 3-D histogram for DCT blocks

140 are cases where the average is smaller than 4
there

impact on the results for QF1 = 70, indicating that thewhose enhancement code is not a single EOB

becausethe blocks with no enhancement codes are120

for image u01_Books, and the contour plot of thealso
100counted as the number of DCT blocks. It canenhancement code length constraint incorporated to balance thevisual distortion
has a greater
effect in proportion
to thesize offrequency
diagram.
Comparing
this result
withbe seen that the enhancement code length increases
80

theof
payload.
that
Figure 7, we can see that the enhancement

steeply in the range of VC from 1 to about 4, and60

Next, the SNR values of enhanced high-resolution and JPEG-

8
tends to saturate at about VC = 5. This confirms the
40

Start
decoded low-resolution images
against VC values are shown in

SNR results in Figure 10.

Figure 10 in order to examinei ← 0 the interaction effects when

20
The

blockiness and blurring are mixed. As VC becomes large, more

6

4 u re 10
sat u rated S N R va lue s i n Fig

a re s0 how n i n Table 3 ( i nd ic at ed2 by VQB )50

Compute Dhigh = S[i]/zxyi n compa r ison100w it 150h t ho200
se obt a i ned by t he

the resolution
enhancement
effect, but onYes
the other hand, thei←i+1
PL＜ΣCL[i]?

(indicated by Conv.). For enhanced high-resolution

Encode S[i] andCL[i],because more visualhigh-frequency coefficients
aredetermine
retained250

distortion in higher frequency is taken into account. This leads toconventional steganographic compression system(b) Dwat(Sb, Se)

blockiness increases as a result
No of the distortion tradeoff. In otherFig. 9 Histogram
its contour ofby
perceptualforimages,the SNRanddecreases
about distance
1 dB whenwords, as VC increases,
the SNR of the enhanced high-resolution
Embed CL[i] bits to lower DCT coefficients,

u01_Books
VQBisalgorithm(QF1,
QF2)70),= (95,
thequalityusing
factor(QF1, with
QF2)
= (95,
but it70). (Except DCT blocks having only EOB code)

Compute Dlow = Dwat[i] – Dwat[i−1]image tends to improve, while that of the JPEG-decoded lowYes

Dlow < VC·Dhigh?
No
Code termination

Fig. 8 Visual quality balancing (VQB) algorithm.

9

scarcely changes when (QF1, QF2) = (70, 70). Thisis due to the fact that the relationship between
the enhancement code length and the perceptual
distance almost unchanged by the VQB algorithm
in the latter case. On the other hand, for JPEG-― 44 ―

Visual Quality Balancing between Enhanced High-resolution and Ordinary JPEG-decoded Images in Steganographic Image Compression System

160
140

SNR (dB)

120
100
80
60

8

40

6

20
0

4
50

100

QF1=

2
150

200

250

q

u01_Books
u04_Kimono

VC( ) (a)
D D(S (SS, )S )
wat

95
70
95
70

(a) Enhanced high-resolution image

e

160
140
120

SNR (dB)

100
80
60

8

40

6

20
0

4
50

100

QF1=

2
150

200

250

95
70
95
70

u01_Books
u04_Kimono

VC(b) Dwat(Sb, Se)(b) JPEG-decoded low-resolution image

Fig. 9 Histogram and its contour of perceptual distance for
u01_Books using VQB algorithm with (QF1, QF2) = (95, 70).
(Except DCT blocks having only EOB code)

Fig. 10 SNR values of enhanced high-resolution image andJPEG-decoded low-resolution image.

decoded low-resolution images, the SNR is greatly
improved when (QF 1, QF 2) = (95, 70). In the
proposed steganographic compression system,by the VQB algorithm shown in Figure 8, theobjective is to improve the quality of the JPEGdecoded low-resolution image while minimizing
the quality degradation of the enhanced highresolution image. The results in Table 3 show thatthe goal depicted in Figure 2 has been achieved:

Average code length (bits)

which determines the enhancement code length

great improvement is obtained for SNR values of
JPEG-decoded low-resolution images in exchange

QF1=

for slight degradation in enhanced high-resolution
images.

95
70
95
70

u01_Books
u04_Kimono

VC

Fig. 11― 45 ―

Average enhancement code length per block.

Isao FURUKAWA and Junji SUZUKI
Table 3

Recovered image
Coding method
Quality (95, 70)
factor(70, 70)

Comparison of SNR (dB).

enhancement codes in relatively complex texture(a) u01_Books

regions of images, which in turn reduces the

Enhanced
Conv.
VQB
29.24
28.26
26.97
27.19

JPEG-decoded
Conv.
VQB
37.76
44.86
32.60
34.37

perceptual distance in JPEG-decoded low-resolution
images.

6. CONCLUTION
In the steganographic image compression system(b) u04_Kimono

Recovered image
Coding method
Quality (95, 70)
factor(70, 70)

Enhanced
Conv.
VQB
31.05
30.05
28.67
28.95

JPEG-decoded
Conv.
VQB
38.44
47.92
33.82
35.65

with information embedding, there is a tradeoff
between the lower and higher frequency visual
distortions, and an algorithm for determining theallowable enhancement code length to balance the

Figure 12 demonstrates some examples ofvisual distortions was proposed, and simulation

enhanced high-resolution images and JPEG-decoded

experiments were made on actual standard 4K

low-resolution images for u01_Books with both

images. As a result, compared with the conventional

the conventional and the proposed VQB methods.

method of embedding as much enhancement codes

In the simulation experiment, the parameters foras possible within the payload, which intended tocompressing images are (QF1, QF2) = (95, 70), nLSB

improve the quality of only enhanced high-resolution= 6, and Vc = 10, so that the visual quality balancing

image, it was confirmed that the proposed visualalgorithm is working in the saturation region. Notequality balancing (VQB) algorithm can improvethat the upper left image in the figure is identicalthe fidelity of JPEG-decoded low-resolution image

to the image (a) in Figure 5. It can be seen thatin exchange for a slight decrease in the fidelity ofthe blockiness is reduced in both resolution images

enhanced high-resolution image.

when the compression system brings the VQB

The perceptual distances for 16 ×16 higher

algorithm into action. In addition, the enhanced high-frequency DCT coefficients used here are obtained

resolution images with VQB are slightly low-pass

by extrapolating from those obtained by Watson’s

filtered compared to those without VQB, and themodel for 8×8 lower frequency DCT coefficients,

JPEG-decoded low-resolution images achieve a moreso they are only based on approximate calculations.

reasonable level of quality than the conventional

In particular, when blockiness occurs at lower

method. In particular, in the low-resolution image,frequencies in conjunction with blurring at highera large blockiness occurs in the shadow under thefrequencies, the optimization of mutual balancing isbookshelf slightly above the center of the cropped

challenging.
In this study, we adopted an algorithm thatimage for the conventional method, but the distortion

multiplies the higher frequency visual distortion

is no longer visible in the proposed method.
Figure 13 shows the results of the enhancement

by a certain constant and compares the distortion

code length per block and the lower frequency

with the lower frequency perceptual distance, andperceptual distance for the entire image u01_Books

continues the enhancement code embedding as longby comparing those for the conventional method

as the perceptual distance caused by bit-embedding(without VQB) and the proposed method (with VQB).

is smaller than the visual distortion caused by

The simulation parameters (QF1, QF2), nLSB, and Vc

discarding the corresponding AC coefficients.

are the same as those in Figure 12. Each value in

The optimal constant value used in the inequality

the figure is shown as a luminance with a maximumthat determines whether the run-length Huffman

value of 255, and the perceptual distance is 50

encoding and the associated bit-embedding should betimes the original value. Therefore, the brighter

continued varies from image to image, and may evenarea corresponds to larger values. From the figure,vary from region to region within a single image, so

we can see that the proposed image compressionthat the results obtained here should be considered

method with VQB algorithm restricts the length ofas a guideline for practical constructions of the― 46 ―

Visual Quality Balancing between Enhanced High-resolution and Ordinary JPEG-decoded Images in Steganographic Image Compression System

Without VQB(a) Enhanced high-resolution images

Without VQBWith VQBWith VQB(b) JPEG-decoded low-resolution images

Fig. 12Comparison of recovered images for u01_Books with and without visual quality balancing (VQB).

Without VQB(a) Enhancement code length per block

Without VQBWith VQBWith VQB(b) Lower frequency perceptual distance per block

Fig. 13 Comparison of enhancement code length and lower frequency perceptual distance for u01_Books with and withoutvisual quality balancing (VQB).
― 47 ―

Isao FURUKAWA and Junji SUZUKI

algorithm. However, it is generally confirmed that a[3] I . Furukawa and J. Suzuki, “JPEG-based image

slight decrease in SNR of enhanced high-resolution

enhancement method with steganographic data

images results in a large increase in SNR of JPEG-

embedding and its performance evaluation,” Bull.

decoded low-resolution images. It will provide a rule-

Hiroshima Inst. Tech. Research, Vol. 49, pp.97-108,of-thumb technique for visual quality balancing in2015.

the conventional steganographic image compression[4] I. Furukawa and J. Suzuki, “Steganographic image

system.

compression method with adaptive enhancement

Our future topics are to evaluate subjectivecode embedding and its performance evaluation

image quality in detail and to develop an optimumon 4K standard images,” Bull. Hiroshima Inst.

control method for the amount of enhancement

Tech. Research, Vol. 55, pp. 63-76, Feb. 2021.

codes by balancing lower and higher frequency[5] J. Cox, M. L. Miller, J. A. Bloom, J. Fridrich, and T.

visual distortions with the use of diverse images andKalker, Digital Watermarking and Steganography,

simulation parameters.

2 nd Ed . , pp. 2 6 9 -2 7 3 , Morga n K au f ma n n
Publishers, 2008.

REFERENCES[6] J. Ahumada and H. A. Peterson, “Luminance-[1] D. S. Taubman and M. W. Marcellin, JPEG2000

model-based DCT quantization for color image

Image Compression Fundamentals, Standards

compression,” Proceedings of the SPIE, 1666,

and Practice, Springer, 2002.

pp.365-374, 1992.

[2] I . Furukawa and J. Suzuki, “A novel approach of[7] I. Matsuda, K. Masaoka, and H. Ikegawa, “Ultra-

image enhancement using steganographic datahigh definition/wide-color-gamut standard test

embedding and JPEG scheme,” Bull. Hiroshima

images,” Journal of ITE, Vol. 68, No. 8, pp.643-647,

Inst. Tech. Research, Vol.48, pp.93-103, Feb. 2014.

2014.

― 48 ―