FLEXIBLY-TUNABLE BITCUBE-BASED PERCEPTUAL ENCRYPTION WITHIN
JPEG COMPRESSION
Kosuke Shimizu† and Taizo Suzuki‡†‡

Department of Computer Science, University of Tsukuba, Japan
Faculty of Engineering, Information and Systems, University of Tsukuba, Japan
Email: † kshimizu@wmp.cs.tsukuba.ac.jp, ‡ taizo@cs.tsukuba.ac.jp
ABSTRACT

We propose a perceptual encryption within JPEG compression(EWJ). Although some of the conventional EWJs have the ‘tunability,’ which is a property of how many perceptual degradationlevels can be provided with the single encryption technique, it isinsufficient because either strong level of security or low bitrate
overhead is regarded as important. The proposed EWJ detects the
specific subspaces of the bits in the quantized discrete cosine transform (QDCT) coefficient blocks (‘bitcubes’), such that the non-zero
bits crowd, and then the bits are permuted within each bitcube,
i.e., not only within each bitplane like the conventional EWJs but
also between adjacent bitplanes. The experiments show that the
bitcube-based EWJ actually provides more flexible tunability thanthe conventional EWJs, while compromising the relation between
the bitrate overhead and the attack robustness.
Index Terms— bitplane, flexible tunability, JPEG, perceptual
encryption, QDCT domain.
1. INTRODUCTION
Many perceptual encryptions (PEs) that visually protect multimedia
contents have been presented [1–8]. Since the PEs can preview thecontents in the visually protected state, they can be included in theapplications of the subscription-aware broadcasting services (SBSs),
e.g., video on demand (VOD) and stock photo, and of the social
networking services (SNSs), e.g., Twitter and Facebook. The PEs for
those applications should be able to tune the perceptual degradationlevels to achieve the various security demands by the providers andcustomers. This study focuses on the tunable encryption. Hereafter,let ‘tunability’ be a property of how many perceptual degradationlevels can be provided with the single encryption technique.
Most of the PEs are used in conjunction with the international
image and video compression standards. Especially, since JPEG [9]is the most popular image compression standard and also inheritedas the base layer part of the emerging alternative standard, JPEG
XT [10], many PEs for JPEG were proposed [1–8]. The PEs for
JPEG are roughly classified in terms of three types of techniques before, after, and within the compression (EBJ, EAJ, and EWJ). The
PEs before the JPEG compression (EBJs) [1, 2] are friendly with
JPEG, but cannot tune the perceptual degradation levels in each of
their encryption modules, i.e., no tunability. As same as the case of
EBJs, the PEs after the JPEG compression (EAJs) [3, 4] are alsofriendly with JPEG, but has no tunability. On the other hand, somePEs within the JPEG compression (EWJs) achieved the tunability.
For example, the EWJ by Li and Lo [5] embedded random signflips with the alternative transforms for achieving the low bitrate978-1-5090-6631-5/20/$31.00 ©2020 IEEE

overheads, but it has low tunability. Intra-bitplane shuffling (IBS)
proposed by Mao and Wu [6] has the potential to provide the EWJ
with flexible tunability, but it significantly increases the bitrate overheads for achieving higher security. In addition, the EWJs by Li and
Yuan [7] and by Khan et al. [8] provide a certain level of tunabilityof the perceptual degradation levels in the encrypted-decoded images thanks to the encryption of several low AC coefficients in eachquantized discrete cosine transform (QDCT) coefficient block. However, all of [6–8] are vulnerable against the replacement attack [5],which overwrites the encrypted portion as all zeros to unmask theun-encrypted portion when low compression, i.e., the tunability is
insufficient.
We propose a novel EWJ providing more flexible tunability thanthe conventional EWJs. The proposed EWJ detects the specific subspaces of the bits in the QDCT coefficient blocks (‘bitcubes’), suchthat the non-zero bits crowd, and then the bits are randomly permuted within each bitcube, i.e., not only within a bitplane like theconventional EWJs but also between adjacent bitplanes. The experiments show that the bitcube-based EWJ actually provides flexible
tunability than the conventional EWJs, while compromising the relation between the bitrate overhead and the attack robustness.
2. BUILDING BLOCKS AND PRIOR REVIEWS
2.1. JPEG Compression
JPEG is a lossy compression technique consisting of the following
five modules:
1. Change the image color space from RGB to YCbCr.
2. Downsample the chroma coefficients Cb and Cr.
3. Apply DCT to each of 8 × 8 blocks divided in the luma coefficients Y and the chroma coefficients Cb and Cr.
4. Quantize the DCTed coefficients.
5. Encode the coefficients using Huffman coding after differential coding of DC coefficients and run-length coding of AC
coefficients in the QDCT domain.
In the step 5, the QDCT coefficients are efficiently compressed because of the sparsity obtained by the steps 1-4.
2.2. Bitplane Decomposition(t)When cij and bij are an (i, j)th decimal QDCT coefficient and thetth (t ∈ N[1 D] , D1 ∈ N) bit of binary representation from the
1 It means the number of bits enough to represent the QDCT coefficient
range. In the JPEG reference software libjpeg-turbo [11], it becomes 10
when the quantization factor Q is less than 96.

2702

ICASSP 2020

Authorized licensed use limited to: IEEE Xplore. Downloaded on May 08,2020 at 10:19:47 UTC from IEEE Xplore. Restrictions apply.

tth bitplane

…

…
…

0…1
…

1

…

…

1…0
0…1

tth v × h × d
subspaceV×H
coeﬀicients
bitplanedecomposition

bitcuboid

Fig. 1. Bitcuboid and subspace decomposition.
noD(t) V,H
least significant bit (LSB), let {cij }V,H
i,j=1 and {bij }i,j=1

be

t=1

the V × H (V, H ∈ N) coefficients and the D bitplanes composedof a sign-bit as most significant bit (MSB) and D − 1 bits of abso(t)lute part, respectively, i.e., {bij }V,H
i,j=1 is the tth bitplane. Even ifnoD(t) V,H
the bitplanes {bij }i,j=1
are encrypted, the encrypted binary
t=1

representation e
bij can be re-decimalized to the encrypted decimal
QDCT coefficient e
cij asD−1e(D) Xecij = −1bij

t=1

(t)

2t−1e
bij .

(1)

As an exception, when the encrypted binary representation ebij
is formed as negative zeros, i.e., −0 · · · 0, we redefine the redecimalizaed value ase
cij = −2D−1 ,(2)because the negative zeros are re-decimalized to zero if without (2).

domly permuted among blocks and within each block: the AC coefficients in the same position are permuted among the blocks and the
AC coefficients in the different position are permuted within eachblock. It can suppress the bitrate overheads like one by Li and Yuan,
but is also vulnerable against the replacement attack, because theencrypted coefficients are only up to six in 64 coefficients.
From the above, the compromises between the security and thebitrate overheads should be considered. The bitrate overhead problem of the conventional EWJ by Mao and Wu is because of significantly different number of the non-zero coefficients after encryption
compared with the original one. Also, the security problem of theconventional EWJs by Li and Yuan and by Khan et al. is because ofthat the number of non-zero bits to be encrypted per each block is
not enough.
3. BITCUBE-BASED EWJ
We propose a novel EWJ, which encrypts the non-zero bits without changing the number of the zero bits, i.e., the sparsity, in eachblock much from the original ones, by extending the IBS by Mao
and Wu [6].

2.3. Conventional Tunable EWJs
This subsection introduces three conventional EWJs [6–8] with thetunability based on QDCT domain.
We first introduce the EWJ derived from applying the IBS by
Mao and Wu [6] to each 8 × 8 QDCT coefficient block in this study.
The IBS decomposes the several (ℓe ∈ Z[1 64] ) QDCT coefficients
detected in the zig-zag order in each QDCT coefficient block into the
bitplanes and randomly permutes the bits within each bitplane. The
resulting EWJ has the potential for providing flexible tunability byincreasing ℓe , setting different conditions in each bitplane, and/or soon. However, it may be vulnerable against the replacement attack,which overwrites the encrypted portion as all zeros to unmask the unencrypted portion when low compression, because fewer coefficientsshould be encrypted for suppression of the bitrate overheads.
We second introduce the EWJ by Li and Yuan [7]. If the tunability is required, several (ℓe ) QDCT coefficients are detected in thezig-zag order in each QDCT coefficient block and the coefficients inthe same positions among the blocks are randomly permuted. It can
suppress the bitrate overheads of the encrypted-encoded bitstreams
efficiently unlike one by Mao and Wu, even when using more QDCT
coefficients, but may be also vulnerable against the replacement attack.
We third introduce the EWJ by Khan et al. [8]. It decomposes
all DC coefficients into the bitplanes and randomly permutes the bits
within each of the several (ℓDC ∈ N[1 7] ) bitplanes from the LSB
plane. In addition, the several (ℓAC ∈ N[1 5] ) AC coefficients detected in the zig-zag order in each QDCT coefficient block are ran-

3.1. Definitions of Bitcuboid and Bitcube
This subsection introduces a more general representation of bitplane and its subspace representation. The cuboid containing
bits (‘bitcuboid’) is constructed by bundling the bitplanes from
the MSB plane to the LSB plane as shown in Fig. 1. The set of bitnoD(t)planes {bij }V,H
are redefined as the V × H × D bitcuboidi,j=1{βijk }V,H,D
i,j,k=1 as

t=1

oD
n
(t) V,H{βijk }V,H,D.
i,j,k=1 = {bij }i,j=1

(3)

t=1

Constructing the bitcuboid, the encryptor can move the bits in the
bitcuboid not only within a bitplane but also between adjacent bitplanes.
However, the encryptor should move the bits in the smaller subspaces of the bitcuboid because the encrypted non-zero coefficients
cause the bitrate overheads when the sparsity is lost. Therefore, thebitcuboid is divided to several v × h × d (≥ 2; v ∈ N[1 V ] , h ∈N[1 H] , d ∈ N[1 D] ) subspaces. When the {βijk }V,H,D
i,j,k=1 is divided
to the L subspaces not overlapping mutually, the subspaces are denoted asoL
nv,h,d (t)= {βijk }V,H,D(4){βijk }i,j,k=1
i,j,k=1 ,t=1v,h,d (t)where L = ⌊V /v⌋ × ⌊H/h⌋ × ⌊D/d⌋. The {βijk }i,j,k=1 is thetth subspace of the {βijk }V,H,D
i,j,k=1 and contains the v × h × d bits

2703

Authorized licensed use limited to: IEEE Xplore. Downloaded on May 08,2020 at 10:19:47 UTC from IEEE Xplore. Restrictions apply.

Fig. 2. Comparison of bitrate overheads induced with the various
m-cubes.

Fig. 3. Comparison of bitrate overhead suppressions with the EWJs.
that pass through the d bit sequences of the corresponding v × hnoD(t)coefficients in the {bij }V,H
. For the JPEG application,
i,j=1
t=1

a bitcuboid {βijk }8,8,D
is constructed from the set of bitplanesnoD i,j,k=1(t) 8,8{bij }i,j=1
except for the MSB plane (sign-bit plane) andt=1
LSB plane, because each 8 × 8 QDCT block has 8 × 8 coefficientsin D-bit depth. Also, when v = h = d, the subspace is termed as a‘bitcube.’
3.2. Encryption Algorithm
There are several possible ways to encrypt the subspaces, but thisstudy introduces a scrambling approach randomly permuting the bits
within each bitcube. The encrypted subspace {eai } P
i=1 (P = v ×h × d) is obtained by randomly permuting the bits in the original
subspace {ai }P
i=1 asP{eai } Pi=1 = {⊙ (ai )}i=1 ,⊙ : x → y (x ̸= y; x, y ∈ {ai }P
i=1 )

(5)

with the random numbers calculated from the pseudo random number generator initialized with the seeds extracted from the encryption
key long enough. The number of the non-zero bits (ones) of the subspace does not change even if permuted. Moreover, by selecting the

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Fig. 4. Comparison of the perceptual degradations produced by thevarious types of m-cubes (JPEG Q = 70): (a) all 7-cubes (ℓ7 =100), (b) all 6-cubes (ℓ6 = 100), (c) all 5-cubes (ℓ5 = 100), (d)all 4-cubes (ℓ4 = 100), (e) all 3-cubes (ℓ3 = 100), (f) all 2-cubes(ℓ2 = 100), (g) all 1-cubes (ℓ1 = 100), and (h) all {1, 2, 3}-cubes(ℓ1 , ℓ2 , ℓ3 = 100).

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 5. Comparison of various perceptual degradation levels withthe various percentages for encrypting 1-cubes (JPEG Q = 70): (a)20% (ℓ1 = 20), (b) 40% (ℓ1 = 40), (c) 50% (ℓ1 = 50), (d)60 % (ℓ1 = 60), (e) 80% (ℓ1 = 80), and (f) 100% (ℓ1 = 100).

specific subspaces to be encrypted carefully, we can achieve the flexible tunability in each bitcuboid. The subspaces containing the very
few number of non-zero bits must be sparse to preserve the sparsityof the QDCT domain, i.e., the coding performance.
Hereafter, this study considers the subspace as the bitcube forsimplicity and let the bitcube whose number of ones is m (∈ Z[0 P ] )be ‘m-cube.’ We can take the advantages of the percentages that how
many the m-cubes are encrypted to achieve the flexible tunability.
The ratio of encrypting m-cubes for all bitcubes of bitcuboids (ℓm ∈Z[0 100] ) contributes to the tunability. When we select the T (≤ P )types of m-cubes efficient for the encryption, 100T ways are enable
to achieve the tunability of the encrypted-decoded images.
3.3. Security Analysis
For the security analysis, we first measure the number of the bruteforce attacks, under the condition that the attacker tries to reconstruct the all encrypted bitcubes in a bitcuboid (block) without understanding that which m-cube was encrypted and the encryption key.
The number of the brute-force attacks to reconstruct each bitcube
in a block is 2P and the number of the m-cubes contained in a
bitcuboid is l. Consequently, the number of the brute-force attacks

2704

Authorized licensed use limited to: IEEE Xplore. Downloaded on May 08,2020 at 10:19:47 UTC from IEEE Xplore. Restrictions apply.

(a)

(b)

(c)

(d)

(e)

Fig. 6. Attacked-decoded images after the replacement attacks (JPEG Q = 70): (left-to-right) original one, encrypted one by Khan et al.
(ℓDC = 7, ℓAC = 5), attacked encrypted one by Khan et al., encrypted one by ours (ℓ1 , ℓ2 , ℓ3 = 100), and attacked encrypted one by ours.

2. Measure the bitrates of the encrypted and not encrypted bitstreams.
3. Decode them.
4. Measure the SSIMs [14] of the decoded images.
5. Iterate from 1. to 4. for all test images and calculate the mean
results.

Fig. 7. Comparison of mean SSIMs between the encrypted-decode
images and the original images.

to all bitcubes in a block isl(2P ) = 2P l .

(6)

For example, when the number in the case of 2 × 2 × 2 bitcubeslis (28 ) = 28l and l is greater than 32, the number of the bruteforce attacks against one bitcuboid is greater than the number of thebrute-force attacks against the SHA-256 hash digest [12, 13], whichis regarded as a security criterion. Even if l is less than or equal to32, encrypting the multiple blocks can reinforce the robustness ofthe encrypted region.
We second discuss the robustness against the replacement attack [5]. When the bitcubes of the 1-cubes to the (P − 1)-cubes({1, · · · , P − 1}-cubes) are encrypted, the remain un-encrypted
bitcubes in a bitcuboid are 0-cubes and P -cubes. If all of the{1, · · · , P − 1}-cubes are overwritten to the 0-cubes and the P cubes are hardly existing, the bits representing the original featuresof the block are also hardly existing. Therefore, the replacement
attacks cannot sufficiently recover the original features of each encrypted block and we can consider that the bitcube-based EWJ isrobust against the replacement attack.
4. EXPERIMENTS
We compared the effectiveness of our bitcube-based EWJ with theconventional EWJs as following procedures:

We set the bitcube size to v = h = d = 2 and used the 100 full-color
images of the Un-Compressed Image Dataset (UCID) database [15]as the test images. One can see that the bitcubes with ℓ1 = 100
caused the most bitrate overheads compared with the other bitcubeswith ℓ2 = ℓ3 = · · · = ℓ7 = 100 as shown in Fig. 2. Also, although the bitcube-based EWJ could not suppress more than ones
by Li and Yuan and by Khan et al., it could suppress more of thebitrate overheads than one by Mao and Wu [6] as shown in Fig. 3.
We also compared the perceptual degradation levels of ourencrypted-decoded images at the various m-cubes. The bitcubeswith ℓ1 = 100 induced the strongest perceptual degradation into
the encrypted-decoded image and the combination of the bitcubeswith ℓ1 , ℓ2 , ℓ3 = 100 as shown in Fig. 4 (h) induced much stronger
degradation than the other patterns as shown in Fig. 4 (a-g). Besides,
according to the increase of the percentage (the value of ℓm ), thebitcube-based EWJ could flexibly tune the perceptual degradationlevels as shown in Fig. 5.
The images, whose all 1-cubes, all 2-cubes, and all 3-cubes wereencrypted by the bitcube-based EWJ, are obviously robust against
the replacement attack, because the encrypted image by Khan et al.
was recovered to the rough textures that can be previewed but one
by ours was not recovered as shown in Fig. 6. We also confirmed theobjective robustness with the mean SSIMs between the encrypteddecoded images and the original images as shown in Fig. 7. It it clearthat the attacked encrypted-decoded images by ours were not recovered the visual qualities with the replacement attack unlike three conventional EWJs.
5. CONCLUSION
We proposed a novel EWJ providing flexible tunability. The proposed bitcube-based EWJ was placed into the JPEG QDCT domain
and randomly permuted the bits of the bitcubes in each of the QDCT
coefficient blocks. The experiments showed that the bitcube-based
EWJ actually provided more flexible tunability than the conventional
EWJs, while compromising the relation between the bitrate overhead
and the security.

1. Encode the test image with the libjpeg-turbo [11] while encrypting the QDCT coefficients using the EWJ (JPEG quality
factor Q = 10, 20, · · · , 90)2705

Authorized licensed use limited to: IEEE Xplore. Downloaded on May 08,2020 at 10:19:47 UTC from IEEE Xplore. Restrictions apply.

6. REFERENCES[1] K. Kurihara, M. Kikuchi, S. Imaizumi, S. Shiota, and H. Kiya,“An encryption-then-compression system for JPEG/Motion
JPEG standard,” IEICE Trans. Fundamentals, vol. E98-A, no.
11, pp. 2238–2245, Nov. 2015.
[2] T. Chuman, W. Sirichotedumrong, and H. Kiya, “Encryption then-compression systems using grayscale-based image
encryption for JPEG images,” IEEE Trans. Inf. Forensics Security, vol. 14, no. 6, pp. 1515–1525, Nov. 2019.
[3] V. Itier, P. Puteaux, and W. Puech, “Recompression of JPEG
crypto-compressed images without a key,” IEEE Trans. Circuits Syst. Video Technol., Jan. 2019, Early Access.
[4] J. Ting, K. Wong, and S. Ong, “Format-compliant perceptual
encryption method for JPEG XT,” in Proc. of ICIP’19, Taipei,
Taiwan, Sept. 2018, pp. 4559–4563.
[5] P. Li and K.-T. Lo, “Joint image compression and encryption
based on order-8 alternating transforms,” J. Vis. Commun. Image, vol. R, no. 44, pp. 61–71, Apr. 2017.
[6] Y. Mao and M. Wu, “A joint signal processing and cryptographic approach to multimedia encryption,” IEEE Trans. Image Process., vol. 15, no. 7, pp. 2061–2075, July 2006.
[7] W. Li and Y. Yuan, “A leak and its remedy in JPEG image
encryption,” Int. J. Comput. Math, vol. 84, no. 9, pp. 1367–
1378, Sept. 2007.
[8] M. I. Khan, V. Jeoti, and M. A. Khan, “Perceptual encryptionof JPEG compressed images using DCT coefficients and splitting of DC coefficients into bitplanes,” in Proc. of ICIAS’10,
Kuala Lumpur, Malaysia, June 2010, pp. 1–6.
[9] “Overview of JPEG,” https://jpeg.org/jpeg.
[10] “Overview of JPEG XT,” https://jpeg.org/jpegxt.
[11] “JPEG software,” https://jpeg.org/jpeg/software.html.
[12] K. Dmitry, R. Christian, and S. Alexandra, “Bicliques forpreimages: Attacks on Skein-512 and the SHA-2 family,” inProc. FSE’19, Washington, DC, Mar. 2012, pp. 244–263.
[13] National Institute of Standards and Technology, FIPS PUB
180-4: Secure Hash Standard (SHS), pub-NIST, Aug. 2015.
[14] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–
612, Apr. 2004.
[15] G. Schaefer and M. Stich, “UCID: An uncompressed color
image database,” in Proc. SPIE 5307, Storage and Retrieval
Methods and Applications for Multimedia, San Jose, CA, Jan.
2004, pp. 472–480.

2706

Authorized licensed use limited to: IEEE Xplore. Downloaded on May 08,2020 at 10:19:47 UTC from IEEE Xplore. Restrictions apply.