2021 IEEE 10th Global Conference on Consumer Electronics (GCCE)

Extended JPEG Decoders With Luma Restorationand Luma-Guided Chroma Restorations
Koushi Yamazaki and Taizo Suzuki
University of Tsukuba, Ibaraki, Japanyamazaki@wmp.cs.tsukuba.ac.jp, taizo@cs.tsukuba.ac.jp

Abstract—This study presents two extended JPEG decoders
that restore the degraded images quickly and accurately. They
have a luma restoration and two luma-guided chroma restorations before/after chroma upsampling in addition to the standard
decoder. The experiments based on the typical classical methodssuch as the block-matching and 3D filtering (BM3D) and guided
filter (GF) show that the proposed methods restore the noiseadded JPEG-compressed images faster and better than theclassical one that does not consider the JPEG process.
Index Terms—image restoration, JPEG, luma, chroma

faster and better than the classical one that does not consider
the JPEG process.
II. P RELIMINARIES
A. JPEG Decoder
The JPEG decoder consists of the following steps:
1) Decimalize the bitstream into the quantized DCT coefficients.
2) Dequantize the quantized DCT coefficients.
3) Convert the coefficients in the spectral domain into one
luma Y and two chromas Cb and Cr in the spatial one.
4) Upsample the Cb and Cr downsampled on the encoderwith the default sampling factor [4 : 2 : 0] to the originalsize.
5) Convert the Y , Cb , and Cr into the colors R, G, and B.

I. I NTRODUCTION
With the development of multimedia devices and broadband
networks, image compression is one of the most necessary
technologies and it should be noted that the JPEG is still thede-facto image coding standard. However, when the JPEG
compresses an image at a low bitrate, various annoying
artifacts may appear on the textures and colors. As oneof the most known them, the blocking is appeared due tothe discontinuity generated with the DCT and quantizationin each 8 × 8/16 × 16 block. As the other one, the color
bleeding is caused because the decoder upsamples the chroma
downsampled on the encoder forcibly.
Although many images are commonly compressed with theJPEG as described above, some high-precision image restoration algorithms [1], [2] tend to observe only the resulting
images and do not consider the JPEG process sufficiently.
As a result, the conflicts between the conventional ones andthe JPEG process may other undesirable artifacts. Also, manyof algorithms that consider the JPEG process [3], [4] still
involve some redundant processes, i.e., the restoration after
whole decoding.
To reduce such artifacts and redundancies, this study
presents two extended JPEG decoders inspired by a joint
bilateral filter (JBF) for color debleeding in [5]. The proposed
methods have a luma restoration (LR) and two luma-guided
chroma restorations (CRs) before/after chroma upsampling in
addition to the standard decoder. Only adding new simple
algorithms directly to the existing decoder not only eliminates
many of the redundant processes, which are required in
the conventional methods, but also improves the restoration
performance. The experiments based on the typical classical algorithms such as the block-matching and 3D filtering(BM3D) [6] and guided filter (GF) [7] show that the proposed
methods restore the noise-added JPEG-compressed images

978-1-6654-3675-5/21/$31.00 ©2021 IEEE

Note that the JPEG is lossy compression due to the quantization and chroma subsampling.
B. Related Algorithms
We introduce three classical algorithms that are closely
related to this study.
The JBF is one of the most common edge-preserving
smooth filters with a guidance image. The kernel consists ofboth of spatial and range kernels. In [5] for color debleeding, the JBF guided with Y is applied to Cb and Cr after
the decimalization, dequantization, inverse DCT, and chroma
upsampling on the JPEG decoder.1
The BM3D [6] is one of the most common and highestprecision algorithms utilizing image self-similarity. It has threesteps: 3D transformation of each 3D data array grouped fromsimilar 2D image blocks, shrinkage of the transform spectrum,and inverse 3D transformation.
The GF [7] is one of the most common and fastest edgepreserving smooth filters with a guidance image like the JBF.
The kernel is derived from a local linear model.
III. JPEG D ECODERS W ITH LR AND L UMA -G UIDED CR S
This study presents two extended JPEG decoders inspiredby the JBF in [5].
1 Although the reference presented the JBF guided with not only Y but also
Cb and Cr , this study omits the trivial extension for simplicity.

757

2021 IEEE 10th Global Conference on Consumer Electronics (GCCE)

Fig. 1. Flows of the proposed methods (US, DS, and dashed arrow indicate the upsampling, downsampling, and guidance, respectively): (left) the Prop-I and(right) the Prop-II .

A. Extended JPEG Decoder Type-I (Prop-I)

TABLE IPARAMETERS OF THE BM3D AND GF.

Prop-I has an LR and two luma-guided CRs “after” thechroma upsampling in addition to the standard decoder (seethe left of Fig. 1). The Prop-I consists of the following steps:
1) Decimalize the bitstream into the quantized DCT coefficients.
2) Dequantize the quantized DCT coefficients.
3) Convert the coefficients in the spectral domain into one
luma Y and two chromas Cb and Cr in the spatial one.
4) [NEW] Restore the Y with an arbitrary LR method.
5) Upsample the Cb and Cr downsampled on the encoderwith the default sampling factor [4 : 2 : 0] to the originalsize.
6) [NEW] Restore the Cb and Cr with the GF guided bythe restored Y .
7) Convert the Y , Cb , and Cr into the colors R, G, and B.
Note that we can employ arbitrary LR, CR, and upsamplingalgorithms for the decoder.

BM3DGFσpsd
iteration
iteration
radius for Prop-I
radius for Prop-II
regularization parameter

25
1
2
2
1
0.04

Fig. 2. R-D curves (Q = 10, 20, · · · , 90).

B. Extended JPEG Decoder Type-II (Prop-II)
The chroma upsampling before the CR in the Prop-I diffuses
the artifacts in Cb and Cr . To suppress the diffusion in theProp-I, we also introduce Prop-II that has an LR and two lumaguided CRs “before” the chroma upsampling (see the right ofFig. 1). The Prop-II consists of the following steps:
1) Decimalize the bitstream into the quantized DCT coefficients.
2) Dequantize the quantized DCT coefficients.
3) Convert the coefficients in the spectral domain into one
luma Y and two chromas Cb and Cr in the spatial one.
4) [NEW] Restore the Y with an arbitrary LR method.
5) [NEW] Downsample the restored Y into the same sizeas the downsampled Cb and Cr .
6) [NEW] Restore the Cb and Cr with the GF guided bythe restored-downsampled Y .
7) Upsample the Cb and Cr downsampled on the encoderwith the default sampling factor [4 : 2 : 0] to the originalsize.
8) Convert the Y , Cb , and Cr into the colors R, G, and B.
Like the Prop-I, we can employ arbitrary LR, CR, and upsampling algorithms for the decoder, but note that the downsampling of the luma is required to apply to the downsampled
chroma.

IV. E XPERIMENTS
This study compared the proposed methods with a classicalalgorithm, full BM3D [6] applied after the whole decoding, atthe restoration of the noise-added JPEG-compressed images.
For simplicity, we employed the typical classical algorithms
such as the BM3D [6], GF [7], and bilinear interpolation
for the LR, CR, and chroma upsampling/downsampling, respectively. Table I shows the parameters for the BM3D andGF. We used 24 images of the Kodak Lossless True Color
Image Suite [8] as test images and added the AWGN with thestandard deviation 0.1 to the images.
Fig. 2 shows the rate-distortion (R-D) curves at mean peaksignal-to-noise ratios (mPSNRs) against various JPEG quality
factors Q and Fig. 3 shows the actual resulting images. TheProp-II outperformed the full BM3D and Prop-I in almost
experiments. Also, the mean execution times for the PropI and II were approximately 30-40 % compared to that for
the full BM3D because of the very fast implementation of
the GF. In addition, the time for Prop-II was slightly shorter
than that for the Prop-I because the CR-applied chromas of
the Prop-II were smaller than them of the Prop-I. Therefore,
it was proved that the proposed decoders can eliminate the758

2021 IEEE 10th Global Conference on Consumer Electronics (GCCE)

Fig. 3. Resulting images when Q = 80: (1st column) original, (2nd column) JPEG compression after noise addition, (3rd column) full BM3D, (4th column)
Prop-I, (5th column) Prop-II, (1st/2nd rows) kodim05, (3rd/4th rows) kodim07, and (5th/6th rows) kodim24.

redundant processes and improve the restoration performance.
V. C ONCLUSIONSThis study presented two extended JPEG decoders thatconsist of an LR and two luma-guided CRs before/after
chroma upsampling in addition to the standard decoder. Only
adding new simple algorithms directly to the existing decoder
not only eliminated many of the redundant processes, whichwere required in the conventional methods, but also improved
the restoration performance. The experiments showed that theproposed methods restored the noise-added JPEG-compressed
images faster and better than the classical one that does notconsider the JPEG process.

[3] P. Liu, H. Zhang, K. Zhang, L. Lin, and W. Zuo, “Multi-level waveletCNN for image restoration,” in Proc. CVPR, Salt Lake City, UT, June
2018, pp. 773–782.
[4] H. Chen, X. He, C. An, and T. Q. Nguyen, “Deep wide-activated residualnetwork based joint blocking and color bleeding artifacts reduction for
4:2:0 JPEG-compressed images,” IEEE Trans. Signal Process., vol. 26,
no. 1, pp. 79–83, Jan. 2019.
[5] N. Wada, M. Kazui, and M. Haseyama, “Extended joint bilateral filter
for the reduction of color bleeding in compressed image and video,” ITE
Trans. Media Tech. Appl., vol. 3, no. 1, pp. 95–106, Jan. 2015.
[6] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising bysparse 3D transform-domain collaborative filtering,” IEEE Trans. Image
Process., vol. 16, no. 8, pp. 2080–2095, Aug. 2007.
[7] K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397–1409, June 2013.
[8] “Kodak Lossless True Color Image Suite,” Kodak [Online], Available:
http://r0k.us/graphics/kodak/.

R EFERENCES[1] K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep CNN denoiser
prior for image restoration,” in Proc. CVPR, Honolulu, HI, July 2017,
pp. 3929–3938.
[2] M. Jiu and N. Pustelnik, “A deep primal-dual proximal network for image
restoration,” IEEE J. Select. Topics. Signal Process., vol. 15, no. 2, pp.
190–203, Jan. 2021.

759