Pre-trained and Shared Encoder in Cycle-Consistent
Adversarial Networks to Improve Image Quality
Runtong ZHANG1 ， Yuchen WU1， Keiji YANAI2
1 University of Electronic Science and Technology of China, Chengdu, China

2 The University of Electro-Communications, Tokyo, Japan

Content
1.

Introduction

2.

Method3.

Experiment

4.

Evaluation Score

5.

Result Images

6.

Conclusion

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications2

1. Introduction

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications3

1.1 CycleGAN

Fig.1 Process in CycleGAN
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial
networkss. In: Computer Vision (ICCV), 2017 IEEE International Conference on (2017)◼

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications4

1.2 Problem
The words in the right-bottom of the picture is much blurrier compared with input.

Input

OutputFig.2 Enlarged images

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications5

1.2 Problem
Edge information will lost in encoding process and cannot be restored in decoder.

Fig.3 Generator Structure of CycleGANNov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

62. Method

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

72.1 ED-Block
ED-Block consists of an encoder and decoder and is trained at first to get a difference map bysubtraction.

difference map

Input

recovered image

Fig.4 Structure of ED-Block(1)

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications(2)8

2.2 Our Structure
After ED-Block being trained well, encoders in generators of CycleGAN share its parameters and will
be frozen during training. Finally, by adding the output from generators to the difference map, we canget images with much better quality.

Fig.5 Our Structure
Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

92.3 Formulas

(3)

(4)

(5)

(6)

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications10

3. ExperimentNov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

11

3.1 Training ED-Block
In images of 10,50,100 epochs, some color information are remained. But in higher training
epochs, the smaller color distortion effect. However, the time expense of 500 is much higher
than that of 200.

Fig.6 Difference map from different training epochs

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

12

3.2 Loss Graph

Fig.7 Loss values of generator and discriminator in each epoch

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications13

4. Evaluation ScoresNov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

144.1 SSIM&PSNR Scores
we use ''Apple2Orange'', ''Summer2Winter'' and ''blond-hair2brown-hair'' datasets. The
third one is selected from celebA dataset by us.
Table 1. SSIM and PSNR scores on ''Apple2Orange'' dataset.

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications15

4.1 SSIM&PSNR Scores
Table 2. SSIM and PSNR scores on ''Summer2Winter'' dataset.

Table 3. SSIM and PSNR scores on ''Blond-hair2Brown-hair'' dataset.

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

16

4.2 Comparison

Table 4. SSIM and PSNR scores comparison.

Nov. 2019

CycleGAN

Unit

DiscoGAN

SSIM+7.50%+15.77%+31.70%

PSNR+1.20%+4.78%+20.76%

University of Electronic Science and Technology of China
The University of Electro-Communications

17

5. Result Images

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications18

5. Resulted Images

Fig.8 Result Images in Apple2Orange dataset
Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

19

5. Resulted Images

Fig.9 Result Images in Summer2Winter dataset
Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

20

5. Resulted Images

Fig.10 Result Images in Blond-hair2Brown-hair dataset

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications21

6. Conclusion

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications22

6. Conclusion◼

We designed a novel block, called ED-Block, to extract edge information andsave it in difference map.

◼

We proposed ED-CycleGAN and by adding the output from generators to thedifference map, we can get images with much better quality.

◼

Our model get notable better SSIM and PSNR scores compared with CycleGAN,
Unit and DiscoGAN.

Nov. 2019

University of Electronic Science and Technology of China
The University of Electro-Communications

23

Thanks for your attention.