Iconify: Converting Photographs
into Icons
Takuro Karamatsu1, Gibran Benitez-Garcia2,3, Keiji Yanai2,and Seiichi Uchida1
Kyushu University1, The University of Electro-Communications2,
OMRON SINIC X Corporation3
MMArt-ACM ‘201

UEC Yanai Lab. Tokyo.

2020.10.26ⓒ 2019 UEC Tokyo.

Motivation– Automatic generation of icons from segmented images, capable ofshowing sufficient abstraction and simplification of the original object
appearance.

2

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Difficulties– A domain conversion task between two different sample sets, naturalimages & icons (no paired data between both sets).
– Large style difference between both domains (severe abstraction).
– Large appearance variations in each domain. Even icons have large
variations in their shapes to represent various objects.

3UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Proposal• Objective:– Generate icon images automatically from natural photographs by using
machine learning techniques (Generative Adversarial Networks)• Approach:– No paired data of photos and icons is provided. Therefore, we have to adopt
unsupervised image-to-image translation methods, such as CycleGAN.

GAN
4UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Generative Adversarial Networks• CycleGAN*:– Based on a mapping between two image sets, 𝑋𝑋 and 𝑌𝑌, without giving any
image-to-image correspondence. Introducing the cycle consistency loss.

ⓒ 2020UEC Tokyo.
5 * Zhu, Jun-Yan, et al. "Unpaired image-to-image translation using
cycle-consistent
adversarial networks." ICCV 2017.
UEC Yanai Lab. Tokyo.

Generative Adversarial Networks• UNIT*:– Based on CycleGAN, introducing a condition that both input and output
should be represented by the same variable in the latent space 𝑍𝑍.

ⓒ 2020UEC Tokyo.
6 * Liu, Ming-Yu, et al. "Unsupervised image-to-image translation
networks."
NIPS 2017.
UEC Yanai Lab. Tokyo.

Datasets to Iconify• Photograph data:– 11,041 individual objects from 5,000
images of the MS-COCO dataset• Icon data:– 8,830 augmented black & white icons
from MS Power Point• Logo data:– 20,000 color logos randomly chosenfrom the LLD-logo dataset*
7 * S, Alexander, et al. "Logo synthesis and manipulation with clustered generative adversarial networks." CVPR 2018.
UEC Yanai Lab. Tokyo.

Iconify human photos (i)– Train both models CycleGAN andUNIT with the same dataset, andhyperparameters from originalimplementations– 1,440 augmented icon images(from 72 original icons), and 1,684
person photos– All images resized to 256x256 pixels

8

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Iconify human photos (Results)
Input

GAN results

UNIT results

9

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Iconify general objects– Train CycleGAN in a coarse-to-fine strategy with the complete datasets

10

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Iconify general objects to logos– Train CycleGAN in a coarse-to-fine strategy with the complete datasets

11

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Conclusion• Conclusions:– We experimentally proved that the transformation of natural photos into
icon images is possible by using GANs.
– CycleGAN has a sufficient “abstraction” ability to generate icon-like images• Future work:– Subjective and objective quality evaluation of the iconified images– Generate a larger icon dataset to improve the results quality– Analysis of the trained GANs for understanding how the abstraction has
been made12

UEC Yanai Lab. Tokyo.

ⓒ 2020 UEC Tokyo.

Thank You

13

UEC Yanai Lab. Tokyo.

ⓒ 2019 UEC Tokyo.