<figure><img src="image_1.png" /></figure>

HAL
open science

# Developing Japanese Ikebana as a Digital Painting Tool
via AI

Cong Hung Mai, Ryohei Nakatsu, Naoko Tosa

<figure><img src="image_2.png" /></figure>

## To cite this version:

Cong Hung Mai, Ryohei Nakatsu, Naoko Tosa. Developing Japanese Ikebana as a Digital Painting Tool via AI. 19th International Conference on Entertainment Computing (ICEC), Nov 2020, Xi'an, China. pp.297-307, 10.1007/978-3-030-65736-9_27. hal-03686042

HAL Id: hal-03686042

https://inria.hal.science/hal-03686042

Submitted on 2 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Developing Japanese Ikebana as a Digital Painting Tool
via AI

Mai Cong Hung <fnref n="1" />,<fnref n="4" />, Nakatsu Ryohei<fnref n="2" /> and Naoko Tosa<fnref n="3" />

**Abstract.** In this research, we have carried out various experiments to perform mutual transformation between a domain of Ikebana (Japanese traditional flower arrangement) photos and other domains of images (landscapes, animals, portraits) to create new artworks via CycleGAN, a variation of GANs (Generative Adversarial Networks) - new AI technology that can perform deep learning with less training data. With the capability of achieving transformation between two image sets using CycleGAN, we obtained several interesting results in which Ikebana plays the role of a digital painting tool due to the flexibility and minimality of the Japanese culture form. Our experiments show that Ikebana can be developed as a painting tool in digital art with the help of CycleGAN and opens a new way to create digital artworks of high-abstracted level by applying AI techniques to elements from traditional culture.

**Keywords:** GANs, Cycle GAN, Digital art, Ikebana

## 1 Introduction

In recent years, the rapid development of AI, especially "Deep Learning [1]," has re-
shaped many fields of research. Not only the natural sciences and engineering have
been changed into new ways, but AI also has had a deep impact on the development of
social sciences and humanities. Among others, the application of deep learning in art is
of interest. Art has been widely considered as one of the most humanized areas and it
has been considered questionable that AI can learn the way people create artworks.
However, recently the power of deep learning has raised the question about the ability
of AI to learn, analyze, and even create art.

In this paper, we have tried to answer some of these questions by investigating the
relationship between art and AI. Also, we will propose a method to use Ikebana (Japa-
nese traditional flower arrangement) as a digital painting tool to create new artworks.
Firstly, in Section 2 we will explain Ikebana focusing on the role of Ikebana in the long

<footnote n="1">¹ Graduate School of Faculty of Science, Kyoto University, Kyoto, Japan</footnote>

<footnote n="2">² Graduate School of Advanced Integrated Studies in Human Survivability, Kyoto University,
Kyoto, Japan</footnote>

<footnote n="3">³ Design School, Kyoto University, Kyoto, Japan</footnote>

<footnote n="4">⁴RIKEN, Osaka, Japan
hungmcuet@gmail.com, nakatsu.ryohei@gmail.com,
naokotosa@gmail.com</footnote>
history of art. And we will explain why Ikebana can work as a digital painting tool by using AI.

Then in Section 3, we will explain CycleGAN [3], one of the recent new AI technologies, that can work as a basic method to make Ikebana work as a digital painting tool. A detailed explanation of CycleGAN including the explanation of how it could be applied to Ikebana will be given in Section 3. But here we briefly explain the basic concept behind the technology. Firstly, we explain the generative model in deep learning. The generative model along with the discriminative model are two kinds of neural networks in deep learning. Informally, the name "generative" came from the fact that the generative model can generate new data instances while the discriminative model can discriminate between different categories of data. In probability terms, the generative model learns the joint probability p(X, Y) of data instances set X and label set Y while the discriminative model learns the conditional probability p(Y|X). Generative models study the distribution of X and how likely a given example belongs to that distribution. In art style transfer, a generative model would transfer a photo into a specific style or can generate new photos from random input which are likely to belong to the distribution representing an artist's style.

Among generative models, GANs (Generative Adversarial Networks) [2] have been extensively studied in recent years because of their powerful efficiency while not requiring a large number of training data. GANs is popular because it can perform deep learning with a relatively small number of training data (see [3]). The composition of GANs includes two networks, a generator network (G) and a discriminator network (D), as in Fig. 1. In the training of GANs, network G learns to generate data from random noise while D tries to identify the generated data whether it is real or fake. The training process can be interpreted as a zero-sum game between G and D if we use the terms from Game Theory [4] when the training process on G tries to maximize the probability of the generated data to lie on the distribution of target sets and the training process on D tries to minimize it. With this minimax mechanism, the networks can converge even with a relatively small number of learning data. With this concept, a large number of GANs variation has been developed by modifying the basic configuration.

<figure><img src="image_4.png" /><figcaption>Fig. 1. the basic configuration of GANs</figcaption></figure>
## 2 The Art of Ikebana

In this research, we study the usage of the generative models in deep learning to generate digital art with Ikebana to be the main painting tool. Ikebana is the art of flower arrangement in Japanese culture [5][6]. The word "Ikebana" comes from the Japanese word "Ikeru" ("be living" or "to have a life") and "Hana" ("flower"). It means that the flowers are given life under the arrangements of the artists. As an important element in Japanese culture under the influence of Zen [7], Ikebana has a strong connection with other Japanese art forms such as Tea Ceremony [8].

Viewing plants and flowers is a notable tradition of the Japanese along with the long history of its culture and is related to the Japanese aesthetics of harmony to nature. Ikebana has a very deep root to the Japanese philosophy of art. Ikebana is established in Japan in the Heian period (794-1185) after Buddhist monks brought the tradition of arranging flowers on Buddha from China. In the early stage, it was just placing a flower in vases but in the next centuries, it grew to be an art form under the influence of Zen Buddhism. Like other Zen-influenced art forms such as Haiku [9] or Tea Ceremony, Ikebana is not just beautifully arranging flowers but it gives the path to be harmonic to nature.

The philosophy of Ikebana includes two important properties which are influenced by the aesthetic of Zen. First is the minimality as Ikebana not only considers the importance of the materials but also the space surrounding them. An Ikebana artist can represent a complete scene with a lot of space to give a minimal view. Secondly, Ikebana let natural living materials - flowers, leaves, and branches - free to grow and change. We consider it as flexibility when the materials can be placed in various shapes and arrangements. The minimal form of Ikebana gives us the idea that it is considered as a minimally condensed form of nature and also beauty in nature. This also gives us an inspiration using Ikebana as a source of creating various new natural forms that could work as new artworks.

The development of Ikebana has continued until today and has not been limited within a traditional art. In the modern digital art scene, one example of Ikebana working as the inspiration source is "Sound Ikebana" created by one of the authors, Naoko Tosa (Fig. 2), which used fluid dynamics to create Ikebana-like forms from different types of water-based solution [10][11]. The resemblance between Sound of Ikebana and actual Ikebana also motivated us to the idea of integrating this traditional art form into the temporary scene of digital art.

<figure><img src="image_5.png" /><figcaption>Fig. 2. Naoko Tosa's Sound of Ikebana</figcaption></figure>
## 3 CycleGAN

Among the variations of GAN, CycleGAN is an elegant method to study the mutual transformation between two sets of data [3]. We consider the generative models on CycleGAN as the main tool to do the task of creating digital Ikebana artworks.

The architecture of a CycleGAN network is illustrated as in Fig 3. In comparison to traditional GANs, we add an inverse transformation $G_{BA}$ of the generator network $G_{AB}$, which has the data of domain A as input to transform them into an element of domain B. We also use two discriminator $D_A$ and $D_B$ for the domains A and B, respectively. We would measure the difference between A and A' (the reconstruction data in A by applying $G_{AB}$ then $G_{BA}$) and the error caused by the difference between B and the domain given by applying $G_{AB}$ to A. The training process would minimize the sum of these two errors. The data generated by $G_{AB}$ and $G_{BA}$ provides the mutual transformation between the two domains.

<figure><img src="image_6.png" /><figcaption>Fig.3. The basic configuration of CycleGAN ([12])</figcaption></figure>

The most important difference between GANs and CycleGAN as generative models is GANs learn to generate data to fit in a target set while CycleGAN learns the set-to-set level of transformation. Because of this feature, CycleGAN could be used to establish mutual conversion between these two groups of images (for example art styles of two artists). Figure 4 shows how horses are converted into zebras and vice versa. The development of CycleGAN has opened a new way to create (or correctly, transform) artworks that have the specific style of an artist. Moreover, in Art, our goal would not be limit to copy the style, we are also desired to create original ideas. We have an assumption that if we use CycleGAN to perform mutual conversion between two groups of images with one group behaves like the painting tools, the transformation would result in a new kind of art. In this research, we use Ikebana as the painting tools to create new artworks via CycleGAN.
<figure><img src="image_7.png" /><figcaption>Fig. 4. Horses-Zebras transfer (Image source [3])</figcaption></figure>

# 4 Experiments

As described above, CycleGAN can perform the mutual transformation between 2 sets of images and does not require one-to-one correspondence between images belonging to each image set. It means that AI can perform high abstract style transfer because the generative models in CycleGAN will learn the distribution (style or characteristic features) of a domain of images.

In the classic example of CycleGAN, the generator G can be used to transfer digital landscape photos into Monet's style and make a mutual transformation between horse and zebra or winter landscape scenes and summer landscapes scenes. In other words, these examples worked with images of relatively similar in size, theme, and category. Artworks by Monet and landscapes photos are representing the same category of subject, similar to winter and summer landscapes things and horses and zebras are animals with similar size and shape. The transformations between these domains are imaginable without using AI.

In this research, we consider another viewpoint of mutual transformation by applying CycleGAN to relatively different domains of objects which are difficult to imagine the mutual transformation. The mutual transformation between macro and micro-size world or between plants and animals would be unusual but unusual (but harmonic) concepts are a key point to create art. We predict this "unusual transformation" would give a high-abstracted representation of things, and it could generate artworks when one domain can play the role of a painting tool.

Inspired by the ability to represent nature in a minimal way of Ikebana, we set a sequence of experiments of CycleGAN in which a set of Ikebana images is one of the
two domains. To perform the "unusual transformation", we choose image sets that are relatively different from flowers: landscapes, portraits, and large-sized animals. We use the datasets below and make an experiment of mutual transformation via CycleGAN between A and B1, B2, B3 respectively.

Dataset A: Ikebana photos in Flickr

Dataset B1: Landscape photos in Kaggle

(https://www.kaggle.com/arnaud58/landscape-pictures)

Dataset B2: Portrait photos in Kaggle (partially)

(https://www.kaggle.com/laurentmih/aisegmentcom-matting-human-datasets)

Dataset B3: Elephants and Horses in Kaggle Animal-10 dataset.

(https://www.kaggle.com/alessiocorrado99/animals10)

The classic transformation between Ikebana and Landscapes comes from the traditional idea of Ikebana. Our experiment would re-investigate the fundamental ability of Ikebana as an art form. The second pair Ikebana and portraits are inspired by the comparison between flowers and human faces in many traditional Oriental pieces of literature and poems. We consider the last transformation because the transformation between large-sized moving animals and small-sized still plants is ideal for our concept of "unusual transformation".

# 5 Results and discussion

In this section, we give some notable results and discussion of each experiment.

## 5.1 A - B1 transformation

Some interesting results are shown in Fig. 5.

In this experiment, we could see Ikebana can minimize the landscape from different spatial perspectives. The representation gives quite similar results as the traditional Ikebana. It is compatible with the pre-assumption that the traditional Ikebana was mostly inspired by the natural shape of landscapes.
<figure><img src="image_8.png" /><figcaption>Fig. 5. Some transformation results from landscape photos to Ikebana images.</figcaption></figure>

## 5.2 A - B2 transformation

Some successful transformation from human faces into Ikebana are shown in Fig. 6.

We find it interesting when the shapes of original human faces are remaining. It proves that the arrangement of flowers can inspire a new kind of portrait art.
<figure><img src="image_9.png" /><figcaption>Fig. 6. Some transformation results from portrait photos to Ikebana images.</figcaption></figure>

5.3 A - B3 transformation

Some successful examples are given in Fig. 7. Similar to A-B2 transformation, the shapes of the animals are also preserved after being transformed into Ikebana. In our opinion, these results could be regarded as one kind of Surrealism Art, which reflects an unreal world.
<figure><img src="image_10.png" /><figcaption>Fig. 7. Some transformation results from animal photos to Ikebana images.</figcaption></figure>

5.4 Other results

We also give some fail examples. The first type of fail examples shown in Fig. 8 is the over-transformation when we could not see the original shape of the input photo. We consider that this problem arises because our training data set of A includes many
photos that are over transformed. We plan to improve it by replacing them with some
successful results for the next experiments.

<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Several failed examples of over transformation.</figcaption></figure>

The second type of fail examples shown in Fig. 9 is that the network failed to perform
good transformation into Ikebana. We consider the reason as the generator failed at
transforming some highly complex photos. We predict this problem would also be re-
solved if we could improve the training sets by adding successful results into the train-
ing set of A.

<figure><img src="image_12.png" /><figcaption>**Fig. 9.** Several failed examples of transformation.</figcaption></figure>

In the inverse direction of the transformation A-B2 shown in Fig. 10, we could see
that CycleGAN can reconstruct some local parts of the human face instead of a whole
face as expected (there is some exception though). This is the limit of generative models
in deep learning when it is noise-vulnerable and performs the transformation locally (in
terms of pixel neighborhood) when one domain has a complex structure. While in the
reverse, the flexibility and minimality of Ikebana transform avoid that limit.
<figure><img src="image_13.png" /><figcaption>Fig. 10. Several failed inverse transformation examples.</figcaption></figure>

# 6 Conclusion

As can be seen from the results of our sequences of experiments, we can conclude that CycleGAN can help an artist generate original artworks by transforming a target set photos into its representation by Ikebana. With the flexibility and minimality in its nature, Ikebana could be developed into a new kind of digital painting tools. With the help of AI, especially the generative models inspired by GANs, we believe other traditional art forms such as Chinese Sanshui Painting can also be used in modern digital art in the same way.

We also note about the concept of “unusual transformation” used in this research. The mutual transformation via CycleGAN would open a new way to create art when we perform mutual transformation between objects of relatively different categories. Via CycleGAN, other transformation could be experimented to investigate the power of AI in creating new art.

# Reference

1. Jorn D. Kelleher, “Deep Learning,” MIT Press (2019).

2. Antonia Creswell, et al., “Generative Adversarial Networks: An Overview,” IEEE Signal Processing Magazine, Vol.35, No.1, pp.53-65 (Jan. 2018).

3. Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks,” The IEEE International Conference on Computer Vision (ICCV), pp.223-2232 (2017).

4. Steven Tadelis, “Game Theory: An Introduction,” Princeton University Press (2013).

5. https://en.wikipedia.org/wiki/Ikebana

6. Shozo Sato, “Ikebana: The Art of Arranging Flowers,” Tuttle Publishing (2013).

7. Suzuki, D., “Zen to Nihonbunka (Zen Buddhism and its Influence on Japanese Culture, “Iwanami (1981)

8. A. L. Sadler, Laura C. Martin, “The Japanese Tea Ceremony: Cha-no-Yu and the Zen Art of Mindfulness,” Tuttle Publishing (2019).

9. Judith Patt, Barry Till, Michiko Warkentyne, “Haiku: Japanese Art and Poetry,” Pomegranate (2010).
10. Naoko Tosa, Yunian Pang, Qin Yang, Ryohei Nakatsu, "Pursuit and Expression of Japanese Beauty Using Technology," Special Issue "The Machine as Artist (for the 21st Century)," Arts journal, MDPI, Vol.8, No.1, 38 (2019).

11. Naoko Tosa, Pang Yunian, Liang Zhao, Ryohei Nakatsu, "Genesis: New Media Art Created as a Visualization of Fluid Dynamics," Entertainment Computing – ICEC2017, LNCS 10507, Springer, pp.3-13 (2017).

12. https://bellchen.me/research/cs/gan/cyclegan-for-unsupervised-translation-in-anime/

13. Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, Wenzhe Shi; "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network" (2016).