# Choshi Design System from 2D Images

Natsuki Takayama, Shubing Meng, Takahashi Hiroki

► To cite this version:

Natsuki Takayama, Shubing Meng, Takahashi Hiroki. Choshi Design System from 2D Images. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.358-365, 10.1007/978-3-642-15399-0_38. hal-01055624

HAL Id: hal-01055624

https://inria.hal.science/hal-01055624

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Choshi design system from 2D images

Natsuki Takayama¹, Shubing Meng², and Takahashi Hiroki²

¹SI ELECTRONICS.LTD.,

3-12-8 Sotokanda, Chiyoda-ku, Tokyo

²Graduate School of Informatics and Engineering, University of
Electro-Communications,

1-5-1 Chofugaoka, Chofu-shi, Tokyo

msb8063@yahoo.co.jp, rocky@hc.uec.ac.jp

**Abstract.** This paper proposes a Choshi design system. Choshi is a new method for carving of paper, which is uneven 3D shape and unique colors of papers. Choshi, derived from carving overlaid colored papers, has the following three features:

1. Each layer consists of a single piece of paper of one color.

2. The color must be selected from a number of existing colors.

3. Choshi has an overlaid structure where carved papers are overlaid on other carved papers.

The goal of the proposed Choshi design system has two issues: to enable a wider variety of people to easily and successfully create Choshi art, and to reduce the difficulty and tedium, of creating a Choshi art piece.

**Key words:** Choshi, Segmentation, Selecting colored paper, Overlaid structure

## 1 Introduction

Choshi[6] is a new carving method which can represent uneven 3D structures with unique colored papers as shown in Figure 1.

The process of Choshi art, shown in Figure 2, includes:

1. A color sketch should be prepared in advance. A number to each colored regions corresponding to the carving sequence is assigned.

2. Papers which are similar colors to those in the sketch are selected. All colored papers are put together according to the order.

3. The topmost layer should carve away the color regions of all the other colors. The next layer should carve away the color regions of all the remaining colors. This proceeds layer by layer to the bottom. The visible color regions in all papers result in a color sketch with sharp curves and a complex overlapping structure.

Compared with graphic works and three-dimensional arts, Choshi provides a better aesthetic perception to the viewers. It is, however, very difficult to popularize
<figure><img src="image_2.png" /><figcaption>Fig. 1. Choshi</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. Making processing of Choshi</figcaption></figure>

this art form due to its sophisticated structure, complicated and time-consuming
production processes, and high requirement for the artistic knowledge.

NPR is the short form of Non-Photorealistic Rendering. It assists artists in
creating real artworks starting with digital images, and provides the possibility
free the artists from labor of complex handwork and get the same or even better
artistic effect than that made by hand. Some of previous NPR methods which are
used for representing three dimensional shapes, similar to Choshi, are introduced.

Bas-relief is a form of relief in which the contents silhouette slightly stands
out from the surrounding surface. The overall depth of bas-relief is shallow so it
sets up an intermediate step between 2D image and 3D sculpture. For example
on most coins, images are in bas-relief. A presented digital bas-relief system[4]
assists artists in converting virtual 3D models into digital bas-relief sculptures.
During the conversion the input 3D models should be squeezed into a flat surface,
while maintaining as much as possible the perception of the full 3D scene. As
an NPR system it can generate real bas-relief automatically with the aid of
computer-driven milling equipment. In addition, this bas-relief system has a
few functions to assist artists in adjusting aesthetic qualities such as the camera
viewpoint or the compressed height in order to create digital Bas-relief intuitively
based on the their personal aesthetic perception.

Another virtual sculpture system[3] allows artists to realize traditional sculp-
ture in virtual space by CSG(Constructive Solid Geometry) and height-map. The
system provides an intuitive interactive interface to artist. Artists can carve any
complicated polygon by using a digital graver to cut forms on stone, wood, brick or any other material in virtual space.

Neither digital Bas-relief nor virtual sculpture can be used in Choshi generation for the discrete overlapping structure of color papers.

The rest of paper is organized as follows; section 2 describes Choshi system by giving the details of each process, and describes the application of the Choshi design system to the generation of carving data. Also, some real Choshi are carved by a cutting plotter based on the data. Section 3 concludes this paper.

# 2 Choshi design system

## 2.1 Generation of rough sketch

Region segmentation of Mean Shift algorithm[7] can not only keep the details and structures of the original features in images but also can facilitate the labeling process better. In this paper, $q = (p, c) = (x, y, r, g, b)$ is used as a feature vector, where $p = (x, y)$ represents a vector in spatial domain $H^2$, and $c = (r, g, b)$ represents a vector in color space $S^3$. In the five-dimensional feature space, mean shift vector $M(q)$ is obtained by the following equation.

$$M(q) = \frac{1}{n_x} \sum_{p \in H^2, c \in S^3} (q_i - q) \quad (1)$$

Where, $q$ and $q_i$ denote five-dimensional feature vectors of the data. $n_x$ represents the number of data within the radius of $d_H \in H^2$, $d_S \in S^3$ at the center $q$.

Local mode is obtained by shifting the target vector $q$ until the mean shift vector $M(q)$ becomes smaller than a threshold. This process is applied for the whole pixels in a given image. Color value in each pixel is replaced the mode in the local distribution. The adjacent pixels in spatial domain are merged into one region, where the distances in the feature space are within the range of $d_H$ and $\frac{d_S}{\sqrt{2}}$.

The adjacency relation of regions in the image space is represented by RAG. Color difference within the range of $\frac{d_S}{\sqrt{2}}$ in the color feature space is merged into one region. This process is repeated 3 times. The obtained image is used as rough sketch as shown in Figure 3(a).

## 2.2 Colored paper selection

In representation of the painting style like Choshi, it is most desirable to calculate color difference in a color coordinate system which is close to human color perception. Munsell color system has three axes H(Hue), V(Value) and C(Chroma) which indicate three attributes of human color perception respectively.

The representative color value of each color region in the rough sketch and colored papers are represented by HVC coordinate in approximate Munsell color
<figure><img src="image_4.png" /><figcaption>Fig. 3. Result of rough sketch image</figcaption></figure>

system by using MTM[9]. Then the color differences between the representative color in the rough sketch and the colored paper are calculated by Godlove Color different formula. A colored paper is selected when the color difference is minimum.

MTM corrects the evenness of Adams color space which approximates Munsell color space. At first, RGB color represented 8 bits in each component is transformed into CIE(1934)XYZ color value by using the following formula.

$$ \begin{pmatrix} X \\ Y \\ Z \end{pmatrix} = \frac{1}{255} \begin{pmatrix} 0.608 & 0.174 & 0.200 \\ 0.299 & 0.587 & 0.144 \\ 0.0 & 0.066 & 1.112 \end{pmatrix} \begin{pmatrix} R \\ G \\ B \end{pmatrix} \qquad (2) $$

Next, Munsell Value function $V(A) = 11.6A^{\frac{1}{3}} - 1.6$ is applied to calculate ($M_1, M_2, M_3$) as three components in Adams color space according to equation (3).

$$ \left\{ \begin{array}{l} M_1 = V(1.02X) - V(Y) \\ M_2 = 0.4[V(0.847Z) - V(Y)] \\ M_3 = 0.23V(Y) \end{array} \right. \qquad (3) $$

$M_1$ and $M_2$ indicate color plane and $M_3$ is for Value. The approximate value ($\hat{H}, \hat{V}, \hat{C}$) of HVC in Adams color space is given by the following equation(4).

$$ \hat{H} = \tan^{-1} \left( \frac{M_2}{M_1} \right), \quad \hat{V} = M_3, \quad \hat{C} = (M_1^2 + M_2^2)^{\frac{1}{2}} \qquad (4) $$

Then, new axes $S_1$ and $S_2$ are obtained by equation (5) which aims to correct the distortion of $M_1$ and $M_2$.

$$ \left\{ \begin{array}{l} S_1 = 8.88 + 0.966\cos(\hat{H})M_1 \\ S_2 = 8.025 + 2.558\sin(\hat{H})M_2 \end{array} \right. \qquad (5) $$

Approximate value of HVC derived from MTM can be given by the following equation by using $S_1, S_2$ and $M_3$.

$$ \tilde{H} = \tan^{-1} \left( \frac{S_2}{S_1} \right), \quad \tilde{V} = M_3, \quad \tilde{C} = (S_1^2 + S_2^2)^{\frac{1}{2}} \qquad (6) $$
Godlove color difference formula gives weighted Euclidean distance between the two color samples in Munsel color space, where the distance is weighted by the perceptual ratio of C and V. Godlove color difference formula is shown as

<figure><img src="image_5.png" /><figcaption>Fig. 4. Result of colored paper determination</figcaption></figure>

equation(7), where a perceptual ratio of C and V is $C : V = 1 : n$. In this paper, $n$ is set to 4 according to the experimental result of Godlove[10].

$$ \Delta E = ((\Delta \tilde{C})^2 + 2\tilde{C}_1\tilde{C}_2(1 - \cos\Delta\tilde{H}) + (n\Delta\tilde{V})^2)^{\frac{1}{2}} \quad (7) $$

Figure 4 shows the selected colored paper according to equation (7).

## 2.3 Choshi description

In every colored paper, the regions which are not seen also exist, so the overlapping structure must be considered. Based on the result of color region segmentation, the whole image is indicated as $I$, and the set of all the color regions in

<figure><img src="image_6.png" /><figcaption>Fig. 5. Determine the sequence of colored papers</figcaption></figure>

tation, the whole image is indicated as $I$, and the set of all the color regions in
the image are indicated as $R_i \in I(i = 1, 2, ..., n|n)$, n indicates how many pieces of paper in use. For $I = \cup_{i=1}^n R_i$, $\bar{P}_i$ indicates the cut-off part in each colored paper. $P_i$ denotes the left part. When the papers are piled up in the order of i, $P_i$ is represented as follows;

$$ \left\{ \begin{array}{l} P_1 = \\ P_i = P_{i-1} \cup R_i \quad (i = 2, ..., n) \end{array} \right. \qquad (8) $$

The cut-off part of each colored paper $\bar{P}_i$, therefore, derives from $\bar{P}_i = I - P_i = \cup_{k=i+1}^n R_k$ obtained by the following equation;

$$ \left\{ \begin{array}{l} \bar{P}_n = \\ \bar{P}_i = \bar{P}_{i+1} \cup R_{i+1} \quad (i = n-1, ..., 1) \end{array} \right. \qquad (9) $$

The overlapping structure model is illustrated in Figure 5.

The proposed Choshi system assigns certain numbers to the color regions based on the sequence which is determined by the order of generated regions in the rough sketch generation process. The number indicates a height.

After the selection of colored papers, user can adjust the height of each colored paper by exchanging the numbers of the regions. Moreover, user can merge some of the regions in order to modify the shapes of regions and the number of colors.

It is realized by interactive operations that are mouse operation and number assignment. The Choshi model is generated in a virtual space based on the equations (8) and (9) as shown in Figure 6.

Then, the cutting plotter is used to cut each colored paper according to the output data and the colored paper is piled up in the order. Finally, a real Choshi can be obtained. The generated Choshi is shown in Figure 7.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Virtual Choshi model</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Fig. 7. Carved Choshi</figcaption></figure>

## 3 Conclusion and Future Works

The system proposed in this paper makes it possible to generate virtual Choshi beginning with an input image. It takes into account the following three features of Choshi in order to keep the generated overlapping structure as similar as possible to the image.

1. Choshi is composed by a set of colored papers and each color region is represented by a piece of colored paper. Each region, therefore, should hold a uniform color.

2. Segmented regions show many kinds of colors, but the resulting Choshi will be realized by colored papers. Thus the categories of colors have to be limited to the existing colored papers.

3. In every colored paper, the regions which are not seen also exist, so the overlapping structure must be considered.

Moveover, as an automatic method, it provides users a simple and pleasant way to make Choshi. It is helpful to spread the Choshi as a common art form.

However, this system is not mature enough. It still needs further improvement.

In proposed system, the quality of the result is very dependent on the quality of the segmentation. Experiment results show that the Mean Shift algorithm is more adapt for defining color regions of illustration as shown in Figure 8(a), however, it does not seem to robustly define ideal regions of input image with complex scenes, like photograph. As shown in Figure 8(b), the discrete leaves and windows did not get enough merging for matching unique color regions of Choshi.

In addition, the Mean Shift algorithm can not intelligently judge the relationship among regions in an image. Some unnatural effects exist as the results of incorrect judgement of automatic segmentation, like the separated sky regions shown in Figure 8(b).

Such points will lead to the results that a Choshi artist would not be satisfied with. Therefore, it might be preferable to provide the artist with tools to
<figure><img src="image_9.png" /><figcaption>Fig. 8. Experimental results</figcaption></figure>

guide the segmentation process and edit the results, rather than to rely on the
described region segmentation algorithm.

References

1. Miyada, K.: Technical trend of rendering. The Journal of the Institute of Image Information and Television Engineers 57, 4, 442–443(2003)

2. Takagi, S.: Research trend of non-photorealistic rendering. Technical report of IE-ICE. HCS 23, 63, 1–6(1999)

3. Okada, M., Ichiro, Toriwaki, J., Mizuno, S.: Virtual sculpting and virtual woodblock printing by a model-driven scheme. Information Processing Society of Japan 1, 2, 74–84(2002)

4. Weyrich, T., Deng, J., Barnes, C., Rusinkiewicz, S., Finkelstein, A.: Digital bas-relief from 3D scenes. ACM Transactions on Graphics (Proc. SIGGRAPH) 26, 3, 32–38 (2007)

5. Ikuta, D., Tanaka, T., Horiba, I., Yamamoto, S., Suqie, N.: Relief design system with multiple modeling methods. Technical Report of IEICE 99, 79, 19–24(2000)

6. Hayashi, K.: Choshi arts. Patent Laid-Open Number: 2004-255858.

7. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans on PAMI. 24, 5, 603–619(2002)

8. Tremeau, A., Colantoni, P.: Regions adjacency graph applied to color image segmentation. IEEE Trans. Image. Process 9, 4, 735–744(2000)

9. Miyahara, M., Yasuhiro, Y.: Mathematical transform of (r, g, b) color data to munsell (h, v, c) color data (high efficiency coding technic for recording and transmission). The Journal of the Institute of Television Engineers of Japan 43, 10, 1129–1136(1989)

10. Godlove, I.: Improved color-difference formula, with applications to the perceptibility and acceptability of fadings. 760–772