# Line Drawing Extraction from Cartoons Using a Conditional Generative Adversarial Network
## 3 Line Drawing Extraction Using a Conditional GAN
### Line Drawing Extraction from a 2D Cartoon Image
#### 3.2.2 Conditional Generative Adversarial Network  
A conditional GAN was proposed to generate data by adding a conditional term to the GAN. A conditional GAN can generate data that meet certain conditions. The objective  
Figure 7: Feature line extraction of the 3D model according to the number of subdivision surfaces performed: (**a**) 1 time, wire frame; (**b**) 1 time, first line combination; (**c**) 1 time, second line combination; (**d**) 2 times, wire frame; (**e**) 2 times, first line combination and (**f**) 2 times.  
function of the conditional GAN can be expressed by adding to the objective function of the GAN, which represents a specific condition, as shown in Equation (9) [19; 20; 21; 22].  
\[\underset{G}{\text{minmax}}V(D,G)=E_{x\sim p_{data}(x)}[log(D(x|y))]+E_{z\sim p_{ z}(z)}[log(1-D(G(z|y)))]. \tag{9}\]  
First, the conditional GAN learns by using a generator network consisting of convolution and deconvolution layers. The generator consists of eight convolution layers and eight deconvolution layers, each one using a U-net network, as shown in Figure 8. In the encoding process (convolution), key features are detected while reducing the size of the feature map to understand the image. According to the feature map after the end of the encoding, the cartoon data are finally restored as line drawing data through decoding (deconvolution). Skip connections are used to restore the lost information in this process.  
The discriminator creates a patch on the basis of the input image and determines the input using this patch. The discriminator uses the patch GAN to determine whether the image generated by the generator corresponds with the input image. It does not determine whether the whole image is genuine or fake but determines whether a patch size is of a specific size and then it averages the result [19; 20; 21; 22]. The structure of the discriminator is shown in Figure 9.  
The input data consist of a 2D cartoon image \(x\) and a corresponding line drawing \(y\). The generator \(G\) learns the inputted 2D cartoon image, as shown in Figure 10, and generates fake data \(G(x)\) similar to the line drawing. The discriminator \(D\) alternately learns the 2D cartoon image \(x\), the real data \(y\) and the fake data \(G(x)\) gene  
Figure 8: The structure of the generator.  
Figure 9: The structure of the discriminator.  
and it determines whether \(y\) and \(G(x)\) are genuine data or not. Equation (10) is the objective function of the conditional GAN used.  
\[\underset{G}{\text{min}}\underset{D}{\text{max}}E_{y}[log(D(y))]+E_{x}[log(D(1-D (G(x)))]+E_{x,y}[\|y-G(x)\|]. \tag{10}\]  
As shown in Equation (10), the generator needs to generate an image that is as close as possible to the inputted 2D cartoon image so that the result of the objective function is minimized. The discriminator has the maximum value of the objective function to determine whether the line drawing generated by the generator corresponds with the 2D cartoon image. The learning is repeated by optimization based on the value of the objective function.  
The discriminator can distinguish whether the line drawing generated by the constructor at the beginning of the learning is the data corresponding with the 2D cartoon image. However, as the learning progresses, the generator generates fake data \(G(x)\) near \(y\), making it difficult to distinguish whether it is genuine or fake. By inputting a 2D cartoon image through this iterative learning, we can generate data that are close to the actual line drawing.  
Feature line detection and generation are performed by modifying pix2 pix [12] using a conditional GAN. pix2 pix is a network that generates images from images and the existing pix2 pix generates color images from edge images. However, we modified pix2 pix to detect line drawings in cartoon images for this study.