General Virtual Sketching Framework for Vector Line Art— Supplemental Material —HAORAN MO, Sun Yat-sen University, China
EDGAR SIMO-SERRA, Waseda University, JapanCHENGYING GAO∗ , Sun Yat-sen University, China
CHANGQING ZOU, Huawei Technologies Canada, Canada
RUOMEI WANG, Sun Yat-sen University, China
1

FRAMEWORK DETAILS

1.1

Stroke Representation

1.1.1 Width Factor Updating. The strokes are represented in a relative manner in our framework. That is, we need
to use the ending point of a stroke from last time step as the starting position for the next stroke, so that a quadratic
Bézier curve can be formed. Since the stroke representation 𝑎𝑡 in Eq. (1) in main paper contains the width factor 𝑤𝑡 , weneed to use the width factor 𝑤𝑡 −1 from last time step as the width (𝑟 0 )𝑡 for starting position for the next Bézier curve
𝑞𝑡 , as shown in Eq. (3) in main paper.
Because of the scalable window in our model, the size of two consecutive windows might be different. As a relativevalue based on the window size, when the width factor from last time step is used for the next window, it should bescaled to adapt to the new window. Practically, we re-define the (𝑟 0 )𝑡 in Eq. (3) in main paper as:(𝑟 0 )𝑡 = 𝑤𝑡′−1,(1)where 𝑤𝑡′−1 is the scaled width from last time step. The scaling process is:c′ = 𝑤𝑡 × (𝑊𝑡 −1 /𝑊𝑡 ), 𝑤 ′ = min(1.0, 𝑤
c′ ),
𝑤
𝑡
𝑡
𝑡

(2)

where 𝑊 denotes the window size andb· the original values after relative scaling. min operations performing value
clipping is adopted to avoid the out-of-bounds issue. In the experiments, we set initial values 𝑤 0′ = 0.01.
1.1.2 Differentiable Pen State Binarizing. The differentiable softargmax operation introduced in Section 3.2.1 is formulated as:
softargmax(𝑥) =𝑦𝑖where Í𝑒 𝑒 𝑦 𝑗 is the standard softmax operation and𝑒 𝛽𝑥𝑖
Í 𝛽𝑥 𝑗 𝑖,
𝑗𝑒
𝑖

Õ

(3)

𝑖 𝑧𝑖 𝑖 is the expectation for the index of the maximum probability.

Í𝑗The 𝛽 is to raise the maximum value and lower the others for a more accurate index. We note that this operation isa trade-off between accuracy and the intensity of gradients, which means a bigger 𝛽 leads to a smaller amount ofgradients.
∗ Corresponding author.

Authors’ addresses: Haoran Mo, Sun Yat-sen University, Guangzhou, China, mohaor@mail2.sysu.edu.cn; Edgar Simo-Serra, Waseda University, Tokyo,
Japan, ess@waseda.jp; Chengying Gao, Sun Yat-sen University, Guangzhou, China, mcsgcy@mail.sysu.edu.cn; Changqing Zou, Huawei Technologies
Canada, Markham, Canada, aaronzou1125@gmail.com; Ruomei Wang, Sun Yat-sen University, Guangzhou, China, isswrm@mail.sysu.edu.cn.

1

2

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

1.2

Stroke Generator

1.2.1 CNN Encoder. Apart from the patches cropped from the input image ∈ R3×128×128 and canvas ∈ R1×128×128 , aswell as the resized entire image ∈ R3×128×128 and canvas ∈ R1×128×128 , we use the cursor position ∈ R2 as additional
input. The cursor position is tiled to spatial maps ∈ R2×128×128 .
The architecture of CNN encoder is shown in Table 1. At the first layer of the encoder, we employ CoordConv [Liuet al. 2018] to boost the encoding of spatial relation between global images and local patches. The image features∈ R512×4×4 are flattened to feature vector ∈ R8192 . Then a fully connected layer converts the feature vector to image
embedding 𝑧𝑡 ∈ R128 .
Table 1. Architecture of CNN encoder.

Layer Type

Kernel Size

StrideNormalization Function

Activation Function

Output

Input

-

-

-

-

10 × 128 × 128

CoordConv

-

-

-

-

12 × 128 × 128

Convolutional

3×32

Instance Norm.

ReLU

32 × 64 × 64

Convolutional

3×3

1

Instance Norm.

ReLU

32 × 64 × 64

Convolutional

3×32

Instance Norm.

ReLU

64 × 32 × 32

Convolutional

3×3

1

Instance Norm.

ReLU

64 × 32 × 32

Convolutional

3×32

Instance Norm.

ReLU

128 × 16 × 16

Convolutional

3×3

1

Instance Norm.

ReLU

128 × 16 × 16

Convolutional

3×3

1

Instance Norm.

ReLU

128 × 16 × 16

Convolutional

3×32

Instance Norm.

ReLU

256 × 8 × 8

Convolutional

3×3

1

Instance Norm.

ReLU

256 × 8 × 8

Convolutional

3×3

1

Instance Norm.

ReLU

256 × 8 × 8

Convolutional

3×32

Instance Norm.

ReLU

512 × 4 × 4

Convolutional

3×3

1

Instance Norm.

ReLU

512 × 4 × 4

Convolutional

3×3

1

Instance Norm.

ReLU

512 × 4 × 4

1.2.2 RNN Decoder. At each time step 𝑡, the RNN decoder takes as input the image embedding 𝑧𝑡 ∈ R128 andthe previous hidden state ℎ𝑡 −1 ∈ R256 . In the implementation, besides the two factors above, we use the following
additional three inputs: (1) previous width factor 𝑤𝑡 −1 ∈ R for the prediction of 𝑤𝑡 because the their corresponding
endpoints form a shared stroke; (2) previous cursor 𝑄𝑡 −1 ∈ R2 to tell the model not to draw outside the full canvas;(3) Δ𝑆𝑡 −1 = 𝑊𝑡 −1 /𝑊𝐼 ∈ R and Δ𝑆𝑡𝑏𝑜𝑡𝑡𝑜𝑚= 𝑊𝑡 −1 /𝑊𝑚𝑖𝑛 ∈ R to suggest the model not to scale the window beyond the−1𝑢𝑝upper and bottom bound. The output 𝑜𝑡 ∈ R256 of RNN is then converted to stroke parameters 𝑎𝑡 ∈ R7 through a fully
connected layer.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —1.3

3

Coordinate System Change in Differentiable Pasting

0

1

2

3

4

5

6

x

0

Coord

1

2

x

1

Coord’

2

1

3
4

2

win1

win2

5

6

y

y

(a) Aligned Cropping(b) Differentiable PastingFig. 1. Aligned cropping and differentiable pasting.

In Fig. 1, we define original image space as coordinate system 𝐶𝑜𝑜𝑟𝑑 and the rendered (or cropped) patch space as
coordinate system 𝐶𝑜𝑜𝑟𝑑 ′ . 𝑄/𝑊 and 𝑄 ′ /𝑊 ′ denote the cursor/window size under 𝐶𝑜𝑜𝑟𝑑 and 𝐶𝑜𝑜𝑟𝑑 ′ , respectively. 𝑤𝑖𝑛1
represents cropping window (black lines) in Fig. 1-(a), and 𝑤𝑖𝑛2 the one (black lines of yellow region) in Fig. 1-(b). For′′
example, 𝑊𝑤𝑖𝑛1 = 3.7 and 𝑊𝑤𝑖𝑛1= 2. Note that 𝑊𝑤𝑖𝑛1= 𝑊𝑟 .

The aligned cropping operation depends on cursor and window size (𝑄 𝑤𝑖𝑛1,𝑊𝑤𝑖𝑛 ) from cropping window 𝑤𝑖𝑛1
under the base coordinate 𝐶𝑜𝑜𝑟𝑑. So during pasting (i.e., another kind of aligned cropping), our goal is to compute′′(𝑄 𝑤𝑖𝑛2,𝑊𝑤𝑖𝑛2) for cropping window 𝑤𝑖𝑛2 under 𝐶𝑜𝑜𝑟𝑑 ′ . This is done by first computing (𝑄 𝑤𝑖𝑛2,𝑊𝑤𝑖𝑛2 ) under 𝐶𝑜𝑜𝑟𝑑and then conversing them to 𝐶𝑜𝑜𝑟𝑑 ′ . The computation steps are as follows:↖

↘

(1) Position of 𝑤𝑖𝑛1 under 𝐶𝑜𝑜𝑟𝑑: the top-left position 𝑃 𝑤𝑖𝑛1 and bottom-right position 𝑃 𝑤𝑖𝑛1 of 𝑤𝑖𝑛1 are:↖

𝑃 𝑤𝑖𝑛1 = 𝑄 𝑤𝑖𝑛1 − 𝑊𝑤𝑖𝑛1 /2.0,↘

𝑃 𝑤𝑖𝑛1 = 𝑄 𝑤𝑖𝑛1 + 𝑊𝑤𝑖𝑛1 /2.0.

(4)(2) Position of 𝑤𝑖𝑛2 under 𝐶𝑜𝑜𝑟𝑑: 𝑤𝑖𝑛2 is the bounding box of rendered patch (dashed grid in Fig. 1-(b)) within 𝑤𝑖𝑛1.
Its top-left and bottom-right positions are:jk↖
↖
𝑃 𝑤𝑖𝑛2 = 𝑃 𝑤𝑖𝑛1 ,lm↘
↘
𝑃 𝑤𝑖𝑛2 = 𝑃 𝑤𝑖𝑛1 .

(3) Cursor and window size of 𝑤𝑖𝑛2 under 𝐶𝑜𝑜𝑟𝑑: they are calculated according to the bounding box:


↖
↘
↘
↖
𝑄 𝑤𝑖𝑛2 = 𝑃 𝑤𝑖𝑛2 + 𝑃 𝑤𝑖𝑛2 /2.0, 𝑊𝑤𝑖𝑛2 = 𝑃 𝑤𝑖𝑛2 − 𝑃 𝑤𝑖𝑛2 .

(5)

(6)

(4) Changing from 𝐶𝑜𝑜𝑟𝑑 to 𝐶𝑜𝑜𝑟𝑑 ′ :

 
↖
′
′
𝑄 𝑤𝑖𝑛2= 𝑊𝑤𝑖𝑛1/𝑊𝑤𝑖𝑛1 × 𝑄 𝑤𝑖𝑛2 − 𝑃 𝑤𝑖𝑛1 ,

′
′
𝑊𝑤𝑖𝑛2
= 𝑊𝑤𝑖𝑛1/𝑊𝑤𝑖𝑛1 × 𝑊𝑤𝑖𝑛2 .

(7)

4

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
After the coordinate system change for the cursor and size of window 𝑤𝑖𝑛2 in Fig. 1-(b), differentiable pasting is

performed as an aligned cropping process, and produces the patch in yellow region. By padding the left pixels withconstant values, a pasted canvas in full resolution is obtained, which is able to propagate the gradients derived from theraster-level loss to the floating-number cursor and window size.
1.4

Neural Renderer

The neural renderer in our work is similar to that in Learning-To-Paint [Huang et al. 2019]. The main difference is that
we discard the RGB and the transparency parameters, which are unnecessary in our task. The architecture is as follows:
Table 2. Architecture of neural renderer.

Layer Type

Kernel Size

Activation Function

Output

Input

-

-

10

Fully Connected

-

ReLU

512

Fully Connected

-

ReLU

1024

Fully Connected

-

ReLU

2048

Fully Connected

-

ReLU

4096

Reshape--

16 × 16 × 16

Convolutional

3×3

ReLU

32 × 16 × 16

Convolutional

3×3-

32 × 16 × 16

Pixel Shuffle--

8 × 32 × 32

Convolutional

3×3

ReLU

16 × 32 × 32

Convolutional

3×3-

16 × 32 × 32

Pixel Shuffle--

4 × 64 × 64

Convolutional

3×3

ReLU

8 × 64 × 64

Convolutional

3×3-

4 × 64 × 64

Pixel Shuffle-

Sigmoid1 × 128 × 128

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —2

5

DATASETS

2.1

Sketches for Vectorization and Rough Sketches Simplification

During training and evaluation, we use library gizeh 1 to render raster sketches with vector sequential points in
QuickDraw [Ha and Eck 2018] dataset. In training, we render sketches in multiple resolutions ranging from 128px to278px. All the raster images are rendered on the fly during training. For different resolutions, we use different numbersof sketch objects. For each object, we render it into different sizes with different line thickness, and then place them todifferent position. All the information are shown below:
Table 3. Information of raster sketch rendering during training.

Resolution

Object Number

Object Size

PositionLine Thickness[128, 172]1

𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙

Center3(172, 225]

1

𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙

Center3 or 4(172, 225]2

[128, 𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙 × 0.75]

Random3 or 4(225, 278]

2

[128, 𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙 × 0.75]

Random3 or 4

During evaluation, we render sketches at four resolutions: 128px, 256px, 384px and 512px. All have only one object
in original size and line thickness 3. All the objects are placed in the center.
We use 10 categories for training: airplane, bus, car, sailboat, bird, cat, dog, tree, flower and zigzag. During evaluation,
we augment three categories: rabbit, circle and line.
2.2

Rough Sketches for Simplification

In rough sketches simplification task, we use the clean sketches for vectorization as target images and use the pencil art
generation technique in [Simo-Serra et al. 2018] to produce the rough sketches as input images. We use the released codeand model 2 . There are two drawing styles from two artists, and we use both during training for better generalization.
2.3

Photograph to Line Drawing

There are face images and the corresponding segmentation masks in CelebAMask-HQ [Lee et al. 2020] dataset, whichcontains 19 categories for the facial details. We discard categories (l_ear, r_ear, hair, hat, neck, ear_r, neck_l and cloth)
and adopt (skin, nose, eye_g, l_eye, r_eye, mouth, u_lip, l_lip, l_brow and r_brow). For mouth, u_lip and l_lip, we merge
their masks to form an entire shape. For l_brow and r_brow, we use skeleton extraction algorithm to convert them into
thin lines. The original size for face images is 1024px and annotated mask 512px. We resize them to both 256px for
training and testing. The facial sketches are obtained by applying canny edge detection algorithm to the mask image.
The canny edge is in 1-pixel width, and then morphological dilation method is employed to thicken the lines. Sometraining or evaluation examples are shown as follows:

1 https://github.com/Zulko/gizeh

2 https://github.com/bobbens/sketch_simplification

6

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Image

SketchImage

SketchImage

SketchFig. 2. Training or evaluation examples of photograph to line drawing.

Image

SketchGeneral Virtual Sketching Framework for Vector Line Art— Supplemental Material —3

7IMPLEMENTATION DETAILS

3.1

Avoid Recursive Gradient Propagation

When training the sequential model, some outputs from last time step are used as inputs for the next step, like thecanvas 𝐶, cursor 𝑄, etc. To avoid recursive gradient back-propagation, we break the gradients of all the inputs of CNN
encoder and the additional inputs of RNN decoder. Gradient breaking should also be done when updating the cursor 𝑄𝑡
and window size 𝑊𝑡 in Eq. (4) in main paper, where gradients should not be propagated recursively from 𝑄𝑡 and 𝑊𝑡 to
𝑄𝑡 −1 and 𝑊𝑡 −1 .
3.2

More about Training

For each experiment of our method, the training is done on 2 GPUs, each with different resolutions (except forphotograph to line drawing which is trained on images of single resolution).
3.3

Random Movement of Cursor during Testing

During testing or evaluation, in order to help the window slide to distant undrawn area more efficiently, the cursor is
randomly moved to another position. We use different strategies of random cursor movement for different types ofimages:Real Clean Sketches. Since the input clean sketches are also the target ones, we are able to detect the undrawn pixels
by comparing the target and the canvas. After each round of drawing, we first divide the full-size sketch and canvas
into several 128 × 128 grids, and calculate their number of undrawn stroke pixels and stroke drawing accuracy. Then,
when the stroke drawing accuracy of all grids is higher than 𝑘%, the drawing is stopped early without using up all
the rounds for drawing. Otherwise, we select the grid with the most undrawn stroke pixels, and move the cursor to arandom position inside that grid. 𝑘 = 95 is used in the experiments.
Rough Sketches and Photographs. Different from clean sketches, rough sketches and photographs cannot be used astarget during testing. So the cursor movement strategy is different from the one for clean sketches. We move the cursor
to a random position without any restriction.

8Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

4

RESULTS

4.1

Effectiveness of Moving and Scaling

Figure 3 shows results of how to move, break the continuous strokes and slide to undrawn region. Figure 4 shows
results of how to enlarge the window to search for undrawn region, and then slide there with long strides. Figure 5
shows the window size distributions of our dynamic window-based model on vectorization task. In contrast, fixed-size
window-based model uses a window size of 128px. Visual comparisons between dynamic window-based model andfixed-size window-based one is shown in Figure 6.
Drawing order
Input

Output (vector)

Stroke orderMoving Trajectory

Input

Output (vector)

128px

256px

384px

512px

Fig. 3. Results of moving, sliding and breaking.

Stroke orderMoving Trajectory

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —
Input

128px

256px

384px

512px

9
Intermediate canvases

52

58

59

45

55

66

42

46

67

70

77

99

79

89

102

69

91

58

54

92

58

67

60

95

77

108

57

54

132

136

133

85

82

74

58

77

100

121

71

56

55

66

91

118

138

95

93

97

65

92

122

144

163

56

50

86

Fig. 4. Results of window scaling. Red boxes represent the windows. Numbers on top right corner indicate the window size. Blue lines
are the slide trajectory which are not drawn finally.

10

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
0.45
0.4
0.35
0.3

Res=128

Res=25680

100

Res=384

Res=512Percentage

0.25
0.2
0.15
0.1

0.05
0

30

40

50

60

70

90

110

120

130

140

150

Window size
Fig. 5. Distributions of window sizes in our dynamic window-based model without stroke regularization. Here we only consider
windows used to draw strokes instead of sliding or breaking.

Input

Fixed-size

Ours (scalable)

Input

Fixed-size

Ours (scalable)

128px

256px

384px

512px

Fig. 6. Comparisons between scalable/dynamic window-based model (ours) and fixed-size window-based one.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —4.2

11Effectiveness of Differentiable Pasting

Figure 7 shows the distributions in model with or without differentiable pasting at a low and a high resolution on
vectorization task. Figure 8 shows how the model with non-differentiable pasting scales the window. Figure 9 shows

Percentage

visual comparisons between differentiable and non-differentiable pasting.

Differentiable

0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.91

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9(a) Resolution = 128px
Percentage

Non-differentiable2

Scaling value

0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.91

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9(b) Resolution = 512px

2

Scaling value

Fig. 7. Distributions of scaling factor values in model with differentiable or non-differentiable pasting. Here we only consider thescaling values of windows used to draw strokes instead of sliding or breaking.

12

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Input

Sequential frames
35

68

33

63

34

67

34

83

42

61

49

72

41

60

65

35

67

34

66

36

65

45

69

46

70

47

67

47

128px

256px

384px

512px

Fig. 8. Window scaling of the model with non-differentiable pasting.

Drawing order

Input

Vector output(diff.)

Vector output(non-diff.)

Stroke order(non-diff.)

Input

Vector output(diff.)

Vector output(non-diff.)

128px

256px

384px

512px

Fig. 9. Comparisons between differentiable (“diff.”) and non-differentiable (“non-diff.”) pasting.

Stroke order(non-diff.)

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —4.3

13Ablation Study of Raster-level Loss

4.3.1 Other Raster Losses. Besides pixel-wise difference loss (𝐿1 difference) and perceptual loss, we also study adversarialloss. For a fair comparison, we use perceptual loss as the base supervision loss and add it to the objective function ofgenerator. Out-of-bounds penalty loss and stroke regularization loss are also added to the objective function of generator
during training. Figure 12 shows visual comparisons among different raster losses. When adopting the adversarial loss,
no significant difference is observed from the visual results of different resolutions. Though there is slight improvementin the quantitative results (Table 4), it is not significant enough. Given that an additional learnable discriminator leads
to higher training complexity, we discard the adversarial loss and use perceptual loss only.
Input

L1

Adversarial

Perceptual

Input

L1

Adversarial

128px

256px

384px

512px

Fig. 10. Comparisons among different raster losses.

Table 4. Quantitative results of different raster losses.

Raster Loss

Perceptual score(↓)

Chamfer distance(↓)

Perceptual

1.053

1.577

Perceptual + Adversarial

1.041

1.343

Perceptual

14

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

4.3.2 Different Layer Combinations. Figure 11 shows visual results of model with different perceptual layer combinations
from VGG-16 [Simonyan and Zisserman 2015]. Table 5 shows quantitative results.
Input image

Perc.12Perc.12_22

Perc.12_22_33

Perc.22_33_51

Perc.33_51

Perc.51

Ours

Fig. 11. Comparisons among different perceptual layer combinations. Our model uses “Perc.12_22_33_51”.

Table 5. Quantitative results of different perceptual layer combinations. Our model uses ∪(12, 22, 33, 51).

Perceptual Layers

Perceptual score(↓)

Chamfer distance(↓)

∪(12)

1.481

3.556

∪(12, 22)

1.399

3.242

∪(12, 22, 33)

1.058

1.219

∪(12, 22, 33, 51)

1.053

1.577

∪(22, 33, 51)

1.076

1.150

∪(33, 51)

1.513

2.995

∪(51)

1.764

4.079

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —15

4.3.3 Loss Value Normalization. Figure 12 shows results from models with and without loss normalization. Model
without loss normalization is similar to model dominated by deep perceptual layers, which produces results with goodcompleteness but worse details.
Input

W/O Norm.

With Norm.

Input

W/O Norm.

128px

256px

384px

512px

Fig. 12. Comparisons between models with and without loss normalization.

With Norm.

16

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

4.4

Effectiveness of Out-of-Bounds Penalty

Figure 13 shows results on vectorization from model without out-of-bounds penalty. Clearly, we can see empty canvases
in the output results. The high out-of-bounds penalty loss (calculated but not applied to the total loss function) during
training explains the reason. The window in our model moves in a relative manner, so it is possible that the window
moves outside the valid region of input image. At the beginning of training, where the model hasn’t learned how todraw well, it tends to move the window outside to produce an empty canvas so that a high raster loss can be avoided. To
this end, out-of-bounds penalty is necessary to tell the model not to do like this and make it learn to draw in a correct
direction.

Resolution

Input

OutputInput

Output

128px

256px

384px

512px

Fig. 13. Results of model without out-of-bounds penalty.

Input

OutputGeneral Virtual Sketching Framework for Vector Line Art— Supplemental Material —4.5

17Stroke Regularization

4.5.1 Redundancy Removal. After obtaining the output stroke data in vector format, we are able to detect the redundantor overlapped strokes: we draw each stroke step by step onto the canvas. Then, for each new stroke, we calculate
its intersection with the already drawn pixel region. Finally, if the intersection area is larger than half of the stroke
area, we regard this stroke as an overlapped one. With this detecting method, we are able to remove the overlappedstrokes as a post-processing operation. Figure 14 shows the performance of redundancy removal with different stroke
regularization weights 𝜆𝑟𝑒𝑔 .
Drawing order

Input

𝜆𝑟𝑒𝑔 = 0.0

𝜆𝑟𝑒𝑔 = 0.02

𝜆𝑟𝑒𝑔 = 0.1

𝜆𝑟𝑒𝑔 = 0.2

Fig. 14. Redundancy removal of stroke regularization with different weights 𝜆𝑟𝑒𝑔 . Red strokes in the grayscale vector outputs
represent the redundant or overlap strokes.

18
4.5.2

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Compactness Improvement. Figure 15 shows the compactness improvement by stronger stroke regularization.
Stroke number: 55
Average length: 14.82

Stroke number: 49
Average length: 16.08

Stroke number: 36
Average length: 19.04

Stroke number: 26
Average length: 26.13

Stroke number: 56
Average length: 17.49

Stroke number: 58
Average length: 16.96

Stroke number: 44
Average length: 19.78

Stroke number: 33
Average length: 25.62

Stroke number: 72
Average length: 16.56

Stroke number: 66
Average length: 19.32

Stroke number: 62
Average length: 19.20

Stroke number: 38
Average length: 30.27

Stroke number: 132
Average length: 17.99

Stroke number: 102
Average length: 23.78

Stroke number: 99
Average length: 23.64

Stroke number: 79
Average length: 30.00

Stroke number: 116
Average length: 18.08

Stroke number: 101
Average length: 20.05

Stroke number: 88
Average length: 21.40

Stroke number: 64
Average length: 28.86

𝜆𝑟𝑒𝑔 = 0.0

𝜆𝑟𝑒𝑔 = 0.02

𝜆𝑟𝑒𝑔 = 0.1

𝜆𝑟𝑒𝑔 = 0.2

256px

256px

256px

512px

512px
Input

Fig. 15. Compactness improvement of stroke regularization with different weights 𝜆𝑟𝑒𝑔 . Larger weights means stronger regularization(𝜆𝑟𝑒𝑔 = 0.0 indicates no regularization). Orange dots represent the endpoints of each drawn stroke.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —19

4.5.3 Parameter Sensibility. Figure 16 and Figure 17 show visual comparisons among different stroke regularization
weights 𝜆𝑟𝑒𝑔 . Model with a large 𝜆𝑟𝑒𝑔 suffers from bad completeness.

Input image (640px)

𝜆𝑟𝑒𝑔 = 0.0

𝜆𝑟𝑒𝑔 = 0.02

𝜆𝑟𝑒𝑔 = 0.1

𝜆𝑟𝑒𝑔 = 0.2

𝜆𝑟𝑒𝑔 = 0.5

𝜆𝑟𝑒𝑔 = 1.0

Fig. 16. Comparisons among different stroke regularization weights 𝜆𝑟𝑒𝑔 .

Input

𝜆𝑟𝑒𝑔 = 0.0

𝜆𝑟𝑒𝑔 = 0.02

𝜆𝑟𝑒𝑔 = 0.1

𝜆𝑟𝑒𝑔 = 0.2

𝜆𝑟𝑒𝑔 = 0.5

Fig. 17. Comparisons among different stroke regularization weights 𝜆𝑟𝑒𝑔 .

𝜆𝑟𝑒𝑔 = 1.0

20
4.6

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Vectorization

4.6.1 Qualitative Results. We compare with Learning-To-Paint [Huang et al. 2019] on sketches from QuickDraw datasetof different resolutions and different stroke numbers. Results are shown in Fig. 18, Fig. 19 and Fig. 20. More results andcomparisons with existing vectorization methods on real clean sketches are shown from Fig. 21 to Fig. 27.

Input(128px)LTP
5 strokes

LTP
LTP
LTP
LTP
LTP
10 strokes 16 strokes 24 strokes 32 strokes 48 strokes

Ours(vector)Fig. 18. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on clean sketch vectorization.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Input(256px)

LTP
16 strokes

LTP
24 strokes

21

LTP
36 strokes

LTP
48 strokes

Ours(vector)Fig. 19. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on clean sketch vectorization.

22

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

Input
(384px)

LTP
16 strokes

LTP
24 strokes

LTP
36 strokes

Ours(vector)Fig. 20. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on clean sketch vectorization.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Muten (1024px)

23Fidelity-vs-Simplicity[Favreau et al. 2016]

PolyVectorization[Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Drawing orderFig. 21. Comparisons with existing approaches on real clean sketch vectorization.

24

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

Sheriff (1024px)

Fidelity-vs-Simplicity[Favreau et al. 2016]

PolyVectorization[Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Drawing orderFig. 22. Comparisons with existing approaches on real clean sketch vectorization.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Elephant (640px)

25

Fidelity-vs-Simplicity[Favreau et al. 2016]

PolyVectorization[Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Drawing orderFig. 23. Comparisons with existing approaches on real clean sketch vectorization.

26

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

Hippo (700px)

Drawing order

Fidelity-vs-Simplicity [Favreau et al. 2016]

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke orderFig. 24. Comparisons with existing approaches on real clean sketch vectorization.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Penguin (720px)

27

Fidelity-vs-Simplicity[Favreau et al. 2016]

PolyVectorization[Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Drawing orderFig. 25. Comparisons with existing approaches on real clean sketch vectorization.

28

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

Banana tree (872px)

Fidelity-vs-Simplicity[Favreau et al. 2016]

PolyVectorization[Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Drawing orderFig. 26. Comparisons with existing approaches on real clean sketch vectorization.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Duck (887px)

29

Fidelity-vs-Simplicity[Favreau et al. 2016]

PolyVectorization[Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Drawing orderFig. 27. Comparisons with existing approaches on real clean sketch vectorization.

30
4.7

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Rough Sketch Simplification

We compare with Learning-To-Paint [Huang et al. 2019] on “Rough QuickDraw” dataset of different resolutions anddifferent stroke numbers. Results are shown from Fig. 28 to Fig. 31. More results and comparisons with existing methods
on complex rough sketches are shown in Fig. 32, Fig. 33 and Fig. 34.

Input(128px)LTP
10 strokes

LTP
16 strokes

Ours(vector)TargetFig. 28. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on rough sketch simplification.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Input(128px)LTP
10 strokes

LTP
16 strokes

31

Ours
(vector)TargetFig. 29. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on rough sketch simplification.

32

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang

Input
(256px)

LTP
36 strokes

Ours(vector)TargetFig. 30. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on rough sketch simplification.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —Input(256px)

LTP
36 strokes

33

Ours
(vector)TargetFig. 31. Comparisons with Learning-To-Paint [Huang et al. 2019] (LTP) with different stroke numbers on rough sketch simplification.

34

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Drawing order

Rocket (384px)

Sketch Simplification (pixel) [Simo-Serra et al. 2018]

Sketch Simplification (pixel) + PolyVectorization

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Duck (512px)

Sketch Simplification (pixel) [Simo-Serra et al. 2018]

Sketch Simplification (pixel) + PolyVectorization

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke orderFig. 32. Comparisons with existing approaches on rough sketch simplification.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —35
Drawing orderHouse (384px)

Sketch Simplification (pixel) [Simo-Serra et al. 2018]

Sketch Simplification (pixel) + PolyVectorization

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Kitten (512px)

Sketch Simplification (pixel) [Simo-Serra et al. 2018]

Sketch Simplification (pixel) + PolyVectorization

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke orderFig. 33. Comparisons with existing approaches on rough sketch simplification.

36

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Drawing order

Car (512px)

Sketch Simplification (pixel) [Simo-Serra et al. 2018]

Sketch Simplification (pixel) + PolyVectorization

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke order

Penguin (640px)

Sketch Simplification (pixel) [Simo-Serra et al. 2018]

Sketch Simplification (pixel) + PolyVectorization

PolyVectorization [Bessmeltsev et al. 2019]

Our results (vector)

Stroke orderFig. 34. Comparisons with existing approaches on rough sketch simplification.

General Virtual Sketching Framework for Vector Line Art— Supplemental Material —4.8

37Photograph to Line Drawing

More results and comparisons with existing methods are shown in Fig. 35 and Fig. 36.

Drawing order

Input

Learning-To-Paint[Huang et al.2019]

Photo-Sketching (pixel) Photo-Sketching (pixel)[Li et al. 2019]+ PolyVectorization
Our results (vector)

Fig. 35. Comparisons with existing approaches on photograph to line drawing.

Stroke order

38

Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang
Drawing order

Input

Learning-To-Paint[Huang et al.2019]

Photo-Sketching (pixel) Photo-Sketching (pixel)[Li et al. 2019]+ PolyVectorization
Our results (vector)

Fig. 36. Comparisons with existing approaches on photograph to line drawing.

Stroke orderGeneral Virtual Sketching Framework for Vector Line Art— Supplemental Material —39

REFERENCES
David Ha and Douglas Eck. 2018. A Neural Representation of Sketch Drawings. In International Conference on Learning Representations.
Zhewei Huang, Wen Heng, and Shuchang Zhou. 2019. Learning to paint with model-based deep reinforcement learning. In Proceedings of the IEEE
International Conference on Computer Vision. 8709–8718.
Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. 2020. MaskGAN: Towards Diverse and Interactive Facial Image Manipulation. In IEEE Conference
on Computer Vision and Pattern Recognition (CVPR).
Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski. 2018. An Intriguing Failing of Convolutional
Neural Networks and the CoordConv Solution. In Advances in Neural Information Processing Systems.
Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa. 2018. Mastering sketching: adversarial augmentation for structured prediction. ACM Transactions
on Graphics (TOG) 37, 1 (2018), 1–13.
Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on
Learning Representations.