# A Joint Training Model for Face Sketch Synthesis
## 4 Evaluation Experiments
### Synthesized Sketch Results Comparison  
Figure 3 shows some synthesized face sketches from different methods on the CUFS dataset. The first two rows are from the CUHK student dataset, the middle two rows are from the AR dataset, and the last two rows are from the XM2 VTS dataset. Moreover, the corresponding local blocks of the synthesized sketches are displayed in Figure 4. From Figure 3, it can be seen that the sketches synthesized by the LLE and MRF methods suffer serious block effects. The MWF method obtains better performance than the LLE and MRF methods in the CUHK student and AR datasets. However, the results are unsatisfying in the XM2 VTS dataset, because it contains more face variations, such as aging, race, and hair styles. The RSLCR method generates fine textures and structures, because more candidate patches are incorporated via random sampling and the locality constraint. However, this method results in blurred outputs. The FCN and GAN methods overcome the blurring effect by using pixel-to-pixel mapping from a photo to a sketch. However, they tend to have undesirable artifacts because of instabilities in training, while generating high-resolution images. Our proposed method  
Figure 2: Example of face sketch-photo pairs in the CUFS dataset (first two rows) and the CUFSF dataset (last two rows). The first and the third row are face photos and the second and the last rows are corresponding face sketches drawn by the artist. achieves much better performance than these six benchmarked methods in all the three datasets. More detailed information is preserved in the synthesized sketches, such as the double-fold eyelids and hair grain. This illustrates that the proposed method is capable of generating identity-preserved sketches. Figure 4: Local cut-out effect on Figure 3; same marshalling sequence as Figure 3.  
Figure 3: Synthesized sketches on the CUFS dataset by locally linear embedding (LLE) [7], Markov random fields (MRF) [9], Markov weight fields (MWF) [10], random sampling and locality constraint (RSLCR) [13], fully convolutional network (FCN) [15], generative adversarial network (GAN) [17], and our proposed method, respectively. Face photos in the first two rows are from the CUHK student dataset; second two rows are from the AR dataset; and the last two rows are from the XM2 VTS dataset, respectively. We also investigated the robustness of the proposed method against shape exaggeration and illumination variations on the CUFSF dataset. Figure 5 shows some synthesized face sketches from different methods used on the CUFSF dataset. The block effect still exists in the LLE and MRF results. The MWF and RSLCR methods obtain similar performance on the CUFSF dataset. The FCN method suffers from serious noise and artifacts. The GAN results show much improvement, but distortion occurs in the synthesized sketches by this method. By comparison, the proposed method achieves the most vivid and clear sketches, reflecting the robustness of our proposed method. Overall, the synthesized face sketch images on the CUFS and CUFSF datasets by our method have the following superiorities compared to the benchmarked methods: (1) Less block artifacts and noise, because we calculated the target sketch patch by weighted averaging hundreds randomly sampled training sketch patches but not few nearest patches; (2) rich facial detail, due to the high-pass filtered components were adopted to build the joint training model, which is able to generate high-quality face sketch patches; (3) complete facial structures, as a result of the training sketch images were took into consideration when computing the reconstruction weights, which weakens the influence by the misalignment in training images. Table 1 shows the average time consumption of different methods on different datasets. Here, only exemplar-based methods are compared, because it takes a very long time to train the neural network for deep learning-based methods, though the test time is quite fast once the model is trained.