: A demo of generating face/sketch from small patches using the proposed recursive generation by bidirectional transformation networks (r-BTN). The left column shows the given sketch or face patches. The right columns show the results of recursive generation at different iteration steps in the correspondingly transferred domain. The proposed r-BTN gradually generates realistic faces/sketches consistent to the given small patches. We start by asking an interesting yet challenging question, "If a large proportion (e.g., more than 90% as shown in Fig. 1 ) of the face/sketch is missing, can a realistic whole face sketch/image still be estimated?" Existing face completion and generation methods either do not conduct domain transfer learning or can not handle large missing area. For example, the inpainting approach tends to blur the generated region when the missing area is large (i.e., more than 50%). In this paper, we exploit the potential of deep learning networks in filling large missing region (e.g., as high as 95% missing) and generating realistic faces with high-fidelity in cross domains. We propose the recursive generation by bidirectional transformation networks (r-BTN) that recursively generates a whole face/sketch from a small sketch/face patch. The large missing area and the cross domain challenge make it difficult to generate satis-factory results using a unidirectional cross-domain learning structure. On the other hand, a forward and backward bidirectional learning between the face and sketch domains would enable recursive estimation of the missing region in an incremental manner ( Fig. 1) and yield appealing results. r-BTN also adopts an adversarial constraint to encourage the generation of realistic faces/sketches. Extensive experiments have been conducted to demonstrate the superior performance from r-BTN as compared to existing potential solutions.
For instance, Song et al. REF present the Bidirectional Transformation Network (BTN), which generates a whole face/sketch recursively by using a small number of facial patches.
Recursive Cross-Domain Face/Sketch Generation from Limited Facial Parts
ArXiv
ArXiv
Computer Science