: We propose a geometry-aware framework for unsupervised image-to-image translation, which is robust to arbitrary shape variations between domains. We show the results of both near-rigid and non-rigid objects. (left) Cow and cheetah rendered from CAD models. (center) Cat and human face from in-the-wild datasets. (right) Horse and giraffe from Flickr. Unsupervised image-to-image translation aims at learning a mapping between two visual domains. However, learning a translation across large geometry variations always ends up with failure. In this work, we present a novel disentangle-and-translate framework to tackle the complex objects image-to-image translation task. Instead of learning the mapping on the image space directly, we disentangle image space into a Cartesian product of the appearance and the geometry latent spaces. Specifically, we first introduce a geometry prior loss and a conditional VAE loss to encourage the network to learn independent but complementary representations. The translation is then built on appearance and geometry space separately. Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks. In addition, by taking different exemplars as the appearance references, our method also supports multimodal translation. Project page: https://wywu.github. io/projects/TGaGa/TGaGa.html
Wu et al. REF propose disentangling the space of images into a Cartesian product of appearance and geometry latent spaces to allow for a geometry-aware translation.
TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Computer Science