Integrating Categorical Semantics into Unsupervised Domain Translation
Authors: Samuel Lavoie-Marchildon, Faruk Ahmed, Aaron Courville
Venue: ICLR 2021 Poster
Keywords: Unsupervised Domain Translation, Unsupervised Learning, Image-to-Image Translation, Deep Learning, Representation Learning
Abstract: While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical semantic features could broaden its applicability. In particular, we demonstrate that categorical semantics improves the translation between perceptually different domains sharing multiple object categories. We propose a method to learn, in an unsupervised manner, categorical semantic features (such as object labels) that are invariant of the source and target domains. We show that conditioning the style encoder of unsupervised domain translation methods on the learned categorical semantics leads to a translation preserving the digits on MNIST$\leftrightarrow$SVHN and to a more realistic stylization on Sketches$\to$Reals.