  We consider the novel task of learning disentangled representations of object
shape and appearance across multiple domains (e.g., dogs and cars). The goal is
to learn a generative model that learns an intermediate distribution, which
borrows a subset of properties from each domain, enabling the generation of
images that did not exist in any domain exclusively. This challenging problem
requires an accurate disentanglement of object shape, appearance, and
background from each domain, so that the appearance and shape factors from the
two domains can be interchanged. We augment an existing approach that can
disentangle factors within a single domain but struggles to do so across
domains. Our key technical contribution is to represent object appearance with
a differentiable histogram of visual features, and to optimize the generator so
that two images with the same latent appearance factor but different latent
shape factors produce similar histograms. On multiple multi-domain datasets, we
demonstrate our method leads to accurate and consistent appearance and shape
transfer across domains.
