  Image translation methods typically aim to manipulate a set of labeled
attributes (given as supervision at training time e.g. domain label) while
leaving the unlabeled attributes intact. Current methods achieve either: (i)
disentanglement, which exhibits low visual fidelity and can only be satisfied
where the attributes are perfectly uncorrelated. (ii) visually-plausible
translations, which are clearly not disentangled. In this work, we propose
OverLORD, a single framework for disentangling labeled and unlabeled attributes
as well as synthesizing high-fidelity images, which is composed of two stages;
(i) Disentanglement: Learning disentangled representations with latent
optimization. Differently from previous approaches, we do not rely on
adversarial training or any architectural biases. (ii) Synthesis: Training
feed-forward encoders for inferring the learned attributes and tuning the
generator in an adversarial manner to increase the perceptual quality. When the
labeled and unlabeled attributes are correlated, we model an additional
representation that accounts for the correlated attributes and improves
disentanglement. We highlight that our flexible framework covers multiple image
translation settings e.g. attribute manipulation, pose-appearance translation,
segmentation-guided synthesis and shape-texture transfer. In an extensive
evaluation, we present significantly better disentanglement with higher
translation quality and greater output diversity than state-of-the-art methods.
