L EARNING D ISENTANGLED R EPRESENTATIONS FOR I MAGE T RANSLATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent approaches for unsupervised image translation are strongly reliant on generative adversarial training and architectural locality constraints.
Despite their appealing results, it can be easily observed that the learned class and content representations are entangled which often hurts the translation performance.
To this end, we propose OverLORD, for learning disentangled representations for the image class and attributes, utilizing latent optimization and carefully designed content and style bottlenecks.
We further argue that the commonly used adversarial optimization can be decoupled from representation disentanglement and be applied at a later stage of the training to increase the perceptual quality of the generated images.
Based on these principles, our model learns signiﬁcantly more disentangled representations and achieves higher translation quality and greater output diversity than state-of-the-art methods.
I NTRODUCTION
Learning disentangled representations from a set of observations is a fundamental problem in machine learning.
Such representations can facilitate generalization to downstream discriminative and generative tasks as well as improving interpretability (Hsu et al., 2017), reasoning (van Steenkiste et al., 2019) and fairness (Creager et al., 2019).
Recent advances have contributed to various tasks such as novel image synthesis (Zhu et al., 2018) and person re-identiﬁcation (Eom & Ham, 2019).
Image translation is an extensively researched task that beneﬁts from disentanglement.
Its goal is to generate an analogous image in a target domain (e.g. cats) given an input image in a source domain (e.g. dogs).
Although this task is generally poorly speciﬁed, it is often satisﬁed under the assumption that images in different domains share common attributes (e.g. head pose) which can be transferred during translation - we name those content .
In many cases, the class (domain) and common attributes do not uniquely specify the target image e.g. there are many dog breeds with the same head pose.
This multi-modal translation motivates the speciﬁcation of the particular classspeciﬁc attributes that we wish the target image to have - we name those style .
The ability to transfer the content of a source image to a target class and style has been the objective of several methods e.g. MUNIT (Huang et al., 2018), FUNIT (Liu et al., 2019) and StartGAN-v2 (Choi et al., 2020).
Unfortunately, we show that despite their visually pleasing results, the translated images still retain many class-speciﬁc attributes of the original image resulting in limited translation quality.
For example, when translating dogs to wild animals, current methods are prone to transfer facial shapes which are unique to dogs and should not be transferred precisely to wild animals.
As demonstrated in Fig.
1, our model transfers the semantic head pose more reliably.
In this work, we analyze the class-supervised setting and present a principled objective for disentangling image class and attributes.
We explain why LORD, introduced by (Gabbay & Hoshen, 2020), cannot be applied for multi-modal translation.
We then show that introducing an additional style representation overcomes this issue and propose a practical method for high-ﬁdelity image translation by learning disentangled representations.
Our method achieves this in two stages; i)
Disentanglement: disentangled representation learning in a non-adversarial framework, leveraging latent optimization and well-motivated content and style bottlenecks.
ii)
Synthesis: the disentangled representations learned in the previous stage are used to ”supervise” a synthesis network that generalizes to unseen images and classes.
As synthesis network training is well-conditioned, we can effectively incorporate a GAN loss resulting in a high-ﬁdelity image translation model.
Our approach illustrates that adversarial optimization, which is typically used for domain translation,
[Figure]
is not necessary for disentanglement, and its main utility lies in generating perceptually pleasing images.
Our model learns disentangled representations and achieves better translation quality and output diversity than current methods.
Our contributions are: i) Introducing a non-adversarial disentanglement method that enables multimodal solutions.
ii) Learning statistically disentangled representations.
iii)
Extending domain translation methods to cases with many ( 10 k ) domains.
iv) State-of-the-art results in image-translation.
1. R ELATED W ORK
Image Translation Translating the content of images across different domains has attracted much attention.
In the unsupervised setting, CycleGAN (Zhu et al., 2017) introduces a cycle consistency loss to encourage translated images preserves the domain-invariant attributes (e.g. pose) of the source image.
MUNIT (Huang et al., 2018) recognized that a given content image could be transferred to many different styles (e.g. colors and textures) in a target domain and extends UNIT (Huang & Belongie, 2017) to learn multi-modal mappings by learning style representations.
DRIT (Lee et al., 2018) tackles the same setting using an adversarial constraint at the representation level.
MSGAN (Mao et al., 2019) added a regularization term to prevent mode collapse.
StarGAN-v (Choi et al., 2020) and DMIT (Yu et al., 2019) extend previous frameworks to translation across more than two domains.
FUNIT (Liu et al., 2019) further allows translation to novel domains.
Class-Supervised Disentanglement
In this parallel line of work, the goal is to anchor the semantics of all the images within each class into a separate class representation while modeling all the remaining class-independent attributes by a content representation.
Several methods encourage disentanglement by adversarial constraints (Denton et al., 2017; Szab ´ o et al., 2018; Mathieu et al., 2016) while other rely on cycle consistency (Harsh Jha et al., 2018) or group accumulation (Bouchacourt et al., 2018).
LORD (Gabbay & Hoshen, 2020) takes a non-adversarial approach and trains a generative model while directly optimizing over class and content codes.
Most works in this area demonstrate domain translation results on simple datasets but not in the multi-modal (many-tomany) settings.
Moreover, their focus is to achieve disentanglement at the representation level rather than designing architectures for high-resolution image translation resulting in weak performance on competitive benchmarks.
In this work, we draw inspiration from LORD in relying on the inductive bias conferred by latent optimization to learn a disentangled content representation.
In contrast to
LORD, we tackle the multi-modal image translation setting by modeling style.
Moreover, we add an adversarial term in the synthesis network that increases the image quality and resolution.
B ACKGROUND : R EPRESENTATION L EARNING IN I MAGE -T RANSLATION
Image translation takes as input a set of N images and corresponding class labels ( x 1 , y 1 ) , ( x 2 , y 2 ) , ..., ( x N , y N ) .
Let us assume that an image x
i is fully speciﬁed by its class y i and attributes a i .
As a motivational example, let us consider the images x i to be of animals, and the class label y
i denotes the species.
Attributes a i , may include attributes a c i common to all classes such as head angle, and class-speciﬁc attributes a si such as dog or cat breed.
An unknown function
G ∗ maps y i , a ci and a si to the image x i :
[Equation]
The goal of image translation is to replace the common attributes a ci of target x
i by a cj of source x j :
[Equation]
The main challenge is that during training, only the class y i of each image is given, but the attributes a ci and a si are unknown.
We deﬁne three representations corresponding to each of the physical properties above - the class embedding e y i represents the class y i , the content code c i represents the common attributes a ci and the style code
s i represents the class-speciﬁc attributes a si .
The computational task is to learn the representations e y i , s i , c
i such that they faithfully encode their corresponding physical properties of image x i and are sufﬁcient to reconstruct the image using a generator G :
[Equation]
This constraint however is insufﬁcient to specify the representations uniquely.
A popular constraint, utilizes the fact that common attributes a c i are independent of the class y i .
In the motivating animals example, we assume that head pose acts similarly on cats or dogs.
It therefore requires independence between the content code c i and class y i .
Adversarial methods: Adversarial methods implement this constraint by training a domain confusion discriminator on the translated image x ij =
G ( e y i , s i , c j ) .
The discriminator attempts to distinguish translated images from original images of the target class.
Unfortunately, we empirically show that state-of-the-art methods do not learn content codes
c i that are disentangled from the class
y i .
The consequence is that c i transfers not only common attributes but also class-dependent attributes.
We hypothesize that this failure is due to the challenging adversarial optimization.
In order to constrain the style s i , current methods (e.g. StarGAN-v2, FUNIT) rely on localitypreserving architectures that bias the content code c i to represent the structure, while the style operates in a global manner per channel and is applied at a relatively deep level within the generator (typically from 16 × 16 spatial resolution).
Unfortunately, such style is unable to model class-speciﬁc attributes a si of spatial nature e.g. the spatial facial shape of different dog breeds.
Non-adversarial methods: LORD is a non-adversarial approach for disentangled representation learning.
It assumes that all attributes are common to all classes
I ( a c ; y )
=
and H ( a s ) =- where I denotes mutual information and H denotes entropy.
It therefore learns only class and content representations e y i
and c
i but not style codes ( s i = 0 ).
In order to learn disentangled class and content representations it uses a simple but effective constraint of minimal information in the content codes c i .
After learned disentangled representations e y i and
c i , LORD learns a synthesis network which generalizes to unseen images and classes.
When the assumptions of LORD are satisﬁed ( H ( a s ) = 0 ), the content codes c
i are indeed shown to recover the information of the common attributes a ci .
However, when class-speciﬁc attributes exist,
LORD does not achieve its goals.
The reason is that the content code c i , contains the information of both the common a ci and class speciﬁc a si attributes ( I ( a s ; c ) > 0 ).
This has two issues: i) as the content code now contains class-speciﬁc information (e.g. cat breed), it is no longer disentangled from the class (we can predict if the content is of a cat); ii) when transferring content codes across classes (e.g. cats to dogs), we are missing the required class speciﬁc information in the target class (i.e. given the class code of e dog and content of a cat c cat , the generator is not provided with the information of what dog breed to generate).
Due to its effective non-adversarial approach, LORD learns disentangled representations and achieves state-of-the-art results on uni-modal synthetic and low-res datasets.
However, due to i) ignoring class-speciﬁc attributes as highlighted above; ii) using a non-adversarial synthesis network, it does not produce high ﬁdelity images and cannot scale to high-resolution.
O VER LORD: P RINCIPLED D ISENTANGLEMENT IN I MAGE T RANSLATION
Based on the above analysis, we present our disentangled image-translation method, OverLORD.
[Figure]
Disentanglement with Latent Optimization
[Figure]
[Figure]
3. L EARNING A DISENTANGLEMENT MODEL
In order to learn disentangled representations for class, content and style, we propose a nonadversarial framework with several principles.
Reconstruction: As the image should be fully speciﬁed by the representations e y i , s i , c
i , we employ a reconstruction term: L rec =
(cid:96) ( G ( e y i , s i , c i ) , x i ) - where (cid:96) is a measure of similarity between images (we use the VGG perceptual loss).
Content bottleneck: In order to constrain the amount of information in each content
variable c i , we parameterize it as a noisy channel consisting of a vector c (cid:48)
i and an additive Normal noise z ∼
N (0 , I ) , c i
=
c (cid:48)
i + z .
Using the Shannon-Hartley theorem, the information capacity of the channel is a function of the signal-to-noise ratio.
For the noisy channel c i , the SNR ratio is given by (cid:107) c (cid:48)
i (cid:107) 2 .
We therefore deﬁne the content bottleneck loss as L cb =
(cid:80) i (cid:107) c (cid:48)
i (cid:107) [Equation]
Style: We train a style encoder E s : X −→ S to infer the class-speciﬁc attributes a s .
To encourage invariance to the common attributes a c , the input image ﬁrst undergoes a random transformation.
This creates a transformed version x transi of the input image that shares the same class speciﬁc attributes a s , but exhibits random common attributes a c (as style should be invariant to these attributes).
The nature of the transformation is setting dependent e.g. if common attributes include spatial locations, then crop and rotation transformations will remove spatial attributes from the transformed image and make the style code s i invariant to them.
[Equation]
Our complete objective can be summarized as follows:
[Equation]
There are several fundamental differences from the cVAE objective: i) the noise variance is a ﬁxed hyper-parameter and is not learned; ii) there is no content encoder, instead the latent content code for each variable is learned directly, in an unamortized way often referred to as latent optimization; iii) the input to the style encoder is a transformed version of the image.
Gabbay & Hoshen (2020) discovered that latent optimization improves disentanglement signiﬁcantly over encoder-based methods.
The difference lies in the initialization as latent optimization starts with no class-content correlation (code initialization is IID) while a content encoder starts with near perfect correlation (class can be predicted from the output of a randomly initialized content encoder).
We further elaborate on latent optimization and its inductive bias in Appendix A.3.
[Figure]
3. G ENERALIZATION AND P ERCEPTUAL Q UALITY
After the disentanglement stage, we possess a content code c i and shared class embedding e
y i for every training image, and also a trained style encoder E s .
In order to generalize to unseen images and novel classes, we follow LORD (Gabbay & Hoshen, 2020) and train a content encoder E c : X −→ C and a class encoder E y : X −→ Y to directly obtain a content code c ∈ C and a class code y ∈ Y :
L gen =
[Equation]
By optimizing the two terms, we ensure that the training set can be reconstructed in an amortized fashion ( L gen ), without violating the disentanglement criteria established in the ﬁrst stage ( L enc ).
Note that the targets for L enc are the ones learned by our own model in the ﬁrst stage.
To increase the visual quality, we jointly train a discriminator D and employ an adversarial loss:
[Equation]
Our objective of the second stage can be summarized as follows (the style encoder is unchanged):
[Equation]
A sketch of the entire method is shown in Fig.
E XPERIMENTS
Our method is evaluated against state-of-the-art methods for multi-modal domain translation and class-supervised disentanglement.
Note that we do not present comparisons to unsupervised disentanglement methods such as betaVAE as they are not competitive on class-content disentanglement with methods that use class-supervision.
Implementation details are provided in Appendix A.1.
[Table]
4. B ASELINES
Multi-modal Domain Translation We compare to StarGAN-v2 and FUNIT which are the stateof-the-art domain translation methods and the established MUNIT, DRIT and MSGAN frameworks (which must be trained multiple times for every possible pair of domains).
As standard LORD does not support multi-modal settings, we use its ImageNet K-MEANS augmented setting.
Class-supervised Disentanglement We compare to LORD and FUNIT as they can handle ﬁnegrained class labels (i.e. 10 k face identities), and generalize to unseen classes.
Single Attribute Manipulation We further assess the performance of our disentanglement approach in the task of translating Males to Females against FaderNetworks (Lample et al., 2017) and mGANprior (Gu et al., 2020) which operates in the latent space of StyleGAN.
4. D ATASETS
AFHQ (Choi et al., 2020) 15 , 000 high quality images categorized into three domains: cat, dog and wildlife.
We follow the protocol used in StarGAN-v2 and use the images at 256 × 256 resolution, holding out 500 images from each domain for testing.
CelebA (Liu et al., 2015) 202,599 images of 10,177 celebrities.
We designate the person identity as class.
We crop the images to 128 × 128 and use 9,177 classes for training and 1,000 for testing.
CelebA-HQ (Karras et al., 2018) 30,000 high quality images from CelebA.
We set the gender as class.
We resize the images to 256 × 256 and leave 1,000 images from each class for testing.
4. E VALUATION P ROTOCOL
We assess the disentanglement at two levels: the learned representations and the generated images.
At the representation level, we follow the protocol in LORD (Gabbay & Hoshen, 2020) and train a two-layer multi-layer perceptron to classify class labels from the learned content codes (lower accuracy indicates better disentanglement).
In CelebA, where annotations of part of the common attributes are available (for evaluation only) such as 68-facial landmark locations, we train a linear regression model to locate the landmarks given the learned class (identity) codes (higher error indicates better disentanglement).
At the image level, we follow StarGAN-v2 and translate all images in the test set to each of the other domains multiple times, borrowing style codes from random reference images in the target domain.
We then train a classiﬁer to classify the original class of the content image from the generated image.
A lower accuracy indicates better disentanglement as the source class does not leak into the translated image.
We also compute FID (Heusel et al., 2017) in [Figure] FUNIT preserves head pose but fails to model facial expression or transfer the exact facial identity. LORD captures the facial identity but generates low ﬁdelity images.
Our method preserves head pose and facial expression well while transferring the identity and generating appealing images.
[Table] a conditional manner to measure the discrepancy between the distribution of images in each target domain and the corresponding translations generated by the models. A lower FID score indicates that the translations are more reliable and better ﬁt to the target domain. FID between real train and test images of the same class forms the optimal score for this metric.
In order to assess the diversity of translation results, we measure the perceptual pairwise distances using LPIPS (Zhang et al., 2018) between all translations of the same input image.
Higher average distances indicate greater diversity in image translation.
In cases where external annotation methods are available (for evaluation only), such as face recognition (Cao et al., 2018) and head pose (Ruiz et al., 2018) and landmark detection for CelebA, we further measure the similarity of the identity of the generated face and the reference, as well as expression (represented by landmarks) and head pose errors.
While we do not have ground truth for translation in CelebA as well, we present a reconstruction error (LPIPS similarity) by inferring the class (identity) representation from one image and reconstructing a second of the same person by extracting the content representation from an image of a different identity which has the closest pose (nearest neighbour in the 68 facial-landmarks space).
For the single attribute manipulation experiment on CelebA-HQ, we measure the accuracy of fooling a target classiﬁer.
4. R ESULTS
We present a quantitative evaluation of multi-modal domain translation on AFHQ in Tab. 1.
Our method outperforms all baselines achieving near perfect disentanglement at the representation level [Figure] I npu t F a d e r
S t y l e
GAN
O u r s [Table] as the classiﬁer could barely guess class labels from content codes by a random chance. Similar results are obtained by trying to classify the original class of the translated image.
We hypothesize that the minor degradation in this metric is due to using a deeper classiﬁer at the image level (VGG-based).
It can be seen that the baselines leak a little bit less information at the output image compared to the representation level, which might indicate that their generator ignores some parts of the representations.
Moreover, we achieve both better visual quality (FID) and higher translation diversity (LPIPS).
As can be seen in the qualitative examples in Fig.
3, StarGAN-v2 and FUNIT leak a signiﬁcant amount of details from the input image which results in unreliable translations [Table]
Our method generates much more disentangled translations and captures the exact style of the reference image.
In order to assess the performance in cases where ﬁne-grained class labels are available and the ability to generalize to unseen classes at inference time is essential, we provide comparison on CelebA with 10K different identities in Tab. 2.
It can be seen that our method achieves better expression and pose transfer while preserving the face identity more faithfully than the baselines.
To validate that content is distributed evenly across identities, we use landmark annotations together with pose-related attributes from CelebA (Open Mouth and Smiling) and train a classiﬁer to infer the identity.
The accuracy of this classiﬁer (0.001) forms the optimal result for the representation metric.
Examples from this experiment are presented in Fig.
We also assess the performance of our method in the single attribute manipulation task of translating males to females on CelebA-HQ in Tab. 3 and Fig.
FaderNetworks makes minor changes to the given face which do not result in reliable translations.
mGANprior modiﬁes the input face signiﬁcantly which reduces the similarity to the original identity, as many subtle facial attributes are lost when inverting real images within the StyleGAN latent space.
Moreover, some of the images were not well reconstructed even though StyleGAN is trained on CelebA-HQ.
Our model achieves near perfect classiﬁcation score and generates visually pleasing results.
More qualitative results are provided in Appendix A.2.
In order to isolate the contribution of our main principles, we provide an assessment of several variants of our method. Style modeling for multi-modal translation We train our model without style modeling (treating content and style as a single representation, both are complementary to the class). As can be seen in
Tab. 1, this strategy violates the disentanglement criteria as class-speciﬁc attributes are now modeled by the content.
As a consequence, the animals are translated in an unreliable and entangled manner between classes, as can be seen in Fig.11 in Appendix A.4.
Adversarial loss for perceptual quality We train our method without the adversarial loss in the synthesis stage.
The results in Tab. 1 suggest that the disentanglement criteria are achieved by our non-adversarial framework, while the additional adversarial loss contributes for increasing the output ﬁdelity (FID). Qualitative evidence for this claim is presented in Fig.11 in Appendix A.4. Limitations There are two main limitations of the proposed method; i)
The disentanglement between content and style (both are complementary to the class) is achieved in an unsupervised manner and therefore relies on an inductive bias.
In our framework it is driven by the design of a set of random transformations to which the style needs to be invariant.
While we show that common settings in image translation can be solved effectively with simple operators, other settings may require more sophisticated transformations.
ii)
Our model is not well optimized for cases in which the object is not the major part of the image e.g. where the background contains other objects or large variation.
We hypothesize that this can be better solved by introducing an unsupervised segmentation objective into the entire framework.
We leave this direction to future research.
C ONCLUSION
We present a principled approach for learning disentangled representations in a non-adversarial framework, leveraging latent optimization and well-motivated content and style bottlenecks.
We show how adversarial optimization can be decoupled from representation disentanglement and be applied only to increase the ﬁdelity of the generated images.
Our proposed model learns disentangled representations and achieves state-of-the-art performance compared to both adversarial and non-adversarial disentanglement methods in different settings of image translation
A A PPENDIX A. I MPLEMENTATION DETAILS
Architectures We borrow the generator and discriminator architectures from StyleGAN2 with two modiﬁcations; (i)
The latent code is adapted to our formulation and forms a concatenation of three separate latent codes: content, class and style.
(ii)
We do not apply any noise injection during training.
A brief summary of the architectures is presented in Tab. 4 and 5 for completeness.
The architectures of the content, class and style encoders is inﬂuenced by StarGAN-v2 (Choi et al., 2020) and presented in Tab. 6.
Note that we do not use any domain-speciﬁc layers and our model is able to generalize to unseen domains at inference time.
For cases where the style is minimal and should mainly model low-level image features (e g. the information that is not described by identity and pose in CelebA) we use a shallow style encoder with a small receptive ﬁeld, as can be seen in Tab.
Optimization In the ﬁrst stage, we optimize over a single content embedding per image (content dim = 64), a single class embedding per class (class dim = 512) and the parameters of the style encoder (style dim = 64) and the generator.
We set the learning rate of the latent codes to 0.01, the learning rate of the generator to 0.001 and the learning rate of the style encoder to 0.0001.
The penalty of the content bottleneck is set to λ cb = 0 .
We train the disentanglement stage forepochs.
For each mini-batch, we update the parameters of the models and the latent codes with a single gradient step each.
In the synthesis stage, we add two encoders to infer the content and class codes learned in the ﬁrst stage directly from an image.
We also optimize a discriminator in an endto-end manner to increase the perceptual ﬁdelity of the images.
This stage is trained for 100 epochs and the learning rate for all the parameters in this stage is set to 0.0001, except for the style encoder whose weights are kept ﬁxed.
Transformations As stated in Sec. 3, the design of the transformation applied on the input to the style encoder is setting dependent.
For AFHQ, in order to encourage the style to be invariant to the head pose of the animal, we randomly ﬂip the image horizontally and crop 0 .
6 to 1 portion of the image.
In CelebA, as we are given ﬁne grained class labels (person identity), the desired style should ideally model only low level features and therefore we opt for a shallow style encoder with a small receptive ﬁeld (5x5) except for the same augmentations as of AFHQ. The low receptive ﬁeld avoids the style from encoding common-attributes as facial expression (which might retain even after the transformation). Baseline models For the evaluation of competing methods, we use the following ofﬁcial publicly available pretrained models: StarGAN-v2 (for AFHQ) and StyleGAN (for mGANprior on CelebAHQ). We train the rest of the baselines using the ofﬁcial repositories of their authors and make an effort to select the best conﬁgurations available for the target resolution (for example, FUNIT trained by us for AFHQ achieves similar results to the public StarGAN-v2 which was known as the SOTA on this benchmark).
[Table]
[Table] Training resources Training takes approximately 3 days for 256x256 resolution on a single NVIDIA RTX-2080 TI
[Table]
A. A DDITIONAL RESULTS
We provide additional qualitative results on AFHQ in Fig.
6,7, on CelebA in Fig.
8, 9 and on CelebAHQ in Fig.
A. L ATENT OPTIMIZATION
In this work
[Table]
we opt for learning the content representation using latent optimization, similarly to
LORD (Gabbay & Hoshen, 2020).
Autoencoders assume a parametric model, usually referred to as the encoder [Table] to compute a vector c i from image x i .
on the other hand, we jointly optimize the inputs and the model parameters.
Since the vector c i is learned directly and is unconstrained by a parametric encoder function, our model can recover all the solutions that could be found by an autoencoder
[Table]
and reach some others.
In order to justify this design choice
[Table]
we validate the observation presented in
LORD and train our disentanglement stage in an amortized fashion using a content encoder.
As can be seen in Fig.
14, amortized training fails to reduce the correlation between the content and class representations.
We have experimented with several content decay factors ( λ cb in Eq. 5).
Although [Table] [Table] the disentanglement improves as λ cb increases, the reconstruction gets worse and the model fails to converge with λ cb > 0 .
. A. V ISUALIZATION OF ABLATION ANALYSIS
Examples from the ablation analysis are provided in Fig. Visualization of the three factors modeled by our method is provided in Fig.
12 and Fig.
A. F AILURE CASES
As our method strongly relies on the style image to infer the class-speciﬁc attributes e.g. identity and facial shape, we have found that when the animal in the style image is present in an extreme pose such as wide open mouth, the style encoder can sometime confuse the extreme pose as a property of the breed and transfer it from the style image.
Examples are provided in Fig.
Style
C o n t e n t
S t a r
GAN - v
F UN
I T
O u r s
S t a r
GAN - v
F UN
I T
O u r s
S t a r
GAN - v
F UN
I T
O u r s
Style
C o n t e n t
S t a r
GAN - v
F UN
I T
O u r s
S t a r
GAN - v
F UN
I T
O u r s
S t a r
GAN - v
F UN
I T
O u r s
[Figure] [Figure]
I npu t
F a d e r N e t w o r k s
S t y l e
GAN
O u r s
[Figure]
I npu t
F a d e r N e t w o r k s
S t y l e
GAN
O u r s
[Figure] [Figure]
C o n t e n t w / o s t
y l e w / o GAN O u r s w / o s t y l e w / o GAN
O u r s w / o s t y l e w / o
GAN O u r s [Figure] Borrowing the style from a reference image allows for speciﬁcation of the exact target breed.
Note that the class and style are correlated and therefore transferring style across classes can generate unreliable translations (e.g. ﬁrst two images in ”To Wild”.) [Figure] [Figure] [Figure]