Title: LEARNING DISENTANGLED REPRESENTATIONS WITH REFERENCE-BASED VARIATIONAL AUTOENCODERS

Abstract: Learning disentangled representations from visual data, where different high-level generative factors are independently encoded, is of importance for many computer vision tasks. Supervised approaches, however, require a significant annotation effort in order to label the factors of interest in a training set. To alleviate the annotation cost, we introduce a learning setting which we refer to as "referencebased disentangling". Given a pool of unlabelled images, the goal is to learn a representation where a set of target factors are disentangled from others. The only supervision comes from an auxiliary "reference set" that contains images where the factors of interest are constant. In order to address this problem, we propose reference-based variational autoencoders, a novel deep generative model designed to exploit the weak supervisory signal provided by the reference set.
During training, we use the variational inference framework where adversarial learning is used to minimize the objective function. By addressing tasks such as feature learning, conditional image generation or attribute transfer, we validate the ability of the proposed model to learn disentangled representations from minimal supervision.

1. INTRODUCTION:

Natural images can be considered the result of a generative process involving many factors of variation. For instance, the appearance of a face is determined by the interaction between many latent variables including the pose, the illumination, the subject's age and expression. Given that the interaction between these underlying explanatory factors is usually very complex, inverting the generative process is extremely challenging. From this perspective, learning disentangled representations where different high-level generative factors are independently encoded, can be considered one of the most relevant problems in computer vision (Bengio et al., 2013) . For instance, these representations can be applied to complex classification tasks given that features correlated with the image labels can be identified. We can find another example in conditional image generation (van den Oord et al., 2016; Yan et al., 2016) , where disentangled representations allow to manipulate desired attributes in synthesized images.

Motivation: Latent variable models have a long history as tools to learn abstract data representations. The main idea behind these methods is to define a probabilistic model relating the observations, e.g. images, with a set of latent variables encoding the factors of variation underlying the data. In recent years, Variational autoencoders (VAEs) (Kingma & Welling, 2014) have emerged as a powerful latent variable model coupling deep learning with variational inference. However, even though VAEs are able to build generic data representations, they are trained in an unsupervised manner and, thus, they lack a mechanism to impose specific high-level semantics on the latent variables. In order to address this limitation, different semi-supervised autoencoders have been proposed (Rifai et al., 2012; Kingma et al., 2014) . In these approaches, a subset of the latent factors are assumed to be manually labelled in a training set. These annotations provide supervision to the model, and allow to disentangle the labelled variables from the remaining generative factors. The main drawback of this strategy is that it require a significant effort in order to annotate the targeted factors in a dataset.

In this context, we are motivated by the following question: "Is it possible to disentangle specific factors of variation with minimal supervision?". Contributions. In this paper, we introduce "reference-based disentangling". A learning setting in which, given a training set of unlabelled images, the goal is to learn a representation where a specific set of generative factors are disentangled from the rest. For that purpose, supervision comes in the form of an auxiliary "reference set" containing images where the factors of interest are constant. See Fig. 1 for illustrative examples. Different from a semi-supervised scenario, explicit labels are not available during training. In contrast, reference-based disentangling is a weakly-supervised task, where the reference set only provides implicit information about the generative factors that we aim to disentangle. Note that a collection of reference images is generally easier to obtain compared to explicitly annotations of target factors. For instance, if we are interested in disentangling facial gesture information from faces, we would need to annotate images according to different expression classes. Despite the fact that this is feasible for a reduced number of basic gestures, naturalistic expressions depend on a combination of a large number of facial muscle activations with their corresponding intensities (Ekman & Rosenberg, 1997) . As a consequence, annotating facial expression datasets typically requires a very expensive process. By contrast, collecting a large collection of neutral faces is much easier and can be carried out by non-expert annotators.

The main contributions of our paper are summarized as follows:

• We propose reference-based variational autoencoders (Rb-VAEs). Different from standard unsupervised VAEs, our model is able to impose high-level semantics into the latent variables by exploiting the weak supervision provided by the reference set.

• We identify critical limitations of the VAE objective function when used to train our model. For this reason, we propose an alternative training procedure based on recently introduced ideas in the context of variational inference and adversarial learning.

• By learning disentangled representations from minimal supervision, we show how the proposed framework is able to naturally address different tasks such as feature learning, conditional image generation, and attribute transfer.

2. RELATED WORK:

Deep Generative Models have been extensively explored to model visual and other types of data during recent years. Variational autoencoders (Kingma & Welling, 2014) and generative adversarial networks (GANs) (Goodfellow et al., 2014) have emerged as two of the most effective frameworks.

VAEs use variational inference in order to learn an encoder network that maps images to a posterior distribution over latent variables. Similarly, a decoder network is learned that produces the conditional distribution on images given the latent variables. GANs are also composed of two differentiable networks. The generator network synthesizes images from latent variables, similar to the VAE decoder. The discriminator network discriminates between real training images and generated synthetic images. During training, GANs employ an adversarial learning procedure which allows to simultaneously optimize the discriminator and generator parameters. Even though GANs have been shown to generate more realistic samples than VAEs, their main limitation is the lack of an inference mechanism able to map images into their corresponding latent variables. In order to address this drawback, there have been several attempts to combine ideas from VAEs and GANs (Larsen et al., 2015; Dumoulin et al., 2017; Donahue et al., 2017) . Interestingly, it has been shown that adversarial learning can be used to minimize the variational objective function of VAEs (Makhzani et al., 2016; Huszár, 2017) . Inspired by this observation, various methods such as adversarial vari-ational Bayes (Mescheder et al., 2017) , α-GAN (Rosca et al., 2017) , and symmetric-VAE (sVAE) (Pu et al., 2018) have incorporated adversarial learning into the VAE framework.

Different from this prior work, our Rb-VAE model is a deep generative model specifically designed to solve the reference-based disentangling problem. During training, adversarial learning is used in order to minimize a variational objective function inspired by the one employed in sVAE (Pu et al., 2018) . Although sVAE was originally motivated by the limitations of the maximum likelihood criterion used in unsupervised VAE, we show how the variational formulation of sVAE offers specific advantages in the context of our proposed model.

Learning disentangled representations is a long standing problem in machine learning and computer vision (Bengio et al., 2013) . In the literature, we can differentiate three main paradigms to address it: unsupervised, supervised, and weakly-supervised. Unsupervised models are trained without specific information about the generative factors of interest (Desjardins et al., 2012; Chen et al., 2016) . To address this task, the most common approach consists in imposing different constraints on the latent representation. For instance, unsupervised VAEs typically define the prior over the latent variables with a fully-factorized Gaussian distribution. Given that high-level generative factors are typically independent, this prior encourage their disentanglement in different dimensions of the latent representation. Based on this observation, different approaches such as β-VAE (Higgins et al., 2017) , DIP-VAE (Kumar et al., 2018) , FactorVAE Kim & Mnih (2018) or β-TCVAE (Chen et al., 2018) have explored more sophisticated regularization mechanisms over the distribution of inferred latent variables. Although unsupervised approaches are able to identify simple explanatory variables of visual data, these methods do not allow latent variables to model specific high-level factors of variation.

A straight-forward approach to overcome this limitation is to use a fully-supervised strategy. In this scenario, models are learned by using a training set where target factors of interest are explicitly labelled. Following this paradigm, we can find different semi-supervised (Kingma et al., 2014; Narayanaswamy et al., 2017), and conditional (Yan et al., 2016; Pu et al., 2016) variants of autoencoders. In spite of the effectiveness of supervised approaches in different applications (Ma et al., 2017; Pu et al., 2016; Tran et al., 2017) , obtaining explicit labels is not feasible in scenarios where we aim to disentangle a large number of factors or their annotation is difficult.

An intermediate solution between unsupervised and fully-supervised methods are weaklysupervised approaches. In this case, only implicit information about factors of variation is provided during training. Several works have explored this strategy by using different forms of weaksupervision such as: temporal coherence in sequential data (Hsu et al., 2017; Denton et al., 2017; Villegas et al., 2017) , pairing relations between images sharing the same generative factors (Mathieu et al., 2016; Donahue et al., 2018) , grouping information for training samples (Bouchacourt et al., 2017) , or partial knowledge about the rendering process in computer graphics (Yang et al., 2015; Kulkarni et al., 2015) .

Other than previous approaches relying on other forms of weak supervision, our proposed method addresses the reference-based disentangling problem. In this scenario, the challenge is to exploit the implicit information provided by a training set of images where the generative factors of interest are constant, e.g. a set of faces with neutral expression. Note that for this problem, supervised approaches are not a directly applicable since explicit labels are not available.

3. PRELIMINARIES: VARIATIONAL AUTOENCODERS:

Variational autoencoders (VAEs) (Kingma & Welling, 2014) are generative models defining a joint distribution p θ (x, z) = p θ (x|z)p(z), where x is an observation, e.g. an image, and z is a latent variable with a simple prior p(z), e.g. a Gaussian with zero mean and identity covariance matrix. Moreover, p θ (x|z) is typically modeled as a factored Gaussian, whose mean and diagonal covariance matrix are given by a non-linear function of z, implemented as a deep (convolutional) neural network. The latter network is referred to as the "decoder", or "generator".

Given a training set of i.i.d. samples from an unknown data distribution p(x), the goal of VAEs is two-fold: (i) To learn the optimal generator parameters θ so that is possible to synthesize samples following p(x) from the prior p(z), (ii): To approximate the intractable posterior p θ (z|x) in order to infer latent variables z from a given observation x. For this purpose, VAEs define a variational distribution q ψ (x, z) = q ψ (z|x)p(x). The approximate posterior q ψ (z|x) is defined as another factored Gaussian, whose mean and diagonal covariance matrix are given as the output of an "encoder" or "inference" network with parameters ψ and taking x as an input. During training, the parameters of the decoder and encoder are learned jointly by solving the following optimization problem:

EQUATION

which is equivalent to the minimization of the KL divergence between q ψ (x, z) and p θ (x, z). Note that the first term can be interpreted as a regularization mechanism encouraging the distribution q ψ (z|x) to be similar to the prior p(z). The second term is known as the reconstruction error, measuring the negative log-likelihood of a generated sample x from its inferred latent variables q ψ (z|x). The described minimization problem is usually solved by using stochastic gradient descent (SGD) where p(x) is approximated by the training set. The "re-parametrization trick" (Rezende et al., 2014) is used to enable gradient back-propagation across samples from q φ (z|x).

4. REFERENCE-BASED DISENTANGLED REPRESENTATIONS:

Consider a training set of unlabelled images (e.g. human faces) x ∈ R W ×H×3 sampled from a given distribution p u (x). Our goal is to learn a latent variable model defining a joint distribution over x and latent variables e ∈ R De and z ∈ R Dz . Whereas e is expected to encode information about a set of generative factors of interest, e.g. facial expressions, z should model the remaining factors of variation underlying the images, e.g. pose, illumination, age, etc. From now on, we will refer to e and z as the "target" and "common factors", respectively. In order to disentangle them, we are provided with an additional set of reference images sampled from p r (x), representing a distribution over x where target factors e are constant e.g. neutral faces. Given p r (x) and p u (x), we define an auxiliary binary variable y ∈ {0, 1} indicating whether an image x has been sampled from the unlabelled or reference distributions, i.e. p(x|y = 0) = p u (x) and p(x|y = 1) = p r (x).

In reference-based disentangling we aim to exploit the weak-supervision provided by y in order to effectively disentangle target and common factors.

4.1. REFERENCE-BASED VARIATIONAL AUTOENCODERS:

In this section, we present our reference-based variational autoencoder (Rb-VAE): a deep latent variable model that defines a joint distribution p θ (x, z, e, y) as

EQUATION

where conditional dependencies are designed to explicitly address the reference-based disentangling problem. See Figure 2 for a schematic illustration of our model. We define p θ (x|z, e) = L(x|G θ (z, e), λ), where G θ (z, e) is a decoder network that maps a pair of latent variables (z, e) to an image defining the mean of a Laplace distribution L with fixed scale parameter λ. Note that we use a Laplace distribution instead of the Gaussian usually employed in the VAEs. The reason is that the negative log-likelihood is equivalent to the 1 -loss which is known to encourage sharper image reconstructions with better visual quality.

To reflect the assumption of constant target factors across reference images, we define the conditional distribution over e given y = 1 as a delta peak centered on a learned vector e r ∈ R De , i.e. p(e|y = 1) = δ(e -e r ). In contrast, for y = 0, the conditional distribution is set to a unit Gaussian: p(e|y = 0) = N (e|0, I). In the following, we denote p(e|y = 0) = p(e). Contrary to the case of target factors e, the prior over common factors z is equal for reference and unlabelled images, and taken to be a unit Gaussian y for p(z) = N (z|0, I). Finally, we assume a uniform prior over y, i.e. p(y = 0) = p(y = 1) = 1 2 .

4.2. CONVENTIONAL VARIATIONAL LEARNING:

Following the VAE framework described in Section 3, we can define a variational distribution q ψ (z, e, x, y) = q ψ (z|x)q ψ (e|x, y)p(x, y), and learn the model parameters θ by minimizing the KL divergence between q ψ and p θ . In this case, note that the conditionals q ψ (e|x, y) and q ψ (z|x) provide a factored approximation of the intractable posterior p θ (e, z|x, y), allowing to infer target and common factors e and z given an image x. Given a reference image, i.e. with y = 1, the target factors q ψ (e|x, y = 1) are known to be equal to the reference value e r . On the other hand, given an non-reference image, i.e. with y = 0, we define the approximate posterior q ψ (e|x, y = 0) = N (e|E µ (x), E σ (x)), where the means and diagonal covariance matrices of a conditional Gaussian distribution are given by non-linear functions E µ (x) and E σ (x). Similarly, we use an additional network to model q ψ (z|x) = N (z|Z µ (x), Z σ (x)).

Optimization with SGVB: In Appendix (A) we show that the minimization of the KL divergence between q ψ and p θ can be expressed as: min θ,ψ,e r E p u (x) KL(q ψ (z|x)q ψ (e|x) p(z)p(e)) -E q ψ (z|x)q ψ (e|x) log(p θ (x|z, e))

EQUATION

where the second and fourth terms of the expression correspond to the reconstruction errors for unlabelled and reference images respectively. Note that for reference images, no inference over target factors e is needed. Instead, the generator reconstructs them using the optimized parameter e r . Similar to standard VAEs, the remaining terms consist of KL divergences between approximate posteriors and priors over the latent variables. The minimization problem defined in Eq. ( 3) can be solved using SGD and the re-parametrization trick in order to back-propagate the gradient when sampling from q ψ (e|x) and q ψ (z|x).

4.3. VARIATIONAL LEARNING WITH SYMMETRIC KL DIVERGENCE:

The main limitation of the variational objective defined in Eq. ( 3) is that it does not guarantee that common and target factors will be effectively disentangled in z and e respectively. In order to understand this phenomenon, it is necessary to analyze the role of the conditional prior p(e|y) in Rb-VAEs. By defining p(e|y = 1) as a delta function, the model is forced to encode into z all the generative factors of reference images, given that they must be reconstructed via p θ (x|z, e r ) with constant e r . Therefore, p(e|y) is implicitly encouraging q ψ (z|x) to encode common factors present in reference and unlabelled samples. However, this mechanism does not avoid the scenario where target factors are also encoded into latent variables z. More formally, the minimization of (3) does not prevent a degenerate solution p θ (x|z, e) = p θ (x|z), where the inferred latent variables by q ψ (e|x) are uninformative and the decoder ignores them.

In order to address this limitation, we propose to optimize an alternative variational expression inspired by unsupervised Symmetric VAEs (Pu et al., 2018) . Specifically, we add into the minimized objective the reversed KL between q ψ and p θ as:

min θ,ψ

KL(q ψ (z, e, x, y) p θ (x, z, e, y)) + KL(p θ (x, z, e, y) q ψ (z, e, x, y)),

In order to understand why this additional term allows to mitigate the degenerate solution p θ (x|z, e) = p θ (x|z), it is necessary to observe that its minimization is equivalent to:

EQUATION

see Appendix A for a detailed derivation. In the defined expression, the two KL divergences encourage images generated using p(z), p(e) and e r to be similar to samples from the real distributions p r (x) and p u (x). On the other hand, the remaining terms correspond to reconstruction errors over latent variables z, e inferred from generated images drawn from p θ . As a consequence, the minimization of these errors is encouraging the decoder p θ (x|z, e) to generate images x by taking into account latent variables e, since the latter must be reconstructed via q ψ (e|x). In conclusion, the addition of the reversed KL in the objective avoids the degenerate solution that ignores variables e.

Optimization via Adversarial Learning. Given the introduction of the reversed KL divergence, the learning procedure described in Section 4.2 can not be directly applied to the minimization of Eq. ( 4). However, note that the objective function defined in Eq. ( 4) is equivalent to:

EQUATION

corresponding to the minimization of: (i) A difference of two expectations over the log-density ratio of q ψ (e, z|x)p u (x) and p θ (x|e, z)p(z)p(e). (ii) An analogous expression for distributions q ψ (z|x)p r (x) and p θ (x|e r , z)p(z)). By approximating both density ratios with two auxiliary functions:

EQUATION

)

we can express the minimization problem in Eq. ( 6) as:

min θ,ψ

E q ψ (e,z|x)p u (x) d ξ (x, z, e) -E p θ (x|e,z)p(z)p(e) d ξ (x, z, e)

EQUATION

which can be solved with SGD. Concretely, we can evaluate functions d ξ (x, z, e), d γ (x, z) and back-propagate the gradients w.r.t parameters ψ and θ by using the re-parametrization trick over samples of x,e and z.

The main challenge of approximating the log-density ratio using the functions defined in Eq. ( 7), is that parameters ξ and γ are not known and must be also optimized. Fortunately, the log-ratio between two distributions can be estimated by using logistic regression (Bickel et al., 2009) . For instance, given fixed encoder and generator parameters θ and ψ, we can optimize ξ by solving:

EQUATION

where σ(•) refers to the sigmoid function. A similar strategy can be applied to optimize parameters of d γ (x, z). This approach is analogous to adversarial unsupervised methods such as ALI (Dumoulin et al., 2017) , where the function d γ (•) acts as a discriminator trying to distinguish whether pairs of reference images x and latent variables z generated by q ψ and p θ . However, in our case we have an additional discriminator d ξ operating over unlabelled images and its corresponding latent variables z and e. Interestingly, note that d ξ encourages the distribution q ψ (z, e|x) to be similar to the prior p(z, e). Therefore, it forces representations z and e to be conditionally independent. This is especially interesting in our context since we assume that target and common factors are uncorrelated. To conclude, it is also interesting to observe that the discriminator d γ (x, z) is implicitly encouraging latent variables z to encode only information about the common factors. The reason is that samples generated from p θ (x|z, e r )p(z) are forced to be similar to reference images. As a consequence, z can not contain information about target factors, which must be encoded into e.

Using previous definitions, we use an adversarial procedure where model and discriminators parameters (θ,ψ), and (ξ,γ) are simultaneously optimized by minimizing and maximizing Eqs. ( 8) and ( 9) respectively. The algorithm used to process one batch during SGD is shown in Appendix B. In Rb-VAEs, the discriminators d γ (•) and d ξ (•) are also implemented as deep convolutional networks.

Explicit log-likelihood maximization. As shown in Eqs. ( 3) and ( 5), the minimization of the symmetric KL divergence encourages low reconstruction errors for images and inferred latent variables. However, by using the proposed adversarial learning procedure, the minimization of these terms becomes implicit. As shown in (Dumoulin et al., 2017; Donahue et al., 2017) , this can cause original samples to differ substantially from their corresponding reconstructions. In order to address this drawback, we use a similar strategy as (Pu et al., 2018; Li et al., 2017) , and add the reconstruction terms:

E q ψ (e|x)q ψ (e|x)p u (x) log(p θ (x|z, e)) + E q ψ (z|x)p r (x) log(p θ (x|z, e r ))+ E p θ (x|z,e)p(z)p(e)

[log(q ψ (z|x)) + log(q ψ (e|x))] + E p θ (x|z,e r )p(z) log(q ψ (z|x)) (10) into the learning objective, minimizing them together with Eq. ( 8). In preliminary experiments, we found that the explicit addition of these reconstructions terms during learning is important to achieve low reconstruction errors and increase stability during the adversarial training procedure. Figure 5 in the Appendix illustrates all the different losses minimized during learning in sRBD-VAE.

5. EXPERIMENTS:



5.1. DATASETS:

To validate our approach and to compare to existing work, we consider two different reference-based disentangling problems.

Digit style disentangling::

The goal is to model style variations from hand-written digits. Concretely, we consider the digit style as a set of three different properties: scale, width and color. In order to address this task from a reference-based perspective, we consider half of the original training images in the MNIST dataset (LeCun et al., 1998) as our reference distribution (30k examples). The unlabelled set is synthetically generated by applying different transformations over the remaining half of images: (1) Simulation of stroke widths by using a dilation with a given filter size; (2) Digit colorization by multiplying the RGB components of each pixel by a 3D RGB color vector; (3) Size variations by down-scaling the image by a given factor and applying zero-padding to recover the original resolution. We randomly transform each image twice to obtain a total of 60k unsuperivsed images. More details can be found in Appendix C.

Facial expression disentangling::

We consider disentangling of facial expressions by using a reference set of neutral faces. For the unlabelled dataset we use a subset of the AffectNet dataset (Mollahosseini et al., 2017) , which contains a large quantity of facial images. This dataset is especially challenging since images were collected "in the wild" and exhibit a large variety of natural expressions. A subset of the images are annotated according to different basic facial expressions: happiness, sadness, surprise, fear, disgust, anger, and contempt. We use these labels for quantitative evaluation. Given that we found that many neutral images in the original database were not correctly annotated, we collected a separate reference set, see Appendix C for details. The unlabelled and reference sets consist of 150k and 10k images, respectively.

5.2. EXPERIMENTAL SETUP:

Baselines: We compare the two different variants of our proposed method: Rb-VAE, trained using the standard variational objective (Section 4.2), and sRb-VAE, learned by minimizing the symmetric KL divergence (Section 4.3). To demonstrate the advantages of exploiting the weak-supervision provided by reference images, we compare both methods with various state-of-the-art unsupervised approaches based on the VAE framework: β-VAE (Higgins et al., 2017) , β-TCVAE (Chen et al., 2018) sVAE (Pu et al., 2018) , DIP-VAE-I and DIP-VAE-II (Kumar et al., 2018) . Note that β-VAE DIP-VAE and have β-TCVAE been specifically proposed for learning disentangled representations, showing better performance than other unsupervised methods such as InfoGAN (Chen et al., 2016) . On the other hand, sVAE is trained using a similar variational objective as sRb-VAE, and can therefore be considered an unsupervised version of our method. We also use standard VAE ( Welling, 2014) as an additional baseline. Finally, in order to evaluate an alternative approach able to exploit the reference-set, we have implemented (Mathieu et al., 2016) adapting it to our context. Note that the amount of supervision used in the cited method is larger than the available in referencebased disentangling. Concretely, we do not have information about unlabelled samples sharing the same target factors (e.g the same expression). In contrast, this information is only available for reference images. This fact renders the original learning algorithm proposed in (Mathieu et al., 2016) inapplicable. However, we have modified it in order to use only pairing information from reference images by removing the reconstruction losses assuming known pairs for unlabelled samples.

Implementation details::

The different components of our method are implemented as deep neural networks. For this purpose, we have used an architecture based on the main building blocks employed by (Karras et al., 2018) . More concretely, generator networks are implemented as a sequence of convolutions, Leaky-ReLU non-linearities, and nearest-neighbour up-sampling operations. Encoders and discriminators follow a similar architecture, using average pooling for down-sampling. Channel normalization is used for generators and encoders. See Appendix D for more details on the precise architecture. Our code will be made available online upon publication. For a fair comparison, we have developed our own implementation for all the evaluated methods in order to use the same network architectures and hyper-parameters. During optimization, SGD is employed using the Adam optimizer (Kingma & Ba, 2015) (with α = 10 -4 , β 1 = 0.5, β 2 = 0.99, = 10 -8 ), and a batch size of 36 images. For the MNIST and AffectNet databases, the models are learned for 30 and 20 epochs respectively. The number of latent variables for the encoders has been set to 32 for all the experiments and models. The λ parameter in the Laplace distribution is set to 0.01.

5.3. QUANTITATIVE EVALUATION: FEATURE LEARNING:

A common strategy to evaluate the quality of learned representations is to measure the amount of information that they convey about the underlying generative factors (Eastwood & Williams, 2018) . In the context of reference-based disentangling, we are interested in modelling the target factors that are constant in the reference distribution. In this experiment, we use this observation in order to quantitatively evaluate our method. For this purpose, we use the learned representations as feature vectors and train a low-capacity model estimating the target factors involved in each problem. Concretely, in the MNIST dataset we employ a set of linear-regressors predicting the scale, width and color parameters for each digit. To predict the different expression classes in the AffectNet dataset, we use a linear classifier. For evaluation, we split each dataset in three subsets.

The first is used to learn each generative model. Then, the second is used for training the regressors or classifier. The third is used to evaluate the predictions in terms of the mean absolute error and per-class accuracy for the MNIST and AffectNet datasets, respectively. In MNIST, the second and third subset have been randomly generated from the original MNIST test set using the procedure described in Section 5.1 (5k images each). For AffectNet, we randomly select 500 images for each of the seven expressions from the original dataset, yielding 3,500 images per fold.

Table 1 shows the results obtained by the unsupervised models considered1 and the proposed Rb-VAE and sRb-VAE. For Rb-VAE and sRb-VAE, we used the inferred latent variables e as features since the distribution over these variables is expected to encode the information regarding the target factors. For the unsupervised models we use all the latent variables. By analyzing the results, we obtain the following conclusions. First, the unsupervised approach DIP-VAE-I achieves better average results than Rb-VAE for MNIST. Moreover, for facial expression recognition, β-TCVAE achieves comparable or better performance in several cases. This may seem counter-intuitive because, unlike Rb-VAE, DIP-VAE-I is trained without the weak-supervision provided by reference images. However, it confirms our hypothesis that the learning objective of Rb-VAE does not explicitly encourage the disentanglement between target and common factors. In contrast, we can see that sRb-VAE obtains comparable or better results than rest of the methods in most cases. Moreover, it achieves the best average performance in both datasets. This demonstrates that the information provided by the reference distribution is effectively exploited by the symmetric KL objective used to train sRb-VAE. In order to further validate the ability of the proposed method to disentangle the target factors, we have followed the same evaluation protocol for Rb-VAE and sRb-VAE but considering the latent variables z as features. The average performance obtained by Rb-VAE is .349 and .195 for AffectNet and MNIST respectively. On the other hand, sRb-VAE achieves .335 and .189.

Note that for both methods these results are significantly worse compared to using e as a representation in Table 1 . This suggests that latent variables z are mainly modelling the common factors between reference and unlabelled images. The qualitative results presented in the next section confirm this. To conclude, note that sRBD-VAE also obtains better performance than (Mathieu et al., 2016) in both data-sets. So even though this method also uses reference images during training, our results confirms that sRBD-VAE better exploits the weak-supervision provided in reference-based disentangling.

5.4. QUALITATIVE EVALUATION:

In contrast to unsupervised approaches, reference images can be used by our model in order to explicitly encode the target and common factors into two different subsets of latent variables. This directly enables tasks such as conditional image synthesis or attribute transfer. In this section, we illustrate the potential applications of our proposed model in this type of applications.

Conditional image synthesis. The goal is to transform real images by modifying only the target factors e. For instance, given a face of an individual, we aim to generate images of the same subject exhibiting different facial expressions. For this purpose, we use our model in order to infer the common factors z. Then, we sample a vector e ∼ N (0, 1) and use the generator network to obtain a new image from e and z. In Fig. 3 we show examples of samples generated by Rb-VAE and sRb-VAE following this procedure. As we can observe, sRb-VAE generates more convincing results than its non-symmetric counterpart. In the AffectNet database, the amount of variability in Rb-VAE samples is quite low. In contrast, sRb-VAE is able to generate more diverse expressions related with eyes, mouth and eyebrows movements. Looking at the MNIST samples, we can draw similar conclusions. Whereas both methods generate transformations related with the digit color, Rb-VAE does not model scale variations in e, while sRb-VAE does. This observation is coherent with results reported in Table 1 , where Rb-VAE offers a poor estimation of the scale.

Visual attribute transfer. Here we transfer target factors e between a pair of images A and B. For example, given two samples from the MNIST dataset, the goal is to generate a new image with the number in A modified with the style in B. Using our model, this can be easily achieved by synthesizing a new image from latent variables e and z inferred from A and B respectively. Figure 4 shows images generated by sRb-VAE and Rb-VAE in this scenario. In this case, we can draw similar conclusions than the previous experiment. Rb-VAE is not able to swap target factors related with the digit scale in the MNIST dataset, unlike sRb-VAE which better model this variation factor. On the AffectNet images, both methods are able to keep most of the information regarding the identity of the subject, but again Rb-VAE leads to weaker expression changes than sRb-VAE.

These qualitative results demonstrate that the standard variational objective of VAE is sub-optimal to train our model, and that the symmetric KL divergence objective used in sRb-VAE allows to better disentangle the common and target factors.

6. CONCLUSIONS:

In this paper we have introduced reference-based disentangling, a novel weakly supervised learning problem, and proposed reference-based variational autoencoder models to address it. We have shown that the standard variational learning objective used to train VAE can lead to degenerate solutions when it is applied in our setting, and proposed an alternative training strategy that exploits adversarial learning. Comparing the proposed model with previous state-of-the-art unsupervised approaches, we have demonstrated its ability to learn disentangled representations from minimal supervision, and its potential application to tasks such as feature learning, conditional image generation, and attribute transfer.

Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. Attribute2image: Conditional image generation from visual attributes. In ECCV. Springer, 2016.

Jimei Yang, Scott E Reed, Ming-Hsuan Yang, and Honglak Lee. Weakly-supervised disentangling with recurrent transformations for 3d view synthesis. In NIPS, 2015.

APPENDIX A MATHEMATICAL DERIVATIONS:

Equivalence between KL(q ψ (z, e, x, y) p θ (x, z, e, y)) and Eq. ( 3):

y∈[0,1] x,e,z

q ψ (e|x, y)q ψ (z|x)p(x|y)p(y) log q ψ (z, e|x, y)p(x|y)p(y) p θ (x|e, z)p(z)p(e|y)p(y) dxdzde (11) = 1 2 x,e,z q ψ (e|x)q ψ (z|x)p u (x) log q ψ (e|x)q ψ (z|x)p u (x) p θ (x|e, z)p(z)p(e) dxdzde

+ 1 2 x,z q ψ (z|x)p r (x) log q ψ (z|x)p r (x) p θ (x|e r , z)p(z) dxdz (12) = 1 2 E p u (x) E q ψ (e|x)q ψ (z|x) log q ψ (e|x)q ψ (z|x) p(z)p(e) -log(p θ (x|e, z)) -H u (x) + 1 2 E p r (x) E q ψ (z|x) log q ψ (z|x) p(z) -log(p θ (x|e r , z)) -H r (x) (13) = 1 2 E p u (x) KL(q ψ (z|x)q ψ (e|x) p(z)p(e)) -E q ψ (z|x)q ψ (e|x) log(p θ (x|z, e)) + 1 2 E p r (x) KL(q ψ (z|x) p(z)) -E q ψ (z|x) log(p θ (x|z, e r )) -H r (x) -H u (x)

We denote H r (X) and H u (X) as the entropy of the reference and unlabelled distributions p r (x) and p u (x) respectively. Note that they can be ignored during the minimization since are constant w.r.t. parameters θ and ψ. For the second equality, we have used the definitions p(x|y = 0) = p u (x), p(x|y = 1) = p r (x) and assumed p(y = 0) = p(y = 1) = 1 2 . Moreover, we have exploited the fact that q ψ (e|x, y = 1) and p(e|y = 1) are defined as delta functions and, therefore, E p(e|y=1) log( p(e|y=1) q ψ (e|y=1) ) = 0. We denote p(e|y = 0) = p(e) and q ψ (e|x, y = 0) = q ψ (e|x) for the sake of brevity.

Equivalence between KL(p θ (x, z, e, y) q ψ (z, e, x, y)) and the expression in Eq. ( 5) y∈[0,1] x,e,z p θ (x|e, z)p(z)p(e|y)p(y) log p θ (x|e, z)p(z)p(e|y)p(y) q ψ (z, e, x, y)p(x|y)p(y) dxdzde

EQUATION

We have used the same definitions and assumptions previously discussed. Moreover, we denote H(z) and H(e) as the entropy of the priors p(z) and p(e). Again, we can ignore these terms when we are optimizing w.r.t parameters ψ and θ.

Equivalence between the minimization of the symmetric KL divergence in Eq. ( 4) and the expression in Eq. ( 6) KL(q ψ (z, e, x, y) p θ (x, z, e, y)) + KL(p θ (x, z, e, y) q ψ (z, e, x, y)) =

= E q ψ (e|x,y)q ψ (z|x)p(x|y)p(y) log q ψ (e|x, y)q ψ (z|x)p(x|y)p(y) p θ (x|e, z)p(z)p(e|y)p(y)

EQUATION

APPENDIX B PSEUDO-CODE FOR ADVERSARIAL LEARNING PROCEDURE:

Alg. 1 shows pseudo-code for the adversarial learning algorithm described in Sec. 4.3 of the paper. 4: Sample {e1, ..., e M } using q φ (e|x) 5: Sample {z1, ..., z M } using q φ (z|x) 6: Sample {z r 1 , ..., z r M } using q φ (z|x r ) 7: Compute gradient of Eq. ( 8) w.r.t ψ using the reparametrization trick for stochastic variables z, e and z r : We use slightly modified version of the MNIST images: the size is increased to 64×64 pixels and an edge detection procedure is applied to keep only the boundaries of the digit. We obtain the samples in the unlabelled dataset by applying the following transformations over the MNIST images:

EQUATION

1. Width: Generate a random integer in the range {1, . . . , 10} using a uniform distribution. Apply a dilation operation over the image using a squared kernel with pixel-size equal to the generated number. 2. Color: Generate a random 3D vector c ∈ [0, 1] 3 using a uniform distribution. Normalize the resulting vector as ĉ = c/||c|| 1 . Multiply the RGB components of all the pixels in the image by ĉ. 3. Size: Generate a random number in the range [0.5, 1] using a uniform distribution. Downscale the image by a factor equal to the generated number. Apply zero-padding to the resulting image in order to recover the original resolution.

The data and code to generate it will be made available online upon publication.

C.2 AFFECTNET:

Reference set collection. We collected a reference set of face images with neutral expression. We applied specific queries in order to obtain a large amount of faces from image search engines. Then, five different annotators filtered them in order to keep only images showing a neutral expression. This was motivated because we found that many neutral images in the AffectNet dataset Mollahosseini et al. (2017) were not accurate. As detailed in the original paper, each image was only labelled by a single subject. As a consequence, the agreement between annotators was shown to be low for neutral images. In contrast, in our reference-set, each image was annotated in terms of "neutral" / "non-neutral" by two different annotators. In order to ensure a higher label quality compared to 

:

For DIP-VAE-I, DIP-VAE-II, β-VAE and β-TCVAE we tested different regularization parameters in the range [1, 50], and report the best results.

Figures and Tables:

Figure 1: Illustration of different reference-based disentangling problems. (a) Disentangling style from digits. The reference distribution is composed by numbers with a fixed style (b) Disentangling factors of variations related with facial expressions. Reference images correspond to neutral faces.

Figure 2: Overview of the proposed Rb-VAE model. Shaded circles correspond to observed variables in each scenario. (a) Generative process where p θ (x|z, e) maps latent variables z (common factors) and e (target factors) to images x. Reference images are known to be generated by constant e r . (b) The decoder is a deep network G with inputs z,e. (c) Encoders map images x to the corresponding common and target factors z and e respectively. (d) The encoders E and Z are implemented as deep nets modelling two Gaussians distribution over e and z respectively.

Figure 3: Conditional image synthesis for MNIST (top) and AffectNet (bottom) using sRb-VAE and Rb-VAE. Within each column images are generated using the same random target factors e.

Figure 4: Transferring target factors e from image A to an image B on the AffectNet (expression) and MNIST (style) using sRb-VAE and Rb-VAE.

illustrates all the different losses minimized during learning in sRBD-VAE. Algorithm 1 sRb-VAE Advesarial Learning (Batch processing during SGD) 1: /*** Gradient φ ***/ 2: Sample {x1, ..., x M } from p u (x) 3: Sample {x r 1 , ..., x r M } from p r (x)

Figure 5: Illustration of the different losses minimized during learning in sRBD-VAE. (a) Model encoders and generator are trained to minimize the divergence between distributions q ψ (z, e|x)p u (x) and p θ (x|e, z)p(z)p(e). The discriminator d ξ (x, z, e) measures the log-density ratio between both distributions. (b) A similar loss is minimized for reference images using an additional discriminator d γ (x, z) (c,d) Encoder and generator aim to reconstruct unlabelled and reference images from their inferred latent variables. Note that for reference images, no inference over constant factors e is needed. (e,f) For unlabelled and reference images, sRDB-VAE also minimizes the reconstruction error over latent variables of images generated using p(z), p(e) and e r

Figure 8: Qualitative results of sRb-VAE and Rb-VAE applied to conditional image generation. See Sec. (5.5) of the paper for details.

Figure 9: Qualitative results of sRb-VAE and Rb-VAE applied to visual attribute transfer. See Sec. (5.5) of the paper for details.

Figure 10: Images generated by sRB-VAE from random noise over latent variables e and z. Images in the same panel share the same target factors e (expression). Images sharing the same position in each panel are generated from the same common factors z (identity)

Kingma & Prediction of target factors from learned representations. For Rb-VAE and sRb-VAE only the latent variables e are used as feature vectors. All the latent variables are employed for unsupervised models. We report accuracy and Mean Absolute Error as evaluation metrics in the AffectNet and MNIST datasets respectively. Two best methods shown in bold, best result in brackets.

Table content: <table><tr><td/><td/><td/><td/><td colspan="2">AffectNet</td><td/><td/><td/><td/><td/><td colspan="2">MNIST</td><td/><td/></tr><tr><td/><td>Happ</td><td>Sad</td><td>Sur</td><td>Fear</td><td>Disg</td><td>Ang</td><td>Compt</td><td>Avg.</td><td>R</td><td>G</td><td>B</td><td colspan="2">Scale Width</td><td>Avg.</td></tr><tr><td>VAE</td><td>.554</td><td>.279</td><td>.383</td><td>.357</td><td>.256</td><td>.415</td><td>.439</td><td>.383</td><td>.099</td><td>.104</td><td>.101</td><td>[.034]</td><td>.085</td><td>.085</td></tr><tr><td>DIP-VAE-I</td><td>.561</td><td>.269</td><td>[.401]</td><td>.367</td><td>.258</td><td>.397</td><td>.463</td><td>.388</td><td>[.055]</td><td>.064</td><td>.063</td><td>.038</td><td>.100</td><td>.064</td></tr><tr><td>DIP-VAE-II</td><td>.548</td><td>.245</td><td colspan="2">[.401] [.389]</td><td>.268</td><td>.391</td><td>.463</td><td>.386</td><td>.077</td><td>.069</td><td>.076</td><td>.035</td><td>.098</td><td>.071</td></tr><tr><td>β VAE</td><td>.581</td><td>.283</td><td>.373</td><td>.323</td><td>.250</td><td>.415</td><td>.467</td><td>.384</td><td>.093</td><td>.099</td><td>.094</td><td>.039</td><td>.089</td><td>.083</td></tr><tr><td>sVAE</td><td>.583</td><td>.251</td><td>.389</td><td>.349</td><td>.260</td><td>.391</td><td>.469</td><td>.384</td><td>.094</td><td>.092</td><td>.084</td><td>.036</td><td>.104</td><td>.082</td></tr><tr><td>β-TCVAE</td><td>.563</td><td>.277</td><td>.393</td><td>.349</td><td>.256</td><td>[.427]</td><td>.467</td><td>.390</td><td>.098</td><td>.100</td><td>.099</td><td colspan="2">[.034] [.084]</td><td>.083</td></tr><tr><td>[Mathieu et. al, 2016]</td><td>.567</td><td>.388</td><td>.312</td><td>.330</td><td>.295</td><td>.353</td><td>[.512]</td><td>.395</td><td>.116</td><td>.116</td><td>.114</td><td>.039</td><td>.104</td><td>.098</td></tr><tr><td>RBD-VAE</td><td>.536</td><td>.393</td><td>.379</td><td>.311</td><td>.320</td><td>.383</td><td>.421</td><td>392</td><td>.065</td><td>.069</td><td>.062</td><td>.061</td><td>.095</td><td>.070</td></tr><tr><td>sRBD-VAE</td><td colspan="2">[.587] [.405]</td><td>.387</td><td>.327</td><td>[.344]</td><td>.425</td><td>.483</td><td>[.422]</td><td>.057</td><td colspan="2">[.053] [.055]</td><td>.038</td><td>.095</td><td>[.060]</td></tr></table>

Peer review (iclr2019):
 The authors address the problem of representation learning in which data-generative factors of variation are separated, or disentangled, from each other. Pointing out that unsupervised disentangling is hard despite recent breakthroughs, and that supervised disentangling needs a large number of carefully labeled data, they propose a “weakly supervised” approach that does not require explicit factor labels, but instead divides the training data in to two subsets. One set, the “reference set” is known to the learning algorithm to leave a set of generative “target factors” fixed at one specific value per factor, while the other set is known to the learning algorithm to vary across all generative factors. The problem setup posed by the authors is to separate the corresponding two sets of factors into two non-overlapping sets of latents. Pros: To address this problem, the authors propose an architecture that includes a reverse KL-term in the loss, and they show convincingly that this approach is indeed successful in separating the two sets of generative factors from each other. This is demonstrated in two different ways. First, quantitatively on an a modified MNIST dataset, showing that the information about the target factors is indeed (mostly) in the set of latents that are meant to capture them. Second, qualitatively on the modified MNIST and on a further dataset, AffectNet, which has been carefully curated by the authors to improve the quality of the reference set. The qualitative results are impressive and show that this approach can be used to transfer the target factors from one image, onto another image. Technically, this work combines and extends a set of interesting techniques into a novel framework, applied to a new way of disentangling two sets of factors of variation with a VAE approach. Cons: The problem that this work solves seems somewhat artificial, and the training data, while less burdensome than having explicit labels, is still difficult to obtain in practice. More importantly, though, both the title and the start of the both the abstract and the introduction are somewhat misleading. That’s because this work does not actually address disentangling in the sense of “Learning disentangled representations from visual data, where high-level generative factors correspond to independent dimensions of feature vectors…” What it really addresses is separating two sets of factors into different parts of the representation, within each of which the factors can be, are very likely are, entangled with each other. Related to the point that this work is not really about disentangling, the quantitative comparisons with completely unsupervised baselines are not really that meaningful, at least not in terms of what this work sets out to do. All it shows is whether information about the target factors is easily (linearly) decodable from the latents, which, while related to disentangling, says little about the quality of it. On the positive side, this kind of quantitative comparison (where the authors approach has to show that the information exists in the correct part of the space) is not pitted unfairly against the unsupervised baselines. === Update: The authors have made a good effort to address the concerns raised, and I believe the paper should be accepted in its current form. I have increased my rating from 6 to 7, accordingly.