DAVA: D ISENTANGLING A DVERSARIAL V ARIATIONAL A UTOENCODER
Anonymous authors
Paper under double-blind review
A BSTRACT
The use of well-disentangled representations poses many advantages for downstream tasks, e.g. an increased sample efficiency, or better interpretability.
The quality of disentangled interpretations is however often highly dependent of the choice of dataset-specific hyperparameters, most notably the regularization strength.
To address the issue, we introduce DAVA, a novel training procedure for variational auto-encoders.
DAVA completely alleviates the issue of hyperparameter selection.
We compare DAVA against models with optimal hyperparameters.
Without any hyperparameter tuning, DAVA is competitive across a diverse range of commonly used datasets.
Underlying DAVA, we discover a necessary condition for unsupervised disentanglement we call PIPE .
We demonstrate the ability of PIPE to positively predict performance of downstream models in abstract reasoning.
We also exhaustively examine correlations with existing supervised and unsupervised metrics.
I NTRODUCTION
Real-world data is generally highly structured, and full with symmetry and patterns.
This implies that there exists a lower-dimensional set of ground truth factors that is able to explain a significant part of the variations present in real-world data.
The goal of disentanglement learning is to recover these factors, so that changes in a single ground truth factor are only reflected in a single latent dimension of a model (see Figure 1 for an example).
Such an abstraction allows for more efficient reasoning (Van Steenkiste et al., 2019) and improved interpretability (Higgins et al., 2017a).
It further shows positive effects on zero-shot domain adaption (Higgins et al., 2017b) and data efficiency (Duan et al., 2020; Schott et al., 2022).
DAVA
visibly
If the generative ground-truth factors are known and labeled data is available, one can train a model in a supervised fashion to extract the ground-truth factors.
What if the generative factors are unknown, but one still wants to profit from the aforementioned benefits for a downstream task?
This can be necessary if the amount of labeled data for the downstream task is limited or training is computationally expensive.
Learning disentangled representations in an unsupervised fashion is generally impossible without the use of some priors (Locatello et al., 2019b).
These priors can be present both implicitly in the model architecture and explicitly in the loss function (Tschannen et al., 2018).
An example of such a prior present in the loss function is low total correlation between latent variables of a model (Chen et al., 2018; Kim & Mnih, 2018).
Reducing the total correlation has been shown to have a positive effect on disentanglement (Locatello et al., 2019b).
Unfortunately, as we show in more detail in this work, how much total correlation should be reduced to achieve good disentanglement is highly dataset-specific.
The optimal hyperparameter setting for one dataset might yield bad results on another dataset.
To optimize regularization strength, we need a way to evaluate disentanglement quality.
So how can we identify well disentangled representations?
The evaluation of representation quality, even given labeled data, is no easy task.
Perhaps as an instance of unfortunate nomenclature, the often-used term “ground-truth factor” implies the existence of a canonical set of orthogonal factors.
However, often there exist multiple equally valid sets of ground truth factors, for example affine transformations of coordinate axes spanning a space, different color representations, or various levels of abstraction for group properties.
This poses a problem for supervised disentanglement metrics, since they fix the ground truth factors for evaluating a representation, judging the models too harshly if they learned another equally valid representation.
Furthermore, acquiring labeled data in a practical setting is usually a costly endeavor.
The previously stated reasons hinder the usability of supervised metrics for model selection.
In this work, we overcome these limitations both for learning and evaluating disentangled representations.
Our improvements are based on the following idea:
We define two distributions that can be generated by a VAE.
Quantifying the distance between these two distributions yields a disentanglement metric independent of the specific choice of ground truth factors and reconstruction quality.
The further away these two distributions are, the less disentangled is the VAEs latent space.
We show that the similarity of the two distributions is a necessary condition for disentanglement.
Further, we can use this property at training time by introducing an adversarial loss into classical training of
VAEs.
For this, we introduce a discriminator network into training and use the decoder of the VAE as generator.
During training, we control the weight of the adversarial loss.
We correspondingly adjust the capacity of the latent space information bottleneck, inspired by (Burgess et al., 2017).
By doing so, we allow the model to increase the complexity of its representation as long as it is able to disentangle.
This dynamic training procedure resolves the issue of dataset-specific hyperparameters and allows our approach to reach competitive disentanglement on a diverse set of commonly used datasets without hyperparameter tuning.
Our contributions are as follows:
• We identify a novel unsupervised aspect of disentanglement called PIPE and demonstrate its usefulness in a metric with correlation to supervised disentanglement metrics as well as a downstream task.
• We propose an adaptive adversarial training procedure (DAVA) for variational auto-encoders, resolving the common issue of disentanglement performance being largely dependent on dataset-specific regularization strength.
• We provide extensive evaluations on different commonly used disentanglement datasets to support our claims.
R ELATED W ORK 2. M ODEL A RCHITECTURES
The β -VAE by Higgins et al. (2017a) is a cornerstone model architecture for disentanglement learning.
The loss function of the β -VAE, the evidence lower bound (ELBO), consists of a reconstruction term and a KL-divergence term weighed by β , forcing the aggregated posterior latent distribution to closely match the prior distribution.
The KL-divergence term seems to be encouraging disentanglement as shown in (Rolinek et al., 2019).
The β -TCVAE architecture proposed by Chen et al.
(2018) decomposes the KL divergence term of the ELBO further into an index-code mutual information, a total correlation and a dimension-wise KL term.
They are able to show that it is indeed the total correlation that encourages disentanglement and propose a tractable but biased Monte Carlo estimate.
Similar in spirit is the FactorVAE architecture (Kim & Mnih, 2018) using the density ratio trick with an adversarial network to estimate total correlation.
The AnnealedVAE architecture (Burgess et al., 2017) as well as ControlVAE (Shao et al., 2020) build on a different premise, arguing that slowly increasing the information bottleneck capacity of the latent space leads to the model learning new latent dimensions step by step and therefore disentangling them.
We will use a similar but optimized approach for DAVA.
More recent promising approaches are presented by Wei et al.
(2021) using orthogonal Jacobian regularization and by Chen et al.
(2021) applying regulatory inductive bias recursively across a compositional feature space.
2. I NTRODUCTION OF A DVERSARIAL T RAINING TO VAE S
When combining autoencoders (AEs) with an adversarial setting, one can connect the adversarial network either on the latent space or on the output of the decoder.
Makhzani et al.
(2016) proposed an adversarial AE (AAAE) that uses an adversarial discriminator network on the latent space of an AE to match its aggregated posterior to an arbitrary prior.
There, the encoder of the AE acts as the generator of the GAN and is the only part of the AE that gets updated with respect to the discriminator’s loss.
This kind of training is strongly connected to VAE training, with the adversarial loss taking on the role of KL divergence in the classical VAE training objective but without the constraint of a Gaussian prior.
The previously mentioned FactorVAE (Kim & Mnih, 2018) implements an adversarial loss on the latent space to reduce total correlation.
Larsen et al.
(2016) proposed using a discriminator network with the decoder of the VAE acting as generator to improve the visual fidelity of the generated images, but with no focus on disentanglement.
The difference to DAVA is that they used the discriminator on the real observations, while we propose a discriminator that only sees observations generated by the decoder of the VAE.
Zhu et al.
(2020) introduce an recognition network based loss on the decoder that encourages predictable latent traversals.
In other words, given a pair of images where all but one latent dimensions are kept constant, the recognition network should be able to predict in which latent dimension the change occurred.
Applied on top of a baseline VAE model, this loss slightly improves the disentanglement performance of the baseline VAE.
In a semi-supervised setting, Carbajal et al.
(2021) and Han et al.
(2020) propose adversarial losses on the latent space of VAEs to disentangle certain parts of the latent space from information present in labels.
Unfortunately, such an approach does not work in an unsupervised setting with the goal of disentanglement of the complete latent space.
2. M EASURING D ISENTANGLEMENT
All supervised metrics have in common that they are dependent on a canonical factorization of ground truth factors.
Given access to the generative process of a dataset, the FVAE metric (Kim &
Mnih, 2018) can be used to evaluate a model.
It first creates a batch of data generated by keeping aforementioned ground truth factor fixed and randomly sampling from the other ground truth factors.
It then uses a majority vote classifier to predict the index of a ground truth factor given the variance of each latent dimension computed over said batch.
Without access to the generative process, but given a number of fully labeled samples, one can use metrics like DCI Disentanglement by Eastwood &
Williams (2018) and MIG by Chen et al.
While DCI in essence assesses if a latent variable of a model captures only a single ground truth factor, MIG evaluates if a ground truth factor is captured only by a single variable.
As a result, DCI compared to MIG does not punish multi-dimensional representations of a single ground truth factor, for example the RGB model of color or a sine/cosine representation of orientation.
There exists a small number of unsupervised disentanglement metrics.
The unsupervised disentanglement ranking (UDR) by Duan et al.
(Duan et al., 2020) evaluates disentanglement based on the assumption that a representation should be disentangled if many models, differently initialized but trained with the same hyperparameters, learn the same representation.
To achieve this, they compute pairwise similarity (up to permutation and sign inverse) of the representations of a group of models.
The score of a single model is the average of its similarity to all the other models in the group.
ModelCentrality (MC) (Lin et al., 2020) builds on top of UDR by improving the pairwise similarity evaluation.
A drawback is the high computational effort, as to find the optimal hyperparameter setting, multiple models need to be trained for each setting.
UDR and MC do not assume any fixed set of ground truth factors.
Nevertheless, a weakness of these approaches is that they do not recognize similarity of a group of models that each learn a different bijective mapping of the ground truth factors.
The latent variation predictability (VP) metric by Zhu et al.
(2020) is based on the assumption that if a representation is well disentangled, it should be easy for a recognition network to predict which variable was changed given two input images with a change in only one dimension of the representation.
Easiness is quantified by evaluating the accuracy of the recognition network in a few-shot learning problem, where higher accuracy indicates a better disentangled model.
An advantage of the
VP metric is its ability to evaluate GANs as it is not dependent on the model containing an encoder.
In comparison to our proposed metric, VP, UDR and MC are dependent on the size of the latent space or need to define additional hyperparameters to recognize inactive dimensions.
Estermann et al.
(2020) showed that the UDR has a strong positive bias for low-dimensional representations, as low-dimensional representations are more likely to be similar to each other.
One could see the same issue arise for the VP metric, as accuracy for the recognition network will likely be higher when there is only a low number of change-inducing latent dimensions available to choose from.
2. A PPLICATIONS OF D ISENTANGLED R EPRESENTATIONS
The effects of disentangled representations on down-stream task performance are manifold.
Van Steenkiste et al.
(2019) showed that for abstract reasoning tasks, disentangled representations enabled quicker learning.
We show in Section 4.2 that the same holds for our proposed metric.
In work by Higgins et al. (2017b), disentangled representations provided improved zero-shot domain adaption for a multi-stage reinforcement-learning (RL) agent.
The UDR metric (Duan et al., 2020) correlated well with data efficiency of a model based RL agent introduced by Watters et al.
(Watters et al., 2019).
Disentangled representations further seem to be helpful in increasing the fairness of down-stream prediction tasks (Locatello et al., 2019a).
Contrary to previous qualitative evidence (Eslami et al., 2018;
Higgins et al., 2018), Montero et al.
(2020) present quantitative findings indicating that disentanglement does not have an effect on out-of-distribution (OOD) generalization.
Recent work by
Schott et al.
(2022) supports the claims of Montero et al.
(2020), concluding that disentanglement shows improvement in downstream task performance but not so in OOD generalization.
P OSTERIOR I NDIFFERENCE P ROJECTION E QUIVALENCE
We first introduce the notation of the VAE framework, closely following the notation used by Kim &
Mnih (2018).
We assume that observations x ∼ ˜ D are generated by an unknown process based on some independent ground-truth factors.
The goal of the encoder is to represent x in a latent vector z ∈ R d .
We introduce a Gaussian prior p ( z ) =
N (0 , I ) with identity covariance matrix on the latent space.
The variational posterior for a given observation x is then q θ ( z | x )
= (cid:81)
dj
=1 N ( z j | µ j ( x ) , σ 2 j ( x )) , where the encoder with weights θ outputs mean and variance.
The decoder with weights ϕ projects from the latent space z back to observation space p ϕ ( x | z ) .
We can now define the distribution q ( z ) .
Definition 3.1 (EP) .
The empirical posterior (EP) distribution q ( z ) of a VAE is the multivariate distribution of the latent vectors z over the data distribution ˜ D .
More formally, q ( z ) =
E x ∼ ˜ D
[ q θ ( z | x )] .
We can reconstruct an observation x ∼ ˜ D the following way: We sample z ∼ q θ ( z | x ) and then get the reconstruction ˆ x ∼ p ϕ
( x | z ) .
We informally call observations generated by this process reconstructed samples and denote them as ˆ x .
The decoder is not constrained to only project from q ( z ) to observation space.
We can sample observations from the decoder by using different distributions on z .
We define a particularly useful distribution.
Definition 3.2 (FP) .
The factorial posterior (FP) distribution ¯ q ( z ) of a VAE is a multivariate distribution with diagonal covariance matrix.
We define it as the product of the marginal EP distrbution: ¯ q ( z ) =
(cid:81)
dj
=1 q ( z j ) .
We can use the decoder to project z ∼ ¯ q ( z ) to observation space ˜ x ∼ p ϕ
( x | z ) .
We informally call observations created by this process generated samples and denote them as ˜ x .
We can now define the data distributions that arise when using the decoder to project images from either the EP or the FP.
Definition 3.3 ( ˜ D EP , ˜ D FP ) .
˜
D EP is generated by the decoder projecting observations from the EP latent distribution q ( z ) , i.e. reconstructed samples.
˜ D
EP =
E z ∼ q ( z ) [ p ϕ ( x | z )] .
˜ D FP is generated by the decoder projecting observations from the FP distribution ¯ q ( z ) , i.e. generated samples.
˜ D
FP =
E
z ∼ ¯ q ( z ) [ p ϕ ( x | z )] .
We can now define the core concept of this paper.
We look at the similarity of two data distributions generated by the decoder, ˜ D EP and ˜ D FP .
[Figure]
Definition 3.4 (PIPE) .
The posterior indifference projection equivalence ( PIPE ) of a VAE represents the similarity of the data distributions ˜ D EP and ˜ D FP .
In other words, PIPE is a measure of the decoder’s indifference to the latent distribution it projects from.
PIPE
[Equation]
θ
[Equation]
x | z )] , where ω is a general similarity measure.
Unsupervised disentanglement learning without any inductive biases is impossible as has been proven by Locatello et al.
(2019b).
Given a disentangled representation, it is always possible to find a bijective mapping leading to a representation which is entangled.
Without knowledge of the ground truth factors, it is impossible to differentiate the two representations.
We argue that PIPE is necessary for a disentangled model, even when it is not sufficient.
Assume that a model has learned a disentangled representation, but ˜ D EP is not equivalent to ˜ D FP .
There are two cases in which this could occur.
The first option is that the latent dimensions of the model are not independent.
This violates the independence assumption of the ground truth factors, consequently the model cannot be disentangled.
The second option is that the model has not learned a representation of the ground truth factors and generates samples which are not represented by the ground truth factors.
This model would also not be disentangled.
Therefore we conclude that PIPE is a necessary condition for disentanglement.
In Figure 2 we present reconstructed and generated samples of two different models to support our chain of reasoning.
Generated samples of the entangled model (Figure 2 (b)) look visibly out of distribution compared to the reconstructed samples.
Generated samples of the disentangled model (Figure 2 (d)) seem to be equivalent to the distribution of reconstructed samples.
PIPE M ETRIC
We propose a way to quantify the similarity metric ω of PIPE with a neural network and call this the PIPE metric.
Given a model M with encoder weights θ , decoder weights ϕ , and corresponding ˜
D EP , ˜ D FP (see Appendix B.2 for details on how to sample from ˜ D EP and tildeD FP ).
Create a set of observations S EP by sampling from ˜ D EP .
Informally, these are the reconstructed samples.
Create a set of observations S FP by sampling from ˜ D FP .
Informally, these are the generated samples.
Randomly divide S EP ∪ S FP into a train and a test set.
Train a discriminator network on the train set.
Evaluate accuracy acc of the discriminator on the test set.
Since a random discriminator will guess half of the samples accurately, we report a score of 2 · (1 − acc ) , such that 1 is the best and 0 is the worst score.
We train the discriminator network for 10 , 000 steps to keep the distinction of S EP and S FP sufficiently difficult.
We further use a uniform factorial distribution instead of FP for a slight improvement in performance.
Details on the implementation can be found in Appendix B.
4. R ELATION TO E XISTING M ETRICS
To classify the performance of the PIPE metric, we evaluate correlations with supervised metrics on a diverse set of commonly used datasets.
We namely consider Shapes3D (Burgess & Kim, 2018),
AbstractDSprites (Van Steenkiste et al., 2019) and Mpi3d Toy (Gondal et al., 2019).
All datasets are part of disentanglement-lib
(Locatello et al., 2019b), which we used to train the models we evaluated our metric on.
More details on the implementation of the PIPE metric and the evaluated models can be found in Appendix B. Results are displayed in Figure 3.
They show that the PIPE metric correlates positively with existing supervised metrics DCI, MIG and FVAE, surpassing the correlations of the unsupervised baselines.
While the performance of UDR on Mpi3d Toy and the performance of MC on AbstractDSprites is lacking, PIPE demonstrates consistent performance across all datasets.
Further, as opposed to UDR and MC, PIPE can be evaluated on a single model only, whereas UDR and MC are only able to evaluate sets of models.
[Figure]
[Table]
[Figure]
[Table]
Shapes3D
P I PE ( ou r s )
UD R M C R ec D C
I M
I G F VA E 35-18 2158 28 -3756 39 4 5064 43 8 43 7242 13 18 37 72
AbstractDSprites
P I PE ( ou r s )
UD R M C R ec D C
I M
I G F VA E -7434 2-44 24 -7645 -18 67 -5724 -11 29 -25 5754 -30 53 -52 63
Mpi3d Toy
We include reconstruction loss Rec as a trivial unsupervised baseline.
DCI, MIG and FVAE are the corresponding supervised metrics.
Low or even negative correlations of our metric with UDR and MC show that our metric captures a different aspect of disentanglement.
Correlation of our metric with supervised disentanglement metrics is mostly consistent across datasets.
We note that the correlation with supervised metrics on the Mpi3d Toy dataset changes direction for both UDR and
Rec, while MC expresses difficulties with AbstractDSprites .
4. C ORRELATION WITH D OWNSTREAM T ASK P ERFORMANCE
To further quantify the usefulness of the PIPE metric, we analyze the predictive performance of the metric for an abstract reasoning downstream task.
For the downstream task, a VAE is trained to learn a disentangled representation of a dataset.
The downstream model then only gets access to this representation then trying to solve the downstream task.
Van Steenkiste et al.
(2019) evaluated different supervised disentanglement metrics in terms of predictive performance of accuracy in an abstract reasoning task on the datasets Shapes3D and AbstractDSprites .
They showed that good disentanglement is an indicator of better accuracy during the early stages on training.
We reproduce their experiment on a reduced scale by only considering up to 6,000 training steps.
As can be seen in Figure 4, our metric is on par with supervised metrics and clearly outperforms the unsupervised baselines.
More importantly this means that PIPE is a positive predictor of accuracy of downstream models in a few-sample setting and is therefore a desirable property for unsupervised model selection.
[Figure] [Figure]
[Figure] [Figure]
AbstractDSprites
T RAINING P ROCEDURE FOR V ARIATIONAL A UTO - ENCODERS (DAVA)
[Figure]
Dis A discriminator attached to the decoder of a VAE.
The goal for the discriminator is to differentiate samples coming from the EP from those coming from the FP.
In the previous chapter we have established that
PIPE is desirable for a model.
We now present how to encourage this property at training time by designing an adversarial training procedure.
We train a discriminator and a VAE at the same time.
The discriminator needs to differentiate ˜ D
EP from ˜ D FP , which is achieved with the following loss:
[Equation]
The loss function for the VAE is compromised of a reconstruction term with the weighted negative objective of the discriminator:
[Equation]
with reconstruction loss
[Equation]
While this works in practice, it faces the issue of the weight µ being very dataset-specific.
It is also closely related to FactorVAE, with the difference that FactorVAE applies the adversarial loss to the latent space and to the encoder only.
Let us consider an alternative approach.
Work by Burgess et al.
(2017) looked at the KL-Divergence from an information bottleneck perspective.
They proposed using a controlled latent capacity increase by introducing a loss term on the deviation of some goal capacity C that increases during training:
[Equation]
Such a loss provides a disentanglement prior complementary to the minimizing total correlation prior used in β -TCVAE and FactorVAE.
Unfortunately, it too depends on specific choice of hyperparameters, mainly the max capacity C and the speed of capacity increase.
We now demonstrate how to incorporate both the adversarial loss and the controlled capacity increase into a single loss function.
We call this approach disentangling adversarial variational auto-encoder DAVA (also see Figure 5):
[Equation]
The main strength of DAVA is its ability to dynamically tune C and µ by using the accuracy of its discriminator.
This yields a model that performs well on a diverse range of datasets.
We now outline the motivation and the method for tuning C and µ during training.
[Figure]
The goal of the controlled capacity increase in the AnnealedVAE architecture is to allow the model to learn individual factors at a time into individual latent dimensions.
The order in which they are learned corresponds to their respective contribution to the reconstruction loss.
In DAVA, we want to encourage the model to learn new factors by increasing C as long as it has learned a disentangled representation of the currently learned factors .
This is the case when the discriminator cannot distinguish ˜ D EP from ˜ D FP .
As soon as the accuracy of the discriminator increases, we want to stop increasing C and increase the weight of the adversarial loss to ensure that the model does not learn any new factors of variation while helping it to disentangle the ones it has picked up into individual dimensions.
As can be seen in Figure 6, this results in dataset-specific schedules of C over the course of the training.
Algorithm 1 in Appendix C.1 describes the training of DAVA in more detail.
5. E XPERIMENTS
To quantify the performance of DAVA, we perform an extensive study including multiple model architectures and datasets.
We include the best disentangling models according to (Locatello et al., 2019b) as baseline, namely the β -TCVAE
(Chen et al., 2018) and FactorVAE (Kim & Mnih, 2018).
We further include AnnealedVAE (Burgess et al., 2017), as it relates to DAVA too.
As discussed earlier, the performance of each of these approaches is highly dependent on the regularization strength.
For this reason, we perform a grid-search over a range of values curated by roughly following the work of Locatello et al.
(2019b)
and adapted where necessary to ensure best possible performance.
The exact hyperparameters considered can be found in appendix C.1.
The training of DAVA does not rely on such dataset-specific regularization, therefore all hyperparameters for the training of
DAVA were kept constant across all considered datasets.
All other hyperparameters closely follow (Locatello et al., 2019b) and were kept consistent across all architectures (baseline and DAVA) and are also reported in appendix C.1.
We run 5 different random seeds for each configuration to establish statistical significance.
We provide summarized results for the most insightful datasets in Tables 1,and 3.
For the baselines we report the performance of the mean hyperparameter setting according to the average score of all considered metrics.
The reader is advised to consult Appendix D for complete results of all evaluated datasets.
[Table] [Table]
[Table]
Appendix D also contains the results of the best performing hyperparameter choice for each baseline model, demonstrating their variance across datasets.
Note that the PIPE scores of FactorVAE in Table 3 are exploiting a design decision discussed in depth in Appendix B.3.
The decoders of these models mapped all latent values to a single image, therefore we exclude their scores.
The results showcase the strong consistent performance of DAVA across different datasets.
Compared to the baselines,
DAVA is able to deliver improvements of up to 50% for the supervised metrics, and over 100% for
PIPE .
This allows DAVA to be used in practice as consistent performance can be expected.
PIPE metric ( · 10 − 2 )
F I D s c
o r e β -TCVAE
AnnealedVAEFactorVAEControlVAEDAVA
We further validate DAVA on three real-world datasets.
CelebA (Liu et al., 2015) is dataset commonly used in the disentanglement literature.
It compromises of over 200,000 images of faces of celebrities.
As shown in Figure 7,
DAVA achieves a competitive FID while still having a comparatively high PIPE metric score.
This means that DAVA accomplishes a similar visual fidelity of its generated samples as the best baseline methods while better disentangling its latent space.
Results for datasets Mpi3d Real (Gondal et al., 2019) and CompCars (Yang et al., 2015) can be found in Appendix D and E, respectively.
C ONCLUSIONS
In this work, we presented a new way to understand and formalize disentangled representations from an unsupervised point of view.
The main idea is that the distribution of reconstructed samples of a VAE should be similar to the distribution of generated samples.
We quantify this notion in a new unsupervised disentanglement metric called PIPE metric to evaluate disentangled representations.
Without access to ground truth, PIPE provides results in line with supervised metrics.
Furthermore, we apply the same idea to DAVA, a novel VAE training procedure.
DAVA can self-tune its regularization strength and produces results competitive with existing models that require more dataset-specific hyperparameter tuning.
Robust unsupervised disentanglement performance is an important requirement for applicability in practice.
With DAVA, we provide a method that can be used by practitioners with only a relatively small amount of compute needed.