Trading off Image Quality for Robustness is not Necessary with Regularized Deterministic Autoencoders
Abstract
The susceptibility of Variational Autoencoders (VAEs) to adversarial attacks indicates the necessity to evaluate the robustness of the learned representations along with the generation performance.
The vulnerability of VAEs has been attributed to the limitations associated with their variational formulation.
Deterministic autoencoders could overcome the practical limitations associated with VAEs and offer a promising alternative for image generation applications.
In this work, we propose an adversarially robust deterministic autoencoder with superior performance in terms of both generation and robustness of the learned representations.
We introduce a regularization scheme to incorporate adversarially perturbed data points to the training pipeline without increasing the computational complexity or compromising the generation fidelity when compared to the robust VAEs by leveraging a loss based on the two-point Kolmogorov–Smirnov test between representations.
We conduct extensive experimental studies on popular image benchmark datasets to quantify the robustness of the proposed approach based on the adversarial attacks targeted at VAEs.
Our empirical findings show that the proposed method achieves significant performance in both robustness and fidelity when compared to the robust VAE models.
An implementation is available at https://github.com/boschresearch/Robust_GMM_DAE .
Introduction
Variational autoencoders (VAEs) offer a powerful probabilistic framework to learn deep generative models.
They are widely employed in various domains such as computer vision, natural language processing and representation learning [24, 16, 25, 26, 10].
A VAE is composed of two coupled but independently parameterized models, an encoder or a recognition model that computes a latent representation of the input space with stochastic latent variables and a decoder or a generative model to map the latent representations back to the original input space [18].
One of the major advantages of VAEs is that they provide semantically meaningful latent representations of highdimensional complex input distributions which can be further utilized for various downstream tasks [17, 14, 28, 11].
However, VAEs are often limited due to theoretical and practical limitations such as over-regularization, prior-posterior mismatch resulting in trade-offs between generation and reconstruction fidelity [31, 9, 12].
Recently proposed deterministic versions of VAEs offer promising alternatives to overcome these limitations [12, 27].
The semantically meaningful representations learned by VAEs can still be corrupted by so-called adversarial attacks [30, 19, 20], where even small but specifically crafted changes to the input can lead to very different reconstructions.
This observation reveals a lack of generalization within such models and is therefore a serious concern with respect to many practical applications.
While it is harder to attack VAEs when compared to classifier networks [13] it is essential to analyze the robustness of VAEs along with their generative performance, to validate whether the learned latent representations are meaningful.
Hence, there has been increasing research interest in the deep learning field towards training robust models, both for classifier models and for autoencoders, i.e. for robust representation learning.
Borrowed from the training of robust classification models [22, 33], the concept of adversarial training has proven to be able to smoothen the VAE encoder and improve the robustness of the learned representations [6].
Other attempts toward learning robust representation spaces introduce either complex network architectures or expensive regularization mechanisms to improve the robustness of VAEs [32, 2].
Further, it has been pointed out in previous works that the robustness of VAEs can be improved by generating disentangled latent representations or by encouraging the smoothness or consistency of the encoding-decoding process [32, 5].
However, regularizing the VAE objective to enhance robustness often leads to poor generation ability compared to its non-robust counterpart.
Hence, we seek to focus on improving the robustness of autoencoders while still maintaining the generation performance.
In this work, we introduce a simple and easy to train deterministic autoencoder which exhibits superior performance in terms of both generation and adversarial robustness.
We argue that the deterministic approach enhances the robustness of VAEs when the latent codes are properly regularized.
Consequently, we extend the training objective of the multi-modal deterministic autoencoders [27] to incorporate adversarially perturbed input data points in the latent space.
The proposed adversarial learning scheme is comparatively less expensive and easier to implement than existing adversarially trained robust VAE models.
We conduct extensive experimental analysis to evaluate the robustness of the trained model on popular benchmarks such as MNIST, FASHIONMNIST, SVHN and CELEBA images.
Our empirical evaluations show that the proposed model consistently exhibits high adversarial robustness and significantly better generation performance compared to state-of-the-art robust VAE baseline models.
We also show that by improving the robustness of the learned representations, a classifier trained on the learned latent space of the model also exhibits better robustness.
Related Work
The ability to defend against adversarial attacks is closely related to the sensitivity of the learned latent representations to slight changes in the input data points.
In this section we first discuss seminal works proposed to generate rich and meaningful latent codes.
This is followed by the review on adversaries for VAEs and strategies proposed to defend against such adversarial attacks.
Variational Autoencoders (VAEs)
In standard VAEs, the encoder and the decoder are jointly trained to maximize the evidence lower bound (ELBO) of the log likelihood of the model.
The resulting training objective is a combination of a reconstruction loss and the Kullback–Leibler (KL) divergence between the learned latent representations and a chosen prior - usually a uni-modal
Gaussian normal distribution.
Higgins et.al [16] improved the learned representation in VAEs by encouraging disentanglement in the latent space and introduces β -VAE for disentangled factor learning.
β -VAE modifies VAEs by introducing a hyperparameter to balance the latent regularization term with the reconstruction performance.
Chen et.al [7] further decompose the ELBO term in VAEs to introduce total correlation (TC) regularization and propose β -TCVAE as a promising alternative to β -VAEs.
Willets et.al.
[32] show that addition of TC term to the VAE objective also improves the robustness of VAEs.
Several methods were proposed to include complex and flexible priors to the training pipeline of VAEs to enhance the semantics of the learned latent representations [8, 35, 31].
Miao et.al.
[23] introduces an approach to incorporate the inductive bias into VAEs without explicitly changing the prior by utilizing intermediary set of latent variables.
Hierarchical VAEs [26, 3, 34, 21] extend the standard VAE framework by introducing a hierarchy of latent variables and offer superior modelling capabilities.
Gosh et.al.
[12] question the variational formulation of VAEs and introduce a simple and effective deterministic model without any prior assumptions followed by a post-hoc density estimation to approximate the learned posterior.
A multimodal version to this approach, that utilizes the capacity of expressive multi-modal latent distributions to yield high quality generation, was proposed in [27].
Adversarial attacks on VAEs Adversarial attacks targeted towards autoencoders were first discussed in [13].
Common attacks on VAEs follow procedures similar to attacks against classifiers, i.e. they aim to maximize the network’s loss.
Usually, slight perturbations are added to the input images to make the reconstructions similar to a specific target image (targeted/supervised attack) or a completely different image (untargeted/unsupervised) [30, 19] such as to maximize the reconstruction loss.
In [32] Willets et.al.
show that applying the TC regularization introduced in TC-VAEs to hierarchical VAEs yields robust representations.
Although the resulting model improves adversarial robustness, the training complexity is high when compared to a VAE.
Cemgil at.al [6] relate the robustness of VAEs to the smoothness of the encoding process.
Similarly to Madry et.al.
[22], they introduce a regularization scheme based on a selection mechanism in the latent space to generate additional data points to minimize the entropy regularized Wasserstein distance between latent representations.
Following the same direction, Cemgil et.al.
[5] argue that the lack of consistency between the encoding-decoding process cause the susceptibility to adversarial attacks in VAES.
In contrast to the previous works,
Camuto et.al.
[4] provide a theoretical insight to the robustness of VAEs and introduce a novel criterion for robustness in VAEs.
Barret et.al.
[2] propose to constrain the Lipschitz constants for both encoder and decoder to ensure certifiable adversarial robustness of VAEs.
Although these methods comparatively improve the adversarial robustness of VAEs, they are often accompanied by complex network architectures and expensive training procedures.
In contrast, our approach adopts an inexpensive adversarial training scheme to the latent space of deterministic autoencoders by an elegant extension to the regularization proposed in [27] to ensure both robustness and fidelity.
Towards robust deterministic autoencoders
Deterministic Autoencoders [12, 27] offer a promising alternative to VAEs for learning meaningful representations of complex input spaces with high fidelity.
Motivated by this fact, we aim to further explore the robustness of the learned representations of the resulting model.
We are particularly interested in the multimodal prior setting in [27], since a flexible and expressive Gaussian mixture model (GMM) prior assumption facilitates encoding similar data points closer together while distancing dissimilar points far in the latent space - a behaviour that has also been found to be beneficial in learning robust classifier models [11].
Consequently, we propose to adopt the regularization technique proposed in [27] to regularize the learned latent representations of our model towards a predefined
GMM prior.
For a model to be inherently robust, slight perturbations in the input space should not result in substantial variations in the encoding space and the corresponding reconstructions.
This could be also attributed to the smoothness of the learned encoder.
Hence, it is essential to investigate the smoothness of the learned representations and the reconstructions of the model.
In the following subsections, we resume the regularization loss from [27] for the sake of completeness and illustrate how it can be extended to adversarial training samples.
3.1 Regularization of the learned representations – Encoder smoothness
Following the idea from [27], we aim to regularize the D -dimensional latent representations { z 1 , ..., z N } of input datapoints { x 1 , ..., x N } learned by the encoder g of the model towards a k -modal GMM prior.
For k ≤ K , let µ k and Σ k be the mean and covariance matrix of the k -th mode in the model.
Further, let w k > 0 be the weight of the k -th mode, (cid:80)
Kk
=
w k = 1 .
Inspired by the statistical Kolmogorov-Smirnov (KS) test, the authors propose to regularize the empirical cumulative distribution function (CDF) of the latent samples towards the CDF Φ( z ) of the prior distribution,
[Equation]
where z = ( z 1 , . . . , z D ) , t = ( t 1 , . .
. , t D ) .
While the original KS-test quantifies a metric between joint CDFs, the minimization of this KSdistance can be approximated by marginalization, i.e. by regularizing the empirical marginal CDFs
[Equation]
of the latent representations { z 1 , ..., z N } in dimensions d = 1 , . . .
, D towards the marginal CDFs
[Equation]
of the GMM prior.
Here, [ · ] d indicates the d -th dimension of a vector, i.e. [ µ k ] d is the d -th entry of the mean vector µ k = ( µ k, 1 , . . . , µ k,D ) of mixture component k , i.e. a scalar.
Similarly, [ · ] d,d indicates the entry in row and column d of the matrix Σ k = diag( σ k, 1 , . . .
, σ k,D ) .
The respective loss is computed as
[Equation]
An additional loss compensates for the ambiguities on the target distribution introduced by the marginalization, i.e.
[Equation]
with Σ GMM =
(cid:80)
Kk
=
w k Σ
k
+ (cid:80)
Kk =1 w k ( µ k − ¯ µ ) ( µ k − ¯ µ ) T , ¯ µ = 1 k (cid:80)
Kk =1 µ k and ¯Σ being the empirical covariance matrix of the latent representations.
The total loss of the model in [27] is a combination of both regularization terms to enforce the latent representations of the encoded data to match a predefined multi-modal prior distribution and a reconstruction loss.
Motivated by [11], for equidistantly chosen modes µ i as in Figure 1, we expect such a model to already provide not only an improved generation fidelity as observed in [27] but also a more robust behavior.
We provide empirical evidence for this conjecture in an ablation study in
Section 4.
The GMM regularization implicitly clusters latent points such that similar points are close to one another while dissimilar points are distant.
To further improve the robustness of our model we employ adversarial training - a widely popular defense strategy utilized to learn robust deep networks [22, 33].
That is, during training we utilize adversarial samples z adv n that are explicitly optimized to fall in an ϵ − ball around the original latent representations z n for n ≤ N and to decode to damaged or semantically altered images.
In [1] it was observed that such adversarial samples tend to explore the underrepresented regions in the latent space.
In the following, we extend the losses in (4) and (5) to overcome this undesired behaviour.
This approach allows for a cheaper yet very effective adversarial training, while preserving the reconstruction and generation ability of the original model from [27].
3.2 Adversarial training data augmentation – Improving Robustness
To generate adversarial inputs, we adapt the fast gradient sign method [33] to the latent space of the model.
For a given ϵ > 0 and datapoint x n , the objective of the attack is to find the corresponding adversary x adv n that would introduce maximum distance in the encoding space.
That is, x adv n = x n + δ x n , where δ x n is the solution to the optimization problem
[Equation]
(6) where g is the encoder of the model.
To prevent adversarial samples from exploring unexplored regions of the latent space, we assume that the joint distribution of latent encodings ( z n , z adv n ) (here z n = g ( x n ) and z adv n = g ( x n + δ x n )) of data points and their adversarial samples ( x n , x adv n ) to follow the same multi-modal GMM prior (Figure 1).
One possible straight forward extension of the approach in [27] would be to consider adversarial examples as a specific data augmentation and regularize z 1 ,...,N and z adv1 ,...,N to the same GMM prior independently and ignoring cross-covariance between z 1 ,...,N and z adv1 ,...,N (here the off-diagonal elements in the covariance matrix of the GMM prior, that is the last matrix mentioned in equation (8), are zero) .
The corresponding losses in eqs.
(4) and (5) take the form
[Equation]
and
L aug
[Equation]
where ¯ Σ and ¯ Σ adv are the empirical covariance matrices of the latent representations z 1 ,...,N and their adversaries z adv1 ,...,N respectively, and ¯ Σ cross is the empirical cross-covariance between benign and adversarial samples.
While such a regularization preserves the overall distribution even under adversarial attacks, it can not control the distance of a specific adversarial sample to its benign z n .
In the worst case scenario, an adversarial example z adv n can be mapped to a different Gaussian mixture component than z n and therefore cause maximum damage in the reconstruction as shown in
Figure 1(left).
To evenly spread out the learned representations, we inject Gaussian noise to the latent vectors during training.
Let x ϵ,n be the output of the decoder at z n + ϵ n , where ϵ n ∼ N (0 , I D ) .
The reconstruction loss equals the mean squared error between inputs x n and their noisy reconstructions x ϵ,n .
3.3 Coupling of z and z adv – A Two-Point KS-distance loss
To ensure that the adversarial examples remain in close proximity to the original mapping in the learned latent space, we establish a strong coupling between the two distributions, z 1 ,...,N and z adv1 ,...,N .
Hence we propose to match the the empirical CDFs of z 1 ,...,N and z adv1 ,...,N and introduce a novel regularization based on the two-point KS-test [29].
By analogy to the one-point KS-test that tests whether a sample is drawn from a given, continuous distribution, the two-sample KS test determines whether two samples with empirical CDF are drawn from the same distribution.
To this end, the two-sample KS test evaluates the supremum of the distance between the two CDFs.
Here, we propose to minimize this distance computed from the marginalized two-point KS-test in order to align the distributions of benign points and their adversaries.
The resulting loss is consistent with the previous regularization and allows to efficiently establish the desired coupling between the representations.
The first regularization loss of the adversarially extended model with pairwise coupling takes the following form
[Equation]
) where ¯ F d , ¯ F adv d are the empirical CDFs of z and z adv respectively.
The correlations between the latent representations and their adversarial samples need to be considered separately.
The degree or strength of the coupling is controlled by a coupling parameter | α |≤ 1 , where α = 1 indicates the border condition where z
=
z adv .
The covariance loss of the extended model becomes,
[Equation]
[Equation]
[Equation]
The total training objective of the model takes the following form,
[Equation]
The weights λ REC , λ KS and λ CV can be calculated by taking the statistics of samples from the GMM prior [27], see Appendix.
Experiments
We conduct an extensive experimental analysis to evaluate the robustness of the proposed model.
We consider the state of the art latent space attacks targeted at VAEs [13, 19, 2] and evaluate the robustness based on the quantitative metrics as described in Section 4.
Since the latent spaces of VAEs are often further utilized for various downstream applications, we also consider the impact of such adversarial attacks on a classifier trained in the latent space of the model.
To evaluate the fidelity of the learned representations, we report the Fréchet inception distance (FID) [15] of the generated images.
Our model is compared with the following baseline models, Variational Autoencoder (VAE) [18], β -VAE [16], β -TCVAE [7], LipschitzVAE [2], Smooth Encoders (SE) [6] and Autoencoding Variational Autoencoder (AVAE)
The experimental study is conducted on important image benchmark datasets such as MNIST, FASHIONMNIST, SVHN and CELEBA.
For simplicity we consider a fully connected network architecture for experiments on MNIST and
FASHIONMNIST images and a convolutional architecture for experiments on SVHN and CELEBA images.
Further details of the experiments are given in the Appendix 1 .
Adversarial attacks.
Adversarial attacks targeted at VAEs attempt to add small noise perturbations to the input data points that fool the model to reconstruct the input image to a target adversarial image or a completely different image.
Following recent literature [2, 32], we consider two types of adversarial attacks in our experiments.
Latent space attacks or supervised attacks are considered the most effective mode of attacks on
VAEs.
Here, the attacker tries to add a noise perturbation δ to a data point x , such that the latent representation z x + δ of the perturbed input x + δ is close to the latent representation z t of a chosen target adversarial image, x t .
The attack involves solving the following optimization problem,
[Equation]
Further, we consider maximum damage or output space attack.
In this setting, the adversary perturbs the input data point to cause maximum damage in the reconstruction of the decoder f of the model and optimizes the following objective,
[Equation]
In both scenarios, the noise perturbation is explicitly constrained by some constant λ > 0 to ensure a consistent comparison with the baseline models.
Evaluation.
The qualitative results for CELEBA images for latent space and maximum damage attack are shown in Figure 3.
We consider three evaluation metrics to quantitatively estimate the robustness of the different models for the above-mentioned adversarial attacks.
First, we evaluate the attack loss of the adversary.
That is, we report the achieved value of the optimization objectives in (12) and (13).
The observed attack losses for the two forms of adversarial attacks are shown in Figure 2 (MNIST results are given in the Appendix).
Higher attack losses correspond to less
FASHIONMNIST
SVHN
CELEBA  5 0.
A tt a c k l o ss
VAEBetaTCVAELipschitzVAESEAVAEOurs
A tt a c k l o ss
VAEBetaTCVAELipschitzVAESEOurs  5 0.
A tt a c k l o ss
VAEBetaTCVAELipschitzVAESEOurs [Figure]
A tt a c k l o s s
VAEBetaTCVAELipschitzVAESEAVAEOurs
A tt a c k l o s s
VAEBetaTCVAELipschitzVAESEOurs
A tt a c k l o s s
VAEBetaTCVAELipschitzVAESEOurs
[Figure]
successful attacks and hence better robustness.
Due to the different regularization methods used in the baseline models, the inherent scale of the aggregated posterior changes and hence the value of the attack objective in eqn 12.
Hence for latent space attacks, the values reported in Figure 2 might not be directly comparable.
Another strong indicator for the robustness of the considered models is the similarity between the reference image x r and the reconstructions ˜ x a of the adversarially perturbed variants x a .
In the case of latent space attacks, we consider the target image x r = x t as the reference images and compare with the reconstruction ˜ x a = f ( z x + δ ) of the perturbed latent representation.
For a maximum damage attack, we consider the original images x r = x as the reference images and compare them to ˜ x a .
Similar to [20], we use the perception-based similarity metric Multi-Scale Structural Similarity Index Measure (MSSSIM ∈ [0 , 1] ) to compare the images.
We also report the l 2 -distance as an alternative similarity metric to MSSSIM in the Appendix.
Lower values of MSSSIM( x r , ˜ x a ) indicate less similarity between the reference (target) image and the adversarial reconstructions and correspond to less successful latent space attacks.
And higher value of MSSSIM( ˜ x r , ˜ x a ) corresponds to less successful maximum damage attacks as they correspond to high similarity between reference (original) and the adversarial reconstructions.
We report the observed values in Table 1.
Finally, it is also essential to evaluate how similar the adversarial images are to the original image.
Ideally, a successful attack implies that both adversarial and original images look similar in appearance (MSSSIM ≈ 1 ).
Hence we consider the MSSSIM between the original and adversarial images in Table 1.
We also report the FID of the generated samples to compare the fidelity of the representations.
Results.
Overall we see that the proposed model outperforms all the considered baselines in terms of both robustness and offers superior generation performance.
Even for complex datasets like SVHN and CELEBA, we observe the same trend with FID and robustness measures. These results are especially promising since we did not employ any extensive hyperparameter search for training.
Our results further confirm that both robust and high fidelity models are possible. Since we employ an FGSM based adversarial training, the training time required is cheaper when compared to the expensive PGD-based training used in smooth encoders (SE). The computation time for a single iteration of SEs is two times more when compared to our method. Ablation Study.
In this section, we compare three different variants of regularized deterministic autoencoders to evaluate the importance of joint regularization of the original and adversarial samples. We begin with the model proposed in [27], which we denote as GMM-DAE. Second, we study the augmented model defined by equations (7) and (8) in Section 3.2, but without the coupling of original and adversarial latent representations (Augmented). We compare the robustness and fidelity of these models with our proposed model, i.e. where original and adversarial latent representations are not
[Table] [Table]
Ours
[Table]
FASHIONMNIST [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
TCVAE
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table]
72 . 29 AVAE
[Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
MSSSIM
[Table]
˜
[Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table]
58 SE
[Table] [Table]
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Figure] [Table]
only regularized towards the same prior but coupled according to equations (9), (10) [Table] and (11) in Section 3.3 (Ours).
The observed metrics are reported in Table 2.
We observe that the proposed method yields comparatively better performance in terms of robustness while still maintaining the generation fidelity when compared to the non-robust version, GMM-DAE.
It can also be seen that enforcing coupling between the latent representations of the original and adversarial samples (Ours) leads to better performance than simply augmenting them (Augmented).
It is worth pointing out that the GMM-DAE maintains better performance than a standard VAE model (Table 1, MNIST VAE results). This is due to the well-structured latent space in GMM-DAE. This observation further confirms our hypothesis in Section 3 that when similar-looking samples are modelled together in the latent space of the model, robustness can be improved.
Robustness to downstream applications.
Since the learned representations of VAEs are often used for various downstream tasks, it is also vital to verify how adversarial attacks affect the performance of the same.
To showcase the effect of adversarial attacks to downstream classification tasks, we train a MLP classifier on the latent space of the model and observe the accuracy drop in the presence of the latent space attacks for a constrained noise norm λ = 1 .
The observed values along with the clean accuracy are shown in Table 3.
The classifier trained on the latent space of the proposed model achieves better accuracies when compared to the baseline models under latent space attack.
Discussion & Conclusion
Limitations.
Although cheaper when compared to the current adversarially trained robust VAE models, the FGSM based adversarial training scheme is still expensive when compared to the nonrobust counterpart.
The coupling parameter in the proposed method is an additional hyperparameter to tune and enforcing strong coupling i.e. α = 1 , might compromise the generation fidelity of the resulting model.
While we need the coupling to perform adversarial training, overly strong coupling will necessarily lead to deteriorated reconstructions, similar as in VAEs.
It is worth noting that we did not perform intensive hyperparameter optimization in this line of experiments.
Hyperparameter optimization might be needed in other datasets for this reason.
Further, datasets with large inter class variance might not benefit from the underlying multimodal prior assumptions in the current approach.
It would be interesting to explore the impact of the adversarial attacks in other potential downstream applications such as optimization in the latent space of VAE models. Potential societal impact.
Variational autoencoders enable learning meaningful representations of complex high dimensional data without any supervision. This further enhances the usability of these learned representations for various downstream tasks and applications where limited data is available for example, due to privacy/security concerns. Hence, it is important to study the robustness of these representations along with their accuracy, especially when employed in real-world applications. In the present paper, we propose a method to train robust VAEs with high fidelity in the learned latent space.
Since our work is a step towards more robust models, we hope to see a positive social impact. There is currently limited work in this direction and we believe that our method encourages potential future work in developing robust VAE models. On the other hand, we should take into consideration the possible negative social impact of this research, especially in safety-critical applications.
Although we observe superior robustness in our model against the existing attacks, similar performance cannot be guaranteed on newly discovered attacks on VAEs. Hence, when deployed in real-world applications we highly recommend testing the model continuously against newly designed attacks.
We also urge the machine learning community to pursue this work responsibly to enable potential future research without any misuse.
Conclusion.
Developing robust VAE models is crucial since the learned representations of VAEs are frequently used for various applications.
Motivated by the recent research towards deterministic alternatives to VAEs, we study the robustness of deterministic autoencoders.
We extend recently developed regularization schemes to efficiently couple the adversarial examples and the learned representations during training.
Our experiments show that with proper regularization, adversarially trained multimodal deterministic autoencoders offer significantly improved adversarial robustness and high fidelity in the learned latent space.