Trading off Image Quality for Robustness is not Necessary with Regularized Deterministic Autoencoders
Authors: Amrutha Saseendran
Kathrin Skubch
Stefan Falkner
Margret Keuper
Venue: NeurIPS 2022 Accept (2022)
Keywords: Adversarial robustness
Generative models
Deterministic autoencoder

Abstract: The susceptibility of Variational Autoencoders (VAEs) to adversarial attacks indicates the necessity to evaluate the robustness of the learned representations along with the generation performance. The vulnerability of VAEs has been attributed to the limitations associated with their variational formulation. Deterministic autoencoders could overcome the practical limitations associated with VAEs and offer a promising alternative for image generation applications. In this work, we propose an adversarially robust deterministic autoencoder with superior performance in terms of both generation and robustness of the learned representations. We introduce a regularization scheme to incorporate adversarially perturbed data points to the training pipeline without increasing the computational complexity or compromising the generation fidelity by leveraging a loss based on the two-point Kolmogorov–Smirnov test between representations. We conduct extensive experimental studies on popular image benchmark datasets to quantify the robustness of the proposed approach based on the adversarial attacks targeted at VAEs. Our empirical findings show that the proposed method achieves significant performance in both robustness and fidelity when compared to the robust VAE models.

Trading off Image Quality for Robustness is not Necessary with Regularized Deterministic Autoencoders
Amrutha Saseendran', Kathrin Skubch', Stefan Falkner', and Margret Keuper
Bosch Center for Artificial Intelligence
University of Siegen, MMax Planck Institute for Informaties, Saarland Informatics Campus Amrutha. Saseendran@de bosch.com
Abstract
The susceptibility of Variational Autoencoders (VAEs) adversarial attacks indi- cates the necessity to evaluate the robustness of the learned representations along with the generation performance. The vulnerability of VAEs has been attributed to the limitations associated with their variational formulation. Deterministic au- toencoders could overcome the practical limitations associated with VAEs and offer promising alternative for image generation applications. In this work, we propose an adversarially robust deterministic autoencoder with superior perfor- mance in terms of both generation and robustness of the learned representations. We introduce a regularization scheme to incorporate adversarially perturbed data points to the training pipeline without increasing the computational complex- ity or compromising the generation fidelity when compared to the robust VAEs by leveraging loss based on the two-point Kolmogorov-Smmimoo test between representations. We conduct extensive experimental studies on popular image benchmark datasets quantify the robustness of the proposed approach based on the adversarial attacks targeted at VAFs. Our empirical findings show that the proposed method achieves significant performance in both robustness and fidelity when compared to the robust VAE models. An implementation available at http://tttubb.ccooooccceeeeeeeee wwwpsrrcchhoouutttooo DMMD_DDAt__GMM_DEEEEEEEE
Introduction
Variational autoencoders (VAEs) offer powerful probabilistic framework to learn deep generative models. They are widely employed in various domains such as computer vision, natural language processing and representation learning 24 but independently parameterized models, an encoder or a recognition model that computes a latent representation of the input space with stochastic latent variables and decoder
16
25
26
10]. A VAE is composed of two coupled
generative model to map the latent representations back to the original input space [18]. One of the major advantages of VAEs that they provide semantically meaningful latent representations of high- dimensional complex input distributions which can be further utilized for various downstream tasks [17 14 28 I.]. However, VAEs are often limited due theoretical and practical limitations such as over-regulaiization prior-posterior mismatch resulting in trade-off's between generation and reconstruction fidelity [1 9 [2] Recently proposed deterministic versions of VAEs offer promising alternatives to overcome these limitations [2] 27].
The semantically meaningful representations learned by VAEs still be corrupted by so-called adversarial attacks [3 [19 20) where even small but specifically crafted changes to the input can lead to very different reconstructions. This observation reveals a lack of generalization within such models and is therefore a serious concern with respect to many practical applications. While
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

is harder to attack VAEs when compared to classifier networks [13] it is essential to analyze the robustness of VAEs along with their generative performance, validate whether the learned latent representations are meaningful. Hence, there has been increasing research interest in the deep learning field towards training robust models, both for classifier models and for autoencoders, i.e. for robust representation learning. Borrowed from the training of robust classification models [22 [3,, the concept of adversarial training has proven to be able to smoothen the VAE encoder and improve the robustness of the learned representations [6]. Other attempts toward learning robust representation spaces introduce either complex network architectures or expensive regularization mechanisms to improve the robustness of VAEs [32 21. Further, it has been pointed out in previous works that the robustness of VAEs can be improved by generating disentangled latent representations or by encouraging the smoothness consistency of the encoding- -ecloding process [32 [5] However, regularizing the VAE objective to enhance robustness often leads to poor generation ability compared to its non-robust counterpart. Hence, seek to focus improving the robustness of autoencoders while still maintaining the generation performance.
In this work, we introduce a simple and easy to train deterministic autoencoder which exhibits superior performance in terms of both generation and adversarial robustness. We argue that the deter- ministic approach enhances the robustness of VAEs when the latent codes are properly regularized. Consequently, we extend the training objective of the multi-modal deterministic autoencoders [27] to incorporate adversarially perturbed input data points in the latent space. The proposed adversarial learning scheme is comparatively less expensive and casier to implement than existing adversarially trained robust VAE models. We conduct extensive experimental analysis to evaluate the robustness of the trained model on popular benchmarks such as MNIST, FASHIONMNIST, SVHN and CELEBA images. Our empirical evaluations show that the proposed model consistently exhibits high adversar- ial robustness and significantly better generation performance compared to state-of-the-ar robust VAE baseline models. We also show that by improving the robustness of the learned representations, a classifier trained on the learned latent space of the model also exhibits better robustness.
2 Related Work
The ability to defend against adversarial attacks closely related to the sensitivity of the learned latent representations to slight changes in the input data points. In this section we first discuss seminal works proposed to generate rich and meaningful latent codes. This followed by the review on adversaries for VAEs and strategies proposed to defend against such adversarial attacks.
Variational Autoencoders (VAEs) In standard VAEs, the encoder and the decoder are jointly trained to maximize the evidence lower bound (ELBO) of the log likelihood of the model. The resulting training objective is combination of a reconstruction loss and the Kullback-Leibler (KL) divergence between the learned latent representations and a chosen prior usually a uni-modal Gaussian normal distribution. Higgins et.al (16) improved the learned representation in VAEs by encouraging disentanglement in the latent space and introduces 3 8-VAE for disentangled factor learning. V-VAE modifies VAEs by introducing hyperparameter to balance the latent regularization term with the reconstruction performance. Chen et.al 7 further decompose the ELBO term in VAEs to introduce total correlation (TC) regularization and propose 3-TCVAE as a promising alternative to 3- VAEE.. Willets et.al. 322 show that addition of TC term to the VAE objective also improves the robustness of VAEs. Several methods were proposed to include complex and flexible priors to the training pipeline of VAEs to enhance the semantics of the learned latent representations R 35 31.
Miao et.al. 23 introduces an approach to incorporate the inductive bias into VAEs without ex explicitly changing the prior by utilizing intermediary set of latent variables. Hierarchical VAES (26 extend the standard VAE framework by introducing a hierarchy of latent variables and offer superior modelling capabilities. Gosh et.al. [12] question the variational formulation of VAEs and introduce a simple and effective deterministic model without any prior assumptions followed by a post-hoc density estimation to approximate the learned posterior. A multimodal version to this approach, that utilizes the capacity of expressive multi-modas latent distributions to yield high quality generation, was proposed in (27]
3
Adversarial attacks on EA Adversarial attacks targeted towards autoencoders were first discussed in 13 Common attacks on VAFS follow procedures similar to attacks against classifiers, i.e. they aim to maximize the network's 's loss. Usually, slight perturbations are added to the input images to make the reconstructions similar to a specific target image (targeted/supervised attack) or completely different

---

image (untargeteddamsupervvsee)) [3 19 such as to maximize the reconstruction loss. In [32 Willets et.al. show that applying the TC regularization introduced in TC-VAES to hierarchical VAEs yields robust representations. Although the resulting model improves adversarial robustness, the training complexity is high when compared to a VAE. Cemgil at.al [ relate the robustness of VA to the smoothness of the encoding process. Similarly to Madry et.al. [22], they introduce a regularization scheme based on a selection mechanism the latent space to generate additional data points to minimize the entropy regularized Wasserstein distance between latent representations. Following the same direction, Cemgil et.al. [5] argue that the lack of consistency between the gecoding-decoding process cause the susceptibility to adversarial attacks in VAES. In contrast to the previous works, Camuto et.al. [4] provide a theoretical insight to the robustness of VAEs and introduce a novel criterion for robustness in VAEs. Barret et.al. [2] propose to constrain the Lipschitz constants for both encoder and decoder to ensure certifiable adversarial robustness of VAFs. Although these methods comparatively improve the adversarial robustness of VAFs, they often accompanied by complex network architectures and expensive training procedures. In contrast, our approach adopts an inexpensive adversarial training scheme the latent space of deterministic autoencoders by elegant extension to the regularization proposed in [27 to ensure both robustness and fidelity.
3 Towards robust deterministic autoencoders
Deterministic Autoencoders 12 27
offer promising alternative VAEs for learning meaningful representations of complex input spaces with high fidelity. Motivated by this fact, we aim to further explore the robustness of the learned representations of the resulting model. We are particularly inter- ested in the multimodal prior setting in [27]. since a flexible and expressive Gaussian mixture model (GMM) prior assumption facilitates encoding similar data points closer together while distancing dissimilar points far in the latent space behaviour that has also been found to be beneficial in learning robust classifier models [1] Consequently, we propose to adopt the regularization technique proposed in [27] to regularize the learned latent representations of our model towards a predefined GMM prior. For model to be inherently robust, slight perturbations in the input space should not result in substantial variations in the encoding space and the corresponding reconstructions. This could be also attributed to the smoothness of the learned encoder. Hence, it is essential to investigate the smoothness of the learned representations and the reconstructions of the model. In the following subsections, we resume the regularization loss from [27] for the sake of completeness and illustrate how it can be extended to adversarial training samples.
3.1 Regularization of the learned representations - Encoder smoothness
Following the idea from [27] we aim to regularize the D-dimensional latent representations {zi...., 2N3 of input datapoints {xi,..., ., learned by the encoder g of the model towards k-modal GMM prior. For S K, let μ4 and Σg be the mean and covariance matrix of the k-th mode
in the model. Further, let wy >> 0 be the weight of the k-th mode, Σk= = 1. Inspired by the statistical Polmogorov-Smimon (KS) test, the authors propose regularize the empirical cumulative distribution function (CDF) of the latent samples towards the CDF ((z) of the prior distribution,
EE 1 m2) dl dl
exp
(2)
Wk
22T"
Exl
k=
where z = (zi...., 2D), (41 (p).
While the original KS-test quantifies a metric between joint CDFs, the minimization of this KS- distance can be approximated by marginalization, i.c. by regularizing the empirical marginal CDFs
1 N
Pa(z)=
Mp.las
(2)
x=1
of the latent representations (21,..,,NN) 2N) in dimensions d = 1,..., D towards the marginal CDFs
K
K 12- Mkkd
FGMM.d(=) = Σ
Pk Φ
Σ
(3)
σk,d
kml
im
3

---

of the GMM prior. Here, Fl indicates the d-th dimension of a vector, i.c. [pgld is the d-th entry of the mean vector H4g (fs.1..... HR.D) of mixture component k, i.e. a scalar. Similarly, Ha, indicates the entry in row and column d of the matrix Σg = diag 1.,...... .k..)) Ok,D) The respective loss is computed as
D MSE
C D
(Fa((zula), FoMM,,
An additional loss compensates for the ambiguities on the target distribution introduced by the marginalization, i.e.
Ccv.v(z.....))
1
Σ E
[Eeed edd
[EGMM]ed le,)) )
(5)
D2
with EGMM -k= wh W + +E We (14 (1) (IE 1²)² 1 E My and Σ being the empirical covariance matrix of the latent representations.
The total loss of the model in [27] is a combination of both regularization terms to enforce the latent representations of the encoded data to match a predefined multi-modal prior distribution and a reconstruction loss. Motivated by (Л]. for equidistantly chosen modes μt; in Figure [ we expect such a model to already provide not only an improved generation fidelity as observed 127 but also more robust behavior. We provide empirical evidence for this conjecture in an ablation study in Section 4. The GMM regularization implicitly clusters latent points such that similar points are close to one another while dissimilar points distant.
To further improve the robustness of our model we employ adversarial training a widely popular defense strategy utilized to learn robust deep networks [22 (3). That is, during training we utilize adversarial samples zadv that are explicitly optimized to fall in an c- ball around the original latent representations 2n for S N and to decode to damaged was observed that such adversarial samples tend to explore the underrepresented regions the latent space. In the following, we extend the losses in 4 and (5) to overcome this undesired behaviour. This approach allows for a cheaper yet very effective adversarial training, while preserving the reconstruction and generation ability of the original model from [27].
semantically altered images. In [ it
3.2 Adversarial training data augmentation Improving Robustness
To generate adversarial inputs, we adapt the fast gradient sign method 33 to the latent space of the model. For a given e >> 0 and datapoint X,, the objective of the attack is to find the corresponding adversary xddv that would introduce maximum distance in the encoding space. That is, xadv X,, + 8x.,, where 8x, is the solution to the optimization problem
arg max (x,, + 6) - g(x,)1l2 s.t. ||0]| S,
(6)
where g is the encoder of the model. To prevent adversarial samples from exploring unexplored
regions of the latent space, assume that the joint distribution of latent encodings (z.,, z.dv (here
2, g(xn) and 2nd7 g(Hn )z..)) of data points and their adversarial samples (Xn, x xadv) ) to
follow the same modal GMM prior (Figure []. One possible straight forward extension of
the approach in (27] would be to consider ad adversarial examples
specific data augmentation and
regularize I and gadv
I,.,,N to the same GMM prior independently and ignoring cross-covariance
between 2I....N and " (here the off-diagonal elements in the covariance matrix of the GMM prior, that the last matrix mentioned equation (8), are zero). The corresponding losses in eqs. and (5) take the form
1
1
adv v) (21..() [Ks,, L (Z....,,N Nr, I.....)
and
2D
Σ
Σ cross
Y
GMM
L (Z,....,, N. adv xp
4D2
Σ
cToss
add
Σ
GMM
(8)
f,d=1
e,d

---

Left: Learned latent representations in deterministic autoencoder regularized towards GMM prior with two components (blue shaded regions). Consider a set of latent points Z 21.-... N (blue dots), in subspace (green shaded region) within component and the corresponding adversarial examples zidv, ..... n (red crosses). The adversarial examples tend to explore regions not covered by the input samples. If we assume that z and z zadv follow the same prior assumptions independently, the adversarial examples tend to move closer the original samples (blue crosses). In the worst case scenario, an adversarial example might reside in a different component. Right: By establishing a strong coupling via 2-point KS-distance regularization, the adversarial examples tend to move more closer to the original samples (orange crosses) after regularization.
Figure 1:
where Σ and Σnd add are the empirical covariance matrices of the latent representations 1 and their adversaries 2i...... respectively, and Eeross is the empirical cross- covariicee between benign and adversarial samples. While such a regularization preserves the overall distribution even under adversarial attacks, it can not control the distance of specific adversarial sample to its benign Zn- In the worst case scenario, an adversarial example z can be mapped to a different Gaussian mixture component than 2, and therefore cause maximum damage in the reconstruction shown in Figure Ileft).
To evenly spread out the learned representations, inject Gaussian noise to the latent vectors during training. Let X be the output of the decoder at Zn + en, where 6, (0, Ip). The reconstruction loss equals the mean squared error between inputs x,, and their noisy reconstructions X.,
3.3 Coupling and zadv Two-Point KS-distance loss
To ensure that the adversarial examples remain in close proximity to the original mapping in the learned latent space, establish strong coupling between the two distributions, 21....N and zadv "i.....N Hence we propose to match the the empirical CDFs of 2i....N and zadv and introduce novel regularization based the two-point KS-test [29]. By analogy the one-point KS-test that tests whether a sample is drawn from a given, continuous distribution, the two-sample KS test determines whether two samples with empirical CDF drawn from the same distribution. To this end, the two-sample KS test evaluates the supremum of the distance between the two CDFs. Here, we propose to minimize this distance computed from the marginalized two-point KS-test in order to align the distributions of benign points and their adversaries. The resulting loss is consistent with the previous regularization and allows efficiently establish the desired coupling between the representations. The first regularization loss of the adversarially extended model with pairwise coupling takes the following form
LKS.A (Z) CI..... Ns (
nan KS (Z),... C,..,, N, ( + 3D
adv
Σ
MSE (Fa(zznJa), Fadv ([2ndy) Ia)) n=1
where Fa, Fudv the empirical CDFs of and adv respectively.
The correlations between the latent representations and their adversarial samples need to be considered separately. The degree strength of the coupling is controlled by coupling parameter a | 1, where indicates the border condition where z adv The covariance loss of the extended
5

---

model becomes,
Σ
E croes
E DGMM
Q
Lady & (B (...,,)) 4D2
Crooss
Σ
Dady
EGMM
Edd
E,d
(10)
The total training objective of the model takes the following form,
adv [X addv I.....N) - ARRECREECX + AKS (2I....N N: I.....N++
Cady
Acv Cev. LV, (21, NN,,..,, w).... (11)
The weights ÅREC, Äks and λcv can be calculated by taking the statistics of samples from the GMM prior (27). see Appendix.
Experiments
We conduct an extensive experimental analysis to evaluate the robustness of the proposed model. We consider the state of the art latent space attacks targeted at VAEs [[3] 19 2] and evaluate the robustness based on the quantitative metrics as described in Section 4. Since the latent spaces of VAEs are often further utilized for various downstream applications, we also consider the impact of such adversarial attacks on classifier trained the latent space of the model. To evaluate the fidelity of the learned representations, we report the Fréchet inception distance (FID) [15] of the generated images. Our model is compared with the following baseline models, Variational Autoencoder (VAE) [18) 3-VAE (16), 3-TCVAE (7]. LipschitzVAE (2). Smooth Encoders (SE) 66] and Autoencoding Variational Autoencoder (AVAE) [ The experimental study is conducted on important image benchmark datasets such as MNIST, FASHIONMNIST, SVHN and CELEBA. For simplicity we consider a fully connected network architecture for experiments on MNIST and FASHIONMNIST images and convolutional architecture for experiments on SVHN and CELEBA images. Further details of the experiments are given in the Appendix
Adversarial attacks. Adversarial attacks targeted at VAFs attempt to add small noise perturbations to the input data points that fool the model to reconstruct the input image to a target adversarial image completely different image. Following recent literature 2 (2). we consider two types of adversarial attacks in our experiments.
Latent space attacks or supervised attacks are considered the most effective mode of attacks on VAEs. Here, the attacker tries add a noise perturbation 8 to a data point x, such that the latent representation Z of the perturbed input x + 5 is close to the latent representation z, of a chosen target adversarial image, Xt. The attack involves solving the following optimization problem,
arg min [1 222 |
(12)
Further, we consider maximum damage or output space attack. In this setting, the adversary perturbs the input data point to cause maximum damage in the reconstruction of the decoder f of the model and optimizes the following objective,
arg max [lf(ze4)- f(z=) 12-
(13)
In both scenarios, the noise perturbation is explicitly constrained by some constant À > 0 to ensure consistent comparison with the baseline models.
Evaluation. The qualitative results for CELEBA images for latent space and maximum damage attack are shown in Figure 3 We consider three evaluation metrics to quantitatively estimate the robustness of the different models for the above-mentionel adversarial attacks. First, we evaluate the attack loss of the adversary. That is, we report the achieved value of the optimization objectives in (1) and (13). The observed attack losses for the two forms of adversarial attacks are shown in Figure 2 (MNIST results are given the Appendix). Higher attack losses correspond to less
All the conducted experiments were part of carbon- neutral framework based GPU cluster and hence did not contribute climate change.

---

FASHIONMNIST
SVHN
CELEBA
VIA
AMA 5
TOAA
Latent space attack.
VAR
1X0
VAE DAATOVAAE
BESETOVAR
AVAZ
(b) Maximum damage attack.
Figure 2: Observed attack losses for (a) latent space attack (eqn (12) and (b) maximum damage attack (eqn (13) with varying À values. We report the observed mean and standard deviation by attacking 100 randomly chosen test images in 10 different trials. Higher loss indicates more robustness.
successful attacks and hence better robustness. Due the different regularization methods used in the baseline models, the inherent scale of the aggregated posterior changes and hence the value of the attack objective in eqn [2] Hence for latent space attacks, the values reported in Figure [ might not be directly comparable. Another strong indicator for the robustness of the considered models is the similarity between the reference image X, and the reconstructions X, of the adversarially perturbed variants X.. In the case of latent space attacks, we consider the target image x,- X, as the reference images and compare with the reconstruction X. f(zx+6) of the perturbed latent representation. For a maximum damage attack, we consider the original images x, = as the reference images and compare them to Xx. Similar to [20]. we use the perception- based similarity metric Multi-Scale Structural Similarity Index Measure (MSSSIM E [0. 1] to compare the images. We also report the ly-distance as an alternative similarity metric to MSSSIM in the Appendix. Lower values of MSSSIM(2,, af,)) indicate less similarity between the reference (target) image and the adversarial reconstructions and correspond to less successful latent space attacks. And higher value of MSSSIM ) corresponds to less successful maximum damage attacks as they correspond to high similarity between reference (original) and the adversarial reconstructions. We report the observed values in Table [ Finally, it is also essential to evaluate how similar the adversarial images are to the original image. Ideally, a successful attack implics that both adversarial and original images look similar in appearance (MSSSIM x 1). Hence we consider the MSSSIM between the original and adversarial images in Table I fidelity of the representations.
We also report the FID of the generated samples to compare the
Results. Overall we see that the proposed model outperforms all the considered baselines in terms of both robustness and offers superior generation performance. Even for complex datasets like SVHN and CELEBA, we observe the same trend with FID and robustness measures. These results are especially promising since we did not employ any extensive hyperparameter search for training. Our results further confirm that both robust and high fidelity models are possible. Since we employ an FGSM based adversarial training, the training time required is cheaper when compared to the expensive PGD-based training used in smooth encoders (SE). The computation time for a single iteration of SEs is two times more when compared to our method.
Ablation Study. In this section, we compare three different variants of regularized deterministic autoencoders to evaluate the importance of joint regularization of the original and adversarial samples. We begin with the model proposed in [27), which we denote as GMM-DAE. Second, we study the augmented model defined by equations (7) and (8) in Section 3.2, but without the coupling of original and adversarial latent representations (Augmented). We compare the robustness and fidelity of these models with our proposed model, i.c. where original and adversarial latent representations are not
7

---

Table 1:
Robustness evaluation across dataset similarities between images the event of latent space and maximum damage attacks in terms of MSSSIM. Here randomly chosen 100 test images are attacked in different trials. Σ, refers to reference image, 2a to adversarial image and 2,, To to their corresponding reconstructions. The maximum input noise perturbation levels À are limited to 1, 3 and Fidelity analysis based on the FID of the generated images.
MNIST
Latent space atinck
Maximum damage attack
FIDX))
MSSSIM(X, X. XL)
MSSSIM &a K()
MSSSIM(X xx1(1)
MSSSIM( xx)()
3
5
3
3
VAE
0.55
0.78
0.89
0.99
0.94
0.87
0.64
0.27
0.08
0.93
43.21
3-VAE
0.52
0.73
0.86
0.99
0.92
0.86
0.65
0.36
0.22
0.98
0.93
0.87
12.72
B-TCVAE
0.53
0.83
0.92
0.73
0.28
0.93
LiscchizVAE
0.50
0.68
0.79
0.98
0.93
0.89
0.75
0.41
0.33
0.98
0.93
0.86
59.45
0.49
0.62
0.68
0.98
0.92
0.86
0.90
0.60
0.54
0.98
0.93
0.86
47.34
AWA
0.49
0.59
0.62
0.98
0.91
0.83
0.80
0.65
0.59
0.97
0.89
0.86
48.47
FASHIONMNIST
Latent space attack
Maximum damage attack
FIX(L)
MSSSIM x.X()
MSSSIM(X, x. K()
MSSSIM(X... x. (())
MSSSIM x. XL)
3
5
3
VAE
0.61
0.65
0.71
0.98
0.91
0.82
0.58
0.29
0.13
0.99
0.94
0.87
70.22
R 8-VAF
0.59
0.61
0.68
0.98
0.92
0.82
0.66
0.32
0.15
0.99
0.94
0.85
73.82
B-TCVAE
0.55
0.64
0.92
0.69
0.27
0.94
73.94
LipschitzVA
0.13
0.59
0.67
0.99
0.91
0.89
0.71
0.31
0.30
0.99
0.94
0.88
79.45
0.24
0.43
0.53
0.98
0.92
0.81
0.90
0.62
0.43
0.99
0.94
0.86
72.29
AVAE
0.32
0.34
0.35
0.98
0.92
0.82
0.79
0.52
0.45
0.99
0.94
0.92
74.45
0.39
SVHN
Latent space attack
Maximum damage attack
FIN(L)
MS x.X()
MSSSIMGK, x, X4)
MSSSIM x,
MSSSIM x. XL)
3
3
3
0.55
0.93
8-VAF
0.44
0.70
0.81
0.99
0.89
0.77
0.52
0.49
0.47
0.99
0.93
0.88
61.65
B-TCVAE
0.39
0.72
0.89
0.63
0.54
02.59
LipschitvVA
0.35
0.62
0.71
0.99
0.89
0.76
0.66
0.65
0.55
0.99
0.93
0.88
65.58
SE
0.19
0.33
0.34
0.99
0.92
0.81
0.79
0.69
0.60
0.99
0.96
0.94
61.28
Own
0.16
0.26
0.28
0.98
0.77
0.76
0.84
0.79
0.75
0.98
0.92
0.86
38.89
CELEBA
Latent space atlack
Maxinum damage altack
FIX
MSSSIMIX.. X.)
MSSSIMMK... x. x.K()
MSSSIM(X. xx((t)
MSSSIMIX.. x,)(L
3
VAE
U.59
0.60
0.66
0.99
0.99
0.97
0.61
0.58
0.55
0.99
0.98
0.988
69.48
0.99
0.59
0.98
--TCAAR
0.54
0.51
0.61
0.99
0.99
0.97
0.76
0.71
0.64
0.98
0.99
0.96
75.11
LipschitZVAE
0.49
0.55
0.99
0.98
0.73
0.70
0.64
0.98
0.96
77.89
SE
0.27
0.31
0.34
0.99
0.98
0.96
0.97
0.91
0.76
0.99
0.98
0.98
72.68
$1.98
Table 2: Ablation study on MNIST images. Augmented refers to the model definition in eqs Here Hy- refers reference image, va to adversarial image and of,, 2i to their corresponding reconstructions. The maximum input noise perturbation level is limited to 1, and 5.
Method
Latent space attack
Maximum damage attack FID(1)
MSSSIM d. (()
MSSSIM F.)(t)
5
5
GMM-DAE 27
0.54
0.70
0.82
0.75
0.37
0.30
38,89
Augmented
0.47
0.59
0.71
0.79
0.54
40.16
Ours
0.38
0.47
0.60
0.92
0.82
0.69
39.37

---

Latent space attack
Maximum damage attack
(a) VAE.
(b) 6-TCVAE.
(c) LipschitzVAE.
(d) SE.
(1) Ours.
Figure 3: Visual appraisal of attacks Qualitative analysis across models on CELEBA images with maximum input noise perturbation level À limited to 5 for latent space and maximum damage attack. (from left to right) Images in each row correspond to input image(x), clean reconstruction, adversarial image(Tr). adversarial reconstruction(x.. and target image(x;) (for latent space attacks) respectively,
Table 3:
Robustness of downstream classifier trained in the latent space of the model under adversarial attack we report the clean accuracy and the accuracy during attack defined in eqn 12 for X
Method
MNIST
FASHIONMNIST
SVHN
clean acc.(f)
À 1(1)
clean acc.(f)
À = 1(t)
clean acc
À = 1(t)
VAE
92.16
58.85
80.65
59.15
61.70
25.63
3-TCVAE
93.02
61.06
81.25
60.77
62.02
30.
LipschitzVAE
90.78
62.00
80.50
62.06
60.99
33.99
SE
93.81
68.65
80.10
66.83
62.36
44.40
Ours
96.08
91.78
85.96
78.86
70.96
59.20
only regularized towards the same prior but coupled according equations (9), (10), and (11) in Section 3.3 (Ours). The observed metrics are reported Table We observe that proposed method yields comparatively better performance in terms of robustness while still maintaining the generation fidelity when compared to the non-robust version, GMM-DAE. can also seen that enforcing coupling between the latent representations of the original and adversarial samples (Ours) leads to better performance than simply augmenting them (Augmented). is worth pointing out that the GMM-DAE maintains better performance than standard VAE model (Table 1, MNIST VAE results). This is due to the well-structured latent space GMM-DAE. This observation further confirms hypothesis in Section 3 that when lomilar-looking samples modelled together in the latent space of the model, robustness can be improved.
Robustness to downstream applications. Since the learned representations of VAEs are often used for various downstream tasks, it is also vital verify how adversarial attacks affect the performance of the same. To showcase the effect of adversarial attacks to downstream classification tasks,

---

train a MLP classifier on the latent space of the model and observe the accuracy drop in the presence of the latent space attacks for a constrained noise norm À 1. The observed values along with the clean accuracy are shown in Table 3 The classifier trained on the latent space of the proposed model achieves better accuracies when compared to the baseline models under latent space attack.
Discussion & Conclusion
Limitations. Although cheaper when compared to the current adversarially trained robust VAE models, the FGSM based adversarial training scheme is still expensive when compared to the non- robust counterpart. The coupling parameter in the proposed method is additional hyperparameter to tune and enforcing strong coupling i.e. a = 1, might compromise the generation fidelity of the resulting model. While need the coupling to perform adversarial training, overly strong coupling will necessarily lead to deteriorated reconstructions, similar as in VAEs. It is worth noting that we did perform intensive hyperparameter optimization in this line of experiments. Hyperparameter optimization might be needed in other datasets for this reason. Further, datasets with large inter class variance might not benefit from the underlying multimodal prior assumptions in the current approach. It would be interesting to explore the impact of the adversarial attacks in other potential downstream applications such as optimization in the latent space VAE models.
Potential societal impact. Variational autoencoders enable learning meaningful representations of complex high dimensional data without any supervision. This further enhances the usability of these learned representations for various downstream tasks and applications where limited data is available for example, due to privacy/security concerns. Hence, it important to study the robustness of these representations along with their accuracy, especially when employed in real-world applications. In the present paper, we propose method to train robust VAEs with high fidelity in the learned latent space. Since our work is step towards more robust models, we hope to see a positive social impact. There currently limited work in this direction and we believe that our method encourages potential future work in developing robust VAE models. On the other hand, we should take into consideration the possible negative social impact of this research, especially in safety-critical applications. Although we observe superior robustness in our model against the existing attacks, similar performance cannot be guaranteed on newly discovered attacks on VAEs. Hence, when deployed in real-world applications we highly recommend testing the model continuously against newly designed attacks. We also urge the machine learning community to pursue this work responsibly to enable potential future research without any misuse.
Developing robust VAE models is crucial since the learned representations of VAEs are frequently used for various applications. Motivated by the recent research towards deterministic alternatives VAEs, we study the robustness of deterministic autoencoders. We extend recently developed regularization schemes to efficiently couple the adversarial examples and the learned representations during training. Our experiments show that with proper regularization, adversarially trained multimodal deterministic autoencoders offer significantly improved adversarial robustness and high fidelity in the learned latent space.
Conclusion.
10

---

References
[1] L. Amsaleg, J. Bailey, A. Barbe, S. M. Erfani, T. Furon, M. E. Houle, M. Radovanović, and
X. V. Nguyen. High intrinsic dimensionality facilitates adversarial attack: Theoretical evidence. IEEE Transactions on Information Forensics and Security, 16:854 865, 2021.
[2] B. Barrett, A. Camuto, M. Willetts, and T. Rainforth. Certifiably robust variational autoencoders.
In NeurlPS 202/ Workshop on Deep Generative Models and Downstream Applications, 2021.
[3] L. Maaloe C. K. Sønderby, T. Raiko and, S. K. Sønderby, and O. Winther. Ladder variational autoencoders. In Proceedings of the 30th International Conference Neural Information Processing Systems, NIPS' 16, page 3745-3753, 2016.
[4] A. Camuto, M. Willetts, S. J. Roberts, C. C. Holmes, and T. Rainforth. Towards a theoretical understanding of the robustness of variational autoencoders. In AISTATS, 2021.
[5] A. Cemgil, S. Ghaisas, K. Dvijotham, S. Gowal, and P. Kohli. The autoencoding variational autoencoder. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS 20, 2020.
[6] T. A. Cemgil, S. Ghaisas, K. (Dj) Dvijotham, and P. Kohli. Adversarially robust representations with smooth encoders. In International Conference on Learning Representations, 2020.
[7] R. T. Q. Chen, X. Li, R. Grosse, and D. Duvenaud. Isolating sources of disentanglement vaes. NIPS'18 18, page 2615-2625, 2018.
[8] X. Chen, D. P. Kingma, T. Salimans, Y. Duan, P. Dhariwal, J. Schulman, I. Sutskever, and Abbeel. Variational lossy autoencoder. ArXiv, abs/1611 .02731, 2017.
[9] B. Dai and D. Wipf. Diagnosing and enhancing VAE models. In International Conference on Learning Representations, 2019.
[10] Z. Ding, Y. Xu, W. Xu, G. Parmar, Y. Yang, M. Welling, and Z. Tu. Guided variational autoencoder for disentanglement learning. In 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 7917-7926, 2020.
[11] P. Ghosh, A. Losalka, and M. J. Black. Resisting adversarial attacks using gaussian mix- ture variational autoencoders. In Proceedings of the Thirty-Third AAAI Conference on Ar- tificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Con- ference and Ninth AAAI Symposium on Educational Advances Artificial Intelligence, AAAI Press, 2019.
[12] P. Ghosh, M. S. M. Sajjadi, A. Vergari, M. Black, and B. Scholkopf. From variational to deterministic autoencoders. In International Conference on Learning Representations, 2020.
[13] G. Gondim-Ribeing P. Tabacof, and E. Valle. Adversarial attacks on variational autoencoders. ArXiv, abs/l 8061100.000 600.04446, 2018.
[14] D. R. Ha and J. Schmidhuber. World models. ArXiv, abs/1803.10112. 2018.
[15] M. Heusel, H. Ramsauer, Т. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge a local nash equilibrium. In NIPS, 2017.
[16] Higgins, Matthey, A. Pal, P. Burgess, X. Glorot, M. M. Botvinick, Mohamed, and
A. Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework.
In ICLR, 2017.
[17] I. Higgins, A. Pal, A. Rusu, L.. Matthey, C. Burgess, A. Pritzel, M. Botvinick, C. Blundell, and A. Lerchner. Darla: Improving zero-shot transfer in reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, page 1480-1490. JMLR.org. 2017.
[18) D. P. Kingma and M. Welling. An introduction to variational autoencoders. Found. Trends Mach. Learn., 12:307-392, 2019.
11

---

[19] J. Kos, I. Fischer, and D. Song. Adversarial examples for generative models. 2018 IEEE Security and Privacy Workshops (SPW), pages 36-42, 2018.
[20] A. Kuzina, M. Welling, and M. J. Tomezak. Diagnosing vulnerability of variational auto- encoders to adversarial attacks. RobusIML Workshop@ ICLR 2021, 2021.
[21] L. Maaløe, M. Fraccaro, V. Liévin, and O. Winther. Biva: A very deep hierarchy of latent variables for generative modeling. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019.
[22] A. Madry, A. Makelov, Schmidt, D. Tsipras, and Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
[23] N. Miao, E. Mathieu, N. Siddharth, Y. W. Teh, and T. Rainforth. Intel-vaes: Adding inductive biases to variational auto-encoders via intermediary latents. ArXiv, abs/2106. 13746, 2021.
[24] Y. Miao, L. Yu, and P. Blunsom. Neural variational inference for text processing. Proceedings of the 33rd International Conference International Conference Machine Learning Volume 48, ICML' 16, page 1727-1736. JMLR.org, 2016.
[25] A. Razavi, A. V. D. Oord, and O. Vinyals. Generating diverse high- figh-fidelity images with vq-vae-2. In Proceedings the 33rd International Conference on Neural Information Processing Systems, 2019.
[26] D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference deep generative models. In Proceedings of the 31st International Conference on International Conference on Machine Learning Volume 32, ICML' 14,, page I1-1278 1----II-225 1286. JMLR.org, 2014.
[27] A. Saseendran, K. Skubeh, S. Falkner, and M. Keuper. Shape your space: gaussian mixture regularization approach to deterministic autoencoders. In Advances in Neural Information Processing Systems, 2021.
[28] L. Schott, J. Rauber, W. Brendel, and M. Bethge. Towards the first adversarially robust neural network model mnist. In International Conference Learning Representations (ICLR), May 2019.
[29] N.V. Smirnov. Approximate distribution laws for random variables, constructed from empirical data" uspekhi mat. Nauk, 10:179-206. In Russian.
[30] Tabacof, Tavares, and E. Valle. Adversarial images for variational autoencoders. ArXiv, abs/161..00155, 2016.
[31] J. M. Tomczak and M. Welling, Vae with vampprior. In AISTATS, 2018.
[32] M. Willetts, A. Camuto, T. Rainforth, S Roberts, and C. Holmes. Improving (vae]s' robustness
to adversarial attack. In International Conference on Learning Representations, 2021.
[33] E. Wong, Rice, and 7. Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2020.
[34] Zhao. Song, and S. Ermon. Learning hierarchical features from deep generative models. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML'17. page 4091-4099. IMLR.org, 2017.
[35] B. Zong, Q Song, M. Renqiang M., W. Cheng, C. Lumezanu, k. Cho, and H. Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In ICLR, 2018.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section
(b) Did you include complete proofs of all theoretical results? [Yes] See Section
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes] See Appendix
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes] See Appendix
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes] See Appendix
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work existing assets, did you citep the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See Appendix
(c) Did you include any new assets either the supplemental material or as URL? [Yes] See Appendix
(d) Did you discuss whether and how consent was obtained from people whose data you'le rou're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "ywT6DgyjiYa", "reviewer": "Reviewer_Xb52", "summary": "The authors extend existing methods for training variational autoencoders to be robust against adversarial attacks. They implement this by training on adversarial examples during training. They show that the resulting networks are less susceptible to adversarial attacks have better MSSSIM metrics and finally that the latent representations are better suited for downstream classification tasks.", "questions": "The authors claim that their method does not increase the computational complexity, yet they use adversarial examples during training. Should the generation of the adversarial examples not increase the computational complexity?\n\nThe authors should provide more details on the sections \"ablation study\" and \"robustness to downstream applications\". Here the text is not clear what they did. I do not understand what the \"ablation study\" is exactly about and for the downstream applications it might help to know what kind of MLP the authors are using.\n\nThe authors present how their method improves the robustness against adversarial attacks but they show little how this affects the baseline performance. How is the MSSSIM for x_r and x_r tilde? Does the new approach provide similar decoded image quality? Also would not MSSSIM for x_a and x_a tile be interesting, how well the adversarial images can be reconstructed by the autoencoder? \n\nThe authors say that the adversarial samples should not explore unexplored regions of the latent space. I do not quite understand why this is beneficial. Shouldn't the newly generated samples try to explore more regions then the normal samples already explore?", "limitations": "The authors discuss the limitations of their work mainly in terms of computational power and additional hyper parameters that need to be tuned and how datasets with different distributions might not work as well with the approach.\n\nThe authors do not discuss potential negative societal impact of their work and just state N/A. Surely if their work is relevant, it should have a potential societal impact of which a part can also be negative. The authors could elaborate a bit on this to address this point.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Here I have to admit that I am not very familiar with the current literature and the math behind the statistics of variational autoencoders. I have some difficulties understanding the exact novelty of the approach the authors have taken. It seems the basic idea is to prevent adversarial input examples during training and ensure some statistical properties of these examples.\nBut I assume the method is explained well for a reader with the sufficient background.\n\nWhat comes a bit short tough is the explanation of the experiments under the headers \"Ablation Study\" and \"Robustness to downstream applications\". There the authors seem to omit the details.\n\nThe results seem to lead to a significant improvement in the results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "dPg7UjEUsms", "reviewer": "Reviewer_Pa7M", "summary": "This paper proposes a technique to improve the robustness of VAEs. It builds upon earlier work by Saseendran et al, Ghosh et al. which provide a deterministic alternative to the variational formulation of VAEs. The formulation by Saseendran et al. uses the Kolmogorov-Smirnov (KS) test to regularize the latent space of deterministic auto encoders and impose a multi modal gaussian prior. \nThis paper extends this idea by imposing the same multi modal gaussian prior on the pair ($z_n$, $z^{adv}_n$) where $z^{adv}_n$ is the adversarial latent code obtained via perturbation at the input image level (added before the encoder). This is as opposed to just imposing the prior on $z_n$ like in earlier work.\nThis means that the regularizer goes from: the one point KS test used in Saseendran et al. to a two sample KS test between latent codes and their adversarial versions.\nThe models used in the paper are VAE, beta VAE, Smooth Encoder, AVAE etc evaluated on MNIST, FashionMNIST, SVHN, CelebA. \nThere are two attack vectors considered:\n1. Latent space attack: adding perturbation to an image such that its latent code moves close to the latent code of a target image.\n2. Maximum damage attack: adding perturbation to an image such that its reconstruction its now far away from its vanilla reconstruction\nResults show that there is consistent improvement in robustness across the board. This is evidenced by close similarity scores between image reconstructions. ", "questions": "1. A it stands, the paper does not have a single image about what samples look like or even what adversarial example would look like for the model. I think having pictures for these kind of papers make them read better.\n2. I found the paper fairly clear overall except the part in the evaluation subsection of Section 4, where the authors introduce a bunch of symbols at the same time. I think having clear definitions for each of reference image ($x_r$), target image ($x_t$) and adversarial image ($x_a$) in addition to clarifying the relationship between each of them would clear things up. (Maybe a diagram/image?)\n3. It seems that the number of modes in dataset has to match those in the prior. I would like to see some comments on this from the authors. What happens if the number of modes don’t match? We know from Saseendran et al. that image quality drops but does robustness also fall in a similar fashion? Or maybe robustness falls faster or slower than image quality?\n\nTypo:\nRef [26] does not have paper name. Listed several times in the paper. Key reference.\n", "limitations": "I think the authors should list that the number of modes in the prior has to match the number of modes in the dataset as a limitation. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Comprehensive Evaluation - The paper uses a variety of datasets to prove their point. A minor criticism could be that larger datasets are not considered (which is valid), however, datasets used in the paper are also used by prior work. \n2. Problem is well motivated and the exposition is clear \n3. Improving robustness in generative models is an important problem so this result is significant. Experiments show large improvements in FID so the results seem solid.\n\n\nWeaknesses:\n1. Unclear scope - How to pick modes to match the prior? What happens if they don't match?\n2. Section 4 seems a little unclear in an otherwise well written paper.\n3. The paper builds upon prior work in the area so the novelty of the technique is not very high. Having said that, it is an application of prior work into a new area which makes this point a less important.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_BaGPVntLGM", "reviewer": "Reviewer_qDc3", "summary": "This paper studies the robustness of deterministic autoencoders by introducing a regularization scheme to incorporate adversarially perturbed data points into the training pipeline. The proposed method increases the generation and robustness of the learned representations without increasing the computational complexity.", "questions": "About the sentence in lines 45 and 46, ``However, regularizing the VAE objective ..., Hence, ... ''. The traditional methods lead to poor reconstruction fidelity, while you aim at maintaining the generation performance? Not both the reconstruction and generation quality of the image?", "limitations": "Yes", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The writing is good, and the paper is easy to follow.\n2. The experiment part demonstrates the proposed method has a large improvement in robustness of the most time.\n\nWeaknesses:\n\nIn the paper, the author may need to show some qualitative results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "HolsdAfBbUs", "reviewer": "Reviewer_ayNS", "summary": "The paper proposes new approaches for improving the robustness of Variational Autoencoders (VAEs). The paper finds that deterministic autoencoder with multimodal prior (proposed in prior work) not only improves the fidelity of generated samples, but is also more robust to adversarial attacks. On top of that, the paper proposes to incorporate adversarial samples into the training process, and add regularizations to ensure that the latents from the adversarially perturbed samples also follow the same prior distribution. Results show that the proposed method achieves better fidelity and robustness across several datasets.", "questions": "\nI appreciate that the authors point out the downside of having a new hyperparameter alpha. However, I think it is still important to at least mention how alpha was tuned for all the experiments, and provide a sensitivity analysis of alpha. This would make it easier for people to adopt the method. I will adjust the score according to the response to this question.\n\n\n\nMy other questions are:\n\n* Table 1: I don't understand how you get x_r and x_a for maximum damage attack, and why lower MSSSIM(x_r,x_a) is better. I could not find related explanations or descriptions in the main text.\n\n* Line 158: z_n and z_n^{adv} are undefined. I suppose they are g(x_n) and g(x_n + \\delta_{x_n})?\n\n* Line 162: Does \"ignoring cross-covariance between z_{1,...,n} and z_{1,...,n}^{adv}\" mean the 0s in last matrix in Eq 8? Please make it more explicit\n\n* Eq. 6: g is undefined. I suppose it means the encoder?\n\n\n\nSome other minor typos and questions:\n\n* The title of the citation [26] is missing.\n\n* Figure 1 and its caption are hard to understand by themselves. It becomes clearer after I read Section 3.2, which appears much later in this paper.\n\n* Should the denominators of Eq 8 and Eq 10 be 4D^2 instead of 2D^2?\n\n* Line 171: left -> right\n\n* Table 2: Consider putting the best metrics of each column in bold?\n\n\n", "limitations": "\nThe paper adequately addressed the limitations. The paper did not discuss the potential negative societal impact of the work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nOriginality: The key techniques of this paper (incorporating adversarial samples in training, KS test) are simple adaptations from prior work. However, this specific combination for improving the robustness of VAE is interesting.\n\nQuality: Overall, the paper is technically sound, and most of the claims are well-supported by experiments.\n\nClarity: The paper is easy to read. However, there are some typos and some unclear statements. See questions below.\n\nSignificance: The paper could be of interest to practitioners as it shows that the proposed technique could improve the robustness without sacrificing fidelity.  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept