Original: Diffusion Visual Counterfactual Explanations
Revised: Diffusion Visual Counterfactual Explanations

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Visual Counterfactual Explanations (VCEs) are an important tool to understandthe decisions of an image classiﬁer.
Revised: Visual Counterfactual Explanations (VCEs) are an important tool to understand the decisions of an image classiﬁer.

Original: They are “small” but “realistic” semanticchanges of the image changing the classiﬁer decision.
Revised: They are “small” but “realistic” semantic changes of the image changing the classiﬁer decision.

Original: Current approaches forthe generation of VCEs are restricted to adversarially robust models and oftencontain non-realistic artefacts or are restricted to image classiﬁcation problemswith few classes.
Revised: Current approaches for the generation of VCEs are restricted to adversarially robust models and often contain non-realistic artefacts, or are limited to image classiﬁcation problems with few classes.

Original: In this paper, we overcome this by generating Diffusion Visual
Revised: In this paper, we overcome this by generating Diffusion Visual

Original: Counterfactual Explanations (DVCEs) for arbitrary ImageNet classiﬁers via adiffusion process.
Revised: Counterfactual Explanations (DVCEs) for arbitrary ImageNet classiﬁers via a diffusion process.

Original: Two modiﬁcations to the diffusion process are key for our
Revised: Two modiﬁcations to the diffusion process are key for our

Original: DVCEs: i) an adaptive parameterization allows us to have hyperparameters thatgeneralize across images and models and together with distance regularization andlate start of the diffusion process allow us to generateclose images and ii) our coneregularization via an adversarially robust model ensures that the diffusion processdoes not converge to trivial non-semantic changes but instead produces realisticimages of the target class which achieve high conﬁdence by the classiﬁer. 
Revised: DVCEs: ﬁrst, an adaptive parameterization, whose hyperparameters generalize across images and models, together with distance regularization and late start of the diffusion process, allows us to generate images with minimal semantic changes to the original ones but different classiﬁcation. Second, our cone regularization via an adversarially robust model ensures that the diffusion process does not converge to trivial non-semantic changes, but instead produces realistic images of the target class which achieve high conﬁdence by the classiﬁer. Code is available under https://github.com/valentyn1boreiko/DVCEs .

Original:  Introduction
Revised:  Introduction

Original: It can be argued that one of the main problems hindering the widespread use of machine learningis the missing possibility to explain the decisions of black-box models such as neural networks.
Revised: It can be argued that one of the main problems hindering the widespread use of machine learning and image classiﬁcation in particular, which is the focus of our paper, is the missing possibility to explain the decisions of black-box models such as neural networks.

Original: This is not only a problem for decisions affecting humans where the current draft for AI regulationin Europe [10] requires “transparency”, but it is a pressing problem in all applications of machinelearning.
Revised: This is not only a problem for decisions affecting humans where the current draft for AI regulation in Europe [10] requires “transparency”, but it is a pressing problem in all applications of machine learning.

Original: The reason, to some extent, is that humans would like to understand but also control if thelearning algorithm has captured the “concepts” of the underlying classes or if it just predicts wellusing spurious features, artefacts in the data set, or other sources of error.
Revised: The reason, to some extent, is that humans would like to understand but also control if the learning algorithm has captured the “concepts” of the underlying classes or if it just predicts well using spurious features, artefacts in the data set or other sources of error.

Original: In this paper, we focus onmodel-agnostic explanations which can be applied to any classiﬁer and do not rely on the speciﬁcstructure of the classiﬁer such as decision trees or linear classiﬁers.
Revised: In this paper, we focus on model-agnostic explanations which can be applied to any classiﬁer and do not rely on the speciﬁc structure of the classiﬁer such as decision trees or linear classiﬁers.

Original: In this area, in particular forimage classiﬁcation, sensitivity based explanations [4], explanations based on feature attributions [3],saliency maps [47, 46, 19, 57, 52], Shapley additive explanations [33], and local ﬁts of interpretablemodels [40] have been proposed.
Revised: In this area, in particular for image classiﬁcation, sensitivity based explanations [4], explanations based on feature attributions [3], saliency maps [48, 47, 18, 57, 52], Shapley additive explanations [32], and local ﬁts of interpretable models [40] have been proposed.

Original: Moreover, [56] proposed counterfactual explanations (CEs), which are instance-speciﬁc explanations.
Revised: Moreover, [56] proposed counterfactual explanations (CEs), which are instance-speciﬁc explanations.

Original: They can be applied to any classiﬁer and it has been argued that they are close to the humanjustiﬁcation of decisions [35] using counterfactual reasoning: “I would have recognized it as zebra(instead of horse), if it would have had black and white stripes.”
Revised: They can be applied to any classiﬁer and it has been argued that they are close to the human justiﬁcation of decisions [35] using counterfactual reasoning: “I would have recognized it as zebra (instead of horse) if it had had black and white stripes.”

Original: For a given classiﬁer almost allmethods for the generation of CEs [56, 16, 36, 6, 39, 55, 44] try to solve the following problem:“Given a target class c what is the minimal change  of input x , such that x +  is classiﬁed as classc with high probability and is a realistic instance of my data generating distribution?”
Revised: For a given classiﬁer almost all methods for the generation of CEs [56, 16, 36, 6, 39, 55, 45] try to solve the following problem: “Given a target class c what is the minimal change  of input x , such that x +  is classiﬁed as class c with high probability and is a realistic instance of my data generating distribution?”

Original: From theperspective of “debugging” existing machine learning models, CEs are interesting as they construct aconcrete input x +  with a different classiﬁcation that allows the developer to test if the model haslearned the correct features.
Revised: From the perspective of “debugging” existing machine learning models, CEs are interesting as they construct a concrete input x +  with a different classiﬁcation that allows the developer to test if the model has learned the correct features.

Original: [Figure]
Revised: [Figure]

Original: The main problem why visual counterfactual explanations (VCEs), that is CEs for image classiﬁcation,are not widely used is that the problem of generating CEs is very related to the problem of adversarialexamples [53] and thus even imperceivable changes of the image can already change the prediction,which clearly does not showto the user if the classiﬁer has picked up the right class-speciﬁc features.
Revised: The main problem with visual counterfactual explanations (VCEs), that is CEs for image classiﬁcation, and the reason why they are not widely used is that the tasks of generating CEs and the generation of adversarial examples [53] are very related. Thus even imperceivable changes of the image can already change the prediction of the classiﬁer, however, the resulting noise patterns do not show the user if the classiﬁer has picked up the right class-speciﬁc features.

Original: One possible solution is using adversarially robust models [43, 1, 7], which have been shown toproduce semantically meaningful VCEs by directly maximizing the probability of the target class inimage space.
Revised: One possible solution is using adversarially robust models [44, 1, 7], which have been shown to produce semantically meaningful VCEs by directly maximizing the probability of the target class in image space.

Original: These approaches have the downside that they only can generate VCEs for robust modelswhich is a signiﬁcant restriction as these models are not competitive in terms of prediction accuracy.
Revised: These approaches have the downside that they only can generate VCEs for robust models which is a signiﬁcant restriction as these models are not competitive in terms of prediction accuracy.

Original: The second approach is to restrict the generation of VCEs using a generative model or constrainingthe set of potential image manipulations [21, 22, 42, 8, 20, 45, 29].
Revised: The second approach is to restrict the generation of VCEs using a generative model or constraining the set of potential image manipulations [20, 21, 43, 8, 19, 46, 28].

Original: However, these approaches areeither restricted to datasets with a small number of classes, cannot provide explanations for arbitraryclassiﬁers, or the generated images look realistic but are far away from the original one, so that the VCE and the original image have so little in common that not much insight can be gained.
Revised: However, these approaches are either restricted to datasets with a small number of classes, cannot provide explanations for arbitrary classiﬁers, or the generated images look realistic but are far away from the original one so that the VCE and the original image have so little in common that not much insight can be gained.

Original: Recently, [28] trained a StyleGAN model to discover and manipulate class attributes.
Revised: Recently, [27] trained a StyleGAN2 [25] model to discover and manipulate class attributes.

Original: While theirapproach yields impressive results on smaller-scale datasets with few similar classes (for example,different bird species), the authors did not demonstrate that the method scales to complex tasks suchas ImageNet with hundreds of classes where different classes require different sets of attributes.
Revised: While their approach yields impressive results on smaller-scale datasets with few similar classes (for example, different bird species), the authors did not demonstrate that the method scales to complex tasks such as ImageNet with hundreds of classes where different classes require different sets of attributes.

Original: Another disadvantage is that the StyleGAN model needs to be retrained for every classiﬁer, making itprohibitively expensive to explain multiple large models.
Revised: Another disadvantage is that the StyleGAN model needs to be retrained for every classiﬁer, making it prohibitively expensive to explain multiple large models.

Original: Moreover, [27] have proposed a loss to do
Revised: Moreover, [26] have proposed a loss to do

Original: VCEs in the latent space of a GAN/VAE.
Revised: VCEs in the latent space of a GAN/VAE.

Original: While they show promising results for MNIST/FMNIST,there is no code available for generating VCEs on ImageNet.
Revised: While they show promising results for MNIST/FMNIST, those for ImageNet are not clear, and there is no code available.

Original: Furthermore, they rely on a conditional GAN, which might introduce a bias as it interferes with the classiﬁer, and it is unclear, how well anunconditional generative model will work.
Revised: Furthermore, to explain a classiﬁer, the conditional information during the generation of explanations should ideally come only from the classiﬁer itself, but [26] rely on a conditional GAN which might introduce a bias.

Original: In this paper, we overcome this by generating our Diffusion Visual Counterfactual Explanations(DVCEs) for arbitrary ImageNet classiﬁers.
Revised: In this paper, we overcome the aforementioned challenges by generating our Diffusion Visual Counterfactual Explanations (DVCEs) for arbitrary ImageNet classiﬁers (an example can be seen in Fig 1).

Original: We use the progress in the generation of realistic imagesusing diffusion processes [48, 26, 49, 51] reaching recently the level of GANs [15].
Revised: We use the progress in the generation of realistic images using diffusion processes [49, 24, 50, 51] reaching recently the level of GANs [15].

Original: Similarly to[38, 2], we use a classiﬁer and a distance-type regularization to guide the generation of the images.
Revised: Similarly to [38, 2], we use a classiﬁer and a distance-type regularization to guide the generation of the images.

Original: Two modiﬁcations to the diffusion process are key elements for our DVCEs: i) a combinationof distance regularization and starting point of the diffusion process together with an adaptivereparameterization lets us generate VCEs visually close to the original image in a controlled way sothat hyperparameters can be ﬁxed across images and even across models, ii) our cone regularizationof the gradient of the classiﬁer via an adversarially robust model ensures that the diffusion processdoes not converge to trivial non-semantic changes but instead produces realistic images of the targetclass which achieve high conﬁdence by the classiﬁer.
Revised: Two modiﬁcations to the diffusion process are key elements for our DVCEs: i) a combination of distance regularization and starting point of the diffusion process together with an adaptive reparameterization lets us generate VCEs visually close to the original image in a controlled way so that hyperparameters can be ﬁxed across images and even across models, ii) our cone regularization of the gradient of the classiﬁer via an adversarially robust model ensures that the diffusion process does not converge to trivial non-semantic changes but instead produces realistic images of the target class which achieve high conﬁdence by the classiﬁer.

Original: Our approach can  be employed for any datasetwhere a generative diffusion model and an adversarially robust model are available.
Revised: Our approach can in principle be employed for any dataset where a generative diffusion model and an adversarially robust model are available.

Original: In a qualitativecomparison and a user study in App. C we show that our DVCEs outperform the recent method of[7] as well as an adaptation of [2] for the generation of VCEs.
Revised: In a qualitative comparison (Sec. 4.1) and a user study (App. C) we show that our DVCEs outperform the recent method of [7] as well as an adaptation of [2] for the generation of VCEs.

Original:  Diffusion models
Revised:  Diffusion models

Original: Diffusion models [15,48, 49, 26, 37] are generative models that consist of two steps reverse diffusion process was closer investigated in [51].
Revised: Diffusion models [15, 49, 50, 24, 37] are generative models that consist of two steps: forward diffusion process (and thus a Markov process) that transforms the data distribution to a prior distribution (which is usually assumed to be a standard normal distribution) and the reverse diffusion process that transforms the prior distribution back to the data distribution. The existence of the (continuous-time) reverse diffusion process was closer investigated in [51].

Original: In the discrete-time setting, a Markov chain { x 1 , ..., x T } for any data point x 0 in the forward directionis deﬁned via a Markovian process that adds noise to the data point at each timestep:
Revised: In the discrete-time setting, a Markov chain { x 1 , ..., x T } for any data point x 0 in the forward direction is deﬁned via a Markovian process that adds noise to the data point at each timestep:

Original: [Equation]
Revised: [Equation]

Original: where {  1 , ...,  T } is some variance schedule, chosen such that q ( x T ) ⇠ N (0 , I ) .
Revised: where {  1 , ...,  T } is some variance schedule, chosen such that q ( x T ) ⇠ N (0 , I ) .

Original: Note that given83 x 0 , it is possible to sample from q ( x t | x 0 ) in closed form instead of applying noise t times using:
Revised: Note that given x 0 , it is possible to sample from q ( x t | x 0 ) in closed form instead of applying noise t times using:

Original: [Equation]
Revised: [Equation]

Original: where ↵ t :=
Revised: where ↵ t :=

Original: Q tk =1 (1   k ) .
Revised: Q tk =1 (1   k ) .

Original: As the number oftotal time steps, T approaches inﬁnity, and the reversetransitions q ( x t  1 | x t  ) approach diagonal Gaussian distributions.
Revised: Knowing these conditional distributions, one can use Bayes theorem to derive the conditional distribution of the reverse step q ( x t  1 | x t , x 0 ) [Equation] where ˜ µ t ( x t , x [Equation]

Original: Then one  uses a DNN to predictmean µ ✓ and covariance ⌃ ✓ for this reverse process with transition probabilities:
Revised: ) = p [Equation] x t and ˜  t = 1  ↵ t  1 1  ↵ t  t , as shown in [33]. In practice, one then uses a neural network to approximate q ( x t  1 | x t , x 0 ) by dropping conditioning on x 0 :

Original: [Equation]
Revised: [Equation]

Original: To sample from p ✓ ( x 0 ) , one samples x T from N (0 , I ) and then follows the reverse process byrepeatedly sampling from the transition probabilities p ✓ ( x t  1 | x t ) .
Revised: To sample from p ✓ ( x 0 ) , one samples x T from N (0 , I ) and then follows the reverse process by repeatedly sampling from the transition probabilities p ✓ ( x t  1 | x t ) .

Original: Originally, diffusion models weretrained by optimizing a variational lower bound of  thedata likelihood [48]. Later, [26] proposed asimpliﬁed loss L simple that resembles  a denoising auto-encoder:
Revised: Instead of predicting µ ✓ ( x t , t ) directly, it has been shown in [24] that the best performing parameterization uses a neural network ✏ ✓ ( x t , t ) to approximate the source noise ✏ in (3) and the loss used for training resembles that of a denoising model:

Original: [Equation]
Revised: [Equation]

Original: where µ  can be recovered as: µ ✓ ( x t 
Revised: The mean of the reverse step µ ✓ in (5) can be derived by plugging (3) in ˜ µ t ( x t , x 0 ) as:

Original: [Equation]
Revised: [Equation]

Original: t ) = 1 p ↵ t ( x t  1  ↵ t p 1  ↵ t ✏ ✓ ( x t , t )) . [37, 15] combined both lossesto learn ⌃ and µ .In the continuous-time setting, [50, 25] have derived the respective lower bounds.
Revised: This is, however, only one of the possible equivalent parameterization of the learning objective as shown in [33]. Because the objective L simple does not give a learning signal for ⌃ ✓ ( x t , t ) , in practice one combines L simple with another loss based on a variational lower bound of the data likelihood [49], which unlike L simple allows us to learn the covariances ⌃ ✓ ( x t , t ) .

Original: 2.1 Class conditional sampling
Revised: 2.1 Class conditional sampling

Original: There exist diffusion models trained with and without knowledge about classes in the dataset [15, 24].
Revised: There exist diffusion models trained with and without knowledge about classes in the dataset [15, 23].

Original: For our experiments, we only use the class-unconditional diffusion model, such that all classconditional features are introduced by the classiﬁer which we want to explain.
Revised: For our experiments, we only use the class-unconditional diffusion model, such that all classconditional features are introduced by the classiﬁer which we want to explain.

Original: For the noise-awareclassiﬁer, the reverse process transitions are then of the form (Proof Appendix H in [15]):
Revised: For the noise-aware classiﬁer p  ( y |· , · ) : R d ⇥ { 1 , ..., T } ! [0 , 1] , with parameters  , that is trained on noisy images corresponding to the various time-steps [15], the reverse process transitions are then of the form:

Original: [Equation]
Revised: [Equation]

Original: (5) for a normalization constant Z .
Revised: p  ( y | x t , t ) (8)for a normalization constant Z .

Original: Typically p ✓ ( y |· ) is a time-step aware classiﬁer speciﬁcally trainedon noisy images belonging to time-step t [15].

Original: Asfurther we would like to explain any classiﬁer andnot only noise-aware ones, we follow the approach of [2], where a classiﬁer p ✓ ( y |· ) is given as aninput the denoised sample ˆ x 0 = f dn ( x t ) of x t , using the mapping:
Revised: As we would like to explain any classiﬁer and not only noise-aware ones, we follow the approach of [2], where a classiﬁer p  ( y |· ) is given as input the denoised sample ˆ x 0 = f dn ( x t ) of x t , using the mapping:

Original: [Equation]
Revised: [Equation]

Original: This mapping is derived from Eq. 2, from which we have that [Equation] (7) and because we use our diffusion model ✏ ✓ ( x t , t ) to predict ✏ .
Revised: This mapping, derived from (3) estimates the noise-free image x 0 by using the noise approximated by the model ✏ ✓ ( x t , t ) for a given time-step t .

Original: With this, we can deﬁne a time-stepaware posterior p ✓ ( y | x t , t ) for any classiﬁer p ✓ ( y |· ) as p ✓ ( y | x t , t ) =p ✓ ( y | f
Revised: With this, we can deﬁne a time-step aware posterior p  ( y | x t , t ) for any classiﬁer p  ( y |· ) as p  ( y | x t , t ) =

Original: [Equation] ( x t , t )) .
Revised: p  ( y | f dn ( x t , t )) .

Original: The issue is that efﬁcient sampling from the original diffusion model is possible only because thereverse process is made of normal distributions.
Revised: The issue is that efﬁcient sampling from the original diffusion model is possible only because the reverse process is made of normal distributions.

Original: As we need to sample from p ✓  ( x t  1 | x t , y ) thousandsof times to obtain a single sample from the data distribution, it is not possible to use MCMC-samplerswith high complexity to sample from each of the individual transitions.
Revised: As we need to sample from p ✓ ,  ( x t  1 | x t , y ) hundreds of times to obtain a single sample from the data distribution, it is not possible to use MCMC-samplers with high complexity to sample from each of the individual transitions.

Original: In [15], they proposed to solveit by approximating p ✓  ( x t  1 | x t , y ) with slightly shifted versions of p ✓ ( x t  1 | x t ) to make closed-form110 sampling possible.
Revised: In [15], they proposed to solve it by approximating p ✓ ,  ( x t  1 | x t , y ) with slightly shifted versions of p ✓ ( x t  1 | x t ) to make closed-form sampling possible.

Original: Such transition kernels are given by:
Revised: Such transition kernels are given by:

Original: [Equation]
Revised: [Equation]

Original: 8) where [Equation] [Figure]
Revised: which we further adapt for the goal of generating VCEs and use in our experiments.

Original:  Diffusion Visual Counterfactual Explanations
Revised:  Diffusion Visual Counterfactual Explanations

Original: A VCE z for a chosen target class c and a given classiﬁer f  and  input x should satisfy the followingcriteria: i) validity: the VCE z should be classiﬁed by f as the desired target class c with highprobability, ii) realism: the VCE z should be indistinguishable or at least close to a natural image, iii)minimality/closeness: the difference of the VCE z and the original image x should be the minimalsemantic modiﬁcation necessary to change the class, in particular, the generated image z should beclose to x while being valid and realistic, e.g. by changing the object in the image and leaving thebackground unchanged.
Revised: A VCE z for a chosen target class c , a given classiﬁer f , and an input x should satisfy the following criteria: i) validity: the VCE z should be classiﬁed by f as the desired target class c with high probability, ii) realism: the VCE z should be indistinguishable or at least close to a natural image, iii) minimality/closeness: the difference between the VCE z and the original image x should be the minimal semantic modiﬁcation necessary to change the class, in particular, the generated image z should be close to x while being valid and realistic, e.g. by changing the object in the image and leaving the background unchanged.

Original: Note thatwhile targeted adversarial examples are valid, they do not showmeaningful semantic changes into the target class for a non-robust model and arealso not realistic.
Revised: Note that targeted adversarial examples are valid but do not show meaningful semantic changes in the target class for a non-robust model and are not realistic.

Original: Although adversarial l 1 perturbations are (almost) imperceivable for humans, adversarial images arevery unlikely to occur in nature. The l 1 . 5 -SVCE method of [7] optimizes the predicted probability of  classiﬁer into the target class bymanipulating the image and is very close to the generation of adversarial examples and thus worksonly foradversarially robust classiﬁers. 
Revised: The l 1 . 5 -SVCEs of [7] optimize the predicted probability of a classiﬁer into the target class by manipulating the image and is very close to the generation of adversarial examples and thus works only for robust classiﬁers. They have used an adversarially robust ImageNet classiﬁer that was trained to be multiple-norm robust (MNR) as proposed in [12]. In our paper we use this model under the name MNR-RN50 (see Sec. 4.2).

Original: The realism of the image comes purely from the generativeproperties of robust classiﬁers [43] which can lead to artefacts.
Revised: The realism of the image comes purely from the generative properties of robust classiﬁers [44] which can lead to artefacts.

Original: In contrast, our Diffusion Visual
Revised: In contrast, our Diffusion Visual

Original: Counterfactual Explanations (DVCEs) work for any classiﬁer and our generated images are morerealistic due to the integration of diffusion models.
Revised: Counterfactual Explanations (DVCEs) work for any classiﬁer and our generated images are more realistic due to the integration of diffusion models.

Original: An approach similar to our DVCE framework isblended diffusion of [2] which manipulates the image inside a masked region.
Revised: An approach similar to our DVCE framework is Blended Diffusion of [2] which manipulates the image inside a masked region.

Original: However, it is easy toadapt it to the task of VCEs by allowing the whole image to be manipulated.
Revised: In fact, it is easy to adapt it to the task of VCEs by allowing the whole image to be manipulated.

Original: DVCE and blendeddiffusion share the same diffusion model, but, as we show in the Appendix, blended diffusion cannotbe applied to arbitrary classiﬁers and requires image-speciﬁc hyperparameter tuning, see Fig. 
Revised: DVCE and Blended Diffusion share the same diffusion model, but, as we show in the Appendix, Blended Diffusion cannot be applied to arbitrary classiﬁers and requires image-speciﬁc hyperparameter tuning, see Fig. 2.

Original: 3.1 Adaptive Parameterization
Revised: 3.1 Adaptive Parameterization

Original: For  the DVCE of the original image ˆ x , we don’t want to sample just any image x from p ( x | y ) (highrealism and validity) but also to make sure that d ( x, ˆ x ) is small.
Revised: For generating the DVCE of the original image ˆ x , we do not want to sample just any image x from p ( x | y ) (high realism and validity) but also to make sure that d ( x, ˆ x ) is small.

Original: To do this, we have to condition ourdiffusion process on ˆ x .
Revised: To do this, we have to condition our diffusion process on ˆ x .

Original: Using the introduced denoising function f dn and analogously to the derivation in [15] and following application in [29], the mean of the transition kernel becomes:
Revised: Using the introduced denoising function f dn and analogously to the derivation in [15] and following application in [28], the mean of the transition kernel becomes:

Original: [Equation]
Revised: [Equation]

Original: where C c is the coefﬁcient of the classiﬁer, and C d the one of the distance guiding loss.
Revised: where C c is the coefﬁcient of the classiﬁer, and C d the one of the distance guiding loss.

Original: Intuitively,we take a step in the direction that increases the classiﬁer score while staying close to ˆ x .
Revised: Intuitively, we take a step in the direction that increases the classiﬁer score while staying close to ˆ x .

Original: Note that thelast term can be interpreted as the log gradient of a distribution with density exp(  d t (ˆ x, · )  ) , where = 1 C d and d t (ˆ x, · ) =
Revised: Note that the last term can be interpreted as the log gradient of a distribution with density exp(  d t (ˆ x, · )  ) , where  = 1 C d and d t (ˆ x, · ) =

Original: d (ˆ x, f dn ( · , t )) , thus we introduce a time-aware prior distribution that enforcesour output to be similar to ˆ x .
Revised: d (ˆ x, f dn ( · , t )) , thus we introduce a time-aware prior distribution that enforces our output to be similar to ˆ x .

Original: In our work, we use the l 1 -distance as it produces sparser changes.
Revised: In our work, we use the l 1 -distance as it produces sparse changes.

Original: Several works have tried to minimize some distance during the diffusion process, implicitly in [9] orexplicitly, but for the background in [2].
Revised: Several works have tried to minimize some distance during the diffusion process, implicitly in [9] or explicitly, but for the background, in [2].

Original: However, there is no principled way to choose the coefﬁcientfor such a regularization term.
Revised: However, there is no principled way to choose the coefﬁcient for such a regularization term.

Original: It turns out that it is impossible to ﬁnd a parameter setting for blendeddiffusion [2] that works across images and classiﬁers.
Revised: It turns out that it is impossible to ﬁnd a parameter setting for Blended Diffusion [2] that works across images and classiﬁers.

Original: Thus, we propose an adaptive parametrization
Revised: Thus, we propose a parameterization

Original: [Equation]
Revised: [Equation]

Original: relative to the predicted mean of the diffusion model that we use to change µ t in Eq. 9 to
Revised: which adapts to the predicted mean of the diffusion model that we use to change µ t in (11) to

Original: [Equation]
Revised: [Equation]

Original: The adaptive parameterization allows for ﬁne-grained control of the inﬂucence of the classiﬁer anddistance regularization so that now the hyperparameters C c and C d have the same inﬂuence acrossimages and even classiﬁers which facilitates the generation of DVCEs as otherwise hyperparameterﬁnetuning would be necessary for each image as in blended diffusion, see Fig.
Revised: [Figure] This adaptive parameterization allows for ﬁne-grained control of the inﬂuence of the classiﬁer and distance regularization so that now the hyperparameters C c and C d have the same inﬂuence across images and even classiﬁers. It facilitates the generation of DVCEs as otherwise hyperparameter ﬁnetuning would be necessary for each image as in Blended Diffusion, see Fig.

Original: 2 for a comparison.
Revised: 2 for a comparison.

Original: However, even with our adaptive parameterization, it is still not easy to produce semanticallymeaningful changes close to ˆ x as can be seen in App.
Revised: However, even with our adaptive parameterization, it is still not easy to produce semantically meaningful changes close to ˆ x as can be seen in App.

Original: B.2.
Revised: B.2.

Original: Thus, as in [2], we vary the starting pointin the diffusion process and observe in App.
Revised: Thus, as in [2], we vary the starting point in the diffusion process and observe in App.

Original: B.2 that starting from step T 2 of the forward diffusionprocess, together with the adaptive parameterization and using as the distance the l 1 -distance providesus with sparse but semantically meaningful changes.
Revised: B.2 that starting from step T 2 of the forward diffusion process, together with the adaptive parameterization and using as the distance the l 1 -distance, provides us with sparse but semantically meaningful changes.

Original: In our experiments, we set T = 200 .
Revised: In our experiments, we set T = 200 .

Original: 3.2 Cone-Projection for Classiﬁer Guidance
Revised: 3.2 Cone Projection for Classiﬁer Guidance

Original: A major component of our DVCEs is that they can be applied to any image classiﬁer regardless ofwhether it is adversarially robust or not.
Revised: A goal for our DVCEs is to be applied to any image classiﬁer regardless of whether it is adversarially robust or not.

Original: However, it turns out that naively applying diffusion with thenew mean (Eq. 12) to a non-robust classiﬁer does not work and leads to very small modiﬁcations ofthe image, which are similar to adversarial examples without meaningful changes.
Revised: This is because using diffusion with the new mean (14) to a non-robust classiﬁer does not work and leads to very small modiﬁcations of the image, which are similar to adversarial examples without meaningful changes.

Original: The reason is thatgradients of non-robust classiﬁers aremuch noisier and less semantically meaningful.
Revised: The reason is that the gradients of non-robust classiﬁers are noisier and less semantically meaningful than those of the robust classiﬁers.

Original: We illustratethis for a non-robust Swin-Transformer[30] in Fig.
Revised: We illustrate this for a non-robust Swin-Transformer[29] in Fig.

Original: 3 where hardly any changes are generated.
Revised: 3 where hardly any changes are generated.

Original: Notethat this is even the case if we  denoise the sampleﬁrst using the denoising function f dn (Eq. 6).
Revised: Note that this is even the case if we ﬁrst denoise the sample using the denoising function f dn (9).

Original: As a solution, we suggest projecting the gradient of a robust classiﬁer, r x t log p robust , ✓ ( y | f dn ( x t , t )) ,onto a cone centered at the gradient of the classiﬁer, r x t log p ✓ ( y | f dn ( x t , t )) , (we use ↵ = 30  in allexperiments for non-robust classiﬁers).
Revised: As a solution, we suggest projecting the gradient of a robust classiﬁer with parameters  , r x t log p robust ,  ( y | f dn ( x t , t )) , onto a cone centered at the gradient of the classiﬁer, r x t log p  ( y | f dn ( x t , t )) .

Original: More precisely, we deﬁne
Revised: More precisely, we deﬁne

Original: [Equation]
Revised: [Equation]

Original: i , where cone ( ↵ , v ) := { w 2 R d : \ ( v, w )  ↵ } is the cone of angle ↵ around vector v , and theprojection P cone ( ↵ ,v ) onto cone ( ↵ , v ) is given as:
Revised: where cone ( ↵ , v ) := { w 2 R d : \ ( v, w )  ↵ } is the cone of angle ↵ around vector v , and the projection P cone ( ↵ ,v ) onto cone ( ↵ , v ) is given as:

Original: [Equation]
Revised: [Equation]

Original:  else , where P v ? ( w ) :
Revised: , v > ↵ w, else , where P v ? ( w ) :

Original: = w  h w,v i h v,v i v
Revised: = w  h w,v i h v,v i v

Original: and we deﬁne u = sin( ↵ ) P v ?
Revised: and we deﬁne u = sin( ↵ ) P v ?

Original: ( w ) kP v ?
Revised: ( w ) k

Original: ( w ) k 2 + cos( ↵ ) v k v k 2 (note thatk u k2 = 

Revised: [Equation] ↵ [Equation] 2 (note that k u k

Original:  and h u, v i = cos( ↵ ) ).
Revised: = and h u, v i = cos( ↵ ) ).

Original: The idea of this projection is that in this way we reduce drasticallythe noise in the gradient of the non-robust classiﬁer.
Revised: The idea of this projection is that in this way we reduce drastically the noise in the gradient of the non-robust classiﬁer.

Original: Note, however, that the projection of the gradientof the robust classiﬁer onto the cone generated by the non-robust classiﬁer is always an ascentdirection for log p ✓ ( y | f dn ( x t , t )) and we still have the goal of maximizing this objective (note thatg proj is not necessarily an ascent direction for log p robust , ✓ ( y | f dn ( x t , t )) ).
Revised: Note, however, that the projection of the gradient of the robust classiﬁer onto the cone generated by the non-robust classiﬁer is always an ascent direction for log p  ( y | f dn ( x t , t )) and we still have the goal of maximizing this objective (note that g proj is not necessarily an ascent direction for log p robust ,  ( y | f dn ( x t , t )) ).

Original: Thus, one can see the coneprojection as a form of regularization of r x t log p ✓ ( y | f dn ( x t , t )) which guides the diffusion processinto semantically meaningful changes of the classiﬁer.
Revised: Thus, one can see the cone projection as a form of regularization of r x t log p  ( y | f dn ( x t , t )) which guides the diffusion process into semantically meaningful changes to the images.

Original: [Figure]
Revised: [Figure]

Original: 3.3 Final Scheme for Diffusion Visual Counterfactuals
Revised: 3.3 Final Scheme for Diffusion Visual Counterfactuals

Original: We summarize here the full scheme of DVCEs for a non-adversarially robust classiﬁer p ✓ ( y | x ) wherewe assume that we are given additionally the adversarially robust model p robust , ✓ ( y | x ) :
Revised: Our solution for a non-adversarially robust classiﬁer p  ( y |· ) is to use Algorithm 1 of [15] by replacing the update step with: [Equation] [Figure] [Equation]

Revised: [Figure]

Revised: [Equation]

Revised: [Figure]

Revised: ( x t

Revised: [Equation]

Revised: t )))

Revised: h r x

Original: [Equation]
Revised: [Equation]

Original: For a robust model, the only difference is that the cone projection is omitted and one uses g proj =
Revised: log p robust ,   y | f dn ( x t [Equation] t [Equation]

Original: r x t log p robust , ✓
Revised: i , g update = C c g proj k [Equation] proj

Original: [Equation]
Revised: [Equation]

Original:  )  .
Revised: kr [Equation] For a robust model, the only difference is that the cone projection is omitted and one uses g proj = r x t log p robust ,   y | f dn ( x t , t )  .

Original: We use in all our experiments  C c = 0 .
Revised: In all our experiments we use C c = 0 .

Original: 1 , and C d 2 0 .
Revised: 1 , and C d =0 .

Original: 15 unless weshow ablations for one of the parameters.
Revised: 15 unless we show ablations for one of the parameters.

Original: The angle ↵ for the cone projection is ﬁxed to 30  .
Revised: The angle ↵ for the cone projection is ﬁxed to 30  .

Original: Instrong contrast to blended diffusion, these parameters generalize across images and even classiﬁersand allow us to generate VCEs for arbitrary classiﬁers for which a diffusion model is available.
Revised: In strong contrast to Blended Diffusion, these parameters generalize across images and even classiﬁers and allow us to generate VCEs for arbitrary classiﬁers for which a diffusion model is available.

Original:  Experiments
Revised:  Experiments

Original: In this section, we evaluate the quality of thevisual counterfactual explanations of DVCE.
Revised: In this section, we evaluate the quality of the DVCE.

Original: We start bycomparing DVCE to existing works in Sec. 4.1.
Revised: We start by comparing DVCE to existing works in Sec. 4.1.

Original: In Sec. 4.2, we compare counterfactuals of DVCEfor various state-of-the-art ImageNet models and show, how our method can be used to interpretdifferences between classiﬁers.
Revised: In Sec. 4.2, we compare DVCEs for various state-of-the-art ImageNet models and show how our method can be used to interpret differences between classiﬁers.

Original: For all experiments, we use the same set of hyperparameters for DVCE given in the last section.
Revised: For all experiments, we use the same set of hyperparameters for DVCE given in the last section without any further hyperparameter tuning.

Original: The diffusion model used for DVCE is taken from [15] and is trainedclass-unconditionally on ImageNet images of size 256x256 and uses a modiﬁed UNet architecture(more details in [15]).
Revised: The diffusion model used for DVCE is taken from [15], is trained classunconditionally on ImageNet images of size 256x256, and uses a modiﬁed UNet[41] architecture (more details in [15]).

Original: User study, further experiments, ablations, and details are in the Appendix.
Revised: User study, further experiments, ablations, and details are in the Appendix.

Original: 4.1 Method comparison
Revised: 4.1 Method comparison

Original: Creating VCEs for ImageNet with 1000 vastly different classes is a challenging task and there onlyexist a small number of methods that try to solve it without reducing dataset size or image resolution.

Original: For evaluation purposes, we compare our method to blended diffusion [2] anda recent method for VCEs based purely on optimization [7].
Revised: We compare DVCEs with VCEs produced by Blended Diffusion [2] and recent method for VCEs based purely on optimization [7].

Original: As the latter only works for adversarially robust classiﬁers,we use the multiple-norm robust model from [7], which they call Madry [17] + FT, as the classiﬁer tocreate VCEs for all three methods. 
Revised: As the latter only works for adversarially robust classiﬁers, we use the multiple-norm robust model from [7], MNR-RN50, as the classiﬁer to create VCEs for all three methods. As this model is robust on its own, we do not use the cone projection for its DVCEs.

Original: Results can be found in Fig.
Revised: First, we present a qualitative evaluation in Fig.

Original: 4, where we transform one imageinto two different classes that are close  in the WordNet hierarchy[34].
Revised: 4, where we transform one image into two different classes that are close to the true one in the WordNet hierarchy[34].

Original: The radius of the l 1 . 5 -ball of [7]  is chosen as the smallest r 2 { 50 , 75 , 100 , 150 } such that theconﬁdence in the target class is larger than 0 .
Revised: The radius of the l 1 . 5 -ball of [7] [Figure] wolf, white wolf, and alp). l 1 . 5 -SVCEs show artefacts (white wolf, volcano, night snake) and have often lower image quality. is chosen as the smallest r 2 { 50 , 75 , 100 , 150 } such that the conﬁdence in the target class is larger than 0 .

Original: 90 per image - if this is not reached, we show the onewith highest conﬁdence.
Revised: 90 per image.

Original: For blended diffusion, we select the image with the smallest classiﬁer andregularization weight that reaches conﬁdence larger than 0.9 from the set of parameters discussed in
Revised: For Blended Diffusion, we select the image with the smallest classiﬁer and regularization weight that reaches conﬁdence larger than 0.9 from the set of parameters discussed in

Original: Sec. 3.3.
Revised: Sec. 3.3.

Original: If 0.9 is not reached by any setting, weagain show the one with  highest conﬁdence. For
Revised: If 0.9 is not reached by any setting, we show the one with the highest conﬁdence for both baselines.

Original: DVCEs, we only use the single set of parameters provided in the previous section.

Original: As the model isrobust on its own, we do not use the cone projection in DVCE.

Original: [Figure]

Original: We show our DVCEs (left column), the l 1 . 5 -SVCEs of [7] (

Original: ), and the adaptation of blended diffusion [2] for VCEs (right).

Original: For each original image (outer left column) we show the VCEs for two target classes.

Original: See the text for the parameter selection of all methods.

Original: Only DVCEs satisfy all desired properties of VCEs.

Original: Blended diffusion fails for leopard and tiger and often produces images far away from the original one (pizza, potpie, timber wolf, white wolf, and alp).

Original: l 1 . 5 -SVCEs show artefacts (white wolf, volcano, night snake) and have often lower image quality.

Original: As Fig.
Revised: As Fig.

Original: 4 shows, DVCEs is the only method that satisﬁes all desired properties of VCEs: i) theproduced VCEs have high conﬁdence in the target class, ii) are close to natural images, and iii)show minimal semantic changes in the target class.
Revised: 4 shows, DVCEs is the only method that satisﬁes all desired properties of VCEs.

Original: For example, for “mashed potato”, DVCEpreserves the bowl and only changes the content into either guacamole or carbonara.
Revised: For example, for “mashed potato”, DVCE preserves the bowl and only changes the content into either guacamole or carbonara.

Original: Our qualitativecomparison shows that the same hyperparameter setting ofDVCE can handle different classes andtransfer between similar classes, such as different snakes or wolf types, as well as different objectsizes. For example, it works for the “cheetah” which covers the entire image, but also for the “snake”which only [Figure] [Table]
Revised: Our qualitative comparison shows that the same hyperparameter setting of

Original: [Figure]

Original: [Table] [Figure] [Table] [Figure]
Revised: DVCE can handle different classes and transfer between similar classes, such as different snakes or wolf types, as well as different object sizes.

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table] [Figure] covers a fraction of the input. In contrast, both l 1 . 5 -SVCEs and blended diffusion VCEs requiredifferent hyperparameters for different images to achieve high conﬁdence in the target class.
Revised: In contrast, both l 1 . 5 -SVCEs and Blended Diffusion VCEs require different hyperparameters for different images to achieve high conﬁdence in the target class.

Original: Evenwith the six parameter conﬁgurations, blended diffusion is not able to produce always images withhigh conﬁdence in the target class.
Revised: Even with the six parameter conﬁgurations, Blended Diffusion is not able to produce always images with high conﬁdence in the target class.

Original: Even more problematic for VCEs: sometimes the resulting imageshave high conﬁdence butdo not look realistic (cheetah ! tiger) or do not resemble the original imageat all (dingo !
Revised: More problematic for VCEs is that the resulting images can have high conﬁdence but not look realistic (cheetah ! tiger) or do not resemble the original image at all (dingo !

Original: timber wolf/white wolf).
Revised: timber wolf/white wolf).

Original: Even if the method works with the given parameters, forexample mashed potato ! guacamole or carbonara, the overall image quality cannot match that of DVCE as often the images contain overly bright colors.
Revised: Even if the method works with the given parameters, for example, mashed potato ! guacamole or carbonara, the overall image quality cannot match that of DVCE as often the images contain overly bright colors.

Original: In the case of the volcano VCE, DVCEshows class features like lava whereas blended diffusion VCE can not clearly be labeled as a volcano.
Revised: In the case of the volcano VCE, DVCE shows class features like lava whereas Blended Diffusion VCE can not clearly be labeled as a volcano.

Original: For the l 1 . 5 -SVCEs, one often needs large radii to achieve the desired conﬁdence of 0.90, which oftenresults incolor artefacts and images thatgenerally do not look realistic. 
Revised: For the l 1 . 5 -SVCEs, one often needs large radii to achieve the desired conﬁdence of 0.90, which often results in images that do not look realistic. For further comparison of VCEs, we provide in Tab.

Revised: 1 the quantitative evaluation of realism, validity, and closeness, where we discuss the results.

Original: 4.2 Model comparison
Revised: As in [7] the authors noticed that there is a risk of having low FID scores for the VCEs that do not change the starting images signiﬁcantly, we propose the simple “crossover” evaluation of FID and other metrics, such that the distribution of labels of original images is different from the distribution of the target classes for VCEs (more details are in App. E). Moreover, we have conducted a user study ( 20 users), in which participants have decided, what are the percentage of total images to have one of the three different properties (numbers for 3 methods sorted in the same order as in Tab. 1): meaningful - 62.0% , 48.4%, 38.7%; realism - 34.7%, 24.6%, 52.2% ; subtle - 45.0%, 50.6% , 31.0%. This conﬁrms that DVCEs generate more meaningful features in the target classes. More details are in App.

Revised: C.

Revised: Closeness

Revised: Validity

Revised: Realism

Revised: Metric l 1 # l 1 .

Revised: [Table]

Revised: 4.2 Model comparison

Revised: Non-robust models.

Original: In Fig.
Revised: In Fig.

Original: 5 we show that DVCEs can be generated for various state-of-the-art ImageNet [41] models.
Revised: 5 we show that DVCEs can be generated for various state-of-the-art ImageNet [42] models.

Original: Weuse a Swin-Transformer[30], a ConvNeXt [31] and a Noisy-Student EfﬁcientNet [58, 54].
Revised: We use a Swin-Transformer[29], a ConvNeXt [30] and a Noisy-Student EfﬁcientNet [58, 54].

Original: Both the

Original:  Swin-Transformer and the ConvNeXt are pretrained on ImageNet21k [14] whereas the EfﬁcientNetuses noisy-student self-training on a large unlabeled pool.
Revised: Both the Swin-Transformer and the ConvNeXt are pretrained on ImageNet21k [14] whereas the EfﬁcientNet uses noisy-student self-training on a large unlabeled pool.

Original: As all models do not yield perceptuallyaligned gradients, we use the cone projection with 30  angle described in Sec.
Revised: As all models do not yield perceptually aligned gradients, we use the cone projection with 30  angle described in Sec.

Original: 3.2 with the robustmodel from the previous section.
Revised: 3.2 with the robust model from the previous section.

Original: All other parameters are identical to the previous experiment.
Revised: All other parameters are identical to the previous experiment.

Original: We highlight that DVCE is the ﬁrst method capable of visualizing arbitrary classiﬁers on a taskaschallenging as ImageNet [41].
Revised: We highlight that DVCE is the ﬁrst method capable of explaining (in the sense of VCEs) arbitrary classiﬁers on a task as challenging as ImageNet.

Original: Overall, DVCEs satisfy all desired properties of VCEs.
Revised: Overall, DVCEs satisfy all desired properties of VCEs.

Original: It also allowsus to inspect the most important features of each model and class.
Revised: It also allows us to inspect the most important features of each model and class.

Original: Forexample, all three models havelearned to associate the speciﬁc texture pattern of the box turtle with that particular class. For thestupa and church classes,  it seems like all models use different roof and tower structures as the mostprominent feature as they spent most of their budget on changing those. 
Revised: For the stupa and church classes, for example, it seems like all models use different roof and tower structures as the most prominent feature as they spent most of their budget on changing those. Robust models. Here, we want to examine the inﬂuence of the different architectures that were trained robustly as in [12] on the resulting DVCEs.

Original: Limitations, Future Work and Societal impact
Revised: There, the authors take inspiration from the result for linear classiﬁers in Theorem 3 . 1 [11], which states that when such a classiﬁer is robust with respect to the l 1 - and l 1 -balls, it is robust also in the largest l p ball, which ﬁts in the convex hull for any 1  p  1 . This motivated them further to devise an adversarial training scheme that with some probability performs for each batch either l 1 - or l 1 -attack and yields classiﬁers more adversarially robust for all p  1 . We will call this scheme multiple-norm robust ﬁnetuning. For models trained in such a way, [7] have shown that the generated VCEs are better (both in terms of FID and qualitatively) if they are generated with respect to a sparse norm, such as l 1 . 5 .

Revised: While our DVCEs are biased by the l 1 regularization, we do not have hard constraints in any l p -norm.

Revised: Thus, we decide to use multiple-norm robust models to be robust to a broad range of perturbation

Revised: [Figure]

Revised: types.

Revised: For this, we compare in Fig.

Revised: 6 multiple-norm (MNR) ﬁne-tuned models : MNR-RN50 based on a l 2 -robust ResNet50 [17], MNR-XCiT based on a l 1 -robust ViT [13] and MNR-DeiT based on a robust data-efﬁcient image transformer [5].

Revised: It can be seen here that all the models that use multiple-norm robust ﬁnetuning have equally good DVCEs, each of which provides slightly different explanations.

Revised: More examples and further experiments are in App.

Revised: B.4.

Revised: [Figure]

Revised:  Limitations, Future Work and Societal impact

Original: In comparison to GANs, diffusion-based approaches can be expensive to evaluate as they requiremultiple iterations of the reverse process to create one sample.
Revised: In comparison to GANs, diffusion-based approaches can be expensive to evaluate as they require multiple iterations of the reverse process to create one sample.

Original: This can be an issue when creating alarge amount of VCEs and makes deployment in a time-sensitive setting challenging.
Revised: This can be an issue when creating a large amount of VCEs and makes deployment in a time-sensitive setting challenging.

Original: We also rely on a robust model during the creation of VCEs for the cone projection.
Revised: We also rely on a robust model during the creation of VCEs for the cone projection.

Original: While weshow that standard classiﬁers do not yield the desired gradients, training robust models can bechallenging.
Revised: While we show that standard classiﬁers do not yield the desired gradients, training robust models can be challenging.

Original: An interesting direction for future research is thus to replace the cone projection withanother “denoising” procedure for the gradient of a non-robust model.
Revised: An interesting direction for future research is thus to replace the cone projection with another “denoising” procedure for the gradient of a non-robust model.

Original: From a theoretical standpoint, using conditional sampling with reverse transitions of the form Eq. 5is justiﬁed, in practice, however, we have to approximate the reverse transitions with shifted normaldistributions of the form Eq. 8.
Revised: From a theoretical standpoint, using conditional sampling with reverse transitions of the form (8) is justiﬁed, in practice, however, we have to approximate the reverse transitions with shifted normal distributions of the form (10).

Original: This approximation relies on the conditioning function having lowcurvature, and it can be hard to verify this once we start adding a classiﬁer, distance and other possibleterms and it is unclear how this inﬂuences the outcome of the diffusion process.
Revised: This approximation relies on the conditioning function having low curvature, and it can be hard to verify this once we start adding a classiﬁer, distance and other possible terms and it is unclear how this inﬂuences the outcome of the diffusion process.

Original: The quantitative evaluation of VCEs is difﬁcult, as the standard FID metric compares the distributionof features of a classiﬁer over a test and a generated dataset.
Revised: The quantitative evaluation of VCEs is difﬁcult, as the standard FID metric compares the distribution of features of a classiﬁer over a test and a generated dataset.

Original: However, for VCEs it is not onlyimportant to generate realistic images, but also to achieve high conﬁdence and to create meaningfulchanges.
Revised: However, for VCEs it is not only important to generate realistic images, but also to achieve high conﬁdence and to create meaningful changes.

Original: Moreover, metrics such as IM1, IM2 [32] for VCEs rely on a well-trained (V)AE for everyclass, which is difﬁcult to achieve for a dataset with 1000 classes and high-resolution images.
Revised: Moreover, metrics such as IM1, IM2 [31] for VCEs rely on a well-trained (V)AE for every class, which is difﬁcult to achieve for a dataset with 1000 classes and high-resolution images.

Original: Futureresearch should therefore try to develop metrics for the quantitative evaluation of VCEs.
Revised: Future research should therefore try to develop metrics for the quantitative evaluation of VCEs and we think that the evaluation in Tab. 1 is the ﬁrst step in that direction.

Original: DVCEs and VCEs in general help to discover biases of the classiﬁers and thus have a positive societalimpact, however one can abuse them for unintended purposes as any conditional generative model.
Revised: DVCEs and VCEs in general help to discover biases of the classiﬁers and thus have a positive societal impact, however one can abuse them for unintended purposes as any conditional generative model.

Original:  Conclusion
Revised:  Conclusion

Original: We have proposed DVCEs, a novel way to create VCEs for any state-of-the-art ImageNet classiﬁerusing diffusion models and our cone projection.
Revised: We have proposed DVCEs, a novel way to create VCEs for any state-of-the-art ImageNet classiﬁer using diffusion models and our cone projection.

Original: Our DVCEs  satisfy all desired properties of VCEs. 
Revised: DVCEs can handle vastly different image conﬁgurations, object sizes, and classes and satisfy all desired properties of VCEs. Acknowledgement

Original: Throughout the paper, we show that DVCEs can handle vastly different image conﬁgurations, objectsizes, and classes.
Revised: M.H. and V.B. acknowledge support by the DFG Excellence Cluster Machine Learning - New Perspectives for Science, EXC 2064/1, Project number 390727645.