Original: A DVERSARIAL TRAINING DESCENDS WITHOUT DESCENT : F INDING ACTUAL DESCENT DIRECTIONS BASED ON D ANSKIN ’ S T HEOREM .
Revised: A DVERSARIAL TRAINING DESCENDS WITHOUT DESCENT : F INDING ACTUAL DESCENT DIRECTIONS BASED ON D ANSKIN ’ S T HEOREM .

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Adversarial Training using a strong first-order adversary (PGD) is the gold standard for training Deep Neural Networks that are robust to adversarial examples.
Revised: Adversarial Training using a strong ﬁrst-order adversary (PGD) is the gold standard for training Deep Neural Networks that are robust to adversarial examples.

Original: We show that, contrary to the general understanding of the method, the gradient at an optimal adversarial example may increase, rather than decrease, the adversarially robust loss.
Revised: We show that, contrary to the general understanding of the method, the gradient at an optimal adversarial example may increase, rather than decrease, the adversarially robust loss.

Original: This holds independently of the learning rate.
Revised: This holds independently of the learning rate.

Original: More precisely, we provide a counterexample to a corollary of Danskin’s Theorem presented in the seminal paper of Madry et al.
Revised: More precisely, we provide a counterexample to a corollary of Danskin’s Theorem presented in the seminal paper of Madry et al.

Original: (2018) which states that a solution of the inner maximization problem can yield a descent direction for the adversarially robust loss.
Revised: (2018) which states that a solution of the inner maximization problem can yield a descent direction for the adversarially robust loss.

Original: Based on a correct interpretation of Danskin’s Theorem, we propose Danskin’s
Revised: Based on a correct interpretation of Danskin’s Theorem, we propose Danskin’s

Original: Descent Direction (DDD) and we verify experimentally that it provides better directions than those obtained by a PGD adversary.
Revised: Descent Direction (DDD) and we verify experimentally that it provides better directions than those obtained by a PGD adversary.

Original: Using the CIFAR10 dataset we further provide a real world example showing that our method achieves a steeper increase in robustness levels in the early  stages of training, and is more stable than the PGD baseline.
Revised: Using the CIFAR10 dataset we further provide a real world example showing that our method achieves a steeper increase in robustness levels in the early training stages of smooth-activation networks without BatchNorm, and is more stable than the PGD baseline.

Revised: As a limitation, PGD training of ReLU+BatchNorm networks still performs better, but current theory is unable to explain this.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Adversarial Training (AT) (Goodfellow et al., 2015; Madry et al., 2018) has become the de-facto algorithm used to train Neural Networks that are robust to adversarial examples (Szegedy et al., 2014).
Revised: Adversarial Training (AT) (Goodfellow et al., 2015; Madry et al., 2018) has become the de-facto algorithm used to train Neural Networks that are robust to adversarial examples (Szegedy et al., 2014).

Original: Variations of AT together with data augmentation yield the best-performing models in public benchmarks (Croce et al., 2020).
Revised: Variations of AT together with data augmentation yield the best-performing models in public benchmarks (Croce et al., 2020).

Original: Despite lacking optimality guarantees for the inner-maximization problem, the simplicity and performance of AT are enough reasons to embrace its heuristic nature.
Revised: Despite lacking optimality guarantees for the inner-maximization problem, the simplicity and performance of AT are enough reasons to embrace its heuristic nature.

Original: From an optimization perspective, the consensus is that AT is a sound algorithm: based on Danskin’s Theorem, Madry et al.
Revised: From an optimization perspective, the consensus is that AT is a sound algorithm: based on Danskin’s Theorem, Madry et al.

Original: (2018, Corollary C.2) posit that by finding a maximizer of the inner non-concave maximization problem, i.e., an optimal adversarial example, one can obtain a descent direction for the adversarially robust loss.
Revised: (2018, Corollary C.2) posit that by ﬁnding a maximizer of the inner non-concave maximization problem, i.e., an optimal adversarial example, one can obtain a descent direction for the adversarially robust loss.

Original: What if this is not true?
Revised: What if this is not true?

Original: are we potentially overlooking issues in its algorithmic framework?
Revised: are we potentially overlooking issues in its algorithmic framework?

Original: As mentioned in (Dong et al., 2020, Section 2.3), Corollary C.2 in Madry et al.
Revised: As mentioned in (Dong et al., 2020, Section 2.3), Corollary C.2 in Madry et al.

Original: (2018) can be considered the theoretical optimization foundation of the non-convex non-concave min-max optimization algorithms that we now collectively refer to as Adversarial Training .
Revised: (2018) can be considered the theoretical optimization foundation of the non-convex non-concave min-max optimization algorithms that we now collectively refer to as Adversarial Training .

Original: It justifies the two-stage structure of the training loop: first we find one approximately optimal adversarial example and then we update the model using the gradient (with respect to the model parameters) at the perturbed input.
Revised: It justiﬁes the two-stage structure of the training loop: ﬁrst we ﬁnd one approximately optimal adversarial example and then we update the model using the gradient (with respect to the model parameters) at the perturbed input.

Original: The only drawbacks of a first-order adversary seem to be its computational complexity and its approximate suboptimal solver nature.
Revised: The only drawbacks of a ﬁrst-order adversary seem to be its computational complexity and its approximate suboptimal solver nature.

Original: Ignoring the computational complexity issue, suppose we have access to a theoretical oracle that provides a single solution of the inner-maximization problem.
Revised: Ignoring the computational complexity issue, suppose we have access to a theoretical oracle that provides a single solution of the inner-maximization problem.

Original: In such idealized setting, can we safely assume AT is decreasing the adversarially robust loss on the data sample?
Revised: In such idealized setting, can we safely assume AT is decreasing the adversarially robust loss on the data sample?

Original: According to the aforementioned theoretical results, it would appear so.
Revised: According to the aforementioned theoretical results, it would appear so.

Original: In this work, we scrutinize the optimization paradigm on which Adversarial Training (AT) has been founded, and we posit that finding multiple solutions of the inner-maximization problem is necessary to find good descent directions of the adversarially robust loss.
Revised: In this work, we scrutinize the optimization paradigm on which Adversarial Training (AT) has been founded, and we posit that ﬁnding multiple solutions of the inner-maximization problem is necessary to ﬁnd good descent directions of the adversarially robust loss.

Original: In doing so, we hope to improveour understanding of the non-convex/non-concave min-max optimization problem that underlies the Adversarial Training methodology, and potentially improving its performance.
Revised: In doing so, we hope to improve

Original: − 1 .− 1 .− 0 .0 .0 .1 .1 .
Revised: − 1 .− 1 .− 0 .0 .0 .1 .1 .

Original: 5 Contourplot − 1 .− 1 .− 0 .0 .0 .1 .1 .
Revised: 5 Contourplot − 1 .− 1 .− 0 .0 .0 .1 .1 .

Original:               8 DDDPGD (a)     50 Iteration 1 .1 .
Revised:               8 DDDPGD (a)     50 Iteration 1 .1 .

Original: R o bu s t L o ss
Revised: R o bu s t L o ss

Original: DDDPGD (b) 0 25 50 75 100 125 150 175 200 Epoch 0 .0 .
Revised: DDDPGD (b) 0 25 50 75 100 125 150 175 200 Epoch 0 .0 .

Original: R o bu s t a cc u r a c y
Revised: R o bu s t a cc u r a c y

Original: DDDPGD (c) 
Revised: DDDPGD (c) our understanding of the non-convex/non-concave min-max optimization problem that underlies the Adversarial Training methodology, and potentially improving its performance.

Original: Our contributions: We present two counterexamples to Madry et al.
Revised: Our contributions: We present two counterexamples to Madry et al.

Original: (2018, Corollary C.2), the motivation behind AT.
Revised: (2018, Corollary C.2), the motivation behind AT.

Original: They show that using the gradient (with respect to the parameters of the model) evaluated at a single solution of the inner-maximization problem, can increase the robust loss, i.e., it can harm the robustness of the model.
Revised: They show that using the gradient (with respect to the parameters of the model) evaluated at a single solution of the inner-maximization problem, can increase the robust loss, i.e., it can harm the robustness of the model.

Original: In particular, in counterexample 2 many descent directions exist, but they cannot be found if we only compute a single solution of the inner-maximization problem.
Revised: In particular, in counterexample 2 many descent directions exist, but they cannot be found if we only compute a single solution of the inner-maximization problem.

Original: In Section 2 we explain that the flaw in the proof is due to a misunderstanding of the directional derivative notion that is used in the original work of Danskin (1966).
Revised: In Section 2 we explain that the ﬂaw in the proof is due to a misunderstanding of the directional derivative notion that is used in the original work of Danskin (1966).

Original: Based on our findings, we propose Danskin’s Descent Direction (DDD, Algorithm 1).
Revised: Based on our ﬁndings, we propose Danskin’s Descent Direction (DDD, Algorithm 1).

Original: It aims to overcome the problems of the single adversarial example paradigm of AT by exploiting multiple adversarial examples, obtaining better update directions for the network.
Revised: It aims to overcome the problems of the single adversarial example paradigm of AT by exploiting multiple adversarial examples, obtaining better update directions for the network.

Original: For a data-label pair, DDD finds the steepest descent direction for the robust loss, assuming that ( i ) there exists a finite number of solutions of the inner-maximization problem and ( ii ) they can be found with first-order methods.
Revised: For a data-label pair, DDD ﬁnds the steepest descent direction for the robust loss, assuming that ( i ) there exists a ﬁnite number of solutions of the inner-maximization problem and ( ii ) they can be found with ﬁrst-order methods.

Original: In Section 5 we verify experimentally that: ( i ) it is unrealistic to assume a unique solution of the inner-maximization problem, hence making a case for our method DDD, ( ii ) our method can achieve more stable descent dynamics than the vanilla AT method in synthetic scenarions and ( iii ) on the
Revised: In Section 5 we verify experimentally that: ( i ) it is unrealistic to assume a unique solution of the inner-maximization problem, hence making a case for our method DDD, ( ii ) our method can achieve more stable descent dynamics than the vanilla AT method in synthetic scenarions and ( iii ) on the

Original: CIFAR10 dataset DDD is more stable and achieves higher robustness levels in the early stages of traning, compared with a PGD adversary of equivalent complexity.
Revised: CIFAR10 dataset DDD is more stable and achieves higher robustness levels in the early stages of traning, compared with a PGD adversary of equivalent complexity.

Original: This is observed in a setting where the conditions of Danskin’s Theorem holds, i.e., using differentiable activation functions.
Revised: This is observed in a setting where the conditions of Danskin’s Theorem holds, i.e., using differentiable activation functions and removing BatchNorm.

Revised: As a limitation, PGD training of ReLU+BatchNorm networks still performs better, but there is no theory explaining this.

Original: Remark.
Revised: Remark.

Original: The fact that (Madry et al., 2018, Corollary C.2) is false, might be well-known in the optimization field.
Revised: The fact that (Madry et al., 2018, Corollary C.2) is false, might be well-known in the optimization ﬁeld.

Original: In the convex setting it corresponds to the common knowledge that a negative subgradient of a non-smooth convex function might not be a descent direction c.f., (Boyd, 2014,
Revised: In the convex setting it corresponds to the common knowledge that a negative subgradient of a non-smooth convex function might not be a descent direction c.f., (Boyd, 2014,

Original: Section 2.1).
Revised: Section 2.1).

Original: However, we believe this is not well-known in the AT community given that ( i ) its practical implications i.e., methods deriving steeper descent updates using multiple adversarial examples, have not been previously introduced, and ( ii ) the results in Madry et al.
Revised: However, we believe this is not well-known in the AT community given that ( i ) its practical implications i.e., methods deriving steeper descent updates using multiple adversarial examples, have not been previously introduced, and ( ii ) the results in Madry et al.

Original: (2018) have been central in the development of AT.
Revised: (2018) have been central in the development of AT.

Original: Hence, our contribution can be understood as raising awareness about the issue, and demonstrating its practical implications for AT.
Revised: Hence, our contribution can be understood as raising awareness about the issue, and demonstrating its practical implications for AT.

Original:  A C OUNTEREXAMPLE TOM ADRY ET AL . (2018, C OROLLARY C.2)
Revised:  A C OUNTEREXAMPLE TO

Revised: M ADRY ET AL . (2018, C OROLLARY C.2)

Original: Preliminaries.
Revised: Preliminaries.

Original: Let θ ∈ R
Revised: Let θ ∈ R

Original: d be the parameters of a model, ( x, y ) ∼ D a data-label distribution, δ a perturbation in a compact set S {+
+} and L a loss function.
Revised: d be the parameters of a model, ( x, y ) ∼ D a data-label distribution, δ a perturbation in a compact set S {+
+} and L a loss function.

Original: The optimization objective of AT is:
Revised: The optimization objective of AT is:

Original: [Equation]
Revised: [Equation]

Original: where

Original: ρ

Original: [Equation]

Original: θ

Original: [Equation]

Original: E

Original: [Equation]

Original: x

Original: [Equation] y [Equation] In this setting ρ ( θ ) is referred to as the adversarial loss or robust loss .
Revised: In this setting ρ ( θ ) is referred to as the adversarial loss or robust loss .

Original: In order to optimize Eq.
Revised: In order to optimize Eq.

Original: (1) via iterative first-order methods, we need access to an stochastic gradient of the adversarial loss ρ or at least, the weaker notion of stochastic descent direction i.e., a direction along which the function
Revised: (1) via iterative ﬁrst-order methods, we need access to an stochastic gradient of the adversarial loss ρ or at least, the weaker notion of stochastic descent direction i.e., a direction along which the function

Original: [Equation]
Revised: [Equation]

Original: decreases in value.
Revised: decreases in value.

Revised: We have collected the perturbations δ i ∈ S 0 on the batch { ( x i , y i ) } ki =1 as the columns of a matrix δ =

Revised: [ δ 1 , . . .

Original:  To obtain a descent direction for partial maximization functions like ϕ wenormally resort to Danskin’s Theorem. We first recall its original formulation:
Revised: , δ k ] ∈ S := S k 0 which is also a compact set. To obtain a descent direction for partial maximization functions like φ we resort to Danskin’s Theorem:

Original: Theorem 1 (Danskin (1966)) .
Revised: Theorem 1 (Danskin (1966)) .

Original: Let S be a compact topological space, and let g : R d × S be a continuous function such that g ( · , δ ) is differentiable for all δ ∈ S and ∇ θ g ( θ, δ ) is continuous on R d × S .
Revised: Let S be a compact topological space, and let g : R d × S be a continuous function such that g ( · , δ ) is differentiable for all δ ∈ S and ∇ θ g ( θ, δ ) is continuous on R d × S .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: Let γ ∈ R d with ∥ γ ∥ 2 = 1 be an arbitrary unit vector.
Revised: Let γ ∈ R d with (cid:107) γ (cid:107) 2 = 1 be an arbitrary unit vector.

Original: The directional derivative D γ ϕ ( θ ) of ϕ in the direction γ at the point θ exists, and is given by the formula
Revised: The directional derivative D γ φ ( θ ) of φ in the direction γ at the point θ exists, and is given by the formula

Original: [Equation]
Revised: [Equation]

Revised: Remark.

Revised: γ (cid:54) = 0 is called a descent direction of φ at x if and only if D γ φ ( x ) < 0 , i.e., if the directional derivative is strictly negative.

Original: Corollary 1 is an equivalent rephrasing of Madry et al.
Revised: Corollary 1 is an equivalent rephrasing of Madry et al.

Original: (2018, Corollary C.2.), and is derived as a consequence of Theorem 1.
Revised: (2018, Corollary C.2.), and was originally claimed to be a consequence of Theorem 1.

Original: Unfortunately counterexample 1 shows that it is false:
Revised: Unfortunately counterexample 1 shows that the corollary is false.

Revised: As Theorem 1 (Danskin’s Theorem) is true, this means that there is some mistake in the proof of the corollary provided in Madry et al.

Original: Corollary 1.
Revised: Corollary 1.

Original: Let δ ⋆ ∈ S ⋆ ( x ) .
Revised: Let δ (cid:63) ∈ S (cid:63) ( x ) .

Original: If −∇ θ g ( θ, δ ⋆ ) ̸ = 0 , then it is a descent direction for ϕ at θ .Remark.
Revised: If −∇ θ g ( θ, δ (cid:63) ) (cid:54) = 0 , then it is a descent direction for φ at θ .

Original: For ease of exposition, we have adapted the previous statements from the original sources, without altering the fundamental results.

Original: In particular, in Danskin (1966) the problem studied is of the form max θ min δ g ( θ, δ ) whereas adversarial training follows the structure min θ max δ g ( θ, δ ) .

Original: Theorem 1 is precisely the analogous result in the min-max setting.

Original: Remark.

Original: γ ̸ = 0 is called a descent direction of ϕ at x if and only if D γ ϕ ( x ) < 0 , i.e., if the directional derivative is strictly negative.

Original: Counterexample 1.
Revised: Counterexample 1.

Original: Let S :=
Revised: Let S :=

Original: [ − 1 , 1] and g ( θ, δ ) = θδ .
Revised: [ − 1 , 1] and g ( θ, δ ) = θδ .

Original: The conditions of Danskin’s theorem clearly hold in this case, andϕ
Revised: The conditions of Danskin’s theorem clearly hold in this case, and

Original: [Equation]
Revised: [Equation]

Original: Note that at θ = 0 , we have S ⋆ (0) =
Revised: Note that at θ = 0 , we have S (cid:63) (0) =

Original: [ − 1 , 1] .
Revised: [ − 1 , 1] .

Original: Choosing δ = 1 ∈ S ⋆ (0) we have that g ( θ, 1)
Revised: Choosing δ = 1 ∈ S (cid:63) (0) we have that g ( θ, 1)

Original: = θ
Revised: = θ

Original: and so −∇ θ g (0 , 1) = − 1 ̸ = 0 .
Revised: and so −∇ θ g (0 , 1) = − 1 (cid:54) = 0 .

Original: Hence, Corollary 1 would imply that − 1 is a descent direction for ϕ ( θ ) =
Revised: Hence, Corollary 1 would imply that − 1 is a descent direction for φ ( θ ) =

Original: | θ | .
Revised: | θ | .

Original: However, θ = 0 is a global minimizer of the absolute value function, which means that there exists no descent direction.
Revised: However, θ = 0 is a global minimizer of the absolute value function, which means that there exists no descent direction.

Original: This is a contradiction.
Revised: This is a contradiction.

Original: To cast more clarity on why Corollary 1 is false, we explain what is the mistake in the proof provided in Madry et al.
Revised: To cast more clarity on why Corollary 1 is false, we explain what is the mistake in the proof provided in Madry et al.

Original: The main issue is the definition of the directional derivative , abasic concept in multivariable calculus, but one that can be defined in slightly different ways in the literature.
Revised: The main issue is the deﬁnition of the directional derivative , a concept in multivariable calculus that is deﬁned in slightly different ways in the literature.

Original: Definition 1.
Revised: Deﬁnition 1.

Original: Let ϕ : R d → R .
Revised: Let φ : R d → R .

Original: For a nonzero vector γ ∈ R d , the one-sided directional derivative of ϕ in the direction γ at the point θ is defined as the one-sided limit:
Revised: For a nonzero vector γ ∈ R d , the one-sided directional derivative of φ in the direction γ at the point θ is deﬁned as the one-sided limit:

Original: [Equation]
Revised: [Equation]

Original: The two-sided directional derivative is defined as the two-sided limit:
Revised: The two-sided directional derivative is deﬁned as the two-sided limit:

Original: [Equation]
Revised: [Equation]

Original: Unfortunately, it is not always clear which one of the two notions is meant when the term directional derivative is used.
Revised: Unfortunately, it is not always clear which one of the two notions is meant when the term directional derivative is used.

Original: Indeed, as our notation suggests, the one-sided definition Eq.
Revised: Indeed, as our notation suggests, the one-sided deﬁnition Eq.

Original: (6) is the one used in the statement of Danskin’s Theorem (Danskin, 1966).
Revised: (6) is the one used in the statement of Danskin’s Theorem (Danskin, 1966).

Original: In contrast, the proof of Corollary 1 provided in Madry et al.
Revised: However, the proof of Corollary 1 provided in Madry et al.

Original: (2018) only works for the two-sided definition Eq.
Revised: (2018) mistakenly assumes the two-sided deﬁnition Eq.

Revised: (7), and inadvertently uses the following property that holds for ˆ D γ φ ( θ ) (Eq.

Revised: but not for D γ φ ( θ ) (Eq.

Revised: Lemma 1.

Original: (7): it starts by noting that for a solution δ of the inner-maximization problem, the  directional derivative in the direction γ  = ∇ θ g ( θ, δ ) is positive, as implied by Danskin’s Theorem:{+> 
 then ˆ D − γ φ ( θ ) < 0 .+}
Revised: For the two-sided directional derivative deﬁnition (7) it holds that − ˆ D γ φ ( θ ) = ˆ D − γ φ ( θ ) provided that ˆ D γ exists. In particular, if ˆ D γ φ ( θ ) {+> 
 then ˆ D − γ φ ( θ ) < 0 .+}

Revised: However this is not true for the one-sided directional derivative (6) , as the example φ ( θ ) =

Revised: | θ | at θ =shows (both directional derivatives are strictly positive).

Revised: We provide a proof of this fact in Appendix E. The (ﬂawed) proof of Corollary 1 provided in Madry et al.

Revised: (2018) starts by noting that for a solution δ of the inner-maximization problem, the directional derivative in the direction γ = ∇ θ g ( θ, δ ) is positive, as implied by Danskin’s Theorem:

Original: [Equation]
Revised: [Equation]

Original:  assuming that ∇ θ g ( θ, δ ) is non-zero.
Revised: (8) assuming that ∇ θ g ( θ, δ ) is non-zero.

Original: This means that moving in the direction of γ increases the value of the function. However, we cannot guarantee that the function will decrease if we move in the opposite direction, that is, it is not necessarily true that D − γ ϕ ( θ ) < 0 .The property D γ ϕ ( θ ) > 0 ⇒ D − γ ϕ ( θ ) < 0 does not hold for the one-sided directional derivative Eq. (6), which is the one used in Danskin’s Theorem.
Revised: The mistake in the proof lies in concluding that D − γ φ ( θ ) < 0 .

Original: Rather, it only holds for the two-sided definition, incorrectly assumed by Madry et al. We summarize this in the following remark: Remark. The two-sided directional derivative definition (7) implies − ˆ D γ ϕ ( θ )
Revised: Following Lemma 1, this property does not hold for the one-sided directional derivative deﬁnition

Original: =
Revised: Eq.

Original: ˆ D − γ ϕ ( θ ) provided that ˆ D γ exists. However this is not true for the one-sided directional derivative (6),as the example ϕ ( θ ) = | θ | at θ = 0 shows (both directional derivatives are positive).
Revised: (6), the one used in Danskin’s Theorem.

Original:  A C OUNTEREXAMPLE AT A POINT THAT IS NOT LOCALLY OPTIMAL
Revised:  A C OUNTEREXAMPLE AT A POINT THAT IS NOT LOCALLY OPTIMAL

Original: The question remains whether a slightly modified version of Corollary 1 holds true: it might be the case that by adding some mild assumption, we exclude all possible counterexamples.
Revised: The question remains whether a slightly modiﬁed version of Corollary 1 holds true: it might be the case that by adding some mild assumption, we exclude all possible counterexamples.

Original: In the particular case of counterexample 1, θ = 0 is a local optimum of the function ϕ ( θ ) =
Revised: In the particular case of counterexample 1, θ = 0 is a local optimum of the function φ ( θ ) =

Original: | θ | .
Revised: | θ | .

Original: At such points, descent directions do not exist.
Revised: At such points, descent directions do not exist.

Original: However, in the trajectory of an iterative optimization algorithm we are mostly concerned with non-locally-optimal points.
Revised: However, in the trajectory of an iterative optimization algorithm we are mostly concerned with non-locally-optimal points.

Original: Hence, we explore whether adding the assumption that θ is not locally optimal can make Corollary 1 true.
Revised: Hence, we explore whether adding the assumption that θ is not locally optimal can make Corollary 1 true.

Original: Unfortunately, we will show that this is not the case.
Revised: Unfortunately, we will show that this is not the case.

Original: To this end we construct a family of counterexamples to Corollary 1 with the following properties: ( i ) there exists a descent direction at a point θ (that is, θ is not locally optimal) and ( ii ) , it does not coincide with −∇ θ g ( θ, δ ) , for any optimal δ ∈ S ⋆ ( θ ) .
Revised: To this end we construct a family of counterexamples to Corollary 1 with the following properties: ( i ) there exists a descent direction at a point θ (that is, θ is not locally optimal) and ( ii ) , it does not coincide with −∇ θ g ( θ, δ ) , for any optimal δ ∈ S (cid:63) ( θ ) .

Original: Moreover, all the directions −∇ θ g ( θ, δ ) are in fact ascent directions i.e., they lead to an increase in the function ϕ ( θ ) .
Revised: Moreover, all the directions −∇ θ g ( θ, δ ) are in fact ascent directions i.e., they lead to an increase in the function φ ( θ ) .

Original: Counterexample 2.
Revised: Counterexample 2.

Original: Let S :
Revised: Let S :

Original: =
Revised: =

Original: [0 , 1] and let u, v ∈ R 2 be unit vectors such that − 1 < ⟨ u, v ⟩ < 0 .
Revised: [0 , 1] and let u, v ∈ R 2 be unit vectors such that − 1 < (cid:104) u, v (cid:105) < 0 .

Original: That is, u and v form an obtuse angle.
Revised: That is, u and v form an obtuse angle.

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: Clearly, the function satisfies all conditions of Theorem 1.
Revised: Clearly, the function satisﬁes all conditions of Theorem 1.

Original: At θ = 0 , we have that S ⋆ (0) = 
Revised: At θ = 0 , we have that S (cid:63) (0) = arg [Equation]

Original: arg max δ ∈ [0 , 1] δ ( δ − 1) = 
Revised: [Equation] [Equation] [Equation] ) = { 0 , 1 } .

Original: At δ = 

Revised: At δ =

Original: we have ∇ θ g ( θ, 0)
Revised: we have ∇ θ g ( θ, 0)

Original: = ∇ θ ⟨ θ, v ⟩ = v and at δ
Revised: = ∇ θ (cid:104) θ, v (cid:105) = v and at δ

Original: =we have ∇ θ g ( θ, 1) = ∇ θ ⟨ θ, u ⟩ = u .
Revised: =we have ∇ θ g ( θ, 1) = ∇ θ (cid:104) θ, u (cid:105) = u .

Original: We compute the value of the directional derivatives in the negative direction of such vectors.
Revised: We compute the value of the directional derivatives in the negative direction of such vectors.

Original: According to Danskin’s Theorem we have
Revised: According to Danskin’s Theorem we have

Original: [Equation]
Revised: [Equation]

Original: where −⟨ v, u ⟩ > 0 holds by construction.
Revised: where −(cid:104) v, u (cid:105) > 0 holds by construction.

Original: Analogously, D − u ϕ (0) > 0 .
Revised: Analogously, D − u φ (0) > 0 .

Original: This means that all such directions are ascent directions.
Revised: This means that all such directions are ascent directions.

Original: However, for the direction γ = − ( u + v ) we have
Revised: However, for the direction γ = − ( u + v ) we have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality also follows by construction.
Revised: where the last inequality also follows by construction.

Original: Hence, − ( u + v ) is a descent direction.
Revised: Hence, − ( u + v ) is a descent direction.

Original: As counterexample 2 shows, Adversarial Training has the following problem: even if we are able to compute one solution of the inner-maximization problem δ ∈ S it can be the case that moving in the direction −∇ θ g ( θ, δ ) increases the robust training loss i.e., the classifier becomes less, rather than more, robust.
Revised: As counterexample 2 shows, Adversarial Training has the following problem: even if we are able to compute one solution of the inner-maximization problem δ ∈ S it can be the case that moving in the direction −∇ θ g ( θ, δ ) increases the robust training loss i.e., the classiﬁer becomes less, rather than more, robust.

Original: This can happen at any stage, independently of the local optimality of θ .
Revised: This can happen at any stage, independently of the local optimality of θ .

Original: For a non-locally-optimal θ ∈ R d , the construction of the counterexamples relies on the following observation: if for any gradient computed at one inner-max solution, there exist another gradient (at a different inner-max solution) forming an obtuse angle, then no single inner-max solution yields a descent direction.
Revised: For a non-locally-optimal θ ∈ R d , the construction of the counterexamples relies on the following: if for any gradient computed at one inner-max solution, there exist another gradient (at a different inner-max solution) forming an obtuse angle, then no single inner-max solution yields a descent direction.

Original: Consequently, it suffices to ensure that for any gradient that can be found by solving the inner problem, there exists another one that has a negative inner product with it.
Revised: Consequently, it sufﬁces to ensure that for any gradient that can be found by solving the inner problem, there exists another one that has a negative inner product with it.

Original: Precisely, our counterexample 2 is carefully crafted so that this property holds.
Revised: Precisely, our counterexample 2 is carefully crafted so that this property holds.

Original: We conclude that the updates of Adversarial Training might have an effect that is contrary to its goal of increasing the robustness of a model.

Original: In order to address this problem, we now turn to improving the Adversarial Training approach by leveraging Danskin’s Theorem (Theorem 1) in a correct way.

Original:  D ANSKIN ’ S D ESCENT D IRECTION
Revised:  D ANSKIN ’ S D ESCENT D IRECTION

Original: Danskin’s Theorem implies that the directional derivative depends on all the solutions of the innermax problem S ⋆ ( θ ) c.f., Eq.
Revised: Danskin’s Theorem implies that the directional derivative depends on all the solutions of the innermax problem S (cid:63) ( θ ) c.f., Eq.

Original: One possible issue in Adversarial Training is relying on a single solution, as it does not necessarily lead to a descent direction c.f. counterexample 2.
Revised: One possible issue in Adversarial Training is relying on a single solution, as it does not necessarily lead to a descent direction c.f. counterexample 2.

Original: To fix this, we design an algorithm that uses multiple adversarial perturbations per data sample.
Revised: To ﬁx this, we design an algorithm that uses multiple adversarial perturbations per data sample.

Original: In theory, we can obtain the steepest descent direction for the robust loss on a batch { ( x i , y i ) :
Revised: In theory, we can obtain the steepest descent direction for the robust loss on a batch { ( x i , y i ) :

Original: i = 1 , . .
Revised: i = 1 , . .

Original: . , k } by solving the following min-max problem:
Revised: . , k } by solving the following min-max problem:

Original: [Equation]
Revised: [Equation]

Original: Unfortunately, the set  S ⋆ ( θ ) could beinfinite and thus remains out of reach for computationally tractable methods.
Revised: On the one hand, if the set of maximizers S (cid:63) ( θ ) is inﬁnite, Eq. (12) would be out of reach for computationally tractable methods.

Revised: On the other hand, the solution is trivial if there is a single maximizer , but we verify experimentally in Section 5 that such assumption is wrong in practice.

Original: In conclusion, a compromise has to be made in order to devise an efficient algorithm.
Revised: In conclusion, a compromise has to be made in order to devise an tractable algorithm that is relevant in practical scenarios.

Original: We will assume that the set of optimal adversarial perturbations is finite and given as:
Revised: First, we assume that the set of optimal adversarial perturbations is ﬁnite:

Original: [Equation]
Revised: [Equation]

Original: Under such assumption, it is possible to compute the steepest descent direction in Eq.
Revised: Under such assumption, it is possible to compute the steepest descent direction in Eq.

Original: (12) efficiently.
Revised: (12) efﬁciently.

Original: Theorem 2.
Revised: Theorem 2.

Revised: Let ∆ m be the m -dimensional simplex i.e., α ≥ 0 , (cid:80) m i

Original:  Suppose that S ⋆ ( θ ) =
Revised: = α i = 1 . Suppose that S (cid:63) ( θ ) =

Revised: S (cid:63)m ( θ ) :=

Original: { δ (1) , . . .
Revised: { δ (1) , . . .

Original: , δ ( m ) } . Denote by ∇ θ g ( θ, S ⋆ ( θ ))
Revised: , δ ( m ) } and denote by ∇ θ g ( θ, S (cid:63)m ( θ ))

Original: the matrix with columns ∇ θ g ( θ, δ ( i ) ) for i = 1 , . . .
Revised: the matrix with columns ∇ θ g ( θ, δ ( i ) ) for i = 1 , . . .

Original: , m .
Revised: , m .

Original: As long as θ is not a local minimizer of the robust loss ϕ ( θ ) = max δ ∈S g ( θ, δ ) , then the steepest descent direction of ϕ at θ can be computed as:
Revised: As long as θ is not a local minimizer of the robust loss φ ( θ ) = max δ ∈S g ( θ, δ ) , then the steepest descent direction of φ at θ can be computed as:

Original: [Equation]
Revised: [Equation]

Revised: We present the proof of Theorem 2 in Appendix C.

Original: where ∆ m is the m -dimensional simplex i.e., α ≥ 0 , (cid:80) mi =1 α i = 1 .
Revised: We now relax our initial ﬁniteness assumption Eq. (13), as it might not hold in practice. We show that it might sufﬁce to approximate the (possibly inﬁnite) set of maximizers S (cid:63) ( θ ) with a ﬁnite set S (cid:63)m ( θ ) .

Original: We present the proof of Theorem 2 in Appendix C.
Revised: If the direction γ (cid:63) deﬁned in Eq. (14) satisﬁes an additional inequality involving the ﬁnite set S (cid:63)m ( θ ) , it will be a certiﬁed descent direction. Theorem 3.

Original: This result motivates Danskin’s Descent Direction (Algorithm 1).
Revised: Suppose that ∇ θ g ( θ, δ ) is L -Lipschitz as a function of δ , i.e., (cid:107)∇ θ g ( θ, δ ) − ∇ θ g ( θ, δ (cid:48) ) (cid:107) 2 ≤ L (cid:107) δ − δ (cid:48) (cid:107) 2 . Let S (cid:63) ( θ ) be the set of solutions of the inner maximization problem, and let S (cid:63)m ( θ ) : = { δ (1) , . . .

Original: The assumptions in Theorem 2 that  the set of maximizers for the robust loss is finite is hard to verify in practice.
Revised: , δ ( m ) } be a ﬁnite set that (cid:15) -approximates S (cid:63) ( θ ) in the following sense: for any δ ∈ S (cid:63) ( θ ) there exists δ ( i ) ∈ S (cid:63)m ( θ ) such that (cid:107) δ − δ ( i ) (cid:107) 2 ≤ (cid:15) .

Original: Nevertheless, in Section 5 we verify that assuming a single solution isnot realistic. Hence, assuming a finite (possibly large) number of solutions is the least restrictive assumption we can make, while keeping the problem computationally tractable.
Revised: Let γ (cid:63) be as in Eq. If max δ ∈S (cid:63)m ( θ ) (cid:104) γ (cid:63) , ∇ θ g ( θ, δ ) (cid:105) < − L(cid:15) then γ (cid:63) is a descent direction for φ at θ . The Lipschitz gradient assumption in Theorem 3 is standard in the optimization literature.

Original: Exact access to these solutions is also out of reach.
Revised: We provide a proof of Theorem 3 in Appendix D.

Original: Rather, we assume there exists a heuristic algorithm providing a finite set ofapproximately optimal adversarial perturbations (Line 5 in Algorithm 1), which is an assumption that is anyways present in Adversarial Training.
Revised: This results motivate Danskin’s Descent Direction (Algorithm 1). We assume an oracle providing a ﬁnite set of adversarial perturbations S (cid:63)m ( θ ) that satiﬁes the approximation assumption in Theorem 3.

Original: Examples of such heuristic oracle algorithms are the so-called Fast Gradient Sign Method (FGSM) or Iterative FGSM (Kurakin et al., 2017) (referred to asPGD in Madry et al.
Revised: In particular, this does not require solving the inner-maximization problem to optimality, which is out of reach for computationally tractable methods and requires expensive branch-and-bound or MIP techniques (Zhang et al., 2022; Tjeng et al., 2019). Given S (cid:63)m ( θ ) , we compute γ (cid:63) as in Eq.

Original: The complexity ofan iteration in Algorithm 1 depends heavily on this choice.
Revised: (14), which corresponds to Line 7 of Algorithm 1.

Original: In Section 5 we will explore different choices and how it affects the the performance of the method. Using the set of adversarial perturbations S ⋆ ( θ ) provided by the oracle, in Line 7 of Algorithm 1 we obtain the value of α ⋆ as specified in Eq.
Revised: If the values of L and (cid:15) in Theorem 3 are not available (they might be hard to compute), we cannot certify that γ (cid:63) is a descent direction. However, note that given a set of adversarial examples S (cid:63)m ( θ ) , γ (cid:63) is still the best choice as it ensures we improve the loss on all elements of S (cid:63)m ( θ ) .

Original: Note that the optimization problem defining α ⋆ can be solved to arbitrary accuracy efficiently: It corresponds to the minimization of a smooth convex objective subject to the convex constraint α ∈ ∆ m .
Revised: Note that the optimization problem deﬁning α (cid:63) and γ (cid:63) can be solved to arbitrary accuracy efﬁciently: It corresponds to the minimization of a smooth convex objective subject to the convex constraint α ∈ ∆ m .

Original: First-order methods like Projected Gradient Descent (PGD) are well suited for such task.
Revised: First-order methods like Projected Gradient Descent (PGD) are well suited for such task.

Original: We use the accelerated PGD algorithm proposed in (Parikh et al., 2014, section 4.3) and pair it with the efficient simplex projection algorithm given in Duchi et al.
Revised: We use the accelerated PGD algorithm proposed in (Parikh et al., 2014, section 4.3) and pair it with the efﬁcient simplex projection algorithm given in Duchi et al.

Original: As the problem is smooth, a fixed step-size choice guarantees convergence.
Revised: As the problem is smooth, a ﬁxed step-size choice guarantees convergence.

Original: We set it as the inverse of the spectral norm of ∇ θ g ( θ, S ⋆ ( θ )) ⊤ ∇ θ g ( θ, S ⋆ ( θ )) and run the algorithm for a fixed number of iterations.
Revised: We set it as the inverse of the spectral norm of ∇ θ g ( θ, S (cid:63) ( θ )) (cid:62) ∇ θ g ( θ, S (cid:63) ( θ )) and run the algorithm for a ﬁxed number of iterations.

Revised: In practice, the theoretical oracle algorithm that computes the set S (cid:63)m ( θ ) is replaced by heuristics like performing multiple runs of the Fast Gradient Sign Method (FGSM) or Iterative FGSM (Kurakin et al., 2017) (referred to as PGD in Madry

Revised: et al.

Original: This allows us to obtain an approximation of α ⋆ which determines the steepest descent direction γ ⋆ , that we then use to update the parameters of the model.
Revised: The complexity of an iteration in Algorithm 1 depends on this choice. In Section 5 we explore different choices and how it affects the the performance of the method. heuristic like PGD

Original:  E XPERIMENTS 5. E XISTENCE OF MULTIPLE OPTIMAL ADVERSARIAL SOLUTIONS
Revised:  E XPERIMENTS 5. E XISTENCE OF MULTIPLE OPTIMAL ADVERSARIAL SOLUTIONS

Original: This section provides evidence that the set of optimal adversarial examples for a given sample is not a singleton.
Revised: This section provides evidence that the set of optimal adversarial examples for a given sample is not a singleton.

Original: The hypothesis is tested by using a ResNet-18 pretrained on CIFAR10 and computing▷ Using a heuristic like PGD multiple randomly initialized PGD-7 attacks for each image with ε = 8 255 .
Revised: The hypothesis is tested by using a ResNet-18 pretrained on CIFAR10 and computing multiple randomly initialized PGD-7 attacks for each image with ε = 8255 .

Original: We compute all pairwise ℓ 2 -distances between attacks for a given image and plot a joint histogram for 10 examples in Figure 2.
Revised: We compute all pairwise (cid:96) 2 -distances between attacks for a given image and plot a joint histogram for 10 examples in Figure 2.

Original: There is a clear separation away from zero for all pairwise distances indicating that the attacks are indeed distinct in the input space.
Revised: There is a clear separation away from zero for all pairwise distances indicating that the attacks are indeed distinct in the input space.

Original: Additionally, we plot a histogram over the adversarial losses for each image.
Revised: Additionally, we plot a histogram over the adversarial losses for each image.

Original: An example is provided in Figure 2, which is corroborated by similar results for other images (see Figure 6§B).
Revised: An example is provided in Figure 2, which is corroborated by similar results for other images (see Figure 6 § B).

Original: We find that the adversarial losses all concentrate with low variance far away from the clean loss.
Revised: We ﬁnd that the adversarial losses all concentrate with low variance far away from the clean loss.

Original: This confirms that all perturbations are in fact both strong and distinct. 
Revised: This conﬁrms that all perturbations are in fact both strong and distinct. [Figure]

Original:  .k
Revised: .1 .k

Original: δ − δ 0 k0.0 .1 .1 . 
Revised: δ − δ 0 k0 . [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original:  5. E XPLORING THE O PTIMIZATION L ANDSCAPE OF DDD AND S TANDARD A DVERSARIAL T RAINING
Revised: D e n s i t y [Figure] 5. E XPLORING THE O PTIMIZATION L ANDSCAPE OF DDD AND S TANDARD A DVERSARIAL T RAINING

Original: Having established that there exist multipe adversarial examples, we now show that the gradients computed can exhibit the behaviors discussed in Section 3.
Revised: Having established that there exist multipe adversarial examples, we now show that the gradients computed can exhibit the behaviors discussed in Section 3.

Original: In a first synthetic example we borrow from (Orabona, 2019, Chapter 6), we consider the function g
Revised: In a ﬁrst synthetic example we borrow from (Orabona, 2019, Chapter 6), we consider the function g

Original: ( θ, δ )
Revised: ( θ, δ )

Original: =
Revised: =

Original: δ (cid:0) θ 21 + ( θ 2 + 1) 2 (cid:1) + (1 − δ ) (cid:0) θ 21 + ( θ 2 − 1) 2 (cid:1) where θ ∈ R 2 and δ ∈ [0 , 1] .
Revised: δ (cid:0) θ 21 + ( θ 2 + 1) 2 (cid:1) + (1 − δ ) (cid:0) θ 21 + ( θ 2 − 1) 2 (cid:1) where θ ∈ R 2 and δ ∈ [0 , 1] .

Original: As can be seen from Figure 1a and Figure 1b, following a gradient computed at a single example leads to a increase in the objective and an unstable optimization behavior despite the use of a decaying step-size.
Revised: As can be seen from Figure 1a and Figure 1b, following a gradient computed at a single example leads to a increase in the objective and an unstable optimization behavior despite the use of a decaying step-size.

Original: In a second synthetic examples, we consider robust binary classification with a feed-forward neural network on a synthetic 2 -dimensional dataset, trained with batch gradient descent.
Revised: In a second synthetic examples, we consider robust binary classiﬁcation with a feed-forward neural network on a synthetic 2 -dimensional dataset, trained with batch gradient descent.

Original: We observe that during training, after an initial phase where all gradients computed at different perturbations point roughly in the same direction, we begin to observe pairs of gradients with negative innerproducts (see Figure 3 (left)).
Revised: We observe that during training, after an initial phase where all gradients computed at different perturbations point roughly in the same direction, we begin to observe pairs of gradients with negative innerproducts (see Figure 3 (left)).

Original: That means that following one of those gradients would lead to an increase of the robust loss, as shown by the different optimization behavior (see Figure 3 (center)) [Figure]
Revised: That means that following one of those gradients would lead to an increase of the robust loss, as shown by the different optimization behavior (see Figure 3 (center)).

Original: Therefore, the benefits DDD kick in later in training, once the loss has stabilized and the innersolver starts outputting gradients with negative inner products.
Revised: Therefore, the beneﬁts DDD kick in later in training, once the loss has stabilized and the innersolver starts outputting gradients with negative inner products.

Original: Indeed, we see that in the middle of training (iteration 250), DDD finds a descent direction of the (linearized) robust objective
Revised: Indeed, we see that in the middle of training (iteration 250), DDD ﬁnds a descent direction of the (linearized) robust objective, whereas all individual gradients lead to an increase.

Original: [Figure]
Revised: [Figure]

Original: whereas all individual gradients lead to an increase.

Original: [Figure]
Revised: [Figure]

Original: 5. A CCURACY /R OBUSTNESS COMPARISON OF DDD VS A DVERSARIAL T RAINING
Revised: A CCURACY /R OBUSTNESS COMPARISON OF DDD VS A DVERSARIAL T RAINING

Original: We compare the robust test and training error of Adversarial Training vs our proposed method DDD, on the CIFAR10 benchmark.
Revised: We compare the robust test and training error of Adversarial Training vs our proposed method DDD, on the CIFAR10 benchmark.

Original: As baseline we use ℓ ∞ -PGD with ϵ = 8 / 255 , α = 2 / 255 , n inner = 7 .
Revised: As baseline we use (cid:96) ∞ -PGD with (cid:15) = 8 / 255 , α = 2 / 255 , n inner = 7 .

Original: We train a ResNet18 with SGD, using the settings from Pang et al.
Revised: We train a ResNet18 with SGD, using the settings from Pang et al.

Original: (2021), Table 1 except for some modifications noted below.
Revised: (2021), Table 1 except for some modiﬁcations noted below.

Original: This means SGD with hyperparameters lr = 0 .
Revised: This means SGD with hyperparameters lr = 0 .

Original: 1 , momentum= 0 . 

Revised: 1 , momentum= 0 .

Original: (not the default 0 .
Revised: (not the default 0 .

Original: 9 , we explain why below), batch size = 128 and weight decay = 5 e − 4 .
Revised: 9 , we explain why below), batch size = 128 and weight decay = 5 e − 4 .

Original: We run for 200 epochs, no warmup, decreasing lr by a factor of 0 .
Revised: We run for 200 epochs, no warmup, decreasing lr by a factor of 0 .

Original: 1 at 50% and 75% of the epochs.
Revised: 1 at 50% and 75% of the epochs.

Original: Satisfying theoretical assumptions: Real world architectures are often not covered by theory while simple toy examples are often far removed from practice.
Revised: Satisfying theoretical assumptions: Real world architectures are often not covered by theory while simple toy examples are often far removed from practice.

Original: To demonstrate the real world impact of our results, we therefore study a setting where the conditions of Danskin’s Theorem hold, but which also uses standard building blocks used by practitioners, specifically replacing ReLU with CELU(Barron, 2017), replacing BatchNorm (BN) (Ioffe & Szegedy, 2015) with GroupNorm (GN)
Revised: To demonstrate the real world impact of our results, we therefore study a setting where the conditions of Danskin’s Theorem hold, but which also uses standard building blocks used by practitioners, speciﬁcally replacing ReLU with CELU(Barron, 2017), replacing BatchNorm (BN) (Ioffe & Szegedy, 2015) with GroupNorm (GN)

Original: (Wu & He, 2018) and removing momentum.
Revised: (Wu & He, 2018) and removing momentum.

Original: This ensures differentiability, removes intrabatch dependencies and ensures each update depends only on the descent direction found at that step respecively.
Revised: This ensures differentiability, removes intrabatch dependencies and ensures each update depends only on the descent direction found at that step respecively.

Original: We present more detailed justification in Appendix B.2 due to space constraints and additionally show an ablation study on the effect of our modifications in (Fig. 4b) 1 .
Revised: We present more detailed justiﬁcation in Appendix B.2 due to space constraints and additionally show an ablation study on the effect of our modiﬁcations in (Fig. 4b) 1 .

Original: Our main results can be seen in Fig. 4a.
Revised: Our main results can be seen in Fig. 4a.

Original: The robust accuracy of the DDD-trained model increases much more rapidly in the early stages, it increases more after the first drop in the learning rate, and is more stable when compared to the baseline.
Revised: The robust accuracy of the DDD-trained model increases much more rapidly in the early stages, it increases more after the ﬁrst drop in the learning rate, and is more stable when compared to the baseline.

Original: Figure 4b also gives evidence that our method has (generally positive or neutral) effects in all settings.
Revised: Figure 4b also gives evidence that our method has (generally positive or neutral) effects in all settings.

Original: Using ReLU instead of CELU re-introduces the characteristic bump in robust accuracy that has led to early stopping becoming standard practice in robust training.
Revised: Using ReLU instead of CELU re-introduces the characteristic bump in robust accuracy that has led to early stopping becoming standard practice in robust training.

Original: It also diminishes the benefit of DDD, but DDD remains on par with PGD in terms of training speed and decays slightly less towards the end of the training.
Revised: It also diminishes the beneﬁt of DDD, but DDD remains on par with PGD in terms of training speed and decays slightly less towards the end of the training.

Original: Adding momentum does not help either method in terms of training speed and makes them behave almost identically.
Revised: Adding momentum does not help either method in terms of training speed and makes them behave almost identically.

Original: Finally, BN seems to significantly ease the optimisation for both methods, raising overall performance and amplifying the bump on both methods.
Revised: Finally, BN seems to signiﬁcantly ease the optimisation for both methods, raising overall performance and amplifying the bump on both methods.

Original: Here, PGD actually reaches a higher maximum robust accuracy and rises faster initially, but then converges to a lower value.
Revised: Here, PGD actually reaches a higher maximum robust accuracy and rises faster initially, but then converges to a lower value.

Original: This implies that some benefits of DDD remain even outside the setting covered by the theory. 
Revised: This implies that some beneﬁts of DDD remain even outside the setting covered by the theory. [Figure]

Original: Although these are promising results indicating that DDD can give real world benefits in terms of iterations and reduce the need for early stopping, it is worth asking whether once could get the same 
Revised: Although these are promising results indicating that DDD can give real world beneﬁts in terms of iterations and reduce the need for early stopping, it is worth asking whether once could get the same beneﬁt with a simpler or cheaper method. The ﬁnal robust accuracies obtained are very close

Original: [Figure]
Revised: [Figure]

Original: benefit with a simpler or cheaper method. The final robust accuracies obtained are very close, and the increased convergence rate in terms of steps comes at a more than 10x slowdown due to having to perform 10 independent forward-backward passes and then solving an additional inner problem.
Revised: and the increased convergence rate in terms of steps comes at a more than 10x slowdown due to having to perform 10 independent forward-backward passes and then solving an additional inner problem.

Original: Additionally, it could be argued that these results are to be expected and trivial: we are spending 10x the compute to get 10x the gradients.
Revised: Additionally, it could be argued that these results are to be expected and trivial: we are spending 10x the compute to get 10x the gradients.

Original: One might even say there is no need to solve the inner product and a simpler method to select the best adversary would suffice.
Revised: One might even say there is no need to solve the inner product and a simpler method to select the best adversary would sufﬁce.

Original: In Fig.
Revised: In Fig.

Original: 5a we address these concerns by comparing Fig.
Revised: 5a we address these concerns by comparing Fig.

Original: 4a to the results of the following variants attempting to match the computational complexity: PGD- 70 runs a single PGD adversary for 10x the number of steps, PGD- 70 − 1 t runs a single PGD adversary for 10x the number of steps, using a 1 /t learning rate decay after leaving the ”standard” PGD regime (i.e. after 8 adversary steps) to converge closer to an optimal adversarial example, PGD-maxruns ten parallel, independent PGD adversaries for each image and select the adversarial example that induces the largest loss.
Revised: 4a to the results of the following variants attempting to match the computational complexity: PGD- 70 runs a single PGD adversary for 10x the number of steps, PGD- 70 − 1 t runs a single PGD adversary for 10x the number of steps, using a 1 /t learning rate decay after leaving the ”standard” PGD regime (i.e. after 8 adversary steps) to converge closer to an optimal adversarial example, PGD-maxruns ten parallel, independent PGD adversaries for each image and select the adversarial example that induces the largest loss.

Original: Finally, PGD-min-10 runs ten parallel, independent PGD adversaries for each image, then computes the gradients and selects the one with the lowest norm.
Revised: Finally, PGD-min-10 runs ten parallel, independent PGD adversaries for each image, then computes the gradients and selects the one with the lowest norm.

Original: This is an approximation of DDD that avoids solving Line 7 in Algorithm 1.
Revised: This is an approximation of DDD that avoids solving Line 7 in Algorithm 1.

Original: Additionally

Original: [Figure]

Original: in Fig.
Revised: In Fig.

Original: 5b we create a DDD variant based on the FAST adversary (Wong et al., 2020) (using ϵ = 8 / 255 , α = 10 / 255 ), while retaining PGD for the evaluation attack and compare against vanilla FAST in our setting (so without BN, momentum and using CELU) as well as a FAST-maxvariant analoguous to PGD-max- [Figure]
Revised: 5b we create a DDD variant based on the FAST adversary (Wong et al., 2020) (using (cid:15) = 8 / 255 , α = 10 / 255 ). Using PGD for the evaluation attack, we compare against vanilla FAST in our setting (no BN, momentum and using CELU) as well as a FAST-max-10 variant analoguous to PGD-max-10.

Original: As we can see in Fig.
Revised: As we can see in Fig.

Original: 5a, every step of the pipeline of DDD seems to be necessary, with none of the PGD variants achieving the fast initial rise in robustness.
Revised: 5a, every step of the pipeline of DDD seems to be necessary, with none of the PGD variants achieving the fast initial rise in robustness.

Original: PGD- 70 − 1 t and PGD-min-10 reach a higher final robust accuracy, which we attribute to the higher quality adversarial example and informed selection respectively.
Revised: PGD- 70 − 1 t and PGDmin-10 reach a higher ﬁnal robust accuracy, which we attribute to the higher quality adversarial example and informed selection respectively.

Original: This isalso corroborated in Fig.
Revised: This is corroborated in Fig.

Original: 5b.
Revised: 5b.

Original: Using a single step adversary is sufficient to speed up convergence in the early stages of training, but does not reach the same final robust accuracy.
Revised: Using a single step adversary is sufﬁcient to speed up convergence in the early stages of training, but does not reach the same ﬁnal robust accuracy.

Original: PGD and DDD seem to behave similarly in the later stages of training.
Revised: PGD and DDD seem to behave similarly in the later stages of training.

Original: Hence, we would suggest aslightly computationally cheaper DDD variant which uses single ascent steps (FAST) in the beginning of training andswitches to PGDvariants in the later stages.
Revised: We would suggest a computationally cheaper DDD variant which uses single ascent steps (FAST) in the beginning of training and PGD in the later stages.

Original: In any case, the bulk of the overhead lies in the subroutine in Line 7 of Algorithm 1.
Revised: In any case, the bulk of the overhead lies in the subroutine in Lineof Algorithm 1.

Original: A faster approximate solution could also speed up the method significantly.
Revised: A faster approximate solution could also speed up the method signiﬁcantly.

Original: Such incremental improvements are left for future work.
Revised: Such incremental improvements are left for future work Neverthelss, in Appendix B.4 we explore some modiﬁcations that can reduce the runtime of Algorithm 1 by at least 70% while retaining its beneﬁts.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Wang et al.
Revised: Wang et al.

Original: (2019) derive suboptimality bounds for the robust training problem, under a locally strong concavity assumption on the inner-maximization problem.
Revised: (2019) derive suboptimality bounds for the robust training problem, under a locally strong concavity assumption on the inner-maximization problem.

Original: However, such results do not  extend to Neural Networks, as the inner-maximization problem is not strongly concave, in general.
Revised: However, such results do not [Figure] extend to Neural Networks, as the inner-maximization problem is not strongly concave, in general.

Original: In contrast, we do not make unrealistic assumptions like strong concavity, and we deal with the existence multiple solutions of the inner-maximization problem.
Revised: In contrast, we do not make unrealistic assumptions like strong concavity, and we deal with the existence multiple solutions of the inner-maximization problem.

Original: In Nouiehed et al.
Revised: In Nouiehed et al.

Original: (2019), it is shown that if the inner-maximization problem is unconstrained and satisfies the PL-condition, it is differentiable, and the gradient can be computed after obtaining a single solution of the problem.
Revised: (2019), it is shown that if the inner-maximization problem is unconstrained and satisﬁes the PL-condition, it is differentiable, and the gradient can be computed after obtaining a single solution of the problem.

Original: However, in the robust learning problem the adversary is usually constrained to a compact set, and the PL condition does not hold generically.
Revised: However, in the robust learning problem the adversary is usually constrained to a compact set, and the PL condition does not hold generically.

Original: This renders such assumptions hard to justify in the AT setting.
Revised: This renders such assumptions hard to justify in the AT setting.

Original: Tramer & Boneh (2019); Maini et al.
Revised: Tramer & Boneh (2019); Maini et al.

Original: (2020) study robustness to multiple perturbation types , which might appear similar to our approach, but is not.
Revised: (2020) study robustness to multiple perturbation types , which might appear similar to our approach, but is not.

Original: Such works strike to train models that are simultaneously robust against ℓ ∞ - and ℓ 2 -bounded perturbations, for example.
Revised: Such works strike to train models that are simultaneously robust against (cid:96) ∞ - and (cid:96) 2 -bounded perturbations, for example.

Original: In contrast, we focus on a single perturbation type, and we study how to use multiple adversarial examples of the same sample to improve the update directions of the network parameters.
Revised: In contrast, we focus on a single perturbation type, and we study how to use multiple adversarial examples of the same sample to improve the update directions of the network parameters.

Original: Finally, we back our claim that the falseness of Madry et al.
Revised: Finally, we back our claim that the falseness of Madry et al.

Original: (2018, Corollary C.2.) is not wellknown in the literature on Adversarial Training.
Revised: (2018, Corollary C.2.) is not wellknown in the literature on Adversarial Training.

Original: For example, such result is included in the textbook (Vorobeychik et al., 2018, Proposition 8.1).
Revised: For example, such result is included in the textbook (Vorobeychik et al., 2018, Proposition 8.1).

Original: It has also been either reproduced or mentioned in conference papers like Liu et al.
Revised: It has also been either reproduced or mentioned in conference papers like Liu et al.

Original: (2020, Section 2), Viallard et al.
Revised: (2020, Section 2), Viallard et al.

Original: (2021, Appendix B), Wei & Ma (2020, Section 5) and possibly many others.
Revised: (2021, Appendix B), Wei & Ma (2020, Section 5) and possibly many others.

Original: This supports our claim that raising awareness about the mistake in the proof is an important contribution.
Revised: This supports our claim that raising awareness about the mistake in the proof is an important contribution.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper we presented a formal proof, counter examples and evidence about the real world impact of the fact that a foundational corollary of the Adversarial Training literature is in fact false.
Revised: In this paper we presented a formal proof, counter examples and evidence about the real world impact of the fact that a foundational corollary of the Adversarial Training literature is in fact false.

Original: Raising awareness about an incorrect claim that has been present in the Adversarial Training literature may provide opportunities to develop improved variants of the method.
Revised: Raising awareness about an incorrect claim that has been present in the Adversarial Training literature may provide opportunities to develop improved variants of the method.

Original: Indeed, we see some improvents in an implementable algorithm that align with our theoretical arguments: DDD exploits multiple approximate solutions of the inner-maximization problem, yields better updates for the parameters of the network and improves the optimization dynamics.
Revised: Indeed, we see some improvents in an implementable algorithm that align with our theoretical arguments: DDD exploits multiple approximate solutions of the inner-maximization problem, yields better updates for the parameters of the network and improves the optimization dynamics.

Original: However, it is important toalso remember the limitations and opportunities for future work: our algorithm requires multiple forward backward passes followed by an expensive additional inner problem.
Revised: However, it is important to remember the limitations and opportunities for future work: our algorithm requires multiple forward-backward passes and one additional optimization problem.

Original: Reducing the number of forward backward passes as well as the cost of the 2nd inner problem would be the highest priority to makeexploiting our results truly practical.
Revised: Reducing the overhead over the vanlla PGD method would certainly make our results truly practical.

Original: All the while, non-smooth activations and the use of Batch Normalization or momentum still falls outside the scope of existing theory but might achieve better performance in benchmarks.
Revised: Non-smooth activations and the use of Batch Normalization or momentum still falls outside the scope of existing theory but might achieve better performance in benchmarks.

Original: To date, this requires using precise hyperparameters and tricks like early-stopping, that have only been found to work a-posteriori through extensive trial and error.
Revised: To date, this requires using precise hyperparameters and tricks like early-stopping, that have only been found to work a-posteriori through extensive trial and error.

Original: Since we observe lower decay even in the non-smooth BatchNorm setting, future work extending the analysis to cover this case might help alleviate this cost.
Revised: Since we observe lower decay even in such setting, future work extending the analysis to cover this case might help alleviate this cost.

Original: A M ORE ON COUNTEREXAMPLES
Revised: A M ORE ON COUNTEREXAMPLES

Original: Here we give more details on the construction of the counterexamples.
Revised: Here we give more details on the construction of the counterexamples.

Original: First observe that for a given point θ 0 , and a direction γ , if there exists a δ 0 ∈ S ⋆ ( θ 0 ) such that ⟨ γ, ∇ θ g  θ 0 , δ ) ⟩ > 0 , then γ is not a descent direction since D
Revised: First observe that for a given point θ 0 , and a direction γ , if there exists a δ 0 ∈ S (cid:63) ( θ 0 ) such that (cid:104) γ, ∇ θ g ( θ 0 , δ ) (cid:105) > 0 , then γ is not a descent direction since D

Original: [Equation]
Revised: [Equation]

Original: In order to ensure that no descent directions can be recovered by solving the inner-maximization, it suffices to guarantee that for any δ ∈ S ⋆ ( θ 0 ) , there exists 
Revised: In order to ensure that no descent directions can be recovered by solving the inner-maximization, it sufﬁces to guarantee that for any δ ∈ S (cid:63) ( θ 0 ) , there exists δ (cid:48) ∈ S (cid:63) ( θ 0 ) such that [Equation] [Equation] [Equation]

Original: δ ′ ∈ S ⋆ ( θ 0 ) such that ⟨∇ θ g ( θ 0 , δ ′ ) , ∇ θ g ( θ 0 , δ ) ⟩ < 0 .
Revised: (cid:48) ) , ∇ [Equation] g ( θ 0 , δ ) (cid:105) < 0 .

Original: This way, neither −∇ θ g ( θ 0 , δ ) nor −∇ θ g
Revised: This way, neither −∇ θ g ( θ 0 , δ ) nor −∇ θ g

Original: θ 0 , δ ′ ) would be descent directions.
Revised: θ 0 , δ (cid:48) ) would be descent directions.

Original: It easy to generate instances verifying the above using linear functions.
Revised: It easy to generate instances verifying the above using linear functions.

Original: More formally, by taking any family of vectors V = { v 1 , . .
Revised: More formally, by taking any family of vectors V = { v 1 , . .

Original: . , v n } such that for any i ∈ { 1 , . . . , n } there exists j ∈ { 1 , . .
Revised: . , v n } such that for any i ∈ { 1 , . . . , n } there exists j ∈ { 1 , . .

Original: . , n }
Revised: . , n }

Original: such that ⟨ v i , v j ⟩ < 0 , we can construct the objective g ( θ, δ )
Revised: such that (cid:104) v i , v j (cid:105) < 0 , we can construct the objective g ( θ, δ )

Original: = (cid:80)
Revised: = (cid:80)

Original: δ i v ⊤ i ( θ − θ 0 )
Revised: δ i v (cid:62) i ( θ − θ 0 )

Original: − H ( δ ) , where δ is in the n -dimensional Simplex and H is the Shannon entropy.
Revised: − H ( δ ) , where δ is in the n -dimensional Simplex and H is the Shannon entropy.

Original: Solving the inner-maximization would yield any one of the vectors { v 1 , . .
Revised: Solving the inner-maximization would yield any one of the vectors { v 1 , . .

Original: . , v n } , and by construction, none of them are descent directions.
Revised: . , v n } , and by construction, none of them are descent directions.

Original: B E XPERIMENTS B. 
Revised: B E XPERIMENTS B.

Original: M
Revised: [Figure]

Original: ULTIPLE ATTACKS

Original: [Figure]
Revised: [Figure]

Original: n s i t
Revised: much clean

Original: [Figure]
Revised: (see Figure 6 for comparison with the clean loss)

Original: B. J USTIFYING OUR MODIFICATIONS
Revised: B. J USTIFYING OUR MODIFICATIONS

Original: For Danskin’s Theorem Theorem 1 to hold, we require the function to be differentiable.
Revised: For Danskin’s Theorem Theorem 1 to hold, we require the function to be differentiable.

Original: To satisfy differentiability, we replace ReLU with CELU (Barron, 2017) , which has been found to have comparable performance and sometimes outperform ReLU (Dubey et al., 2022) [Figure]
Revised: To satisfy differentiability, we replace ReLU with CELU (Barron, 2017) , which has been found to have comparable performance and sometimes outperform ReLU (Dubey et al., 2022).

Original: To operate on individual images and remove the batch-wise correlations across samples we replace
Revised: To operate on individual images and remove the batch-wise correlations across samples we replace

Original: BatchNorm (BN) (Ioffe & Szegedy, 2015) with GroupNorm (GN) (Wu & He, 2018) 2 .
Revised: BatchNorm (BN) (Ioffe & Szegedy, 2015) with GroupNorm (GN) (Wu & He, 2018) 2 .

Original: Finally, to make each update depend only on the current state, we set momentum = 0 . 0 .
Revised: Finally, to make each update depend only on the current state, we set momentum = 0 . 0 .

Original: Since momentum is standard practice in the CV community and works like Yan et al.
Revised: Since momentum is standard practice in the CV community and works like Yan et al.

Original: (2018) argue that it can improve generalisation, we rely on our ablation to show that removing it is safe.
Revised: (2018) argue that it can improve generalisation, we rely on our ablation to show that removing it is safe.

Original: B. F URTHER DETAILS ON SYNTHETIC EXPERIMENTS
Revised: B. F URTHER DETAILS ON SYNTHETIC EXPERIMENTS

Original: The synthetic experiment in Fig.
Revised: The synthetic experiment in Fig.

Original: 1a is conducted with the following settings.
Revised: 1a is conducted with the following settings.

Original: The innermaximization is approximated with 10 steps of projected gradient ascent in order to match the traditional AT setting.
Revised: The innermaximization is approximated with 10 steps of projected gradient ascent in order to match the traditional AT setting.

Original: The outer iterations have a decaying 0 .
Revised: The outer iterations have a decaying 0 .

Original: 5 √ k step-size schedule.
Revised: 5 √ k step-size schedule.

Original: We observe the same erratic behavior for PGD with a fixed outer stepsize, while DDD consitently remains well-behaved.
Revised: We observe the same erratic behavior for PGD with a ﬁxed outer stepsize, while DDD consitently remains well-behaved.

Original: The synthetic experiment in Fig.
Revised: The synthetic experiment in Fig.

Original: 3 is conducted on a dataset of size 100 in dimension 2 where the coordinates are standard Gaussian.
Revised: 3 is conducted on a dataset of size 100 in dimension 2 where the coordinates are standard Gaussian.

Original: The neural network is a 2-layer network with ELU activation with a hidden layer of width 2 .
Revised: The neural network is a 2-layer network with ELU activation with a hidden layer of width 2 .

Original: The inner solver is PGD with 10 steps with stepsize 0 .
Revised: The inner solver is PGD with 10 steps with stepsize 0 .

Original: 1 and optimizes over the unit cube.
Revised: 1 and optimizes over the unit cube.

Original: The outer step-size is 0 .
Revised: The outer step-size is 0 .

Original: and the weights are optimized with full batch gradient descent.
Revised: and the weights are optimized with full batch gradient descent

Original:  The linear approximation at iteration 250 of the robust loss consits of taking the 10 adversarial examples computed at iteration 250 and approximating it with
Revised: [Figure] The linear approximation at iteration 250 of the robust loss consits of taking the 10 adversarial examples computed at iteration 250 and approximating it with

Original: [Equation]
Revised: [Figure]

Original: Interestingly we do not observe the same drastic improvement over PGD when observing the nonlinearized loss at iteration250 .
Revised: Interestingly we do not observe the same drastic improvement over PGD when observing the nonlinearized loss at iteration

Original: [Figure]
Revised: [Figure]

Original: R o bu s t L o ss
Revised: B. I MPROVING THE RUNNING TIME

Original: [Table]
Revised: [Figure]

Revised: (b)

Revised: Epoch evolution of a more efﬁcient implementation

Revised: DDD

Revised: While the focus of this paper is not to obtain a state-of-the-art method, it does matter whether it is feasible to efﬁciently capture the beneﬁt of DDD.

Revised: The naive implementation has about a 10 −times overhead compared to PGD, mainly due to three bottlenecks (in descending impact)

Revised: 1. for k -DDD, generating k adversarial examples with PGD as the base attack involves a ktimes overhead 2.

Revised: then k separate gradient samples need to be computed on these adversarial examples, which involes k forward-backward passes 3.

Revised: ﬁnally, one additional optimization problem needs to be solved.

Revised: While steps 1) and 2) can be somewhat parallelized, they still cause a massive increase in compute and memory.

Revised: We therefore adopt two heuristic approaches to speed up the algorithm while (hopefully) maintaining it’s beneﬁts:

Revised: since later in training the beneﬁts of DDD appear to diminish, we linearly decay the number of gradients sampled k from 10 down to 1 along the 200 epochs (referred to as decay )

Revised: we also adopt a method of creating k unique batches from only 2 independent adversarial attacks (described below in Appendix B.4.1, referred to as comb ).

Revised: We evaluate this method using both PGD and FAST as base attacks and show the results in Fig.

Revised: 8a and Fig.

Revised: 8b.

Revised: As can be seen, DA-PGD-decay-comb and DA-PGD-comb both enjoy a massive speedup in wallclock time (reducing the 12 × overhead to about 3 × ) while retaining the improved per-step progress of base DDD .

Revised: B.4. C OMBINATORIAL BATCH CONSTRUCTION

Revised: Suppose we have a batch of data-label pairs ( x i , y i ) of size B .

Revised: In order to construct k ≤ 2 B different gradients by computing only 2 adversarial examples per data sample x i in the batch we do the following:

Revised: for each i = 1 , . . .

Revised: , B compute δ i, 0 , δ i, 1 two adversarial examples using the data-label pair ( x i , y i ) in the batch.

Revised: for each j = 1 , . . .

Revised: , k repeat the following steps: 3.

Revised: Deﬁne ∆ =

Revised: [ ] as an empty list. 4. generate a random bitvector b ⊆ { 0 , 1 } B of length B 5.

Revised: when b i is 0 we append δ i, 0 to ∆ , otherwise when b i is 1 we append δ i, 1 to ∆ .

Revised: compute the gradient w.r.t.

Revised: the network parameters using the perturbations in ∆

Revised: While this still incures overhead of computing k gradients, it greatly reduces running time as seen in Fig.

Revised: 8b and could further improved by e.g. reusing gradients from past epochs to construct the examples.

Original: C P ROOF OF T HEOREM 2.
Revised: C P ROOF OF T HEOREM 2.

Original: The steepest descent direction is computed, following Eq.
Revised: The steepest descent direction is computed, following Eq.

Original: (4) as:
Revised: (4) as:

Original: [Equation]
Revised: [Equation]

Original: Whenever θ is not a local optimum, there exists a non-zero descent direction.
Revised: Whenever θ is not a local optimum [Equation] there exists a non-zero descent direction.

Original: In this case we can relax the constraint that ∥ γ ∥ 2 = 1 to ∥ γ ∥ 2 ≤ 1 without changing the solutions or optimal value of (15), which is strictly negative: 
Revised: In this case we can relax the constraint that (cid:107) γ (cid:107) 2 = 1 to (cid:107) γ (cid:107) 2 ≤ 1 without changing the solutions or optimal value of (15), which is strictly negative: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We can now transform (15) into a bilinear convex-concave min-max problem, subject to convex and compact constraints:
Revised: We can now transform (15) into a bilinear convex-concave min-max problem, subject to convex and compact constraints:

Original: [Equation]
Revised: [Equation]

Original: By Sion’s minimax Theorem Sion (1958), we can solve Eq.
Revised: By Sion’s minimax Theorem Sion (1958) [Equation] we can solve Eq.

Original: (17) by swapping the operator order:
Revised: (17) by swapping the operator order:

Original: [Equation]
Revised: [Equation]

Original: Finally, by noting that squaring the objective function in the right-hand side of Eq.
Revised: Finally, by noting that squaring the objective function in the right-hand side of Eq.

Original: (18) does not change the set of solutions, we arrive at the formula for α ⋆ in Eq.
Revised: (18) does not change the set of solutions, we arrive at the formula for α (cid:63) in Eq.

Original: Indeed for a solution α ⋆ to this problem we have
Revised: Indeed for a solution α (cid:63) to this problem we have

Original: [Equation]
Revised: [Equation]

Revised: where the denominator is nonnegative as the optimal objective value is nonzero c.f. Eq.

Revised: D P ROOF OF T HEOREM 3.

Revised: For any δ ∈ S (cid:63)

Revised: ( θ ) let i ( δ ) ∈ { 1 , . . .

Revised: , m } be such that (cid:107) δ

Revised: ( i ( δ ))

Revised: − δ (cid:107) 2 ≤ (cid:15) .

Revised: That is, we map any maximizer δ to an index i ∈ { 1 , . . .

Revised: , m } such that the corresponding perturbation δ ( i ) in the ﬁnite set S (cid:63)m ( θ ) is at most at an (cid:15) distance.

Revised: This map can be constructed by the assumption on S (cid:63)m ( θ ) .

Original: {+For any γ such that (cid:107) γ (cid:107) 
+}
Revised: {+For any γ such that (cid:107) γ (cid:107) 
+}

Revised: = 1 we have

Original: [Equation]
Revised: [Equation]

Revised: Taking the supremum over δ ∈ S (cid:63) ( θ ) on the left-hand-side we obtain

Revised: [Equation]

Revised: [Equation]

Original: where the denominator is nonnegative as the optimal objective value is nonzero c.f. Eq.
Revised: Hence if the supremum on the right-hand-side is strictly smaller than − L(cid:15) we have that D γ φ ( θ ) < 0 , which yields the desired result. E P ROOF OF L EMMA Assume the limit that deﬁnes ˆ D γ φ ( θ ) exists (and is ﬁnite).

Revised: [Equation]