Abstract: Adversarial training is an effective methodology to train deep neural networks which are robust against adversarial, norm-bounded perturbations. However, the computational cost of adversarial training grows prohibitively as the size of the model and number of input dimensions increase. Further, training against less expensive and therefore weaker adversaries produces models that are robust against weak attacks but break down under attacks that are stronger. This is often attributed to the phenomenon of gradient obfuscation; such models have a highly non-linear loss surface in the vicinity of training examples, making it hard for gradient-based attacks to succeed even though adversarial examples still exist. In this work, we introduce a novel regularizer that encourages the loss to behave linearly in the vicinity of the training data, thereby penalizing gradient obfuscation while encouraging robustness. We show via extensive experiments on CIFAR-10 and ImageNet, that models trained with our regularizer avoid gradient obfuscation and can be trained significantly faster than adversarial training. Using this regularizer, we exceed current state of the art and achieve 47% adversarial accuracy for ImageNet with ∞ adversarial perturbations of radius 4/255 under an untargeted, strong, white-box attack. Additionally, we match state of the art results for CIFAR-10 at 8/255.

Chongli Qin
James Martens
Sven Gowal
Dilip Krishnan
Google Krishnamurthy
Dvijotham Deepmind
Alhussein Fawzi
Soham De
Robert Stanforth Deepmind

Adversarial Robustness through Local Linearization

Introduction In a seminal paper, Szegedy et al.  [22]  demonstrated that neural networks are vulnerable to visually imperceptible but carefully chosen adversarial perturbations which cause them to output incorrect predictions. After this revealing study, a flurry of research has been conducted with the focus of making networks robust against such adversarial perturbations  [14, 16, 17, 25] . Concurrently, researchers devised stronger attacks that expose previously unknown vulnerabilities of neural networks  [24, 4, 1, 3] . Of the many approaches proposed  [19, 2, 6, 21, 15, 17] , adversarial training  [14, 16]  is empirically the best performing algorithm to train networks robust to adversarial perturbations. However, the cost of adversarial training becomes prohibitive with growing model complexity and input dimensionality. This is primarily due to the cost of computing adversarial perturbations, which is incurred at each step of adversarial training. In particular, for each new mini-batch one must perform multiple iterations of a gradient-based optimizer on the network's inputs to find the perturbations.  1  As each step of this optimizer requires a new backwards pass, the total cost of adversarial training scales as roughly the number of such steps. Unfortunately, effective adversarial training of ImageNet often requires large number of steps to avoid problems of gradient obfuscation  [1, 24] , making it significantly more expensive than conventional training. One approach which can alleviate the cost of adversarial training is training against weaker adversaries that are cheaper to compute. For example, by taking fewer gradient steps to compute adversarial examples during training. However, this can produce models which are robust against weak attacks, but break down under strong attacks -often due to gradient obfuscation. In particular, one form of gradient obfuscation occurs when the network learns to fool a gradient based attack by making the loss surface highly convoluted and non-linear (see  Fig 1) , an effect which has also been observed by Papernot et al  [18] . This non-linearity prevents gradient based optimization methods from finding an adversarial perturbation within a small number of iterations  [4, 24] . In contrast, if the loss surface was linear in the vicinity of the training examples, which is to say well-predicted by local gradient information, gradient obfuscation cannot occur. In this paper, we take up this idea and introduce a novel regularizer that encourages the loss to behave linearly in the vicinity of the training data. We call this regularizer the local linearity regularizer (LLR). Empirically, we find that networks trained with LLR exhibit far less gradient obfuscation, and are almost equally robust against strong attacks as they are against weak attacks. The main contributions of our paper are summarized below: • We show that training with LLR is significantly faster than adversarial training, allowing us to train a robust ImageNet model with a 5× speed up when training on 128 TPUv3 cores  [9] . • We show that LLR trained models exhibit higher robustness relative to adversarially trained models when evaluated under strong attacks. Adversarially trained models can exhibit a decrease in accuracy of 6% when increasing the attack strength at test time for CIFAR-10, whereas LLR shows only a decrease of 2%. • We achieve new state of the art results for adversarial accuracy against untargeted white-box attack for ImageNet (with = 4/255 2 ): 47%. Furthermore, we match state of the art results for CIFAR 10 (with = 8/255): 52.81% 3 . • We perform a large scale evaluation of existing methods for adversarially robust training under consistent, strong, white-box attacks. For this we recreate several baseline models from the literature, training them both for CIFAR-10 and ImageNet (where possible).  4  2 Background and Related Work We denote our classification function by f (x; θ) : x → R C , mapping input features x to the output logits for classes in set C, i.e. p i (y|x; θ) = exp (f i (x; θ)) / j exp (f j (x; θ)), with θ being the model parameters and y being the label. Adversarial robustness for f is defined as follows: a network is robust to adversarial perturbations of magnitude at input x if and only if argmax i∈C f i (x; θ) = argmax i∈C f i (x + δ; θ) ∀δ ∈ B p ( ) = {δ : δ p ≤ }. (1) In this paper, we focus on p = ∞ and we use B( ) to denote B ∞ ( ) for brevity. Given the dataset is drawn from distribution D, the standard method to train a classifier f is empirical risk minimization (ERM), which is defined by: min θ E (x,y)∼D [ (x; y, θ)]. Here, (x; y, θ) is the standard cross-entropy loss function defined by (x; y, θ) = −y T log (p(x; θ)) , (2) where p i (x; θ) is defined as above, and y is a 1-hot vector representing the class label. While ERM is effective at training neural networks that perform well on heldout test data, the accuracy on the test set goes to zero under adversarial evaluation. This is a result of a distribution shift in the data induced by the attack. To rectify this, adversarial training  [17, 14]  seeks to perturb the data distribution by performing adversarial attacks during training. More concretely, adversarial training minimizes the loss function E (x,y)∼D max δ∈B( ) (x + δ; y, θ) , (3) where the inner maximization, max δ∈B( ) (x + δ; y, θ), is typically performed via a fixed number of steps of a gradient-based optimization method. One such method is Projected-Gradient-Descent (PGD) which performs the following gradient step: δ ← Proj (δ − η∇ δ (x + δ; y, θ)) , (4) where Proj(x) = argmin ξ∈B( ) x − ξ . Another popular gradient-based method is to use the sign of the gradient  [8] . The cost of solving Eq (3) is dominated by the cost of solving the inner maximization problem. Thus, the inner maximization should be performed efficiently to reduce the overall cost of training. A naive approach is to reduce the number of gradient steps performed by the optimization procedure. Generally, the attack is weaker when we do fewer steps. If the attack is too weak, the trained networks often display gradient obfuscation as shown in Fig  1 . Since the introduction of adversarial training, a corpus of work has researched alternative ways of making networks robust. One such approach is the TRADES method  [27] , which is a form of regularization that optimizes the trade-off between robustness and accuracy -as many studies have observed these two quantities to be at odds with each other  [23] . Others, such as work by Ding et al  [7]  adaptively increase the perturbation radius by find the minimal length perturbation which changes the output label. Some have proposed architectural changes which promote adversarial robustness, such as the "denoise" model  [25]  for ImageNet. The work presented here is a regularization technique which encourages the loss function to be well approximated by its linear Taylor expansion in a sufficiently small neighbourhood. There has been work before which uses gradient information as a form of regularization  [20, 17] . The work presented in this paper is closely related to the paper by Moosavi et al  [17] , which highlights that adversarial training reduces the curvature of (x; y, θ) with respect to x. Leveraging an empirical observation (the highest curvature is along the direction ∇ x (x; y, θ)), they further propose an algorithm to mimic the effects of adversarial training on the loss surface. The algorithm results in comparable performance to adversarial training with a significantly lower cost.
 Motivating the Local Linearity Regularizer As described above, the cost of adversarial training is dominated by solving the inner maximization problem max δ∈B( ) (x + δ). Throughout we abbreviate (x; y, θ) with (x). We can reduce this cost simply by reducing the number of PGD (as defined in Eq (4)) steps taken to solve max δ∈B( ) (x + δ). To motivate the local linearity regularizer (LLR), we start with an empirical analysis of how the behavior of adversarial training changes as we increase the number of PGD steps used during training. We find that the loss surface becomes increasingly linear (as captured by the local linearity measure defined below) as we increase the number of PGD steps.
 Local Linearity Measure Suppose that we are given an adversarial perturbation δ ∈ B( ). The corresponding adversarial loss is given by (x + δ). If our loss surface is smooth and approximately linear, then (x + δ) is well approximated by its first-order Taylor expansion (x) + δ T ∇ x (x). In other words, the absolute difference between these two values, g(δ; x) = (x + δ) − (x) − δ T ∇ x (x) , (5) is an indicator of how linear the surface is. Consequently, we consider the quantity γ( , x) = max δ∈B( ) g(δ; x), (6) to be a measure of how linear the surface is within a neighbourhood B( ). We call this quantity the local linearity measure.   6 )) is large (on the order of 10) when we train with just one or two steps of PGD for inner maximization, (2a). In contrast, γ( , x) becomes increasingly smaller (on the order of 10 −1 ) as we increase the number of PGD steps to 4 and above, (2b). The x-axis is the number of training iterations and the y-axis is γ( , x), here = 8/255 for CIFAR-10.
 Empirical Observations on Adversarial Training We measure γ( , x) for networks trained with adversarial training on CIFAR-10, where the inner maximization max δ∈B( ) (x + δ) is performed with 1, 2, 4, 8 and 16 steps of PGD. γ( , x) is measured throughout training on the training set  5  . The architecture used is a wide residual network  [26]  28 in depth and 10 in width (Wide-ResNet-28-10). The results are shown in Fig  2a and 2b . Fig  2a  shows that when we train with one and two steps of PGD for the inner maximization, the local loss surface is extremely non-linear at the end of training. An example visualization of such a loss surface is given in Fig A1a  .  However, when we train with four or more steps of PGD for the inner maximization, the surface is relatively well approximated by (x) + δ T ∇ x (x) as shown in Fig 2b  .  An example of the loss surface is shown in Fig A1b  .  For the adversarial accuracy of the networks, see Table  A1 .
 Local Linearity Regularizer (LLR) From the section above, we make the empirical observation that the local linearity measure γ( , x) decreases as we train with stronger attacks  6  . In this section, we give some theoretical justifications of why local linearity γ( , x) correlates with adversarial robustness, and derive a regularizer from the local linearity measure that can be used for training of robust models.
 Local Linearity Upper Bounds Adversarial Loss The following proposition establishes that the adversarial loss (x + δ) is upper bounded by the local linearity measure, plus the change in the loss as predicted by the gradient (which is given by |δ T ∇ x (x)|). Proposition 4.1. Consider a loss function (x) that is once-differentiable, and a local neighbourhood defined by B( ). Then for all δ ∈ B( ) | (x + δ) − (x)| ≤ |δ T ∇ x (x)| + γ( , x). (7) See Appendix B for the proof. From Eq  (7)  it is clear that the adversarial loss tends to (x), i.e., (x + δ) → (x), as both |δ ∇ x (x)| → 0 and γ( ; x) → 0 for all δ ∈ B( ). And assuming (x + δ) ≥ (δ) one also has the upper bound (x + δ) ≤ (x) + |δ T ∇ x (x)| + γ( , x).
 Local Linearity Regularization (LLR) Following the analysis above, we propose the following objective for adversarially robust training L(D) = E D (x) + λγ( , x) + µ|δ T LLR ∇ x (x)| LLR , (8) where λ and µ are hyper-parameters to be optimized, and δ LLR = argmax δ∈B( ) g(δ; x) (recall the definition of g(δ; x) from Eq (5)). Concretely, we are trying to find the point δ LLR in B( ) where the linear approximation (x) + δ T ∇ x (x) is maximally violated. To train we penalize both its linear violation γ( , x) = (x + δ LLR ) − (x) − δ T LLR ∇ x (x) , and the gradient magnitude term δ T LLR ∇ x (x) , as required by the above proposition. We note that, analogous to adversarial training, LLR requires an inner optimization to find δ LLR -performed via gradient descent. However, as we will show in the experiments, much fewer optimization steps are required for the overall scheme to be effective. Pseudo-code for training with this regularizer is given in Appendix E.
 Local Linearity Measure γ( ; x) bounds the adversarial loss by itself Interestingly, under certain reasonable approximations and standard choices of loss functions, we can bound |δ ∇ x (x)| in terms of γ( ; x). See Appendix C for details. Consequently, the bound in Eq  (7)  implies that minimizing γ( ; x) (along with the nominal loss (x)) is sufficient to minimize the adversarial loss (x + δ). This prediction is confirmed by our experiments. However, our experiments also show that including |δ ∇ x (x)| in the objective along with (x) and γ( ; x) works better in practice on certain datasets, especially ImageNet. See Appendix F.3 for details.
 Experiments and Results We perform experiments using LLR on both CIFAR-10 [13] and ImageNet  [5]  datasets. We show that LLR gets state of the art adversarial accuracy on CIFAR-10 (at = 8/255) and ImageNet (at = 4/255) evaluated under a strong adversarial attack. Moreover, we show that as the attack strength increases, the degradation in adversarial accuracy is more graceful for networks trained using LLR than for those trained with standard adversarial training. Further, we demonstrate that training using LLR is 5× faster for ImageNet. Finally, we show that, by linearizing the loss surface, models are less prone to gradient obfuscation.
 CIFAR-10: The perturbation radius we examine is = 8/255 and the model architectures we use are Wide-ResNet-28-8, Wide-ResNet-40-8  [26] . Since the validity of our regularizer requires (x) to be smooth, the activation function we use is softplus function log(1 + exp(x)), which is a smooth version of ReLU. The baselines we compare our results against are adversarial training (ADV)  [16] , TRADES  [27]  and CURE  [17] . We recreate these baselines from the literature using the same network architecture and activation function. The evaluation is done on the full test set of 10K images.
 ImageNet: The perturbation radii considered are = 4/255 and = 16/255. The architecture used for this is from  [11]  which is ResNet-152. We use softplus as activation function. For = 4/255, the baselines we compare our results against is our recreated versions of ADV  [16]  and denoising model (DENOISE)  [25] .  7  For = 16/255, we compare LLR to ADV  [16]  and DENOISE  [25]  networks which have been published from the the literature. Due to computational constraints, we limit ourselves to evaluating all models on the first 1K images of the test set. To make sure that we have a close estimate of the true robustness, we evaluate all the models on a wide range of attacks these are described below.
 Evaluation Setup To accurately gauge the true robustness of our network, we tailor our attack to give the lowest possible adversarial accuracy. The two parts which we tune to get the optimal attack is the loss function for the attack and its corresponding optimization procedure. The loss functions used are described below, for the optimization procedure please refer to Appendix F.1.
 Loss Functions: The three loss functions we consider are summarized in Table  1 . We use the difference between logits for the loss function rather than the cross-entropy loss as we have empirically found the former to yield lower adversarial accuracy.
 Attack Name Loss Function Metric Random-Targeted max δ∈B( ) f r (x + δ) − f t (x + δ) Attack Success Rate Untargeted max δ∈B( ) f s (x + δ) − f t (x + δ) Adversarial Accuracy Multi-Targeted [10] max δ∈B( ) max i∈C f i (x + δ) − f t (x + δ) Adversarial Accuracy Table  1 : This shows the loss functions corresponding to the attacks we use for evaluation and also the metric we measure on the test set for each of these attacks. Notation-wise, s = argmax i =t fi(x + δ) is the highest logit excluding the logits corresponding to the correct class t, note s can change through the optimization procedure. For the Random-Targeted attack, r is a randomly chosen target label that is not t and does not change throughout the optimization. C stands for the set of class labels. For the Multi-Targeted attack we maximize fi(x + δ) − fT (x + δ) for all i ∈ C, and consider the attack successful if any of the individual attacks on each each target class i are successful. The metric used on the Random-Targeted attack is the attack success rate: the percentage of attacks where the target label r is indeed the output label (this metric is especially important for ImageNet at = 16/255). For the other attacks we use the adversarial accuracy as the metric which is the accuracy on the test set after the attack. For CIFAR-10, the main adversarial accuracy results are given in Table  2 . We compare LLR training to ADV  [16] , CURE  [17]  and TRADES  [27] , both with our re-implementation and the published models  8  . Note that our re-implementation using softplus activations perform at or above the published results for ADV, CURE and TRADES. This is largely due to the learning rate schedule used, which is the similar to the one used by TRADES  [27] .
 Results for Interestingly, for adversarial training (ADV), using the Multi-Targeted attack for evaluation gives significantly lower adversarial accuracy compared to Untargeted. For ImageNet, we compare against adversarial training (ADV)  [16]  and the denoising model (DE-NOISE)  [25] . The results are shown in Table  3 . For a perturbation radius of 4/255, LLR gets 47% adversarial accuracy under the Untargeted attack which is notably higher than the adversarial accuracy obtained via adversarial training which is 39.70%. Moreover, LLR is trained with just two-steps of PGD rather than 30 steps for adversarial training. The amount of computation needed for each method is further discussed in Sec 5.2.1. Further shown in Table  3  are the results for = 16/255. We note a significant drop in nominal accuracy when we train with LLR to perturbation radius 16/255. When testing for perturbation radius of 16/255 we also show that the adversarial accuracy under Untargeted is very poor (below 8%) for all methods. We speculate that this perturbation radius is too large for the robustness problem. Since adversarial perturbations should be, by definition, imperceptible to the human eye, upon inspection of the images generated using an adversarial attack (see Fig  F4 ) -this assumption no longer holds true. The images generated appear to consist of super-imposed object parts of other classes onto the target image. This leads us to believe that a more fine-grained analysis of what should constitute "robustness for ImageNet" is an important topic for debate.
 Runtime Speed For ImageNet, we trained on 128 TPUv3 cores  [9] , the total training wall time for the LLR network (4/255) is 7 hours for 110 epochs. Similarly, for the adversarially trained (ADV) networks the total wall time is 36 hours for 110 epochs. This is a 5× speed up.
 Accuracy Degradation: Strong vs Weak Evaluation The resulting model trained using LLR degrades gracefully in terms of adversarial accuracy when we increase the strength of attack, as shown in Fig 3  .  In particular, Fig  3a  shows that, for CIFAR-10, when the attack changes from Untargeted to Multi-Targeted, the LLR's accuracy remains similar with only a 2.18% drop in accuracy. Contrary to adversarial training (ADV), where we see a 5.64% drop in accuracy. We also see similar trends in accuracy in Table  2 . This could indicate that some level of obfuscation may be happening under standard adversarial training. As we empirically observe that LLR evaluates similarly under weak and strong attacks, we hypothesize that this is because LLR explicitly linearizes the loss surface. An extreme case would be when the surface is completely linear -in this instance the optimal adversarial perturbation would be found with just one PGD step. Thus evaluation using a weak attack is often good enough to get an accurate gauge of how it will perform under a stronger attack.  We use either the standard adversarial training objective (ADV-1, ADV-2) or the LLR objective (LLR-1, LLR-2) and taking one or two steps of PGD to maximize each objective. To train LLR-1/2, we only optimize the local linearity γ( , x), i.e. µ in Eq. (  8 ) is set to zero. We see that for adversarial training, as shown in Figs 4a, 4c, the loss surface becomes highly non-linear and jagged -in other words obfuscated. Additionally in this setting, the adversarial accuracy under our strongest attack is 0% for both, see Table  F3 . In contrast, the loss surface is smooth when we train using LLR as shown in Figs  4b, 4d . Further, Table  F3  shows that we obtain an adversarial accuracy of 44.50% with the LLR-2 network under our strongest evaluation. We also evaluate the values of γ( , x) for the CIFAR-10 test set after these networks are trained. This is shown in
 Conclusions We show that, by promoting linearity, deep classification networks are less susceptible to gradient obfuscation, thus allowing us to do fewer gradient descent steps for the inner optimization. Our novel linearity regularizer promotes locally linear behavior as justified from a theoretical perspective. The resulting models achieve state of the art adversarial robustness on the CIFAR-10 and Imagenet datasets, and can be trained 5× faster than regular adversarial training. Figure 1 : 1 Figure 1: Example of gradient obfuscated surface. The color of the surface denotes the prediction of the network.
 Figure 2 : 2 Figure 2: Plots showing that γ( , x) (Eq (6)) is large (on the order of 10) when we train with just one or two
 2 Figure 4 : 24 Figure 4: Comparing the loss surface, (x), after we train using just 1 or 2 steps of PGD for the inner maximization of either the adversarial objective (ADV) max δ∈B( ) (x + δ) or the linearity objective (LLR) γ( , x) = max δ∈B( ) (x + δ) − (x) − δ T ∇ (x) . Results are shown for image 126 in test set of CIFAR-10, the nominal label is deer. ADV-i refers to adversarial training with i PGD steps, similarly with LLR-i.
 Fig F3. The values of γ( , x) are comparable when we train with LLR using two steps of PGD to adversarial training with 20 steps of PGD. By comparison, adversarial training with two steps of PGD results in much larger values of γ( , x).
 Table 2 : 2 Model accuracy results for CIFAR-10. Our LLR regularizer performs the best under the strongest attack (highlighted column). (S) denotes softplus activation; (R) denotes ReLU activation; and models with (S, R) are our implementations. Robustness CIFAR-10: Wide-ResNet-28-8 (8/255) Methods Nominal FGSM-20 Untargeted Multi-Targeted Attack Strength Weak Strong Very Strong ADV[16] 87.25% 48.89% 45.92% 44.54% CURE[17] 80.76% 39.76% 38.87% 37.57% ADV(S) 85.11% 56.76% 53.96% 48.79% CURE(S) 84.31% 48.56% 47.28% 45.43% TRADES(S) 87.40% 51.63 50.46% 49.48% LLR (S) 86.83% 54.24% 52.99% 51.13% CIFAR-10: Wide-ResNet-40-8 (8/255) ADV(R) 85.58% 56.32% 52.34% 46.89% TRADES(R) 86.25% 53.38% 51.76% 50.84% ADV(S) 85.27% 57.94% 55.26% 49.79% CURE(S) 84.45% 49.41% 47.69% 45.51% TRADES(S) 88.11% 53.03% 51.65% 50.53% LLR (S) 86.28% 56.44% 54.95% 52.81%
 Table 3 : 3 The accuracy obtained are 49.79% and 55.26% respectively. Evaluation using Multi-Targeted attack consistently gave the lowest adversarial accuracy throughout. Under this attack, the methods which stand out amongst the rest are LLR and TRADES. Using LLR we get state of the art results with 52.81% adversarial accuracy. LLR gets 47% adversarial accuracy for 4/255 -7.30% higher than DENOISE and ADV. For 16/255, LLR gets similar robustness results, but it comes at a significant cost to the nominal accuracy. Note Multi-Targeted attacks for ImageNet requires looping over 1000 labels, this evaluation can take up to several days even on 50 GPUs thus is omitted from this table. The column of the strongest attack is highlighted. ImageNet: ResNet-152 (4/255) Methods PGD steps Nominal Untargeted Random-Targeted Accuracy Success Rate ADV 30 69.20% 39.70% 0.50% DENOISE 30 69.70% 38.90% 0.40% LLR 2 72.70% 47.00% 0.40% ImageNet: ResNet-152 (16/255) ADV [25] 30 64.10% 6.30% 40.00% DENOISE [25] 30 66.80% 7.50% 38.00% LLR 10 51.20% 6.10% 43.80%
 The annotations on each node denotes no. of PGD steps × no. of random restarts (see Appendix F.1). (3a), background color denotes whether the attack is Untargeted (blue) or Multi-Targeted (orange). (3b), we only use Untargeted attacks. Adversarial Accuracy 48 50 52 54 56 58 60 62 1 × 1 1 × 1 10 × 1 10 × 1 Stronger Attack → Untargeted 20 × 1 20 × 1 50 × 1 50 × 1 100 × 1 100 × 1 200 × 1 200 × 1 Multi Targeted 1000 × 20 1000 × 20 1000 × 1 1000 × 1 200 × 20 200 × 20 LLR ADV Adversarial Accuracy 38 40 42 44 46 48 50 1 × 1 1 × 1 10 × 1 10 × 1 Stronger Attack → 20 × 1 50 × 1 20 × 1 50 × 1 100 × 1 100 × 1 1000 × 1 200 × 1 1000 × 1 200 × 1 LLR ADV (a) CIFAR-10 (8/255) (b) ImageNet (4/255) Figure 3: Adversarial accuracy shown for CIFAR-10, (3a), and ImageNet, (3b), as we increase the strength of attack. (3a) shows LLR's adversarial accuracy degrades gracefully going from 53.32% to 51.14% (-2.18%) while ADV's adversarial accuracy drops from 54.43% to 48.79% (-5.64%). (3b) LLR remains 7.5% higher in terms of adversarial accuracy (47.20%) compared to ADV (39.70%). For ImageNet, see Fig3b, the adversarial accuracy trained using LLR remains significantly higher (7.5%) than the adversarially trained network going from a weak to a stronger attack.
			 While computing the globally optimal adversarial example is NP-hard [12] , gradient descent with several random restarts was empirically shown to be quite effective at computing adversarial perturbations of sufficient quality. 2  This means that every pixel is perturbed independently by up to 4 units up or down on a scale where pixels take values ranging between 0 and 255. 3  We note that TRADES [27]  gets 55% against a much weaker attack; under our strongest attack, it gets 52.5%. 4  Baselines created are adversarial training, TRADES and CURE [17] . Contrary to CIFAR-10, we are currently unable to achieve consistent and competitive results on ImageNet at = 4/255 using TRADES.
			 To measure γ( , x) we find max δ∈B( ) g(δ; x) with 50 steps of PGD using Adam as the optimizer and 0.1 as the step size. 6  Here, we imply an increase in the number of PGD steps for the inner maximization max δ∈B( ) (x + δ).
			 We attempted to use TRADES on ImageNet but did not manage to get competitive results. Thus they are omitted from the baselines.
			 Note the network published for TRADES [27]  uses a Wide-ResNet-34-10 so this is not shown in the table but under the same rigorous evaluation we show that TRADES get 84.91% nominal accuracy, 53.41% under Untargeted and 52.58% under Multi-Targeted. We've also ran ∞ DeepFool (not in the table as the attack is weaker) giving ADV(S): 64.29%, CURE(S): 58.73%, TRADES(S): 63.4%, LLR(S): 65.87%.


Abstract: Michael Smith, in his impressive book The Moral Problem , 1 claims that (1) 'convergence in the hypothetical desires of fully rational creatures is required for the truth of normative reason claims' (  173 ) and (2) we have reason to 'have some confidence [… that] there will be a convergence in our desires under conditions of full rationality.' (187). These claims play a crucial role in supporting Smith's distinctive and interesting meta-ethical position. However, I will here ignore the context of Smith's broader ambitions and focus on these two theses. I will argue that Smith does not give us adequate justification for either claim. This paper will have four sections. In §1 I will explain Smith's conception of fully rational deliberation. In §2 I discuss what it would be for desires to converge as Smith suggests. In §3 I consider Smith's conceptual claim that such convergence is necessary for there to be normative reasons at all. Finally, I address Smith's reasons for optimism about convergence. The first three sections consider Smith's first thesis above while the fourth section considers his second thesis.

Do the desires of rational agents converge?

1. The plausibility of the claim that the desires of all agents will converge after proper deliberation hinges crucially on how one characterizes such deliberation. One could simply claim that a person only counts as having deliberated properly if she reaches certain approved conclusions. This path would assure Smith's first thesis at the cost of invoking a substantive, nonproceduralist conception of proper deliberation. The interest in Smith's claims stems from his willingness to invoke an understanding of proper deliberation which is not conceptually tied to the deliberator arriving at any particular motivations. In this paper I will say that a conception of normative reasons for action is Humean iff (1) it claims that one's rational desires give one normative reasons (hereafter, reasons), and (2) it invokes a proceduralist notion of proper deliberation in which what makes a desire rational is that it would be arrived at if one successfully followed the procedure, and (3) this procedure is specified such that, at least in principle, any particular option could be rationally desired after successfully following the procedure, and (4) it is relativist in the sense that different agents might arrive in the right way at different desires and hence have divergent reasons. This last condition shows that, for the Humean, the normativity for A of the desires that A would arrive at in the approved way is not undermined should other agents arrive in the approved way at other desires. Smith clearly rejects the fourth thesis. He must also be rejecting the first or second thesis. Perhaps he would want to say that successfully following the procedure makes one's resultant desires rational, but that (contra thesis 1) rational desires only give one reasons when all rational agents converge in their desires. Alternatively, Smith might want to say that one's desires are only rational if everyone would reach them after proper deliberation (contra thesis 2). Smith finds himself in substantial agreement with Bernard Williams's account of fully rational deliberation, complaining only that Williams's account requires supplementation. 2 (158) Thus Smith and Williams largely agree about what it is for an agent to arrive at a desire in the right way. Smith glosses Williams's position in this way: in order for a deliberator to counts as fully rational '(i) the agent must have no false beliefs, (ii) the agent must have all relevant true beliefs, (iii) the agent must deliberate correctly.' (  156 ) Conditions (i) and (ii) claim that we can only be fully rational deliberators if we get the facts right. Smith's requirement of true belief shows that he is not trying to capture the notion of proper deliberation given one's limited time and access to the truth. Thus while we might have been rational in light of the information available to us (because, for example, the scientific community had not discovered certain facts) or that it was reasonable for us to collect, we were not fully rational deliberators in Smith's sense for we did not know all the facts. One sense of 'rational deliberation' is to make proper use of the information one has (or could reasonably be expected to get). Thus a person might have been rational in this sense to leave the building after the fire alarm went off even though it turns out that the building was not really on fire. Smith is clearly after another notion of rational deliberation, one perhaps well designed to determine what it is advisable to do or what will get us what we really want. Smith's targeted notion might be called 'ideally rational deliberation.' Several philosophers, myself included, have recently argued that this notion of ideally rational deliberation involves serious conceptual problems, but I want to focus on other issues here.  3  Condition (iii) adds possibilities for practical reasoning such as these: the agent adequately exercises her imagination in considering the options available, the agent makes no mistakes of means-end reasoning, the agent engages in constitutive reasoning 'such as deciding what would make for an entertaining evening, granted that one wants entertainment.'  (Williams 1981 : 104) Smith's main additions to Williams's above criteria are that the agent is not suffering from depression, apathy, or other debilitating attitudes, and the agent has a concern that her overall desire pattern be coherent and justified. (158-61) Crucially these possibilities for practical reason do not amount to the claim that it is criterial of proper deliberation that one reach a particular specified motivational profile. 2. Before considering Smith's arguments for his two theses we must better understand what we mean when we speak of the desires of all ideally rational agents converging. This is more complicated than it might seem. Different things might be thought to constitute convergence. A weak understanding of convergence would be that all fully rational agents agree about every claim of this form: A has normative reason to O in circumstances C. If a Humean account of reasons were correct and this was accepted, then, presumably, we would all agree, when fully rational, about all such reason claims. In fact, the most likely way to achieve such agreement would be for all rational agents to agree on an account of practical rationality; that is, agree about what makes it the case that A has reason to O in circumstances C. This kind of agreement is compatible with different agents having arbitrarily divergent ideally rational desires. Call this 'convergence about each agent's reasons.' This, it turns out, is not the notion of convergence that Smith is pointing to. Or rather, such agreement would seem necessary but not sufficient for the kind of convergence that Smith has in mind. Smith's understanding of convergence would be for all rational agents to converge on desires that have the same de se content. (169-70) If we both want you rather than me to get the larger slice of cake there is a sense in which we converge and a sense in which we do not. We both want the same state of the world to obtain, but we do not both want the cake for ourselves. If we both wanted cake for ourselves our desires would have the same de se content. A simple case of this sort of convergence would be if all fully rational agents desired only their own pleasure. In this case rational agents agree about what kind of things are desirable. Convergence in the objects of desire can be thicker or thinner. We might both want to listen to the Rolling Stones album Exile on Main Street very loud, or we might both want to listen to music. The former agreement in desire is thicker in that the shared object of appropriation is specified more specifically. It is likely that some very thin convergence can always be found between two people or perhaps between all rational agents. Perhaps all rational agents desire to be rational, to live a good life, or to avoid mistakes. If the convergence that Smith has in mind is convergence in the objects of desire, then it is crucial to know what level of generality he intends. Thick convergence among all rational agents would be very surprising and, perhaps, entail moral agreement, whereas very thin convergence would not be very interesting and would not entail moral agreement. Further, if we are not given a specific level of generality at which rational agents are supposed to converge, then one could worry that Smith could, in the face of any seeming disagreement, simply move in each such case to thinner and thinner areas of convergence. There could also be varying degrees of the extent of convergence. Rational agents might agree (at a fixed level of generality) about all, most, or just a few objects of desire. Smith's claim is that there is full convergence in desire amongst rational agents. A problem for the thesis that rational agents will converge on the objects of desire thickly described is that on matters of mere tastes agreement does not seem forthcoming. It is not plausible that every rational deliberator will like chocolate ice cream more than vanilla or Seinfeld more than Friends. Our desires diverge over such issues. Yet in many cases our divergent desires seem for all the world informed in the relevant sense and well insulated from consistency or justificatory pressures from other areas of our motivational make-up. There are strong grounds for thinking such divergence in tastes will survive rational deliberation. Smith concedes this point, but tries to show that this would not prevent the kind of full convergence he has in mind.  4  Smith admits that his preference for beer over wine provides him a reason to get beer rather than wine where our opposite preference provides us reason to get wine instead. Smith claims that this is no threat to full convergence because the preference is 'a relevant feature of our circumstances....' (171) This can be seen, Smith tells us, 'from the fact that I can quite happily agree with you that if I were in your circumstances -if I preferred wine to beer -then the fact that the local wine bar sells very good wine would constitute a reason for me to go there as well ….' (171) Smith continues: 'even if an agent's preferences may enter into a specification of the circumstances that she faces it may still be the case that whether or not she is rationally justified in taking her own preferences into account, and the way in which she is justified in taking them into account if she is, depends on whether fully rational agents would all converge on a desire which makes the preferences she has relevant to her choice.' (171) Smith is here claiming that rational agents might, despite their divergent tastes, nonetheless converge on desires of the following form: if I were in circumstances C (which includes having a certain preference profile P), then I want myself to choose O. At first sight such convergence might appear to be nothing more than convergence on instrumental rationality itself. It appears that the thought is simply this: whatever desires I find myself having, I want myself to satisfy those desires. But things are not so simple. Allan Gibbard notes that 'Ideals differ from tastes: I dislike spinach but think it a matter of taste; that means in part that although I dislike spinach I am willing to eat it if I like it. The norms I accept endorse eating it if I like it and not otherwise. I oppose cruelty unconditionally: I want myself not to be cruel even if, hypothetically, I should want to be.'  (Gibbard, 1990: 167)  Thus, for those that hold ideals in Gibbard's sense (and moral claims seem paradigmatically to be such ideals), what one would want for oneself if one had an altered preference ranking need not simply be maximally to satisfy those altered desires. Smith's claim seems to be that all rational agent's might converge in their de se desires about what they would want for themselves if they were in a particular circumstance (which includes having a certain preference profile). There are at least two different ways in which such convergence might be achieved. The first way would be for all rational agents to treat every preference as a matter of taste and never as an ideal. Such rational agents might agree that for any situation they find themselves in, they would want to maximally satisfy the preferences that they have in that situation. Convergence of this sort, I take it, would be compatible with a thoroughgoing Humean account of an agent's reasons for action. Thus I suppose it must not be this method of reaching convergence that Smith has in mind for he calls his proposal an 'anti-humean theory of normative reasons.' Smith, it would seem, must therefore have in mind a picture in which fully rational agents converge in their desires in a way that crucially involves them having ideals. How might such a picture go? The second way such convergence might occur, in this case involving agents with ideals, would be for all rational agents to agree fully about what counts as a matter of mere taste and what counts as a matter of ideals. It would not be important that the rational agents agree in their tastes, rather only that they agree about which of their preferences are to be treated as mere tastes. For where we treat our preferences as mere matters of taste, we presumably agree about what we want in situations in which we have this taste or that; namely, indulge our tastes as we find them. In this way Smith has found a nice way of acknowledging that fully rational agents would not converge on preferring chocolate ice cream to vanilla, yet still holding out the prospect that they might fully converge in their desires in an interesting and important sense. It would seem however that the rational agents would have to converge on their ideals in order for the full kind of convergence in desires that Smith envisions to occur. When two agents disagree in their ideals, they will sometimes disagree in their wants for themselves in circumstances C (which includes having a certain preference profile P). Thus for the kind of convergence that Smith has in mind to occur all rational agents must both (1) agree about what counts as a mere matter of taste and what counts as an ideal and (2) agree in their ideals. Imperfectly rational agents clearly disagree about what are matters of mere taste and what are ideals. Some want themselves to dress fashionably even in the situation in which they do not want to look fashionable, others treat this as a matter of taste. It must be claimed that this kind of disagreement would be extinguished upon becoming fully rational. It is worth noting that for two people to agree in their ideals they must agree about significantly more than just morality. For as the above example of fashion shows, a person can hold a preference as a personal ideal even when they think others would not be immoral or irrational for failing to share that ideal. Gibbard called such ideals 'existential: it is a choice of what kind of person to be, in a fundamental way, come what might, which the chooser does not take to be dictated by considerations of rationality.' (ibid.: 168) Thus it would seem that to vindicate Smith's conception fully rational agent's would have to converge fully not only about morality, but also about personal ideals. This would seem to necessitate that the process of becoming fully rational extinguish all existential ideals, or at least make false the claim that the holder of the existential ideal makes: namely that commitment to this ideal is not dictated by rationality. And as holding false beliefs is supposed to be incompatible with being fully rational, it would seem that Smith must hold that existential ideals must go if convergence is to take place. Let me here parenthetically acknowledge a clear and important virtue of Smith's attempt at an 'anti-Humean Theory of Normative Reasons.' It deals as successfully with matters of mere taste as does the Humean theory and this to my mind had been one of the clearest advantages of the do the desires of rational agents converge? 143 Humean account. Smith shows us how the anti-Humean can, perhaps effectively, avoid conceding this front. 3. Why must proper deliberation lead to convergence if we are to have reasons at all? Smith's conclusion here is remarkably strong. He claims the very concept of reasons demands convergence. Thus, apparently, the dominant Humean understanding of reason is, in a straightforward way, conceptually confused. I find Smith's clearest arguments for this to be the following: (1) such an assumption is necessary to underwrite the platitude that 'we are … potentially in agreement or disagreement with each other about what constitutes a reason and what doesn't.' (172), and (2) absent this sort of convergence I must see my reasons as being entirely dependent on the fact that my actual desires are as they are. But the shape of my actual desires (which, by hypothesis, will determine the shape of my desires after rational deliberation), Smith tells us, is an 'entirely arbitrary matter, one without any normative significance of its own.' (172) I will consider these arguments in turn. Consider first Smith's case that convergence is necessary to vindicate the thought that we can dispute reasons claims. He writes there is … a sense in which we can talk about rational justification or desirability simpliciter. When you and I talk about the reasons that there are for acting [in this sense], we are therefore talking about the same thing …. On the relative conception, however, matters are quite different. For in order to give a truth condition to the schematic claim 'It is desirable that p in c' we need to know from whose perspective the truth of the claim is to be assessed ….  [Thus]  if I say to you 'There is a reason for O-ing', and you deny this, we are therefore potentially talking about different things …. (166-67) All of this is true. But it does not support either the claim that on the Humean view there could not be a common subject matter when we discuss reasons, or that 'rational justification [concerning reasons] is itself a relative matter; that really there is only rational-justification-relative-tothis-person or rational-justification-relative-to-that.' (171) In fact these two claims are false. Smith claims that on Williams's Humean view when we speak of reasons 'it typically means reason me out of my mouth, reason you out of your's, reason her out of her's and so on.' (172) But, of course, we might both be talking about the reasons of the same person. We might be debating if A has reason to O in circumstances C. On the Humean account when we debate whether A has reason to O in circumstances C we are debating what A's motivations would be like if she were deliberating properly. The Humean thus holds that we can be right or wrong in such claims and we have a common subject matter when we discuss them. Someone who claimed that a person had an intrinsic reason to O, when that person had nothing in her motivational set that O answered to, would just be wrong. Thus the Humean framework allows us to vindicate the platitude that we have a common subject matter when we speak of a particular agent's reasons. Perhaps it is the case that, on the Humean view, we cannot find a common subject matter for other kinds of reason claims. Likely the Humean cannot vindicate certain kinds of reason claims that we might have hoped to vindicate (e.g., that there are reasons, such as moral reasons, which apply to all rational agents as such). But even if this is granted there is no reason here to suppose, as Smith does, that the Humean is forced to abandon the very notion of reasons altogether. Further, the way to justify reason claims such as A has reason to O in circumstances C is not relative to each individual. The Humean holds that there is a fact of the matter about this which is determined by the agent's motivations in counterfactual situations (which will admittedly be very difficult to determine) just as there is a fact of the matter about the effects of red wine on the body. And we justify believing the former sort of facts in the same non-person relative way that we justify the latter sort of facts. Smith's second argument claims that unless desires converged amongst fully rational deliberators, one's rational desires would be determined by what one (perhaps idiosyncratically) happened to find attractive. Smith claims that if this were so one's rational desires would depend on 'an entirely arbitrary matter' and 'arbitrariness is precisely a feature of a consideration that tends to undermine any normative significance it might initially appear to have.' (172-73) Smith could mean one of two things. First, he could mean that as a psychological matter an agent cannot take her own desires seriously, cannot herself find them to be a source of reasons for her, if they are arbitrary in this way. This is clearly false. The prevalence of sincere Humeans demonstrates this. The Humean may be wrong about reasons for action, but it must be allowed that she really believes her theory. The second interpretation of Smith's claim is that there is no justification for treating one's desires that are arbitrary in this way as creating reasons. But this is exactly what is at issue between the Humean and Smith. My rationally arbitrary preference (that is, a preference I could be rational without) for chocolate ice cream gives me a reason to choose it. What is incoherent about all my reasons being such? Smith has baldly claimed that there is a conceptual confusion in the notion of rationally arbitrary desires by themselves creating reasons. The predominant neo-Humean view holds that this is precisely what creates reasons. Thus Smith will have to do better. I myself have trouble seeing anything conceptually confused or problematically arbitrary in the thought that my rationally optional projects and commitments create reasons for me and not for you. Before moving on I want to briefly consider the plight of the agent that deliberates by successfully following the procedure that Smith recommends but unhappily discovers that other rational agents arrived at divergent desires. Smith tells us that this agent has no reason to satisfy her desires, in fact she has no reasons (recall I use 'reason' to refer to Smith's 'normative reason') at all for there are no reasons without convergence. Thus a lack of convergence, according to Smith, would make it the case that I had no reason to avoid pain or choose a flavour of ice cream I like rather than one I find disgusting. I find this suggestion difficult to take seriously. Does Smith really think the existence of such reasons is contingent on universal convergence? Perhaps Smith should only claim that an agent has a reason to O iff all rational agents would converge on a desire for O in the relevant circumstances rather than claim that for an agent to have any reasons all rational agents must converge on all desires. I would still disagree with this claim, but it would not flaunt what I regard as the absurdities mentioned in the previous paragraph. In some cases, like the preference for chocolate rather than vanilla ice cream, our bare liking provide us with a reason to choose one thing rather than another. Even if all rational agents do not converge in their desires, I like chocolate ice cream better than vanilla, and this is all I need to have a reason to choose it. It would seem that Smith must either claim that, in such cases, my getting chocolate ice cream does not benefit me more than getting vanilla or that I have no reason whatsoever in such a case to seek my benefit rather than my detriment. Neither claim strikes me as plausible. 4. Smith argues that convergence in desires amongst rational deliberators is quite plausible. Smith's main argument here is that 'the empirical fact that moral argument tends to elicit the agreement of our fellows gives us reason to believe that there will be a convergence in our desires under conditions of full rationality. For the best explanation of that tendency is our convergence upon a set of extremely unobvious a priori moral truths.' (187) Smith bolster this argument with three supplementary arguments: (1) often by focusing only on moral disagreement we ignore substantial agreement and the fact that 'we' share thick evaluative language which entails a fair degree of agreement in attitude, (2) although current moral disputes appear at times deadlocked, 'we must remember that in the past similarly entrenched disagreements were removed inter alia via a process of moral argument' and (3) some moral disagreement certainly can be explained as clearly arising from lack of rational deliberation. (  188 ) But much of the historical moral argumentation that has actually managed to produce consensus has been (1) factually and logically imperfect, (2) addressed to those poorly positioned to object, (3) addressed to those who share substantial common moral vocabulary, moral education, and cultural identification, and (4) offered by those who are persuasive for reasons other than the cogency of their position. Any historical tendency towards convergence that results from some combination of these four causes (and no doubt others) will not constitute evidence that rational agents will converge in their moral views. Since such causes undoubtedly played a very significant role in producing convergence, we cannot simply point towards a general historical tendency towards convergence (even if there were one) and claim it constitutes inductive evidence for Smith's case. Thus, as we might have suspected, Smith's historical case about a tendency towards convergence will have to be genuinely historical. It will have to persuade us of the crucial role of facts, logic, and reason is explaining the history of convergence and the secondary role of force, guile, and a shared thick moral vocabulary. The sort of convergence Smith holds out real hope for is convergence amongst people that start off as different in their desires, moral vocabulary, and moral education as we can imagine. The convergence in the objects of desire that Smith needs is not simply convergence amongst all actual humans (as if we could make ourselves right by killing dissenters) but rather amongst all possible agents no matter how initially divergent. But what is the history of convergence amongst agents with radically divergent desires, moral vocabularies, and moral educations when they are confronted with factually informed and logically impeccable argumentation? Certainly Smith will have to remind us of such cases. Smith needs a string of cases in which (1) convergence occurred, (2) it occurred for the right reasons, and (3) it occurred between radically different cultures. Smith relies quite plainly and inappropriately on convergence within an extended community. Smith points to convergence within a community that shares a common moral vocabulary of thick evaluative language. (188) This commitment to a common thick evaluative language, Smith rightly points out, reveals considerable moral agreement within the community. (188) But just because this shared thick evaluative language presupposes and reveals considerable moral agreement, convergence amongst those that share thick evaluative language is of very limited use in demonstrating the power of rational argumentation to itself manufacture this agreement. Rational argumentation has some power to lead people who actually agree about morality to see that they agree. This is not do the desires of rational agents converge? 147 doubted. But Smith needs to show that history provides a good inductive case that all fully rational agents, no matter how initially divergent they are prior to becoming fully rational, will agree on moral matters. Smith finds himself 'in substantial agreement with Derek Parfit' (214) when Parfit claims that Belief in God, or in many gods, prevented the free development of moral reasoning. Disbelief in God, openly admitted by a majority, is a very recent event, not yet completed. Because this event is so recent, Non-Religious Ethics is at a very early stage. We cannot yet predict whether, as in Mathematics, we will all reach agreement.  (Parfit 1984: 454)  But betraying such hopes undermines Smith's inductive historical argument for it suggests that the factually informed deliberators will not predicate their moral opinions on religious authority. And surely an overwhelming amount of historical ethical argument that has produced convergence has relied on religious beliefs and authority. It might be true, as Parfit says, that absent such a religious basis we lack any reason to suppose that there will not be convergence. But surely we must therefore also admit that past moral practices do not provide sufficient inductive grounds to suppose that such agreement will be forthcoming. Thus I conclude that Smith's arguments in support of the plausibility of extensive convergence are unconvincing and that he has failed to show that such convergence is necessary for the existence of reasons.  5  Bowling Green State University Bowling  Green,   			 Smith, 1994.  All otherwise unattributed references are to this work.
			 Williams 1981  spells out his neo-Humean account of instrumental rationality.
			  See Velleman 1988 , Sobel 1994 , Rosati 1995 , and Loeb 1995 Interestingly all these critiques are offered against the full information account of well-being rather than the remarkably comparable full information account of reasons for action that Williams and Smith are developing. These two theories, for all their similarity, seem to have developed and be discussed in isolation from each other.
			 Seemingly, Smith might have conceded that the case of mere tastes shows that there would not be full convergence on the objects of desire after fully rational deliberation, while continuing to insist that we have reason to be 'optimistic about the possibility of an agreement about what is right and wrong being reached under more idealized conditions of reflection and discussion.' (189)
			 I would like to thank David Copp, Michael Smith, Sara Worley and an anonymous referee for Analysis for their help with this paper.