Title: ADVERSARIAL INTERPOLATION TRAINING A SIMPLE APPROACH FOR IMPROVING MODEL ROBUSTNESS

Abstract: We propose a simple approach for adversarial training. The proposed approach utilizes an adversarial interpolation scheme for generating adversarial images and accompanying adversarial labels, which are then used in place of the original data for model training. The proposed approach is intuitive to understand, simple to implement and achieves state-of-the-art performance. We evaluate the proposed approach on a number of datasets including CIFAR10, CIFAR100 and SVHN. Extensive empirical results compared with several state-of-the-art methods against different attacks verify the effectiveness of the proposed approach.

1. INTRODUCTION:

Deep learning-based techniques have achieved outstanding performance in many tasks such as image classification (Krizhevsky et al., 2012) , speech recognition (Hinton et al., 2012) and video game playing (Mnih et al., 2015) . Despite of these encouraging progress, it has been shown that these models could be easily attacked by adversarial examples (Szegedy et al., 2014; Biggio et al., 2013; Carlini & Wagner, 2018; Lin et al., 2017) . The ubiquitous of adversarial examples across many tasks (Szegedy et al., 2014; Lin et al., 2017; Eykholt et al., 2018; Carlini & Wagner, 2018) and the fact that they are transferable between different models (Tramèr et al., 2017; Charles et al., 2019; Moosavi-Dezfooli et al., 2017) raise great concerns on security of such models and hinder their actual deployment in real world applications. Researchers have been actively working on understanding the cause of adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015) and approaches for improving model robustness against them (Madry et al., 2018; Tramèr et al., 2018; Liao et al., 2018; Yan et al., 2018) . A number of theories have been developed for explaining the existence of adversarial examples. Szegedy et al. (2014) explain that adversarial examples are possible because the image space is densely filled with low probability adversarial pockets. Goodfellow et al. (2015) argue that the adversarial examples are caused by the linear nature of deep networks. Tanay & Griffin (2016) provide the perspective that adversarial examples exist because the class boundary extends beyond the data sub-manifold and can be lying close to it in some cases. It has been further shown in some recent work that adversarial examples can be decomposed into categories with different causes including off-manifold ones (Stutz et al., 2019; Jacobsen et al., 2019) and those due to natural test error (Stutz et al., 2019; Jacobsen et al., 2019; Ford et al., 2019) .

Recently, Ilyas et al. (2019) provide a perspective that adversarial vulnerability is caused by nonrobust features. The reasoning is that there are abundant of useful correlations that exist in natural data, thus it is natural to expect the models could learn to exploit any of them if no preference is given (c.f. Table 7 ). However, models relying on superficial statistics (non-robust features) can be brittle and generalize poorly, thus suffering from adversarial attacks (Ilyas et al., 2019) . The natural idea is therefore using only robust features for learning. However robust features are not easy to construct directly (Madaan & Hwang, 2019) . In contrast, non-robust features are much easier to construct, using the standard adversarial example generation procedure (Szegedy et al., 2014; Goodfellow et al., 2015; Madry et al., 2018) . Therefore, we can leverage non-robust features instead for robust learning. This is typically achieved by constructing a robustified dataset where the new (perturbed) images are constructed by adding non-robust features to the clean images, and then performing model training using the perturbed images in place of the original ones. This procedure is essentially the standard adversarial training approach (Goodfellow et al., 2015; Madry et al., 2018) , which has been shown to be effective for defending against adversarial attacks (Athalye et al., 2018) .

In this paper, we present a simple approach for improving model robustness under the framework of adversarial training. We craft adversarial examples by performing a so-called adversarial interpolation operation (Section 3) between samples, and we expect the model should perform robustly against adversarial interpolation-induced perturbations within a constrained neighborhood.

The contribution of this work can be summarized as follows:

• we propose an adversarial interpolation approach for generating adversarial samples, which is simple, intuitive and effective; • we analyze the proposed approach and its connections with previous methods;

• we leverage adversarial interpolation for adversarial training and verify its effectiveness in improving model robustness compared with a number of strong baselines across several datasets.

2. PRELIMINARIES 2.1 ADVERSARIAL ATTACK AND DEFENSE:

Adversarial examples have been investigated in the seminal work of Biggio et al. (2013) ; Szegedy et al. (2014) and have attracted increasing attention recently (Biggio et al., 2013; Goodfellow et al., 2015; Tramèr et al., 2018; Madry et al., 2018; Athalye et al., 2018; Biggio & Roli, 2017; Wang et al., 2019) . Szegedy et al. (2014) investigated the vulnerabilities of CNNs to adversarial examples and proposed an L-BFGS-based approach for attack generation. Goodfellow et al. (2015) developed a fast gradient sign method (FGSM) for generating adversarial images. After that, many types of attacks have been developed in the past few years (Moosavi-Dezfooli et al., 2016; Carlini & Wagner, 2017; Su et al., 2017; Xiao et al., 2018; Brown et al., 2017; Brendel et al., 2018) . On the defense side, many researchers are actively working on improving model robustness against adversarial attacks Meng & Chen (2017) ; Xie et al. (2018) ; Metzen et al. (2017) ; Liu et al. (2018) ; Song et al. (2017) ; Liao et al. (2018) ; Samangouei et al. (2018) ; Prakash et al. (2018) ; Wang & Zhang (2019) .

Recently, Athalye et al. (2018) pointed out a phenomenon called gradient masking and showed that many existing defence methods did not actually improve model robustness thus giving a false sense of robustness because of it. According to Athalye et al. (2018) , adversarial training (Goodfellow et al., 2015; Madry et al., 2018) is one of the effective defense method against adversarial attacks. It improves model robustness by solving a minimax problem (Goodfellow et al., 2015; Madry et al., 2018) as min θ max x∈B(x, ) L(x, y; θ) , where the inner maximization is responsible for providing the most challenging example for the current model while the outer optimization loop improves the model performance on these adversarial examples (Madry et al., 2018) . The inner optimization can be solved approximately by a one-step approach such as FGSM (Goodfellow et al., 2015) , or a multi-step projected gradient descent (PGD) method (Madry et al., 2018) .

2.2. INCORPORATING REGULARIZATION IN MODEL TRAINING:

Regularization is important for deep learning models with a large number of parameters, as there are multiple possible solutions that perform equally well on clean data (Garipov et al., 2018; Draxler et al., 2018; Athiwaratkun et al., 2019) , the model by default has no particular preference over any of them if no preferences are given. In the case of model robustness, we need to have a way to convey our preference on robustness to the learning process. This can be viewed as a form of regularization. Conventional regularization techniques include weight decay or other forms of constraints on component function (Arjovsky et al., 2017) . Data augmentation is another way to regularization by manipulating the input. The idea is that an nuisance factor can be de-emphasized by generating training data that incorporates variations of the factor, leading to invariance w.r.t.this factor. Following this idea, conventional data augmentation transforms the training images using label preserving transformations (Krizhevsky et al., 2012) . Zhang et al. (2018) proposed a mixup approach which jointly transforms images and labels. (Ororbia et al., 2017) . Since adversarial training incorporates preference to robustness against perceptual insensitive perturbations, the models trained this way have been shown to have better interpretability (Engstrom et al., 2019; Stutz et al., 2019) . @D @f < l a t e x i t s h a 1 _ b a s e 6 4 = " 4 8 s 7 s X Z Q d j F 3 u G p e S J 9 c B j X w i r 0

D < l a t e x i t s h a 1 _ b a s e 6 4 = " v h c S Z K B 5 R E t y 1 f M y h k x M k / A 5 F w M = " > A A A B 8 n i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s x U Q Z d F X b i s Y B 8 w H U o m z b S h m W R I M k I Z + h l u X C j i 1 q 9 x 5 9 + Y a W e h r Q c C h 3 P u J e e e M O F M G 9 f 9 d k p r 6 x u b W + X t y s 7 u 3 v 5 B 9 f C o o 2 W q C G 0 T y a X q h V h T z g R t G 2 Y 4 7 S W K 4 j j k t B t O b n O / + 0 S V Z l I 8 m m l C g x i P B I s Y w c Z K f j / G Z k w w z + 5 m g 2 r N r b t z o F X i F a Q G B V q D 6 l d / K E k a U 2 E I x 1 r 7 n p u Y I M P K M M L p r N J P N U 0 w m e A R 9 S 0 V O K Y 6 y O a R Z + j M K k M U S W W f M G i u / t 7 I c K z 1 N A 7 t Z B 5 R L 3 u 5 + J / n p y a 6 D j I m k t R Q Q R Y f R S l H R q L 8 f j R k i h L D p

x W M S q E 1 C N g k t s G W 4 E d h K F N A o E P g T j 2 5 n / 8 I R K 8 1 j e m 0 m C f k S H k o e c U W O l p u q X K 2 7 V n Y O s E i 8 n F c j R 6 J e / e o O Y p R F K w w T V u u u 5 i f E z q g x n A q e l X q o x o W x M h 9 i 1 V N I I t Z / N D 5 2 S M 6 s M S B g r W 9 K Q u f p 7 I q O R 1 p M o s J 0 R N S O 9 7 M 3 E / 7 x u a s J r P + M y S Q 1 K t l g U p o K Y m M y + J g O u k B k x s Y Q y x e 2 t h I 2 o o s z Y b E o 2 B G / 5 5 V X S r l W 9 i 2 q t e V m p 3 + R x F O E E T u E c P L i C O t x B A 1 r A A O E Z X u H N e X R e n H f n Y 9 F a c P K Z Y / g D 5 / M H 3 m + M + g = = < / l a t e x i t > f < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 y O a 2 E O P w 3 I x i f n l U q v l T i C E / s g = " > A A A B 8 X i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s x U Q Z d F N y 4 r 2 A e 2 Q 8 m k d 9 r Q T G Z I M k I Z + h d u X C j i 1 r 9 x 5 9 + Y t r P Q 1 g O B w z n 3 k n N P k A i u j e t + O 4 W 1 9 Y 3 N r e J 2 a W d 3 b / + g f H j U 0 n G q G D Z Z L G L V C a h G w S U 2 D T c C O 4 l C G g U C 2 8 H 4 d u a 3 n 1 B p H s s H M 0 n Q j + h Q 8 p A z a q z 0 2 I u o G Q V h F k 7 7 5 Y p b d e c g q 8 T L S Q V y N P r l r 9 4 g Z m m E 0 j B B t e 5 6 b m L 8 j C r D m c B p q Z d q T C g b 0 y F 2 L Z U 0 Q u 1 n 8 8 R T c m a V A Q l j Z Z 8 0 Z K 7 + 3 s h o p P U k C u z k L K F e 9 m b i f 1 4 3 N e G 1 n 3 G Z p A Y l W 3 w U p o K Y m M z O J w O u k B k x s Y Q y x W 1 W w k Z U U W Z s S S V b g r d 8 8 i p p 1 a r e R b V 2 f 1 m p 3 + R 1 F O E E T u E c P L i C O t x B A 5 r A Q M I z v M K b o 5 0 X 5 9 3 5 W I w W n H z n G P 7 A + f w B 4 y 2 R D g = = < / l a t e x i t > f 0 < l a t e x i t s h a 1 _ b a s e 6 4 = " X c r H u M M j 0 X A v L b S G p J o d P C 2 7 K b E = " > A A A B 8 n i c b V D L S g M x F M 3 U V 6 2 v q k s 3 w S K 6 K j N V 0 G X R j c s K 9 g H T o W T S T B u a S Y b k j l C G f o Y b F 4 q 4 9 W v c + T d m 2 l l o 6 4 H A 4 Z x 7 y b k n T A Q 3 4 L r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 5 1 j E o 1 Z W 2 q h N K 9 k B g m u G R t 4 C B Y L 9 G M x K F g 3 X B y l / v d J 6 Y N V / I R p g k L Y j K S P O K U g J X 8 f k x g H E Z Z N D s f V G t u 3 Z 0 D r x K v I D V U o D W o f v W H i q Y x k 0 A F M c b 3 3 A S C j G j g V L B Z p Z

= " > A A A C D 3 i c b V D L S s N A F J 3 U V 6 2 v q E s 3 g 0 V x V Z I q 6 L K o C 5 c V 7 A O a U G 6 m k 3 b o 5 M H M R C g h f + D G X 3 H j Q h G 3 b t 3 5 N 0 7 a I N p 6 Y O B w z r 1 3 7 j 1 e z J l U l v V l l J a W V 1 b X y u u V j c 2 t 7 R 1 z d 6 8 t o 0 Q Q 2 i I R j 0 T X A 0 k 5 C 2 l L M c V p N x Y U A o / T j j e + y v 3 O P R W S R e G d m s T U D W A Y M p 8 R U F r q m 8 e O L 4 C k T g x C M e B O A G p E g K f X W f Y j Y j / r m 1 W r Z k 2 B F 4 l d k C o q 0 O y b n 8 4 g I k l A Q 0 U 4 S N m z r V i 5 a T 6 Q c J p V n E T S G M g Y h r S n a Q g B l W 4 6 v S f D R 1 o Z Y D 8 S + o U K T 9 X f H S k E U k 4

C T 1 f m + 8 p 5 L x f / 8 3 q J 8 i / c l I V x o m h I Z h / 5 C c c q w n k 4 e M A E J Y p P N A E i m N 4 V k x H o g J S O s K J D s O d P X i T t e s 0 + r d V v z 6 q N y y K O M j p A h + g E 2 e g c N d A N a q I W I u g B P a E X 9 G o 8 G s / G m / E + K y 0 Z R c 8 + + g P j 4 x u Q I J 2 d < / l a t e x i t > @D @f @f @x < l a t e x i t s h a 1 _ b a s e 6 4 = " r Z w Y H Z t h t D g J s 3 A q 8 i s 8 T S I j z X

Q = " > A A A C L X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j N V 0 G X R L l x W s A / o l J J J M 2 1 o 5 k G S E c s w P + T G X x H B R U X c + h t m 2 k F t 6 4 H A 4 d x z k 5 z j R p x J Z V k T Y 2 V 1 b X 1 j s 7 B V 3 N 7 Z 3 d s 3 D w 6 b M o w F o Q 0 S 8 l C 0 X S w p Z w F t K K Y 4 b U e C Y t / l t O W O b r J 5 6 4 E K y c L g X o 0 j 2 v X x I G A e I 1 h p q W f W H E 9 g k j g R F o p h 7 v h Y D Q n m S S 1 N f 0 T k p W j e p p V f / p j 2 z J J V t q Z A y 8 T O S Q l y 1 H v m q 9 M P S e z T Q B G O p e z Y V q S 6 S X Y h 4 T Q t O r G k E S Y j P K A d T Q P s U 9 l N p m l T d K q V P v J C o U + g 0 F T 9 u 5 F g X 8 q x 7 2 p n l k Y u z j L x v 1 k n V t 5 V N 2 F B F C s a k N l D X s y R C l F W H e o z Q Y n i Y 0 0 w E U z / F Z E h 1 r 0 o X X B R l 2 A v

R l 4 m z U r Z P i 9 X 7 i 5 K 1 e u 8 j g I c w w m c g Q 2 X U I V b q E M D C D z B C 0 z g 3 X g 2 3 o w P 4 3 N m X T H y n S O Y g / H 1 D S z A q o E = < / l a t e x i t > @D @f < l a t e x i t s h a 1 _ b a s e 6 4 = " 4 8 s 7 s X Z Q d j F 3 u G p e S J 9 c B j X w i r 0

= " > A A A C D 3 i c b V D L S s N A F J 3 U V 6 2 v q E s 3 g 0 V x V Z I q 6 L K o C 5 c V 7 A O a U G 6 m k 3 b o 5 M H M R C g h f + D G X 3 H j Q h G 3 b t 3 5 N 0 7 a I N p 6 Y O B w z r 1 3 7 j 1 e z J l U l v V l l J a W V 1 b X y u u V j c 2 t 7 R 1 z d 6 8 t o 0 Q Q 2 i I R j 0 T X A 0 k 5 C 2 l L M c V p N x Y U A o / T j j e + y v 3 O P R W S R e G d m s T U D W A Y M p 8 R U F r q m 8 e O L 4 C k T g x C M e B O A G p E g K f X W f Y j Y j / r m 1 W r Z k 2 B F 4 l d k C o q 0 O y b n 8 4 g I k l A Q 0 U 4 S N m z r V i 5 a T 6 Q c J p V n E T S G M g Y h r S n a Q g B l W 4 6 v S f D R 1 o Z Y D 8 S + o U K T 9 X f H S k E U k 4

C T 1 f m + 8 p 5 L x f / 8 3 q J 8 i / c l I V x o m h I Z h / 5 C c c q w n k 4 e M A E J Y p P N A E i m N 4 V k x H o g J S O s K J D s O d P X i T t e s 0 + r d V v z 6 q N y y K O M j p A h + g E 2 e g c N d A N a q I W I u g B P a E X 9 G o 8 G s / G m / E + K y 0 Z R c 8 + + g P j 4 x u Q I J 2 d < / l a t e x i t > f < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 y O a 2 E O P w 3

I x i f n l U q v l T i C E / s g = " > A A A B 8 X i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s x U Q Z d F N y 4 r 2 A e 2 Q 8 m k d 9 r Q T G Z I M k I Z + h d u X C j i 1 r 9 x 5 9 + Y t r P Q 1 g O B w z n 3 k n N P k A i u j e t + O 4 W 1 9 Y 3 N r e J 2 a W d 3 b / + g f H j U 0 n G q G D Z Z L G L V C a h G w S U 2 D T c C O 4 l C G g U C 2 8 H 4 d u a 3 n 1 B p H s s H M 0 n Q j + h Q 8 p A z a q z 0 2 I u o G Q V h F k 7 7 5 Y p b d e c g q 8 T L S Q V y N P r l r 9 4 g Z m m E 0 j B B t e 5 6 b m L 8 j C r D m c B p q Z d q T C g b 0 y F 2 L Z U 0 Q u 1 n 8 8 R T c m a V A Q l j Z Z 8 0 Z K 7 + 3 s h o p P U k C u z k L K F e 9 m b i f 1 4 3 N e G 1 n 3 G Z p A Y l W 3 w U p o K Y m M z O J w O u k B k x s Y Q y x W 1 W w k Z U U W Z s S S V b g r d 8 8 i p p 1 a r e R b V 2 f 1 m p 3 + R 1 F O E E T u E c P L i C O t x B A 5 r A Q M I z v M K b o 5 0 X 5 9 3 5 W I w W n H z n G P 7 A + f w B 4 y 2 R D g = = < / l a t e x i t > f 0 < l a t e x i t s h a 1 _ b a s e 6 4 = " X c r H u M M j 0 X A v L b S G p J o d P C 2 7 K b E = " > A A A B 8 n i c b V D L S g M x F M 3 U V 6 2 v q k s 3 w S K 6 K j N V 0 G X R j c s K 9 g H T o W T S T B u a S Y b k j l C G f o Y b F 4 q 4 9 W v c + T d m 2 l l o 6 4 H A 4 Z x 7 y b k n T A Q 3 4 L r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 5 1 j E o 1 Z W 2 q h N K 9 k B g m u G R t 4 C B Y L 9 G M x K F g 3 X B y l / v d J 6 Y N V / I R p g k L Y j K S P O K U g J X 8 f k x g H E Z Z N D s f V G t u 3 Z 0 D r x K v I D V U o D W o f v W H i q Y x k 0 A F M c b 3 3 A S C j G j g V L B Z p Z 8 a l h A 6 I S P m W y p J z E y Q z S P P 8 J l V h j h S 2 j 4 J e K 7 + 3 s h I b M w 0 D u 1 k H t E s e 7 n 4 n + e n E N 0 E G Z d J C k z S x U d R K j A o n N + P h 1 w z C m J q C a G a 2 6 y Y j o k m F G x L F V u C t 3 z y K u k 0 6 t 5 l v f F w V W v e F n W U 0 Q k 6 R R f I Q 9 e o i e 5 R C 7 U R R Q o 9 o 1 f 0 5 o D z 4 r w 7 H 4 v R k l P s H K M / c D 5 / A E d 6 k T 8 = < / l a t e x i t > f ✓ (x) < l a t e x i t s h a 1 _ b a s e 6 4 = " S Q 7 y 1 c l b f t J H b G U D U d b 4 V Y k n m k U = " > A A A B 9 H i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 B I t Q L y W p g h 6 L X j x W s B / Q h r L Z b t q l m 0 3 c n R R L 6 O / w 4 k E R r / 4 Y b / 4 b t 2 0 O 2 v p g 4 P H e D D P z / F h w j Y 7 z b e X W 1 j c 2 t / L b h Z 3 d v f 2 D 4 u F R U 0 e J o q x B I x G p t k 8 0 E 1 y y B n I U r B 0 r R k J f s J Y / u p 3 5 r T F T m k f y A S c x 8 0 I y k D z g l K C R v K C X d n H I k E z L T + e 9 Y s m p O H P Y q 8 T N S A k y 1 H v F r 2 4 / o k n I J F J B t O 6 4 T o x e S h R y K t i 0 0 E 0 0 i w k d k Q H r G C p J y L S X z o + e 2 m d G 6 d t B p E x J t O f q 7 4 m U h F p P Q t 9 0 h g S H e t m b i f 9 5 n Q S D a y / l M k 6 Q S b p Y F C T C x s i e J W D 3 u W I U x c Q Q Q h U 3 t 9 p 0 S B S h a H I q m B D c 5 Z d X S b N a c S 8 q 1 f v L U u 0 m i y M P J 3 A K Z X D h C m p w B 3 V o A I V H e I Z X e L P G 1 o v 1 b n 0 s W n N W N n M M f 2 B 9 / g C V B p H 4 < / l a t e x i t > f ✓ (x 0 )

< l a t e x i t s h a 1 _ b a s e 6 4 = " q 9 E k q 4 d L l j l x E 9 2 T 5 q T

C P c J O E t E = " > A A A B 9 X i c b V D L T g J B E O z 1 i f h C P X q Z S I x 4 I b t o o k e i F 4 + Y y C O B l c w O s z B h 9 p G Z X p V s + A 8 v H j T G q / / i z b 9 x g D 0 o W E k n l a r u d H d 5 s R Q a b f v b W l p e W V 1 b z 2 3 k N 7 e 2 d 3 Y L e / s N H S W K 8 T q L Z K R a H t V c i p D X U a D k r V h x G n i S N 7 3 h 9 c R v P n C l R R T e 4 S j m b k D 7 o f A F o 2 i k e 7 + b d n D A k Y 5 L T y e n 3 U L R L t t T k E X i Z K Q I G W r d w l e n F 7 E k 4 C E y S b V u O 3 a M b k o V C i b 5 O N 9 J N I 8 p G 9 I + b x s a 0 o B r N 5 1 e P S b H R u k R P 1 K m Q i R T 9 f d E S g O t R 4 F n O g O K A z 3 v T c T / v H a C / q W b i j B O k I d s t s h P J M G I T C I g P a E 4 Q z k y h D I l z K 2 E D a i i D E 1 Q e R O C M / / y I m l U y s 5 Z u X J 7 X q x e Z X H k 4 B C O o A Q O X E A V b q A G d W C g 4 B l e 4 c 1 6 t F 6 s d + t j 1 r p k Z T M H 8 A f W 5 w / 6 F J I p < / l a t e x i t > min x D(f , f 0 ) < l a t e x i t s h a 1 _ b a s e 6 4 = " / W z D U c Z d C 2 I Z m D / q 7 7 W / Y D r r I F E = " > A A A C F n i c b Z D L S g M x F I Y z 9 V b r b d S l m 2 A R K 2 i Z q Y I u i 7 p w W c F e o C 1 D J s 3 U 0 C Q z J B m x D P M U b n w V N y 4 U c S v u f B v T d k B t / S H w 8 Z 9 z y D m / H z G q t O N 8 W b m 5 + Y X F p f x y Y W V 1 b X 3 D 3 t x q q D C W m N R x y E L Z 8 p E i j A p S 1 1 Q z 0 o o k Q d x n p O k P L k b 1 5 h 2 R i o b i R g 8 j 0 u W o L 2 h A M d L G 8 u y j D q f C S + 5 T 2 O F I 3 2 L E k s u 0 N G Y / S I L 0 E P 7 w / o F n F 5 2 y M x a c B T e D I s h U 8 + z P T i / E M S d C Y 4 a U a r t O p L s J k p p i R t J C J 1 Y k Q n i A + q R t U C B O V D c Z n 5 X C P e P 0 Y B B K 8 4 S G Y / f 3 R I K 4 U k P u m 8 7 R j m q 6 N j L / q 7 V j H Z x 1 E y q i W B O B J x 8 F M Y M 6 h K O M Y I 9 K g j U b G k B Y U r M r x L d I I q

x N k g U T g j t 9 8 i w 0 K m X 3 u F y 5 P i l W z 7 M 4 8 m A H 7 I I S c M E p q I I r U A N 1 g M E D e A I v 4 N V 6 t J 6 t N + t 9 0 p q z s p l t 8 E f W x z e A O p + O < / l a t e x i t >

x < l a t e x i t s h a 1 _ b a s e 6 4 = " h L + F a L t O T 9 l u w f L W 3 U t 0 8 x l 3 P c w 

= " > A A A B 6 H i c b V D L T g J B E O z F F + I L 9 e h l I j H x R H b R R I 9 E L x 4 h k U c C G z I 7 9 M L I 7 O x m Z t Z I C F / g x Y P G e P W T v P k 3 D r A H B S v p p F L V n e 6 u I B F c G 9 f 9 d n J r 6 x u b W / n t w s 7 u 3 v 5 B 8 f C o q e N U M W y w W M S q H V C N g k t s G G 4 E t h O F N A o E t o L R 7 c x v P a L S P J b 3 Z p y g H 9 G B 5 C F n 1 F i p / t Q r l t y y O w d Z J V 5 G S p C h 1 i t + d f s x S y O U h g m q d c d z E + N P q D K c C Z w W u q n G h L I R H W D H U k k j 1 P 5 k f u i U n F m l T 8 J Y 2 Z K G z N X f E x M a a T 2 O A t s Z U T P U y 9 5 M / M / r p C a 8 9 i d c J q l B y R a L w l Q Q E 5 P Z 1 6 T P F T I j x p Z Q p r i 9 l b A h V Z Q Z m 0 3 B h u A t v 7 x K m p W y d 1 G u 1 C 9 L 1 Z s s j j y c w C m c g w d X U I U 7 q E E D G C A 8 w y u 8 O Q / O i / P u f C x a c 0 4 2 c w x / 4 H z + A O e H j Q A = < / l a t e x i t > x 0 < l a t e x i t s h a 1 _ b a s e 6 4 = " f E 3 A S V S e V b J p P 9 + E Y m b Y O a S F Z R g = " > A A A B 6 X i c b V D L T g J B E O z F F + I L 9 e h l I j F 6 I r t o o k e i F 4 9 o 5 J E A I b N D L 0 y Y n d 3 M z B r J h j / w 4 k F j v P p H 3 v w b B 9 i D g p V 0 U q n q T n e X H w u u j e t + O 7 m V 1 b X 1 j f x m Y W t 7 Z 3 e v u H / Q 0 F G i G N Z Z J C L V 8 q l G w S X W D T c C W 7 F C G v o C m / 7 o Z u o 3 H 1 F p H s k H M 4 6 x G 9 K B 5 A F n 1 F j p / u m 0 V y y 5 Z X c G s k y 8 j J Q g Q 6 1 X / O r 0 I 5 a E K A 0 T V O u 2 5 8 a m m 1 J l O B M 4 K X Q S j T F l I z r A t q W S

3. ADVERSARIAL INTERPOLATION TRAINING:

We present the adversarial interpolation training method in this section. The intuition is that the semantic meaning of the image is unchanged under mild perturbations, thus the network should be robust against them as well. On the one hand, it is well-known that adding random perturbations to the input is ineffective for improving model robustness. On the other hand, adversarial training using classification loss generated perturbations suffers from label leaking (Kurakin et al., 2017) . These motivate the development of the proposed method as detailed in the sequel.

3.1. ADVERSARIAL INTERPOLATION:

We first review several existing types of perturbations in the sequel and then introduce our proposed approach. We will use x to denote the perturbed version of x. Definition Random Perturbation. Given a sample x, the randomly perturbed sample x is generated by sampling from the neighborhood ball B(x, ) centered at x with radius as x ∼ B(x, ).

It is clear that the random perturbations are data-agnostic and isotropic in each direction. It has been shown that this types of perturbation is equivalent to Tikhonov regularization (Bishop, 1995) and is less effective in improving model robustness due to its data and model agnostic nature. Note that we have omitted the pixel value range constraint to avoid notation clutter. Instead of randomly sampling, (discriminative) adversarial perturbations are generated with the guidance of a classification loss. Definition Discriminative Perturbation. Given a sample x and its label y, we define the discriminative perturbation as: x * arg min x∈B(x, ) -CE(x, y), where CE(•) denotes the network taking x as input and computes the cross-entropy (CE) loss between the predicted class probabilities and the input label y. Discriminative perturbation is datadependent and model/task-aware (i.e. white-box), representing a form of strong attacks for a given model. Naturally, it has been used in adversarial training for online generation of training data that is adversarial to the current model. It is known that the discriminative perturbations are highly correlated with the decision boundary. The high correlation between the perturbation and class label poses the training to the danger of label leaking (Kurakin et al., 2017) . Definition Adversarial Interpolation. Given two images x, x , we define the adversarial interpolation of x (towards x ) as the solution to the following optimization problem x * adv interp(x, x ) = arg min

EQUATION

where x is another instance that is different from x. In practice, x can be set as another instance from the current batch, thus introducing a negligible computational overhead. D is a distance measure in a feature space. As minimization of D(x, x ) w.r.t.x has the effect of "interpolating" x towards x , we therefore term the proposed approach for generating adversarial attacks as adversarial interpolation. The adversarial interpolation procedure is illustrated in Figure 1 . It leverages the currently learned model for feature extraction and distance computation. It then generates the adverse examples by back-propagating the distance induced gradient (feature space difference) through the network back to pixels. It is more effective than random perturbation as it leverages structures between data points. And it is potentially more flexible than discriminative perturbations as adversarial interpolation is used for generating perturbations which is untied from the classification loss. For a particular sample, when viewed collectively over time, adversarial interpolation essentially constructs a randomly connected graph (Hein et al., 2007) and performs interpolation along different edge each time randomly. More on understanding of the proposed approach is in Section 3.3. 

for k = 1 to K do for random batch {x i , y i } n i=1 ∼dataset do xi ∼ B(x i , ), ỹi = y i , x i = x n-i+1 , ȳ i = 1 C-1 (1-y n-i+1 ) P(x i ) = min zi∈S z i -xi , S = B(x i , ) ∩ [0, 255] #pixel adversarial interpolation: for l = 1 to L do • xi ← P xi -• sign ∇ xi D(x i , x i )

generating adversarial images

• ỹi ← ỹi -y • (ỹ i -ȳ i )

generating adversarial labels end for adversarial training:

θ ← θ-α • 1 n n i=1 ∇ θ L(x i , ỹi ; θ)

updating model parameters end for end for Output: model parameter θ.

We utilize adversarial interpolation for robust model training. Given a set of data, S = {(x i , y i )}, we first generate perturbed examples S = {(x i , ỹi )} using adversarial interpolation, and then use the generated examples for model training (Goodfellow et al., 2015; Madry et al., 2018) . Formally, we perform robust model training by solving the following problem:

EQUATION

where the loss is the standard cross-entropy-based loss as L θ (x, y) CE(g • f (x), ỹ), with f the feature extraction network shared with the adversarial interpolation process and g denotes some additional layers following f (e.g., FC layer followed by softmax layer). Here we overload θ to represent the union of parameters for g and f . The training samples (x, ỹ) are generated via adversarial interpolation as follows:

x * = arg min

EQUATION

EQUATION

which perturbs image x and y towards different (or reverse) directions within a permissible neighborhood. D and D y are proper distance measures for image and label respectively. By perturbing the image and label in directions reverse to each other, the new image-label pair (x, ỹ) will potentially induce a higher classification loss when evaluated together and represent a strong adversarial sample to the model (Wang & Zhang, 2019) . Here y denotes the maximum of the allowed perturbation for the label and it will be properly set to ensure the entry corresponding to the ground-truth label will remain to be the largest one after perturbation. All the samples in B(y, y ) are assumed to have non-negative elements and the entries of each sample sum to one. We reformulate Eqn.(4) using the following approximate instantiation

ỹ * = arg min ỹ∈B(y, y ) ỹ -ȳ 2 2 (5)

where Euclidean distance is used and ȳ = 1-y C-1 with y the label of x , with C the total number of classes. Note that labels are in vector form thus y and y are one-hot vectors. More interpretations on Eqn.(5) are provided in Section 3.3. Based on Eqn.(3) and Eqn.(5), we can derive the updates as follows. For two points (x, y) and (x , y ), we generate the perturbed training sample as follows

EQUATION

Eqn.( 6) can be interpreted as interpolating point x towards x for the data, while interpolating the label y away from y . The training procedure is summarized in Algorithm 1 and the implementation is in Section A.1. In practice, D(x, x ) is implemented using deepnet-based feature extractor combined with a standard distance measure such as Euclidean (or other metrics such as Cosine distance),

EQUATION

)

We have used D as a shorthand for D θ in the above but without explicitly specifying the dependency on θ. Minimizing this term makes the feature thus the perturbed input image incorporate some features from the other sample, while retaining visually similar to the original image, thus acting as an effective form of adversarial perturbation. More importantly, we perturb the image towards a target instance, while perturb the label against this instance (to all other classes but this target one). When the target instance is of different class with the current instance, the label interpolation operation essentially reinforces that the perturbed image is not from the target class (c.f. Table 7 ), which helps to further break the correlation between δ and y , beyond decreasing the correlation between δ and y. When the target instance is of the same class with the current instance, the label interpolation operation is equivalent to conventional label smoothing (proof is deferred to Appendix A.3).

3.3. INTERPRETING AND UNDERSTANDING OF THE PROPOSED APPROACH:

From the perspective of robust and non-robust features, the key to robust model training is to effectively prevent the classifier from learning useful but non-robust features (Ilyas et al., 2019) . In order to achieve this, we need a way to convey this preference to the learning algorithm. Currently, one of the most straightforward way to communicate this is by manipulating the data. In fact, by manipulate the data in different ways, we can convey different preferences thus receiving different results from the machine learning algorithm (c.f. Table 7 ). As our major objective is to obtain a robust model, this can be achieved by breaking the correlation between the adversarial perturbation δ and label. Conventionally, it is achieved by constructing δ ∼ ∆, where ∆ is the set of non-robust features of all the other classes. In this way, by training the model using the original label y and perturbation δ from all other classes, the correlation between δ and y is reduced. Conversely, if we construct δ as from one class (y ) and use y as the label for training, then we essentially imply that δ represents useful feature for the prediction of y . Therefore the model will learn to use these nonrobust features for prediction, which has been exploited by Ilyas et al. (2019) recently. The proposed approach can be interpreted from two complementary perspectives. Firstly, we also de-emphasize δ as a family by constructing it from different classes while retaining the original target specified by y, thus reducing the correlation between δ and y. Secondly, we further de-emphasize each individual δ by reinforcing that δ is not useful features for y , as represented by (1-y ), ("not y "), thus further breaking the correlations between δ and y . While we present one concrete instantiation in this work, the reasoning is general and can be implemented with other configurations as well.

3.4. CONNECTIONS WITH PREVIOUS WORKS:

A feature-scattering approach for generating adversarial perturbations for training has been proposed recently (Zhang & Wang, 2019) , advocating the usage of unsupervised adversarial generation to avoid label leaking. Intuitively, it generates perturbations by moving away from a set of similar samples, discovered through an additional matching process. Differently, adversarial-interpolation generates adversarial examples by moving towards a distinct sample, which does not require the expensive matching step and is complementary to feature-scattering (Zhang & Wang, 2019) .

There is also an interesting connection with Mixup (Zhang et al., 2018) . Specifically, in the case of f θ (x) = x with Euclidean distance, adversarial interpolation for images (Eqn.( 6)) is equivalent to the operation applied to image in Mixup (Zhang et al., 2018) (proof deferred to Section A.4). Therefore our approach can be interpreted as feature space induced adverse image generation instead of linearly mixing the raw pixels directly. Manifold-mixup (Verma et al., 2019) has been proposed recently by generalizing the mixup operation from input to latent features. While it also has an interpolation step in feature space, its motivation is different from ours and it is used differently. Manifold-mixup performs feature interpolation to generate the mixed feature, which is then passed to the subsequent layers till the cross-entropy loss. The cross-entropy loss-induced gradients backpropagate through the interpolation for model training. Manifold-mixup plays the role of a regularizer in order to obtain neural networks with smoother decision boundaries at multiple levels of representation, which helps but does not significantly improve robustness against multi-step PGD attacks (Verma et al., 2019) simiar to the original Mixup (Zhang et al., 2018) . The authors of Verma et al. ( 2019) have further integrated Manifold-mixup with adversarial training to mitigate this issue in another recent work (Lamb et al., 2019) . Different from these works, the proposed method uses features interpolation for inducing input perturbation, which is done by backpropogating the gradient from feature space distance Eqn.(8) to the input (Figure 1 ). The perturbed inputs are then used as new inputs in place of the original ones for model training. Furthermore, empirically different from Mixup (Zhang et al., 2018) and Manifold-mixup (Verma et al., 2019) , the proposed adversarial image interpolation scheme clearly helps to improve model robustness as shown in Table 5 . Another crucial difference with Mixup (Zhang et al., 2018) and Manifold-mixup (Verma et al., 2019) is that the label in the proposed approach is interpolated in a direction "reverse" to that of the image interpolation to further reinforce robustness. This difference can be explained by the fact that mixup focuses more on the on-manifold generalization while adversarial robustness further extends the focus to off-manifold robustness (Stutz et al., 2019) . Anonymous (2020) presents an interesting interpretation of Mixup as belonging to a class highly analogous to adversarial training. Without evaluation of adversarial robustness provided in the paper, Anonymous ( 2020) is mainly on interpreting and understanding of Mixup, which is different from our goal of improving model robustness. Christopher Beckham (2019) generalizes Manifold-mixup (Verma et al., 2019) for image generation, where feature interpolation is used to train a model that is able to combine the attributes of multiple inputs in a resynthesised image. The purpose of feature interpolation in (Christopher Beckham, 2019) is to introduce human sensible visual attributes in the generated image, and the image is generated by forwarding the mixed feature through a decoder in the auto-encoding framework. Differently, the purpose of feature interpolation in the proposed method is to introduce human imperceptible, non-robust features in the generated image, which is generated by back-propagating the feature difference through the feature extractor (c.f. Figure 1 ). A number of recent works that exploit interpolation from a different angle have also been presented by Wang et al. (2018b; a) ; Wang & Osher (2019) , with the interpolation operates at the output layer and no adversarial label interpolation.

The recent Bilateral method (Wang & Zhang, 2019 ) also jointly perturbs image and label, but uses the classification loss directly. Our approach is very different in that the classification loss is not used in generating perturbed image or label. Instead, it uses an adversarial interpolation scheme which is not directly connected with the classification loss thus mitigating the problem of label leaking or data manifold tilting and shrinking (Tanay & Griffin, 2016; Zhang & Wang, 2019) .

We have used an unsupervised approach to generate adversarial perturbations. As another form of utilizing unsupervised learning, Uesato et al. (2019) improves adversarial generalization by using additional unlabeled data, which is orthogonal to our work and could potentially be used together.

4. EXPERIMENTS:

The PyTorch implementation of the proposed Adv-Interp approach is provided in Section A.1. We conduct extensive experiments to verify the effectiveness of the proposed approach. We compare the performance of the proposed method with a number of baseline methods:

• Natural: the model trained with standard approach using natural images (Krizhevsky, 2009) ;

• Madry: the PGD-based approach from Madry et al. (2018) , which is one of the most effective and representative defense method; • Bilateral: a method that performs adversarial training with both image and label adversarial perturbations generated using the classification loss (Wang & Zhang, 2019 ); • Feature-Scatter: a recent method which generates adversarial perturbations using an unsupervised feature-scattering scheme for attack generation (Zhang & Wang, 2019) .

Following (Madry et al., 2018) , the full network (image to logits) is implemented as the Wide ResNet (WRN-28-10) (Zagoruyko & Komodakis, 2016) and f θ (•) (feature extractor) is implemented by excluding the last layer for the logits. D is implemented as the Cosine distance, which is free-from additional tuning parameter. For training, the initial learning rate α is 0.1 for CIFAR and 0.01 for SVHN. We set the number of epochs for Natural and Madry methods as 100 with transition epochs set as {60, 90} following Madry et al. (2018) ; Wang & Zhang (2019) . The training scheduling of 200 epochs with the same transition epochs is used similar to Wang & Zhang (2019) . Standard data augmentation is used during training, i.e., random crops with 4 pixels of padding and random horizontal flips (Krizhevsky, 2009) . We use a perturbation budget of =8 (Madry et al., 2018) , y = 0.5, and L=1 in training. The trained models are evaluated by measuring the accuracy performance against different adversarial attacks. For white-box attacks, we use:

• FGSM: the Fast Gradient Sign Method (Goodfellow et al., 2015) , which is a one-step gradientbased approach for generating adversarial attacks; • PGD: Projected Gradient Descent Method (Madry et al., 2018) , which is a multi-step gradientbased approach for generating adversarial attacks (PGDT denotes PGD attack with T iterations); (Madry et al., 2018) , Bilateral (Wang & Zhang, 2019) , Feature-Scatter (Zhang & Wang, 2019) and the proposed Adv-Interp method under different attacks. Table 2 : Performance against stronger white-box attacks with increasing attack budgets. The models are trained with the attack budget =8, and are evaluated against attacks with larger budgets.

• CW: Carlini-Wagner loss based attack (Carlini & Wagner, 2017) , which is also a multi-step gradient-based approach for generating adversarial attacks. It is implemented using the CWloss (Carlini & Wagner, 2017) within the PGD framework following Madry et al. (2018) . For black-box attacks, we use both gradient-based and gradient-free ones as detailed in Section 4.4.

4.1. EVALUATION AGAINST WHITE-BOX ATTACKS ON CIFAR10:

We conduct experiments on CIFAR10 (Krizhevsky, 2009) in this section. CIFAR10 is a dataset with 10 classes, 5K training images per class and 10K test images. It has been widely use in adversarial training literature (Madry et al., 2018; Wang & Zhang, 2019) . Evaluation under Standard Setting. We report the accuracy on the original test images (Clean) as well as under standard adversarial settings (Madry et al., 2018; Carlini & Wagner, 2017; Carlini et al., 2019) and the results are summarized in Table 1 . From Table 1 , we have several observations:

• it is observed that model trained with natural images (Natural) fails drastically under different white-box attacks. Madry method improves the model robustness significantly over the Natural model and achieves about 45.0% accuracy under the standard PGD20 attack. The Bilateral approach outperforms Madry and achieves a performance of 57.5% under PGD20 attack. The recent Feature-Scatter method further boost the performance to 70.5% under PGD20 attack. The proposed approach achieves 73.5% accuracy under the standard 20 steps PGD attack, outperforming all the compared methods; • it is also noted that while the Feature-Scatter method achieves competitive performance under PGD20 attack, its performance has a noticeable drop when the strength of the attack is increased (e.g. PGD20→PGD100). The proposed Adv-Interp approach, on the other hand, maintains its performance when the strength of the attack is increased (PGD20→PGD100) and outperforms Feature-Scatter with a even large gap under PGD100 attack; • it is interesting to observe that under the CW metric, the performances of Madry and Bilateral are similar to their respective performances under PGD attack. For Feature-Scatter, there is a large drop for the performance under CW attack compared with that under PGD attack. The proposed Adv-Interp approach, on the other hand, has a smaller performance gap between PGD and CW metric and outperforms Feature-Scatter and all other methods with a large margin; • in terms of the worst case performance among all evaluation criteria, the proposed Adv-Interp method improves over Madry by 24.2%, Bilateral by 14.9% and Feature-Scatter by 8.1%. Table 5 : Ablation studies on different perturbation schemes for (a) image and (b) label.

Models:

Evaluation against Stronger White-box Attacks. All the robust models are trained under the standard setting with a maximum attack budget = 8. While it is standard to restrict the maximum budget at test-time to be = 8 as well, it is also illuminating to conduct evaluations beyond it for a comprehensive understanding of the model robustness. We therefore evaluate model robustness against PGD20 and CW20 attacks with increasing attack budgets and the results are summarized in Table 2 . Unsurprisingly, the performance of Natural model remains at 0. Madry improves the model robustness significantly over Natural across a wide range of attack budgets beyond 8, demonstrating the effectiveness of the standard adversarial training on improving model robustness.

The proposed Adv-Interp approach further improves the performance over Madry by a large margin for all the evaluated attack budgets. It is interesting to note that Adv-Interp maintains a high level of robustness even beyond the regime it is trained on. Note that if the attack budget is further increased to = 255, the performance of all methods will be eventually approaching 0. We further conduct another set of experiments towards evaluation against stronger attacks, by increasing the number of attack iterations with a fixed attack budget of 8. The results are summarized in Table 3 . As can be observed from Table 3 , Madry method performs stable when the number of attack iterations is increased. The proposed Adv-Interp method can also maintain a fairly stable performance across number of attack iterations is increased. It is interesting to note that the performance of Adv-Interp under PGD1000 still outperforms the performance of all other approaches under PGD20 by a large margin (c.f. Table 1 ). We will use a PGD/CW attackers with =8 and attack step 20 and 100 in the sequel as the standard setting for evaluating model robustness.

4.2. WHITE-BOX ATTACK EVALUATION:

ON MORE DATASETS CIFAR100. We also experiment with CIFAR100 dataset, with 100 classes, 50K training and 10K test images (Krizhevsky, 2009) . This data set is more challenging as the number of training images per class is much smaller (Zhang & Wang, 2019) . We set training epoch as 300 and y =0.9. As shown by the results in Table 4 (a), the proposed approach outperforms Madry and Bilateral by a large margin under both PGD and CW attacks. Furthermore, it achieves performance on-par with Feature-Scatter under the strongest attack (CW100) on this challenging dataset. SVHN. We further report results on the SVHN dataset (Netzer et al., 2011) (Tramèr et al., 2018; Uesato et al., 2018) .

Gradient-based Black-box Attack. We first use gradient-based approach for generating the attacks, but using models that are different from the one to be evaluated. Specifically, we use the Natural model and Adv-Interp', which is another model trained with the proposed method in a different training session. The results are summarized in Table 6 (a). It is observed that the model trained with the proposed approach is robust against different types of gradient-based black-box attacks.

Gradient-free Black-box Attack. We further conducted evaluation using two gradient-free blackbox attacks. SPSA is a technique developed for high-dimensional optimization problem (Spall, 1992) and is used in Uesato et al. (2018) for gradient-free attack-based evaluations. Local-Search is a local search-based black-box attack method (Narodytska & Kasiviswanathan, 2017) . The evaluation results of the proposed model against gradient-free attacks are summarized in Table 6 (b). All these results together with the white-box attack results in Table 1 verify that the improvements are indeed due to the improved model robustness, not because of gradient-masking caused by a degenerate solution (Tramèr et al., 2018) .

5. CONCLUSION AND FUTURE WORK:

An adversarial interpolation training method is presented in this paper. This approach is intuitive to understand and simple to implement, yet achieving state-of-the-art performance. Extensive experiments on several standard datasets have been done to evaluate and understand the performance of the proposed method. Empirical results demonstrate that the proposed approach compares favorably with existing methods in the literature. There are some limitations of the proposed approach due to the usage of an unsupervised surrogate function for generating image perturbations. While this offers the potential of mitigating some issues of the discriminative perturnation, the perturbations generated this way are not necessarily the most effective ones due to potential mis-alignment with the downstream task. Also, as there is no explicit force for further encouraging the clustering of features and the increasing of the separation margins between them, the feature points are typically more scattered (c.f. Figure 3 ), offering more opportunity to feature space attacks. Nevertheless, the fact that such a simple approach can achieve encouraging performance suggests that this is a valuable direction to explore further and the observation that it behaves differently with existing methods against different attacks suggests the potential to combine their complementary strength for further improvements (Pang et al., 2019) . The proposed method is generally applicable can potentially be extended to applications beyond image classification. Furthermore, as the proposed adversarial interpolation-based attack generation does not require label information, it would be interesting to extend this approach to other types of learning tasks such as unsupervised learning and reinforcement learning. We leave the exploration of these interesting directions as future work. 

EQUATION

which leads to the following update equation for label

EQUATION

where the label target ȳ is constructed as ȳ = 1-y C-1 with C the total number of classes. Here we show that this can be derived from a complementary perspective as follows.

Given a training sample (x, y), the original way for model training is as follows:

EQUATION

where

z = h(x) = g • f (x).

By adding a perturbation δ which fools the predictor towards label y , we have x = x + δ, which semantically still belongs to the original class. Therefore, we can use the following formulation to dis-emphasize δ min

EQUATION

Secondly, while the perturbed sample x fools the predictor towards label y , x is not a valid sample of class y , which is represented as

EQUATION

By combining Eqn.(12) with Eqn.( 14), we have

EQUATION

which leads to the same update equation for label as in Eqn.(10).

A.3 CONNECTION OF ADVERSARIAL LABEL INTERPOLATION WITH LABEL SMOOTHING:

Here we provide the proof that when the target instance is of the same class with the current instance, i.e., y = y , the label interpolation operation is equivalent to conventional label smoothing.

Recall that for adversarial interpolation of label, we have ỹ = (1y )y + y ȳ ,

where ȳ = 1-y C-1 . C is the number of classes. In the case that y = y, we have

EQUATION

which is equivalent to label-smoothing with the strength of C C-1 y (Szegedy et al., 2016) .

A.4 CONNECTION OF ADVERSARIAL IMAGE INTERPOLATION WITH MIXUP:

There is a connection between the proposed method and mixup (Zhang et al., 2018) as mentioned in the main paper. Specifically, in the case of f θ (x) = x with Euclidean distance, the adversarial interpolation for images (Eqn.( 6)) is equivalent to mixup (Zhang et al., 2018) . Concretely, for

D(x, x ) = x -x 2 2 (17)

we have its gradient as

EQUATION

Plugging it into adversarial interpolation update Eqn.( 6), we have

EQUATION

This shows that for image manipulation, when the proposed approach is performed at pixel level we recover mixup (Zhang et al., 2018) . In this case, the new image x is obtained as a linear combination of the data points x and x in the original data space, which are pixels for images. This formally shows that the image manipulation in mixup can be interpreted as one gradient step for minimizing the pixel value distance between two samples. As for labels, mixup (Zhang et al., 2018) shares a similar update form as Eqn.( 7) but with ȳ = y , i.e. ỹ = (1y )y + y y .

(20)

A.5 CATEGORIZATION OF DIFFERENT TRAINING METHODS:

Here we provide a categorization of different training methods as in Table 7 . By manipulating image and label in different ways, we can convey different messages to the learning algorithm and obtain the corresponding results. The proposed approach can be intuitively understood as leveraging two complementary aspects for improving model robustness, i.e., by reducing the the correlations between δ and y as well as the correlations between δ and y .

A 

A.10 MORE RESULTS AND COMPARISONS:

We have discussed the connection of the proposed approach with a number of related recent works in Section 3.4. Here we provide more comparison with them in Table 8 . Specifically, we compared with Mixup (Zhang et al., 2018) , Manifold-mixup (Verma et al., 2019) as well as their adversarial extensions IAT-Mixup and IAT-Manifold-mixup (Lamb et al., 2019) , and UAT (Uesato et al., 2019) , which is an adversarial training method augmented with an unsupervised loss computed on additional unlabeled data. The results in Table 8 for UAT is obtained using 20k additional images per class for training (Uesato et al., 2019) .

Attack Methods Models:

Mixup Here we provide more evaluations against adaptive attacks following reviewer's suggestions as well as Carlini et al. (2019) . We include the following aspects of "adaptiveness" in our evaluations: 1) we assume the full knowledge of the model as well as other necessary information and perfrom evaluations with multiple runs; 2) we include feature-based attack, which has the full knowledge of the model as well as the defense method. Specifically, we use the following attacks for evaluation:

• PGD-min: we run the evaluation agasint PGD20 attack 10 times with different random starts for the same test sample and record the worst performance for each sample; the average performance over the whole test set is reported as the final result.

:

x → clamp(x + • rand sign like(x), 0,

255), where rand sign like(x) returns a tensor with the same shape as x and the elements are randomly set to -1 or 1. clamp ensures the input is within the specified range.denotes the perturbation budget which is the same as used in training.

Figures and Tables:

5 Z g o p j N i s g Y K 0 y M b a l i S / C W T 1 4 l n U b d u 6 g 3 H i 5 r z Z u i j j K c w C m c g w d X 0 I R 7 a E E b C E h 4 h l d 4 c 4 z z 4 r w 7 H 4 v R k l P s H M M f O J 8 / d i u R X g = = < / l a t e x i t > r < l a t e x i t s h a 1 _ b a s e 6 4 = " m c x b H M l p e x U Z O Q 8 0 I 6 5 L 7 N 8 g j U M = " > A A A B 6 H i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 L B b B U 0 m q o M e i F 4 8 t 2 F p o Q 9 l s J + 3 a z S b s b o Q S + g u 8 e F D E q z / J m / / G b Z u D t j 4 Y e L w 3 w 8 y 8 I B F c G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o r e N U M W y

8 a l h A 6 I S P m W y p J z E y Q z S P P 8 J l V h j h S 2 j 4 J e K 7 + 3 s hI b M w 0 D u 1 k H t E s e 7 n 4 n + e n E N 0 E G Z d J C k z S x U d R K j A o n N + P h 1 w z C m J q C a G a 2 6 y Y j o k m F G x L F V u C t 3 z y K u k 0 6 t 5 l v f F w V W v e F n W U 0 Q k 6 R R f I Q 9 e o ie 5 R C 7 U R R Q o 9 o 1 f 0 5 o D z 4 r w 7 H 4 v R k l P s H K M / c D 5 / A E d 6 k T 8 = < / l a t e x i t >

Figure 1: Adversarial Interpolation for adversarial image generation. It takes a clean image x to be perturbed, and another distinct image x , performs feature extraction using the feature extractor f θ (•) and then measures the distance in the feature space using a distance measure D. The perturbed image is generated by minimizing the feature space distance with respect to x .

Performance under increasing T (20→1000) with PGD Performance under increasing T (20→1000) with CW 20 Madry 45.0 44.6 44.5 44.5 44.5 44.5 44.4 45.7 45.7 45.4 45.3 45.3 45.3 45.2 Adv-Interp 73.5 73.0 73.0 73.0 72.9 72.8 72.8 69.7 69.0 68.7 68.4 68.3 68.3 68.3

Figure 2: Adversarial loss curves of different models on different images under PGD attack with increasing iterations (1→100). Each column corresponds to the loss curves generated using the same input image on the Natural model (top) and the Adv-Interp model (bottom).

Figure 3: Feature distribution visualization of Natural and Adv-Interp models on (top) clean images, (middle) PGD attack perturbed images, and (bottom) adversarial interpolation perturbed images. It is observed that the distribution of features extracted using the Natural model changes drastically in the presence of attacks. On the other hand, the Adv-Interp model can better preserve the feature distribution against attacks.

It generates training examples (both image and label) by linear interpolation between pairs of natural examples, thus introducing a linear inductive bias in the vicinity of training samples. Similar to standard data augmentation, adversarial training incorporates

Table content: <table/>



Table content: <table/>

Algorithm 1 Adversarial Interpolation TrainingInput: training epochs K, learning rate α, budget , y , attack iterations L, number of classes C

Table content: <table/>

CIFAR10 results. Comparison of performance (classification accuracy) for Natural, Madry

Table content: <table><tr><td>Models</td><td>Clean</td><td colspan="6">Performance under White-box Attack ( = 8) Worst FGSM PGD20 PGD100 CW20 CW100</td></tr><tr><td>Natural</td><td>95.6</td><td>37.0</td><td>0.0</td><td>0.0</td><td>0.0</td><td>0.0</td><td>0.0</td></tr><tr><td>Madry</td><td>86.7</td><td>54.9</td><td>45.0</td><td>44.5</td><td>45.7</td><td>45.3</td><td>44.5</td></tr><tr><td>Bilateral</td><td>91.2</td><td>70.7</td><td>57.5</td><td>55.2</td><td>56.2</td><td>53.8</td><td>53.8</td></tr><tr><td>Feature-Scatter</td><td>90.0</td><td>78.4</td><td>70.5</td><td>68.6</td><td>62.4</td><td>60.6</td><td>60.6</td></tr><tr><td>Adv-Interp</td><td>90.3</td><td>78.0</td><td>73.5</td><td>73.0</td><td>69.7</td><td>68.7</td><td>68.7</td></tr></table>

Performance against stronger white-box attacks with increasing attack iterations. The models are trained with the attack budget =8.

Table content: <table><tr><td>Models</td><td>Clean</td><td colspan="6">Performance under White-box Attack Worst FGSM PGD20 PGD100 CW20 CW100</td><td>Models</td><td>Clean</td><td colspan="6">Performance under White-box Attack Worst FGSM PGD20 PGD100 CW20 CW100</td></tr><tr><td>Natural</td><td>78.6</td><td>9.7</td><td>0.0</td><td>0.0</td><td>0.0</td><td>0.0</td><td>0.0</td><td>Natural</td><td colspan="2">96.6 36.0</td><td>0.3</td><td>0.2</td><td>0.3</td><td>0.0</td><td>0.0</td></tr><tr><td>Madry</td><td colspan="3">59.9 28.5 22.6</td><td>22.2</td><td colspan="3">23.2 23.1 22.2</td><td>Madry</td><td colspan="3">93.9 68.4 47.9</td><td>46.0</td><td colspan="3">48.7 47.5 46.0</td></tr><tr><td>Bilateral</td><td colspan="3">68.2 60.8 26.7</td><td>25.3</td><td>-</td><td colspan="2">22.1 22.1</td><td>Bilateral</td><td colspan="3">94.1 69.8 53.9</td><td>50.3</td><td>-</td><td colspan="2">48.9 48.9</td></tr><tr><td colspan="4">Feature-Scatter 73.9 61.0 47.2</td><td>46.2</td><td colspan="3">34.6 30.6 30.6</td><td colspan="4">Feature-Scatter 96.2 83.5 62.9</td><td>52.0</td><td colspan="3">61.3 50.8 50.8</td></tr><tr><td>Adv-Interp</td><td colspan="3">73.6 58.3 41.0</td><td>40.2</td><td colspan="3">32.4 31.2 31.2</td><td>Adv-Interp</td><td colspan="3">94.1 75.6 65.8</td><td>64.0</td><td colspan="3">63.4 60.4 60.4</td></tr></table>

More evaluation results. Performance comparisons on (a) CIFAR100 and (b) SVHN.

Table content: <table><tr><td>Models</td><td>Clean</td><td colspan="5">Performance under White-box Attack Worst FGSM PGD20 PGD100 CW20 CW100</td><td>Models</td><td>Clean</td><td colspan="2">Performance under White-box Attack Worst FGSM PGD20 PGD100 CW20 CW100</td></tr><tr><td>image-org</td><td colspan="2">95.7 62.4 11.3</td><td>2.3</td><td>16.7</td><td>8.7</td><td>2.3</td><td>label-org</td><td colspan="2">91.6 57.3 38.1</td><td>36.5</td><td>39.3 38.0 36.5</td></tr><tr><td colspan="3">image-mixup 95.4 79.9 32.4</td><td>19.4</td><td colspan="3">31.8 23.1 19.4</td><td colspan="3">label-mixup 93.3 65.5 19.4</td><td>13.4</td><td>18.8 14.1 13.4</td></tr><tr><td>image-cls</td><td colspan="2">90.3 71.2 60.0</td><td>57.7</td><td colspan="3">56.3 54.1 54.1</td><td colspan="3">label-smooth 91.1 77.0 68.4</td><td>66.2</td><td>62.9 61.3 61.3</td></tr><tr><td colspan="3">Adv-Interp 90.3 78.0 73.5</td><td>73.0</td><td colspan="3">69.7 68.7 68.7</td><td>Adv-Interp</td><td colspan="2">90.3 78.0 73.5</td><td>73.0</td><td>69.7 68.7 68.7</td></tr></table>

, which is a 10-way house number classification dataset, with 73257 training images and 26032 test images. The additional training images are not used in experiment. The results are summarized in Table4(b). Experimental results show that the proposed method outperforms other baseline methods with a clear margin under both PGD and CW attacks. Notably, it outperforms Feature-Scatter by about 10% under the worst case metric, further demonstrating the effectiveness of the proposed approach.4.3 ABLATION STUDIES: THE IMPACTS OF ADVERSARIAL INTERPOLATIONWe conduct a number of ablation studies to investigate the impacts of the components in the adversarial interpolation scheme. We replace a single component while keeping all others the same. Impacts of Adversarial Image Interpolation. We investigate the impacts of image adversarial interpolation by comparing with the following: i) image-org: use the original image for training; ii) image-mixup: use the mixup operation for generating perturbed images (Eqn.(19)); iii) image-cls: use classification loss for generating perturbed image; The results are presented in Table5(a). It is observed that there is a significant performance drop when removing image adversarial interpolation (image-org). image-mixup performs better than image-org with improved robustness. image-cls is more effective due to the usage of classification loss in generating adver-Black-box attack evaluations. Performance of the model trained with the Adv-Interp method under (a) gradient-based and (b) gradient-free black-box attacks. sarial images. The proposed method, which uses adversarial interpolation for generating perturbed images, achieves the best performance, implying the importance of adversarial image interpolation. Impacts of Adversarial Label Interpolation. Similarly, we replace the adversarial label interpolation part with different approaches: i) label-org: the original label is used for training; ii) label-mixup: a mixup operation is applied to the label (Eqn.(20)); iii) label-smooth: label smoothing is applied to the label (Eqn.(16)); The results are summarized in Table5(b). It is observed that, without adversarial interpolation for label (label-org), the performance drops significantly. label-mixup improves performance under weak attacks (FGSM), but the performance under stronger attacks is decreased, possibility due to its inability in de-emphasizing the correlations between the perturbation and y . label-smooth achieves a much better performance than label-org and label-mixup. The Adv-Interp method further boosts the performance over label-smooth, demonstrating the importance and effectiveness of adversarial label interpolation.4.4 PERFORMANCE UNDER BLACK-BOX ATTACKSTo further verify that the improvement is due to inherent improvement in model robustness instead of a degenerate solution, we evaluate the robustness of the model trained with the proposed approach w.r.t.black-box attacks following previous evaluation setup

Table content: <table><tr><td>Attack Gradient</td><td colspan="4">Performance under Black-box Attack</td><td>Gradient-Free</td><td>Performance under</td></tr><tr><td>Generation Models</td><td>PGD20</td><td>PGD100</td><td>CW20</td><td>CW100</td><td>Attack Methods</td><td>Black-box Attack</td></tr><tr><td>Natural</td><td>89.3</td><td>89.1</td><td>89.4</td><td>89.2</td><td>SPSA</td><td>89.2</td></tr><tr><td>Adv-Interp'</td><td>82.9</td><td>82.5</td><td>81.8</td><td>82.0</td><td>Local-Search</td><td>90.1</td></tr></table>

.6 LABEL ADV-INTERPOLATION PARAMETERWe investigate the impacts of the adversarial label interpolation parameter y and the results are summarized in the table on the right. It is observed from the results that the proposed approach performs well for a range of values of y .A.7 INVESTIGATION OF THE ADVERSARIAL LOSSESFollowingMadry et al. (2018), we investigate and visualize the variations of the losses caused by the adversary (adversarial losses). We run PGD adversary with random starts for 10 times, against Natural model as well as the proposed Adv-Interp model. The results are presented in Figure2. It can be observed that the loss achieved with PGD adversary against our model increases in a fairly consistent way and plateaus rapidly for 10 different runs with random starts, with relatively smaller variance and final loss value, compared with those of Natural model. These observations are consistent with properties of robust models observed inMadry et al. (2018), further verifying the inherent robustness improvement of the proposed model.

Table content: <table><tr><td>y</td><td>Clean</td><td colspan="5">Performance under White-box Attack ( = 8) FGSM PGD20 PGD100 CW20 CW100</td></tr><tr><td>0</td><td>91.6</td><td>57.3</td><td>38.1</td><td>36.5</td><td>39.3</td><td>38.0</td></tr><tr><td colspan="2">0.1 90.9</td><td>73.3</td><td>57.3</td><td>52.9</td><td>57.2</td><td>54.2</td></tr><tr><td colspan="2">0.2 90.5</td><td>74.9</td><td>61.6</td><td>58.5</td><td>63.4</td><td>61.5</td></tr><tr><td colspan="2">0.3 90.6</td><td>78.2</td><td>73.5</td><td>72.6</td><td>68.7</td><td>68.0</td></tr><tr><td colspan="2">0.4 90.8</td><td>77.8</td><td>73.7</td><td>73.0</td><td>67.7</td><td>66.6</td></tr><tr><td colspan="2">0.5 90.3</td><td>78.0</td><td>73.5</td><td>73.0</td><td>69.7</td><td>68.7</td></tr><tr><td colspan="2">0.6 90.5</td><td>74.9</td><td>61.6</td><td>58.4</td><td>63.5</td><td>61.5</td></tr><tr><td colspan="2">0.7 90.2</td><td>73.5</td><td>59.8</td><td>48.7</td><td>59.2</td><td>50.8</td></tr></table>



Table content: <table><tr><td>Natural</td><td>96.8</td><td>96.9</td><td>93.6</td><td>93.5</td><td>85.9</td><td>90.3</td></tr><tr><td>FGSM</td><td>67.4</td><td>61.6</td><td>66.2</td><td>64.8</td><td>-</td><td>78.0</td></tr><tr><td>PGD20</td><td>0.7</td><td>1.7</td><td>50.1</td><td>44.8</td><td>62.2</td><td>73.5</td></tr></table>

More results and comparisons with several recent methods on CIFAR10.A.11 EVALUATIONS AGAINST ADAPTIVE ATTACKS

Table content: <table/>

Peer review (iclr2020):
Contribution: This paper proposed an adversarial interpolation approach for generating adversarial samples and then training robust deep nets by leveraging the generated adversarial samples. However, I have the following concerns: 1. In adversarial interpolation training, why 	ilde{y}_i* is set to 1/(C-1)(1-y_{n-i+1})? 2. This work lack of interpretation of why the proposed method is more effective than PGD adversarial training. 3. How about training deep nets with replicas of the training data but replace the true labels with random labels? I want to see such a result. 4. Can the authors provide the black-box attack results also? I want to see the performance of PGD adversarially trained deep nets on the adversarial images crafted by attacking Adv-Interp trained deep nets, and vice versa. 5. Can the authors provide the visualization of a few adversarial interpolated images? 6. The authors should compare with the existing efforts that using interpolation to improve adversarial robustness. Below are some Related works on using interpolation in deep nets to improve their robustness 1). Bao Wang, Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi, Stanley J. Osher. Deep Neural Nets with Interpolating Function as Output Activation, NeurIPS, 2018 2). Bao Wang, Alex T. Lin, Zuoqiang Shi, Wei Zhu, Penghang Yin, Andrea L. Bertozzi, Stanley J. Osher. Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization, arXiv:1809.08516, 2018 3). B. Wang, S. Osher. Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning, arXiv:1907.06800, 2019 7. Moreover, the following paper provides a theoretical interpretation of adversarial vulnerability of deep nets, and proposed a nice ensemble of neural SDEs to improve deep nets* robustness. 1). Bao Wang, Binjie Yuan, Zuoqiang Shi, Stanley J. Osher. ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies, arXiv:1811.10745, NeurIPS, 2019 Please address the previously mentioned concerns in rebuttal, and this paper can be acceptable if all my concerns are addressed.