Compressive Recovery Defense: A Defense Framework for $\ell_0, \ell_2$ and $\ell_\infty$ norm attacks.

COMPRESSIVE RECOVERY DEFENSE : A DEFENSE FRAMEWORK FOR ` 0 , ` 2 , AND ` ∞ NORM ATTACKS . Anonymous authors Paper under double-blind review 1 ABSTRACT . We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in Bafna et al . ( 2018 ) to defend neural networks against ` 0 , ` 2 , and ` ∞-norm attacks . In the case of ` 0-norm noise , we provide recovery guarantees for Iterative Hard Thresholding ( IHT ) and Basis Pursuit ( BP ) . For ` 2-norm bounded noise , we provide recovery guarantees for BP , and for the case of ` ∞-norm bounded noise , we provide recovery guarantees for a modified version of Dantzig Selector ( DS ) . These guarantees theoretically bolster the defense framework introduced in Bafna et al . ( 2018 ) for defending neural networks against adversarial inputs . Finally , we experimentally demonstrate the effectiveness of this defense framework against an array of ` 0 , ` 2 and ` ∞-norm attacks . 2 INTRODUCTION . Signal measurements are often corrupted by noise . The theory of compressive sensing ( Candes et al . ( 2006 ) ) allows us to retrieve the original signal from a corrupted measurement , under some structural assumptions on the measurement mechanism and the signal . Let us consider the class of machine learning problems where the inputs are compressible ( i.e. , approximately sparse ) in some domain . For instance , images and audio signals are known to be compressible in their frequency domain and machine learning algorithms have been shown to perform exceedingly well on classification tasks that take such signals as input ( Krizhevsky et al . ( 2012 ) ; Sutskever et al . ( 2014 ) ) . However , it was found in Szegedy et al . ( 2013 ) that neural networks can be easily forced into making incorrect predictions by adding adversarial perturbations to their inputs ; see also Szegedy et al . ( 2014 ) ; Goodfellow et al . ( 2015 ) ; Papernot et al . ( 2016 ) ; Carlini & Wagner ( 2017 ) . Further , the adversarial perturbations that led to incorrect predictions were shown to be very small ( in either ` 0 , ` 2 , or ` ∞-norm ) and often imperceptible to human beings . For this class of machine learning tasks , we show how to approximately recover original inputs from adversarial inputs and thus defend the neural network ` 0-norm , ` 2-norm and ` ∞-norm attacks . In the case of ` 0-norm attacks on neural networks , the adversary can perturb a bounded number of coordinates in the input vector but has no restriction on how much each coordinate is perturbed in absolute value . In the case of ` 2-norm attacks , the adversary can perturb as many coordinates of the input vector as they choose as long as the ` 2-norm of the perturbation vector is bounded . Finally , in ` ∞-norm attacks , the adversary is only constrained by the amount of noise added to each coordinate of the input vector . The contribution and structure of this paper is as follows . In Section 3.1 , we describe the Compressive Recovery Defense ( CRD ) framework , a compressive-sensing-based framework for defending neural networks against adversarial inputs . This is essentially the same framework introduced in Bafna et al . ( 2018 ) , though Bafna et al . ( 2018 ) considered only ` 0 attacks . In Section 3.2 , we present the recovery algorithms which are used in the CRD framework to approximately recover original inputs from adversarial inputs . These algorithms include standard Basis Pursuit ( BP ) , ( k , t ) -sparse Iterative Hard Thresholding ( IHT ) and Dantzig Selector ( DS ) with an additional constraint . In Section 3.3 , we state recovery guarantees for the recovery algorithms in the presence of noise bounded in either ` 0 , ` 2 , or ` ∞-norm . The guarantees apply to arbitrary ` 0 , ` 2 , and ` ∞-norm attacks ; they do not require prior knowledge of the adversary ’ s attack strategy . The recovery guarantees are proved rigorously in Appendix A . In Section 4 , we experimentally demonstrate the performance of the CRD framework in defending neural network classifiers on CIFAR-10 , MNIST , and FashionMNIST datasets against state-of-the-art ` 0 , ` 2 and ` ∞-norm attacks . Notation . Let x be a vector in CN . Let S ⊆ { 1 , . . . , N } and S = { 1 , . . . , N } \ S. The cardinality of S is |S| . If A ∈ Cm×N is a matrix , then AS ∈ Cm×|S| is the column submatrix of A consisting of the columns indexed by S. We denote by xS either the sub-vector in CS consisting of the entries indexed by S or the vector in CN that is formed by starting with x and setting the entries indexed by S to zero . For example , if x = [ 4 , 5 , −9 , 1 ] T and S = { 1 , 3 } , then xS is either [ 4 , −9 ] T or [ 4 , 0 , −9 , 0 ] T . It will always be clear from context which meaning is intended . Note that , under the second meaning , xS = x − xS . The support of x , denoted by supp ( x ) , is the set of indices of the non-zero entries of x , i.e. , supp ( x ) = { i ∈ { 1 , . . . , N } : xi 6= 0 } . The ` 0-quasinorm of x , denoted ‖x‖0 , is defined to be the number of non-zero entries of x , i.e . ‖x‖0 = card ( supp ( x ) ) . We say that x is k-sparse if ‖x‖0 ≤ k. We use xh ( k ) to denote a k-sparse vector in CN consisting of the k largest ( in absolute value ) entries of x with all other entries zero . For example , if x = [ 4 , 5 , −9 , 1 ] T then xh ( 2 ) = [ 0 , 5 , −9 , 0 ] T . Note that xh ( k ) may not be uniquely defined . In contexts where a unique meaning for xh ( k ) is needed , we can choose xh ( k ) out of all possible candidates according to a predefined rule ( such as the lexicographic order ) . We also define xt ( k ) = x− xh ( k ) . If x = [ x1 , x2 ] T ∈ C2n with x1 , x2 ∈ Cn , and if x1 is k-sparse and x2 is t-sparse , then x is called ( k , t ) -sparse . We define xh ( k , t ) = [ ( x1 ) h ( k ) , ( x2 ) h ( t ) ] T , which is a ( k , t ) -sparse vector in C2n . 3 THEORY . 3.1 COMPRESSIVE RECOVERY DEFENSE ( CRD ) . Bafna et al . ( 2018 ) introduced a framework for defending machine learning classifiers against ` 0- attacks . We extend the framework to ` 2 and ` ∞ attacks . The defense framework is based on the theory of compressive sensing , so we call it Compressive Recovery Defense ( CRD ) . We explain the idea behind the CRD framework in the context of an image classifier . Suppose x ∈ Cn is a ( flattened ) image vector we wish to classify . But suppose an adversary perturbs x with a noise vector e ∈ Cn . We observe y = x + e , while x and e are unknown to us . Let F ∈ Cn×n be the Discrete Fourier Transform ( DFT ) matrix . The Fourier coefficients of x are x̂ = Fx . It is well-known that natural images are approximately sparse in the frequency domain . So we expect that x̂ is approximately sparse , meaning that x̂t ( k ) is small for some small k. We can write y = F−1x̂+ e ( 1 ) If ‖e‖2 ≤ η or ‖e‖∞ ≤ η , with η small ( as in a ` 2 or ` ∞-attack ) , then we can use an appropriate sparse recovery algorithm with y and F−1 as input to compute a good approximation x # to x̂ . Precise error bounds are given in Section 3.3 . Then , since F is unitary , F−1x # will be a good approximation ( i.e. , reconstruction ) of x = F−1x̂ . So we can feed F−1x # into the classifier and expect to get the same classification as we would have for x . For an ` 0-attack where e is t-sparse , the approach is only slightly different . We set A = [ F−1 , I ] and write y = F−1x̂+ e = F−1x̂h ( k ) + e+ F −1x̂t ( k ) = A [ x̂h ( k ) , e ] T + F−1x̂t ( k ) , ( 2 ) so that [ x̂h ( k ) , e ] T is ( k , t ) -sparse . This structure lets us use a sparse recovery algorithm to compute a good approximation to x̂ , as before . Note that the same idea can be applied with audio signals or other types of data instead of images . Moreover , the DFT can be replaced by any unitary transformation F for which x̂ = Fx is approximately sparse . For example , F may be the Cosine Transform , Sine Transform , Hadamard Transform , or another wavelet transform . We now describe the training and testing procedure for CRD . For each training image x , we compute x̂h ( k ) = ( Fx ) h ( k ) , and then compute the compressed the image x′ = F−1x̂h ( k ) . We then add both x and x′ to the training set and train the network in the usual way . Given a ( potentially adversarial ) test image y , we first use a sparse recovery algorithm to compute an approximation x # to x̂ , then we compute the reconstructed image y′ = F−1x # and feed it into the network for classification . 3.2 RECOVERY ALGORITHMS . We provide the recovery algorithms used in this section . For ` 0-attacks , we set A = [ F−1 , I ] as in ( 2 ) . Against ` 2 or ` ∞-attacks , we take A = F−1 as in ( 1 ) . Algorithm 1 : ( k , t ) -Sparse Iterative Hard Thresholding ( IHT ) Procedure : IHT ( y , A , k , t , T ) Input : y ∈ Cn , A ∈ Cn×2n , and positive integers k , t , T . x [ 0 ] = 0 for i : = 0 to T do x [ i+1 ] = ( x [ i ] +A∗ ( y −Ax [ i ] ) ) h ( k , t ) return x # = x [ T+1 ] The IHT algorithm above is used to defend against ` 0-norm attacks . For such attacks , according to ( 2 ) , the vector we need to recover is ( k , t ) -sparse . Thus this IHT is adapted to the structure of our problem as it uses the thresholding operation h ( k , t ) that produces ( k , t ) -sparse vectors . This structured IHT was first considered in Baraniuk et al . ( 2010 ) . It gives better theoretical guarantees and practical performance in our CRD application than the standard IHT , which would instead use the thresholding operation h ( k+t ) that produces ( k + t ) -sparse vectors . For ` 2 or ` ∞ attacks , the recovery error for IHT would ( in general ) be larger due to the need to include a term for the ` 2 norm of the tail of the noise vector e. This , in turn , produces worse expected performance of the recovery defense . Therefore we only use Algorithm 1 for ` 0-norm attacks . We note that the results of Theorem 1 allow for values of k and t greater than or equal to Theorem 2.2. of Bafna et al . ( 2018 ) . Algorithm 2 : Basis Pursuit ( BP ) . Procedure : BP ( y , A , η ) . Input : y ∈ Cm , A ∈ Cm×N , and η ≥ 0. x # = argminz∈CN ‖z‖1 subject to‖Az − y‖2 ≤ η return x # We utilize BP for ` 0 and ` 2 norm attacks . In the ` 0 norm case , BP allows us to provide recovery guarantees for larger values of k and t than IHT . For instance , in the case of MNIST and FashionMNIST , IHT ( equation ( 4 ) of Theorem 1 ) allows us to set k = 4 and t = 3 , whereas BP ( equation ( 7 ) of Theorem 2 ) allows us to set k = 8 and t = 8 . In the case of ` 2 norm attacks , BP is applied withA = F−1 , a unitary matrix . As unitary matrices are isometries in ` 2 norm , BP provides good recovery guarantees for such matrices , and hence against ` 2 norm attacks . Algorithm 3 : Modified Dantzig Selector ( DS ) . Procedure : DS ( y , A , η ) . Input : y ∈ Cm , A ∈ Cm×N , and η ≥ 0. x # = argminz∈CN ‖z‖1 subject to ‖A∗ ( Az − y ) ‖∞ ≤ √ nη , ‖Az − y‖∞ ≤ η return x # We utilize DS for ` ∞ norm attacks . The standard Dantzig Selector algorithm does not have the additional constraint ‖Az − y‖∞ ≤ η . Our modified Dantzig Selector includes this constraint for the following reason . In our application , A = F−1 and we want the reconstruction Ax # = F−1x # to be close to the original image x , so that they are classified identically . Thus , we want to the search space for x # to be restricted to those z ∈ CN such that ‖Az−x‖∞ is small . Note , for any z ∈ CN , ‖Az − x‖∞ ≤ ‖Az − y‖∞ + ‖x − y‖∞ . In an ` ∞-attack , ‖x − y‖∞ = ‖e‖∞ is already small . Thus it suffices to require ‖Az − y‖∞ is small . We experimentally illustrate the improvement in reconstruction due to the additional constraint in Section 4.3 ( Figure 4 , Table 4 ) . Remarks on Reverse-Engineered Attacks . As observed in Bafna et al . ( 2018 ) , x [ 0 ] in Algorithm 1 , can be initialized randomly to defend against a reverse-engineered attack . In the case of Algorithm 2 and Algorithm 3 , the minimization problems can be posed as semi-definite programming problems . If solved with interior point methods , one can use random initialization of the central path parameter and add randomness to the stopping criterion . This makes recovery non-deterministic and consequently non-trivial to create a reverse-engineered attack .

Summary: This paper extends the compressive recovery defense framework introduced by Bafna et al. (2018), which is mainly against l_0 attacks, to l_2 and l_∞ attacks. They provide guarantees for some recovery algorithms in the case of different kinds of norm bounded noises. The difference between their work and the previous work is clearly clarified.