Overcoming the Convex Barrier for Simplex Inputs

1 Introduction . Verification refers to the challenging computational problem of determining whether a neural network satisfies a given specification . Perhaps the most popular specification is to prove or disprove that a neural network classifier is robust to perturbations of the input that lie within an ` p ball [ de Palma et al. , 2021a , Wong and Kolter , 2018 , Zhang et al. , 2018 , Weng et al. , 2018 , Mirman et al. , 2018 , Dvijotham et al. , 2018 ] , with impressive state of the art results recently achieved in Wang et al . [ 2021 ] . The ability to verify neural networks would open the door to better understanding their nature , and allow us to apply deep learning in safety critical domains where errors can have a large cost . Given the importance of the problem , it is unsurprising that it has attracted considerable atttention from the machine learning and automated verification communities [ Katz et al. , 2017 , Ehlers , 2017 ] . Early work in verification lead to the notion of a convex barrier , that is , a limitation in the tightness of the bounds obtainable by the convex relaxation of the output of a neural network [ Salman et al. , 2019 ] . This weakness was seen as a primary reason for the slow convergence of branch-and-bound algorithms for verification , which rely on convex relaxations to compute the bounds , on a large number of specifications on standard datasets . However , recent work has been able to successfully overcome this barrier . Specifically , Anderson et al . [ 2019 ] proposed a tight relaxation that precisely defines the convex hull of a composition of a linear function of a vector within an ` ∞ ball with the ReLU non-linearity . While their relaxation has a large number of constraints ( exponential in the size of the vector ) , they provide an efficient algorithm for identifying the most violated constraint at any infeasible point . This enables the use of efficient cutting plane algorithms to solve the 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . primal [ Anderson et al. , 2019 ] , active sets to solve the dual [ de Palma et al. , 2021a , b ] , and even approximate linear bound propagation algorithms [ Tjandraatmadja et al. , 2020 ] . Buoyed by the possibility of realizing practical verification using tight relaxations , we consider an important specification , namely , robustness to perturbations that lie in a probability simplex . Examples of such a specification include robustness to ` 1 perturbations or to word substitutions [ Huang et al. , 2019 ] . Previous approaches for addressing this specification relied on fairly loose relaxations based on interval bound propagation [ Huang et al. , 2019 , Gowal et al. , 2018 ] . To alleviate this deficiency , we derive the convex hull of the composition of a linear transformation of the probability simplex with a convex non-linearity such as ReLU or SoftPlus . Somewhat surprisingly , we show that , unlike the previously considered case of ` ∞ balls , the probability simplex helps greatly simplify the description of the convex hull . In fact , the number of constraints required is linear in the dimensionality of the simplex . By using a novel technique to propagate the simplex constraints through the hidden layers of the network , we derive a tight relaxation for the output of a network whose size scales linearly with the size of the network . Furthermore , we suitably extend a linear bound propopagation algorithm [ Zhang et al. , 2018 ] to solve the tight relaxation efficiently , thereby realizing practical verification over simplex inputs . We demonstrate the scalability of our approach using the specification of robustness to ` 1 perturbations for MNIST and CIFAR-10 classification . Our method achieves 13.6 % higher verified accuracy on MNIST and up to 14.4 % higher verified accuracy on CIFAR-10 compared to the state of the art baselines on the same networks given the same computational budget . To further demonstrate the benefits of our tight relaxation , we consider a novel and highly challenging specification of global robustness in multi-modal classification . Specifically , we consider the Food 101 dataset [ Wang et al. , 2015 ] , where each sample consists of an image of a food item together with its recipe . Our specification requires a neural network trained to classify the food item to be robust to arbitrary changes in the text of a sample . This specification captures the scenario where the image is carefully curated while the text is crowd-sourced and can therefore be easily manipulated by adversarial actors . While similar specifications have been considered in previous works [ Jia et al. , 2019 , Huang et al. , 2019 ] , they focus on substituting a very small subset of words with their synonyms , which leads to simpler verification problems . We show that , on our significantly more difficult global verification task , our method achieves up to 25 % higher verified accuracy in the same amount of time as the state of the art baseline . 2 Preliminaries . We provide a formal mathematical description of our verification problem , and briefly discuss the existing relaxations in order to contextualise our contributions . 2.1 Problem description . We denote the m-dimensional probability simplex as ∆m , that is , ∆m = { x ∈ Rm | x ≥ 0 , 1 > x ≤ 1 } . As mentioned in the previous section , we are interested in verifying specifications of a given neural network when its input is constrained to lie in a simplex . We focus on networks with a layered architecture to keep notation simple , but our ideas easily extend to any feed-forward network , including residual networks . We consider a neural network with n layers . Each layer is assumed to be composed of two operations : ( i ) an affine operation ( fully connected layer or convolution layer ) , which we denote by Lk ( · ) ; and ( ii ) a non-linear activation function , which we denote by σ ( · ) . In other words , given its input xk−1 ∈ Rnk−1 , the k-th layer performs the operation x̂k = Lk ( xk−1 ) ∈ Rnk , followed by xk = σ ( x̂k ) ∈ Rnk . While we place no restriction on the linear operation , we make the following assumption on the activation function . Assumption 1 . We assume that the activation function σ is an element-wise convex function ( for example , ReLU or SoftPlus ) . Verification problem : Using the above notation , the verification problem we solve can be formulated as min x Ψ ( x̂n ) ( 1a ) s.t . x̂k = Lk ( xk−1 ) , xk = σ ( x̂k ) k ∈ [ n ] , ( 1b ) x0 ∈ ∆n0 , ( 1c ) where [ n ] denotes the set { 1 , · · · , n } . The objective function Ψ is a scalar-valued linear function of the output of the final layer of the the network . For example , say we wish to verify that a classification network is not vulnerable to any adversarial attacks . In this case , we define Ψ as the difference between the true logit and a target logit outputted by the network . The sign of the optimum value of problem ( 1 ) can be used to either prove or disprove the aforementioned specification . We illustrate the practical importance of considering simplex inputs using two examples . These examples will also form the basis of our experimental setup . ` 1 perturbations : Consider a network with a continuous valued input x ∈ Rm . We are interested in verifying the behavior of the network under input perturbations that lie within an ` 1 ball : { x |∥∥x− x0∥∥ 1 ≤ } . This input domain can be reformulated as a simplex as x = x0 + Mz , z ∈ ∆2m , M ( m×2m ) = 1 −1 0 0 . . . 0 0 0 0 1 −1 . . . 0 0 ... 0 0 0 0 . . . 1 −1 . ( 2 ) Bag of words models : Consider a text classification network that takes text as input and makes predictions based on an embedding of the text . A commonly used embedding is the so-called “ bag of words ” . Here , we first take an embedding for every word in the text ( for example , using a precomputed set like GloVe [ Pennington et al. , 2014 ] or Word2Vec [ Mikolov et al. , 2013 ] ) . Next , we take the mean of all the word embeddings to obtain the final representation . We denote the word embeddings as a matrix E ∈ Rd×v , where each d-dimensional column represents an embedding of a putative word in a vocabulary of size v. Using the above matrix , the text embedding is given by Ex where x represents the normalized counts of words from the vocabulary in the text . In other words , x ∈ ∆v assuming that arbitrarily long text with arbitrary numbers of repetitions of each word in the vocabulary are allowed . 2.2 Planet and disjunctive relaxations . Ehlers [ 2017 ] proposed a convex relaxation for the ReLU activation function , which is commonly referred to as Planet in the literature . The Planet relaxation has been widely used in many verification algorithms [ Bunel et al. , 2020b , Dvijotham et al. , 2018 , Bunel et al. , 2020a , Lu and Kumar , 2020 ] . Briefly , it relaxes the sequence of two operations : x̂ = wTx + b where x ∈ Rm , followed by y = ReLU ( x̂ ) . To this end , it utilizes lower and upper bounds on x̂ . In our case , since x ∈ ∆m , we can compute the bounds as ˜̀= wmin + b ≤ wTx + b ≤ wmax + b = ũ . Here , wmin and wmax denote the minimum and maximum entries of w respectively . Using the bounds on x̂ , the Planet relaxation for the set S = { y , x | y = ReLU ( wTx + b ) , x ∈ ∆m } is defined as P∆ : y ≥ wTx + b , y ≥ 0 , y ≤ ũ ũ− ˜̀ ( wTx + b− ˜̀ ) , x ∈ ∆m . ( 3 ) In Anderson et al . [ 2019 , 2020 ] , the authors propose a tighter relaxation and characterize the exact convex hull of the set { ( x , relu ( wTx + b ) ) : ` ≤ x ≤ u } . However , this characterization involves exponentially many inequalities . And efficient algorithms based on it need to resort to cutting plane methods , adding violated inequalities sequentially . Doing so is computationally challenging and requires significant effort to implement in a scalable manner . Further , this does not handle the simplex constraint on the input . Thus using this relaxation would require replacing the simplex constraint with the unit hypercube , a much weaker constraint on the inputs . Anderson et al . [ 2020 ] also propose an ideal formulation for the product of k simplices . Their formulation needs additional variables and still requires cutting planes . However , eliminating the auxiliary variable , as in our case , leads to a concise formulation . Further , our relaxation is derived for convex activation functions , whereas the formulation of Anderson et al . [ 2020 ] is limited to the maximum of two affine functions .

Summary: The paper proposes  a particular relaxation for the verification of neural network robustness under perturbation that can be phrased as functions over the probability simplex (e.g. $\ell_1$-norm).  Mathematically, the verification is first phrased as an optimization problem, that mirrors the layer structure of the neural network. For the ReLU (or other convex activation functions) they employ the well-known convex relaxation from [Ehlers, 2017]* ("Planet") and improve upon it by a new upper bound, which utilizes the fact that the input (and all intermediate neuron values post-activation) are subject to a simplex constraint. This relaxation forms the convex hull of the possible values of a post-activation neuron and thus is tight (for a single neuron). The resulting optimization problem forms a linear program with 2 linear lower bounds, 2 linear upper bounds and a simplex constraint for each neuron. To allow for faster and more scalable evaluation the optimization problem is relaxed following the approach of [Zhang et al. 2018]. The 2 upper and lower bounds are relaxed into one each, that is a linear interpolation weighted by an $\alpha  \in [0, 1]$. In this formulation the optimum of the problem can be found by alternatingly computing the neuron bounds (as a function of fixed $\alpha$) and optimizing the values of $\alpha$ via gradient decent.  Experimentally the authors show that both, the relaxed version and the linear program version outperform Plant relaxations in terms of certified accuracy by a large margin on image classification (MNIST and CIFAR-10) under $\ell_{1}$ perturbations and robustness to text-perturbations for BoW-representations in an multi-modal setting (simplified Food-101). The relaxed version almost matches the linear program version in performance, but is significantly faster.  *Citations in the [Name, Year] format refer to citations in the paper.