N EURAL N ETWORK B RANCHING FOR N EURAL N ETWORK V ERIFICATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Formal veriﬁcation of neural networks is essential for their deployment in safetycritical areas.
Many available formal veriﬁcation methods have been shown to be instances of a uniﬁed Branch and Bound (BaB) formulation.
We propose a novel framework for designing an effective branching strategy for BaB. Speciﬁcally, we learn a graph neural network (GNN) to imitate the strong branching heuristic behaviour.
Our framework differs from previous methods for learning to branch in two main aspects.
Firstly, our framework directly treats the neural network we want to verify as a graph input for the GNN.
Secondly, we develop an intuitive forward and backward embedding update schedule.
Empirically, our framework achieves roughly 50% reduction in both the number of branches and the time required for veriﬁcation on various convolutional networks when compared to the best available hand-designed branching strategy.
In addition, we show that
our
GNN model enjoys both horizontal and vertical transferability.
Horizontally, the model trained on easy properties performs well on properties of increased difﬁculty levels.
Vertically, the model trained on small neural networks achieves similar performance on large neural networks.
I NTRODUCTION
Despite their outstanding performances on various tasks, neural networks are found to be vulnerable to adversarial examples (Goodfellow et al., 2015; Szegedy et al., 2013).
The brittleness of neural networks can have costly consequences in areas such as autonomous driving, ﬁnance and healthcare.
When one requires robustness to adversarial examples, traditional model evaluation approaches, which test the trained model on a hold-out set, do not sufﬁce.
Instead, formal veriﬁcation of properties such as adversarial robustness becomes necessary.
For instance, to ensure self-driving cars make consistent correct decisions even when the input image is slightly perturbed, the required property to verify is that the underlying neural network outputs the same correct prediction for all points within a norm ball whose radius is determined by the maximum perturbation allowed.
Several methods have been proposed for verifying properties on neural networks (NN).
Bunel et al.
(2018) showed that many of the available methods can be viewed as instances of a uniﬁed Branch and Bound (BaB) framework.
A BaB algorithm consists of two key components: branching strategies and bounding methods.
Branching strategies decide how the search space is recursively split into smaller space.
Bounding methods compute bounds of each split space to tighten the bounds of the ﬁnal objective function over the whole search space.
In this work, we focus on improving the branching strategies.
By directly working with a general framework, our identiﬁed algorithmic improvements can be combined with any bounding method, leading to potential performance improvement for BaB based veriﬁcation algorithms.
Branching strategy has a signiﬁcant impact on the overall problem-solving process, as it directly decides the total number of steps, consequently the total time, required to solve the problem at hand.
The quality of a branching strategy is even more important when NN veriﬁcation problems are considered, which generally have a very large search space.
Each input dimension or each activation unit can be a potential branching option and neural networks of interest often have high dimensional input and thousands of hidden activation units.
With such a large search space, an effective branching strategy could mean a large reduction of the total number of branches required, consequently time required to solve a problem.
Developing an effective strategy is thus of signiﬁcant importance to the success of BaB based NN veriﬁcation.
So far, to the best of our knowledge, branching rules adopted by BaB based veriﬁcation methods are either random selection (Katz et al., 2017; Ehlers, 2017) or hand-designed heuristics (Wang et al., 2018b; Bunel et al., 2018;
Royo et al., 2019; Bunel et al., 2019).
Random selection is generally inefﬁcient as the distribution of the best branching decision is rarely uniform.
In practice, this strategy often results in exhaustive search to make a veriﬁcation decision.
On the other hand, hand designed heuristics often involve a trade-off between effectiveness and computational cost.
For instance, strong branching is generally one of the best performing heuristics for BaB methods in terms of the number of branches, but it is computationally prohibitive as each branching decision requires an expensive exhaustive search over all possible options.
The heuristics that are currently used in practice are either inspired by the corresponding dual problem when veriﬁcation is formulated as an optimization problem (Bunel et al., 2018; Royo et al., 2019) or incorporating the gradient information of the neural network (Wang et al., 2018b).
These heuristics normally have better computational efﬁciency.
However, given the complex nature of the search space, it is unlikely that any hand-designed heuristics is able to fully exploit the structure of the problem and the data distribution encountered in practice.
As mentioned earlier, for large size NN veriﬁcation problems, a slight reduction of branching quality strategy could lead to exponential increase in the total number of branches required to solve the problem.
A computationally cheap but high quality branching strategy is thus much needed.
In order to exploit the inherent structure of the problem and the data, we propose a novel machine learning framework for designing a branching strategy.
Our framework is both computational efﬁcient and effective, giving branching decisions that are of a similar quality to that of strong branching.
Speciﬁcally, we make following contributions:
• We use a graph neural network (GNN) to exploit the structure of the neural network we want to verify.
The embedding vectors of the GNN are updated by a novel schedule, which is both computationally cheap and memory efﬁcient.
In detail, we mimic the forward and backward passes of the neural network to update the embedding vectors.
In addition, the proposed GNN allows a customised schedule to update embedding vectors via shared parameters.
That means, once training is done, the trained GNN model is applicable to various veriﬁcation properties on different neural network structures.
• We train GNNs via supervised learning.
We provide ways to generate training data cheaply but inclusive enough to represent branching problems at different stages of a BaB process for various veriﬁcation properties.
With the ability to exploit the neural network structure and a comprehensive training data set, our GNN is easy to train and converges fast.
• Our learned GNN also enjoys transferability both horizontally and vertically.
Horizontally, although trained with easy properties, the learned GNN gives similar performance on medium and difﬁcult level properties.
More importantly, vertically, given that all other parts of BaB algorithms remain the same, the GNN trained on small networks performs well on large networks.
Since the network size determines the total cost for generating training data and is positively correlated with the difﬁculty of learning, this vertical transferability allows our framework to be readily applicable to large scale problems.
• We further enhance our framework via online learning.
For a learned branching strategy, it is expected that the strategy can fail to output satisfactory branching decisions from time to time.
To deal with this issue, we provide an online scheme for ﬁne-tuning the GNN along the BaB process in order to best accommodate the veriﬁcation property at hand.
Finally, we supply a dataset on convolutional NN veriﬁcation problems, covering problems at different difﬁculty levels over neural networks of different sizes.
We hope that by providing a large problem dataset it could allow easy comparisons among existing methods and additionally encourage the development of better methods.
Since most veriﬁcation methods available work on ReLU-based deep neural networks, we focus on neural networks with ReLU activation units in this paper.
However, we point out that our framework is applicable to any neural network architecture.
R ELATED W ORKS
Learning has already been used in solving combinatorial optimization (Bello et al., 2016;
Dai et al., 2017) and mixed integer linear programs (MILP) (Khalil et al., 2016; Alvarez et al., 2017; Hansknecht et al., 2018; Gasse et al., 2019).
In these areas, instances of the same underlying structure are solved multiple times with different data value, which opens the door for learning.
Among them,
Khalil et al.
(2016), Alvarez et al.
(2017), Hansknecht et al.
(2018), and Gasse et al.
proposed learned branching strategies for solving mixed integer programs with BaB algorithms.
All of these methods imitate the strong branching strategy.
Speciﬁcally, Khalil et al.
(2016) and Hansknecht et al.
(2018) learn a ranking function to rank potential branching decisions while Alvarez et al.
(2017) uses regression to assign a branching score to each potential branching choice.
There are two main issues with these methods.
Firstly, they rely heavily on hand-designed features and secondly, they use generic learning structure which is unable to exploit the neural network architecture.
The most relevant to ours is the recent concurrent work by Gasse et al.
Gasse et al.
(2019) managed to reduce feature reliance by exploiting the bipartite structure of MILP through GNN.
The bipartite graph is capable of capturing the network architecture, but cannot exploit it effectively. Speciﬁcally, it treats all the constraints the same and updates them simultaneously using the same set of parameters.
This limited expressiveness can result in a difﬁculty in learning and hence, high generalization error for NN veriﬁcation problems.
Our proposed framework is speciﬁcally designed for NN veriﬁcation problems.
By exploiting the neural network structure, and designing a customized schedule for embedding updates, our framework is able to scale elegantly both in terms of computation and memory.
Finally, we mention that the recently proposed veriﬁcation methods (Katz et al., 2019; Singh et al., 2018; Anderson et al., 2019) are not explicitly formulated as BaB.
Since our focus is on branching, we mainly use the methods in (Bunel et al., 2019) for comparison.
B ACKGROUND
Formal veriﬁcation of neural networks refers to the problem of proving or disproving a property over a bounded input domain.
Properties are functions of neural network outputs.
When a property can be expressed as a Boolean expression over linear forms, we can modify the neural network in a suitable way so the property can be simpliﬁed to checking the sign of the neural network output (Bunel et al., 2018).
Note that all the properties studied in previous works satisfy this form, thereby allowing us to use the aforementioned simpliﬁcation.
Mathematically, given the modiﬁed neural network f , a bounded input domain C , formal veriﬁcation examines the truthfulness of the following statement:
[Equation]
If the above statement is true, the property holds.
Otherwise, the property does not hold.
3. B RANCH AND B OUND
Veriﬁcation tasks are often treated as a global optimization problem.
We want to ﬁnd the minimum of f ( x ) over C in order to compare with the threshold 0 .
Speciﬁcally, we consider an L layer feedforward neural network, f : R | x | → R , with ReLU activation units such that for any x 0 ∈ C ⊂ R | x | , f ( x 0 )
=
ˆ x L ∈ R where
[Equation]
2b [Equation] The terms W i and b i refer to the weights and biases of the i -th layer of the neural network f. Domain C can be an (cid:96) p norm ball with radius (cid:15) .
Finding the minimum of f is a challenging task, as the optimization problem is generally NP hard (Katz et al., 2017).
To deal with the inherent difﬁculty of the optimization problem itself, BaB (Bunel et al., 2018) is generally adopted.
In detail, BaB based methods divide C into sub-domains (branching), each of which deﬁnes a new subproblem.
They compute a relaxed lower bound of the minimum (bounding) on each sub-problem.
The minimum of lower bounds of all generated sub-domains constitutes a valid global lower bound of the global minimum over C .
As a recursive process, BaB keeps partitioning the sub-domains to tighten the global lower bound.
The process terminates when the computed global lower bound is above zero (property is true) or when an input with a negative output is found (property is false).
A detailed description of the BaB is provided in the appendices.
In what follows, we provide a brief description of the two components, bounding methods and branching strategies, that is necessary for the understanding of our novel learning framework.
3. B OUNDING
For NN veriﬁcation problems, bounding consists of ﬁnding upper and lower bounds for the ﬁnal output, the minimum of f ( x ) over C .
An effective technique to compute a lower bound is to transform the original optimization problem into a linear program (LP) by introducing convex relaxations over ReLU activation units.
As we can see in Eq.
(2b), ReLU activation units do not deﬁne a convex feasible set, and hence, relaxations are needed.
Denote the j -th element of the vector x
i as x i [ j ] .
Possible convex relaxations for a hidden node x i [ j ] that have been introduced so far are shown in
Figure 1.
We replace ReLU with the shaded green area.
The tighter the convex relaxation introduced, the more computational expensive it is to compute a bound but the tighter the bound is going to be.
From Figure 1, we note that in order to introduce a convex relaxation, we need intermediate bounds l i [ j ] and u i [ j ] .
Thus intermediate bounds are required for building the LP for the ﬁnal output lower bound.
Given their purpose and the large number of intermediate bound computations, rough estimations are mainly used.
On the other hand, the ﬁnal output lower bound is directly used in making the pruning decision and hence a tighter lower bound is preferred as it avoids further unnecessary split on the sub-problem.
[Figure]
3. B RANCHING
Branching is of equal importance as bounding in the Branch and Bound framework.
Especially for large scale networks f , each branching step has a large number of putative choices.
In these cases, the effectiveness of a branching strategy directly determines the possibility of verifying properties over these networks within a given time limit.
On neural networks, two types of branching decisions are used: input domain split and hidden activation unit split.
Assume we want to split a parent domain D .
Input domain split selects an input dimension and then makes a cut on the selected dimension while the rest of the dimensions remaining the same as the parent for the two sub-domains.
The common choice is to cut the selected dimension in half and the dimension to cut is decided by the branching strategy used.
Available input domain split strategies are Bunel et al.
(2018) and Royo et al.
Royo et al.
(2019) is based on sensitivity test of the LP on D while Bunel et al.
(2018) uses the formula provided in Wong & Kolter (2018) to estimate ﬁnal output bounds for sub-domains after splitting on each input dimension and selects the dimension that results in the highest output lower bound estimates.
In our setting, we refer to a ReLU activation unit x i [ j ] =
max(ˆ x i [ j ] , 0) as ambiguous over D if the upper bound u i
[ j ] and the lower bound l i [ j ] for ˆ x i [ j ] have different signs.
Activation unit split chooses among ambiguous activation units and then divides the original problem into cases of different activation phase of the chosen activation unit.
If a branching decision is made on x i [ j ] , we divide the ambiguous case into two determinable cases: { x i [ j ] = 0 , l i [ j ] ≤ ˆ x i [ j ] ≤ 0 } and { x i [ j ]
= ˆ x i [ j ] , 0 ≤ ˆ x i [ j ] ≤ u i
[ j ] } .
After the split, the originally introduced convex relaxation is removed, since the above sets are themselves convex.
We expect large improvements on output lower bounds of the newly generated sub-problems if a good branching decision is made.
Apart from random selection, employed in Ehlers (2017) and Katz et al.
(2017), available ReLU split heuristics are Wang et al.
(2018a) and Bunel et al.
Wang et al.
(2018a) computes scores based on gradient information to prioritise ambiguous ReLU nodes.
Similarly, Bunel et al.
(2019) uses scores to rank ReLU nodes but scores are computed with a formula developed on the estimation equations in Wong & Kolter (2018).
We note that for both branching strategies, after the split, intermediate bounds are updated accordingly on each new sub-problem.
For NN veriﬁcation problems, either domain split or ReLU split can be used at each branching step.
When compared with each other,
ReLU split is normally a more effective choice for large scale networks, as shown in Bunel et al.
All the aforementioned existing branching strategies use hand-designed heuristics.
In contrast, we propose a new framework for branching strategies by utilizing a GNN to learn to imitate strong branching heuristics.
This allows us to harness the effectiveness of strong branching strategies while retaining the efﬁciency of GPU computing power.
GNN F RAMEWORK
Overview We begin with a brief overview of our overall framework, followed by a detailed description of each of its components.
A graph neural network G is represented by two components: a set of nodes V and a set of edges E , such that G =
( V, E ) .
Each node and each edge has its set of features.
A GNN uses the graph structure and node and edge features to learn a representation vector (embedding vector) for each node v ∈ V .
GNN is a key component of our framework, in which we treat the neural network f as a graph G f .
A GNN takes G f as an input and initializes an embedding vector for each node in V .
The GNN updates each node’s embedding vector by aggregating its own node features and all its neighbour embedding vectors.
After several rounds of updates, we obtain a learned representation (an updated embedding vector) of each node.
To make a branching decision, we treat the updated embedding vectors as inputs to a score function, which assigns a score for each node that constitutes a potential branching option.
A branching decision is made based on the scores of potential branching decision nodes.
Our framework is visualised in Figure 2.
We now describe each component in detail.
[Figure]
Nodes Given a neural network f , V consists of all input nodes v
[ j ] , all hidden activation nodes v i [ j ] and an output node v L .
In our framework, we combine every pre-activation variable and its associated post-activation variable and treat them as a single node.
Pre- and post-activation nodes together contain the information about the amount of convex relaxation introduced at this particular activation unit, so dealing with the combined node simpliﬁes the learning process.
In terms of the
Eq.
(2), let x (cid:48)
i
[ j ] denote the combined node of ˆ x i [ j ] and x i [ j ] .
The nodes v
[ j ] , v i [ j ] and v L are thus in one-to-one correspondence with x 0[ j ] , x (cid:48)
i
[ j ] and x L .
We note that V is larger than the set of all potential branching decisions as it includes unambiguous activation nodes and output nodes.
Node features Different types of nodes have different sets of features.
In particular, input node features contain the corresponding domain lower and upper bounds and primal solution.
For activation nodes, the node features consist of associated intermediate lower and upper bounds, layer bias, primal and dual solutions and new terms computed using previous features.
Finally, the output node has features including the associated output lower and upper bounds, layer bias and primal solution.
Other types of features could be used and some features could be excluded if they are expensive to compute.
We denote input node features as z
[ j ] , activation node features as z i [ j ] and output node feature as z L .
Our framework uses simple node features and does not rely on extensive feature engineering.
Nonetheless, by relying on the powerful GNN framework, it provides highly accurate branching decisions.
Edges E consists of all edges connecting nodes in V , which are exactly the connecting edges in f .
Edges are characterized by the weight matrices that deﬁne the parameters of the network f such that for an edge e ijk connecting x (cid:48) i
[ j ] and x (cid:48) i +
[ k ] , we assign e ijk = W ijk .
Embeddings We associate a p -dimensional embedding vector µ v for each node v ∈ V .
All embedding vectors are initialised as zero vectors.
Forward and Backward embedding updates In general, a graph neural network learns signals from a graph by acting as a function of two inputs: a feature matrix X ∈ R | V |× p , where each row is the embedding vector µ v for a node v ∈ V , and an adjacency matrix A representing the graph structure.
Under this formulation, all node embedding vectors are updated at the same time and there is no particular order between nodes.
In this work, instead, we propose an update scheme where only the nodes corresponding to the same layer of the network f are updated at the same time, so embedding vector updates are carried out in a layer-by-layer forward-backward way.
We argue that the forward-backward updating scheme is a natural ﬁt for our problem.
In more detail, for a given domain D , each branching decision (an input node or an ambiguous activation node) will generate two sub-domains s 1 and s 2 , with each sub-domain having an output lower bound l Ls 1 , l Ls 2 equal to or higher than the lower bound l L D that of D .
Strong branching heuristic uses a predetermined function to measure the combined improvement of l Ls 1 , l Ls 2 over l L D and makes the ﬁnal branching decision by selecting a node that gives the largest improvement.
Thus, to maximise the performance of a graph neural network, we want a node embedding vector to maximally capture all information related to computing l Ls 1 , l Ls 2 if a split is conducted on the node.
For estimating l Ls 1 , l Ls 2 of splitting on a potential branching decision node v , we note that these values are closely related to two factors.
The ﬁrst factor is the amount of convex relaxations introduced at a branching decision node v , if v corresponds to an ambiguous activation node.
The second factor considers that if the node v is split, the impact it will have on the convex relaxations introduced to nodes on layers after that of v .
Recall that, if there are no ambiguous activation nodes, the neural network f is simply a linear operator, whose minimum value can be easily obtained.
When ambiguous activation nodes are present, the total amount relaxation introduced determines the tightness of the lower bound to f .
We thus treat embedding vectors as a measure of local convex relaxation and its contribution to other nodes’ convex relaxation.
As shown in Figure 1, at each ambiguous activation node x (cid:48) i
[ j ] , the area of convex relaxation introduced is determined by the lower and upper bounds of the pre-activate node ˆ x i [ j ] .
Since intermediate lower and upper bounds of a node ˆ x i [ j ] are signiﬁcantly affected by the layers prior to it and have to be computed in a layer-by-layer fashion, we respect this structure and the order of computation by utilising a forward layer-by-layer update on node embedding vectors, which should allow these embedding vectors to capture the local relaxation information.
In terms of the impact of local relaxation change to that of other nodes, we note that by splitting an ambiguous node into two ﬁxed cases, all intermediate bounds of nodes on later layers will affected, leading to relaxation changes at those nodes.
We thus employ a backward layer-by-layer update to account for the impact the local change has over other nodes.
Theoretically, by ﬁxing an ambiguous ReLU node, intermediate bounds of nodes at previous layers and on the same layer might change as well.
For a naturally trained neural network, these changes for these nodes should be relatively small compared to nodes on the later layers.
To account for these changes, we rely on multiple rounds of forward-and-backward updates.
In summary, during the forward update, for i = 1 , . . .
, L − 1 , we have, for all possible j ,
[Equation] [Equation] [Equation]
[Equation]
During backward update, for i
=
L − 1 , . .
, 1 , we have
[Equation] [Equation]
B inp ( z 0[ j ] , µ
[Equation]
e [Equation] [Equation] Update functions F and B take the form of multi-layered fully-connected networks with ReLU activation functions or composites of these simple update networks.
The terms θ i denote the parameters of the networks.
A detailed description of update functions is provided in the appendices.
We point out that our forward-backward update scheme does not depend on the underlying neural network structure and thus should be generalizable to network architectures that differ from the one we use for training.
However, it does rely on the information used to compute convex relaxation, so underlying data distribution, features and bounding methods are assumed to be same when the trained model is applied to different networks.
Furthermore, our forward-backward update is memory efﬁcient, as we are dealing with one layer at a time and only the updated embedding vectors of the layer are used to update the embedding vectors in the next (forward-pass) and the previous (backward-pass) layer.
This makes it readily applicable to large networks.
Scores At the end of the forward-backward updates, embedding vectors for potential branching decision nodes (all input nodes and ambiguous activation nodes) are gathered and treated as inputs for a score function g s ( · ; θ 5 ) : R p
→ R , which takes the form of a fully-connected network with parameters θ 5 .
It assigns a scalar score for each input embedding vector.
The ﬁnal branching decision is determined by picking the node with the largest score.
P ARAMETER E STIMATION
Training We train a GNN via supervised learning.
To estimate Θ := ( θ 0 , θ 1 , θ 2 , θ 3 , θ 4 , θ 5 ) , we propose a new hinge rank loss function that is speciﬁcally designed for our framework.
Before we give details of the loss, we introduce a relative improvement measure m ﬁrst.
Given a domain D , for each branching decision node v , the two generated sub-problems have output lower bounds l Lsand l Ls 2 .
We measure the relative improvement of splitting at the node v over the output lower bound l L D as follows
[Equation]
Intuitively, m ( 0 ≤ m ≤ 1 ) measures the average relative sub-problem lower bound improvement to the maximum improvement possible, that is − l L D .
Any potential branching decision node v can be compared and ranked via its relative improvement value m v .
Since we are only interested in branching nodes with large improvement measures, ranking loss is a natural choice.
Direct pairwise rank loss might be difﬁcult to learn for NN veriﬁcation problems, given the large number of branching decision nodes on each domain D .
In addition, many branching decisions may give similar performance, so it is redundant and potentially harmful to the learning process if we learn a ranking among these similar nodes.
To deal with these issues, we develop our loss by ﬁrst dividing all potential branching nodes into M classes ( M is much smaller than the total number of branching decision nodes) through the improvement value m v of a node.
We denote the class label as Y v for a node v .
Labels are assigned in an ascending order such that Y v > =
Y v (cid:48) if m v > m v (cid:48) .
We then compute the pairwise hinge-rank loss on these newly assigned labels as
[Equation]
where φ ( z ) =
(1 − z ) + is the hinge function, N is the total number of branching decision nodes and
K is total number of pairs where Y j > Y i for any branching decision nodes v i , v j .
The loss measures the average hinge loss on score difference ( g s ( µ j ; Θ ) − g s ( µ i ; Θ ) ) for all pairs of branching decision nodes v i , v j such that Y j > Y i .
Finally, we evaluate Θ by solving the following optimization problem:
[Equation]
where the loss D i is the one introduced in Eq.
(9) and n is the number of training samples.
Fail-safe Strategy We introduce a fail-safe strategy employed by our framework to ensure that consistent high-quality branching decisions are made throughout a BaB process.
The proposed framework uses a GNN to imitate the behavior of the strong branching heuristic.
Although computationally cheap, in some cases, the output decision by the learned graph neural network might be suboptimal.
When this happens, it could lead to considerably deteriorated performances for two reasons.
Firstly, we observed that for a given domain, which requires multiple splits to reach a conclusion on this domain, if a few low-quality branching decisions are made at the beginning or the middle stage of the branching process, the total number of splits required might increase substantially.
The total BaB path is thus, to some extent, sensitive to the quality of each branching decision apart from those made near the end of the BaB process.
Secondly, once a low-quality decision is made on a given domain, a decision of similar quality is likely to be made on the two newly generated sub-domains, leading to exponential decrease in performance.
Features for newly generated sub-domains are normally similar to those of the parent domain, especially in the cases where the branching decision of the parent domain is made on the later layers and loose intermediate bounds are used.
Thus, it is reasonable to expect the GNN fails again on the sub-domains.
To deal with issue, we keep track of the output lower bound improvement for each branching decision, as introduced in Eq.
We then set a pre-determined threshold parameter.
If the improvement is below the threshold, a computationally cheap heuristic is called to make a branching decision.
Generally, the back-up heuristic is able to give an above-threshold improvement and generate subdomains sufﬁciently different from the parent domain to allow the learned GNN to recover from the next step onwards.
Online Learning Online learning is a strategy to ﬁne-tune the network for a particular property after we have learnt Θ .
It can be seen as an extension of the fail-safe strategy employed.
Every time a heuristic branching decision node v h is used instead of the node v gnn chosen by the GNN, we can use v h and v gnn to update GNN accordingly.
Since a correct GNN model should output embedding vector µ h resulting in a higher score g s ( µ h ; Θ ) for the heuristic decision, a loss can be developed based on the two scores g s ( µ h ; Θ ) and g s ( µ gnn ; Θ ) to generate optimization signals for correcting the GNN behaviour.
For example, the loss used in our experimental setting is:
[Equation]
The last term is used to amplify ( γ > 0 ) the loss if the relative improvement made by the heuristic decision is more than t percent higher than that by the GNN.
We update Θ of GNN by taking one gradient step with small learning rate of the following minimization problem.
[Equation]
Online learning is property speciﬁc: it uses the decisions made by heuristics to ﬁne tune the GNN model so it can best accommodate the property at hand.
As will be shown in our experiments, a small but signiﬁcant improvement in performance is achieved when online learning is used.
E XPERIMENTS
We now validate the effectiveness of our proposed framework through comparative experiments against other available NN veriﬁcation methods.
A comprehensive study of NN veriﬁcation methods has been done in Bunel et al.
We thus design our experiments based on the results presented in Bunel et al.
6. S ETUP
We are interested in verifying properties on large network architectures with convolutional layers.
In Bunel et al.
, existing NN methods are compared on a robustly trained convolutional network on MNIST.
We adopt a similar network structure but using a more challenging dataset, namely
CIFAR-10, for an increased difﬁculty level.
We compare against the following two methods: (i)
MIPplanet, a mixed integer solver backed by the commercial solver Gurobi; and (ii) BaBSR, a BaB based method utilising a ReLU-split heuristic.
Our choice is motivated by their superior performance over other methods for MNIST veriﬁcation problems in the previous work (Bunel et al., 2019).
We provide the detailed experimental setup through four perspectives: bounding method, branching strategy, network structure, veriﬁcation properties tested.
( Bounding methods )
We compute intermediate bounds using linear bounds relaxation (Figure 1(b)).
For the output lower bound, we use
Planet relaxation (Figure 1(c)) and solve the corresponding LP with Gurobi.
For the output upper bound, we compute it by directly evaluating the network value at the input provided by the LP solution.
( Branching strategy )
We focus on ReLU split only in our experiments.
As shown in Bunel et al.
(2019), domain split only outperforms ReLU split on low input dimensional and small scale networks.
Also, since one of the Baseline method BaBSR employs a ReLU-split heuristic, we consider ReLU split only for a fair comparison.
However, we emphasize that our framework is readily applicable to work with a combined domain and ReLU split strategy.
( Network structures )
Three neural network structures will be studied.
The base one is of the similar structure and size to the one used in Bunel et al.
It has two convolutional layers, followed by two fully connected layers and is trained robustly using the method provided in Wong & Kolter (2018).
This particular choice of network size is made because the time required for solving each LP increases substantially with the size of the network.
To best evaluate the performance of the branching strategy, we have to work with a medium sized network so that within the given timeout, a sufﬁcient amount of branching decisions can be made to allow effective comparisons.
When testing the transferability of framework, two larger networks will be tested but their sizes are still restricted by the LP bottleneck.
Detailed network architecture is provided in the appendices.
( Veriﬁcation properties )
Finally, we consider the following veriﬁcation properties.
For a image x , that the model correctly predicted the label y c , we randomly choose a label y c (cid:48) such that for a given (cid:15) , we want to prove
[Equation]
where f (cid:48) is the original neural network, e ( c ) and e ( c (cid:48) ) are one-hot encoding vectors for labels y c and y c (cid:48) .
We want to verify that for a given (cid:15) , the trained network will not make a mistake by labelling the image as y c (cid:48) .
Since BaBSR is claimed to be the best performing method on convolutional networks, we use it to determine the (cid:15) values, which govern the difﬁculty level of veriﬁcation properties.
Small (cid:15) values mean that most ReLU activation units are ﬁxed so their associated veriﬁcation properties are easy to prove while large (cid:15) values could lead to easy detection of counter-examples.
The most challenging (cid:15) values are those at which a large number of activation units are ambiguous.
We use binary search with BaBSR method to ﬁnd suitable (cid:15) values.
We only consider (cid:15) values that result in true properties and timed out properties.
Binary search process is simpliﬁed by our choice of robustly trained models.
Since these models are trained to be robust over a δ ball, the predetermined value δ can be used as a starting value for binary search.
6. T RAINING D ATASET
In order to generate training data, we ﬁrstly pick 565 random images and for each image, we randomly select an incorrect class.
For each property, the (cid:15) value is determined by running binary search with BaBSR and 800 seconds timeout, so the ﬁnal set of properties consists of mainly easily solvable properties and a limited number of timed out properties.
We collect training data along a BaB process for solving a veriﬁcation property.
At each given domain, given the large number of potential branching decisions, we perform the strong branching heuristic on a selected subset of all potential branching decisions.
The subset consists of branching decisions that are estimated to be of high quality by the BaBSR heuristic and randomly selected ones, which ensure a minimum 5% coverage on each layer.
To construct a training dataset that is representative enough of the whole problem space, we need to cover a large number of properties.
In addition, within a BaB framework, it is important to include training data at different stages of a BaB process.
However, running a complete BaB process with the strong branching heuristic for hundreds of properties is computationally expensive and considerably time consuming.
We thus propose the following procedure for generating a training dataset to guarantee a wide coverage both in terms of the veriﬁcation properties and BaB stages.
For generated veriﬁcation properties, we randomly select 25% of non-timeout property to conduct a complete BaB process with the strong branching heuristic.
For the rest of the properties, we try to generate at least B = 20 training data for each veriﬁcation property.
Given the maximum number of branches q = 10 and an effective and computationally cheap heuristic, we ﬁrst generate a random integer k from [0 , q ] .
Then, we run a BaB process with the selected cheap heuristic for k steps.
Finally, we call the strong branching heuristic to generate a training sample.
We repeat the process until B training samples are generated or the BaB process terminated.
A detailed algorithm is provided in the appendices.
6. B ASE MODEL
We test our learned model on the same model structure but on properties at three different difﬁculty levels.
Testing veriﬁcation properties are generated by binary search with BaBSR and 3600s timeout.
We categorise veriﬁcation properties solved within 800s as easy, which is consistent with training data generated, between 800s and 2400s as medium and more than 2400s as hard.
In total, we generated 467 easy properties, 773 medium properties and 426 hard properties.
Results are given in the Table 1.
Methods are compared in three perspectives: the average time over all properties, average number of branches required over the properties that are solved by all methods (we exclude timed out properties) and also the ratio of timed out properties.
Since the properties are generated based on BaBSR, the timed out ratios of BaBSR on easy and medium properties are not comparable with that of other methods.
All other numbers should give a fair evaluation of the effectiveness of our branching strategy.
BaBSR, GNN and GNN-online only differ in the branching strategy used.
On all three sets of properties, we see that our learned branching strategy has led to a more than 50% reduction in the total average number of branches required for a property.
As a direct result, the average time required achieves at least a 50% reduction as well.
Our framework is thus an effective scheme and enjoys horizontal transferability.
A further performance improvement is obtained through instance-speciﬁc online learning.
Among all 1666 tested veriﬁcation properties,
GNN with online-learning solves 61 .
52% of properties with fewer number of branches and 60 .
20% of properties in less time when compared to standard GNN.
[Table] [Table]
We also provide a time cactus plot (Figure 3a) for all properties on the Base model.
Time cactus plots for each category of properties can be found in the appendices.
All these time cactus plots look similar.
Although BaBSR performs better than the commercial solver encoded method MIPplanet overall, MIPplanet wins on a subset of properties.
The learned model GNN, however, is capable of giving consistent high quality performance over all properties tested.
6. T RANSFERABILITY : LARGER MODELS
We also robustly trained two larger networks.
One has the same layer structure as the Base model but more hidden units on each layer, which we refer to as Wide model.
The other has a similar number of hidden units on each layer but more layers, which we refer to as Deep model.
The detailed network architecture is provided in the appendices.
Apart from the network structure, everything else is kept the same as for the Base model experiments.
We use BaBSR and timeout of 7200s to generateproperties for the Wide model and 250 properties for the Deep model.
For these two models, each
LP called for solving a sub-problem output lower bound is much more time consuming, especially for the Deep model.
This is reason that the average number of branches considered is much fewer that those of the Base model within the given time limit.
The model learned on the Base network is tested on veriﬁcation properties of large networks.
Experimental results are given in the Table 2 and time cactus plots (Figures 3b, 3c) are also provided.
All results are similar to what we observed on the Base model, which show that our framework enjoys vertical transferability.
[Figure]
r o p e r t i e s v e r
i f i e d
[Figure]
% o f p
r
o p e r t i e s v e r
i f i e d
[Figure]
D ISCUSSION
The key observation of our work is that the neural network we wish to verify can be used to design
a
GNN to improve branching strategies.
This observation can be used in enhancing the performances of other aspects of BaB. Possible future works include employing GNN to ﬁnd fast-converging starting values for solving LPs on a neural network and utilising GNN to develop a lazy veriﬁer, that only solves the corresponding LP on a domain when it could lead to pruning.
A PPENDIX A. B RANCH AND B OUND A LGORITHM
The following generic Branch and Bound Algorithm is provided in Bunel et al.
Given a neural network net and a veriﬁcation property problem we wish to verify, the BaB procedure examines the truthfulness of the property through an iterative procedure.
During each step of BaB, we ﬁrst use the pick out function (line 6 ) to choose a problem prob to branch on.
The split function (line 7 ) determines the branching strategy and splits the chosen problem prob into sub-problems.
We compute output upper and lower bounds on each sub-problem with functions compute UB and compute LB respectively.
Newly computed output upper bounds are used to tighten the global upper bound, which allows more sub-problems to be pruned.
We prune a sub-problem if its output lower bound is greater than or equal to the global upper bound, so the smaller the global upper bound the better it is.
Newly calculated output lower bounds are used to tighten the global lower bound, which is deﬁned as the minimum of the output lower bounds of all remained sub-problems after pruning.
We consider the BaB procedure converges when the difference between the global upper bound and the global lower bound is smaller than (cid:15) .
In our case, our interested veriﬁcation problem Eq.
(1) is a satisﬁability problem.
We thus can simplify the BaB procedure by initialising the global upper bound global ub as 0 .
As a result, we prune all sub-problems whose output lower bounds are above 0 .
In addition, the BaB procedure is terminated early when a below 0 output upper bound of a sub-problem is obtained, which means a counterexample exits.
1: function B A B( net , problem , (cid:15) ) 2: global lb ← compute LB ( net , problem ) 3: global ub ← compute UB ( net , problem ) 4: probs ← [( global lb , problem )] 5: while global ub − global lb > (cid:15) do 6: ( , prob ) ← pick out ( probs ) 7: [ subprob 1 , . . .
, subprob s ] ← split ( prob ) 8: for i = 1 . . .
s do 9: sub lb ← compute LB ( net , subprob i ) 10: sub ub ← compute UB ( net , subprob i ) 11: if sub ub < global ub then 12: global ub ← sub ub 13: prune probs ( probs , global ub ) 14: end if 15: if sub lb < global ub then 16: probs .
append (( sub lb , subprob i ))
17: end if 18: end for 19: global lb ← min { lb | ( lb , prob ) ∈ probs } 20: end while 21: return global ub 22: end function
A PPENDIX B. I MPLEMENTATION OF FORWARD AND BACKWARD PASSES
We give implementation details of forward and backward updates for embedding vectors for the model used in the experiments section.
Choices of forward and backward update functions are based on the bounding methods used.
In our experiments, we used linear bound relaxations for computing intermediate bounds and Planet relaxation for computing the ﬁnal output lower bound.
We start with a graph neural network mimicking the structure of the network we want to verify.
We denote domain lower and upper bounds as l 0 and u 0 respectively.
Similarly, we denote the intermediate bounds (pre-activation) for layers i = 1 , . . .
, L − 1 as l i and u i .
Since an LP solver is called for the ﬁnal output lower bound, we have primal values for all nodes of V and dual values for all ambiguous nodes of V .
Finally, let W 1 , . .
, W L be the layer weights and b 1 , . . .
, b L be the layer biases of the network f , which we wish to verify.
B.1 F ORWARD P ASS
Unless otherwise stated, all functions F ∗ are 2-layer fully connected network with ReLU activation units.
B.1.1 I NPUT NODES
We update the embedding vectors of input nodes only during the ﬁrst round of forward pass.
That is we update µ
[ j ] when it is zero for all j .
After that, input nodes embedding vectors are updated only in backward pass.
For each input node, we form the feature vector z
[ j ] as a vector of l 0[ j ] , u
[ j ] and its associated primal solution.
The input node embedding vectors are computed as
[Equation] [Equation]
B.1.2 A CTIVATION NODES
The update function F act can be broken down into three parts: 1) compute information from local features 2) compute information from neighbour embedding vectors and 3) combine information from 1) and 2) to update current layer’s embedding vectors.
Information from local features Since we compute the ﬁnal lower bound with the Planet relaxation (Figure 1(c)), we introduce a new feature related to the relaxation: the intercept of the relaxation triangle, shown in Figure 4.
We denote an intercept as β and compute it as
[Equation]
The intercept of a relaxation triangle can be used as a measure of the amount of relaxation introduced at the current ambiguous node.
Therefore, the local feature vector z i
[ j ] of an ambiguous node x (cid:48)
i
[ j ] consists of l i [ j ] , u i [ j ] , β i
[ j ] , its associated layer bias value, primal values (one for pre-activation variable and one for post-activation variable) and dual values.
We obtain information from local features via
[Equation]
[Equation]
where R i [ j ] ∈ R p .
Information from neighbour embedding vectors During the forward pass, we focus on embedding vectors of the previous layer only.
To update an embedding vector on layer i , we ﬁrst combine embedding vectors of the previous layer with edge weights via
[Equation]
To compute the information from neighbour embedding vectors to an arbitrary activation node x (cid:48) i
[ j ] , we consider each activation unit as a gate .
We observe that the amount of the information from [Equation]
x [Equation] [Equation] [Figure]
[Equation]
[Figure] [Equation]
[Figure]
neighbour embedding vectors that remains after passing through a gate is dependent on the its lower bound l i [ j ] and upper bound u i [ j ] .
When l i [ j ] and u i [ j ] are of different signs, x (cid:48)
i
[ j ] is an ambiguous node. With relaxation, for any input value between l i [ j ] and u i [ j ] , the maximum output achievable after passing an activation unit is shown by the red slope in Figure 5(a).
The red slope s i
[ j ] is computed as
[Equation]
Thus, the amount of information from neighbour embedding vectors that remains after passing through an ambiguous gate is related to the ratio α := u i [ j ] u i [ j ] − l i [ j ] .
When u i [ j ] is no greater than zero, the activation node x (cid:48)
i
[ j ] completely blocks all information.
For any input value, the output value is zero after passing the activation unit, as shown by the red line in Figure 5(b).
We have α = 0 in this case.
Finally, when l i [ j ] is no less than 0 , the activation node x (cid:48) i
[ j ] allows a complete passing of information and α = 1 .
It is shown by the red line in Figure 5(c).
We incorporate these observations into our evaluations and compute the information from neighbour embedding vectors as
[Equation]
where α (cid:48)
= 1 − α when 0 < α <
and α (cid:48)
=
α otherwise.
Here, we use [ a , b ] to denote the concatenation of two vectors a , b ∈ R p into a vector R 2 p .
We introduce α (cid:48) to be more informative.
We do not consider the information that relate to the intercept β
i
[ j ] in the ambiguous case for the sake of simplicity.
Improved performance could be expected if the β i [ j ] related information is incorporated as well.
Combine previous information Finally, we combine the information from local features and the information from neighbour embedding vectors to update the embedding vectors of activation nodes.
Speciﬁcally,
[Equation]
B.1.3 O UTPUT NODE
Embedding vectors of output nodes are updated in a similar fashion to that of activation nodes.
We ﬁrst compute information from local features.
[Equation]
For output nodes, the vector of local features z L consists of output lower bound, output upper bound, primal solution and layer bias.
F out − lf is a one-layer fully-connected network with ReLU activation units.
We then compute information from neighbour embedding vectors.
Since the output node does not have an activation unit associated with it, we directly compute the information of neighbour embedding vectors as
[Equation]
Finally, we update the embedding vector of the output node as
[Equation]
During backward message passing, for i =
L − 1 , . .
, 1 , we update embedding vectors for activation nodes and input node.
Again, all functions B ∗ are 2-layer fully-connected networks unless speciﬁed otherwise.
B.2.1 A CTIVATION NODES
Similar to updates of embedding vectors carried out for activation nodes in a forward pass, we update embedding vectors of activation nodes using the same three steps in the backward pass, but with minor modiﬁcations.
Information from local features We use the same feature
z i [ j ] as the one used in the forward pass and compute the information from local features as
[Equation]
[Equation] We recall that a dual value indicates how the ﬁnal objective function is affected if its associated constraint is relaxed by a unit. To better measure the importance of each relaxation to the ﬁnal objective function, we further update the information from local features by
[Equation]
otherwise
[Equation]
[Equation]
Here, d i [ j ] is the vector of dual values corresponding to the activation node x (cid:48)
i
[ j ] .
We use (cid:12) to mean that we multiply R bi [ j ] by each element value of d i [ j ] and concatenate them as a singe vector.
Information from neighbour embedding vectors During the backward pass, we focus on embedding vectors of the next layer only.
In order to update an embedding vector on layer i , we compute the neighbourhood embedding vectors as [Equation]
[Equation]
We point out that there might be an issue with computing E i
[ j ] if the layer i + 1 is a convolutional layer in the backward pass.
For a convolutional layer, depending on the padding number, stride number and dilation number, each node x (cid:48)
i
[ j ] may connect to a different number of nodes on the layer i + 1 .
Thus, to obtain a consistent measure of E i [ j ] , we divide E i [ j ] by the number of connecting node on the layer i + 1 , denoted as E b (cid:48)
i
[ j ] and use the averaged E b (cid:48) i
[ j ] instead.
Let
[Equation]
i [ j ] if layer i+1 convolutional,
E b i [ j ] otherwise.
The following steps are the same as the forward pass.
We ﬁrst evaluate
[Equation]
28) and the update embedding vectors as
[Equation]
B.2.2 I NPUT
[Equation]
Finally, we update the input nodes.
We use the feature vector z b 0 , which consists of domain upper bound and domain lower bound.
Information from local features is evaluated as
[Equation]
We compute the information from neighbour embedding vectors in the same manner as we do for activation nodes in the backward pass, shown in Eq (27).
Denote the computed information as E b ∗
[ j ] .
The embedding vectors of input nodes are updated by
[Equation]
A PPENDIX C. A LGORITHM FOR GENERATING T RAINING D ATASET
Algorithm 2 outlines the procedure for generating the training dataset.
The algorithm ensures the generated training date have a wide coverage both in terms of the veriﬁcation properties and BaB stages while at the same time is computationally efﬁcient.
Speciﬁcally, we randomly pick 25% of all properties that do not time out and run a complete BaB procedure on each of them with the strong branching heuristic to generate training samples (line 3 - 5 ).
For the remaining properties, we attempt to generate B training samples for each of them.
To cover different stages of a BaB process of a property, we use a computationally cheap heuristic together with the strong branching heuristic.
Given a property, we ﬁrst use the cheap heuristic for k steps (line 10 - 15 ) to reach a new stage of the
BaB procedure and then call the strong branching heuristic to generate a training sample (line 16 ).
We repeat the process until B training samples are generated or the BaB processs terminates.
p
P 3: α ←− random number from 4: if p is a timed out property and α ≤ 0 .
25 then 5:
Running a complete BaB process with the Strong Branching Heuristic 6: else 7: b =8: while b ≤ B do 9: k ←− random integer from [0 , q ] 10: while k > 0 do 11:
Call a computationally cheap heuristic 12: if terminates then return 13: end if 14: k = k 15: end while 16:
Call and generate a sample 17: if terminates then return 18: end if 19: b = b
20: end while 21: end if 22: end for
A PPENDIX D. E XPERIMENT D ETAILS
All the hyper-parameters used in the experiments are determined by testing a small set of numbers over the validation set.
Due to the limited number of tests, we believe better sets of hyper-parameters could be found.
D.1 T RAINING D ETAILS
Training dataset To generate a training dataset, 565 random images are selected.
Binary serach with BaBSR and 800 seconds timeout are used to determine (cid:15) on the Base model.
Amongveriﬁcation properties determined, we use 430 properties to generate 17958 training samples and the rest of properties to generate 5923 validation samples.
Training samples and validation samples are generated using Algorithm 2 with B = 20 and q = 10 .
For a typical epsilon value, each sub-domain generally contains 1300 ambiguous ReLU nodes.
Among them, approximately 140 ReLU nodes are chosen for strong branching heuristics, which leads to roughly 200 seconds for generating a training sample.
We point out that the total amount of time required for generating a training sample equals the 2*(per LP solve time)*(number of ambiguous ReLU nodes chosen).
Although both the second and the third terms increase with the size of the model used for generating training dataset, the vertical transferability of our GNN enables us to efﬁciently generate training dataset by working with a small substitute of the model we are interested in.
In our case, we trained on the Base model and generalised to Wide and Deep model.
Training We initialise a GNN by assigning each node a 64 -dimensional zero embedding vector.
GNN updates embedding vectors through two rounds of forward and backward updates.
To train the GNN, we use hinge rank loss (Eq. (9)) with M = 10 .
Parameters Θ are computed and updated through Adam optimizer with weight decay rate λ = 1 e − 4 and learning rate 1 e − 4 .
If the validation loss does not decrease for 10 consecutive epochs, we decrease the learning rate by a factor of 5 .
If the validation loss does not decrease for 20 consecutive epochs, we terminate the learning procedure.
The batch size is set to 2 .
In our experiments, each training epoch took less than 400 seconds and the GNN converges within 60 epochs.
In terms of the training accuracy, we ﬁrst evaluate each branching decision using the metric deﬁned by Eq.
Since there are several branching choices that give similar performances at each subdomain, we considered all branching choices that have m v above 0.9 as correct decisions.
Under this assumption, our trained GNN achieves 85 . 8% accuracy on the training dataset and 83 . 1% accuracy on the validation dataset. D.2 V ERIFICATION E XPERIMENT D ETAILS
We ran all veriﬁcation experiments in parallel on 16 CPU cores, with one property being veriﬁed on one CPU core.
We observed that although we speciﬁcally set the thread number to be one for MIPplanet (backed by the commercial solver Gurobi), the time required for solving a property depends on the total number of CPUs used.
For a machine with 20 cpu cores, MIPplanet requires much less time on average for proving the same set of properties on fewer (say 4) CPU cores in parallel than on many (say 16) CPU cores in parallel (the rest of CPU cores remain idle).
Since BaBSR, GNN and GNN-online all use Gurobi for the bounding problems, similar time variation, depending on the number of CPU cores used, are observed.
We ran each method in the same setting and on 16 CPUs in parallel, so our reported results and time are comparable.
However, we remind readers to take time variation into consideration when replicating our experiments or using our results for comparison.
Fail-safe strategy Since, to the best of our knowledge, the branching heurisitc of BaBSR is the best performing one on convolutional neural networks so far, we choose it for our fail-safe strategy.
The threshold is set to be 0 .
Every time when the relative improvement m gnn of a GNN branching decision v gnn is less than 0 .
2 , we call the heuristic to make a new branching decision v h . We solve
8) performs the best.
The node with higher relative improvement is chosen to be the ﬁnal branching decision.
Online learning We take a conservative approach in terms of online learning.
We refer to a GNN decision as a failed decision if the relative improvement offered by heuristic branching is better than the one offered by the GNN.
We record all GNN failed decisions and only update the GNN model online when the same failed decision is made at least twice.
To update the GNN model, we use
Adam optimizer with weight decay rate λ = 1 e − 4 and learning rate 1 e − 4 .
The GNN model is updated with one gradient step only with respect to the optimization problem Eq.
(12), where γ =and t = 0 .
1 in the loss function loss online , deﬁned in Eq.
D.3 B ASELINES
We decided our baselines based on the experiment results of Bunel et al.
In Bunel et al.
(2019), methods including MIPplanet, BaBSR, planet (Ehlers, 2017), reluBaB and reluplex (Katz et al., 2017) are compared on a small convolutional MNIST network.
Among them, BaBSR and MIPplanet signiﬁcantly outperform other methods.
We thus evaluate our methods against these two methods only in the experiments section.
In order to strengthen our baseline, we compare against two additional methods here. Neurify (Wang et al., 2018a) Similar to BaBSR, Neurify splits on ReLU activation nodes. It makes a branching decision by computing gradient scores to prioritise ReLU nodes.
Since the updated version of Neurify’s released code supports veriﬁcation, we conducted a comparison experiment between between Neurify and BaBSR for inclusiveness. Neurify does not support CIFAR dataset. To evaluate the performance of Neurify, we obtained the trained ROBUST MNIST model and corresponding veriﬁcation properties from Bunel et al.
We ranked all veriﬁcation properties in terms of the BaBSR solving time and selected the ﬁrst 200 properties, which are solved by BaBSR within one minute, as our test properties. For a fair comparison, we have restricted Neurify to use one CPU core only and set the timeout limit to be two minutes. Among all test properties, Neurify timed out on 183 out of 200 properties.
BaBSR thus outperforms Neurify signiﬁcantly.
Combining with the results of Bunel et al. (2019), BaBSR is indeed a fairly strong baseline to be compared against. MIP based algorithm (Tjeng et al., 2019) We also compared our MIPplanet baseline against a new MIP based algorithm (Tjeng et al., 2019), published in ICLR 2019.
To test these two methods, we randomly selected 100 veriﬁcation properties from the CIFAR Base experiment with timeout 3600s.
In terms of solving time, MIPplanet requires 1732.18 seconds on average while the new MIP algorithm requires 2736.60 seconds.
Speciﬁcally, MIPplanet outperforms the new MIP algorithm on 78 out of 100 properties.
MIPplanet is therefore a strong baseline for comparison.
As a caveat, we mention that the main difference between MIPplanet and the algorithm of (Tjeng et al., 2019) is the intermediate bound computation, which is complementary to our focus.
If better intermediate bounds are shown to help veriﬁcation, we can still use our approach to get better branching decisions corresponding to those bounds.
D.4 M ODEL A RCHITECTURE
We provide the architecture detail of the neural networks veriﬁed in the experiments in the following table. [Table]
A PPENDIX E. A DDITIONAL E XPERIMENT R ESULTS E.1 F AIL - SAFE HEURISTIC DEPENDENCE
In all our experiments
[Table]
we have compared against BaBSR, which employs only the fail-safe heuristic for branching.
In other words, removing the GNN and using only the fail-safe heuristic is equivalent to BaBSR.
The fact that GNN signiﬁcantly outperforms BaBSR demonstrates that GNN is doing most of the job.
To better evaluate the GNN’s reliance on a fail-safe heuristic, we study the ratio of times that a GNN branching decision is used for each veriﬁcation property of a given model.
Results are listed in Table 4.
On all three models, GNN accounts for more than 90% of branching decisions employed on average, ensuring the effectiveness of our GNN framework.
Evaluating GNN’s dependence on the fail-safe strategy.
Given a CIFAR model, we collected the percentage of times GNN branching decision is used and the percentage of times the fail-safe heuristic (BaBSR in our case) is employed for each veriﬁcation property.
We report the average ratio of all veriﬁcation properties of the same model.
To account for extreme cases, we also list the minimum and maximum usage ratios of the fail-safe heuristic for each model.
[Table]
E.2 GNN FEATURE ANALYSIS
We evaluate the importance of different features used in GNN.
We note that two types of features are used in GNN.
The ﬁrst type (including intermediates bounds, network weights and biases) can be collected at negligible costs.
The other type is LP features (primal and dual values) that are acquired by solving a strong LP relaxation, which are expensive to compute but potentially highly informative.
To evaluate their effect, we trained a new GNN with LP features removed and tested the new GNN on 260 randomly selected veriﬁcation properties on the Base model.
Among the selected properties, 140 are categorised as easy, 70 as medium and 50 as hard.
We denote the model trained on all features as GNN and the newly trained model as GNN-R (we use R to indicate reduced features).
[Table]
From Table 5, we observe that removing primal and dual information deteriorates the GNN performance, but GNN-R still outperforms the baseline heuristic BaBSR.
We believe cheap features are the most important.
Depending on the cost of LP, potential users can either remove expensive LP features or train a GNN with a smaller architecture.
E.3 MIP PLANET B RANCHING N UMBER
MIPplanet is implemented with the commercial solver Gurobi.
Since Gurobi outputs internal branch number, we recorded MIPplanet branch number for a subset of veriﬁcation properties for each model.
In detail, we randomly selected 120 properties of various difﬁculty levels for the Base model and 27 properties each for the Wide and Deep model.
Results are summarised in Table 6.
One key observation we made is that Gurobi branch number is not positively related to the solving time.
For instance, on timed out properties of the Wide model, MIPplanet branch number varies between 1 and 7479.
We suspect Gurobi performs cutting before branching, so time spent on branching varies between properties, leading to inconsistent branch number and solving time.
As the result, the MIPplanet branch number is not comparable with that of BaBSR, GNN and GNN-online.
This is also the reason that we did not include MIPplant branch number in Table 1 and Table
[Table]
[Table]
E.4 LP SOLVING TIME AND GNN COMPUTING TIME
We mention that LP solving time is the main bottleneck for branch-and-bound based veriﬁcation methods.
Although both GNN evaluation time and LP solving time increase with the size of network,
LP solving time grows at a signiﬁcantly faster speed.
For instance, in CIFAR experiments, GNN requires on average 0.02, 0.03, 0.08 seconds to make a branching decision on Base, Wide and Deep model respectively but the corresponding one LP solving time on average are roughly 1.1, 4.9, 9. seconds.
GNN evaluation is almost negligible for large neural networks when compared to LP solving time.
E.5 G EOMETRIC M EAN
For all our experiments
[Table]
we based our analyses on the statistics of average solving time and branching number.
To ensure the reported numbers are not biased by potential outliers, we measure methods’ performance with geometric mean as well and summarize results in Table 7 and Table 8.
Statistics of geometric mean are consistent with that of arithmetic mean, validating the analyses of the main paper.
[Table] [Table] E.6 A DDITIONAL P LOTS
We provide cactus plots for the Base model on easy, medium and hard difﬁculty level properties respectively.
[Figure]
A PPENDIX F. MNIST DATASET
We replicate the CIFAR experiments on the MNIST dataset to test the generalization ability of our
GNN framework.
F.1 M ODEL A RCHITECTURE AND V ERIFICATION P ROPERTIES
We trained three different networks on MNIST with the method provided in (Wong & Kolter, 2018).
The base model is mainly used for generating the training dataset and testing the horizontal generalization ability of the trained GNN.
The Wide and Deep models are used for evaluating the vertical generalization.
Veriﬁcation properties are found via binary search with BaBSR.
We set the binary
[Table]
F.2 T RAINING D ETAILS
Training dataset Training dataset is generated on the Base model.
We point out that we explicitly choose a Base model of small network size for efﬁcient and fast training data generation.
To generate a training dataset, 538 random MNIST images are selected.
Binary serach with BaBSR and 600 seconds timeout are used to determine (cid:15) on the Base model.
Among 538 veriﬁcation properties determined, we use 403 properties to generate 18231 training samples and the rest of properties to generate 5921 validation samples.
Training samples and validation samples are generated using
Algorithm 2 with B = 20 and q = 10 .
For a typical epsilon value, each sub-domain generally contains 480 ambiguous ReLU nodes.
Among them, approximately 80 ReLU nodes are chosen for strong branching heuristics, which leads to roughly 45 seconds for generating a training sample.
Training
The same set of parameters and training procedure are used for training a GNN for
MNIST dataset.
The GNN converges in 70 epochs with each epoch took less than 400 seconds.
The trained GNN reached 86 .
5% accuracy on the training dataset and 83 .
1% accuracy on the validation dataset.
F.3 E XPERIMENT R ESULTS
We ﬁrst note that we use veriﬁcation properties with timeout 1800 seconds on the Base model to allow for an integrated evaluation of GNN on both its performance and its horizontal transferability. Vertical transferability is tested on the Wide and Deep model. We observe that MIPplanet outperforms all BaB based methods on veriﬁcation properties of the Base model. Given that the network size of the Base model is particularly small (1226 hidden units only), we believe that MIP algorithms backed by commercial solvers could be the most effective tool on veriﬁcation problems of small size.
Our conjecture is further conﬁrmed by the fact that MIPplanet timed out on almost all properties of both the Wide and Deep models.
On all three models, GNN consistently outperforms BaBSR, demonstrating the transferability of our framework. Finally, when online learning is considered, we found it is effective in ﬁne-tuning the trained GNN and enabling further performance improvements, especially on the Wide model. [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
F.4 F AIL - SAFE HEURISTIC DEPENDENCE
Results of Table 11 ensure that the trained GNN is indeed account for the most branching decisions
[Table]
F.5 G EOMETRIC MEAN
The consistency between the results of Table 12 and Table 10 conﬁrm that our analyses based on arithmetic mean are not biased by outliers. [Table]