REGLO: Provable Neural Network Repair for Global Robustness Properties
Abstract
We present REGLO , a novel methodology for repairing neural networks to satisfy global robustness properties.
In contrast to existing works that focus on local robustness, i.e., robustness of individual inputs, REGLO tackles global robustness, a strictly stronger notion that requires robustness for all inputs within a region.
Leveraging an observation that any counterexample to a global robustness property must exhibit a corresponding large gradient, REGLO first identifies violating regions where the counterexamples reside, then uses verified robustness bounds on these regions to formulate a robust optimization problem to compute a minimal weight change in the network that will provably repair the violations.
Experimental results demonstrate the effectiveness of REGLO across a set of benchmarks.
Introduction
Motivated by the fragility of deep neural networks (DNNs) to small input perturbations known as adversarial examples [1], there is a large, growing body of research on measuring, verifying, and improving the robustness of DNNs against those perturbations [2, 3, 4, 5, 6, 7, 8, 9, 10].
Various notions of robustness have been considered [11, 12, 13, 14, 15] and can be largely categorized into two groups: local robustness and global robustness .
Local robustness is about the robustness of individual input points.
Intuitively, it means that for an input x , a small change of the input (e.g., any p -norm-bounded ∆ x ) would not result in a significant change in the output (e.g., change of a classification result).
On the other hand, global robustness stipulates robustness on all points within a given input region X .
Global robustness is strictly stronger than local robustness, and has the advantage of enforcing robustness on unseen inputs within the given input region.
Techniques developed for verifying local robustness [16, 17, 18, 10] are often leveraged as a subroutine when verifying global robustness and probabilistic notion of robustness.
For instance, Weng et al.
[18] derives a probabilistic certificate based on the worst-case certificate computed for local robustness.
A standard way to compute global robustness bounds is to first construct a twin-network, i.e., two copies of the original network side by side, where one input represents x and the other input represents its adversarial perturbation ∆ x , and then apply techniques for computing local robustness bounds to the twin-network for any x ∈ X and ∆ x within some given perturbation set [14, 19].
Global robustness is known to be harder to verify than local robustness, due to encoding two copies of the network and a larger input domain [14].
Recently, Wang et al.
[20, 21] improved the efficiency of global robustness verification by exploiting the interleaving dependencies in the twin-network encoding.
For ReLU DNNs, it is also possible to exploit their piecewise linearity and enumerate the activation patterns to search for counterexamples to a global robustness specification [22].
Recognizing the importance of global robustness, various methods have also been proposed to train networks with improved global robustness.
Leino et al.
[23] present a method for training and constructing globally-robust classification networks with an additional output class that labels inputs as “non-locally-robust”.
Chen et al.
[13] use a counterexample-guided framework to train classifiers that satisfy global robustness properties.
In addition, the notion of individual fairness (IF), which requires two inputs that differ only on some sensitive features to have similar outputs, can be viewed as a global robustness property [24].
Benussi et al.
[25] presents an MILP formulation whose solution can be used to verify IF properties and guide the training process by modifying the training loss.
Existing works also leverage adversarial-training schemes by using a discriminator to force the classifier to be unbiased towards the sensitive features [26, 27, 28].
In general, verification-in-theloop training approaches can be prohibitively expensive given the high cost of global robustness verification.
Moreover, training-based methods cannot guarantee the satisfaction of global robustness properties.
In this paper, we consider the problem of repairing a trained DNN to satisfy a given global robustness property.
Repair and verification are two sides of the same coin – if a problem warrants (formal) verification, then any bug discovered during verification should necessitate fixing.
Existing DNN repair methods mainly consist of weight modification [29, 30, 31], either via constraint solving or fine-tuning, and DNN architecture extension [32, 33, 34].
Compared with training-based approaches, repair can be applied only once as a post-hoc modification and does not require access to the training data.
A repair method is considered sound if it can guarantee the removal of the discovered violations or the satisfaction of a given property.
Our contributions: We propose REGLO , the first DNN repair technique with provable guarantees on satisfying global robustness properties .
The key idea of REGLO is to leverage an observation that any counterexample of a global robustness property would have a large gradient that indicates the violation, and use verified robustness bounds on the corresponding violating region to formulate a robust optimization problem to compute a minimal weight change in the last hidden layer of the network to fix the violation.
For piecewise linear DNNs, our approach is both sound and complete – the resulting network is guaranteed to satisfy the given global robustness property, and a repair is guaranteed to be found.
We detail our approach below, starting with the technical preliminaries.
Background 2.1 Deep Neural Networks (DNNs)
An R -layer feed-forward DNN f : X → Y is a composition of linear functions and activation function σ , where X ⊆
R m is a bounded input domain and
Y ⊆ R n is the output domain.
The weights and biases of the linear function are parameters of the DNN.
We call the first R − 1 layers hidden layers and the R -th layer the output layer.
We use z ij to denote the i -th neuron (before activation) in the j -th hidden layer.
For DNNs that use only the ReLU activation function σ ( x ) = max ( x, 0) , we call them ReLU DNNs.
For any neuron z ij , we say the neuron is activated for an input if and only if the neuron’s value σ ( z ij ) =
z ij .
We use a binary variable α ij to represent the activation status of z ij (where α ij = 1 means the neuron is activated).
The set of activation statuses { α ij } of all the neurons is called an activation pattern.
It is known that an R m → R function is representable by a ReLU DNN if and only if it is a continuous piecewise linear (CPWL) function [35].
2.2 Linear Regions
A linear region is the set of inputs that are subject to the same activation pattern in a ReLU DNN [36].
Lemma 1.
[37] Consider a ReLU DNN f and an input x ∈ R m .
For every neuron z ij , it induces a feasible set
[Equation]
The intersection A ( x ) = (cid:84) i,j A ij ( x ) is the linear region that includes x .
Note that A ( x ) is essentially the H-representation of the corresponding convex polytope.
2.3 Global Robustness Property
We consider a global robustness property P on X ⊆ X .
Definition 1 (Global Robustness) .
A DNN f satisfies a global robustness property P ( X, Ω , ϵ ) on an input
set X ⊆ X along with a perturbation set Ω ⊆
R m if and only if for any x ∈ X and ∆ x ∈ Ω , ∥ f ( x ) − f ( x + ∆ x ) ∥ ≤ ϵ holds.
Definition 2 (Norm-Bounded Global Robustness) .
A DNN f is ( δ, ϵ ) -globally robust on an input set
X ⊆ X
if and only if for any x ∈ X , we have that ∥ ∆ x ∥ ≤ δ ⇒ ∥ f ( x ) − f ( x + ∆ x ) ∥ ≤ ϵ .[-
-]
This is an instantiation of the global robustness property with Ω = { ∆ x | ∥ ∆ x ∥ ≤ δ } .
The notion of individual fairness [38, 24] can also be viewed as a global robustness property, by taking Ω = { ∆ x | ∆ x NF = 0 } where SF indicates sensitive features and NF indicates the remaining, non-sensitive features, as follows.
Definition 3 (Individual Fairness) .
A DNN f is ϵ -fair with respect to some sensitive input features SF if and only if for any x ∈ X , if x SF = ( x + ∆ x ) NF , then
∥ f
( x ) − f ( x + ∆ x ) ∥ ≤ ϵ .
2.4 Neural Network Repair
We are now ready to define the repair problem.
Definition 4 (Repair for Global Robustness Property) .
Given a global robustness property P ( X, Ω , ϵ ) and a target DNN f ̸| = P ( X, Ω , ϵ ) , the repair problem is to find a modified DNN (cid:98) f such that (cid:98) f | =
P ( X, Ω , ϵ ) .
2.5 Verification for Global Robustness Property
A standard way to verify global robustness is to reduce it to verifying local robustness by constructing a twin-network [39].
ITNE [20, 21] is the state-of-the-art verification technique that uses an interleaving twin-network encoding approach where two copies of the neural network are encoded side-by-side with extra interleaving dependencies added between them.
Bound propagation techniques [17, 40, 10, 41] can then be applied to compute the output bound for a given input area on the twin-network.
In REGLO , we use global robustness verification to guide the repair process which we will describe in detail in the next section.
The REGLO Approach
We present REGLO ’s approach below, starting with an observation that allows us to identify repair regions that violate a given global robustness property.
A preview of the high-level approach in
REGLO is also given in Figure 1.
3.1 Key Observation
A counterexample to a global robustness property P ( X, Ω , ϵ ) is a pair of inputs ( x, ∆ x ) that violates the property P , i.e., x ∈ X and ∆ x ∈ Ω but ∥ f ( x ) − f ( x + ∆ x ) ∥ ≥ ϵ .
Based on the following
[Figure] Iterate [Figure] Mean Value Inequality Theorem [42], we can convert the search of a potential violation area (an area that contains a counterexample pair) for a global robustness property to the search of a single point with a large gradient, as illustrated in Figure 2.
Theorem 1 (Mean Value Inequality [42]) .
For a continues function f : [ a, b ] → R n , if f is differentiable on (a, b), then
[Equation]
Directly applying the Mean Value Inequality Theorem to a global robustness property on a DNN, we obtain the following corollary.
Corollary 1 (Gradient Features of Global Robustness Properties) .
For a DNN f and a global robustness property P ( X, Ω , ϵ ) , if there is a counterexample ( x, ∆ x ) such that ∆ x ∈ Ω and ∥ f
( x + ∆ x ) − f ( x ) ∥ ≥ ϵ , then there exists a differentiable point x ′ between x and x + ∆ x , such that ∥ x − x ′ ∥ ≤ dia (Ω) and ∥ ∂f∂x ( x ′ )
∥ > ϵ dia (Ω) , where dia (Ω) is the diameter of Ω .[-
-]
Remark :
Note that this corollary specifies a necessary but not sufficient condition.
In other words, the presence of a counterexample must exhibit a large corresponding gradient but the reverse is not necessarily true.
The Mean Value Inequality Theorem requires differentiability of the function f on ( a, b ) .
For non-differentiable DNNs, the non-differentiability comes from its activation functions, e.g., ReLU,
LeakReLU, Heaviside, or Maxpooling.
Since the measure of the non-differentiable area is zero for those operations, we can still apply the Mean Value Inequality Theorem in a piecewise manner to obtain the same result.
Remark :
For individual fairness, we can define the norm ∥ x ∥ = ∥ x ∥ F if ∥ x ∥ NF = 0 and else ∥ x ∥ = + ∞ , and only consider the gradient with respect to the sensitive features ∥ ∂f∂x ( x ′ ) ∥
=
∥ ∂f∂x SF ( x ′ ) ∥ .
3.2 Repair Areas
For a ReLU DNN and the L ∞ output bound, we can encode the search of the maximal gradient as a mixed-integer programming (MILP) problem [43, 44]: [Equation] for 0 ≤ j ≤ R
[Equation]
where B is a big enough number and ν is a small enough number.
By solving the MILP problem (3), we obtain the optimal input x ∗ , the optimal activation pattern α ∗ , and the largest gradient norm c ∗ .
Our goal is to find all the linear regions with gradients larger 5 than ϵ dia (Ω) .
We store the linear region { x | Ax ≤ b } that contains x ∗ which can be obtained from Lemma 1 for c ∗ > ϵ dia (Ω) .
Then we add the following additional constraint to the MILP problem (3), which is used to exclude the same activation pattern ˆ α that we have already found, and solve the MILP problem with the additional constraint to find a new violating linear region.
[Equation]
Since the number of linear regions on X is finite, we can find all the linear regions with gradient greater than ϵ dia (Ω) by iteratively solving MILPs and accumulating the exclusion constraint (4) until the optimal solution c ∗ is smaller than ϵ dia (Ω) .
Note that while the total number of linear regions can be very large, the number of violating regions is typically much smaller.
For non-CPWL DNNs, we can use random sampling to search for input x ′ that satisfies the violation constraint ∥ ∂f∂x ( x ′ ) ∥ > ϵ dia (Ω) .
If the target DNN f is twice differentiable, we can apply a projected gradient descent method to improve the sample efficiency.
For CPWL DNNs, we can also start with random sampling and then pivot to the more expensive MILP-based method for better search efficiency.
Now we can define the repair area A as follows.
A = { x | Ax ≤ b } by solving MILP (3) or
A = { x | ∥ x − x ′ ∥ ≤ δ 2 } via random sampling.
We use {A i } i ∈
I to denote all the repair areas found via the aforementioned procedure.
3.3 Verification-Guided Constraints
For each repair area A i , we apply ITNE [21] to estimate ϵ ∗ i , which is an over-approximation of the global robustness bound on A i .
ϵ ∗ i is the optimal value of the following optimization problem.
[Equation]
where θ is the weight in the last hidden layer of the DNN, D li , e li , D ui and e ui are parameters for the linear bounds of ∆ z , and ∆ z is the neurons value difference of the last hidden layer between x and x + ∆ x .
Remark: We consider weight modification in the last hidden layer because it does not change the activation pattern of any input and in turn preserves the boundaries of the linear regions.
Thus, by repairing the violating linear regions iteratively, we can guarantee satisfaction of the given global robustness property.
In theory, we can also consider modifying the weights of an intermediate layer.
However, the objective function of optimization problem (5) will include the subsequent layers and the optimization problem will no longer be convex.
3.4 Repair as Robust Optimization
We use ∆ θ to denote the weight change to the DNN’s last hidden layer.
In order to preserve the functionality (e.g., accuracy) of the network, we aim to find a minimal weight change ∆ θ that can guarantee the satisfaction of global robustness property on all the repair areas {A i } i ∈
I .
Formally, ∆ θ is the solution to the following optimization problem.
[Equation]
Property 1.
Optimization problem (6) is convex and thus any local minimum also achieves the global minimum.
This minimization problem with inner maximal constraints is a form of robust optimization [45].
For minimizing the L 1 or L ∞ norm of ∆ θ , such robust optimization problems can be solved by taking the robust counterpart of the inner constraints and converting it to a linear programming (LP) problem.
3.5 Repair via Barrier Method
For general L p norm of ∆ θ , we apply the barrier method from [46] and formulate it as an unconstrained convex optimization problem.
We use [∆ x ∗ i (∆ θ ) , ∆ z ∗ i (∆ θ )] and ϵ ∗ i ( θ + ∆ θ ) to denote the optimal solution and the optimal value respectively for optimization problem (5).
We have, for a sufficiently large t , the solution of the following barrier problem converges to the solution of the optimization problem (6).
[Equation]
To solve the optimization problem (7), we compute the gradient of ϵ ∗ i ( θ + ∆ θ ) by
[Equation]
3.6 Iterative Repair
In the previous sections, we enumerate all the repair areas {A i } i ∈
I that violate P ( X, Ω , ϵ ) and repair these areas via weight modification.
However, it is possible that the repair increases the gradient ∥ ∂f∂x ( x ′ ) ∥ for x ′ ∈ X \∪ i ∈ I A i and results in a new violation to P ( X, Ω , ϵ ) . To ensure a sound repair, we iteratively search for repair areas to repair.
When solving the optimization problem, we consider both the previously repaired areas and the new repair areas. We call one iteration, including the search of repair areas and the repair itself, a Single Iteration Repair . The Iterative Repair algorithm is given in Algorithm 1.
Let 2: Let repair I 3: while t do 4: while | I | < M do 5:
Let c ∗ and x ∗ be the optimal value and solution of MILP (3), respectively.
6: if c ∗ ≤ ϵ dia (Ω) then 7:
Break 8: end if 9:
Let A i be the linear region containing x ∗ computed using Lemma 1.
I .add
( A i ).
Add a new exclusion constraint (4) to MILP (3).
12: end while 13:
Formulate the constraints on all repair areas I as a robust optimization (6), solve it via
Algorithm 2 (in the Appendix), and obtain an optimal solution ∆ θ .
Update f ’s last-layer weight θ = θ + ∆ θ .
15: end while 16: return f with updated last-layer weight.
Theoretical Guarantees
In this section, we present the theoretical guarantees that REGLO provides, and point the readers to proofs of the theorems in the Appendix.
4.1 Completeness Guarantees
Theorem 2 (Completeness Guarantees) .
We have the following completeness guarantees for REGLO :
For a Single Iteration Repair, REGLO can always find a solution to optimization problem (6).
For an Iterative Repair on a piecewise linear DNN, REGLO always terminates with no more repair areas to be found by solving MILP (3).
4.2 Soundness Guarantees
Ideally, if the target DNN is a ReLU DNN, we can enumerate all the linear regions that violate the global robustness property P ( X, Ω , ϵ ) by solving multiple MILPs and applying REGLO .
By Iterative
Repair , REGLO will terminate when no more repair area can be found.
Thus, we have the soundness guarantee for the resulting DNN.
Specifically, for a Single Iteration Repair , the weight change ∆ θ ensures the satisfaction of P ( X, Ω , ϵ ) on all the repair areas {A i } i ∈ I .
As we discussed in the Iterative Repair Section , the repair may increase the gradient ∥ ∂f∂x ( x ′ )
∥ for x ′ ∈ X \ ∪ i ∈
I
A i and causes it to violate P ( X, Ω , ϵ ) .
We call any such violation in X \ ∪ i ∈
I A i a side effect of our repair.
The following theorem shows that we have guarantees on limiting the side effects of a Single Iteration Repair .
Theorem 3 (Limited Side Effect for Single Iteration Repair) .
Given a global robustness property
P ( X, Ω , ϵ ) , a target DNN f , and weight change ∆ θ from a Single Iteration Repair, we have
for any area B ⊂ ∪ i ∈
I A i ⊂ X , (cid:98) f | =
P ( B , Ω , ϵ ) ; 2. for any area C ⊂ X which is not a subset of ∪ i ∈
I
A
i , (cid:98) f | =
P ( C , Ω , ϵ + 2 L ∥ ∆ θ ∥∥ X ∥ ) , where L is the Lipschitz constant of f from the input layer to the last hidden layer.
Corollary 2 (Soundness Guarantees for Repairing CPWL DNNs) .
Given a global robustness property
P ( X, Ω , ϵ ) , a piecewise linear DNN f , and weight change ∆ θ from Iterative Repair, REGLO will terminate with no more repair areas to be found and the resulting DNN (cid:98) f | = P ( X, Ω , ϵ ) .
For DNNs that are not piecewise linear, we have the following weaker soundness guarantee.
Corollary 3 (Soundness Guarantee for Repairing General DNNs) .
Given a global robustness property
P ( X, Ω , ϵ ) , a DNN f , and weight change ∆ θ from Iterative Repair, REGLO returns a DNN (cid:98) f | =
P ( C , Ω , ϵ + 2 L ∥ ∆ θ ∥∥ X ∥ ) for any repair area C .
Experiments
Our prototype tool is implemented in Python.
We use Gurobi [47] to solve MILP (3) and use CVXPY [48] to solve optimization problem (5).
The global robustness bounds used in (5) are derived by ITNE [21] with a bound propagation technique similar to β -CROWN
[41] (without bound refinements for efficiency).
The verification bounds (VBs) in the experiments are evaluated using ITNE with bound refinements for tighter estimations.
5.1 Baseline Methods
To the best of our knowledge, REGLO is the first study on DNN repair to satisfy a global robustness property.
We consider the following five baseline methods for comparison with REGLO .
• ST: standard training.
• AT: adversarial training using PGD [3] on the training data.
• AT-G: adversarial training with counterexamples that violate the global robustness property (AT-G).
The counterexamples are generated by applying PGD on randomly sampled points in X .
• ST+AT-G: standard training followed by an adversarial fine-tuning for the global robustness property.
Fine-tuning is to fine-tune (train with a smaller learning rate) a pre-trained DNN with additional counterexamples generated similarly to those in AT-G. • AT+AT-G: adversarial training followed by an adversarial fine-tuning for the global robustness property.
For REGLO , we consider two settings where the repair is applied after standard training and after
adversarial training respectively
. • ST+
REGLO :
standard training followed by REGLO .
• AT+ REGLO : adversarial training followed by REGLO .
5.2 Evaluation Metrics
Given that there is no efficient method for exact verification of a global robustness property, we use
ITNE [20] to compute an upper-bound of the true global robustness ϵ ∗ .
In addition, we use PGD to evaluate the empirical robustness of randomly sampled inputs in X , as a lower-bound of ϵ ∗ .
We also report accuracy on testing data and runtime.
• VB: the verification bound given by ITNE for the global robustness property.
• PGD-B: the maximum norm difference on outputs between input pairs ( x, x +∆ x ) computed by PGD; • PGD-R: the ratio of input pairs computed by PGD that violate the global robustness property.
• ERR: the mean absolute error on prediction for testing data (for regression problems); • ACC: the accuracy on testing data (for classification problems).
• T(s): runtime in seconds.
Remark: Note that PGD-B ≤ ϵ ∗ ≤ VB, where ϵ ∗ is the unknown, true global robustness of the target
DNN for X and Ω .
5.3 Benchmark Evaluations
We perform three benchmark evaluations, including individual fairness in a classification problem, as well as norm-bounded global robustness in a regression problem and a classification problem.
All experiments were run on machines with CPUs similar to ten-core 2.6 GHz Intel Xeon E52660v3 without GPU.
Details of the experimental setup such as DNN architectures and training hyperparameters can be found in the Appendix.
German Credit (classification): repair for individual fairness.
We train a ReLU DNN on the
[Table]
German Credit dataset [49] to predict the credit risks (good or bad) for a person based on input features.
An ideal DNN predictor should be fair with respect to the sensitive input feature ‘age’, that is the resulting DNN should satisfy P ( X, Ω , ϵ ) for Ω
= { ∆ x | ∆ x SF = 0 } with ϵ = 0 . 01 , where SF are all input features other than ‘age’.
We consider the individual fairness properties on input domain X as well as regions based on age groups: X [-
-]0 = { x | x age ≤ 24 } , X 1 = { x | 25 ≤ x age ≤ 54 } ,
X 2 = { x | 55 ≤ x age ≤ 64 } , or X 3 = { x | 65 ≤ x age } according to [50].
For REGLO , we search for repair areas by random sampling on X and choose 30 areas to repair.
The results are shown in Table 1.
Auto MPG (regression): sound repair for norm-bounded global robustness.
We train a ReLU
DNN on the Auto MPG dataset [49] to predict the fuel efficiency (mile per gallon) of a car.
We consider a norm-bounded global robustness property P ( X, Ω , ϵ ) , where Ω = { ∆ x | ∥ ∆ x ∥ ≤ δ } , ϵ = 1 . 5 , δ = 0 .
05 and X is the smallest hyper-rectangle that contains all the training inputs.
For
REGLO , we use MILP (3) to search for repair areas and apply an Iterative Repair (Algorithm 1).
REGLO terminates after 2 iterations of repair and in total, 4 repair regions with different ReLU activation patterns were found during the repair.
The results are shown in Table 2.
MNIST (classification): repair for norm-bounded global robustness.
We train a convolutional
DNN with ReLU units on the MNIST dataset [51].
We consider a norm-bounded global robustness property P ( X, Ω , ϵ ) , where Ω = { ∆ x | ∥ ∆ x ∥ ≤ δ } , δ = 0 .
3 , and ϵ = 0 .
[Table] [Table] We consider X to be a class rectangle (the hyper-rectangle that contains all the training inputs with the same class label).
For example, X k
= { x | x l k ≤ x ≤ x u k } , where x l k i = min x ∈ X k x i and x u k i = max x ∈ X k x i for every dimension i of the input space and a fixed class k .
For REGLO , we search repair areas by random sampling on X and choose 30 areas to repair.
A subset of the results is shown in Table 3 and Table 4.
The full set of results can be found in the Appendix.
5.4 Sound Repair with Stronger Constraints
To address the issue of soundness due to sampling and repairing only a subset of the violating linear regions when the input dimension or the number of linear regions is large, we consider imposing a stronger constraint during repair to recover the guarantee on satisfying the global robustness property.
Figure 3 illustrates the results of using a smaller bound ¯ ϵ ≤ ϵ for the max constraint in the optimization problem (6).
We fix the repair regions to the ones sampled with ¯ ϵ = 0 .
3 to eliminate the effect of random sampling for the repair regions (using a smaller ¯ ϵ does not affect the overall set of repair regions since the repair regions are still identified using ϵ ).
At ¯ ϵ = 0 .
1 , we have a VB less than the required ϵ of 0.3 and the accuracy drop is less than 5% .
It is possible to guarantee global robustness over X by repairing only a subset of the violating regions in X because repairs on those regions can generalize to the other regions. [Table]
Concluding Remarks
REGLO is the first work that enables provable repair of neural networks for global robustness properties.
Experimental results demonstrate the effectiveness of the approach against multiple baselines.
Achieving such deterministic guarantees, however, can be challenging when we apply the technique to larger networks.
Compared to local robustness, global robustness is a strictly stronger condition that requires robustness for all infinitely many inputs within a region.
As a result, verifying whether a network satisfies a given global robustness property is fundamentally more challenging than local robustness verification.
ITNE [20], which REGLO uses as a subroutine as necessitated by the need for deterministic guarantees, is the current state-of-the-art technique for verifying global robustness properties.
However, it can still take hours for ITNE to compute a global robustness bound for a network with around 10k neurons on the CIFAR-10 dataset.
Another challenge to scalability lies in finding the violating linear regions. As the size of the network increases, solving the MILP problem in Eq. (3) can become very expensive. In addition, the number of linear regions of a ReLU network grows exponentially in the number of layers and polynomially in the number of neurons (or layer width)
Thus, while in theory REGLO can provide completeness guarantees as stated in Theorem 2, for large ReLU DNNs (and non-CPWL DNNs) we have to resort to random sampling for finding the violating regions as described in Section 3.2.
Future works include improving the efficiencies of identifying repair areas and computing tight verified global robustness bounds.
Another direction is to consider statistical techniques to sidestep the inherent complexity of aiming for deterministic guarantees for global robustness properties.
Acknowledgements.
We gratefully acknowledge the support from the National Science Foundation awards CCF-1646497, CCF-1834324, CNS-1834701, CNS-1839511, IIS-1724341, CNS-2038853,
ONR grant N00014-19-1-2496, the Intelligence Advanced Research Projects Agency (IARPA) under the contract W911NF20C and the US Air Force Research Laboratory (AFRL) under contract number FA8650-16-C-2642.
Appendix 7.1 Proofs of Theorems
Corollary 1 (Gradient Features of Global Robustness Properties) .
For a DNN f and a global robustness property P ( X, Ω , ϵ ) , if there is a counterexample ( x, ∆ x )
such that ∆ x ∈ Ω and ∥ f
( x + ∆ x ) − f ( x ) ∥ ≥ ϵ , then there exists a differentiable point x ′ between x and x + ∆ x such that ∥ x − x ′ ∥ ≤ dia (Ω) and ∥ ∂f∂x ( x ′ )
∥ > ϵ dia (Ω) , where dia (Ω) is the diameter of Ω .
Proof.
Consider g : [0 , 1] → R n defined as g ( t ) =
f ( x + t ∆ x ) and g is piecewise differentiable on [0 , 1] .
Since ϵ < ∥ g (1) − g (0) ∥ , there must exist a differentiable interval ( t i , t i +1 ) , such that
[Equation]
Then by Theorem 1, we have,
[Equation]
Thus, there exists a x ′ =
x + t ′ ∆ x , ∥ x − x ′ ∥ ≤ dia (Ω) (we can simply switch x and x + ∆ x if ∥ x − x ′ ∥ > dia (Ω) ), and
[Equation]
Property 1.
Optimization problem (6) is convex and thus any local minimum also achieves the global minimum.
Proof.
By the definition of a norm, we have any norm ∥ .
∥ is a convex function.
For any ∆ θ 1 , ∆ θ 2 , λ ∈ [0 , 1] and any ∆ z i , we have
[Equation]
Both the objective and the constraints are convex.
Therefore, we have that optimization programming (6) is convex.
Thus, any local minimum achieves the global minimum.
Theorem 2 (Completeness Guarantees) .
We have the following completeness guarantees for REGLO :
For a Single Iteration Repair, REGLO can always find a solution to the optimization problem (6).
For an Iterative Repair on a piecewise linear DNN, REGLO always terminates with no more repair areas to be found by solving MILP (3).
Proof.
For a Single Iteration Repair , the optimization problem (6) is convex over a close domain.
Since ∆ θ = − θ is a feasible solution, the domain where ∆ θ is in is always feasible.
Therefore, optimization problem (6) has an optimal solution.
Since the optimization problem is convex, the optimal solution is unique and REGLO can always find the optimal solution to the optimization problem (6).
For an Iterative Repair on a CPWL DNN, REGLO can always find an optimal solution for every Single
Iteration Repair .
Given that the number of linear regions for a CPWL DNN is finite, REGLO always terminates with no more repair areas to be found by solving the MILP.
Theorem 3 (Limited Side Effect for Single Iteration Repair) .
Given a global robustness property
P ( X, Ω , ϵ ) , a target DNN f , and weight change ∆ θ from a Single Iteration Repair, we have
for any area B ⊂ ∪ i ∈
I A i ⊂ X , (cid:98) f | =
P ( B , Ω , ϵ ) ; 2. for any area C ⊂ X which is not a subset of ∪ i ∈
I
A
i , (cid:98) f | =
P ( C , Ω , ϵ + 2 L ∥ ∆ θ ∥∥ X ∥ ) , where L is the Lipschitz constant of f from the input layer to the last hidden layer.
Proof.
The constraints of optimization problem (6) ask the resulting DNN must satisfy the global robustness property on B ⊂ ∪
i ∈
I A i ⊂ X , a subset of the repair areas.
The difference between the resulting DNN (cid:98) f and the target DNN f on any input x can be bounded by the norm of ∆ θ : ∥ (cid:98) f ( x ) − f ( x ) ∥ =
∥ ( θ + ∆ θ )
f n − 1 ( x ) − θf n − 1 ( x ) ∥ =
∥
∆ θf n − 1 ( x ) ∥ ≤ ∥ ∆ θ ∥ · ∥ f n − 1 ∥ · ∥ x ∥ =
L ∥ ∆ θ ∥ · ∥ x ∥ , where f n − 1 is the DNN function from the input layer to the last hidden layer.
For any x ∈ C ⊂ X , which is not in a subset of ∪ i ∈
I
A i , and any ∆ x ∈ Ω : ∥ (cid:98) f ( x + ∆ x ) − (cid:98) f ( x ) ∥ = ∥ (cid:98) f
( x + ∆ x ) − f ( x + ∆ x ) + f ( x + ∆ x ) − f ( x ) + f ( x ) − (cid:98) f
( x ) ∥ ≤∥ (cid:98) f ( x + ∆ x ) − f ( x + ∆ x ) ∥ + ∥ f ( x + ∆ x ) − f ( x ) ∥ + ∥ f ( x ) − (cid:98) f ( x ) [Equation]
Corollary 2 (Soundness Guarantees for Repairing CPWL DNNs) .
Given a global robustness property
P ( X, Ω , ϵ ) , a piecewise linear DNN f , and weight change ∆ θ from Iterative Repair, REGLO will terminate with no more repair areas to be found and the resulting DNN (cid:98) f | = P ( X, Ω , ϵ ) .
Proof.
For Iterative Repair on a CPWL DNN f , by Theorem 2, REGLO will terminate with no more repair areas to be found.
Therefore, the resulting DNN satisfies P ( X, Ω , ϵ ) outside the repair areas.
In addition, by Theorem 3, we have that the resulting DNN satisfies P ( C , Ω , ϵ ) for any C ⊂ X .
Combining these results, we have that REGLO will terminate with no more repair areas to be found and the resulting DNN (cid:98) f | =
P ( X, Ω , ϵ ) .
Corollary 3 (Soundness Guarantee for Repairing General DNNs) .
Given a global robustness property
P ( X, Ω , ϵ ) , a DNN f , and weight change ∆ θ from Iterative Repair, REGLO returns a DNN (cid:98) f | =
P ( C , Ω , ϵ + 2 L ∥ ∆ θ ∥∥ X ∥ ) for any repair area C .
Proof.
Since Iterative Repair collects both the previously repaired areas and the new repair areas found at one iteration, the last Single Iteration Repair will repair all those collected repair areas.
Therefore, by Theorem 3, the last Single Iteration Repair will return a DNN (cid:98) f that satisfies P ( C , Ω , ϵ + 2 L ∥ ∆ θ ∥∥ X ∥ ) .
Input : Current last-layer weight θ , repair areas {A i } i ∈ I , and Ω .
Parameter :
Initial step size α , initial weight of barrier function t , an early stop threshold δ , and maximal steps K .
Output : ∆ θ 1: Let k = 0 .
2: Let ∆ θ = − θ (start from a feasible solution).
3: while k < K do 4:
Let ϵ ∗ i and ∆ z ∗ i be the optimal value and optimal solution of optimization problem (5), respectively.
Compute the gradient g = ∂ϵ ∗
i ( θ +∆ θ ) ∂ ∆
θ according to Equation (8).
6: if ∥ g ∥ < δ then 7:
Break 8: end if 9:
Update ∆ θ = ∆ θ − α · g 10:
Update α and t .
11: end while 12: return ∆ θ .
Additional Experiment Details 8.1 DNN Architectures and Training/Repair Hyperparameters
German Credit: The DNN is a multi-layer perceptron with ReLU activation functions.
It has an input layer with 20 neurons, 2 hidden layers with 200 neurons in each layer, and a final output layer withneuron.
Auto MPG:
The DNN is a multi-layer perceptron with ReLU activation functions.
It has an input layer with 3 neurons, 2 hidden layers with 20 neurons in each layer, and a final output layer withneuron.
MNIST:
The DNN is a convolutional neural network with ReLU activation functions.
It has two convolutional layers and one dense layer.
The two convolutional layers have 16 and 32 channels, respectively.
The dense layer has 16 neurons.
We set the learning rate to 10 − 3 in all the training experiments.
For fine-tuning, we set a learning rate of 5 ∗ 10 − 4 on the German Credit experiment, 2 ∗ 10 − 4 on the Auto MPG experiment, and 2 ∗ 10 −on the MNIST experiment.
The number of epochs for training is 20 and the batch size is 256 .
For AT-G, the number of points that we sampled in each iteration is 100 .
The step size we used for PGD-B and PGD-R is 0 .
and the number of steps is 200 .
The step size we used for AT and AT-G is 0 .
and the number of steps is 10 .
8.2 Full Results for Norm-bounded Global Robustness Repair on MNIST
Table 5 shows the full set of results for norm-bounded global robustness repair on MNIST.
Overall,
ST+
REGLO achieves the best (smallest) VB and AT-G achieves the best (smallest) PGD-B, while both methods have very small PGD-R (0.0% or close to 0.0%) and do not have any significant drop in accuracy.
Compared with AT-G, ST+ REGLO has the following advantages: (1) repair using REGLO does not require access to the training data (which may not be available due to privacy reasons for instance ), (2) it can be applied to a trained network as a post-hoc modification especially when the global robustness property is only given after training, and (3) a much smaller runtime as REGLO does not require adversarial training.
In addition, AT-G is not able to reduce VB which is required by the specification.
On the other hand, we can use a ¯ ϵ <
ϵ in REGLO to satisfy the specification even if we are only sampling and repairing a subset of the violating linear regions, as shown at the beginning of the Appendix.
[Table]