Title: Provable Editing of Deep Neural Networks using Parametric Linear Relaxation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Provable editing

provable repair

provable training

trustworthiness

linear programming

local robustness

verification
Abstract: Ensuring that a DNN satisfies a desired property is critical when deploying DNNs in safety-critical applications. There are efficient methods that can verify whether a DNN satisfies a property, as seen in the annual DNN verification competition (VNN-COMP). However, the problem of provably editing a DNN to satisfy a property remains challenging. We present PREPARED, the first efficient technique for provable editing of DNNs. Given a DNN $\mathcal{N}$ with parameters $\theta$, input polytope $P$, and output polytope $Q$, PREPARED finds new parameters $\theta'$ such that $\forall \mathrm{x} \in P . \mathcal{N}(\mathrm{x}; \theta') \in Q$ while minimizing the changes $\lVert{\theta' - \theta}\rVert$. Given a DNN and a property it violates from the VNN-COMP benchmarks, PREPARED is able to provably edit the DNN to satisfy this property within 45 seconds. PREPARED is efficient because it relaxes the NP-hard provable editing problem to solving a linear program. The key contribution is the novel notion of Parametric Linear Relaxation, which enables PREPARED to construct tight output bounds of the DNN that are parameterized by the new parameters $\theta'$. We demonstrate that PREPARED is more efficient and effective compared to prior DNN editing approaches i) using the VNN-COMP benchmarks, ii) by editing CIFAR10 and TinyImageNet image-recognition DNNs, and BERT sentiment-classification DNNs for local robustness, and iii) by training a DNN to model a geodynamics process and satisfy physics constraints.

Provable Editing of Deep Neural Networks using Parametric Linear Relaxation

Abstract

Ensuring that a DNN satisfies a desired property is critical when deploying DNNs in safety-critical applications. There are efficient methods that can verify whether a DNN satisfies a property, as seen in the annual DNN verification competition ( VNN -COMP ). However, the problem of provably editing a DNN to satisfy a property remains challenging. We present PREPARED , 1 the first efficient technique for provable editing of DNNs. Given a DNN N with parameters θ , input polytope P , and output polytope Q , PREPARED finds new parameters θ such that ∀ x ∈ P . N (x; θ ) ∈ Q while minimizing the changes ∥ θ -θ ∥ . Given a DNN and a property it violates from the VNN -COMP benchmarks, PREPARED is able to provably edit the DNN to satisfy this property within 45 seconds. PREPARED is efficient because it relaxes the NP-hard provable editing problem to solving a linear program. The key contribution is the novel notion of Parametric Linear Relaxation, which enables PREPARED to construct tight output bounds of the DNN that are parameterized by the new parameters θ . We demonstrate that PREPARED is more efficient and effective compared to prior DNN editing approaches i) using the VNN -COMP benchmarks, ii) by editing CIFAR10 and TINYIMAGENET image-recognition DNNs, and BERT sentiment-classification DNNs for local robustness, and iii) by training a DNN to model a geodynamics process and satisfy physics constraints.

1 Introduction

Ensuring that a DNN is correct and avoids harmful behaviors is critical when deploying them, especially in safety-critical contexts [8]. In such scenarios, it is vital to guarantee that a DNN satisfies a given property, e.g., a safety specification [35]. Towards this end, DNN verifiers aim to determine whether a DNN satisfies the given property [1]. There are efficient methods for DNN verification that can handle a wide-range of DNN architectures and properties, as seen in the annual DNN verification competition ( VNN -COMP ) [5]. However, the success of DNN verification reveals the next challenge: provably editing a DNN to satisfy a property. Formally, we define the provable editing problem as:

Definition 1.1. Given a DNN N with parameters θ , an input polytope P def = { x | Dx ≤ e } and an output polytope Q def = { y | Ay ≤ b } , the provable editing problem is to find new parameters θ that

Provable editing can be used, for instance, to ensure that a classifier DNN is locally robust for an input x . In this case, the input polytope P def = { x ′ | ∥ x -x ′ ∥ ∞ ≤ ε } is the L ∞ ball around x with

x

x

x

y

def

=

ReLU

(

x

)

(a)

x

y

def

=

ReLU

(

x

)

(b)

x

y

def

=

ReLU

(

x

)

(c)

̸

radius ε , and the output polytope Q def = { y | ∧ j = l y j < y l } requires that the output label is l ; viz., arg maxy = l . Moreover, provable editing can be used either during training to guarantee that the DNN satisfies a property, or post-deployment to repair a given DNN.

The provable editing problem is challenging due to the presence of the universal quantifier; viz., finding new parameters such that for all inputs in the polytope P , the output of the edited network lies in the polytope Q . There are no existing approaches for efficiently and effectively solving the provable editing problem. Regularization-based approaches [26, 10, 28, 24] encode the property into the loss during training, but are unable to guarantee the property-satisfaction. SMT-based approaches [14, 16] provide guarantees, but are not efficient because they directly solve an NP-hard problem. Prior LP-based approaches [41, 42] are efficient because they can only handle a restricted class of provable editing problems; e.g., they assume that the vertices of the input polytope or the linear regions of the DNN can be efficiently enumerated, or the DNN architecture can be modified.

This paper presents PREPARED , the first efficient technique for provable editing of DNNs that runs in polynomial time (Theorem 3.4). Given a DNN and a property it violates from the VNN -COMP ′ 22 benchmarks, PREPARED is able to provably edit the DNN to satisfy this property within 45 seconds.

PREPARED approaches the problem of provable editing by constructing tight parametric bounds of the output N (x; θ ) for all inputs x in the input polytope P in terms of the parameters θ , and then constraining these parametric bounds to lie within the output polytope Q . These parametric bounds are constructed compositionally per layer of the DNN, and are expressed as linear constraints, so that efficient Linear Programming (LP) solvers can be used to find an optimal solution.

Our key insight is to represent the parametric bounds indirectly via underapproximations of the epigraph and hypograph of the DNN layer. Consider the ReLU layer y def = ReLU ( x ) . Prior approaches [49, 39] overapproximate its upper bound with a linear function ( in Figure 1(a)) using constant input bounds [ x,x ] . Such a linear relaxation is loose, and is only sound within the given constant input bounds [ x,x ] . Our approach is able to exactly represent the upper bound of ReLU output for any input upper bound variable x ( in Figure 1(b)). This is done by capturing the epigraph ( in Figure 1(c)) of ReLU ( x ) with a linear constraint.

Our key contribution is a novel Parametric Linear Relaxation for DNN layers, which defines tight parametric bounds using linear constraints in terms of the layer parameters. Consider v def = xw , which is a simplified representation of an Affine layer. We would like to construct bounds for all x ∈ [ x,x ] in terms of the variable parameter w . Prior approaches could achieve a sound, but loose linear relaxation that is not in terms of the layer parameters (Figure 2(a)). In contrast, our approach exactly represents the output bounds without any relaxation (Figure 2(b)) building upon our key insight of capturing the epigraph and hypograph of the DNN layer in terms of the parameters (Figure 2(c)).

∀

w

∈

[

w,w

]

v

=

w

x

v

=

w

x

x

x

∀

x

∈

[

x,x

]

∀

x

∈

[

x,x

]

x

v

def

=

xw

(a)

v

=

w

x

x

x

∀

x

∈

[

x,x

]

∀

x

∈

[

x,x

]

x

v

def

=

xw

(b)

v

=

xw

v

=

xw

[

x,x

∀

x

∈

[

x,x

]

w

Our bounds are parameterized

by any variable weight

w

.

w

v

def

=

xw

(c)

We evaluate PREPARED i) using the VNN -COMP benchmarks, ii) by editing CIFAR10 and TINYIMAGENET image-recognition DNNs, and BERT sentiment-classification DNNs for local robustness, and iii) by training a DNN to model a geodynamics process and satisfy physics constraints. Because there were no prior efficient approaches for provable editing, we implemented new provable finetuning baselines by integrating prior DNN editing approaches with a verifier in the loop. PREPARED outperformed such baselines demonstrating its effectiveness and efficiency.

2 Related work

Regularized training [20, 45, 30, 10, 19, 22, 47] incorporates constraints as regularization into the training loss, but does not guarantee constraint-satisfaction in the trained DNN. For the DNN editing problem we are addressing, which involves universally-quantified logical formulas, DL2 [10] is the state-of-the-art regularized training method.

Certified training [26, 15, 36, 2, 31, 28, 24] is a type of regularized training geared towards adversarial robustness. SABR [28] and STAPS [24] are state-of-the-art certified training approaches. However, they also do not guarantee constraint-satisfaction.

SMT-based editing approaches [14, 23, 16] directly solve the NP-hard provable editing problem using an SMT solver. Thus, they are inefficient and do not scale beyond small DNNs. For instance, the recent DeepSaDe approach [16] incorporates an SMT solver during training, but only uses the SMT solver to edit the last layer. However, it still can take 500× longer than regularized training.

LP-based editing approaches [41, 42] relax the provable editing problem to solving an LP problem, which makes them efficient. However, prior approaches can only handle a restricted class of provable editing problems. The state-of-the-art provable editing approach APRNN [42] is efficient for editing input points. However, it does not scale to the general provable editing problem of Definition 1.1 due to the need for enumerating vertices of the input polytope. PRDNN [41] was the precursor to APRNN and suffers from similar limitations; moreover, PRDNN modifies the architecture of DNN and requires enumeration of linear regions to edit an input polytope. REASSURE [12] repairs a linear region of a DNN by adding a patch DNN for each such linear region. Thus, it also requires enumeration of linear regions to edit an input polytope. Further, it has been shown to be unsound and incomplete (Section 5 of [42]), and suffers from significant runtime and memory overheads (Section 7.8 of [42]).

DNN verification [38, 39, 49, 44, 37, 48, 46, 9, 4] aims to determine whether a DNN satisfies a given property. As shown in Section 4, provable editing can be used when a verifier determines that a DNN violates a property. Most verification techniques use linear relaxations to bound the DNN output. However, these linear relaxations cannot be used for provable editing because they are not parameterized in terms of the DNN parameters. Instead, verification techniques assume that the parameters are constant and focus on designing linear relaxations for activation layers like ReLU [34, 43]. However, it is not clear how even these activation layer relaxations could be used for provable editing. For instance, these relaxations require constant input bounds; loose constant input bounds result in loose linear relaxations. In the context of provable editing, if the ReLU layer follows an Affine layer with editable parameters, then the constant input bounds to the ReLU layer would be very loose (e.g., [ -∞ , ∞ ] ) in order to be sound for all possible edits. Hence, the resulting linear relaxation would also be loose. In contrast, our technique does not require constant input bounds for any layer, and produces an exact upper bound for the ReLU layer (Theorem 3.9). Applying our Parametric Linear Relaxation to DNN verification remains future work.

3 Provable editing of DNNs using Parametric Linear Relaxation

We use x ∈ R to denote a scalar, x ∈ R m to denote a column vector, and W ∈ R n × m to denote a matrix. Blue-colored variables with a dot denote LP decision variables, e.g., x , x , W , b , etc. The proofs for all theorems can be found in Appendix A.

Definition 3.1 (DNN) . Let y def = N (x; θ ) denote an L -layer deep neural network (DNN) N with parameters θ , input x and output y . The DNN is defined iteratively as x ( ℓ +1) def = N ( ℓ ) (x ( ℓ ) ; θ ( ℓ ) ) for each layer N ( ℓ ) with parameters θ ( ℓ ) , where x (0) def =x , y def =x ( L ) and 0 ≤ ℓ < L . ■

For ease of exposition, we defer our approach for the general provable editing problem of Definition 1.1 to Appendix B. In this section, we consider the following provable interval editing problem, where the input and output constraints are constant interval bounds:

Definition 3.2. Given a DNN N with parameters θ , constant input bounds [x , x] and constant output bounds [y , y] , the provable interval editing problem is to find new parameters θ that

3.1 Parametric Linear Relaxation

Consider a DNN N with variable parameters θ and input bounds [x , x] , our goal is to construct sound parametric output bounds [y , y] of N in terms of θ and [x , x] . The exact definition is

However, this universally-quantified non-linear formula is expensive to solve [6, 7]. Our key contribution is a novel Parametric Linear Relaxation , a poly-size linear formula that implies Equation 3.

Definition 3.3. The Parametric Linear Relaxation φ N ( y , y , x , x; θ ) for N is a poly-size linear formula that implies ∀ x ∈ [x , x] . N ( x; θ ) ∈ [y , y] . ■

In other words, any satisfying assignment (y , y , x , x , θ ) of the formula φ N ( y , y , x , x; θ ) yields sound output bounds [y , y] for N , viz., ∀ x ∈ [x , x] . N (x; θ ) ∈ [y , y] . The size of the formula φ N ( y , y , x , x; θ ) is polynomial in the size of the DNN, i.e., the number of parameters, layers, and the input and output dimensions of each layer.

3.2 Provable editing via Parametric Linear Relaxation

The following theorem shows how Parametric Linear Relaxation can be used to solve the provable interval editing problem.

Theorem 3.4. Given a provable interval editing problem (Definition 3.2) for DNN N and parameters θ with input bounds [x , x] and output bounds [y , y] , let φ N ( y , y , x , x; θ ) be a Parametric Linear Relaxation for N . Then the following linear program can be solved in polynomial time in the size of the DNN N , and whose solution is a solution to the provable interval editing problem:

3.3 Compositional construction of Parametric Linear Relaxation

Definition 3.5. Given an L -layer DNN N with parameters θ , constant input bounds [x , x] and Parametric Linear Relaxation φ N ( ℓ ) for each layer N ( ℓ ) , we define the compositional Parametric Linear Relaxation φ N ( y , y , x , x; θ ) as follows, where [y , y] def = [x ( L ) , x ( L ) ] and [x (0) , x (0) ] def = [x , x] :

Theorem 3.6. Definition 3.5 is a Parametric Linear Relaxation for the DNN N .

By Theorems 3.4 and 3.6, we see that to solve the provable editing problem, we just need to construct Parametric Linear Relaxations φ N ( ℓ ) ( y , y , x , x; θ ) for each DNN layer N ( ℓ ) . In practice, φ N ( ℓ ) ( y , y , x , x; θ ) is defined by separate formulas φ N ( ℓ ) ( y , x , x; θ ) and φ N ( ℓ ) ( y , x , x; θ ) for the upper and lower bounds, respectively: φ N ( ℓ ) ( y , y , x , x; θ ) def = φ N ( ℓ ) ( y , x , x; θ ) ∧ φ N ( ℓ ) ( y , x , x; θ ) . For brevity, we describe how to construct Parametric Linear Relaxation for ReLU and Affine layers. Appendix C presents Parametric Linear Relaxations for Tanh, Sigmoid and ELU layers.

3.4 Parametric Linear Relaxation for ReLU layer

Definition 3.7. y def = ReLU (x) with input x ∈ R m and output y ∈ R m is defined as y def = max { x , 0 } .

Definition 3.8. For a ReLU layer with variable input bounds [x , x] , its Parametric Linear Relaxation is defined as φ ReLU (y , y , x , x) def = φ ReLU (y , x) ∧ φ ReLU (y , x) where:

and c ∈ R m are constants such that 0 ≤ c i ≤ 1 .

■

Because ReLU is a convex function, as seen in Figure 1(c), the upper bound constraint y i ≥ x i ∧ y i ≥ 0 for each y i exactly captures the epigraph ( ) of ReLU (x i ) . For each lower bound y i , we use a linear relaxation y i =c i x i ( ) of ReLU ( x ) i with a constant 0 ≤ c i ≤ 1 .

Theorem 3.9. Definition 3.8 is a Parametric Linear Relaxation for the ReLU layer that captures the exact upper bound.

3.5 Parametric Linear Relaxation for Affine layer

Definition 3.10. y def = Affine (x;W , b) with input x ∈ R m , output y ∈ R n , weight W ∈ R n × m and bias b ∈ R n is defined as y def =Wx+b , which expands to y j def = ∑ m i =0 x i W ji +b j .

In DNN verification, bounding Affine layers is straightforward, because the parameters are constants. However, in provable editing, this is challenging, because the parameters are variables. In particular, we need to bound the multiplication xw between the universally-quantified input x ∈ [ x,x ] and the variable weight w . As seen in Figure 2(a), if the bounds for xw are not parameterized by w , a linear relaxation would require constant bounds [ w,w ] for w , resulting in loose bounds. Our approach constructs tight parametric output bounds using Parametric Linear Relaxation. We now describe our approach for an Affine layer i) with constant input bounds [x , x] when it is the first layer of a DNN, and ii) with variable input bounds [x , x] when it is not.

x

y

def

=

(

x

)

(a)

x

y

def

=

Sigmoid

(

x

)

(b)

Sigmoid

x

y

def

=

ELU

(

x

)

(c)

ELU

3.5.1 Affine layer with constant input bounds

Definition 3.11. For an Affine layer with constant input bounds [x , x] and variable parameters W , b , its Parametric Linear Relaxation is defined as φ Aff (y , y , x , x , W , b) def = φ Aff (y , x , x , W , b) ∧ φ Aff (y , x , x , W , b) where:

The core of this definition is constructing parametric bounds [ v,v ] for the multiplication xw for all x ∈ [ x,x ] in terms of the variable weight w . As seen in Figure 2(c), for all x ∈ [ x,x ] , the upper bound ( ) of xw is defined by a piecewise-linear convex function f ( w ) def = max { xw,xw } . Hence, the upper bound constraint v ≥ xw ∧ v ≥ xw exactly captures the epigraph ( ) of f ( w ) . Similarly, the lower bound ( ) of xw is defined by a piecewise-linear concave function f ( w ) def = min { xw,xw } . Hence, the lower bound constraint v ≤ xw ∧ v ≤ xw exactly captures the hypograph ( ) of f ( w ) .

Theorem 3.12. Definition 3.11 is a Parametric Linear Relaxation for the Affine layer with constant input bounds [x , x] that captures the exact lower and upper bounds.

3.5.2 Affine layer with variable input bounds

We freeze the parameter weight W to be constant W (e.g., the original weight) when the input bounds are variable to avoid non-linearity.

Definition 3.13. For an Affine layer with variable input bounds [x , x] , constant weight W and variable bias b , its Parametric Linear Relaxation is defined as φ Aff (y , y , x , x , W , b) def = φ Aff (y , x , x , W , b) ∧ φ Aff (y , x , x , W , b) where:

Using constant weight W avoids non-linearity due to the multiplication xw where x is universallyquantified in variable bounds [ x,x ] . Thus, the bounds [ v,v ] of xw for all x ∈ [ x,x ] are determined by the sign of the constant w ; viz., [ v,v ] def = [ xw,xw ] if w ≥ 0 , otherwise [ v,v ] def = [ xw,xw ] .

117

185

423

0

1000

2000

3000

Better

# instances succeeded

Time (s)

PFT

⟨

DL2

⟩

PFT

⟨

APRNN

⟩

PREPARED

1

10

100

600

1

10

100

600

100

×

10

×

1

×

0

.

1

×

PREPARED

time (s)

PFT

⟨

DL2

⟩

time (s)

1

10

100

600

1

10

100

600

100

×

10

×

1

×

0

.

1

×

PREPARED

time (s)

PFT

⟨

APRNN

⟩

time (s)

4548

62

66

0

5000

10000

Better

# instances succeeded

Time (s)

PFT

⟨

DL2

⟩

PFT

⟨

APRNN

⟩

PREPARED

1

10

100

1000

3600

1

10

100

1000

3600

100

×

10

×

1

×

0

.

1

×

PREPARED

time (s)

PFT

⟨

DL2

⟩

time (s)

1

10

100

1000

3600

1

10

100

1000

3600

100

×

10

×

1

×

0

.

1

×

PREPARED

time (s)

PFT

⟨

APRNN

⟩

time (s)

(b) Left : cactus plot of runtime for 66 all-properties editing problems, with 3600 seconds time limit for each problem. PREPARED succeeded on 62 problems, while PFT ⟨ DL2 ⟩ succeeded on 45, and PFT ⟨ APRNN ⟩ succeeded 48. Middle : speed up for PREPARED vs PFT ⟨ DL2 ⟩ . Right : speed up for PREPARED vs PFT ⟨ APRNN ⟩ .

Theorem 3.14. Definition 3.13 is a Parametric Linear Relaxation for the Affine layer with variable input bounds [x , x] that captures the exact upper and lower bounds.

3.6 Parametric Linear Relaxation for Tanh, Sigmoid and ELU layers

Our approach can also handle other activation layers like Tanh, Sigmoid and ELU. Figure 3 illustrates their Parametric Linear Relaxations with detailed description deferred to Appendix C.

3.7 On scalability

As described in Appendix B, our approach can restrict the edits to only the last k layers of the DNN N , freezing the parameters of the first L -k layers. Consequently, the resulting Parametric Linear Relaxation is a linear formula whose size is polynomial in the size of only the last k editable layers N ( k : L ) instead of the entire DNN N . This flexibility enables our approach to scale to large DNNs and BERT transformers, as demonstrated in Section 4.

4 Experimental evaluation

We have implemented PREPARED in PyTorch [32] and use Gurobi [17] as the LP solver. We demonstrate the effectiveness and efficiency of PREPARED on different tasks that use a wide-range of DNN architectures and properties. All experiments were run on a machine with Dual Intel Xeon Silver 4216 Processor 16-Core 2.1GHz with 384 GB of memory, SSD and RTX-A6000 with 48 GB of GPU memory running Ubuntu 20.04. Additional details about these experiments are in Appendix D.

Baselines. Because there were no efficient prior approaches for provable editing, we implemented provable fine-tuning PFT ⟨·⟩ baselines that combine prior (non-provable) DNN editing approaches with

a verifier in the loop: the editing stops when the verifier confirms that the DNN satisfies the property (Appendix D.1). We consider the state-of-the-art DNN editing approaches DL2 [10], APRNN [42], SABR [28] and STAPS [24] as introduced in Section 2. We use PFT ⟨ DL2 ⟩ to denote the provable finetuning baseline instantiated with DL2 . We use verifiers α,β -CROWN [46, 44], MN -BaB [9], or DeepT [4], depending on the task. To the best of our knowledge, ours is the first to provide a comprehensive evaluation of such verifier-in-the-loop baselines for provable editing.

4.1 Provable editing on VNN competition benchmarks

Setup. We compare PREPARED against PFT ⟨ DL2 ⟩ and PFT ⟨ APRNN ⟩ on VNN -COMP ′ 22 benchmarks [29]. The VNN -COMP ′ 22 benchmarks consist of DNNs along with one or more of their associated safety properties. For verification, the task would be to determine whether a DNN satisfies each of its properties. In the context of provable editing, we use these benchmarks in two scenarios: i) Single-property editing : Given a DNN and a property it violates, edit it to satisfy this single property with a time limit of 600 seconds. There are 423 such DNN-property instances. ii) All-properties editing : Given a DNN that violates at least one of its associated properties, edit it to satisfy the conjunction of all (satisfied or violated) properties associated with it with a time limit of 3600 seconds. There are 66 such DNN-property instances.

PFT ⟨ SABR ⟩ and PFT ⟨ STAPS ⟩ are not used because they do not handle all properties and DNN architectures in this experiment. In particular, SABR and STAPS have not discussed how to handle general logical formula with disjunctions, and their current implementations are designed for local robustness training. PFT ⟨ SABR ⟩ and PFT ⟨ STAPS ⟩ are used for local robustness editing in Section 4.2.

Metrics. The effectiveness is measured using the number of provably edited instances, and the efficiency is measured using the runtime. We were unable to determine the impact on the predictive performance (accuracy), because VNN -COMP does not come with such evaluation metrics and data.

Results. As shown in Figure 4, PREPARED is the best provable editing approach , significantly outperforming PFT ⟨ DL2 ⟩ and PFT ⟨ APRNN ⟩ in terms of both effectiveness and efficiency . As shown in the cactus plots ( Left in Figures 4(a) and 4(b)), PREPARED can provably edit more instances in less time. As shown in the speed-up plots ( Middle and Right in Figures 4(a) and 4(b)), PREPARED completes most problems in 10 seconds, taking a maximum of 45 seconds for single-property editing; it achieves from 10× to more than 100× speed-up over PFT ⟨ DL2 ⟩ and PFT ⟨ APRNN ⟩ . Specifically, PREPARED succeeds on all 423 single-property editing problems, while PFT ⟨ DL2 ⟩ and PFT ⟨ APRNN ⟩ succeed on 185 and 117 problems, respectively. PREPARED succeeds on 62 out of 66 multi-property editing problems, while PFT ⟨ DL2 ⟩ and PFT ⟨ APRNN ⟩ only succeed on 45 and 48 problems, respectively.

4.2 Local robustness editing for image-recognition DNNs

Definition 4.1. Consider a classifier DNN N : R m → R n and a dataset D of input-label pairs (x , l ) ∈ R m × Z . The DNN N correctly classifies an input-label pair (x , l ) iff arg max N (x) = l , and the standard accuracy of a DNN on a dataset D is the percentage of correctly classified input-label pairs in D . Given a perturbation ε ∈ R , the DNN N is L ∞ -locally-robust on an input-label pair (x , l ) iff N correctly classifies all perturbed inputs x ′ in the L ∞ box centered at x with perturbation ε , viz., ∀ x ′ ∈ R m . ∥ x ′ -x ∥ ∞ ≤ ε = ⇒ arg max N (x ′ ) = l . The certified accuracy of a DNN on a dataset D is the percentage of L ∞ -locally-robust input-label pairs in D .

Setup. We compare PREPARED against PFT ⟨ DL2 ⟩ , PFT ⟨ SABR ⟩ , and APRNN on L ∞ -local-robustness editing for image-recognition DNNs. We edit CNN7 DNNs for CIFAR10 [21] and TINYIMAGENET [33], trained in prior work [28], to be locally-robust for images in the edit set. The CIFAR10 DNN has 17.2M parameters, 79.24% standard accuracy and 75.77% certified accuracy ( ε = 0.5 / 255). The TinyImageNet DNN has 51.9M parameters, 28.85% standard accuracy and 20.46% certified accuracy ( ε = 1 / 255). For CIFAR10 and TINYIMAGENET, the edit sets consist of 50 misclassified fog-corrupted images from CIFAR10-C [18] and TINYIMAGENET-C [18], respectively. The generalization sets consist of 200 variant images with different corruption-levels, four variants per image in the edit set. We use MN -BaB [9] to compute the certified accuracy.

Metrics. Efficacy is measured using the certified accuracy (Definition 4.1) on the edit set. The original certified accuracy on the edit set is 0%. Efficacy is the most important metric: a provable edit must guarantee 100% efficacy. The standard accuracy and certified accuracy on the full

test set are also important metrics: a good provable edit should have high accuracy. However, those accuracy metrics are relevant only if efficacy is 100%. The generalization accuracy on the generalization set measures how well the edit generalizes to inputs that are similar to the edit set. However, the generalization accuracy is relevant only if the efficacy is 100% and the standard and certified accuracies are good; that is, the predictive power of the edited DNN should not be sacrificed for better generalization. The runtime metric measures the time taken to edit the DNN.

Results. As shown in Table 1, PREPARED is overall the best approach. PREPARED achieves the best standard and certified accuracy, good generalization and short runtime. In particular, PFT ⟨ SABR ⟩ and PFT ⟨ STAPS ⟩ have significantly lower standard and certified accuracy; PFT ⟨ DL2 ⟩ takes significantly longer time, and was unable to achieve 100% efficacy in four hours in the TINYIMAGENET experiment; APRNN was unable to achieve 100% efficacy.

4.3 Local robustness editing for sentiment classification BERT transformers

Setup. We compare PREPARED against PFT ⟨ DL2 ⟩ and APRNN on provable L ∞ local-robustness editing for BERT sentiment-classification transformers. We conduct this experiment on the Stanford Sentiment Treebank (SST) [40] DNNs. We use DeepT [4] as the verifier in this experiment. We take the 'wider' 12-layer BERT transformer used in the DeepT paper [4]. The standard accuracy of this network is 84 . 07% . Because DeepT cannot handle all sentences in the SST dataset with arbitrary ε , we construct two editing sets from the SST validation set: all 66 verifiable sentences with ε = 1e-4, where 60 of them are certified to be locally-robust; all 26 verifiable sentences with ε = 5e-4, where 24 of them are certified to be locally-robust. PFT ⟨ SABR ⟩ and PFT ⟨ STAPS ⟩ are not compared in this experiment because SABR and STAPS do not handle the BERT transformer architecture.

Metrics. Efficacy is measured using the certified accuracy (Definition 4.1) on the edit set. Because DeepT is incomplete, we can only compute a lower bound of the efficacy. Efficacy is the most important metric: a provable edit must guarantee 100% efficacy. The original efficacy (Og. Effic.) is also presented in Table 2. The standard accuracy on the full test set is also an important metric: a good provable edit should have high accuracy. However, the standard accuracy is relevant only if efficacy is 100%. The runtime metric measures the time taken to edit the DNN.

Results. As seen in Table 2, PREPARED is the only approach that achieves 100% efficacy in this task. PREPARED also achieves good accuracy and short runtime. Both PFT ⟨ DL2 ⟩ and APRNN were unable to achieve 100% efficacy, and decreased the efficacy in most experiments.

4.4 Provable training for physics-plausible DNNs

Numerical model. We consider a classic model of a buoyancy-driven mantle flow with a central circular plume [13], which is an incompressible flow with variable viscosity. The state of the system is a vector field F def = ( u, v, η, ρ ) of velocity field U def = ( u, v ) , viscosity η and density ρ , over space ( x, y ) ∈ Ω and time t ∈ T . We consider the following two physics constraints: i) Conservation of mass ∇· U = 0 from the continuity equation; ii) Dirichlet boundary condition U · n = 0 on the boundary ∂ Ω , where n is the outward normal of ∂ Ω .

Setup. We compare GD ⟨ PREPARED ⟩ , a combination of gradient-descent ( GD ) and PREPARED , against DL2 , GD and GD ⟨ APRNN ⟩ on the task of provably training a DNN to model the geodynamics process and satisfy physics constraints. We discretize the space into a 31×31 grid and implement the numerical model to generate data for 150 time steps. We evenly split the data into training, validation and test sets. We train a ReLU ResNet with one single residual block of four conv2d layers with 32 channels and 3×3 kernel size. For DL2 , we add the physics constraints as an unsupervised-learning regularization to the supervised training. For GD ⟨ APRNN ⟩ and GD ⟨ PREPARED ⟩ , we first use vanilla gradientdescent ( GD ) to train a base model, then edit it with the constraints. Specifically, GD ⟨ APRNN ⟩ edits all training points, and GD ⟨ PREPARED ⟩ edits the convex-hull of all training points to satisfy the constraints.

Metrics. The relative error on the test set, the constraint-satisfaction errors, viz., continuity error , for the conservation of mass, and boundary-condition error , and the runtime are the metrics.

Results. As shown in Table 3, GD ⟨ PREPARED ⟩ is overall the best approach. GD ⟨ PREPARED ⟩ has good relative error, and its constraint-satisfaction error is significantly better than other approaches. Notably, its continuity error is at the same magnitude as the reference data, close to the theoretical value, zero, and is 10 5 to 10 8 times better than other approaches. GD achieves the best relative error and the shortest runtime. DL2 has the worst errors and takes significantly longer amount of time.

5 Conclusion

We have presented PREPARED , the first efficient approach for provable editing of DNNs that runs in polynomial time. We presented novel methods for constructing tight parametric bounds for the DNN output using Parametric Linear Relaxation, enabling PREPARED to use an LP solver to find an edit. To demonstrate its effectiveness and efficiency, PREPARED was used to provably edit DNNs and properties from the VNN -COMP ′ 22 benchmark, provably edit CIFAR10 and TINYIMAGENET imagerecognition DNNs, and BERT sentiment-classification DNNs for local robustness, and provably train a geodynamics DNN to satisfy physics constraints.

Societal impact. PREPARED represents a step towards the goal of ensuring safety, trustworthiness, and reliability of DNNs, which is crucial given their use in autonomous safety-critical systems. However, being a general technique for editing DNNs, it could be used to remove or add malicious behavior such as bias, backdoor attacks, etc.

Limitations. PREPARED requires as input a property that we wish the DNN to satisfy. These properties would need to formally encode the notions of safety, reliability, or trustworthiness, which may not always be possible or easy to do. e.g., defining safety properties for LLM-based chatbots. However, the machine learning and formal methods communities are making progress towards this goal; e.g., developing proxy specifications for complex properties that are easier to define and learning safety specifications from data [8].

Acknowledgments and Disclosure of Funding

We would like to thank the anonymous reviewers for their feedback and suggestions, which have greatly improved the quality of the paper. This work is supported in part by NSF grant CCF-2048123 and DOE Award DE-SC0022285.

References

Aws Albarghouthi. Introduction to neural network verification. Found. Trends Program. Lang. , 7(1-2): 1-157, 2021. doi: 10.1561/2500000051. URL https://doi.org/10.1561/2500000051 .

Mislav Balunovic and Martin Vechev. Adversarial training and provable defenses: Bridging the gap. In International Conference on Learning Representations , 2020. URL https://openreview.net/forum? id=SJxSDxrKDr .

Gregory Bonaert, Dimitar I. Dimitrov, Maximilian Baader, and Martin T. Vechev. Fast and precise certification of transformers. https://github.com/eth-sri/DeepT/tree/ 16ffe4075f1f8a7c87fa2a187d8c46cfd51e07bf , 2021.

Gregory Bonaert, Dimitar I. Dimitrov, Maximilian Baader, and Martin T. Vechev. Fast and precise certification of transformers. In Stephen N. Freund and Eran Yahav, editors, PLDI '21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 2021 , pages 466-481. ACM, 2021. doi: 10.1145/3453483.3454056. URL https://doi.org/10.1145/3453483.3454056 .

Christopher Brix, Mark Niklas Müller, Stanley Bak, Taylor T. Johnson, and Changliu Liu. First three years of the international verification of neural networks competition (vnn-comp), 2023.

George E. Collins. Quantifier elimination for real closed fields by cylindrical algebraic decompostion. In H. Brakhage, editor, Automata Theory and Formal Languages , pages 134-183, Berlin, Heidelberg, 1975. Springer Berlin Heidelberg. ISBN 978-3-540-37923-2.

George E. Collins and Hoon Hong. Partial cylindrical algebraic decomposition for quantifier elimination. Journal of Symbolic Computation , 12(3):299-328, 1991. ISSN 0747-7171. doi: https://doi.org/10. 1016/S0747-7171(08)80152-6. URL https://www.sciencedirect.com/science/article/pii/ S0747717108801526 .

David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barret, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, and Joshua Tenenbaum. Towards guaranteed safe ai: A framework for ensuring robust and reliable AI systems. arXiv preprint arXiv:2405.06624 , 2024.

Claudio Ferrari, Mark Niklas Müller, Nikola Jovanovi´ c, and Martin Vechev. Complete verification via multineuron relaxation guided branch-and-bound. In International Conference on Learning Representations , 2022.

Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin T. Vechev. DL2: training and querying neural networks with logic. In Proceedings of the 36th International Conference on Machine Learning (ICML) , volume 97 of Proceedings of Machine Learning Research . PMLR, 2019.

Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin T. Vechev. DL2: Training and querying neural networks with logic. https://github.com/eth-sri/dl2/tree/ 05e2d3fb4e2b00ef5968bf28d26219750a580908 , 2023.

Feisi Fu and Wenchao Li. Sound and complete neural network repair with minimality and locality guarantees. In 10th International Conference on Learning Representations, ICLR 2022, Lisbon, Portugal, Oct 27-28, 2022 . OpenReview.net, 2022. URL https://arxiv.org/abs/2110.07682 .

Taras Gerya. Introduction to numerical geodynamic modelling . Cambridge University Press, 2019.

Ben Goldberger, Guy Katz, Yossi Adi, and Joseph Keshet. Minimal modifications of deep neural networks using verification. In Elvira Albert and Laura Kovács, editors, LPAR 2020: 23rd International Conference on Logic for Programming, Artificial Intelligence and Reasoning, Alicante, Spain, May 22-27, 2020 , volume 73 of EPiC Series in Computing , pages 260-278. EasyChair, 2020. doi: 10.29007/699q. URL https://doi.org/10.29007/699q .

Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models, 2019.

Kshitij Goyal, Sebastijan Dumancic, and Hendrik Blockeel. Deepsade: Learning neural networks that guarantee domain constraint satisfaction. Proceedings of the AAAI Conference on Artificial Intelligence , 38 (11):12199-12207, Mar. 2024. doi: 10.1609/aaai.v38i11.29109. URL https://ojs.aaai.org/index. php/AAAI/article/view/29109 .

Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2022. URL https://www.gurobi. com .

Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview.net/forum? id=HJz6tiCqYm .

Nick Hoernle, Rafael Michael Karampatsis, Vaishak Belle, and Kobi Gal. Multiplexnet: Towards fully satisfied logical constraints in neural networks. Proceedings of the AAAI Conference on Artificial Intelligence , 36(5):5700-5709, Jun. 2022. doi: 10.1609/aaai.v36i5.20512. URL https: //ojs.aaai.org/index.php/AAAI/article/view/20512 .

Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing. Harnessing deep neural networks with logic rules. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 2410-2420, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v1/P16-1228. URL https://aclanthology.org/P16-1228 .

Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). URL http://www.cs.toronto.edu/~kriz/cifar.html .

Zenan Li, Zehua Liu, Yuan Yao, Jingwei Xu, Taolue Chen, Xiaoxing Ma, and Jian L\'{u}. Learning with logical constraints but without shortcut satisfaction. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=M2unceRvqhh .

Dmitry Malioutov and Kuldeep S Meel. Mlic: A maxsat-based framework for learning interpretable classification rules. In International Conference on Principles and Practice of Constraint Programming , pages 312-327. Springer, 2018.

Yuhao Mao, Mark Niklas Mueller, Marc Fischer, and Martin Vechev. Connecting certified and adversarial training. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. URL https: //openreview.net/forum?id=T2lM4ohRwb .

Yuhao Mao, Mark Niklas Mueller, Marc Fischer, and Martin Vechev. TAPS: Connecting certified and adversarial training. https://github.com/eth-sri/TAPS/tree/ 6b5c5d9d3453c482f56f00e6f57b99b07b11ca4f , 2023.

Matthew Mirman, Timon Gehr, and Martin T. Vechev. Differentiable abstract interpretation for provably robust neural networks. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 , volume 80 of Proceedings of Machine Learning Research , pages 3575-3583. PMLR, 2018. URL http://proceedings.mlr.press/v80/mirman18b.html .

Mark Niklas Mueller, Franziska Eckert, Marc Fischer, and Martin Vechev. SABR: Small adversarial bounding boxes. https://github.com/eth-sri/sabr , 2023.

Mark Niklas Mueller, Franziska Eckert, Marc Fischer, and Martin Vechev. Certified training: Small boxes are all you need. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=7oFuxtJtUMH .

Mark Niklas Müller, Christopher Brix, Stanley Bak, Changliu Liu, and Taylor T. Johnson. The third international verification of neural networks competition (vnn-comp 2022): Summary and results, 2022. URL https://arxiv.org/abs/2212.10376 .

Yatin Nandwani, Abhishek Pathak, Mausam, and Parag Singla. A primal dual formulation for deep learning with constraints. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/ cf708fc1decf0337aded484f8f4519ae-Paper.pdf .

Alessandro De Palma, Rudy Bunel, Krishnamurthy Dvijotham, M. Pawan Kumar, and Robert Stanforth. Ibp regularization for verified adversarial robustness via branch-and-bound, 2023.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. CoRR , abs/1912.01703, 2019. URL http://arxiv.org/abs/1912.01703 .

Hadi Pouransari and Saman Ghili. Tiny imagenet visual recognition challenge. CS 231N , 7, 2014.

Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. A convex relaxation barrier to tight robustness verification of neural networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , pages 9832-9842, 2019. URL https://proceedings. neurips.cc/paper/2019/hash/246a3c5544feb054f3ea718f61adfa16-Abstract.html .

Sanjit A. Seshia, Dorsa Sadigh, and S. Shankar Sastry. Toward verified artificial intelligence. Commun. ACM , 65(7):46-55, 2022. doi: 10.1145/3503914. URL https://doi.org/10.1145/3503914 .

Zhouxing Shi, Yihan Wang, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. Fast certified robust training with short warmup. In ICML 2021 Workshop on Adversarial Machine Learning , 2021. URL https: //openreview.net/forum?id=_jUobmvki51 .

Zhouxing Shi, Qirui Jin, J Zico Kolter, Suman Jana, Cho-Jui Hsieh, and Huan Zhang. Formal verification for neural networks with general nonlinearities via branch-and-bound. 2023.

Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin T. Vechev. Fast and effective robustness certification. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , pages 10825-10836, 2018. URL https://proceedings.neurips.cc/paper/ 2018/hash/f2f446980d8e971ef3da97af089481c3-Abstract.html .

Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. An abstract domain for certifying neural networks. Proc. ACM Program. Lang. , 3(POPL):41:1-41:30, 2019. doi: 10.1145/3290354. URL https://doi.org/10.1145/3290354 .

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing , pages 1631-1642, 2013.

Matthew Sotoudeh and Aditya Thakur. Prdnn. https://github.com/95616ARG/PRDNN , 2021.

Zhe Tao, Stephanie Nawas, Jacqueline Mitchell, and Aditya V. Thakur. Architecture-preserving provable repair of deep neural networks. Proc. ACM Program. Lang. , 7(PLDI), jun 2023. doi: 10.1145/3591238. URL https://doi.org/10.1145/3591238 .

Christian Tjandraatmadja, Ross Anderson, Joey Huchette, Will Ma, Krunal Patel, and Juan Pablo Vielma. The convex relaxation barrier, revisited: Tightened single-neuron relaxations for neural network verification. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, MariaFlorina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , 2020. URL https://proceedings.neurips.cc/paper/2020/hash/ f6c2a0c4b566bc99d596e58638e342b0-Abstract.html .

Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J. Zico Kolter. Betacrown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 29909-29921, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/ fac7fead96dafceaf80c1daffeae82a4-Abstract.html .

Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. A semantic loss function for deep learning with symbolic knowledge. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 5502-5511. PMLR, 10-15 Jul 2018. URL https://proceedings.mlr.press/v80/ xu18h.html .

Kaidi Xu, Huan Zhang, Shiqi Wang, Yihan Wang, Suman Jana, Xue Lin, and Cho-Jui Hsieh. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. URL https://openreview.net/forum?id=nVZtXBI6LNn .

Ziwei Xu, Yogesh Rawat, Yongkang Wong, Mohan S Kankanhalli, and Mubarak Shah. Don't pour cereal into coffee: Differentiable temporal logic for temporal action segmentation. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 14890-14903. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ 5f96a21345c138da929e99871fda138e-Paper-Conference.pdf .

Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , pages 4944-4953, 2018. URL https://proceedings.neurips.cc/paper/2018/hash/ d04863f100d59b3eb688a11f95b0ae60-Abstract.html .

Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems , 31, 2018.

Huan Zhang, Zhouxing Shi, Kaidi Xu, Yihan Wang, Shiqi Wang, Linyi Li, Jinqi (Kathryn) Chen, and Zhuolin Yang. auto_LiRPA: An automatic linear relaxation based perturbation analysis library for neural networks and general computational graphs. https://github.com/Verified-Intelligence/auto_ LiRPA , 2023.

A Proofs

Theorem 3.4. Given a provable interval editing problem (Definition 3.2) for DNN N and parameters θ with input bounds [x , x] and output bounds [y , y] , let φ N ( y , y , x , x; θ ) be a Parametric Linear Relaxation for N . Then the following linear program can be solved in polynomial time in the size of the DNN N , and whose solution is a solution to the provable interval editing problem:

Proof. Equation 4 is a linear program that can be solved in polynomial in the size of the DNN N because by Definition 3.3, φ N ( y , y , x , x; θ ) is a poly-size linear formula whose size is polynomial in the size of the DNN N , i.e., the number of parameters, layers, and the input and output dimensions of each layer.

Equation 4 is a solution to the provable interval editing problem (Definition 3.2) because by Definition 3.3, φ N ( y , y , x , x; θ ) implies ∀ x ∈ [x , x] . N ( x; θ ) ∈ [y , y] .

Theorem 3.6. Definition 3.5 is a Parametric Linear Relaxation for the DNN N .

Proof. Recall Definition 3.5:

where [y , y] def = [x ( L ) , x ( L ) ] and [x (0) , x (0) ] def = [x , x] . Our goal is to prove that φ N ( y , y , x , x; θ ) is a poly-size linear formula that implies

We first show that φ N ( y , y , x , x; θ ) is a poly-size linear formula. For Parametric Linear Relaxation φ N ( ℓ ) ( x ( ℓ +1) , x ( ℓ +1) , x ( ℓ ) , x ( ℓ ) ; θ ( ℓ ) ) for any layer N ( ℓ ) , by Definition 3.3, we have φ N ( ℓ ) ( x ( ℓ +1) , x ( ℓ +1) , x ( ℓ ) , x ( ℓ ) ; θ ( ℓ ) ) is a poly-size linear formula whose size is polynomial in the number of parameters, input and output dimensions of layer N ( ℓ ) . Therefore, we have φ N ( y , y , x , x; θ ) is a poly-size linear formula whose size is polynomial in the number of parameters, layers, and the input and output dimensions of each layer.

Then we show that φ N ( y , y , x , x; θ ) implies ∀ x ∈ [x , x] . N ( x; θ ) ∈ [y , y] . By Definition 3.3, we have that the Parametric Linear Relaxation φ N ( ℓ ) ( x ( ℓ +1) , x ( ℓ +1) , x ( ℓ ) , x ( ℓ ) ; θ ( ℓ ) ) for any layer N ( ℓ ) implies the following formula:

Therefore, we have φ N ( y , y , x , x; θ ) def = ∧ 0 ≤ ℓ<L φ N ( ℓ ) ( x ( ℓ +1) , x ( ℓ +1) , x ( ℓ ) , x ( ℓ ) ; θ ( ℓ ) ) implies the following formula:

where [y , y] def = [x ( L ) , x ( L ) ] and [x (0) , x (0) ] def = [x , x] . By induction, the formula above implies ∀ x ∈ [x , x] . N ( x; θ ) ∈ [y , y] . Therefore, we proved that φ N ( y , y , x , x; θ ) is a poly-size linear formula that implies ∀ x ∈ [x , x] . N ( x; θ ) ∈ [y , y] .

A.1 Proofs for Parametric Linear Relaxation of ReLU layers

Lemma A.1. φ ReLU (y , y , x , x) defined in Definition 3.8 is a poly-size linear formula.

Proof. As seen in Definition 3.8, φ ReLU (y , y , x , x) is a linear formula. Let m be the number of input dimensions of ReLU. φ ReLU (y , y , x , x) has 3 m linear constraints over 4 m variables ( x ∈ R m , x ∈ R m , y ∈ R m and y ∈ R m ). Hence, φ ReLU (y , y , x , x) is a poly-size linear formula whose size is polynomial in the number of input dimensions of the ReLU layer.

Lemma A.2. φ ReLU (y , y , x , x) def = φ ReLU (y , x) ∧ φ ReLU (y , x) defined in Definition 3.8 implies ∀ x ∈ [x , x] . ReLU (x) ∈ [y , y] .

Proof. For the upper bound , recall that by Definition 3.8, we have

Our goal is to prove that φ ReLU (y , x) implies

By the definition of the ReLU layer (Definition 3.7), we expand the formula above to

Because max { x , 0 } increases monotonically with respect to x , the above formula is equivalent to

Because max { x , 0 } is a convex function, the above formula is equivalent to

which is the definition of φ ReLU (y , x) in Definition 3.8. Therefore, we proved that φ ReLU (y , x) implies ∀ x ∈ [x , x] . y ≥ ReLU (x) .

For the lower bound , recall that by Definition 3.8, we have

where c ∈ R m are constants such that 0 ≤ c i ≤ 1 . Our goal is to prove that φ ReLU (y , x) implies

By the definition of the ReLU layer (Definition 3.7), we expand the formula above to

Because max { x , 0 } increases monotonically with respect to x , the above formula is equivalent to

Given constant vector c where 0 ≤ c ≤ 1 , φ ReLU (y , x) def = y = cx implies the above formula. Therefore, we proved that φ ReLU (y , x) implies ∀ x ∈ [x , x] . y ≤ ReLU (x) .

which is equivalent to which is equivalent to

which is equivalent to

Lemma A.3. φ ReLU (y , y , x , x) def = φ ReLU (y , x) ∧ φ ReLU (y , x) defined in Definition 3.8 is implied by y = max { ReLU (x) ∣ ∣ x ≤ x ≤ x } , hence captures the exact upper bound.

Proof. Recall that by Definition 3.8, we have

For any solution (y ∗ , x , x) to y = max { ReLU (x) ∣ ∣ x ≤ x ≤ x } , we have

By the definition of ReLU layer (Definition 3.7), because ReLU is a monotonically increasing function, the definition above is equivalent to

To prove that φ ReLU (y , x) captures the exact upper bound, we will show that (y ∗ , x) is a solution to φ ReLU (y , x) . In other words, the following instantiated formula φ ReLU (y ∗ , x) is true:

By substituting y ∗ defined in Equation 9 above, our goal becomes

which is true. Hence, we have proved that φ ReLU (y , y , x , x) captures the exact upper bound.

Theorem 3.9. Definition 3.8 is a Parametric Linear Relaxation for the ReLU layer that captures the exact upper bound.

Proof. By Lemma A.1 and Lemma A.2, we proved that φ ReLU (y , y , x , x) is a poly-size linear formula that implies ∀ x ∈ [x , x] . ReLU (x) ∈ [y , y] , hence is a Parametric Linear Relaxation (Definition 3.3) for ReLU. By Lemma A.3, we proved that φ ReLU (y , y , x , x) is implied by y = max { ReLU (x) ∣ ∣ x ≤ x ≤ x } , hence captures the exact upper bound.

A.2 Proofs for Parametric Linear Relaxation of Affine layers with constant input bounds

Lemma A.4. φ Aff (y , y , x , x , W , b) defined in Definition 3.11 is a poly-size linear formula.

Proof. As seen in Definition 3.11, φ Aff (y , y , x , x , W , b) is a linear formula. Let m and n be the number of input and output dimensions of the Affine layer. φ Aff (y , y , x , x , W , b) has 2 n + 4 mn linear constraints over 2 m +3 n +3 mn variables ( x ∈ R m , x ∈ R m , b ∈ R n , y ∈ R n , y ∈ R n , W ∈ R n × m , V ∈ R n × m and V ∈ R n × m ). Hence, φ Aff (y , y , x , x , W , b) is a poly-size linear formula whose size is polynomial in the number of input and output dimensions of the Affine layer.

Proof. For the upper bound , recall that by Definition 3.11, we have

our goal is to prove that φ Aff (y , x , x , W , b) implies

We prove the above formula by proving for each y j :

By the definition of the Affine layer (Definition 3.10), we expand the formula above to

which is implied by the following formula

Because φ Aff (y , x , x , W , b) implies y j = ∑ m i =0 V ji +b j , by substituting the left-hand-side y j of the formula above with ∑ m i =0 V ji +b j , our goal becomes:

By subtracting b j from both sides, our goal becomes:

Then we prove the above formula by proving for each i :

Because x i W ji changes monotonically with respect to x i , max { x i W ji ∣ ∣ x i ≤ x i ≤ x i } is taken when x i = x i or x i = x i . In other words, max { x i W ji ∣ ∣ x i ≤ x i ≤ x i } = max { x i W ji , x i W ji } . By substituting the right-hand-side of the above formula, our goal becomes:

The formula above is equivalent to V ji ≥ x i W ji ∧ V ji ≥ x i W ji , which is implied by φ Aff (y , x , x , W , b) . Therefore, we have proved that φ Aff (y , x , x , W , b) implies ∀ x ∈ [x , x] . y ≥ Affine (x;W , b) .

For the lower bound , recall that by Definition 3.11, we have

our goal is to prove that φ Aff (y , x , x , W , b) implies

by proving for each j where y j :

By the definition of the Affine layer (Definition 3.10), we expand the formula above to

which is implied by the following formula

Because φ Aff (y , x , x , W , b) implies y j = ∑ m i =0 V ji +b j , by substituting the left-hand-side y j of the formula above with ∑ m i =0 V ji +b j , our goal becomes:

By subtracting b j from both sides, our goal becomes:

Then we will prove for each i :

Because x i W ji changes monotonically with respect to x i , min { x i W ji ∣ ∣ x i ≤ x i ≤ x i } is taken when x i = x i or x i = x i . In other words, min { x i W ji ∣ ∣ x i ≤ x i ≤ x i } = min { x i W ji , x i W ji } . By substituting the right-hand-side of the above formula, our goal becomes:

The formula above is equivalent to V ji ≤ x i W ji ∧ V ji ≤ x i W ji , which is implied by φ Aff (y , x , x , W , b) . Therefore, we have proved that φ Aff (y , x , x , W , b) implies ∀ x ∈ [x , x] . y ≤ Affine (x;W , b) .

Lemma A.6. φ Aff (y , y , x , x , W , b) def = φ Aff (y , x , x , W , b) ∧ φ Aff (y , x , x , W , b) defined in Definition 3.11 is implied by y = max { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } ∧ y = min { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , hence captures the exact upper and lower bounds.

Proof. For the upper bound , recall that by Definition 3.11, we have

Given constant input bounds [x , x] , for any solution (y ∗ , W , b) to y = max { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , we have the exact constant output upper bound defined as

By the definition of Affine layer (Definition 3.10), it is equivalent to

To prove that φ Aff captures the exact upper bound, we will show that by substituting (y ∗ , x , x , W , b) in φ Aff (y , x , x , W , b) , the instantiated formula φ Aff (y ∗ , x , x , W , b) , as shown below, is satisfiable:

which is equivalent to prove the following formula for any index j :

By substituting y ∗ j defined in Equation 10 above, our goal becomes

which is implied by

If W ji ≥ 0 , V ∗ ji def = W ji x i is a solution to this formula; otherwise V ∗ ji def = W ji x i is a solution to this formula. Hence, we have proved that φ Aff (y ∗ , x , x , W , b) is satisfiable and φ Aff (y , x , x , W , b) captures the exact upper bound.

For the lower bound , recall that by Definition 3.11, we have

Given constant input bounds [x , x] , for any solution (y ∗ , W , b) to y = min { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , we have the exact constant output lower bound defined as

By the definition of Affine layer (Definition 3.10), it is equivalent to

To prove that φ Aff captures the exact lower bound, we will show that by substituting (y ∗ , x , x , W , b) in φ Aff (y , x , x , W , b) , the instantiated formula φ Aff (y ∗ , x , x , W , b) , as shown below, is satisfiable:

which is equivalent to prove the following formula for any index j :

By substituting y ∗ j defined in Equation 11 above, our goal becomes

which is implied by

If W ji ≥ 0 , V ∗ ji def = W ji x i is a solution to this formula; otherwise V ∗ ji def = W ji x i is a solution to this formula. Hence, we have proved that φ Aff (y ∗ , x , x , W , b) is satisfiable and φ Aff (y , x , x , W , b) captures the exact lower bound.

Theorem 3.12. Definition 3.11 is a Parametric Linear Relaxation for the Affine layer with constant input bounds [x , x] that captures the exact lower and upper bounds.

Proof. By Lemma A.4 and Lemma A.5, we proved that φ Aff (y , y , x , x , W , b) is a poly-size linear formula that implies ∀ x ∈ [x , x] . Affine ( x;W , b ) ∈ [y , y] , hence is a Parametric Linear Relaxation (Definition 3.3) for the Affine layer. By Lemma A.6, we proved that φ Aff (y , y , x , x , W , b) is implied by y = max { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } ∧ y = min { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , hence captures the exact upper and lower bounds.

A.3 Proofs for Parametric Linear Relaxation of Affine layers with variable input bounds

Lemma A.7. φ Aff (y , y , x , x , W , b) defined in Definition 3.13 is a poly-size linear formula.

Proof. As seen in Definition 3.13, φ Aff (y , y , x , x , W , b) is a linear formula. Let m and n be the number of input and output dimensions of the Affine layer. φ Aff (y , y , x , x , W , b) has 2 n linear constraints over 2 m + 3 n variables ( x ∈ R m , x ∈ R m , b ∈ R n , y ∈ R n and y ∈ R n ). Hence, φ Aff (y , y , x , x , W , b) is a poly-size linear formula whose size is polynomial in the number of input and output dimensions of the Affine layer.

Lemma A.8. φ Aff (y , y , x , x , W , b) def = φ Aff (y , x , x , W , b) ∧ φ Aff (y , x , x , W , b) defined in Definition 3.13 implies that ∀ x ∈ [x , x] . Affine ( x;W , b ) ∈ [y , y] .

Proof. For the upper bound , recall that by Definition 3.13, we have

Our goal is to prove that φ Aff (y , x , x , W , b) implies

We prove the above formula by proving for each y j :

By the definition of the Affine layer (Definition 3.10), we expand the formula above to

which is implied by the following formula

By Definition 3.13, φ Aff (y , x , x , W , b) implies y j = ∑ m i =0 V ji +b j . By substituting the left-handside y j of the formula above with ∑ m i =0 V ji +b j , our goal becomes:

By subtracting b j from both sides, our goal becomes:

Then we prove the above formula by proving for each i :

For the left-hand-side, we have V ji def = { x i W ji W ji ≥ 0 x i W ji otherwise . For the right-hand-side, because x i W ji changes monotonically with respect to x i , and W ji is a given constant, we

have max { x i W ji ∣ ∣ x i ≤ x i ≤ x i } = { x i W ji W ji ≥ 0 x i W ji otherwise , which is the same as the definition of the left-hand-side V ji . Thus, we have proved that φ Aff (y , x , x , W , b) implies ∀ x ∈ [x , x] . y ≥ Affine (x;W , b) .

For the lower bound , recall that by Definition 3.13, we have

Our goal is to prove that φ Aff (y , x , x , W , b) implies

We prove the above formula by proving for each y j :

By the definition of the Affine layer (Definition 3.10), we expand the formula above to

which is implied by the following formula

By Definition 3.13, φ Aff (y , x , x , W , b) implies y j = ∑ m i =0 V ji +b j . By substituting the left-handside y j of the formula above with ∑ m i =0 V ji +b j , our goal becomes:

By subtracting b j from both sides, our goal becomes:

Then we prove the above formula by proving for each i :

For the left-hand-side, we have V ji def = { x i W ji W ji ≥ 0 x i W ji otherwise . For the right-hand-side, because x i W ji changes monotonically with respect to x i , and W ji is a given constant, we have min { x i W ji ∣ ∣ x i ≤ x i ≤ x i } = { x i W ji W ji ≥ 0 x i W ji otherwise , which is the same as the definition of the left-hand-side V ji . Thus, we have proved that φ Aff (y , x , x , W , b) implies ∀ x ∈ [x , x] . y ≤ Affine (x;W , b) .

Lemma A.9. φ Aff (y , y , x , x , W , b) def = φ Aff (y , x , x , W , b) ∧ φ Aff (y , x , x , W , b) defined in Definition 3.13 is implied by y = max { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } ∧ y = min { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , hence captures the exact upper and lower bounds.

Proof. For the upper bound , recall that by Definition 3.13, we have

For any solution (y ∗ , x , x , b) to y = max { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , we have the exact constant output upper bound defined as

By the definition of Affine layer (Definition 3.10), it is equivalent to

To prove that φ Aff captures the exact upper bound, we will show that (y ∗ , x , x , b) is a solution to φ Aff (y , x , x , W , b) . In other words, the following instantiated formula φ Aff (y ∗ , x , x , W , b) is true:

where V ji def =x j W ji if W ji ≥ 0 , otherwise V ji def =x j W ji .

This formula is equivalent to prove the following formula for any index j :

By substituting y ∗ j defined in Equation 12 above, our goal becomes

which is implied by

If W ji ≥ 0 , max { x i W ji ∣ ∣ x ≤ x ≤ x } = x j W ji = V ji , the above formula is true; Otherwise, max { x i W ji ∣ ∣ x ≤ x ≤ x } =x j W ji =V ji , the above formula is true. Hence, we have proved that φ Aff (y ∗ , x , x , W , b) is true and φ Aff (y , x , x , W , b) captures the exact upper bound.

For the lower bound , recall that by Definition 3.13, we have

For any solution (y ∗ , x , x , b) to y = min { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , we have the exact constant output lower bound defined as

By the definition of Affine layer (Definition 3.10), it is equivalent to

To prove that φ Aff captures the exact lower bound, we will show that (y ∗ , x , x , b) is a solution to φ Aff (y , x , x , W , b) . In other words, the following instantiated formula φ Aff (y ∗ , x , x , W , b) is true:

This formula is equivalent to prove the following formula for any index j :

By substituting y ∗ j defined in Equation 13 above, our goal becomes

If W ji ≥ 0 , min { x i W ji ∣ ∣ x ≤ x ≤ x } = x j W ji = V ji , the above formula is true; Otherwise, min { x i W ji ∣ ∣ x ≤ x ≤ x } =x j W ji =V ji , the above formula is true. Hence, we have proved that φ Aff (y ∗ , x , x , W , b) is true and φ Aff (y , x , x , W , b) captures the exact lower bound.

Theorem 3.14. Definition 3.13 is a Parametric Linear Relaxation for the Affine layer with variable input bounds [x , x] that captures the exact upper and lower bounds.

Proof. By Lemma A.7 and Lemma A.8, we proved that φ Aff (y , y , x , x , W , b) is a poly-size linear formula that implies ∀ x ∈ [x , x] . Affine ( x;W , b ) ∈ [y , y] , hence is a Parametric Linear Relaxation (Definition 3.3) for the Affine layer. By Lemma A.9, we proved that φ Aff (y , y , x , x , W , b) is implied by y = max { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } ∧ y = min { Affine ( x;W , b ) ∣ ∣ x ≤ x ≤ x } , hence captures the exact upper and lower bounds.

B General provable editing via Parametric Linear Relaxation

In this section we present our approach for solving the general provable editing problem (Definition 1.1):

Definition B.1. Given Parametric Linear Relaxation φ N ( y , y , x , x; θ ) for DNN N with parameters θ , an input polytope P def = { x | Dx ≤ e } , an output polytope Q def = { y | Ay ≤ b } . Given a hyperparameter k so that we only edit the DNN layers N ( k : L ) starting from the k th layer. Then a solution to the following linear program is a solution to the provable editing problem of Definition 1.1:

where a sound constant interval bound [ x ( k ) , x ( k ) ] for the input x ( k ) to the slice N ( k : L ) to edit is computed by a sound bound propagation tool of DNNs like DeepPoly [39], auto _ LiRPA [50] and DeepT [4]:

The output constraint ψ (y , y , A , b) encodes Ay ≤ b for all y ∈ [ y , y ] using the parametric output bounds [ y , y ] is defined as follows:

which is implied by

B.1 Provable editing for multiple properties via Parametric Linear Relaxation

Given n properties defined by input polytopes P i def = { x | i Dx ≤ i e } , and output polytopes Q i def = { y | i Ay ≤ i b } . This work can directly generalize to provably edit multiple properties:

In particular, most of our experiments in Section 4 involve editing DNNs for multiple properties altogether. Here we present our formulation for provably editing multiple properties, which encodes the conjunction of the constraints for each property.

where [ i x ( k ) , i x ( k ) ] def = Compute _ Bound N (0: k ) ( P i ; θ (0: k ) ) .

B.2 Provable editing for disjunctive properties via Parametric Linear Relaxation

Given a property defined by an input polytope P def = { x | Dx ≤ e } , and n possible output polytopes Q j def = { y | j Ay ≤ j b } , this work can directly generalize to provably edit disjunctive property:

by using a mixed-integer linear programming (MILP) solver. In particular, some properties in VNN -COMP (Section 4.1) involve editing DNNs for such disjunctive properties. Here we present our formulation:

where [ x ( k ) , x ( k ) ] def = Compute _ Bound N (0: k ) ( P; θ (0: k ) ) .

C Parametric Linear Relaxation for Tanh, Sigmoid and ELU layers

C.1 Parametric Linear Relaxation for Tanh layers

Definition C.1. y def = Tanh (x) with input x ∈ R m and output y ∈ R m is defined as y i def = tanh(x i ) .

Definition C.2. For Tanh layer with variable input bounds [x , x] , its Parametric Linear Relaxation is defined as φ Tanh (y , y , x , x) def = φ Tanh (y , x) ∧ φ Tanh (y , x) where:

where x u ∈ R m , a u ∈ R m , b u ∈ R m are constants that satisfy the following two conditions: i) let f ( x ) = a u i x +b u i , the line defined by function f ( x ) is tangent to the concave piece of tanh at some x ≥ 0 ; ii) a u x u +b u = Tanh (x u ) ; viz., the line defined by function f ( x ) intersects with convex piece of tanh at x u i where x u i ≤ 0 .

Similarly, x l ∈ R m , a l ∈ R m , b l ∈ R m are constants that satisfy the following two conditions: i) let f ( x ) = a l i x +b l i , the line defined by function f ( x ) is tangent to the convex piece of tanh at some x ≤ 0 ; ii) a l x l +b l = Tanh (x l ) ; viz., the line defined by function f ( x ) intersects with concave piece of tanh at x l i where x l i ≥ 0 . ■

Theorem C.3. Definition C.2 is a Parametric Linear Relaxation for the Tanh layer.

Proof. As seen in Definition C.2, φ Tanh (y , y , x , x) is a linear formula. Let m be the number of input dimensions of Tanh. φ Tanh (y , y , x , x) has 4 m linear constraints over 4 m variables ( x ∈ R m , x ∈ R m , y ∈ R m and y ∈ R m ). Hence, φ Tanh (y , y , x , x) is a poly-size linear formula, whose size is polynomial in the number of input dimensions of the Tanh layer.

Now our goal is to prove that φ Tanh (y , y , x , x) defined in Definition C.2 implies

We will discuss the upper and lower bounds separately.

For the upper bound constraint φ Tanh (y , x) def =y ≥ Tanh (x u ) ∧ y ≥ a u x + b u , for the case when x i ≤ x u i , we have y i ≥ tanh(x u i ) ≥ tanh(x i ) because tanh is monotonically increasing. For the case when x i > x u i , we have y i ≥ a u i x i +b u i ≥ tanh(x i ) because f ( x ) = a u i x +b u i is tangent to the concave piece of tanh in [0 , ∞ ] , and tanh is monotonically increasing in [x u i , 0] . Therefore, φ Tanh (y , x) defined in Definition C.2 implies that ∀ x ∈ [x , x] . Tanh (x) ≤ y .

For the lower bound constraint φ Tanh (y , x) def =y ≤ Tanh (x l ) ∧ y ≤ a l x + b l , for the case when x i ≥ x l i , we have y i ≤ tanh(x l i ) ≤ tanh(x i ) because tanh is monotonically increasing. For the case when x i < x l i , we have y ≤ a l i x i +b l i ≤ tanh(x i ) because f ( x ) = a l i x +b l i is tangent to the convex piece of tanh in [ -∞ , 0] , and tanh is monotonically increasing in [0 , x l i ] . Therefore, φ Tanh (y , x) defined in Definition C.2 implies that ∀ x ∈ [x , x] . y ≤ Tanh (x) .

C.2 Parametric Linear Relaxation for Sigmoid layers

Definition C.4. y def = Sigmoid (x) with input x ∈ R m and output y ∈ R m is defined as y i def = 1 1+ e -x i .

Definition C.5. For Sigmoid layer with variable input bounds [x , x] , its Parametric Linear Relaxation is defined as φ Sigmoid (y , y , x , x) def = φ Sigmoid (y , x) ∧ φ Sigmoid (y , x) where:

where x u ∈ R m , a u ∈ R m , b u ∈ R m are constants that satisfy the following two conditions: i) let f ( x ) = a u i x +b u i , the line defined by function f ( x ) is tangent to the concave piece of sigmoid at some x ≥ 0 ; ii) a u x u +b u = Sigmoid (x u ) ; viz., the line defined by function f ( x ) intersects with convex piece of sigmoid at x u i where x u i ≤ 0 .

Similarly, x l ∈ R m , a l ∈ R m , b l ∈ R m are constants that satisfy the following two conditions: i) let f ( x ) = a l i x +b l i , the line defined by function f ( x ) is tangent to the convex piece of sigmoid at some x ≤ 0 ; ii) a l x l +b l = Sigmoid (x l ) ; viz., the line defined by function f ( x ) intersects with concave piece of sigmoid at x l i where x l i ≥ 0 . ■

Theorem C.6. Definition C.5 is a Parametric Linear Relaxation for the Sigmoid layer.

Proof. As seen in Definition C.5, φ Sigmoid (y , y , x , x) is a linear formula. Let m be the number of input dimensions of Sigmoid. φ Sigmoid (y , y , x , x) has 4 m linear constraints over 4 m variables ( x ∈ R m , x ∈ R m , y ∈ R m and y ∈ R m ). Hence, φ Sigmoid (y , y , x , x) is a poly-size linear formula, whose size is polynomial in the number of input dimensions of the Sigmoid layer.

Now our goal is to prove that φ Sigmoid (y , y , x , x) defined in Definition C.5 implies

We will discuss the upper and lower bounds separately.

For the upper bound constraint φ Sigmoid (y , x) def = y ≥ Sigmoid (x u ) ∧ y ≥ a u x + b u , for the case when x i ≤ x u i , we have y i ≥ sigmoid (x u i ) ≥ sigmoid (x i ) because sigmoid is monotonically increasing. For the case when x i > x u i , we have y i ≥ a u i x i +b u i ≥ sigmoid (x i ) because

f ( x ) = a u i x +b u i is tangent to the concave piece of sigmoid in [0 , ∞ ] , and sigmoid is monotonically increasing in [x u i , 0] . Therefore, φ Sigmoid (y , x) defined in Definition C.5 implies that ∀ x ∈ [x , x] . Sigmoid (x) ≤ y .

For the lower bound constraint φ Sigmoid (y , x) def = y ≤ Sigmoid (x l ) ∧ y ≤ a l x + b l , for the case when x i ≥ x l i , we have y i ≤ sigmoid (x l i ) ≤ sigmoid (x i ) because sigmoid is monotonically increasing. For the case when x i < x l i , we have y i ≤ a l i x i +b l i ≤ sigmoid (x i ) because f ( x ) = a l i x + b l i is tangent to the convex piece of sigmoid in [ -∞ , 0] , and sigmoid is monotonically increasing in [0 , x l i ] . Therefore, φ Sigmoid (y , x) defined in Definition C.5 implies that ∀ x ∈ [x , x] . y ≤ Sigmoid (x) .

C.3 Parametric Linear Relaxation for ELU layers

Definition C.7. y def = ELU (x) with input x ∈ R m and output y ∈ R m is defined as y i def = elu (x i ) where elu ( x ) def = { x x ≥ 0 α ( e x -1) x < 0 and α > 0 .

Definition C.8. For ELU layer with variable input bounds [x , x] , its Parametric Linear Relaxation is defined as φ ELU (y , y , x , x) def = φ ELU (y , x) ∧ φ ELU (y , x) where:

where a ∈ R m , b ∈ R m are constants such that the line defined by f ( x ) = a i x +b i for any i is tangent to elu . ■

Theorem C.9. Definition C.8 is a Parametric Linear Relaxation for the ELU layer.

Proof. As seen in Definition C.8, φ ELU (y , y , x , x) is a linear formula. Let m be the number of input dimensions of ELU. φ ELU (y , y , x , x) has 3 m linear constraints over 4 m variables ( x ∈ R m , x ∈ R m , y ∈ R m and y ∈ R m ). Hence, φ ELU (y , y , x , x) is a poly-size linear formula, whose size is polynomial in the number of input dimensions of the ELU layer.

Now our goal is to prove that φ ELU (y , y , x , x) defined in Definition C.8 implies

We will discuss the upper and lower bounds separately.

For the upper bound constraint φ ELU (y , x) def =y ≥ x ∧ y ≥ 0 , for the case when x i ≥ 0 , by Definition C.7 we have y i ≥ x i = elu (x i ) . For the case x i < 0 , because elu is monotonically increasing and elu (0) = 0 , we have y i ≥ 0 ≥ elu (x i ) . Therefore, φ ELU (y , x) defined in Definition C.8 implies that ∀ x ∈ [x , x] . ELU (x) ≤ y .

For the lower bound constraint φ ELU (y , x) def = y ≤ ax + b , because the line defined by f ( x ) = a i x +b i is tangent to elu and elu is a convex function, we have y i ≤ a i x i +b i ≤ elu (x i ) . Therefore, φ ELU (y , x) defined in Definition C.8 implies that ∀ x ∈ [x , x] . y ≤ ELU (x) .

D Experiment Details

D.1 Provable fine-tuning PFT ⟨·⟩

Because there are no efficient prior approaches for provable editing, we implemented provable fine-tuning PFT ⟨·⟩ baselines that combine prior (non-provable) DNN editing approaches with a verifier in the loop: the editing stops when the verifier confirms that the DNN satisfies the property (Appendix D.1). Specifically: 1) In each epoch, provable fine-tuning first updates the DNN parameters with the DNN editing approach; 2) If the editing approach or adversarial attack cannot find any constraint violation, we call a verifier to check if all constraints are satisfied; 3) If all constraints

are satisfied, the loop terminates; otherwise the loop continues. According to the DNN architecture and editing property, we use the best verifier from (1) α,β -CROWN [46, 44]: winner of the VNN Competition VNN -COMP 2021, 2022 and 2023; (2) MN -BaB [9]: runner-up of VNN -COMP 2022 and 2023; and (3) DeepT [4]: state-of-the-art (incomplete) transformer verifier.

In order to improve the performance of provable fine-tuning with a non-provable DNN editing approach, viz., preserve the DNN's accuracy, we a) freeze all batch normalization layers, b) only edit the last few layers, c) incorporate changes of parameters as regularization, and d) sample points and incorporate the changes in their outputs as regularization.

D.2 Provable Editing on VNN Competition Benchmarks

In this section we present the further details for Section 4.1.

Benchmarks. We take 12 out of 15 benchmarks from VNN -COMP ′ 22 [29], excluding i) ' carvana _ unet _ 2022 ' and ' nn4sys ': the support for the sigmoid activation function is not implemented in PREPARED , and is not supported by APRNN . ii) ' vggnet16 _ 2022 ': the bound computation for intermediate layers runs out of CPU and GPU memory using auto _ LiRPA .

Setup details. For both PFT ⟨ DL2 ⟩ and PFT ⟨ APRNN ⟩ , we use DL2 to find counterexamples to edit, and we use α,β -CROWN , the winner of VNN -COMP , as the verifier. We only call the verifier when DL2 cannot find counterexamples. For all approaches, we edit the last few layers, which differs for different DNNs. For PREPARED , we use auto _ LiRPA or DeepPoly to compute the constant bounds for the input to the first layer to edit.

Hyperparameters for PFT ⟨ DL2 ⟩ . SGD optimizer with a learning rate of 0.001, DL2 weight of 0.2, batch size of 32 for the single-property setting, and 256 for the all-properties setting.

Hyperparameters for PFT ⟨ APRNN ⟩ . For different DNNs, APRNN edits from the last six layers to the last layer. we set a bound of [-10, 10] for the change of each parameter. We use DL2 to find counterexamples for APRNN to edit in each epoch, and use the same hyperparameters as the DL2 experiments for the DL2 oracle.

Hyperparameters for PREPARED . For different DNNs, PREPARED edits from the last eight layers to the last layer. We use auto _ LiRPA [50] or DeepPoly [39] to compute the input bounds to the first layer to edit. we set a bound of [-10, 10] for the change of each parameter.

Results details. In Table 4 we present the number of succeed instances for provably editing single-properties and all-properties instances in each benchmark.

D.3 Local robustness editing for image-recognition DNNs

In this section we present the experiment details for Section 4.2.

Property. Given an L ∞ local robustness perturbation ε for an DNN N and a pair of input point x and output label l , the repair specification is ∀ x ′ . ∥ x -x ′ ∥ ∞ = ⇒ arg max N (x ′ ) = l . In other words, for all inputs x ′ in the L ∞ ball around x with radius ε , the DNN N should classify them as label l .

Setup details. The validation set is the same 10% of the training set, randomly selected with seed 0. For PFT ⟨ DL2 ⟩ , PFT ⟨ SABR ⟩ and PFT ⟨ STAPS ⟩ , we use MN -BaB , the complete verifier used by SABR and STAPS , as the verifier in the loop. MN -BaB is the runner-up of VNN -COMP . We only call the verifier when DL2 , SABR or STAPS cannot find counterexamples. For PREPARED , we use DeepPoly to compute the constant bounds for the input to the first layer to edit.

Hyperparameters for PFT ⟨ SABR ⟩ . We take the base hyperparameters from the SABR training script [27]. We freeze all batch normalization layers, and search the hyperparameters: i) learning rate in { 5e-4, 1e-4, 5e-5, 1e-5 } ; ii) batch size in { 5 , 10 , 25 } ; iii) lambda in { 0 . 1 , 0 . 2 , . . . , 0 . 9 } ; iv) end_epoch_eps in { 80 , 60 , 40 , 20 } ; v) fine-tune the entire DNN, the last three layers, or the last layer to find the edit with the highest efficacy, then the best validation set accuracy. Other hyperparameters are the same as used for training the CIFAR10 ( ε = 2 / 255 ) and TINYIMAGENET ( ε = 1 / 255 ) CNN7 in the SABR paper. We use the following early-stopping criteria: i) CIFAR10: validation accuracy drops below 60%; ii) TINYIMAGENET: validation accuracy drops below 20%. The final hyperparameters for CIFAR10 are: Adam optimizer with a learning rate of 5e-4, batch size of 5,

(b) Comparison of the number of instances succeeded for provably editing all-properties instances in each benchmark.

lambda of 0.1, end_epoch_eps of 80, and fine-tune the last three layers. The final hyperparameters for TINYIMAGENET are: Adam optimizer with a learning rate of 5e-4, batch size of 5, lambda of 0.1, end_epoch_eps of 80, and fine-tune the last three layers.

Hyperparameters for PFT ⟨ STAPS ⟩ . We take the base hyperparameters from the STAPS training script in the TAPS code [25]. We freeze all batch normalization layers, and search the following hyperparameters: i) learning rate in { 5e-4, 1e-4, 5e-5, 1e-5 } ; ii) batch size in { 5 , 10 , 25 } ; iii) reg_lambda in { 0 . 1 , 0 . 2 , . . . , 0 . 9 } ; iv) end_epoch_eps in { 80 , 60 , 40 , 20 } . v) fine-tune the entire DNN, the last three layers, or the last layer. to find the edit with the highest efficacy, then the best validation set accuracy. Other hyperparameters are the same as used for training CIFAR10 ( ε = 2 / 255 ) and TINYIMAGENET ( ε = 1 / 255 ) CNN7 using STAPS in the TAPS paper We use the following early-stopping criteria: i) CIFAR10: validation accuracy drops below 60%; ii) TINYIMAGENET: validation accuracy drops

below 20%. The final hyperparameters for CIFAR10 are: Adam optimizer with a learning rate of 1e-5, batch size of 10, reg_lambda of 0.9, end_epoch_eps of 80, and fine-tune the last three layers. The final hyperparameters for TINYIMAGENET are: Adam optimizer with a Learning rate of 1e-5, batch size of 10, reg_lambda of 0.8, end_epoch_eps of 80, and fine-tune the last three layers.

Hyperparameters for PFT ⟨ DL2 ⟩ . We take the base hyperparameters from the RobustnessG training script in the DL2 code [11]. We freeze all the batch normalization layers, and search for the following hyperparameters: i) learning rate in { 1e-3, 5e-4, 1e-4, 5e-5 } ; ii) batch size in { 5 , 10 , 25 } ; iii) fine-tune the entire DNN, the last layer or the last three layers. to find the edit with the highest efficacy, then the best validation set accuracy. We use the following early-stopping criteria: i) CIFAR10: validation accuracy drops below 70%; ii) TINYIMAGENET: validation accuracy drops below 20%. The final hyperparameters for CIFAR10 are: SGD optimizer with learning rate of 1e-4, batch size of 10, DL2 weight of 0.2, fine-tune the last three layers. The final hyperparameters for TINYIMAGENET are: SGD optimizer with learning rate of 1e-4, batch size of 10, DL2 weight of 0.2, fine-tune the last three layers.

Hyperparameters for APRNN . We edit up to the fifth last layer, and set a bound [-10, 10] for the change of each DNN parameter. For both CIFAR10 and TINYIMAGENET, the final result comes from editing the last layer.

Hyperparameters for PREPARED . We edit up to the fifth last layer, and set a bound [-10, 10] for the change of each DNN parameter. For both CIFAR10 and TINYIMAGENET, the final result comes from editing the last layer.

D.4 Local robustness editing for sentiment classification BERT transformers

In this section we present the experiment details for Section 4.3.

Setup details. We use the default validation set used in the DeepT training scripts [3]. For PFT ⟨ DL2 ⟩ , we use DeepT as the verifier in the loop. We only call the verifier when DL2 cannot find counterexamples. For PREPARED , we use DeepT to compute the constant bounds for the input to the first layer to edit.

Hyperparameters for PFT ⟨ DL2 ⟩ . We incorporate the default training script and hyperparameters from DeepT [4]. We search the hyperparameters i) fine-tune the last three layers or only the last layer; ii) batch size in { 4 , 8 , 16 , 32 } ; iii) learning rate in {1e-3, 5e-3, 1e-4, 5e-4}; to find the edit with the best efficacy, then the best validation accuracy. For ε =1e-4, the final hyperparameters are: fine-tune the last layer, batch size of 32, Adam optimizer with learning rate of 1e-4, DL2 weight of 0.1 For ε =5e-4, the final hyperparameters are: fine-tune the last layer, batch size of 16, Adam optimizer with learning rate of 1e-4, DL2 weight of 0.1.

Hyperparameters for PFT ⟨ APRNN ⟩ . We edit up to the third last layer, and set a bound [-10, 10] for the change of each DNN parameter. The final result comes from editing the last layer.

Hyperparameters for PREPARED . We edit up to the third last layer, and set a bound [-10, 10] for the change of each DNN parameter. The final result comes from editing the last layer.

D.5 Provable training for physics-plausible DNNs

In this section we present the experiment details for Section 4.4.

Hyperparameters for DL2 . We search the hyperparameters i) batch size in { 4 , 8 , 16 , 32 } ; ii) learning rate in {1e-3, 5e-3, 1e-4, 5e-4}; We train 2000 epochs and take the epoch with the highest validation accuracy. The final hyperparameters are: batch size of 8, SGD optimizer with lr of 1e-3, DL2 weight of 0.2, 1241 epochs.

Hyperparameters for GD . We search the hyperparameters i) batch size in { 4 , 8 , 16 , 32 } ; ii) learning rate in {1e-3, 5e-3, 1e-4, 5e-4}; We train 2000 epochs and take the epoch with the highest validation accuracy. The final hyperparameters are: batch size of 32, SGD optimizer with lr of 1e-3, 635 epochs.

Hyperparameters for GD ⟨ APRNN ⟩ . We edit up to the first layer, and set a bound [-10, 10] for the change of each DNN parameter. The final result comes from editing the last layer.

Hyperparameters for GD ⟨ PREPARED ⟩ . We edit up to the first layer, and set a bound [-10, 10] for the change of each DNN parameter. We use DeepPoly to compute the input bounds to the first layer to edit. The final result comes from editing the last three layers.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The theoretical claims in the abstract and introduction are proved in Section 3 as well as Appendix A, B. The empirical claims in the abstract and introduction are justified in Section 4.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations are discussed in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The full set of assumptions and informal proof sketch are provided in Section 3 complemented by formal proofs in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Section 3 complemented by Appendix B fully disclose all information needed to reproduce the technique, and Section 4 complemented by Appendix D fully disclose all information needed to reproduce the experimental results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The experiments Section 4.1-4.3 use open-access benchmarks. We plan to release on acceptance the data we generated for Section 4.4. For the implementation of our tool, we could not make the code open access due to ongoing IP restrictions.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The experimental settings are presented in Section 4 complemented by further details in Appendix D.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Because the proposed algorithm is deterministic, we instead evaluate our approach on a wide-variety of benchmarks to demonstrate its efficacy and efficiency.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The evaluation platform and runtime are provided in Section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: This is discussed in Sections 1 and 5.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper proposes an efficient and provable DNN editing approach, which does not involve releasing of data and models that have a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Section 4 cites the original papers that produced the baselines, tools, models and benchmarks used in the paper. Appendix D cites commits for open-access code and scripts used in the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce new assets in the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose HRDNN, the first efficient technique for provable editing of DNNs.  HRDNN is able to provably edit the DNN to satisfy this property within 45 seconds. HRDNN is efficient because it relaxes the NP-hard provable editing problem to solving a linear program (LP). It enables HRDNN to construct tight bounds of the DNN that are parameterized by the new parameters $\\theta$. The edited models could be vnn-comp22 benchmarks, CIFAR10,tiny imagenet, bert SST dnns.\n\nSTRENGTHS:\n+Important research topic\n\n+the first verifier-in-the-loop, and the editing stops when the verifier confirms that the DNN satisfies the property\n\n+well writing\n\n+It can apply to many types of networks(sigmoid,tanh,elu DNNs)\n\nWEAKNESSES:\n- In Table1, the generalized accuracy is smaller than PFT<ASBR>. Further, the results in Table 1  are only tested on two models and can not sufficiently demonstrate the effectiveness of HRDNN. \n\n-In Table1, the results with <100\\%efficacy are shaded. However, I think APRNN is more effective in terms of ACC. , Cert. and Time. Are these metrics more important than the efficacy, as editing methods are purposed to improve the certified robustness (Cert.)?\n\n-Unclear metric. Where is the definition of Og. Effic in Table2?\n\nQUESTIONS:\nThere are many robustness-enhancing  methods. What are the results of comparing HRDNN to verified training(lipschitz-based).\n\n\nOther question can be seen in weakness.\n\nI am happy to reconsider my rating, if the authors address satisfactorily my questions.\n\nLIMITATIONS:\nWhat I'm more concerned about: In Table1, the results with <100\\%efficacy are shaded. However, I think APRNN is more effective in terms of ACC. , Cert. and Time. Are these metrics more important than the efficacy, as editing methods are purposed to improve the certified robustness(Cert.)?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of finding a minimal changes to the parameters of a DNN $f(.)$ in order to satisfy pre/post conditions defined as *convex* polytopes, i.e. $\\forall x . (x \\in Pre \\implies f(x) \\in Post ) $.\nThe proposed algorithm is efficient as it considers a Linear Programming (LP) relaxation, while being more general than previous LP-based approaches. The key contribution is a parametric relaxation by defining bound on the affine transformations *in terms of the layers parameters*. The approach is evaluated on multiple benchmarks from the VNN competition and local robustness of classifiers.\n\nSTRENGTHS:\n- The paper is very well written.\n- Quite extensive evaluation that seems to validate the proposed approach\n- Evaluation of HRDNN both as a post-hoc technique and during training\n\nWEAKNESSES:\n- I think that runtime results for an editing algorithm should always be paired with its impact on predictive performance to be meaningful. I don't understand the role of the VNN Comp benchmark in this sense;\n- Some details of the experimental section are not perfectly unclear.\n\nQUESTIONS:\n1) Wouldn't local robustness on the whole test set be a non-convex property (i.e. the union of many $\\epsilon$-balls)? How was this approached in the experimental section? Editing the network independently for each input instance does not guarantee that the constraint holds globally, right (i.e. some fixes might be undone in successive edits)?\n\n\n---\n\n## Minors\n\n\n\n\"This paper presents HRDNN, the first efficient technique for provable editing of DNNs that runs in polynomial time.\"\nPolynomial in..?\n\n---\n\nEq. 1 and 2 both define *the provable editing problem*. In the former, arbitrary convex polytopes are considered, the latter seems restricted to axis-aligned inequalities/halfspaces (or maybe the notation with square brackets misled me?). Which is the correct one? In any case, I would avoid redefining it twice.\n\n---\n\nIn the experiments, I think it would be interesting to read the accuracy of the original (unedited) NNs as a reference.\n\n---\n\nEfficacy means different things in the VNN Comp benchmarks and later benchmarks. I would use a different term.\n\n---\n\nI would define certified accuracy in the text.\n\nLIMITATIONS:\nThe limitations are adequately addressed in my opinion.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper proposes a novel approach named HRDNN to provably edit a DNN to satisfy a given property. Concretely, given a DNN, its input domain P, and desired output range Q, HRDNN edits the parameters of the DNN so that for any input in P, the output of the edited DNN falls into Q. The key contribution of this approach is that it uses a novel notion of Parametric Linear relaxation to construct tight output bounds of the DNN. It finally shows the effectiveness and efficiency over a set of benchmarks.

STRENGTHS:
* The paper is well-written and easy to read.
* It proposes a novel notion of parametric linear relaxation to construct tight parametric bounds of the DNN outputs.
* The paper provides theoretical proof for the soundness of the proposed notion.
* The paper shows experimental results that demonstrate the efficacy and efficiency of the proposed method.

WEAKNESSES:
* The parametric linear relaxation is developed for specific activation functions, like ReLU, Tanh.  It lacks a discussion or summary about a general paradigm to devise parametric bounds for different activation functions.
* The paper does not discuss the scalability of this approach. It is a white-box method in that the weights of neural networks shall be known in advance.

QUESTIONS:
* It is weird to use ";" before "viz". Please replace them with ", viz.,". The same holds for "; e.g.,"
* line 58, " with defines tight", typo?
* The authors distinguish different symbols with different colors. This is good but not friendly to readers with amblyopia. Please consider adding certain marks to colored symbols.
* Is the scalability of this approach restricted by the ability of LP solvers?

LIMITATIONS:
Limitations and social impact are adequately discussed in the paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-15

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel method for provably editing neural networks using neural network verification techniques. The proposed method is based on a new linear programming formulation of neural networks. Unlike typical neural network verification settings, in the neural network editing settings, the weights are also variables that will multiply to neural network features, and this is challenging to handle. The proposed formulation constructs the lower and upper bounds of each neuron as linear program variables separately, which offers some benefits when dealing with weight variables. Experiments were conducted in multiple settings to demonstrate that the proposed method achieves better efficacy and efficiency compared to existing provable neural network editing methods.\n\nSTRENGTHS:\n1. The paper studies a fundamental problem of how to find neural network weights that guarantee to satisfy output properties. The approach is based on NN verification and has theoretical guarantees unless ordinary gradient-based training is used. The relaxations based on the epigraph and hypograph of nonlinear functions are inspiring and effective in the proposed setting. Based on these contributions, I tend to recommend acceptance of this paper overall, although there are weaknesses and questions I hope the authors address below.\n\n2. Experiments include multiple settings, including standardized benchmarks such as the VNN-COMP 2022 benchmarks, as well as novel settings such as physics-plausible DNNs.\n\nWEAKNESSES:\nWeaknesses:\n\n1. Lack of theoretical discussions on how the proposed relaxations (especially for ReLU) connect to the optimal LP relaxation for ReLU neurons (https://arxiv.org/abs/1902.08722 https://arxiv.org/abs/2006.14076). Note that these relaxations are not the ones presented in Figure 1(a). Since now the pre-activation lower and upper bounds are considered as variables, it can introduce looseness in some situations, for example, for stable neurons. To make the paper more insightful, it will be helpful to discuss when the new formulation which keeps the lower and upper bounds as two separate variables helps.\n\n2. The benefits of the proposed method compared to provable fine-tuning, an important baseline, are not well demonstrated. It is only included in one set of experiments (table 1). Also the setting used in this experiment is strange - the typical perturbation used for CIFAR is 2/255 or 8/255 (0.5/255 is too small). It also looks strange SABR and STAPS achieve very low accuracy (far below the accuracy reported in their papers).\n\nQUESTIONS:\nQuestions:\n\n1. It is unclear how the editing approach was extended to the BERT model. How the editing of the Transformer layer was done? Is DeepT handling it as a blackbox? In addition, you can consider using a better verifier for Transformers such as https://files.sri.inf.ethz.ch/wfvml23/papers/paper_24.pdf\n\n2. The notations are somewhat confusingly differentiated using colors. It will be impossible to read if the paper is printed in black and white. It is better to not solely use colors to distinguish among different letters. It makes the paper less accessible and confusing to understand.\n\nLIMITATIONS:\n1. Comparisons to provable fine tuning setting is limited, as pointed out above.\n\n2. The approach's scalability is unclear. How large can the network be using the LP formulation?\n\n3. Extensions to other activation functions were discussed conceptually but without formulations and experiements.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-30"}