Title: Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification
Venue: NeurIPS 2025
Accepted: yes
Keywords: Neuron Network Verification; AI Safety; Robustness; Formal Methods
Abstract: State-of-the-art neural network verifiers demonstrate that applying the branch-and-bound (BaB) procedure with fast bounding techniques plays a key role in tackling many challenging verification properties. 
In this work, we introduce the \emph{linear constraint-driven clipping} framework, a class of scalable and efficient methods to enhance bound propagation verifiers. Under this framework, we develop two novel algorithms that efficiently utilize constraints to 1) reduce portions of the input space that are either verified or irrelevant to a subdomain in the context of branch-and-bound, and 2) directly improve intermediate bounds throughout the network. 
The process novelly uses linear constraints that are readily available during verification in a highly scalable manner compared to using off-the-shelf linear programming (LP) solvers. 
This reduction tightens bounds globally and can significantly reduce the number of subproblems
handled during BaB. We show our clipping procedures can intuitively and efficiently be incorporated into BaB-based verifiers such as $\alpha, \beta$-CROWN, and is amenable to BaB procedures that split upon the input or activation space. We demonstrate the effectiveness of our procedure on a broad range of benchmarks where, in some instances, we witness a 96\% reduction 
in the number of subproblems during branch-and-bound, and also achieve state-of-the-art verified accuracy across multiple benchmarks.

Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification

Abstract

State-of-the-art neural network (NN) verifiers demonstrate that applying the branchand-bound (BaB) procedure with fast bounding techniques plays a key role in tackling many challenging verification properties. In this work, we introduce the linear constraint-driven clipping framework, a class of scalable and efficient methods designed to enhance the efficacy of NN verifiers. Under this framework, we develop two novel algorithms that efficiently utilize linear constraints to 1) reduce portions of the input space that are either verified or irrelevant to a subproblem in the context of branch-and-bound, and 2) directly improve intermediate bounds throughout the network. The process novelly leverages linear constraints that often arise from bound propagation methods and is general enough to also incorporate constraints from other sources. It efficiently handles linear constraints using a specialized GPU procedure that can scale to large neural networks without the use of expensive external solvers. Our verification procedure, Clip-and-Verify, consistently tightens bounds across multiple benchmarks and can significantly reduce the number of subproblems handled during BaB. We show that our clipping algorithms can be integrated with BaB-based verifiers such as α,β -CROWN, utilizing either the split constraints in activation-space BaB or the output constraints that denote the unverified input space. We demonstrate the effectiveness of our procedure on a broad range of benchmarks where, in some instances, we witness a 96% reduction in the number of subproblems during branch-and-bound, and also achieve state-of-the-art verified accuracy across multiple benchmarks. Clip-and-Verify is part of the α,β -CROWN verifier, the VNN-COMP 2025 winner . Code available at https://github.com/Verified-Intelligence/Clip_and_Verify .

1 Introduction

The neural network (NN) verification problem is imperative in mission-critical applications [70, 63, 58, 16, 75, 72] where formally proving properties such as safety and robustness over a specified input domain is essential. Recent approaches make neural network verification more tractable by relaxing the original non-convex problem into convex formulations that are amenable to linear programming (LP) [25, 68], semidefinite programming (SDP) [12, 26, 50, 40, 18, 19], and bound propagationbased solvers [77, 29, 55, 67, 24, 66, 64]. These convex relaxations can be further strengthened by tightening single-neuron relaxations through convex geometric analysis [55], constructing convexhull approximations to capture multi-input dependencies [48], and introducing cutting planes that encode inter-neuron dependencies [76, 79]. To handle properties that cannot be certified by a single relaxation, state-of-the-art verifiers couple bound propagation methods with the branch-and-bound (BaB) paradigm [14, 66, 53, 13] as this technique can be efficiently parallelized and scaled on GPUs. Linear bound propagation methods such as CROWN [77] recursively compute bounds on the activations of each layer, referred to as intermediate bounds , which serve as critical building blocks for determining the tightness of the overall relaxation and for guiding the BaB search.

While BaB and cutting-plane techniques can further refine the relaxation by incorporating additional constraints at the final layer, they cannot directly and efficiently improve the intermediate bounds themselves. In BaB, where the number of subproblems grows exponentially, loose intermediate bounds weaken the relaxation, resulting in deeper branching and longer verification times. Although algorithms such as β -CROWN [66] theoretically support optimizing bounds at intermediate layers, doing so in practice is prohibitively expensive as the number of hidden neurons typically outnumber the output neurons used for property verification by several orders of magnitude. Consequently, updating bounds for all intermediate layers introduces significant computational overhead that far outweigh the gains brought from their tighter convex relaxations. As a result, existing implementations fix the global intermediate bounds computed at initialization (e.g., via α -CROWN [74]) and focus their optimization efforts solely at the final layer. While this design choice preserves scalability, it limits the ability to tighten the relaxation throughout the network, motivating methods that can refine intermediate bounds both effectively and efficiently .

To this end, we introduce Clip-and-Verify , a verification pipeline designed to enhance NN verifiers by opportunistically refining bounds at any layer with minimal computational overhead . Our core insight is that the bounding planes generated by linear bound propagation at all layers naturally align with our pipeline, enabling us to exploit their geometry to eliminate infeasible regions of the input domain and prune redundant subproblems early in verification. We formalize the task of tightening any layer's bounds as the objective of our linear constraint-driven clipping framework, and we propose two novel algorithms: complete clipping, which directly optimizes neurons' bounds via a specialized coordinate ascent procedure, and relaxed clipping, which refines the input domain to enhance the intermediate relaxations and consequently improves the NN's bounds. An intuitive illustration of this refinement is given in Figure 1, and our main contributions are as follows:

We propose two specialized GPU algorithms within our novel linear constraint-driven clipping framework for tightening bounds at any layer , preserving the scalability of state-of-the-art NN verifiers without relying on external solvers. Relaxed clipping optimizes the input domain bounds as a proxy, offering good improvements with little costs, while complete clipping employs a customized coordinate ascent solver to directly refine the bounds at every layer of the NN.

We show that linear bound propagation methods produce linear constraints that can be obtained 'for free' during both input and activation BaB procedures. By leveraging these cheaply available constraints and integrating our two efficient clipping algorithms, we introduce Clip-and-Verify , a verification pipeline that tightens the all neurons bounds with minimal computational overhead.

Across a large number of benchmarks from the Verification of Neural Networks Competition (VNN-COMP) [11, 10] and existing literature, we demonstrate that our Clip-and-Verify framework is capable of reducing the number of BaB subproblems by as much as 96% and consistently verifying more properties on benchmarks from the Verification of Neural Networks Competition.

2 Preliminaries

The NN Verification Problem. Given some input x belonging to the set X , and a feed-forward network f ( · ) with general activation functions, the goal of NN verification can be formulated as verifying f ( x ) ≥ 0 for all inputs in X . One manner of verifying this property involves solving min x ∈X f ( x ) , which is challenging due to the non-convexity of the NN and is generally NP-complete [35]. On the other hand, convex-relaxation algorithms compute a sound, approximate lower bound, f ( x ) , to the network's true minimum such that when f ( x ) > 0 , the property is sufficiently verified, otherwise the problem is unknown without further refinement or falsification.

Bound Propagation. Linear bound propagation methods [56, 55, 64, 5, 77] approximate neuron bounds layer by layer by relaxing the nonlinearities of activation functions, making these techniques a fast and popular approach for NN verification. For an L -layered, feedforward network, the bounds for the j th neuron at the i th layer may be expressed as:

When X is an ℓ ∞ box (i.e. { x | ∥ x -ˆ x ∥ ∞ ≤ ϵ } ), we may 'concretize' the lower and upper bounds of the layer using Lemma B.1: z ( i ) = A ( i ) ˆ x -| A ( i ) | ϵ + c ( i ) and z ( i ) = A ( i ) ˆ x + | A ( i ) | ϵ + c ( i ) . Once concretized, the post-activation neuron, ˆ z ( i ) j , may be bounded (e.g. via Planet relaxation [25] for

(a)

(b)

ReLU). The lower/upper bounding planes, A ( i ) / A ( i ) and c ( i ) / c ( i ) , are produced via backpropagation, and their definitions are given in Appendix A. The lower bound at the final layer, z ( L ) , is used to determine if the problem is verified.

Branch-and-Bound. The BaB paradigm [14, 66, 53, 76, 79, 21, 65, 25, 45] systematically partitions the verification problem into smaller subproblems, X = X 1 ∪X 2 , enabling tighter bounds on each subdomain. BaB can split upon the input space (e.g. axis-aligned constraints) or the activation space (e.g. split activation neurons). See Appendix A.1 for the formal definition and complexity implications. The exponential growth of subdomains can lead to high computational cost but can be mitigated by verifying domains early and often, a direct consequence of Clip-and-Verify .

3 Enhancing Neural Network Verification with Clip-and-Verify

3.1 Motivation and Overview

BaB-based verifiers manage complexity by partitioning the input or activation space into tractable subproblems. Within each branch, linear relaxations provide bounds on the network's neurons which can guide branching decisions and verify properties. Crucially, the constraints introduced at each BaB split implicitly define a tighter feasible input domain and offer opportunities to refine the bounds at any layer. As shown in Fig. 1b, linear constraints can shrink an ℓ ∞ -norm ball during robustness verification, yielding tighter intermediate bounds and enabling more effective search-space pruning.

In large NNs, full linear bound propagation is often limited to an initial pass for efficiency. Alternative methods like using LPs to update intermediate bounds [51] or full re-propagation after each split also prove too costly. Such overhead restricts frequent bound updates in deep NNs or extensive branching.

To fully exploit these opportunities, we propose Clip-and-Verify , a pipeline which tightens the input box and re-concretizes intermediate bounds at each BaB node using linear constraints (e.g. activation split and final layer bound), without running a full pass. It contains two algorithms: Complete Clipping , which performs a fast coordinate-wise dual search with sorted breakpoints per constraint to obtain near-LP tightening at a fraction of the cost, and Relaxed Clipping , which performs perconstraint exact tightening for axis-aligned boxes by solving the one-dimensional dual in closed form and then re-concretizing cached linear constraints over the shrunk box.

3.2 Complete Clipping: Optimizing Intermediate Bounds Directly via Linear Constraints

Exact Bound Refinement with a Single Constraint via Optimized Duality. Suppose we wish to improve the lower bound of a linear function a ⊤ x + c over an input domain X , given a new linear

Instead of directly solving this potentially high-dimensional LP, we formulate its Lagrangian dual. The key advantage and insight is that, for a fixed dual variable β ∈ R + , the inner minimization over x can be solved analytically for box domains, transforming the problem into a simpler optimization over a single dual variable. This leads to the following theorem:

Theorem 3.1 (Exact Bound Refinement under a Single Linear Constraint) . Let a ∈ R n , c ∈ R , g ∈ R n , h ∈ R , and the input domain be X = { x | ˆ x -ϵ ≤ x ≤ ˆ x + ϵ } . The optimal value L ⋆ of the constrained minimization problem (2) is given by the solution to the dual problem:

D ( β ) in (3) is concave and piecewise-linear in β ∈ R + . Its maximum L ⋆ and the optimal β ⋆ can be determined exactly and efficiently by identifying its breakpoints (values of β where ( a + β g ) j = 0 for some j ∈ [ n ] ) and analyzing the super-gradients within the resulting linear segments.

For proof, see Appendix B.2. Theorem 3.1 converts a potentially expensive n -dimensional LP into a 1D concave maximization problem ( D ( β ) ) that can be solved without iterative gradient methods. The process of finding breakpoints and the optimal segment (detailed below and in Algorithm 1 for the multi-constraint case) is highly amenable to efficient computation, making it suitable for refining bounds across many neurons and subdomains in parallel . An analogous theorem holds for tightening the upper bound by solving max x ∈X { a ⊤ x + c | g ⊤ x + h ≤ 0 } .

Before optimizing Eq. (3), infeasibility can be detected a priori : the problem is infeasible iff g ⊤ ˆ x + h -∑ n i =1 | g i | ϵ i > 0 . Such a scenario arises when the property being verified imposes constraints unsatisfiable within the current input domain X (e.g., contradictory ReLU assignments), and may be considered verified without further refinement.

Note that our primal optimization problem is mathematically equivalent to a continuous knapsack problem which can be obtained via a change of variables as detailed in Appendix B.3. The breakpoints β i = -a i /g i in our dual formulation are identical to the efficiency ratios r j /s j used in the standard greedy knapsack algorithm. Thus, our dual-based solver and the greedy knapsack algorithm are equivalent, both finding the provably optimal solution with O ( n log n ) complexity.

Coordinate Ascent for Multiple Constraints When multiple linear constraints Gx + h ≤ 0 (where G ∈ R m × n , h ∈ R m ) are available, the dual problem involves optimizing multiple Lagrange multipliers β ∈ R m + :

̸

The single constraint problem (3) is easy to solve using Theorem 3.1, thus we can use coordinate ascent to solve (4). Algorithm 1 details a single pass of this coordinate ascent procedure. This iterative approach optimizes one dual variable β k at a time, keeping others fixed. Each step thus reduces to solving the 1D problem described in Theorem 3.1 (with a replaced by a + ∑ p = k β p G p, : and g replaced by G k, : ). Since the dual objective remains concave and piecewise-linear along each coordinate β k , we can efficiently exploit the breakpoint structure for each update. An order dependency is discussed in Appendix B.5. Algorithm 1 is significantly more efficient and scalable than using general-purpose LP solvers, even when those solvers are used as fast heuristics to get a bound rather than converge to optimal.We conducted a detailed comparison in Appendix D.3, integrating a state-of-the-art LP solver (Gurobi) into our BaB framework. The results show that even when running dual simplex with 10-iteration limit, the LP solver was over 880 × slower than our GPU-parallelized coordinate ascent (0.0028s vs. 2.47s per round) while achieving comparable bound accuracy (0.00085 vs 0.0007 mean error).

Algorithm 1 Coordinate Ascent with Multiple Constraints

3.3 Relaxed Clipping: Optimizing Intermediate Bounds Indirectly via Input Refinement

Although the coordinate ascent is significantly efficient compared to LP solvers, it is still computationally expensive when there are various subproblems with many unstable neurons and multiple constraints. To address this issue, we propose a more efficient Relaxed Clipping algorithm, which shrinks the input domain by leveraging linear constraints (e.g., from final-layer outputs or activation branchings) to shrink axis-aligned portions of the input. By adopting a box input domain as a proxy, we avoid repeatedly solving linear programs for each intermediate-layer neuron, and can solve the resulting optimization problem efficiently via the dual norm without relying on external solvers. The remaining task is to determine the clipped box X ′ ⊆ X that best respects the constraints.

Formulating Relaxed Clipping for a Hyper-Rectangle. We assume a hyper-rectangular input region and a set of m linear constraints Ax + c ≤ 0 , with A ∈ R m × n and c ∈ R m . To refine the lower and upper bounds of the input along each input dimension i ∈ [ n ] , we solve:

These refined bounds, x i and x i , clip the original domain X to reflect only the portion that satisfies all the linear constraints. We derive a simple closed-form solution when there is only one linear inequality, a ⊤ x + c ≤ 0

, foregoing the need to solve (5) via LP solvers or gradient-based methods. ⊤

̸

Theorem 3.2 (Relaxed clipping under a single constraint) . Let x ∈ X and a x + c ≤ 0 be the sole constraint. For brevity, denote the closed-form solution as x (clip) i = ( -∑ j = i { a j ˆ x j -| a j | ϵ j }-c ) / a i . Then, for each coordinate i , the new upper (or lower) bound is updated as follows:

Appendix B.4 gives a proof showing that, for a single linear constraint a ⊤ x + c ≤ 0 , we can clip the box X to a new box X ′ in one pass using the closed-form updates in Theorem 3.2, without any external solvers. The resulting box is the tightest axis-aligned over-approximation of the feasible set X ∩ { x : a ⊤ x + c ≤ 0 } ⊆ X ′ ⊆ X , and it is component-wise tight : no coordinate bound of X ′ can be further refined while remaining a box that still contains the feasible set. The computation costs only O ( n ) arithmetic operations. For example, re-propagating bounds for a network with 3 intermediate layers (4096, 2048, and 100 neurons respectively) and containing 800-1600 unstable neurons can take 10s per subdomain, versus 0.3s when re-concretizing the intermediate bounds.

This Relaxed Clipping step complements Complete Clipping. Whereas Complete Clipping solves problem (3) for each neuron , Relaxed Clipping tightens the shared input box X ′ once , and then

all intermediate bounds are cheaply re-concretized over this tighter box, yielding network-wide bound improvements from a single cheap update. By keeping the shared domain as a box, we avoid polyhedral operations and repeated per-neuron optimizations, substantially improving scalability.

Clipping for Multiple Constraints. When several linear constraints are present, we apply Theorem 3.2 in parallel to each constraint in Ax + c ≤ 0 . Algorithm 2 outlines this procedure: Given the original box bounds, x and x , we compute its center and radius, ˆ x and ϵ . Then, for each constraint k and dimension i , we apply Theorem 3.2 independently and in parallel to refine x i and x i . After processing all constraints, the resulting clipped bounds are aggregated, and the tightest bounds are selected to form the final clipped domain. This formulation preserves scalability and supports parallelization. We present a sequential variation in Appendix C.2 that foregoes parallelization for further refinement in which the box center and radius are recalculated after each constraint is applied. Nonetheless, we emphasize our current formulation as a core strength of domain clipping, maintaining both efficiency and effectiveness. Similar to direct clipping, Algorithm 2 may identify infeasibility by returning clipped bounds where x is larger than x along some dimension(s).

̸

̸

We have discussed how to use the linear constraints to tighten bounds, the next step is to find these linear constraints. We can use output constraints or activation split constraints during BaB. The next two sections we will discuss how to find the constraints and incorporate our algorithm into BaB.

3.4 Integrating Clipping into Input BaB Verification

In input BaB, the verifier partitions the network's input region (often an ℓ ∞ -box) along axis-aligned splits, generating multiple subdomains {X i } b i =1 . Each subdomain is then fed into a bound-propagation verifier (e.g., CROWN) to obtain linear hyper-planes that are used to lower-bound the final-layer output with respect to the input. Existing verifiers typically use these final-layer hyperplanes only to compute the bounds and decide whether the entire subdomain can be immediately verified or must be further subdivided. It may be the case that the hyperplanes used for computing the final layer bounds are tight enough to verify a subset of the input, but not tight enough to verify the input in its entirety (see Fig. 4 in Appendix). The key insight is that we can remove infeasible parts in a subproblem via linear constraints and then consider a partially verified problem in the next iteration of BaB. These already calculated hyperplanes can then be effectively re-used as constraints, making them appropriate for our clipping paradigms.

Using Final-Layer Bounds for Clipping. After a subdomain has been bounded, the final-layer's bounding plane(s) are retained after concretization if the subdomain cannot be fully verified. For the verification objective, f ( x ) ≥ 0 , it suffices to verify the input region, { x | a ( L ) ⊤ x + c ( L ) ≥ 0 } . In this case, the bounding plane acts as a constraint that separates this verified subset from the complimentary subset of the input domain that requires further analysis, i.e., { x | a ( L ) ⊤ x + c ( L ) ≤ 0 } . For verification problems with multiple output conditions in the form of f 1 ( x ) ≥ 0 ∧ f 2 ( x ) ≥ 0 ∧ · · · ∧ f T ( x ) ≥ 0 , we can have multiple linear bounds (1), one for each clause, and all of them can be used. Rather than treating each property separately, we jointly collect all relevant final-layer constraints and apply them using Algorithms 1 and 2, accelerating batch verification on a GPU.

Modifications to the Standard Input BaB Procedure. Algorithm 4 in Appendix C.3 outlines our modification to the standard input BaB loop, highlighted in brown. Our first key modification after bounding a batch of domains, is that we perform relaxed clipping after we split the domains. This reordering allows two child domains to inherit their parent's constraint after axis-aligned split. The inheritance enables a more tailored domain clipping procedure for each child, which is often more effective than clipping the parent domain directly. The resulting subdomains are reinserted into the domain list along with their associated constraints. In the subsequent bounding step, we perform complete clipping for the unstable neurons in each layer , where constraints are leveraged to directly to tighten the lower/upper bound objectives. When the number of unstable neurons is large, we heuristically select a subset of critical neurons for Complete Clipping; the selection heuristics are detailed in Appendix D.4. Neurons not selected for complete clipping still benefit indirectly, as their concretized bounds are refined due to the clipped domain inherited from the prior BaB iteration.

3.5 Integrating Clipping into Activation BaB Verification

We now demonstrate how our clipping framework naturally extends to branch-and-bound on the activation space. Our method is activation-agnostic , as it operates on general linear constraints derived from any activation split. For a given neuron z ( i ) j , BaB can introduce splits of the form z ( i ) j ≥ s or z ( i ) j ≤ s , where s is a split point ( s = 0 for a ReLU split)[53]. This inequality can be relaxed by bound propagation algorithms and expressed as a linear constraint on the input, g ⊤ x + h ≤ 0 , which is directly usable by our algorithms. This flexibility allows our framework to apply to networks with various activation functions, as demonstrated by our experiments on Vision Transformer models in Section 4. By assigning certain unstable neurons to either regime, we obtain linear relaxation split constraints that can further optimize the intermediate bounds , boosting verification efficacy. A key insight is that many constraints from multiple activation assignments can accumulate, potentially tightening intermediate bounds significantly.

Using Linear Constraints in Activation Space for Clipping. We prioritize neurons whose convex envelopes incur the largest relaxation error; for ReLU this typically coincides with 'unstable' units (Appendix A.1). At any BaB node with input box X t , linear bound propagation yields, for each neuron ( i, j ) ,

Then we can validate of activation-space linear constraints. If a branch assigns z ( i ) j ≥ s , then any feasible x must satisfy

since z ( i ) j ( x ) ≤ A ( i ) ⊤ j x + c ( i ) j . Symmetrically, for z ( i ) j ≤ s we obtain the necessary condition A ( i ) ⊤ j x + c ( i ) j ≤ s . Thus the activation-branching provide sound linear constraints in the input space that encode the assigned activation regime.

We can cache and reuse the initialized activation branching linear constraints. When bound initialization is executed at a node, we cache ( A , c ) and ( A , c ) for all 'unstable' neurons, and further use the constraints based on the branching domains during BaB to do (i) Relaxed Clipping : treat (7) (and its lower-bound analogue) as box-consistency constraints and update X t ↦→X ′ t in closed form following Algorithm. 2). (ii) Complete Clipping : on a selected set C ( p ) of neurons, we directly tighten their per-neuron affine bounds using the same constraints using Algorithm. 1. Neurons not in C ( p ) keep their original bounds, but still tighten indirectly when re-concretized over X ′ t .

Here, C ( p ) is the set of 'critical neurons' that are heuristically expected to benefit most from bound refinement. While one could apply our method to every neuron in the network, this may be challenging for networks with high-dimensional hidden layers. To address this, we propose a top-k objective selection heuristic that adaptively selects neurons based on the BaBSR intercept score [13]. This heuristic strategically prioritizes neurons whose refinement is most likely to tighten the overall verification bounds. A detailed justification for this choice, including an ablation study comparing BaBSR to other common heuristics, is provided in Appendix D.4. Neurons not selected by this heuristic retain their standard bounds as computed by CROWN but may still benefit indirectly from relaxed clipping. Since relaxed clipping is computationally lightweight and highly scalable, it should always be applied to refine the input domain as effectively as possible.

cartpole

Quadrotor-2D

Quad-2D-Large

Method

time(s)

subproblems

time(s)

subproblems

time(s)

subproblems

α,β

-CROWN (No clipping)

1602

54,260,909

timeout

timeout

Clip-and-Verify (Relaxed clipping)

484

16,453,971

↓

69

.

7%

209,504

2,630,043,050

timeout

Clip-and-Verify (Complete clipping)

142

2,438,359

↓

95

.

5%

78,818

1,112,917,436

↓

57

.

7%

104,614

1,472,433,971

(a) acasxu

(c) cifar-cnn-b-adv

(d) cifar10-resnet

(e) cifar100

(f) tinyimagenet

24

20

α,β-CROWN

Clip-and-Verify (Relaxed)

16

α,β-CROWN

Clip-and-Verify (Relaxed)

200

GCP-CROWN + mip cuts

β-CROWN

Clip-and-Verify (Relaxed + Reorder)

Clip-and-Verify (Relaxed + Reorder)

150

BICCOS

16

Clip-and-Verify (Complete)

12

Clip-and-Verify (Complete)

Clip-and-Verify

12

8

Time

8

100

Time Limit (200)

Clip-and-Verify + BICCOS

Clip-and-Verify + mip cuts

4

50

4

108

112

116

Number of Instances Verified

120

124

128

132

136

140

0

10

15

Number of Instances Verified

20

25

30

35

40

0

09

Number of Instances Verified

70

80

90

100

β-CROWN

100

1

B-CROWN

100

B-CROWN

250

GCP-CROWN + mip cuts

1

BICCOS

BICCOS

BICCOS

80

Clip-and-Verify

80

Clip-and-Verify

200

Clip-and-Verify

Clip-and-Verify + BICCOS

Clip-and-Verify + BICCOS

Clip-and-Verify + mip cuts

60

Time Limit (100)

60

Time Limit (100)

150

Clip-and-Verify + BICCOS

Time Limit (300)

40

40

100

50

20

20

0

40

45

50

55

09

0

100

105

110

115

120

130

0

110

115

120

125

130135

140

Number of Instances Verified

Number of Instances Verified

Number of Instances Verified

Modifications to the Standard Activation BaB Procedure. Our activation space BaB Algorithm 5, shown in Appendix C.4, iteratively uses activation split constraints to optimize the intermediate bounds via Complete Clipping (Theorem 3.1) . This strategy fully exploits accumulated constraints through complete clipping on intermediate bounds directly, bypassing costly LP solves.

4 Experiments

We evaluate the effectiveness of Clip-and-Verify on several benchmarks from VNN-COMP 20212024 [6, 49, 11, 10]. We first demonstrate its benefits on three benchmarks and three hard NN control system problems that are commonly solved with the input BaB procedure, then evaluate our approach on challenging activation space BaB benchmarks in VNN-COMPs, as well as SDP-FO benchmarks

*

Results from VNN-COMP 2021 report [6]. † from VNN-COMP 2022 report [49] ‡ from VNN-COMP 2024 report [10]

introduced in previous studies [19, 66]. Fig. 2 shows the overview of results of selected benchmarks. All results visualization and details about the configuration see Appendix D.1.

Verification Results on Input Split Benchmarks and Hard NN Control Systems. For evaluation, we focus on the following benchmarks from VNN-COMP 23 and VNN-COMP 24: acasxu , lsnc , and nn4sys . we have considered all input benchmarks and these three are the mostly challenging ones when solved using input BaB. We compare our approach to other baseline tools as shown in Table 1. Overall, Clip-and-Verify reduces the number of branches by over 50% and accelerating the verification process. For lsnc , we reduced the number of domains by 96%. We further tested our method on three challenging verification tasks from a recent study on provably stable neural network control systems [75, 41, 42]: cartpole , Quadrotor-2D , and Quadrotor-2D-Larger-ROA . These tasks require certifying Lyapunov-based stability over high-volume state domains, and can not be solved by existing verifiers. The challenge of the verification problem is that we need to verify inside the intersection of a large box and a level set of its Lyapunov function. Detailed settings see Appendix D.1. These instances therefore test whether a verifier can reason about Lyapunov decrease and boundary non-escape over large, physically meaningful state ranges, rather than over small adversarial balls. While general-purpose robustness verifiers are not designed for such volumes, our method targets exactly this regime. As shown in Table 2, baseline methods without clipping fail to solve these problems within the time limit. Both relaxed and complete clipping enable verification, with complete clipping demonstrating superior performance by drastically reducing both the number of visited BaB subproblems and the total runtime, turning previously intractable problems into verifiable ones.

Verification Results on General Activation Split Benchmarks Shown in Table 3, we further compare the proposed methods against a wide range of existing neural network verification toolkits on six challenging VNN-COMP benchmarks: oval22 , cifar10-resnet , cifar100-2024 , tinyimagenet-2024 , and vit-2024 Each method is evaluated in terms of average runtime (seconds) and the number of verified properties (# verified). A dash (-) denotes that the corresponding tool

was not applicable or did not support that particular benchmark. Clip-and-Verify with BICCOS attains state-of-the-art verification coverage on the evaluated VNN-COMP benchmarks, pushing closer to the theoretical upper bound in terms of the number of properties verified. When combined with BICCOS, whose tight bounding routines complement Clip-and-Verify's efficient splitting, coverage reaches to 131 and 144, underscoring the method's broad applicability and scalability. Besides of the ReLU nets, we further evaluated our method on a Vision Transformer (ViT) model [53], which contains non-ReLU layers such as Softmax . Verifying such architectures is challenging due to the complex, non-linear constraints introduced by these operators. The model tested has approximately 76k parameters and an input dimension of 3072. Tables 4, 6 show that Clip-and-Verify significantly outperforms the baseline, verifying more properties in about half the time and with nearly 45% fewer subproblems in ViT. This demonstrates the applicability of our framework on general networks.

Ablation Studies. We compare the BaB baseline without clipping, Relaxed Clipping only, and the full pipeline (Relaxed + Complete) in Table 1. We further vary the comparison of complete clipping and LP solvers in Section D.3, and scoring rule for choosing critical neurons (e.g., BaBSR intercept, envelope gap, bound width) in Complete Clipping in Section D.4. For each configuration we report time and visited subproblems under identical branching/timeout settings in Table 4 and Table 6; detailed breakdowns, per-model trends, and ablation protocols are in Appendix D.2.

5 Related Work

Verification becomes easier when we tighten the base relaxations that bound intermediate layers and contract the input region , so that a subsequent branch-and-bound (BaB) search explores fewer and easier subproblems. BaB frameworks unify many complete verifiers and show that stronger node bounds reduce tree size [14, 13, 55, 64, 77, 5]. Subsequent improvements include faster dual/activeset solvers and better branching [20, 66, 27]. Our clipping steps make each node strictly easier (smaller box and tighter intermediate relaxations) without spawning additional subproblems, and are therefore complementary to any branching heuristic. Our method contracts the box domain using linear constraints and injects the same constraints to tighten intermediate affine bounds, thus improving the node-wise base problem before any additional branching is introduced.

For tighter base relaxations, linear-relaxation-based bound propagation (e.g., CROWN [77] and DeepPoly [56]) provide fast base bounds. Beyond this, early single-neuron triangle relaxations such as PLANET [25] and the convex adversarial polytope/Wong-Kolter line [69] underpin modern propagation methods; symbolic/abstract-interpretation variants (DeepZ [55]/AI2 [29]/Neurify [64]/OSIP [31]) further improve hidden-layer bounds in practice. Multi-neuron relaxations overcome singleneuron limits by coupling activations [54, 2, 48, 57]. Beyond k-ReLU and PRIMA [54, 48], tightened layer-wise convex relaxations [59] and barrier-oriented analyses (e.g., for simplex inputs or active-set scaling) show how to systematically overcome single-neuron limits. Beyond pure LP, SDP-based relaxations with linear cuts capture neuron coupling when affordable [8, 40, 44, 18]. Orthogonal to envelopes, NN-specific cutting planes have been injected directly into bound propagation e.g., general cutting planes (GCP-CROWN [76]) and conflict-driven cuts (BICCOS [79]) to strengthen node-wise relaxations before or alongside BaB. Exact formulations (MILP/SMT) and their strengthened variants remain an important baseline and hybrid component [60, 2, 35, 36], though scalability constraints often shift emphasis toward stronger relaxations and BaB. Recent work also tightens bounds via optimization-driven rolling-horizon decomposition, effectively performing layerwise optimizationbased bound tightening to improve intermediate bounds [78]. Our Complete Clipping is orthogonal: instead of inventing new global envelopes, we reuse existing linear constraints produced during verification to tighten per-neuron affine bounds inside the same bound-propagation framework. On the other hand, Contract-Simple [7] contracts a zonotope using one output inequality. We generalize this in a way tailored to bound propagation: Relaxed Clipping applies arbitrary linear constraints from final-layer bounds or activation branchings to a box proxy with closed-form updates, avoiding repeated LP solves while yielding globally useful domain reductions reused across all neurons.

6 Conclusion

We proposed Clip-and-Verify , a framework that enhances the Branch-and-Bound paradigm in neural network verification through our novel linear constraint-driven domain clipping algorithms. An efficient and light-weight paradigm for tightening intermediate-layer bounds has been a remaining gap in neural network verification that has finally been addressed by our framework. Limitation and broader impacts discussed in Appendix E.

References

Matthias Althoff. An introduction to CORA 2015. In Proc. of the 1st and 2nd Workshop on Applied Verification for Continuous and Hybrid Systems , pages 120-151. EasyChair, December 2015.

Ross Anderson, Joey Huchette, Will Ma, Christian Tjandraatmadja, and Juan Pablo Vielma. Strong mixed-integer programming formulations for trained neural networks. Mathematical Programming , 183(1-2):3-39, 2020.

David Applegate, Mateo Díaz, Oliver Hinder, Haihao Lu, Miles Lubin, Brendan O'Donoghue, and Warren Schudy. Practical large-scale linear programming using primal-dual hybrid gradient. Advances in Neural Information Processing Systems , 34:20243-20257, 2021.

David Applegate, Mateo Díaz, Oliver Hinder, Haihao Lu, Miles Lubin, Brendan O'Donoghue, and Warren Schudy. Pdlp: A practical first-order method for large-scale linear programming. arXiv preprint arXiv:2501.07018 , 2025.

Stanley Bak. nnenum: Verification of relu neural networks with optimized abstraction refinement. In NASA Formal Methods Symposium , pages 19-36. Springer, 2021.

Stanley Bak, Changliu Liu, and Taylor Johnson. The second international verification of neural networks competition (vnn-comp 2021): Summary and results. arXiv preprint arXiv:2109.00498 , 2021.

Stanley Bak, Hoang-Dung Tran, Kerianne Hobbs, and Taylor T. Johnson. Improved geometric path enumeration for verifying relu neural networks. In Proceedings of the 32nd International Conference on Computer Aided Verification . Springer, 2020.

Ben Batten, Panagiotis Kouvaros, Alessio Lomuscio, and Yang Zheng. Efficient neural network verification via layer-based semidefinite relaxations and linear cuts. In IJCAI , pages 2184-2190, 2021.

Elena Botoeva, Panagiotis Kouvaros, Jan Kronqvist, Alessio Lomuscio, and Ruth Misener. Efficient verification of relu-based neural networks via dependency analysis. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 3291-3299, 2020.

Christopher Brix, Stanley Bak, Taylor T Johnson, and Haoze Wu. The fifth international verification of neural networks competition (vnn-comp 2024): Summary and results. arXiv preprint arXiv:2412.19985 , 2024.

Christopher Brix, Stanley Bak, Changliu Liu, and Taylor T Johnson. The fourth international verification of neural networks competition (vnn-comp 2023): Summary and results. arXiv preprint arXiv:2312.16760 , 2023.

Robin Brown, Edward Schmerling, Navid Azizan, and Marco Pavone. A unified view of sdp-based neural network verification through completely positive programming, 2022.

Rudy Bunel, Jingyue Lu, Ilker Turkaslan, Philip HS Torr, Pushmeet Kohli, and M Pawan Kumar. Branch and bound for piecewise linear neural network verification. Journal of Machine Learning Research , 21(42):1-39, 2020.

Rudy R Bunel, Ilker Turkaslan, Philip Torr, Pushmeet Kohli, and Pawan K Mudigonda. A unified view of piecewise linear neural network verification. Advances in Neural Information Processing Systems , 31, 2018.

Nicholas Carlini, Guy Katz, Clark W. Barrett, and David L. Dill. Ground-truth adversarial examples. CoRR , abs/1709.10207, 2017.

Shaoru Chen, Lekan Molu, and Mahyar Fazlyab. Verification-aided learning of neural network barrier functions with termination guarantees. arXiv preprint arXiv:2403.07308 , 2024.

Chih-Hong Cheng, Georg Nührenberg, and Harald Ruess. Maximum resilience of artificial neural networks. CoRR , abs/1705.01040, 2017.

Hong-Ming Chiu, Hao Chen, Huan Zhang, and Richard Zhang. Sdp-crown: Efficient bound propagation for neural network verification with tightness of semidefinite programming. In International Conference on Machine Learning (ICML) , 2025. ICML 2025.

Sumanth Dathathri, Krishnamurthy Dvijotham, Alexey Kurakin, Aditi Raghunathan, Jonathan Uesato, Rudy R Bunel, Shreya Shankar, Jacob Steinhardt, Ian Goodfellow, Percy S Liang, et al. Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming. Advances in Neural Information Processing Systems (NeurIPS) , 2020.

Alessandro De Palma, Harkirat Singh Behl, Rudy Bunel, Philip H. S. Torr, and M. Pawan Kumar. Scaling the convex barrier with active sets. International Conference on Learning Representations (ICLR) , 2021.

Alessandro De Palma, Rudy Bunel, Alban Desmaison, Krishnamurthy Dvijotham, Pushmeet Kohli, Philip HS Torr, and M Pawan Kumar. Improved branch and bound for neural network verification via lagrangian decomposition. arXiv preprint arXiv:2104.06718 , 2021.

Hai Duong, Linhan Li, ThanhVu Nguyen, and Matthew Dwyer. A dpll (t) framework for verifying deep neural networks. arXiv preprint arXiv:2307.10266 , 2023.

Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. Output range analysis for deep feedforward neural networks. In NASA Formal Methods Symposium , 2018.

Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, and Pushmeet Kohli. A dual approach to scalable verification of deep networks. Conference on Uncertainty in Artificial Intelligence (UAI) , 2018.

Ruediger Ehlers. Formal verification of piece-wise linear feed-forward neural networks. In Automated Technology for Verification and Analysis: 15th International Symposium, ATVA 2017, Pune, India, October 3-6, 2017, Proceedings 15 , pages 269-286. Springer, 2017.

Mahyar Fazlyab, Manfred Morari, and George J Pappas. Safety verification and robustness analysis of neural networks via quadratic constraints and semidefinite programming. IEEE Transactions on Automatic Control , 2020.

Claudio Ferrari, Mark Niklas Muller, Nikola Jovanovic, and Martin Vechev. Complete verification via multi-neuron relaxation guided branch-and-bound. arXiv preprint arXiv:2205.00263 , 2022.

Matteo Fischetti and Jason Jo. Deep neural networks as 0-1 mixed integer linear programs: A feasibility study. CoRR , abs/1712.06174, 2017.

Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. Ai2: Safety and robustness certification of neural networks with abstract interpretation. In 2018 IEEE Symposium on Security and Privacy (SP) . IEEE, 2018.

Julien Girard-Satabin, Michele Alberti, Francois Bobot, Zakaria Chihani, and Augustin Lemesle. Caisar: A platform for characterizing artificial intelligence safety and robustness. arXiv preprint arXiv:2206.03044 , 2022.

Vahid Hashemi, Panagiotis Kouvaros, and Alessio Lomuscio. Osip: Tightened bound propagation for the verification of relu neural networks. In Software Engineering and Formal Methods: 19th International Conference, SEFM 2021, Virtual Event, December 6-10, 2021, Proceedings , page 463-480, Berlin, Heidelberg, 2021. Springer-Verlag.

Patrick Henriksen and Alessio Lomuscio. Efficient neural network verification via adaptive refinement and adversarial search. In ECAI 2020 , pages 2513-2520. IOS Press, 2020.

Patrick Henriksen and Alessio Lomuscio. Deepsplit: An efficient splitting method for neural network verification via indirect effect analysis. In Proceedings of the 30th international joint conference on artificial intelligence (IJCAI21) , pages 2549-2555, 2021.

IBM. Ibm ilog cplex optimizer, 2022.

Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30 , pages 97-117. Springer, 2017.

Guy Katz, Derek A Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zelji´ c, et al. The marabou framework for verification and analysis of deep neural networks. In International Conference on Computer Aided Verification (CAV) , 2019.

Hans Kellerer, Ulrich Pferschy, and David Pisinger. Knapsack Problems . Springer Berlin, Heidelberg, 2004.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR) , 2015.

Panagiotis Kouvaros and Alessio Lomuscio. Towards scalable complete verification of relu neural networks via dependency-based branching. In IJCAI , pages 2643-2650, 2021.

Jianglin Lan, Yang Zheng, and Alessio Lomuscio. Tight neural network verification via semidefinite relaxations and linear reformulations. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7272-7280, 2022.

Haoyu Li, Xiangru Zhong, Bin Hu, and Huan Zhang. Neural contraction metrics with formal guarantees for discrete-time nonlinear dynamical systems. In 7th Annual Learning for Dynamics \ &Control Conference , pages 1447-1459. PMLR, 2025.

Haoyu Li, Xiangru Zhong, Bin Hu, and Huan Zhang. Two-stage learning of stabilizing neural controllers via zubov sampling and iterative domain expansion. arXiv preprint arXiv:2506.01356 , 2025.

Alessio Lomuscio and Lalit Maganti. An approach to reachability analysis for feed-forward relu neural networks. CoRR , abs/1706.07351, 2017.

Ziye Ma and Somayeh Sojoudi. Strengthened sdp verification of neural network robustness via non-convex cuts. arXiv preprint arXiv:2010.08603 , pages 715-727, 2020.

David R. Morrison, Sheldon H. Jacobson, Jason J. Sauppe, and Edward C. Sewell. Branch-andbound algorithms: A survey of recent advances in searching, branching, and pruning. Discrete Optimization , 19:79-102, 2016.

Christoph Müller, Francois Serre, Gagandeep Singh, Markus Püschel, and Martin Vechev. Scaling polyhedral neural network verification on gpus. Proceedings of Machine Learning and Systems , 3:733-746, 2021.

Mark Niklas Müller, Gleb Makarchuk, Gagandeep Singh, Markus Püschel, and Martin Vechev. Precise multi-neuron abstractions for neural network certification. arXiv preprint arXiv:2103.03638 , 2021.

Mark Niklas Müller, Gleb Makarchuk, Gagandeep Singh, Markus Püschel, and Martin Vechev. Prima: general and precise neural network certification via scalable convex hull approximations. Proceedings of the ACM on Programming Languages , 6(POPL):1-33, 2022.

Mark Niklas Müller, Christopher Brix, Stanley Bak, Changliu Liu, and Taylor T. Johnson. The third international verification of neural networks competition (vnn-comp 2022): Summary and results, 2023.

Aditi Raghunathan, Jacob Steinhardt, and Percy S Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In Advances in Neural Information Processing Systems (NeurIPS) , 2018.

Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. A convex relaxation barrier to tight robustness verification of neural networks. In Advances in Neural Information Processing Systems (NeurIPS) , 2019.

Karsten Scheibler, Leonore Winterer, Ralf Wimmer, and Bernd Becker. Towards verification of artificial neural networks. In MBMV , pages 30-40, 2015.

Zhouxing Shi, Qirui Jin, Zico Kolter, Suman Jana, Cho-Jui Hsieh, and Huan Zhang. Neural network verification with branch-and-bound for general nonlinearities. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems , 2025.

Gagandeep Singh, Rupanshu Ganvir, Markus Püschel, and Martin Vechev. Beyond the single neuron convex barrier for neural network certification. In Advances in Neural Information Processing Systems (NeurIPS) , 2019.

Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. In Advances in Neural Information Processing Systems (NeurIPS) , 2018.

Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An abstract domain for certifying neural networks. Proceedings of the ACM on Programming Languages , 3(POPL):130, 2019.

Harkirat Singh, M Pawan Kumar, Philip Torr, and Krishnamurthy Dj Dvijotham. Overcoming the convex barrier for simplex inputs. In Advances in Neural Information Processing Systems , 2021.

Chuangchuang Sun, Dong-Ki Kim, and Jonathan P How. Romax: Certifiably robust deep multiagent reinforcement learning via convex relaxation. In 2022 International Conference on Robotics and Automation (ICRA) , pages 5503-5510. IEEE, 2022.

Christian Tjandraatmadja, Ross Anderson, Joey Huchette, Will Ma, Krunal Patel, and Juan Pablo Vielma. The convex relaxation barrier, revisited: Tightened single-neuron relaxations for neural network verification. Advances in Neural Information Processing Systems (NeurIPS) , 2020.

Vincent Tjeng, Kai Xiao, and Russ Tedrake. Evaluating robustness of neural networks with mixed integer programming. arXiv preprint arXiv:1711.07356 , 2017.

Vincent Tjeng, Kai Y. Xiao, and Russ Tedrake. Evaluating robustness of neural networks with mixed integer programming. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019.

Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez, Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, and Taylor T Johnson. Nnv: The neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. In International Conference on Computer Aided Verification , pages 3-17. Springer, 2020.

Andreas Venzke and Spyros Chatzivasileiadis. Verification of neural network behaviour: Formal guarantees for power system applications. IEEE Transactions on Smart Grid , 12(1):383-397, 2020.

Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Efficient formal safety analysis of neural networks. In Advances in Neural Information Processing Systems (NeurIPS) , 2018.

Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Formal security analysis of neural networks using symbolic intervals. In USENIX Security Symposium , 2018.

Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. Advances in Neural Information Processing Systems , 34:29909-29921, 2021.

Tsui-Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane Boning, and Inderjit Dhillon. Towards fast computation of certified robustness for relu networks. In International Conference on Machine Learning , pages 5273-5282, 2018.

Eric Wong and J Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In ICML , 2018.

Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning (ICML) , 2018.

Eric Wong, Tim Schneider, Joerg Schmitt, Frank R Schmidt, and J Zico Kolter. Neural network virtual sensors for fuel injection quantities with provable performance specifications. In 2020 IEEE Intelligent Vehicles Symposium (IV) , pages 1753-1758. IEEE, 2020.

Haoze Wu, Omri Isac, Aleksandar Zelji´ c, Teruhiro Tagomori, Matthew Daggitt, Wen Kokke, Idan Refaeli, Guy Amir, Kyle Julian, Shahaf Bassan, et al. Marabou 2.0: a versatile formal analyzer of neural networks. In International Conference on Computer Aided Verification , pages 249-264. Springer, 2024.

Junlin Wu, Andrew Clark, Yiannis Kantaros, and Yevgeniy Vorobeychik. Neural lyapunov control for discrete-time systems. Advances in Neural Information Processing Systems , 36:29392955, 2023.

Kai Yuanqing Xiao, Vincent Tjeng, Nur Muhammad (Mahi) Shafiullah, and Aleksander Madry. Training for faster adversarial robustness verification via inducing relu stability. CoRR , abs/1809.03008, 2018.

Kaidi Xu, Huan Zhang, Shiqi Wang, Yihan Wang, Suman Jana, Xue Lin, and Cho-Jui Hsieh. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. International Conference on Learning Representations (ICLR) , 2021.

Lujie Yang, Hongkai Dai, Zhouxing Shi, Cho-Jui Hsieh, Russ Tedrake, and Huan Zhang. Lyapunov-stable neural control for state and output feedback: a novel formulation. In Proceedings of the 41st International Conference on Machine Learning , ICML'24. JMLR.org, 2024.

Huan Zhang, Shiqi Wang, Kaidi Xu, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. General cutting planes for bound-propagation-based neural network verification. Advances in Neural Information Processing Systems , 35:1656-1670, 2022.

Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems , 31, 2018.

Haoruo Zhao, Hassan Hijazi, Haydn Jones, Juston Moore, Mathieu Tanneau, and Pascal Van Hentenryck. Bound tightening using rolling-horizon decomposition for neural network verification. In Integration of Constraint Programming, Artificial Intelligence, and Operations Research: 21st International Conference, CPAIOR 2024, Uppsala, Sweden, May 28-31, 2024, Proceedings, Part II , page 289-303, Berlin, Heidelberg, 2024. Springer-Verlag.

Duo Zhou, Christopher Brix, Grani A Hanasusanto, and Huan Zhang. Scalable neural network verification with branch-and-bound inferred cutting planes. Advances in Neural Information Processing Systems , 37:29324-29353, 2024.

Appendix

Contents

A Formulations

A.1 Unstable Neurons and Complexity

Unstable neurons for General Activations Let z ( i ) j be the pre-activation of neuron ( i, j ) in layer i number j with interval bounds [ z ( i ) j , z ( i ) j ] , and let the component-wise activation be ϕ ( i ) ( · ) . Given a sound linear relaxation (convex envelope) for the scalar graph of ϕ ( i ) over [ z ( i ) j , z ( i ) j ] , denote by l ( i ) j ( z ) and u ( i ) j ( z ) its lower and upper affine bounds, and define the envelope gap (or relaxation error):

We call neuron ( i, j ) stable if δ ( i ) j = 0 (the relaxation is exact on the interval), and unstable if δ ( i ) j > 0 . This definition is activation-agnostic: for piecewise-linear (PWL) activations, stability is equivalent to [ z ( i ) j , z ( i ) j ] lying inside a single linear piece (e.g., for ReLU it reduces to z ( i ) j ≤ 0 or z ( i ) j ≥ 0 ); for smooth nonlinear activations one typically adopts a PWL relaxation, in which case instability means the interval spans at least two segments (hence a nonzero envelope gap).

Complexity. Let U be the number of unstable neurons at a BaB node. Once every unstable neuron has its activation region (piece) fixed, the network reduces to an affine map over the input box, and the remaining subproblem is convex. For PWL activations where each unstable neuron has at most

K admissible pieces, 1 the number of global activation patterns is bounded by K U . Consequently, any exact verifier that distinguishes activation pieces (e.g., MILP/SMT encodings with one discrete choice per unstable neuron, or BaB that branches on activation pieces) faces, in the worst case,

where T bound is the per-node cost to evaluate/tighten bounds. In particular, for ReLU networks K = 2 , yielding the familiar exponential scaling Ω(2 U ) ; thus any mechanism that reduces U (e.g., by stabilizing neurons via tighter pre-activation intervals) produces a multiplicative reduction in worst-case search, by a factor of K ∆ U when U decreases by ∆ U .

A.2 MIP Formulation and LP & Planet Relaxation

Early approaches pursue exact guarantees for verification through Mixed-Integer Linear Programming (MILP) [17, 43, 23, 28, 61, 73] and Satisfiability Modulo Theories (SMT) [52, 35, 15, 25].

The MIP Formulation The mixed integer programming (MIP) formulation is the root of many NN verification algorithms. Given the RELU activation function's piecewise linearity, the model requires binary encoding variables, or ReLU indicators δ only for unstable neurons. We formulate the optimization problem aiming to minimize the function f ( x ) , subject to a set of constraints that encapsulate the DNN's architecture and the perturbation limits around a given input x , as follows:

To initialize intermediate bounds for each neuron, we replace the original objective f ( x ) with the neuron's pre-activation value z ( i ) j . This lets us solve the following bounds for every neuron j in layer i , with i ∈ [ L -1] and j ∈ J ( i ) :

Here, the set J ( i ) comprises all neurons in layer i , which can be categorized into three groups: 'active' ( I +( i ) ), 'inactive' ( I -( i ) ), and 'unstable' ( I ( i ) ).

Next, the MIP formulation is initialized with the constraints

across all neurons and layers i . These bounds can be computed recursively, propagating from the first layer up to the i -th layer. However, since MIP problems involve integer variables, they are generally NP-hard, reflecting the computational challenge of this approach.

The LP and Planet relaxation. By relaxing the binary variables in (9k) to δ ( i ) j ∈ [0 , 1] , j ∈ I ( i ) , i ∈ [ L -1] , we can get the LP relaxation formulation. By replacing the constraints in (9i), (9j), (9k) with

we can eliminate the δ variables and get the well-known Planet relaxation formulation [25]. Both of these two relaxations are solvable in polynomial time to yield lower bounds.

A.3 Formulating Bounding Hyperplanes in Linear Bound Propagation

In a feedforward network, A ( i ) , A ( i ) , c ( i ) and c ( i ) must be derived for every linear layer preceding an activation layer, as well as the final layer of the network. In order to derive the hyperplane coefficients ( A ( i ) / A ( i ) ) and biases ( c ( i ) / c ( i ) ), at this i th layer, all preceding activation layers must have already had their inputs bounded. The following lemma describes how a ReLU activation layer may be relaxed which will be useful for defining bounding hyperplanes, A ( i ) , A ( i ) , c ( i ) and c ( i ) .

Lemma A.1. (Relaxation of a ReLU layer in CROWN). Given the lower and upper bounds of z ( i -1) j , denoted as l ( i -1) j and u ( i -1) j , respectively, the linear layer proceeding the ReLU activation layer may be lower-bounded element-wise by the following inequality:

where D ( i -1) is a diagonal matrix with shape R n i -1 × n i -1 whose off-diagonal entries are 0, and on-diagonal entries are defined as:

and b ( i -1) j is a vector with shape R n i -1 whose elements are defined as:

In the above definitions, α ( i -1) j is a parameter in range [0 , 1] and may be fixed or optimized as in [74].

Proof. For the j th ReLU at the ( i -1) th layer, it's result may be bounded as follows:

The right-hand side holds as this is the Planet-relaxation defined by equation (10). For the left-hand side, we first consider when z ( i -1) j ≤ 0 . For every input in this range, the result of the ReLU is σ ( z ( i -1) j ) = 0 . α ( i -1) j z ( i ) j forms a line for which inputs in this range will always produce a

non-positive result when α ( i -1) j ∈ [0 , 1] . For inputs in the range z ( i -1) j ≥ 0 , the result of the ReLU is σ ( z ( i -1) j ) = z ( i -1) j . This result is never exceeded by α ( i -1) j z ( i -1) j when α ( i -1) j ∈ [0 , 1] .

When the result, σ ( z ( i -1) j ) , is multiplied by a scalar such as W ( i ) j , a valid lower-bound of W ( i ) j σ ( z ( i -1) j ) requires a lower bound on σ ( z ( i -1) j ) when W ( i ) j ≥ 0 , and an upper bound on σ ( z ( i -1) j ) when W ( i ) j < 0 . Such lower and upper bounds are indeed produced by D ( i -1) j,j and b ( i -1) j , whose definitions are derived from the inequality displayed in equation (14). This concludes the proof.

Lemma A.1 suggests a recursive approach to bounding a neural network as the bounds at the i th layer depends on the bounds of the layer preceding it due to the dependence on l ( i -1) j and u ( i -1) j . This is indeed the case, and we may define our hyperplane coefficients as A ( i ) = Ω ( i, 1) W (1) where

To collect the remaining terms, we set c ( i ) = ∑ i k =2 ( Ω ( i,k ) W ( k ) b ( k -1) ) + ∑ i k =1 ( Ω ( i,k ) b ( k ) ) . To obtain an upper bound, Lemma A.1 and its proof may be adjusted accordingly where appearances of the inequalities W ( i ) ≥ 0 and W ( i ) < 0 are flipped. In doing so, we may repeat this recursive process in order to obtain A ( i ) and c ( i ) .

Though we have described how a ReLU feedforward network may be bounded, appropriately updating the definitions of D ( i ) and b ( i ) allows feedforward networks with general activation functions (that act element-wise) to be bounded. Such a general formulation is described in [77] that is similar to the template described above, and goes into further detail on how this formulation may be extended to quadratic bound propagation.

A.4 Verification Properties Given as Boolean Logical Formulae

Equation (16) is referred to as the canonical formulation [14, 13] as it is general enough to encompass any property involving boolean logical formulas over linear inequalities. To make this idea clear, consider a neural network whose output dimension is n L = 3 . We may wish to verify that the output corresponding to the true label, y 0 , is always greater than all other outputs, y 1 and y 2 , for every input in X . Explicitly, we want to verify:

To represent this formula, we may first introduce a new linear layer whose weight matrix is defined as C , sometimes referred to as a specification matrix :

This specification matrix allows the output neurons to be compared against one another. Applying this matrix to the output vector of the network, denoted as y := [ y 0 , y 1 , y 2 ] ⊤ , results in the following new output:

Next, we append a MinPool layer to the network which returns the minimum of these two results. Now, we may define a new neural network, f ′ ( x ) , which appends the aforementioned MinPool and linear layer to f ( x ) , resulting in the final output, min { y 0 -y 1 , y 0 -y 2 } . It should now be apparent that if the property described by equation (17) can be verified as true, then the following must hold:

We may now proceed to produce a lower bound, f ′ ( x ) , and only until f ′ ( x ) > 0 , can we formally state that the network will correctly classify all inputs in X . In the scenario that we instead want to verify the property ( y 0 > y 1 ∨ y 0 > y 2 ) for all inputs in X , we may simply redefine f ′ ( x ) by replacing the MinPool layer with a MaxPool layer which would return max { y 0 -y 1 , y 0 -y 2 } .

Finally, suppose we want to verify that the output corresponding to the true label is greater than all other outputs with some additional margin m such that ( y 0 > y 1 + m ∧ y 0 > y 2 + m ) , ∀ x ∈ X where m> 0 (i.e. the network should always classify the first label with additional relative confidence defined by m ). Then the only modification is to also incorporate a bias vector, t := [ -m, -m ] ⊤ , at the final linear layer where C was defined. This bias vector is sometimes referred to as the threshold . This results in the final output, f ′ ( x ) = min { y 0 -y 1 -m, y 0 -y 2 -m } . Hence, equation (16) is general enough to encompass more complex queries.

B Proofs

B.1 Statement and proof of Lemma B.1

We first introduce a preparatory result that establishes a closed-form solution for a linear optimization problem over a hyper-rectangular feasible set.

Lemma B.1 (Dual Norm Concretization for Hyper-Rectangular Domains) . Let ϵ ∈ R n with ϵ i > 0 for all i ∈ [ n ] , and define the hyper-rectangular domain:

For any vector v ∈ R n , the following hold:

where | v | ∈ R n + denotes the component-wise absolute value of v .

Proof. We begin by rewriting the feasible set as

where ◦ denotes the Hadamard (component-wise) product. Then, the maximization problem becomes

where the final equality follows from the definition of the dual norm of the ∞ -norm. The derivation for the minimization problem follows analogously by replacing the maximization with a minimization, which flips the sign of | v | ⊤ ϵ . Thus, the claim follows.

B.2 Proof of Theorem 3.1

Proof. The problem (2) is a linear program over a compact convex set X with an additional linear constraint. Let feasible set F = { x ∈ X : g ⊤ x + h ≤ 0 } be the feasible set for (2).

Case 1: F = ∅ , then by definition, L ⋆ = + ∞ . In this case, it implies that for all x ∈ X , g ⊤ x + h > 0 .

Case 2: F is non-empty. Since X is compact and F is a closed non-empty subset of X (as it's the intersection of X with a closed half-space), F is also compact. The objective function a ⊤ x + c is continuous. Therefore, L ⋆ is finite and attained. We introduce the Lagrangian for problem (2) by partially dualizing the constraint g ⊤ x + h ≤ 0 :

The primal problem can be written as:

To swap the min and sup , we can apply Sion's Minimax Theorem. Let K = X (a compact convex set in R n ) and M = { β ∈ R : β ≥ 0 } (a closed convex set in R ). The function L ( x , β ) has the following properties: 1) For any fixed β ∈ M , L ( x , β ) = ( a + β g ) ⊤ x +( c + βh ) is linear in x , and thus convex and continuous on K . 2) For any fixed x ∈ K , L ( x , β ) = ( g ⊤ x + h ) β +( a ⊤ x + c ) is linear in β , and thus concave and continuous on M . Since these conditions are met, Sion's Minimax Theorem states:

Therefore,

Since F is non-empty, L ⋆ is finite, implying that the supremum is attained (or is the limit if approached at infinity, but D ( β ) is continuous), so we can write max instead of sup . Let d ( β ) = min x ∈X L ( x , β ) .

The inner minimization is optimizing a linear function ( a + β g ) ⊤ x over the hyper-rectangle X . Using Lemma B.1 (with v = a + β g ):

where | a + β g | ⊤ ϵ = ∑ n j =1 | a j + βg j | ϵ j . Substituting this into the expression for d ( β ) , we get:

This is exactly the dual objective function D ( β ) defined in the theorem statement (3). Thus, we have established that L ⋆ = max β ≥ 0 D ( β ) . Then we analyze the properties of the dual objective D ( β ) :

Concavity: The dual function d ( β ) is always concave, as it is the pointwise minimum of a family of functions that are affine in β (indexed by x ∈ X ).

̸

Piecewise-Linearity: The term -| a + β g | ⊤ ϵ = -∑ n j =1 | a j + β g j | ϵ j involves the absolute value function. Each term -| a j + β g j | ϵ j is concave and piecewise-linear, with a breakpoint (a point where the slope changes) at β = -a j / g j (if g j = 0 ). The other terms in D ( β ) are linear in β . Since D ( β ) is a sum of concave piecewise-linear functions and linear functions, it is itself concave and piecewise-linear. The breakpoints of D ( β ) are the collection of all values β = -a j / g j ≥ 0 where g j = 0 .

̸

Thus, we need to maximize the concave, piecewise-linear function D ( β ) over the interval [0 , ∞ ) . Since D ( β ) is concave, its maximum over a convex set occurs either at a point where the supergradient contains zero, or potentially at the boundary point β = 0 . Because D ( β ) is piecewiselinear, its super-gradient ∂D ( β ) is constant within the linear segments between breakpoints. At a breakpoint β k , the super-gradient is an interval [ ∂D ( β -k ) , ∂D ( β + k )] (the range between the left and right derivatives). The maximum occurs at a point β ⋆ such that 0 ∈ ∂D ( β ⋆ ) . This β ⋆ must be either β = 0 (if the derivative is non-positive for β > 0 ) or one of the breakpoints β k > 0 where the derivative changes sign from positive to non-positive (i.e., 0 ∈ [ ∂D ( β -k ) , ∂D ( β + k )] ), or the function increases indefinitely (which corresponds to an infeasible or unbounded primal, but we assumed feasibility and the primal is bounded over the compact X , so L ⋆ is finite, thus the dual maximum is finite).

Therefore, the maximum L ⋆ can be found non-iteratively by:

Identifying all non-negative breakpoints β k = -a j / g j ≥ 0 .

Sorting these unique breakpoints 0 = β 0 < β 1 < · · · < β p .

Evaluating the derivative (slope) of D ( β ) within each segment ( β k , β k +1 ) and potentially at β = 0 .

Finding the point β ⋆ (either 0 or some β k ) where the slope transitions from non-negative to non-positive. The value D ( β ⋆ ) is the maximum L ⋆ .

This process involves a finite number of analytical calculations (evaluating slopes and function values at breakpoints) rather than iterative optimization, justifying the claim of efficiency.

B.3 Equivalence to the Continuous Knapsack Problem

The primal problem in (2) is mathematically equivalent to the continuous (or fractional) knapsack problem. We can demonstrate this equivalence through a change of variables. Let x 0 = ˆ x -ϵ be the lower bound and x 1 = ˆ x + ϵ be the upper bound. We transform x ∈ [ x 0 , x 1 ] to y ∈ [0 , 1] n using:

Substituting this into the primal problem min x ∈X { a ⊤ x + c | g ⊤ x + h ≤ 0 } gives:

Let r = -2 ϵ ⊙ a , s = 2 ϵ ⊙ g , and t = -( g ⊤ (ˆ x -ϵ ) + h ) . The problem becomes the standard knapsack form:

This problem, even with negative coefficients, can be solved with a greedy algorithm [37]. The efficiency ratios r j /s j used for sorting in the greedy algorithm are:

These ratios are identical to the breakpoints in our dual objective function D ( β ) . This confirms a line-by-line correspondence: our dual optimization algorithm, which sorts breakpoints to find where the super-gradient contains zero, is algorithmically equivalent to the greedy knapsack algorithm, which sorts by efficiency ratios. Both have the same O ( n log n ) time complexity.

B.4 Proof of Theorem 3.2

̸

Direct Intuitive Proof A more direct and intuitive proof for Theorem 3.2 exists. We wish to solve x ( new ) i = max x ∈X { x i | a ⊤ x + c ≤ 0 } . Since only x i is in the objective, we can set all other variables x j (for j = i ) to values within their box domain [ˆ x j -ϵ j , ˆ x j + ϵ j ] that make the constraint a ⊤ x + c ≤ 0 as loose as possible, thereby maximizing the 'budget' for x i .

̸

To loosen the constraint, we must minimize the term ∑ j = i a j x j . This is achieved by setting each x j to its extreme:

If a j > 0 , we set x j to its lower bound, x j = ˆ x j -ϵ j .

If a j < 0 , we set x j to its upper bound, x j = ˆ x j + ϵ j .

This worst-case minimum for the sum can be written compactly as ∑ j = i ( a j ˆ x j - | a j | ϵ j ) . We substitute this minimum sum back into the constraint:

̸

̸

Assuming a i > 0 , we can solve for x i to find its new upper bound:

̸

̸

This value is x ( clip ) i , as defined in Theorem 3.2. The final bound x ( new ) i is the minimum of this value and the original upper bound x i . The case for a i < 0 (updating the lower bound x i ) follows analogously. A detailed proof is shown below:

Proof. We consider the upper bound; the lower bound can be derived analogously. First, we rewrite the input region as

where ˆ x = x + x 2 and ϵ = x -x 2 . Suppose the linear inequality constraint is given by a ⊤ x + b ≤ 0 , where a ∈ R n and b ∈ R . We note that the intersection X ∩ { x ∈ R n : a ⊤ x + b ≤ 0 } is nonempty if and only if

by Lemma B.1. Henceforth, we will assume this inequality is satisfied.

We now compute

where third line follows from Sion's minimax theorem since X is a compact set, and the final line follows from Lemma B.1.

Rearranging the term yields

̸

̸

We now analyze three cases for a i and derive a closed-form expression for the scalar minimization. Recall that we have assumed 0 ≥ a ⊤ ˆ x + c -∑ n i =1 a i ϵ i or equivalently ∑ j = i ϵ j | a j | -a ⊤ ˆ x -b ≥ -ϵ i | a i | . We have:

a i > 0 : The function ϵ i | 1 -λa i | attains its minimum at λ = 1 a i > 0 , with slope ϵ i a i on the right and -ϵ i a i on the left. Thus:

̸

If | ∑ j = i ϵ j | a j | -a ⊤ ˆ x -b | ≤ ϵ i a i then the minimum of (21) is attained at λ ⋆ = 1 a i and

̸

̸

If ∑ j = i ϵ j | a j | -a ⊤ ˆ x -b > ϵ i a i then λ ⋆ = 0 and

a i < 0 : In this case, the minimizing λ = 1 a i < 0 is infeasible to (21). Hence, λ ⋆ = 0 and x ( new ) i = x i .

̸

̸

with F (0) = X .

(Order-Independent Bounds): The variable-wise bounds satisfy:

These bounds are independent of π if computed via F = ⋂ m j =1 F j .

̸

(Order-Dependent Refinement): If constraints are intersected sequentially (i.e., F ( k ) depends on π ), then there exist permutations π 1 = π 2 such that:

̸

Proof. Order-Independent Bounds. Same as Corollary B.2. The inequalities follow from F ⊆ F j for all j . Simultaneous intersection satisfies:

as the intersection F cannot exceed the tightest bound from any F j .

Order-Dependent Refinement. Let F ( k ) = ⋂ k t =1 F π ( t ) . For dependent constraints (e.g., F 1 bounds x 1 , F 2 depends on x 1 ), the sequence F ( k ) depends on π . A counterexample with F 1 : x 1 + x 2 ≤ 2 and F 2 : x 1 -x 2 ≤ 0 shows:

Intersecting F 1 first gives x 1 ≤ 2 -x 2 , then F 2 further tightens x 1 ≤ x 2 .

Intersecting F 2 first gives x 1 ≤ x 2 , then F 1 tightens x 1 ≤ 1 .

The final bounds differ: x 1 ≤ min(2 -x 2 , x 2 ) vs. x 1 ≤ 1 .

C Algorithms

C.1 Overview

Fig. 3 shows the pipeline of our algorithm.

̸

a i = 0 : Here, the objective function in (21) becomes affine in λ . Since ∑ j = i ϵ j | a j | -a ⊤ ˆ x -b ≥ 0 , we again have λ ⋆ = 0 and x ( new ) i = x i .

In summary, if ϵ ⊤ | a | -a ⊤ ˆ x -b < 0 then ∅ = X ∩ { x ∈ R n : a ⊤ x + c ≤ 0 } ; otherwise,

̸

̸

This completes the proof.

B.5 Proof of Order Dependency

Here we briefly introduce Theorem B.2 to demonstrate why sequential processing the constraints for clipping achieving better results.

Theorem B.2 (Order Dependency of Constraint Intersection) . Let X = { x ∈ R n : x L ≤ x ≤ x U } be a box domain, and let F j = { x ∈ X : a ⊤ j x + c j ≤ 0 } denote the feasible set for the j -th constraint. Define the full feasible set as F = ⋂ m j =1 F j .

Let π : { 1 , . . . , m } → { 1 , . . . , m } be a permutation (i.e., a reordering) of the constraints. For sequential intersection, define:

Linear

Relaxed (Input) Clipping

CROWN Bounding + Complete Clipping

Branching

For every subdomain, use

constraints to clip the input space

Split domains on activation or

input space

Bound domains in parallel

Batch of domains:

Added Method

Split

Linear

Prune and Add

Remove verified domains

(in green) and add the rest

back to the domain list.

(relaxed)

clip

C.2 Sequential Clipping for Multiple Constraints

When several linear constraints are present, we apply Theorem 3.2 sequentially to each row of Ax + c ≤ 0 . Algorithm 3 outlines this procedure: First, for each constraint k , compute ˆ x and ϵ from the current bounds, x and x . Then, use Theorem 3.2 to refine x i and x i for all i . Finally, proceed to the next constraint, using the newly clipped bounds as the domain. Note that if x i > x i occurs in any dimension, the input region of this subproblem is infeasible, and we can directly verify this subproblem without performing further verification.

Because the bounds are updated after each constraint, the final domain is an over-approximation of the true feasible region under all constraints simultaneously. However, it is still far more efficient than solving a multi-constraint system in one shot, making it well suited for large-scale verification where we repeatedly clip domains across many subproblems.

By clipping the domain repeatedly, Algorithm 3 retains enough precision to prune large regions yet remains computationally lightweight enough for repeated invocation on many subdomains during verification. It may even be the case that the constraints passed to our domain clipping algorithm reveal the region is entirely infeasible . Such a scenario may occur when two Activation assignments

̸

̸

̸

admit an infeasible domain, or when the desired property admits multiple constraints that produce infeasibility. In such scenarios, Algorithm 3 will return clipped bounds where x will be smaller than x along some dimension(s). In the context of branch-and-bound, this subdomain may be effectively pruned, avoiding the process of running bound propagation once more on the domain, avoiding unnecessary computations

C.3 Algorithms for Clip-and-Verify in Input Branch-and-bound Scheme

Algorithm 4 Clip-and-Verify for Input Branch-and-bound

Require: f : model to verify; n : batch size; timeout : time-out threshold

1: D Unknown , f ← Init( f, ∅ ) {Initialize the set of unknown subdomains D Unknown and global bound f }

2: while |D Unknown | > 0 and not timed out do

4: {C i } n i =1 ← Top -K _ Heuristic( {X i , A prev i , c prev i } n i =1 ) {Determine critical neurons C i using a top-k heuristic over each domain and its previous hyperplanes.}

3: {X i , A prev i , c prev i } n i =1 ← Batch _ Pick _ Out( D Unknown , n ) {Pick up to n subdomains X i and their associated hyperplanes ( A prev i , c prev i ) from the previous iteration.}

5: ( f X 1 , A X 1 , c X 1 , . . . , f X n , A X n , c X n ) ← Solve _ Bound ( f, { X i , C i } n i =1 ) {Compute bounds and plane coefficients on each subdomain; Refine critical neurons using planes as constraints and Complete Clipping .}

6: {X -i , X + i , A i , c i } n i =1 ← Batch _ Split ( {X i , A i , c i } n i =1 ) {Split each subdomain; share hyperplanes with both children.}

7: { x ( -) i , x (+) i } n i =1 ← Relaxed _ Clipper ( {X -i , X + i , A i , c i } n i =1 ) {Refine input box on each child subdomain using the plane coefficients.}

8: {X -i , X + i } n i =1 ← Domain _ Update ( { x ( -) i , x (+) i } n i =1 ) {Update each child subdomain's input bounds.}

9: D Unknown ← D Unknown ∪ Domain _ Filter ( [ f X -1 , X -1 , A ( -) 1 , c ( -) 1 ] , . . . ) {Filter out verified/infeasible subdomains. Retain unknowns and their bounding hyper-planes.}

Ensure: UNSAT if |D Unknown | = 0 else Unknown

Algorithm 4 describes our modifications to the standard BaB procedure. When all subproblems can be verified, then the verification problem is referred to as UNSAT (i.e., the complementary property, ∃ x ∈ X , f ( x ) < 0 , is unsatisfiable ), and the network is safe from counter-examples. Otherwise, it is insufficient to determine if the property is UNSAT without further refinement or falsification.

C.4 Algorithms for Clip-and-Verify in Activation Branch-and-bound Scheme

Algorithm 5 Clip-and-Verify for Activation Branch-and-bound

Require: f : model to verify; n : batch size; timeout : time-out threshold

1: D Unknown , f ← Init( f, ∅ ) {Initialize the set of unknown subdomains D Unknown and global bound f }

2: { A ( j ) , c ( j ) , A ( j ) , c ( j ) , unstable_neuron(idx) ( j ) } J j =1 ← Get _ Constraints( f ) {Retrieve the set of coefficients and biases for each unstable neurons' upper and lower bound and get the indices of unstable neurons during Bound Propagation.}

3: Domain _ Clipper ← Init _ Clipper( { A ( j ) , c ( j ) , A ( j ) , c ( j ) , idx ( j ) } J j =1 ) {Initialize the Domain Clipper with the set of constraints information.}

4: while |D Unknown | > 0 and not timed out do

6: {Z -i , Z + i } n i =1 ← Batch _ Split ( {Z i } n i =1 ) {Split each subdomain (e.g., Activation split or input split) into two child subdomains.}

5: {Z i } n i =1 ← Batch _ Pick _ Out( D Unknown , n ) {Pick at most n subdomains from the unknown set.}

7: {C ( -) i , C (+) i } n i =1 ← Top -K _ Heuristic( {X i } n i =1 ) {Determine critical neurons using a top-k heuristic (e.g. BaBSR) over each domain.}

8: { x ( -) i , interm _ bds ( -) i , x (+) i , interm _ bds (+) i } n i =1 ← Domain _ Clipper ( {Z -i , Z + i , C ( -) i , C (+) i } n i =1 ) {Apply Relaxed Clipping to child subdomain's input and Complete Clipping on the critical neurons. Constraints are the Activation splits.}

9: {Z -i , Z + i } n i =1 ← Domain _ Update ( { x ( -) i , interm _ bds ( -) i , x (+) i , interm _ bds (+) i } n i =1 ) {Update each child subdomain's input and intermediate bounds.}

10: ( f Z -1 , . . . , f Z + n ) ← Solve _ Bound ( f, { x i , interm _ bounds i , Z i } ) {Compute bounds on each newly clipped subdomain using a bound propagation solver.}

11: D Unknown ←D Unknown ∪ Domain _ Filter ( [ f Z -1 , Z -1 ] , . . . ) {Filter out verified/infeasible subdomains. Keep remaining unknown subdomains in D Unknown .}

Ensure: UNSAT if |D Unknown | = 0

else Unknown

Algorithm 5 begins by initializing the verification procedure. First, we call Init( f, ∅ ) to obtain an empty set of partial Activation assignments (or subdomains) along with an initial global lower bound f for the property to be checked. This global lower bound can, for example, be the result of a quick bounding pass. The set D Unknown is then populated with a single 'root' subdomain representing the entire input domain.

Next, we retrieve the constraint information for all neurons (Line 2). Specifically, Get _ Constraints( f ) returns the linear coefficients and biases used to bound each neuron's activation, distinguishing between the lower-bounding ( A ( j ) , c ( j ) ) and upper-bounding ( A ( j ) , c ( j ) ) linear functions. It also identifies indices of 'unstable' Activation neurons whose ranges straddle zero. These constraints will later be used to restrict the feasible input region using both our Relaxed Clipping and Complete Clipping algorithms.

We then initialize the Domain Clipper (Line 3) with the gathered linear constraints. This Clipper component will be invoked whenever we branch on a Activation neuron, so that the corresponding partial assignment (e.g., x ( j ) k ≥ 0 ) is 'pushed back' onto the input domain. In doing so, we clip the subdomain's input box by applying Theorem 3.2 (or its extensions) to incorporate these newly introduced constraints, thus discarding parts of the input space that violate them. In addition, when given a set of critical neurons, these Activation split constraints will be used to directly refine the intermediate neurons using Theorem 3.1.

The main loop (Lines 5-11) iterates until either no subdomains remain unknown or a time-out is reached. In each iteration, we pick up to n unknown subdomains from D Unknown (Line 5) for batched parallel processing. Each subdomain Z i is then split (Line 6) along one or more unstable Activation neurons, creating child subdomains in which each split neuron is fixed to either the active ( ≥ 0 ) or inactive ( ≤ 0 ) regime.

At Line 7, we invoke a top-k heuristic (e.g. BaBSR) in order to determine the set of 'critical neurons' that would contribute the most to providing a stronger convex relaxation if their bounds were to be refined.

At Lines 8-9, we invoke the Domain Clipper on these newly formed child subdomains. The Clipper translates each Activation assignment into a linear constraint on the input, then refines (or 'clips') the child subdomain's input bounds, and the bounds of the 'critical neurons' in each subdomain with respect to the heuristic choices from the step prior. This ensures that any portion of the parent domain that contradicts the new constraint is removed. Once clipped, the child subdomains' intermediate bounds are also updated (Line 10) so that subsequent bounding calculations reflect the tighter input ranges.

We then compute bounds on the newly clipped subdomains (11) using a chosen method-often a fast bound-propagation tool such as CROWN or a lightweight LP solver. This step yields lower bounds f Z ± i on the network outputs for each subdomain. If these bounds confirm that the property holds (e.g., a robustness margin remains non-negative), the subdomain is verified and can be pruned. If the subdomain is infeasible (e.g., constraints are contradictory), it is also removed. All remaining subdomains (still 'unknown') return to D Unknown for further splitting.

Finally, the loop terminates once there are no unknown subdomains left or the time-out is reached. If D Unknown becomes empty, we conclude UNSAT, signifying that no violating input (counterexample) exists within any subdomain. Otherwise, we return 'Unknown,' indicating that verification was not completed in time.

Overall, this procedure reflects a standard Activation BaB flow, except that an additional 'domain clipping' step (Lines 8-9) is inserted after each split, leveraging partial Activation assignments to refine the input domain and select intermediate neurons before the next bounding pass. By applying our linear constraint-driven clipping algorithms whenever new constraints appear, we gain significantly tighter intermediate-layer bounds and thus reduce the branching burden throughout the verification process.

C.5 2D Toy Example

To illustrate our clipping verification approach, we consider a simple two-layer ReLU feed-forward network defined as

where w (2) , b (1) , x ∈ R 2 , W (1) ∈ R 2 × 2 , and σ ( · ) denotes the element-wise ReLU activation. We aim to verify the property

The input domain X is defined as an ℓ ∞ -box centered at ˆ x = [0 . 5 , -0 . 5] ⊤ with half-widths ϵ = [1 . 5 , 1 . 5] ⊤ . Equivalently, the domain can be expressed using its endpoints x = [ -1 , -2] ⊤ and x = [2 , 1] ⊤ . The network parameters are specified as

CROWN Bound. As a warm up, let's bound the network using the CROWN algorithm without utilizing relaxed nor complete clipping. We begin by computing the pre-activation bounds of the intermediate layer which are obtained by concretizing the first layer's affine transformation:

Substituting the parameters into equation (24b) yields z = [ -2 , -13] ⊤ and z = [22 , 5] ⊤ . These neurons are then passed to a ReLU activation function, and one should notice that for each input, we have that z 1 < 0 < z 1 and z 2 < 0 < z 2 . In this scenario, we have that the ReLU neurons are unstable , meaning that we cannot bound the non-linearity exactly, however, we can still get sound linear bounds on the output of the ReLU activation. Using Lemma A.1, we construct the diagonal matrices and corresponding vectors,

D and b are used to create lower bounding planes on the output of the ReLU activation where α 1 and α 2 are real numbers limited to the range [0 , 1] . These values may be optimized, however we will always fix α 1 = α 2 = 1 when lower bounding unstable neurons in this toy example. D and d are upper bounding planes on the output of the ReLU activation, and its construction is derived from the Planet relaxation [25]. It may be verified that for inputs z in the range [ z , z ] ,

We next lower bound the network output. Because the post-activation vector σ ( z ) is passed through the final linear layer, a coordinate-wise sign on the final-layer weights determines whether to use upper or lower affine bounds for each neuron. Concretely,

so that the final-layer lower bound with respect to z is

where w (2) , + zeros out entries which are negative and w (2) , -zeros out entries which are positive. In our example, these vectors are w (2) , + = [1 , 0] ⊤ and w (2) , -= [0 , -1] ⊤ . The final step is to produce a lower bounding hyperplane of the final layer with respect to the network's input. So far, we've only related this lower bound to the intermediate input, z . Our final step is quite simple as we know that z = W (1) x + b (1) . Using this relation, we can 'back-propagate' our affine relaxations to the input as follows,

Let us introduce the following variables,

By construction, we have that the output of the network is lower bounded by the following linear relaxation,

Using Hölder's inequality, we can solve for the minima of this lower bounding plane,

As this lower bound is too loose, we cannot verify our desired property, f ( x ) ≥ 0 , ∀ x ∈ X .

Relaxed Clipping. We first tackle this example using our efficient yet approximate clipping algorithm termed relaxed clipping . Given a set of constraints, the objective of relaxed clipping is to compute the smallest input box representation that satisfies those constraints. This step is performed once per round of branch-and-bound, and the resulting refined input box is shared by all neurons during the concretization step. Consequently, this refinement has the potential to tighten bounds across multiple neurons, including those in the final layer.

Consider a bound-propagation-based verifier executing branch-and-bound by splitting over the activation space. For the remainder of this subsection, we focus on the case where the first ReLU

neuron is split into its non-positive (inactive) region, i.e., z 1 ≤ 0 . This split decision can be reinterpreted as a linear constraint on the input,

Since the input x is two-dimensional, our goal is to tighten the lower and upper bounds of each input dimension under this constraint. Formally, we solve the following optimization problems:

From Theorem 3.2, one potential solution along each dimension is given by

̸

Each dimension is then updated as follows:

For the first dimension ( i = 1 ), since W (1) 1 , 1 > 0 , the upper limit may be refined if x ( clip ) 1 < x 1 . For the second dimension ( i = 2 ), where W (1) 1 , 2 < 0 , the lower limit may be refined if x ( clip ) 2 > x 2 . Substituting the given parameters yields:

It is indeed the case that x ( clip ) 1 < x 1 and x ( clip ) 2 > x 2 , so the limits of the refined input domain are now,

The new box representation may also be characterized by its center and half-widths:

yielding the refined input domain

We now proceed to bound the network using the CROWN algorithm under X ( rc ) in place of the original domain, X . Substituting this domain into Eq. (24) produces z ( rc ) = [ -2 , -13] ⊤ and z ( rc ) = [2 , -57 / 21] ⊤ . Both neurons exhibit tighter bounds, and notably, z ( rc ) 2 < 0 , which ensures that the input to the second ReLU neuron is strictly non-positive across X ( rc ) . Since the ReLU function is piecewise linear, this implies an exact post-activation bound:

One interesting observation to point out is that with this clipped domain, z ( rc ) 1 = 2 improves dramatically from the original upper bound, z 1 = 22 . However, because our constraint originated

from z 1 ≤ 0 , the true maximum is in fact zero. In this case, one could enforce this upper bound to be zero, but keep in mind that relaxed clipping algorithm is designed for generality, thus constraints may arise from split decisions at any neuron in the network where forcing z ( rc ) 1 = 0 would not be valid. This example illustrates that relaxed clipping, while lightweight and compatible with CROWN, remains an approximation and may yield suboptimal yet informative bounds. Rather than enforcing the true maximum, we retain the relaxed clipping result, z ( rc ) 1 = 2 .

Applying Lemma A.1, we derive the linear post-activation bounds with respect to the pre-activation inputs:

For simplicity, let α 1 = 1 . Notice that for the second neuron, we have that,

confirming that this ReLU neuron is inactive and its output can be exactly bounded as zero.

Finally, performing the same backpropagation procedure as before yields

This represents an improvement over the previous bound, although the property remains unverified.

Complete Clipping. Relaxed clipping can be viewed as an indirect approach towards refining the bounds of the neural network as a smaller input representation can potentially yield improvement in the concretization step when forming the neurons' bounds. Complete clipping on the other hand is a direct approach towards refining the bounds on the network as the optimization objective specifically targets the neuron's bounds rather than the shared input representation.

In practice, relaxed clipping is extremely lightweight, and its operations are well-suited for GPUs. Thus, it is often sensible to combine relaxed and complete clipping in the verification pipeline. In this example, however, we aim to clearly distinguish the two methods. Therefore, we use the original input domain, X , rather than the refined one, X ( rc ) , and retain the ReLU split constraint z 1 ≤ 0 .

Complete clipping operates directly on the bounds of each neuron, refining them via constrained optimization. The first step is to bound the preactivation bounds which may be formulated as:

For this two-dimensional problem, this amounts to targeting the lower and upper bound of both neurons at the intermediate layer, resulting in a total of four constrained optimization subproblems. However, we will only focus on refining the upper bounds as we will soon see that this will be sufficient for verifying our desired property in this toy example.

As discussed earlier, the split constraint on the first neuron implies that its true upper bound is trivially z ⋆ 1 = 0 . With a single constraint, Theorem 3.1 guarantees optimality, i.e., z ( cc ) 1 = z ⋆ 1 = 0 , which can be verified using Algorithm 1 2 . For a single constraint, complete clipping attains the exact optimal solution through enumeration, without relying on projected gradient methods or LP solvers, no longer serving as an approximation such as the case with relaxed clipping. Even when multiple constraints are present (where optimality is not guaranteed), this targeted refinement remains a powerful mechanism for improving intermediate-layer bounds.

Next, we consider the upper bound z ( cc ) 2 . Complete clipping yields the optimal solution z ⋆ 2 , which we derive analytically via Algorithm 1. According to Theorem 3.1, the dual form of this problem and its solution is given by

̸

The dual objective is minimized with respect to the Lagrange multiplier, β ∈ R + . One possible solution occurs at β = 0 , but since relaxed clipping already improved this bound, we expect that β ⋆ = 0 .

Before solving for β ⋆ , it is helpful to examine the structure of the dual objective D ( β ) . Because the primal objective maximizes z 2 , minimizing the dual objective is equivalent. The function D ( β ) is convex and piece-wise linear , so we analyze its sub-gradient, ∂ ∂β D ( β ) :

where q is the vector of breakpoints [( W (1) 2 ,j ) / ( W (1) 1 ,j )] 2 j =1 = [5 , 1 / 7] . Thus, the gradient is uniquely defined on the intervals β ∈ ( -∞ , 1 / 7) , (1 / 7 , 5) , and (5 , ∞ ) . Because D ( β ) is convex, its subgradient is negative on the leftmost interval and positive on the rightmost one. The optimal β ⋆ occurs at the break-point where the sub-gradient changes sign from negative to positive, i.e., the break-point whose sub-gradient interval contains zero. The sub-gradients in each region are:

Note that when transitioning from the case where β ∈ ( -∞ , 1 / 7) to β ∈ (1 / 7 , 5) , the term | W (1) 1 , 2 | ϵ 2 switches sign while -| W (1) 1 , 1 | ϵ 1 remains negative. This ordering follows because in our break-point vector, we have that q 2 < q 1 , and this ensures the sub-gradient is correctly calculated in each sub-interval, providing intuition as to why argsort ( q ) is necessary in Algorithm 1. For this example, the sign change occurs at β = 5 , giving the minimum of the dual objective at β ⋆ = 5 and the solution z ( cc ) = z ⋆ 2 = L ⋆ = -3 .

After performing complete clipping at the intermediate layer, we discover that the optimal upper bounds under our split constraint are given as z ⋆ = [0 , -3] ⊤ . There is clearly an inter-neuron dependency between the two ReLU neurons such that forcing the first neuron to be in-active subsequently causes the the second neuron to also become in-active. Consequently, the post-activation neurons can be exactly bounded using the CROWN algorithm, and in particular, σ ( z ) = 0 for all z ∈ [ z ⋆ , z ⋆ ] . Since the last layer contains no bias vector, it is not necessary to perform complete clipping again, and we have finally verified our desired property for this subproblem,

D Experiments

D.1 Experiments Settings

To allow for comparability of results, all tools for input BaB were evaluated on equal-cost hardware with a 32-vcore CPU, one NVIDIA RTX 4090 GPU with 24 GB memory, and 256 GB CPU memory. For ReLU based BaB experiment, we use a cluster with one AMD EPYC 9534 64-core CPU and the GPU is one NVIDIA RTX 5090 GPU with 32 GB memory and 512 GB CPU memory. Our implementation is based on the open-source α,β -CROWN verifier 3 with Clip-and-Verify related

code added. For input bab, three different set-ups of Clip-and-Verify are tested: Relaxed, Relaxed + Reorder, and Complete. Here Relaxed, Reorder and Complete refers to the methodology discussed in 3.2 and 3.3. All experiments use 32 CPU cores and 1 GPU. The MIP cuts are acquired by the cplex [34] solver (version 22.1.0.0). We use the Adam optimizer [38] to solve both α,β , µ , τ . For the SDP-FO benchmarks, we optimize those parameters for 20 iterations with a learning rate of 0.1 for α and 0.02 for β , µ , τ . We decay the learning rates with a factor of 0.98 per iteration. The timeout is 200s per instance. For the VNN-COMP benchmarks, we use the same configuration as α , β -CROWN used in the respective competition and the same timeouts. For NN control systems, task details are in [42, Appendix C]. For the Lyapunov function level set, we verify on V ( x ) ∈ [0 . 2 , 0 . 20001] for CartPole and Quadrotor-2D , and on V ( x ) ∈ [2 . 0 , 2 . 1] for Quadrotor-2D-Large-ROA . Let Ω final denote the resulting (expanded) box used for training/evaluation once the generator stabilizes. Concretely (all ' ± [ · ] ' are per-coordinate half-widths), for CartPole , Ω final = ± [4 . 8 , 3 . 6 , 13 . 2 , 13 . 2] . For Quadrotor-2D , Ω final = ± [12 , 13 . 2 , 12 , 19 . 2 , 20 . 4 , 88 . 8] .

D.2 Ablation Studies

We conduct a detailed ablation study on multiple adversarially-trained models spanning MNIST, CIFAR, and larger VNN-COMP benchmarks to evaluate Clip-and-Verify and its variants against three baselines: β -CROWN, GCP-CROWN, and BICCOS. Tables 4 and 6 highlight three key metrics: verified accuracy (Ver.%), average per-example verification time (Time), and average per-verifiedexample domain visited (D.V.). Domain visited (D.V.) is a metric specific to branch-and-bound (BaB) methods, indicating how many subproblems (domains) are explored to fully verify an instance. Crucially, a higher D.V. count may reflect verifying more difficult instances or a larger overall coverage, rather than inefficiency in the verification process. We also provide Figures 5 and 6 that visualize these metrics across all benchmarks.

Overall Verified Accuracy and Time From Table 4, Clip-and-Verify variants nearly always achieve higher verified accuracy than the baselines. For instance, on CNN-B-Adv (CIFAR), Clipand-Verify with BICCOS reaches 51.5% verified accuracy-surpassing the 47.0% ( β -CROWN), 49.5% (GCP-CROWN with MIP cuts), and 51.0% (BICCOS alone) of the baselines. On cifar10resnet, Clip-and-Verify (with MIP cuts or with BICCOS) achieves up to 88.89% verified accuracy (outperforming the 83.33% - 87.5% range from baselines). In terms of verification time on this benchmark, Clip-and-Verify with β -CROWN is the fastest overall (6.06s), and Clip-and-Verify with BICCOS (11.80s) remains competitive with standalone BICCOS (16.73s) and GCP-CROWN (17.99s). When scaling to deeper networks such as cifar100-2024 and tinyimagenet-2024, Clip-and-Verify with BICCOS maintains the leading verified accuracy (65.5% and 72.0%, respectively) while sustaining moderate average verification times (e.g., 8.17s for cifar100-2024 and 10.48s for tinyimagenet-2024), underscoring its suitability for larger-scale verification tasks.

Domain Visited (D.V.) vs. Difficulty In Table 6, we further examine the average domain visited (D.V.) across verified examples. While Clip-and-Verify variants may sometimes visit more domains (e.g., on oval22, Clip-and-Verify with MIP cuts visits 18891.25 domains compared to 16614.72 for GCP-CROWN with MIP cuts), this often correlates with achieving higher verified accuracy (90.00% vs 83.33% in this case). This suggests that the method is effectively exploring the space to verify more challenging instances or a broader set of inputs, leading to a net increase in verified accuracy. For example, on CNN-A-Adv (CIFAR), Clip-and-Verify with MIP cuts visits 3704.86 domains on average and attains 48.5% verified accuracy. While its D.V. is slightly higher than standalone BICCOS (3622.71 D.V. for 48.5% accuracy), it's notably lower than β -CROWN (12621.38 D.V. for 45.5% accuracy) and GCP-CROWN with MIP cuts (8186.11 D.V. for 48.5% accuracy), while achieving comparable or better accuracy. In other scenarios (e.g., CNN-A-Adv-4 on CIFAR), Clip-and-Verify with BICCOS achieves 48.5% accuracy with a D.V. of only 843.47. This is the same or higher accuracy with a significantly smaller D.V. compared to standalone β -CROWN (46.5%, 2066.39 D.V.), GCP-CROWN with MIP cuts (48.5%, 3907.30 D.V.), and BICCOS (48.5%, 1319.56 D.V.). This illustrates that when Clip-and-Verify effectively prunes the search space, verification efficiency can improve even while tackling similarly challenging problems and achieving high accuracy.

A Breakdown Comparison between Verifiers Table 5 exhibits the instance-wise comparison on acasxu benchmark, as also illustrated in Figure 2a. Most of the instances in acasxu benchmark are easy to verify. On these instances, all the verifiers share similar verification time and D.V . . Instances

73 and 65 stand out as they're significantly harder to verify. For instance 73, clipping is able to cut down over 94% D.V. and over 80% verification time. While α, β -CROWN is unable to verify instance 65 within given timeout, Clip-and-Verify is able to verify it within 10 seconds. Cactus plots Figures 5 and 6 are also able to give a instance-wise comparison. Please see the following paragraph for a more detailed explanation.

Interpretation of the Cactus Plot Figures 5 and 6 illustrate the number of instances verified as runtime varies. Each line represents a method, with the x-axis showing the cumulative verified instances under a given timeout (y-axis). This style, used in VNN-COMP reports captures the tradeoff between subproblem difficulty and runtime performance. The curve's right end indicates total solved instances-further right means more instances verified within the timeout. A flat initial curve fragment reflects many easy instances solved quickly. A curve below and to the right of another shows consistently faster solving. The x-axis ordering reflects instance difficulty, from easy (left) to hard (right). For example, on the tinyimagenet benchmark (Figure 5d), our Complete Clipping + BICCOS variant solves easy instances faster and solves more hard instances. Across these benchmarks, the proposed Clip-and-Verify framework demonstrates a favorable balance: it reduces the number of hard subproblems, leading to more instances being verified within moderate time thresholds. This suggests that although our clipping procedures incur additional overhead that may be noticeable in easy instances, the overall net gain is highlighted in its ability to solve more instances and hard instances with shorter verification times.

Overall, these results show that Clip-and-Verify's framework of enhanced linear bounding and 'clipping' robustly scales across varying network depths and adversarial training schemes. The additional integration of MIP cuts or BICCOS bounding routines consistently pushes verified accuracy closer to each benchmark's upper bound while balancing verification time and domain exploration. We plot and analyze these ablation studies across all benchmarks in Appendix D.2, confirming that our method not only raises coverage (Ver.%) but also leverages clipping and cutting plane methods to verify some of the hardest instances encountered.

D.3 Detailed Comparison with LP Solvers

To validate our claim that coordinate ascent is more efficient than LP solvers for our task, we integrated LP solvers directly into our BaB algorithm 4 to solve the clipping optimization problem. We compared our method ('Clip (ours)') against the Gurobi solver using dual simplex with varying

iteration limits. The experiment was run on the acasxu benchmark instance 65, and we report the average time and bound error over 30 million LP calls during the entire BaB process. We report only Gurobi's optimize() time, ignoring all Python overhead for problem creation, giving an advantage to the LP solver.

As shown in Table 7, our method is 740x faster than a 1-iteration LP heuristic and 880x faster than a 10-iteration LP, while achieving comparable accuracy (0.00085 vs. 0.0007 mean error). The LP solver only achieves near-zero error with 10+ iterations, at which point it is intractably slow. The high fixed cost of LP solvers (even for 1 iteration) comes from presolve and initial basis factorization routines, which are far more expensive than our simple O ( n log n ) sort.

We also investigated modern GPU-based LP solvers, specifically Google's PDLP [3, 4], which uses a Primal-Dual Hybrid Gradient (PDHG) algorithm. As shown in Table 8, while PDLP achieves high accuracy when it converges, it is not suitable for our use case. PDHG does not maintain a feasible solution during iterations. Under strict iteration limits, it has a very high failure rate, returning a NOT_SOLVED status and thus failing to provide a valid dual bound for our sound verification procedure. In contrast, dual simplex (Gurobi) and our method always return a valid, feasible bound. PDLP is designed for single, massive LPs, whereas our framework requires solving millions of independent, small LPs in parallel, a setting where our specialized GPU solver excels.

D.4 Details of Heuristics

Neuron Selection Heuristic. The performance of Complete Clipping can be sensitive to the choice of which intermediate neurons to refine. To guide this selection, we employ a heuristic based on the Branch-and-Bound for Split Recommendation (BaBSR) score, originally designed to select which neuron to branch on during BaB [13]. Specifically, we use the intercept score from BaBSR, which estimates the potential impact of refining a neuron's bounds on the final output relaxation. The score for a neuron k is calculated as:

where l k and u k are the neuron's pre-activation lower and upper bounds, and mean A k relates to the intercept of the neuron's linear lower bound. A higher score indicates a looser relaxation and a greater potential for improvement. For each layer, we compute this score for all unstable neurons and select the topk neurons for refinement with Complete Clipping.

To validate this choice, we conducted an ablation study on the tinyimagenet-2024 and cifar100-2024 benchmarks, comparing BaBSR against three alternative heuristics: random selection, prioritizing neurons with the largest bound gap ( U -L ), and prioritizing neurons with the largest bound product ( -U × L ). As shown in Tables 9 and 10, the BaBSR heuristic consistently achieves the best verification time and explores the fewest subdomains, especially for the balanced top-20 setting. This confirms that BaBSR makes more effective choices, leading to earlier pruning and a more efficient search.

Table 10: Average number of visited domains per instance for different top-k heuristics.

Table 11 shows the ablation study on the top-k neuron selection heuristic on cifar_cnn_a_mix benchmark, reveals that strategically prioritizing neurons based on their FSB intercept scores significantly enhances verification efficiency. Employing a moderate k (specifically top-k 20 and 50 ) leads to the best outcomes, successfully verifying more properties (86) while substantially reducing both the number of domains visited (by up to 28% compared to no heuristic) and the overall time (by up to 19%). In contrast, not using the heuristic (top-k 0 ) results in a less efficient search, while applying it to all unstable neurons (top-k 'all") drastically cuts down visited domains but incurs a prohibitive time cost and a timeout, indicating that the overhead of processing too many prioritized neurons outweighs the benefits of more targeted branching. This demonstrates a crucial trade-off, with intermediate k values striking an optimal balance between guided search and computational overhead.

Constraint Importance Heuristic The constraint importance heuristic is to sort the constraints for better performance. The constraints corresponding to hyperplanes closer to the center of the current input domain might be more immediately relevant or impactful for tightening the bounds or for cutting off a significant portion of the current feasible region X .

First, for the current input box domain X , determine its centroid ˆ x . We can calculate g ⊤ ˆ x + h -∑ n i =1 | g i | ϵ i > 0 (there is no x ∈ X satisfy the constraint in (2)) to check the infeasibility and

g ⊤ ˆ x + h + ∑ n i =1 | g i | ϵ i ≤ 0 (for all x ∈ X satisfy the constraint in (2)) to remove redundancy. Second, for each available linear constraint g ⊤ x + h = 0 , calculate the geometric distance from the centroid ˆ x to this hyperplane:

Then, sort the constraints in ascending order based on these calculated distances to prioritize constraints that are closer to the centroid. In methods like Algorithm 1 (Complete Clipping), processing more impactful constraints earlier might lead to faster convergence or more significant bound improvements in the coordinate ascent. Such an important metric could also be relevant to Algorithm 3 (Relaxed Clipping).

We tested the bound tightness improvement from the heuristic on acasxu instance 65. Clip-andVerify improves the intermediate bounds for 470,772,542 times during input BaB, and 3.962% of them can be further tightened with constraint importance heuristic. For all these further improved bounds, we computed the empirical quantiles of the relative improvements, shown in Table D.4.

These results reveal that the head of the distribution contains substantial refinements, with the maximum observed improvement reaching up to 4000%. This indicates that the heuristic can produce tightened bounds, potentially leading to earlier branch pruning or faster convergence.

E Limitation and Broader Impacts

Limitation. Our framework's effectiveness is influenced by the trade-off between the precision of clipping and its computational overhead. While Complete Clipping utilizes GPU acceleration, its scalability can be impacted when networks have very large hidden layers combined with a high number of unstable neurons and sparse constraints. The memory complexity scales as O ( B × N × M ) , where B is the number of BaB subproblems, N is the number of neurons in the largest layer, and M is the number of linear constraints. To mitigate this, our top-k neuron selection heuristic (Appendix D.3) strategically prioritizes the most critical neurons, balancing precision and cost.

Conversely, while Relaxed Clipping is highly efficient with a complexity of O ( n ) per input dimension, its effectiveness can diminish in very high-dimensional input spaces due to the curse of dimensionality, where axis-aligned relaxations may become looser.

However, the ultimate scalability of our method is not fundamentally limited by input dimensionality or layer width, but rather by the intrinsic difficulty of the verification query, which is an NPhard problem. As demonstrated by our experiments on neural network control systems 2 even small networks with low-dimensional inputs can pose immense verification challenges. Clip-andVerify proved essential in solving these hard instances. The key insight is that our approach adapts computational effort based on problem characteristics rather than being constrained by absolute dimensional limits, ensuring practical applicability across diverse verification scenarios.

(a) cifar10-resnet

(b) oval22

(c) cifar100

(d) tinyimagenet

300

700

β-CROWN

β-CROWN

250

GCP-CROWN+mip cuts

600

GCP-CROWN+mip cuts

BICCOS

BICCOS

200

Clip-and-Verify

500

Clip-and-Verify

e

150

Clip-and-Verify+mip cuts

Clip-and-Verify+BICCOS

Time

e

400

Clip-and-Verify+BICCOS

Clip-and-Verify+mip cuts

Time Limit (300)

300

Time Limit (720)

100

200

50

100

0

0

40

45

50

55

60

0

5

10

15

20

25

NumberofInstancesVerified

NumberofInstancesVerified

100

B-CROWN

100

B-CROWN

BICCOS

BICCOS

80

Clip-and-Verify

80

Clip-and-Verify

Clip-and-Verify+BICCOS

Clip-and-Verify+BICCOS

Time

60

Time Limit (100)

Time

60

Time Limit (100)

40

40

20

20

0

100

105

110

115

120

125

130

0

110

115

120

125

130

135

140

NumberofInstancesVerified

NumberofInstancesVerified

Broader Impacts Neural network verification is crucial for ensuring the safety and reliability of AI systems in critical applications such as autonomous vehicles, medical diagnosis, and financial trading. By significantly accelerating the verification process through efficient domain reduction, our work makes formal verification more practical for larger and more complex neural networks. This advancement enables broader adoption of verification techniques in real-world applications, potentially preventing catastrophic failures and building trust in AI systems. Clip-and-Verify's integration with existing frameworks ensures immediate applicability, allowing organizations to implement stronger safety guarantees without substantial overhead. While this work strengthens the safety of AI systems, it is important to note that verification tools should be part of an approach to AI safety, including robust testing interpretability, and ethical guidelines.

(a) cifar_cnn_a_adv

(b) cifar_cnn_a_adv4

(c) cifar_cnn_a_mix

(d) cifar_cnn_a_mix4

(e) cifar_cnn_b_adv

(f) cifar_cnn_b_adv4

(g) mnist_cnn_a_adv

200

200

β-CROWN

β-CROWN

GCP-CROWN+mipcuts

GCP-CROWN+mip cuts

150

BICCOS

150

BICCOS

Clip-and-Verify

Clip-and-Verify

Clip-and-Verify+mip cuts

e

Clip-and-Verify+mipcuts

100

Clip-and-Verify+BICCOS

100

Clip-and-Verify+BICCOS

Time Limit (200)

Time Limit (200)

50

50

0

70

75

80

85

90

95

0

80

82

84

86

88

90

92

94

96

NumberofInstancesVerified

NumberofInstancesVerified

200

β-CROWN

200

β-CROWN

GCP-CROWN+mip cuts

GCP-CROWN+mip cuts

150

BICCOS

150

BICCOS

Clip-and-Verify

Clip-and-Verify

Clip-and-Verify+mip cuts

ime

Clip-and-Verify+ mip cuts

100

Clip-and-Verify+BICCOS

100

Clip-and-Verify+BICCOS

Time Limit (200)

Time Limit (200)

50

50

0

60

65

70

75

80

85

90

95

0

95.0

97.5

100.0102.5105.0107.51

110.0

112.5

NumberofInstancesVerified

NumberofInstancesVerified

200

200F

β-CROWN

β-CROWN

GCP-CROWN + mip cuts

GCP-CROWN+mip cuts

150-

BICCOS

150

BICCOS

Clip-and-Verify

Clip-and-Verify

Time

Clip-and-Verify + mip cuts

Time

Clip-and-Verify + mip cuts

100

Clip-and-Verify + BICCOS

100

Clip-and-Verify + BICCOS

Time Limit (200)

Time Limit (200)

50

50

0

0

60

70

80

90

100

80

85

90

95

100

105

110

115

120

Number of Instances Verified

Number of Instances Verified

200

β-CROWN

GCP-CROWN + mip cuts

150

BICCOS

Clip-and-Verify

Time

Clip-and-Verify + mip cuts

100

Clip-and-Verify + BICCOS

Time Limit (200)

50

0

110

120

130

140

150

Number of Instances Verified

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our claims match our theoretical and empirical results. We present results across several benchmarks to demonstrate that they can generalize to other settings.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations have been explicitly discussed in Appendix E.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Theoretical results have been proven.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Algorithms have been clearly described with technical details discussed. Parameters for experiments have been included in the appendix. The code will be opensourced.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code will be open-sourced.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Experimental details have been discussed in the Experiments section and also in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: Verification results are deterministic on the benchmarks, and no error bars need to be provided.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Time has been reported in tables and we only need one machine and one GPU to run all experiments. The specific hardware details are reported in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read and acknowledged the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Societal impacts have been discussed explicitly in Appendix E.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: No data or models are released.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We've cited the sources of the benchmarks we used, such as those from the VNN-COMP

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets are introduced in this paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The paper does not use LLMs for important, original, or non-standard component of the core methods in this research.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nNeural network verification is critical for mission-critical applications, where branch-and-bound (BaB) with bound propagation is a state-of-the-art approach. However, loose intermediate layer bounds in BaB lead to exponential subproblem growth and inefficient verification. This paper introduces Clip-and-Verify, a linear constraint-driven domain clipping framework to enhance bound propagation verifiers.  \n\nThe framework comprises two novel algorithms:  \n1. **Complete Clipping** directly optimizes intermediate layer bounds using Lagrangian duality, transforming high-dimensional linear programs into efficient 1D concave maximization problems. It handles multiple constraints via coordinate ascent, tightening bounds and identifying infeasible regions.  \n2. **Relaxed Clipping** refines the input domain using linear constraints (e.g., from ReLU splits or final-layer bounds), updating input bounds in closed-form to avoid complex polyhedral operations. The clipped domain tightens intermediate bounds globally.  \n\nBoth algorithms integrate seamlessly into BaB-based verifiers (e.g., α, β-CROWN), leveraging existing linear constraints without extra computation. They reduce redundant subproblems by pruning infeasible input regions and improving bound tightness.  \n\n### Key Contributions  \n1. **Innovative Framework**: A scalable linear constraint-driven clipping framework that enhances intermediate layer bounds without expensive solvers, via Complete and Relaxed Clipping.  \n2. **Efficient Integration**: Algorithms adapt to input and activation BaB, reusing constraints from BaB splits to tighten domains and bounds with minimal overhead.  \n3. **State-of-the-Art Performance**: Reduces BaB subproblems by up to 96% on VNN-COMP benchmarks, achieving superior verified accuracy across diverse networks and verification scenarios.\n\nSTRENGTHS AND WEAKNESSES:\n*Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification* presents a novel approach to enhance neural network verification. Here is an assessment of its strengths and weaknesses across the dimensions of Quality, Clarity, Significance, and Originality:\n### Strengths\n1. **Quality**\n    - **Methodological Rigor**: The paper demonstrates high methodological quality. It formulates the problem of tightening intermediate layer bounds in neural network verification as a problem within the linear constraint - driven clipping framework. The two proposed algorithms, Complete Clipping and Relaxed Clipping, are well - defined with detailed mathematical derivations. \n    - **Experimental Validation**: The experiments are comprehensive and well - designed. The authors evaluate Clip - and - Verify on a wide range of benchmarks from VNN - COMP 2021 - 2024, including input BaB and ReLU split BaB benchmarks.\n2. **Clarity**\n    - **Structured Presentation**: The overall structure of the paper is logical and well - organized. It follows a standard academic paper format, with sections on Preliminaries, the proposed method, experiments, related work, and conclusion. Each section builds on the previous one, guiding the reader through the research process in a clear and coherent manner.\n3. **Significance**\n    - **Advancing Neural Network Verification**: The research has significant implications for the field of neural network verification. By improving the efficiency of bound propagation verifiers, it can potentially enable the verification of larger and more complex neural networks. This is crucial for mission - critical applications where the safety and robustness of neural networks need to be formally proven.\n    - **Practical Impact**: The reduction in the number of subproblems during the BaB process and the improvement in verification speed can lead to more practical and scalable verification tools. This can have a direct impact on industries that rely on neural network verification, such as autonomous systems, healthcare, and finance.\n4. **Originality**\n    - **Novel Framework and Algorithms**: The linear constraint - driven clipping framework and the two associated algorithms are novel contributions. While previous work has focused on bound propagation and BaB in neural network verification, this paper introduces a new way of leveraging linear constraints to optimize intermediate layer bounds and input domains.\n\n### Weaknesses\n1. **Quality**\n    - **Scalability in Extreme Cases**: Although the paper claims to preserve the scalability of state - of - the - art neural network verifiers, it is not entirely clear how the proposed algorithms will perform in extremely large - scale neural networks with a vast number of neurons and complex architectures.\n    - **Limited Generalization to Non - Standard Networks**: The experiments are mainly conducted on benchmarks from VNN - COMP, which may not cover all types of neural networks. The performance of Clip - and - Verify on non - standard or emerging network architectures, such as those with novel activation functions or complex topological structures, remains untested.\n2. **Significance**\n    - **Dependency on Benchmark Datasets**: The significance of the results may be somewhat limited by the reliance on specific benchmark datasets. The improvements observed on VNN - COMP benchmarks may not directly translate to real - world scenarios or other types of verification tasks. There is a need to further explore the generalizability of the method in more diverse and realistic settings.\n4. **Originality**\n    - **Conceptual Similarities to Related Work**: Although the proposed framework is novel, there are some conceptual similarities to related work.\n\nQUESTIONS:\n#### **Question1：** Scalability in Extremely Large Neural Networks.  \nThe paper demonstrates efficiency on VNN-COMP benchmarks, but how does Clip-and-Verify perform on extremely large networks (e.g., with millions of neurons or complex architectures like transformers)? \n\n#### **Question2：** Generalization to Non-Standard Network Architectures. \nThe experiments focus on VNN-COMP benchmarks with ReLU networks. How does the framework handle non-standard activations (e.g., Swish, Mish) or specialized layers (e.g., attention mechanisms)?  \n\n#### **Question3：** Real-World Application Scenarios.  \nWhile VNN-COMP benchmarks are standard, how does Clip-and-Verify perform in practical safety-critical applications (e.g., autonomous driving perception networks or medical diagnosis systems)?\n\nLIMITATIONS:\nNo, the authors have not adequately addressed potential negative societal impacts in the provided document. \n\n**Constructive suggestions for improvement:**  \n1. **Address computational resource implications**: Discuss whether the framework’s efficiency gains could inadvertently exacerbate computational resource disparities in academia/industry, potentially limiting access to verification tools.  \n2. **Safety-critical application risks**: Acknowledge that even improved verification might not guarantee 100% safety in mission-critical systems (e.g., autonomous vehicles), and outline strategies to mitigate false negatives/positives.  \n3. **Ethical considerations in verification**: Explore how the framework might influence neural network design (e.g., prioritizing verifiability over performance) and whether this could introduce biases or restrict innovation.  \n\nIncluding such discussions would demonstrate a comprehensive understanding of the research’s limitations and societal role, enhancing the paper’s rigor and responsibility.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe focus of this work is to improve bound tightening within the context of branch-and-bound-based neural network verification, especially bounds for intermediate layers. These bounds are derived from pre-existing linear constraints that are available during typical verification methods, either when using input splitting or ReLU splitting. The paper proposes two methods, \"complete clipping\", which performs coordinate ascent over the dual problem to find tighter bounds, and \"relaxed clipping\", which is a simpler and faster method that considers each linear constraint independently. Experiments are performed on benchmarks from past neural network verification competitions, showing that these bounds help reduce the number of subproblems and improve the number of verified cases.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper provides an effective way to incrementally improve the performance of branch-and-bound verifiers, whether they branch on the input or on the ReLUs. The end-to-end results are significant and the computational setup, including baselines and benchmarks, seems sound. The general idea of tightening bounds based on pre-existing linear constraints from the verifier is interesting and relatively simple, especially if you can produce very fast ways to do so. The overall writing and quality of the paper look solid to me.\n\nHowever, the proposed optimization methodology has a couple of major issues: 1, it is not as novel as it may initially seem due to some connections that the paper seems to miss, and 2, the paper claims that the coordinate ascent algorithm is better than an LP solver without a proper comparison, particularly since the context is to only produce a dual bound rather than solving the problem to optimality. I will go into details in the \"Questions\" section below.\n\nIn my opinion, the optimization portion of the paper needs significant reworking, and thus at the current state, I cannot recommend acceptance. That said, methodology aside, I believe that this work offers a valuable key message: that efficient-but-weak bound tightening techniques for subproblem bounds are helpful in practice. Therefore, I believe that this can be a solid paper if its issues can be resolved, and thus my current recommendation is borderline.\n\nQUESTIONS:\n**Optimization methodology**\n\n1. **Sec. 3.2: Complete Clipping, Exact Bound Refinement**\n\n    For the case where $\\\\mathcal{X}$ is a box domain as in Theorem 3.1, then this is exactly an LP with bounded variables and a single inequality constraint. This can be solved with a greedy algorithm. I originally thought that this would be a textbook result, but this might not be the case: I tried to find a clean reference but unfortunately I could not find it. The closest I got was Kellerer, Pferschy, Pisinger, \"Knapsack Problems\", but you would need to look at both Sec. 1.4 and Sec. 2.2. Specifically, this is a continuous knapsack problem, with the difference that the variable bounds are arbitrary instead of $[0, 1]$ and the objective coefficients and weights can be negative. However, these two differences can be easily resolved via transformation: if $l \\\\leq x \\\\leq u$, then just shift and scale $x$ to $x' = (x - l) / u$ so that $0 \\\\leq x' \\\\leq 1$. Furthermore, following Sec. 1.4 of the above book, we can set $x'$ to 0 or 1 if the objective and weight have different signs, or if they are both negative, we can simply flip the sign of $x'$. Then this is exactly the continuous knapsack problem, and it is well known that it can be solved via a greedy algorithm (Sec. 2.2 of the book above).\n\n    The implication here is that the machinery from Theorem 3.1, while it is a nice alternative way to solve the problem, feels unnecessary. Reading Appendix B.2, your algorithm is possibly equivalent (or almost equivalent) to the greedy algorithm above. I believe your $\\\\beta$s are equivalent to the ratios from the greedy algorithm and you are searching for the breakpoint where you can no longer add to the knapsack.\n\n    **What's actionable:** If you agree with the above discussion, then at the minimum, you would need to incorporate the above context, but it does substantially diminish this part of the paper's contribution and perhaps should be rewritten. It is ok if you want to keep the method along with the above context (especially since it could be a way to generalize to other $\\\\mathcal{X}$), but I would not consider it to be a contribution significantly beyond existing literature. That said, I do not think that diminishing this contribution is too bad for the paper, given that the paper has other interesting contributions.\n\n2. **Sec. 3.2: Complete Clipping, Coordinate Ascent for Multiple Constraints**\n\n   In this section, your problem is exactly an LP, so you seem to be proposing an algorithm for general LPs. However, the difference here is that you are actually proposing a dual heuristic for LPs via coordinate ascent, since your algorithm does not solve the LP to optimality. It is plausible that a dual heuristic would work well here, since you want a fast way to obtain dual bounds. That said, the claim that \"coordinate ascent is significantly efficient compared to LP solvers\" is unfounded since you are assuming the LP solver would be used as an exact solver and not as a heuristic. If you are proposing a new dual heuristic for LPs and claiming that it works better than LPs, then you need to compare it with a standard dual heuristic for LPs. In this case, the most straightforward baseline is simply running dual simplex with a time or iteration limit. By stopping dual simplex early, you still have a dual feasible solution and thus a dual bound for your problem. It is plausible that your algorithm does better specifically for verification than limited dual simplex for some reason, but evidence needs to be presented.\n\n   **What's actionable:** I do not believe you can make the claim that your coordinate ascent heuristic works better than a typical LP solver with a time/iteration limit without a proper experimental comparison. Please make that comparison. If your algorithm outperforms the LP solver, then that is great, you have a very nice contribution here. If it does not, then I do not see the value of the proposed algorithm.\n\n3. **Sec. 3.3: Relaxed Clipping**\n\n   Since Theorem 3.2 is a specific case of Theorem 3.1, the same discussion above applies, and it is possible to obtain a simpler proof without resorting to duality. The argument here is even simpler: since only $x_i$ appears in the objective, you can set all other variables to their lower or upper bounds as to loosen the constraint as much as possible, then set $x_i$.\n\n   The clipping to multiple constraints subsection makes sense: this is a very fast way to tighten your bounds. I would have expected these bounds to be rather weak, but it seems that they are still useful in your computational experiments, so that is good.\n\n   **What's actionable:** Similarly to my first comment, you should discuss the simpler approach here. This one is less important, however, and I do not think that adding contextualization diminishes the contribution much in this case.\n\n**Minor comments**\n\n4. For input splits, critical neurons are selected via a top-k heuristic. Could you please include in the paper what metric exactly is the top-k heuristic looking at? BaBSR is mentioned for ReLU splits, but for input splits this is not explicitly mentioned.\n\n5. Do I understand correctly in that for ReLU splits, you only do complete clipping and not relaxed clipping? This is what I understand from Sec. 3.5, but I am not sure about this. Please make sure that is clearer in the paper.\n\nLIMITATIONS:\nAdequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Clip-and-Verify, a framework that enhances branch-and-bound neural network verifiers by using linear constraints to tighten intermediate bounds and reduce subproblems. It introduces two clipping algorithms, i.e., complete clipping (direct optimization of intermediate bounds) and relaxed clipping (input domain refinement), and integrates them with existing verifiers. The method claims a reduction of up to 96% in subproblems (although a 96% reduction does not necessarily correspond to improved performance) and improved verification performance on several verification benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- The framework repurposes linear constraints from BaB to refine intermediate layers, addressing a gap in prior work that underutilizes such constraints.\n- Experiments on VNN-COMP benchmarks show consistent reductions in subproblems, outperforming existing verifiers in most benchmarks (w.r.t. the verified tasks).\n\nWeaknesses:\n\n- The paper uses a \"top-k objective selection heuristic based on BABSR intercept score\" to choose critical neurons for clipping, but provides little justification. It is unclear how this heuristic ensures consistent performance across networks or input regions. In my view, the selection of objective neurons plays a crucial role in influencing the overall solving efficiency. The paper lacks ablation studies or comparisons with alternative heuristics.\n\n- While the proposed method reduces the number of subproblems, this reduction does not seem to translate into proportional runtime savings, and in some cases, even results in longer total solving time (for example, nn4sys in Table 1), suggesting non-trivial overhead from the clipping framework proposed in the paper. This discrepancy likely stems from the computational costs inherent to the clipping procedures themselves, and the paper does not systematically evaluate this trade-off. This gap hinders a clear understanding of when the clipping procedures provide benefits versus when their overhead outweighs subproblem reductions. \n\n- Figure 1 is not intuitive and difficult to understand.\n\nQUESTIONS:\n1. Could the authors provide a (clear) justification for using the BABSR intercept score as the basis for selecting neurons? Additionally, have the authors conducted any ablation studies to evaluate alternative heuristics? Understanding the impact of this choice is important, as neuron selection can significantly influence the efficiency of the solver.\n\n2. Could the authors provide a breakdown of runtime with respect to the difficulty level of subproblems (e.g., trivial vs. hard)? This would help clarify how the trade-off between fewer subproblems and their individual complexity affects overall performance.\n\n3. For verified instances that are also solvable by existing verifiers, how does the solving time under the proposed method compare? Tables 2 and 3 report overall verified results, but it would be helpful to separate the solving time of instances verified by both methods versus those uniquely verified by the proposed framework.\n\n4. For the results presented in Tables 2 and 3, as well as Figure 2, which variant of Clip-and-Verify is used? Relaxed + Reorder?\n\nI may revise my score based on the authors' responses.\n\nLIMITATIONS:\nHeuristic Sensitivity: The reliance on a top-k heuristic for selecting critical neurons introduces variability; performance may degrade if the heuristic fails to identify impactful neurons.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a lightweight domain clipping framework to improve neural network verification in branch-and-bound (BaB). \nIt introduces two key methods, i.e., Relaxed Clipping to shrink input domains and Complete Clipping to tighten intermediate bounds, \nboth using linear constraints efficiently. \nBy integrating into BaB, this approach reduces subproblems and improves verification accuracy without expensive solvers.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The method is lightweight and efficient, relying only on simple arithmetic operations without LP or external solvers.\n2. It improves both input and intermediate bounds by combining Relaxed Clipping for domain refinement and Complete Clipping for internal bound tightening.\n3. The approach integrates easily into existing BaB-based verifiers such as \\alpha-  \\beta-CROWN with minimal modification.\n4. By clipping irrelevant input regions, it significantly reduces the number of subproblems and BaB splits required.\n5. Subdomains in BaB can inherit linear constraints from their parent, enabling efficient incremental clipping and avoiding redundant computation.\n6. The experiments are comprehensive, and the ablation study is also well-designed.\n\nWeaknesses:\n1. The method is only applicable to ReLU activation functions.\n2. The paper provides few running examples, which limits intuitive understanding.\n3. Complete Clipping only targets the top-k key neurons selected based on heuristics; if the heuristic misidentifies key neurons, it may affect the final bound tightness.\n\nQUESTIONS:\n1. How does Complete Clipping determine the top-k key neurons using the BABSR intercept score, and what criteria are used in this heuristic?\n2. In line 787-788, what are the main limitations of Complete Clipping in handling networks with large hidden layers and sparse constraints?\n3. Why is the proposed method restricted to ReLU activation functions, and what challenges arise if other activations are used?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper puts forward a method for tightening intermediate bounds in symbolic bound propagation frameworks for neural network verification. The method uses constraints already derived from the bound propagation passes (final-layer constraints and ReLU split constraints) to optimise the bounds of hidden neurons expressed as linear functions over the input domain. The overhead of the method is discussed and more efficient alternatives are given (relaxed clipping) when it becomes significant. The empirical evaluation shows performance improvements over the state-of-the-art for a wide range of benchmarks. Still the scalability of the method with respect to input dimensionality and layer sizes is not shown.\n\nSTRENGTHS AND WEAKNESSES:\nI think that the main a advantage of the proposed method is that the further optimisation of the intermediate bounds is done efficiently, with overall verification performance improvements that are demonstrated across a wider range of benchmarks. The optimisation can of course have a significant overhead with an increasing number of unstable nodes, a shortcoming however that the paper addresses with relaxed clipping.\n\nWhile the performance improvements appear marginal, I think that it is only natural for performance jumps to gradually weaken with the evolution of a research area. Although the work is of course incremental to an existing verification framework, said framework exhibits leading performance and its improvement is therefore significant. Still, this means that the proposed method is only applicable to verification queries that the underlying framework is already applicable (thereby accelerating verification as the title accurately describes), rather than scaling verification to bigger models/harder verification queries.\n\nThe paper is superbly written!\n\nQUESTIONS:\nCan you please specify the input dimensionality and layer sizes for which the efficacy of the proposed method may weakened?\n\nLIMITATIONS:\nThe limitations and potential negative societal impact of the work are adequately discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}