Generating and Checking DNN Verification Proofs
Authors: Hai Duong
ThanhVu Nguyen
Matthew B. Dwyer
Venue: NeurIPS 2025 poster ()
Keywords: DNN Verification
Proof Generation
Proof Format
Primary area: other
TL;DR: A new proof format (APTP) and an efficient proof checker (APTPchecker) make neural network verification more reliable and robust across different tools.

Abstract: Deep Neural Networks (DNN) have emerged as an effective approach to implementing challenging subproblems. They are increasingly being used as components in critical transportation, medical, and military systems. However, like human-written software, DNNs may have flaws that can lead to unsafe system performance. To confidently deploy DNNs in such systems, strong evidence is needed that they do not contain such flaws. This has led researchers to explore the adaptation and customization of software verification approaches to the problem of neural network verification (NNV).
Many dozens of NNV tools have been developed in recent years and as a field these techniques have matured to the point where realistic networks can be analyzed to detect flaws and to prove conformance with specifications. NNV tools are highly-engineered and complex may harbor flaws that cause them to produce unsound results.

We identify commonalities in algorithmic approaches taken by NNV tools to define a verifier independent proof format---activation pattern tree proofs (APTP)---and design an algorithm for checking those proofs that is proven correct and optimized to enable scalable checking. We demonstrate that existing verifiers can efficiently generate APTP proofs, and that an APTPchecker significantly outperforms prior work on a benchmark of 16 neural networks and 400 NNV problems, and that it is robust to variation in APTP proof structure arising from different NNV tools.
APTPchecker is available at: https://github.com/dynaroars/APTPchecker.

Generating and Checking DNN Verification Proofs
Hai Duong
Thanh Vu Nguyen George Mason University Fairfax, VA 22030
Matthew B. Dwyer University Virginia Charlottesville, VA 22904
George Mason University Fairfax, VA 22030
hduong22@gmu.cm edu
tvn@gmu.. edu
mathherbbdyerrriiiiinm....
Abstract
Deep Neural Networks (DNN) have emerged as effective approach to implement- ing challenging subproblems. They are increasingly being used as components critical transportation, medical, and military systems. However, like numan-writted software, DNNs may have flaws that can lead to unsafe system performance. To confidently deploy DNNs in such systems, strong evidence is needed that they do not contain such flaws. This has led researchers to explore the adaptation and customization of software verification approaches to the problem of neural network verification (NNV). Many dozens of NNV tools have been developed in recent years and field these techniques have matured to the point where realistic networks can be analyzed to detect flaws and to prove conformance with specifications. NNV tools are highly-engineered and complex may harbor flaws that cause them to produce unsound results.
We identify commonalities in algorithmic approaches taken by NNV tools define a verifier independent proof format-activation pattern tree proofs (APTP)-and design an algorithm for checking those proofs that is proven correct and optimized to enable scalable checking. We demonstrate that existing verifiers can efficiently generate APTP proofs, and that an APTPchecker significantly outperforms prior work on benchmark of 16 neural networks and 400 NNV problems, and that it is robust variation APTP proof structure arising from different NNV tools. APTP checker is available at: httggghhww..oomd/mdaarrr//PPPPPe
1 Introduction
As deep neural networks (DNNs) become integral components of critical systems such as autonomous vehicles [1, medical decision-making [2], and robotics [3], it is imperative to rigorously verify their behavior. In recent years, the research community has developed wide-range of algorithmic techniques to verify DNN properties and incorporated them into tools that now scale to realistic DNN models millions of neurons [4) These advances have enabled verification of properties such as robustness to input perturbations and conformance to safety specifications 4 (
However, despite the progress in algorithmic advances, a fundamental question remains: "How can we trust the results produced by DNN verification tools?" While existing tools emit counterexamples when properties are violated (i.e., SAT results), there is mechanism to independently validate results when properties are proven to hold (i.e., UNSAT claims). Recent competitions such as VNN-COMP [4) have revealed correctness issues in multiple tools, including cases where verifier incorrectly declared a property to be proven even when counterexample exists. These errors are difficult to detect and debug due to the complexity of verifier implementations, which often exceed tens of thousands of lines of code and employ intricate optimization techniques, c.g., top of the line DNN verification tools such a-3 77] and NeuralSAT [9] have 20k SLOC implementations with complex algorithms that may harbor bugs. Without a mechanism to independently validate
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

verification results, correctness of DNN verification tools cannot be assured and therefore posing a serious obstacle to deploying DNNs in cafety-critical domains.
To address this, we propose proofuproducing DNN verification: an approach in which verifiers emit a formal proof object that encodes the reasoning steps behind the verification result, and separate, minimal proof checker certifies the proof's s validity. This paradigm, long established in classical logic and SAT solving
I0 Ii [2], brings transparency, auditability, and trust to the verification process.
More specifically, we analyze the broad class of Branch-and- Bouuddd (BaB) DNN verification algo- rithms and reveal that they share two commonalities: (1) they refine the abstractions they use by performing case splitting to reason about the different phases of neuron activation, and (2) within cases they perform reasoning steps that can be formulated within the broad class of mixed integer linear programming (MILP) problems. Based on these insights, we show that BaB DNN verifi- cation naturally emit activation pattern tree proofs (APTP), which are compact representation of the reasoning steps performed by the verifier ($3.1). We also define a verifier independent APTP format that can be efficiently generated on-the-fly during DNN verification (33.2). Finally, we resent the APTPchecker algorithm along with a suite of optimizations and implement an independent APTPchecker prototype tool that has a small-footprint (800 SLOC) and validates APTP proof' using standard MILP solving.
This paper makes the following contributions:
We identify commonalities in BaB DNN verification algorithms and show how they can be minimally extended to generate proofs of unsatisfiability (S..1).
We define a verifier-indeeeeddnnt, compact, and SMTLib [13] -1aee human-readable proof format, APTP, that captures the reasoning steps of BaB verifiers (83.2).
We implement the APTPchecker tool to independently and efficiently check APTP proofs $4).
We evaluate our work on a benchmark of 400 verification problems involving 16 net- works, including large models (up 1.7M parameters) (55). and demonstrate that APTP and APTPchecker are robust to variation in proof structure arising from different DNN verification algorithms.
is important to note that our goal is not to create a new DNN verifier, but to verify the correctness of results produced by existing verifiers. Similar to SAT/SMT proof checking, where verifiers produce proofs that are independently checked (e.g.. ARAT proofs [00), we propose such an approach for DNN verification. We envision APTP and checerr step toward DNN verifier accountability, standardizes proof formats, supports independent checking, and can be integrated into future VNN- COMP iterations strengthen trust in verification results.
2 Background
Deep Neural Network A neural network consists of an input layer, multiple hidden layers, and an output layer. The output of DNN is obtained by progressively computing the values of neurons in each layer. Specifically, the value of a hidden neuron y is ReLU w + b), where is the bias, ws are the weights of y, us are the neurons of preceding layer, Σ, uepy; + is the affine transformation, and Re: (ar) = max(ar, 0) is the activation function. The Rectified Linear Unit (ReLU) is a representative of broad class of piece-wise linear activations that could be supported by our approach. A ReLU neuron is active if its input value is greater than zero and inactive otherwise.
DNN Verification Given a DNN N and a property 6, the DNN verification problem asks if Φ is a valid property of W. In modern DNN verification, 6(x, v) := (z) + (y) where én is property over the inputs and pout is a property over the outputs of N. This form of properties has been used to encode safety and security requirements of DNNs [5
DNN verification then can be formulated checking the satisfiability of:
(I) where ar is the encoding of N. A DNN verifier attempts to find counterexumple input to A that satisfies dhn but violates Dout. If..q. 1 is unsatisfiable (e.g., no such counterexample exists), & is a valid property of N and invalid otherwise.
A Din -oout
2

---

Alg, 1. The BaBay algorithm with proof generation.
input :DNN N, property Pin = Pout
output (unsal, proof) if property is valid, otherwise (sat, cer)
ActPatterss +- [0] // initialize verification problems
proof + /3// initialize proof tree
while Act do // main loop
Select[ // process problem i-th
Deduce Pen: Dout, 0;) then
(cex, 0s) + Decide(N Φin, bout, ot)
if cen then return (sat, cex) // found valid counter-exampe
PctPtresss + ActParenss U far u A v.) // new activation patterns
else // detect a conflict
10
proof + proof U { // build proof tree
11 return (unsat,proc
For the widely-used ReLU activation problem, this problem becomes search for activation patterns, i... boolean assignments representing activation status of neurons, that lead to satisfaction the formula [8] all adopt this idea and search for
in Eq. 1 Modern DNN verification techniques 7 16
satisfying assignments.
Related Work (more details in Apdxx D) Proof checking is well-establlsserr area constraint solving, particularly in SAT/SMT solving, with significant work on clausal proof generation and verification, such as DRAT for SAT solvers and various proof checkers like AR and I.RAT [I0
12). SMT solvers, such 73 and veriT, also produce proofs that can be reconstructed in proof assistants, and other solvers like Aa5h SMTInterpol, and CVC5 have similar capabilities 200
21 22 23
However, DNN verification is newer field, with limited research on proof checkers.
The only existing proof checking work for DNNs focuses on the Marabou DNN verification tool 25 using Farkas's 's lemma and implemented in the Imandra framework [26). In contrast, we introduces more expressive proof format, APTP, and stronger proof checker, APTPchecker, specifically designed for branch-am aal---nd--buull verification $3.1
3 Proof Generation for DNN Verification
Major DNN verification approaches including 0,3-CROWN 77). NeuralSAT 16], PYRAT 6
nnenum [17]. and Marabou 27 all share common "branch and bound" (BaB) search structure: (i) (branch) split into smaller subproblems by using neuron splitting. which decides boolean values representing neuron activation status, and (ii) (bound) use abstraction and LP solving to approximate bounds on neuron values to determine the satisfiability of the partial activation pattern formed by the split. We leverage this commonality to bring proof generation capabilities with minimal overhead to existing DNN verification tools.
In this paper we focus on checking proofs of unsatisfiability (unsat). A counterexample, c, returned by verifier is an input that purported to violate the property. This constitutes proof of satisfiability (sat) and can easily be checked by evaluating á(c, N (c)). In contrast, unsat proof, which explains why no possible inputs can violate the property, inherently more complex to generate (33). requires a more sophisticated encoding ($3.2). and an efficient checking algorithm ($4)
3.1 Branch-and-Bound DNN Verification
Alg. illustrates BaBay, reference architecture (28 for modern DNN verifiers based on the branch- and-bound (BaB) framework. BaByy takes as input ReL.U-based DNN and property of interest. iteratively alternates between two components: Decide (line 6), which performs neuron-splitingg by assigning an activation status (active/inactive) to a neuron, and Deduce (line 3), which checks the feasibility of the current activation pattern and prunes infeasible branches.
Our key insight is that the BaB architecture of BaBay naturally supports proof generation. To realize this, augment BaBay with proof tree structure, stored in the proof variable (line 2). We also

---

0.2
91
0.2
U.6
0.3
0.3
0.6
0.3
V2
0.5
(a) A simple DNN.
(b) A proof tree.
Fig. 1: Example of verifying (21, E [-2.00, 2.0] [-1.0, 1,0 + (y1 92).
instrument BaBgy so that each branching decision made during the Decide step is explicitly recorded into this tree (line 10). Each node in the binary proof tree represents a neuron, and its left and right children correspond to the two possible activation decisions (active or inactive).
Example Fig. la illustrates a DNN and how BaByy determines unsatisfiability (i.e., verifies the problem) and generates the unsat proof in Fig. 1b First, BaByv initializes the activation pattern set PatPette with an empty activation pattern D. Then BaBxv enters a loop (line l 10) to search for satisfying assignment
proof of unsatisfiability.
1st iteration:
BaBgy selects the only available activation pattern 0 E ActPallerss, and calls Deduce to check the feasibility of the problem based on the current activation pattern. Deduce uses abstraction to approximate that from the input constraints the output values are feasible for the given network. Since Deduce cannot determine infeasibility. BaBay invokes Decide to randomly select neuron to split. Suppose it selects neuron #4, which results the original problem being divided into two independent subproblems: one where is active, and another where 14 is inactive. BaBwy then adds 24 and Vy to Act AetPatttnss
BaBny has two subproblems that can be processed in parallel. For the first subproblem with 04, Deduce cannot decide infeasibility, it selects 02 to split. then conjoins 24 with v2 and then with by and adds both conjuncts to Acl Patterns. For the second subproblem with 8 inactive (i.e.. Dz). Deduce determines that the problem unsatisfiable and BaByy saves Vi to the proof tree, as node 3, to indicate one unsatisfiable pattern, i.c., whenever the network has ty being inactive, the problem is unsatisfiable.
2nd iteration:
3rd iteration: BaByy has two subproblems for v4 A U2 and 14 A U2. For the first subproblem, Deduce cannot decide infeasibility, it selects U1 split. It then conjoins en and then i1 to the current activation pattern and adds them to PctPareesss. For the second one, Deduce determines that the problem is unsatisfiable and BaBwy saves the 04 A U2 the proof tree, as node 5.
4th iteration: BaBay has two subproblems for v4 A U2 A 6i and v4 A 02 A गт. Both subproblems are determined to be unsatisfiable, and BaBgy saves them to the proof tree nodes 6 and 7, respectively.
Finally, BaByy has an empty Act Patterrss stops the search, and returns unsat and the proof tree.
The APTP proof tree The resulting proof tree has a specific structure. First, is a binary tree where each parent node must have children for both activation status values of a neuron. Second, it is proof tree that captures unsatisfiability reasoning, i.e., each leaf holds the constraint showing the activation pattern encoded from the root to this leaf results in unsatisfiability. The tree in Fig. Tb) demonstrates this structure. Each white node corresponds decisions to split neurons. The red leaves correspond to the unsatisfiable patterns that are saved to the proof tree. Note that a leaf node implies the unsatisfiability of the subtree rooted at the leaf, e.g.. node 3 encodes the unsatisfiability of set of 8 activation patterns.
branching node where BaByy makes
We leverage this structure to store the proof in the APTP format 33.2 and check it using the APTPchecker algorithm
3.2 The APTP Proof Language
We have shown in 33 that the broad class of BaBgy DNN verification techniques can generate a binary tree that represents a proof of unsatisfiability (53). We define a standard proof format for specifying DNN proofs, APTP, that is human human-readable, compact, and efficiently generated by verification tools

---

(proof) ::= (declaration) (assertions)
Declare variables
(deciarations) (declaratio)) (declorotion) (declorations)
(declare-const X_0 Real)
(declaration) :: (declare-const (input-vars) Real)
(declare-conet X.1 Real)
(declare-conet Y_0 Real)
[declare-const (output-vare)) Real)
(declare-const Y_1 Real)
| (declare-pwl
(hidden-vars) (activation)
(declare-pH] ReLU)
(declare-pıl H_2 BelU)
(aasut-vars) =: (input-var)
(iaput-var) (ieput-vars)
(declare-pul H_3 ReLU)
(output-vons) (onsput-var)
(outpиt-var) (ourput-vors)
(declare-pel 8,4 BeLU)
(hideen-vars) :: (Nidden-var) (hidden-var) (Nldden-varr)
10 Input constraints
11 (assert (>* X_0 -2.0))
(activation) ReLU Leaky ReLU
12 (assert: (<= x_0 2.0))
(azsertions) : (assertiin)) | (assertion) (assertions)
(assert (>- X_1 -1.0))
14 (assert (<- X_1 1.0))
(assertion) (assert (formda))
15 Output constraints
(formula) :: ((operator) (term) (temm)
16 (assert (<- Y_0 Y_1))
(and (Jormla))+) (or (formule)
17 Hidden constrainte
18 (assert (or
(term) = (Input-var) (outpur-var)
19 (and (< k_4 0))
(Ridden-var) (constont)
20
(and (<
(operator) ==
1 S
21
(>- E_4 0))
22
(>
(input-var)
(constunt)
23
(>- N_1 0)
(output-vor) ==
Y (constonn)
24
(>= K_4 0))
(hidden-var)
(conston))
25
(and
(>=
26
(< R_1 0)
(constont)
Int Real
27
))))
(a) The APTP proof language.
(b) APTP example.
Fig, The APTP format.
and processed by proof checkers. APTP is inspired by the SMTLIB format [3] used for SMT solving, which has also been adopted by the VNNLIB language [29] specify DNN verification problems.
Fig. 2a presents the syntax of the APTP proof language. proof consists of declarations and assertions. Declarations define input/output variables (real numbers) and hidden variables (with PWL activations like ReI.U). Assertions encode preconditions over inputs and postconditions over outputs using logical formulas with comparisons and Boolean operators like and and or. More details on the syntax and semantics of APTP are available in (Apdx. A).
Example The APTP proof in Fig. 2b corresponds to the proof tree in Fig. lb The statement (and (< N_4 0)) corresponds to the rightmost path of the tree with V. decision (leaf 3). The statement (and (< N_2 0) (>= N_4 0)) corresponds to the path with 144 A D2 (leaf 5).
The APTP language is intentionally designed to omit explicit weights and biases reduce the size of the proof structure, and (b) explicitly encode a DNF structure to enable easy parallelization. The weights and biases of the DNN already recorded in the ONNX format [30]. which serves
standard input to both verification tools and APTP checkers, like the one we describe in [
Note that the APTP language can be extended to support other piece-wise linear activation functions, e.g., Leaky ReLU. Particularly, (a, a) = x if 2 else aæ are expressible by APTP, e.g., (>= Ni 0) for and (< Ni 0) for off, which is essentially identical to ReLU, while the MILP encoding described in $4.1 would handle the semantics of Leaky operations.
3.3 Handling Input Splitting
employ input-splitring strategies that partition the input domain rather than branching on neuron activations as in BaB. Our APTP proof generation and format can naturally represent these cases without modification. More specifically, each input split can be encoded as conjunction of input-range constraints describing the corresponding subspace, replacing the "hidden activation" constraints used neuron splitting. For example, splitting on #1 at 0 yields two subproblems, each defined by intervals on and w2, as shown below.
For networks with a small number of inputs (e.g., < 50), some verifiers B1 32
16
(aasert (or
(and (>= I_0 -2.0) (<- x_0
2.0) (>- I_1 0.0) (c- I_1 1.0)) (and (>- I_0 -2.0) (<- x_o 2.0) (>- X_1 -1.0) (<- X_1 0.)))
5

---

Alg, 2. APTPchecker algorithm.
input :DNN N, property Φin 7 Pout, proof
output scerliffed if proof is valid, otherwise uncerti uncerlified
RepOK (proof) then RaiseError (Invalid proof tree)
model < CreateStabilizeddMI..
Win, Pout) // initialize MILP model with inputs
while proo. de
node t Select(proof) // get node check
model t AddConstrs (model, node) // add corresponding constraints
if CheckFeassbbiitty (model) then
return uncerti fied cannot certify
return certified
In this setting, the proof generation step emits input-subspace constraints instead of hidden-layer constraints, and the proof tree structure remains unchanged. Essentially, we treat each input dimension as a "neuron" and split on its range, similar to neuron splitting in BaB.
4 Checking APTP Proofs
After generating proofs, the next step validate them. The goal is to separate proof generation from proof checking, where existing verifiers generate UNSAT proofs while separate, independent checker independently validates these proofs. To this end, we introduce a proof checker, APTPchecker, that validates APTP proofs. First, APTPchecker eschews all optimizations and complexities in modern verifiers. It only needs validate the final activation pattern claims, regardless of the sophisticated search strategies, bound tightening techniques, or pruning heuristics used by the verifiers. Next, APTPchecker achieves substantially reduced trusted code base, requiring only 800 SLOC compared to 20K SLOC in verifiers, while providing verifier independence where the same proof format works across multiple verification tools. Finally, APTPchecker also uses the MILP solver as a black box, allowing multiple solvers to be employed to increase confidence in results.
4.1 The Core APTPchecker Algorithm
The goal of APTPchecker is verify that the APTP tree generated by a DNN verification tool is cor- rect (i.c., the proof tree is a proof of unsatisfiability of the DNN verification problem). APTPchecker thus must verify that the constraint represented by each leaf node in the proof tree is unsatisfiall. To check each node, APTPchecker forms an MILP problem consisting of the constraint in Eg. 1 (the DNN, the input condition, and the negation of the output) with the constraints representing the activation pattern encoded by the tree path to the leaf node. APTPchecker then invokes an LP solver to check that the MILP problem infeasible, which indicates unsatisfiability of the leaf node.
Core Algorithm Alg. 2 shows a minimal (core) APTPchecker algorithm, which takes as input DNN N. property Pin 7 Pout, proof tree proof, , and returns certi fied if the proof tree is valid and uncerti, If fied otherwise. APTPchecker first checks the validity of the proof tree (line 2). i.c., the input must represent proper APTP proof tree (S.22.. If the proof tree is invalid, APTPchecker raises an error. APTPchecker next creates MILP model (Tine 2} representing the input. APTPchecker then enters a loop (line 3) that selects a (random) leaf node from the proof tree (line 4) and adds its MILP constraint to the model (line 5). It then checks the model using an I.P solver to determine whether the leaf node is unsatisfiable If the LP solver returns fcasibility. APTPchecker returns uncerti fied, i.c., it cannot verify the input proof tree. APTPcheckes continues until all leaf nodes are checked and returns certi fied, indicating the proof tree is valid.
Example For the APTP proof in Fig. 2b we need to check that the four leaf nodes 3, 5, 6, and 7 of the proof tree in Fig. lb are unsatisfable. Assume APTPchecked first selects node 3, it forms the MILP problem for leaf node by conjoining the constraint representing 0.6u; + 0.90² 0.1 < 0 (i.e., 02) with the constraints in Eq. Trrepresenting the input ranges and the DNN with the objective of optimizing the output. APTPchecker then invokes an LP solver, which determines that this MILP is infeasible, i.e., leaf node indeed leads unsatisfiability. APTPchecker continues this process for the other three leaf nodes and returns certi fied as all leaf nodes are unsatisfiable.

---

MILP Formulation
APTPchecker formulates MILP problems 33 and checks for feasible solutions using off-the-shelf IP solving. Formally, the MILP problem is defined as:
(a) g(0 + b
(b) y = 2 2L) : ====3: = =00;
(c)
2) 2 2 22
(0) (8) 2/ 20;
(d) aj E (0.1}:
(1)
(2)
(c)
2 - 0j (0) (0
S 2
(1)
1 1
))
where æ is input, y is output, and =00 300, W and ((0) are the pre-activation, post-activation, weight, and bias vectors for layer i, respectively. This encodes precisely the semantics of a ReLU- based DNN: (a) the affine transformation computing the pre-activation value for neuron; (b) the inputs and outputs in the DNN; (c) assertion that post-activation values non-negative and no less than pre-activation values; (d) neuron activation status indicator variables that are either 0 or 1; and (e) constraints on the upper, "j and lower, 11") bounds of the pre-activation value of the jth neuron in the ith layer. Deactivating a neuron, dj (a)
(i) = 0, simplifies the first of the (e) to =(() 0, and
activating neuron simplifies the second to ={4) I "j which is consistent with .6) mar(=;) (1)
Correctness Alg. returns certif fied iff the input APTP proof tree unsatisfiable. This proof tree encodes disjunction of constraints, one per tree path, where each constraint represents activation pattern of the network (the leaf node). Then each problem is reduced to a simple LP that exactly captures the semanties of the DNN for specific activation pattern and thus, the algorithm introduces no approximations, i.e., it is sound and complete.
Note that this correctness argument assumes that the LP solver is correct. In practice multiple solvers could be used to guard against errors in that component. It is standard for proof checkers to assume the correctness of small set of external tools, c.g., checkers that use theorem provers assume the correctness of the underlying prover 34].
Implementation and Validation APTPchecker is written in Python, and uses Gurobi 35 for LP solving. The core APTPchecker algorithm (Alg. 2) consists of 600 SLOC, while optimizations use an additional 200 SLOC. Currently, APTPchecker supports ReLU-based feed-forward (FNNs) and convolutional neural networks (CNNs). APTPchecker uses ONNX for neural networks and outputs APTP proofs. In addition, we used the CrossHair [36] symbolic execution tool to check the correctness of the core algorithm in APTPchecker. Specifically, CrossHair confirmed that key postconditions hold, c.g., that APTPchecker returns certifict fied if and only if all leaf nodes in the proof tree formally proven. While the verification is not exhaustive (CrossHair only explore program paths up to certain depth), this increases confidence in the implementation's 's correctness up certain depth. detailed discussion is provided in AAdx. B
4.2 Optimizations
Our APTPchecker implementation employs several optimizations to improve efficiency, especially for large proof trees. It uses neuron stabilization to identify stable neurons (either active or inac- tive) and replace disjunctive constraints with linear ones, and simplifying the MILP problem and reducing the work of the LP solver. Additionally, it employs pruming of leaf nodes and backtracking to check parent nodes only when necessary. reducing the number of LP problems to be solved. Finally, APTPchecker leverages the tree structure of APTP proof to parallelize the checking of leaf nodes, making the verification process scale better to large proof trees. Additional details on these optimizations are available in Apdx. C
Evaluation
We evaluate our work using the following research questions: RQI 85.1 How does proof generation
and checking perform with existing verifiers? RQ2 (...) What factors impact proof checking
performance? RQ3 ( How does this work compare to prior work? RQ4 65.4): What are some unsound cases that we can detect?
7

---

Tab. 1: Benchmarks consist of 8 neural networks comprised of varying numbers of CNN (C) and FNN (F)
layers, neurons, and parameters, each paired with properties form UNSAT verification instances.
Networks
Instances Num.
Name
Num.
Layers
Neurons
Param.
CNN
8
1-2C:1F
320-3920
41K-180K
200
FNN
2-6F
64-3072
27K-1.7M
200
Tab. 2: APTPchecker performances in seconds (mean/median/maxx)
FNN
CNN
Verifier
Verify
Check Proof
Verify
Check Proof
a.S-CRWN
4/9/481
3/19/870
6/16/50
13/67/866
NeuralSAT
9/16/304
2/10/599
12/30/64
7/42/825
Benchmarks We evaluate on UNSAT verification problems selected from the benchmark suite intro- duced in 19], which includes ACAS Xu, RESNET A/B, CIFAR2020, MNISTFC, and MNIST GDVB. Specifically. MNISTFC contains networks with 4, and layers, each with 256 ReLUs per layer. MNIST GDVB networks, generated using GDVB [37], range from 2-6 layers with 16-512 neurons per layer, yielding 64-3072 ReLU counts total. As with prior work [5] we exclude ACAS Xu. which has networks with low input dimensions and did not even need to use BaB activation space to be solved. We also exclude RESNET, which are currently not supported by the APTPchecker. Note that this straightforward engineering limitation and there is no fundamental reason the checking algorithm is not applicable.
From CIFAR2020, we selected CNN models and varied convolutional sizes and depths; from MNISTFC and MNIST_ -DVB, chose FNNs of diverse sizes. For robustness properties, use local perturbation radii ranging from 0.01 to 0.09 in Loo norm. For each network, we randomly sampled local robustness properties until we obtained 25 UNSAT instances, yielding 200 CNN and 200 FNN problems (400 total) as shown in Tab.
Baselines We adapted verifiers: a. a.3-CRWN and NeuralSAT, generate APTP proofs. We also compare our work with the Marabou verifier and its proof checker (88..
Metrics To assess performance we use the two common metrics in the verification community 4 (i) time to solve and (ii) number of problems solved. We record time to verify, generate, and check proofs, using 1000-second timeout. A problem is "solved" if all steps complete within time limit.
Setup All experiments were run on a Linux machine with an AMD Threadripper 64-core 4.2GHZ CPU, 128GB RAM, and an NVIDIA GeForce RTX 4090 GPU with 24 GB VRAM.
5.1 RQ1: Proof Generation and Checking
shows cactus plots for proof generation and check- ing with the underlying NeuralSAT and a:3-CROWN ver- ifiers. In cactus plots like this, lines that extend further on the x-axis are better more problems solved and lines that are lower are better faster solve times. The dashed lines show the performance of the verifier and the solid lines show the performance of the verifier, proof generation, and the proof checker.
Fig.
n.3 C--C+AATPeusskrr
a3-C
NS- APTP.tckee
Runtimes
600
3000
4000
300
200
100
Fig. 3 with additional timing details given in Tab. show that checking proofs is slower than verification itself. Mod- em verification tools rely on sophisticated abstractions and optimizations (e.g., arB-CROWN and NeuralSAT employ advanced bound propagation, pruning heuristics, and leverage GPU acceleration), but these enhance- ments preciscly what make verification complex and error-prone. In contrast, APTPchecker
0-
1001
2000
300
400
Solved problems
Fig. 3: Runtimes and problems solved.

---

Tab. 3: subproofs vs. MILP complexity.
Num. subproofs
MILP Complexity
Verifier
Mean
Median
Mean
Median
a/9-CROW
230
180
414
179
NeuralSAT
95
36
601
545
10a
10a
FNN (solved)
(unsolved)
10a
Counts
CNN (solved)
10a
CNN (unsolved)
104
10
10
10
500
1000
1500
2000
2500
3000
2000
MILP Complexity
Sub-proofs
Fig. 4: Number of constraints and subproofs per problem.
deliberately eschews all verifier-specific optimizations and systematically validates each proof node using MILP (e.g., we use Gurobi which runs on CPU but can be replaced with other solvers), prioritizing reliability over speed.
Overall, both verifiers were able generate proofs for all problems with APTP, and APTPchecker is able to check between 93.7% and 99.4% of the proofs that are generated. This demonstrates that the APTP is able to encode proofs generated by differing neural network verification algorithms, and that APTPchecker can check them.
5.2 RQ2: Proof Checking Analysis
Tab. 3 shows statistics on the number of subproofs the number of MILP calls made by APTPchecker (cquivalent to the number leaf nodes in the proof tree no pruning is used), and the MILP complexity- the number of constraints in the MILP problem at each MILP call, of proofs generated by MeuralSAT and a:8-CROWN. We can see that NeuralSAT produces fewer subproofs, but with more complex MILP problems. In contrast, a:- B.B-CROWN generates significantly more subproofs, but with simpler MILP problems. This variation suggests directions for future work, such as designing NeuralSAT generate larger proof trees with simpler MILPs for better parallelization.
Fig. 4eeplores the impact MILP complexity and subproofs on APTPchecker performance. The MILP complexity distribution reveals distinct patterns across network architectures, e.g., FNN prob- lems predominantly occupy the lower complexity range (0-1000 constraints), where APTPchecker nearly solves them all. In contrast, CNN problems have broader complexities, with many instances requiring more than I 1000 constraints and exhibiting higher unsolved rates at greater complexities. For subproofs CNN and FNN architectures both generate remarkably similar proof tree sizes, with most problems requiring fewer than 1000 subproofs. However, the key difference lies in the complexity of individual MILP subproblems. FNN proofs involve simpler MILPs that can be efficiently verified, whereas CNN proofs remain challenging even with small proof trees due to the higher computational cost of solving individual MILP subproblems.
This architectural difference highlights that FNN problems generate more tractable MILP formula- tions, whereas CNN problems pose greater computational challenges due to their inherently more complex constraint structures.
5.3 RQ3: APTPchecker vs. Marabou's Checker
To evaluate proof checking performance in isolation, we compared APTPchecker against the proof checker built into the Marabou verifier [38], which uses Farkas's lemma and implemented in the Imandra framework
(25) We used identical verification results from the Marabou verifier for both

---

Tab, 4: Proof checking times of Marabou's 's checker and APTP cheker.
Proof checking time
Checker
Mean
Median
Max
Marabou checker
204
785
APTPchecker
9
38
checkers. Marabou successfully verified 54 problems from our benchmark suite. For each verified problem, extracted the proof in APTP format and measured the time required by each checker to validate the proof.
Tab. presents the proof checking times for both approaches. The similar mean times indicate that both checkers handle simple problems quickly. However, for more challenging proofs, APTPchecker demonstrates significant advantages with over 20x speedup in both median (c.g.. 204 seconds) and maximum (c.g., 38 vs 785 seconds) checking times. This demonstrates the effectiveness of APTPchecker's optimizations for complex proof validation.
5.4 RQ4: Unsound Cases
So far APTPchecker can validate sound proofs generated by existing verifiers. While unsound results are rare in practice, we were able to use this work to identify both real and synthetic examples of soundness bugs existing verifiers, demonstrating
goal ensuring verifier soundness.
Real Bugs One concrete example of such a soundness bug is documented in NeuralSAT Github issuc #8, where NeuralSAT falsely returns UNSAT for SAT instance [39]. We were able extract the APTP proof tree from this false verification result and analyzed it using APTPchecker. APTPchecker successfully found valid counterexample, confirmed by NeuralSAT developers, that violates the claimed UNSAT result.
Synthetic Bugs
Zhou al. 40 explored verifier soundness by introducing synthetic bugs into verifiers. One studied class of bugs involves randomly dropped branches during BaB search, e.g., a proportion of branches created in each BaB iteration randomly discarded. This bug breaks the completeness of the BaB algorithm, as may miss branches containing counterexamples, leading to false UNSAT claims. APTPchecker easily detects such incompleteness bugs because it always first checks is that all branches in the proof tree are properly visited and justified. Thus, APTPchecker will catch missing branches and raise errors indicating invalid proof that it cannot certify.
Conclusion and Future Work
We introduce a proof format APTP which can express proofs generated by state-of-the -h-t-e- DNN verifiers, and a proof checker APTPchecker that can validate proofs in this format. Together, these contributions establish concrete foundation for certifiable and reliable neural network verification, closing the gap between practical verification and assurance required for real-world deployment.
Limitations APTPchecker relies on an external MILP solver such as Gurobi, which itself is not formally verified and can contain bugs (c.g., doating-point errors). While this a limitation as we cannot climinate all trusted components, this work allows to significantly reduce the trusted code base. This can also mitigated by using multiple solvers to cross-check results, e.g., CPLEX and Xpress [2]. or by replacing the MILP solver with formally verified LP solver, e.g., SCIP 43
Potential Negative Societal Impact The research line DNN verification can exploited find issues in DNNs and this work, which aims to improve DNN verification, indirectly supports that. However, DNN verification, and therefore this work, also helps to ensure that DNNs safe and secure for deployment in critical applications.
10

---

Acknowledgments and Disclosure of Funding
We thank the anonymous reviewers for their helpful comments. This work was supported in part by funds provided by the National Science Foundation awards 2129824, 2217071, 2501059, 2422036, 2319131, 2238133, and 2200621, and by an Amazon Research Award and an NVIDIA Academic Grant.
References
[1] Nuro-ai, "Nuro Driver," 2024. https:/www..uro..ffccchhoooo
[2] B. Kovatchev, Castillo, Pryor, L. L. Kollar, C. L. Barntt,, M. D. DeBoer, and S. A. a. Brown, "Neural-net artificial pancreas: A randomized crossover trial of first-in-class automated insulin delivery algorithm," " Diabetes Technology & Therapeutics, vol. 26, no. 6, pp. 375-382. 2024.
[3] D. Hanover, A. Loquercio, L., Bauersfeld, A. Romero, R. Penicka, Y. Yongg G. Cioffi, F. Kauf- mann, and D. Scaramuzza, "Autonomous drone racing: A survey," IEEE Transactions on Robotics, 2024.
[4] C. Brix, S. Bak, C. Liu, and T. Johnson, "The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results," 2023.
[5] H. Zhang, S. Wang. K. Xu, L. Li, B. Li, S. Jana, C.-J Hsieh, and J. 7. Kolter, "Gen- eral cutting planes for bound suund-ropaggtioonbeee based neural network verification," " arXiv preprint
arXiv:2208.052 05744. 2022.
[6] A. Lemesle, J. Lehmann, T. L. Gall, and Z. Chihani, "Verifying neural networks with pyrat," in International Static Analysis Symposium, pp. 11-33, Springer, 2025.
[7] S. Wang, H. Zhang, K. Xu, X. Lin, Jana, C.-J Hsieh, and J. Z. Kolter, "Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification," Advances in Neural Information Processing Systems, vol. 34, pp. 29909-29921, 2021.
[8] H. Duong, T. Nguyen, and M. B. Dwyer, "Neuralsat: A high-performance verification tool for deep neural networks," in International Conference on Computer Aided Verification, pp. 409- 423, Springer, 2025.
[9] H. Duong, D. Xu, T. Nguyen, and M. B. Dwyer, "Harnessing neuron stability to improve dnn verification," " Proc. ACM Softw, Eng., vol. 1, jul 2024.
[10] N. Wetzler, M. Heule, and W. A. Hunt Jr, "Drat-trim: Efficient checking and trimming using expressive clausal proofs," in International Conference on Theory and Applications of Satisfiability Testing, pp. 422 429, Springer, 2014.
[11] Pollitt, M. Fleury, and A. Biere, "Faster Irat checking than solving with cadical," in 26th International Conference on Theory and Applications of Satisfibbility Testing (SAT 2023), Schloss Dagstuhl-Leibnic Lerruuhh--Lebbbiizeeertuuu für Informatik, 2023.
[12] P. Lammich, "Efficient verified (un) sat certificate checking," Journal of Automated Reasoning, vol. 64, no. 3, pp. 513.513-11 513-532, 2020.
[13] C. Barrett, A. Stump, C. Tinelli, et al., "The smt-lib standard: Version 2.0." in Proceedings of the 8th international workshop satisfiability modulo theories (Edinburgh, England), vol. 13,
p. 14, 2010.
[14] G. Katz, Barrett, D. L. Dill, Julian, and M. J. Kochenderfer. "Towards proving the adversarial robustness of deep neural networks," Proc. 1st Workshop on Formal Verification of Autonomous Vehicles (FVAV), pp. 19-26, 2017.
[15] M. Kochenderfer, J. F. Holland, and J. Chryssanthacopollos, "Next-generation airborne col- lision avoidance system," tech. rep., Massachusetts Institute of Tecchnolog--Lincol Laboratory Lexington United States, 2012.
[16] H. Duong, T. Nguyen, and M. Dwyer, "A DPLL(T) Framework for Verifying Deep Neural 2024.
Networks," arXiv preprint arXiv:2307.
[17] Bak, "nnenum: Verification of relu neural networks with optimized abstraction refinement," in NASA Formal Methods Symposium, pp. 19-36, Springer, 2021.

---

[18] C. Ferrari, M. N. Mueller, N. Jovanović, and M. Vechev, "Complete Verification via Multi- Neuron Relaxation Guided BannhhanddBoundee in International Conference on Learning Representations, 2022.
[19] L. Cruz-Filipe, M. J. Heule, W. A. Hunt, M. Kaufmann, and P. Schneider- "Efficient certified rat verification," in Automated Deduction- CADE 26: 26th International Conference on Automated Deduction, Gothenburg, Sweden, August 6-11, 2017, Proceedings, pp. 220-236, Springer, 2017.
[20] I.. De Moura and N. Bjørner, "73: An efficient smt solver," in International conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 337-340, Springer, 2008 [21] T. Bouton, D. Caminha B. de Oliveira, D. Déharbe, and P. Fontaine, "verit: an open, trustable and efficient smt-solver;" in International Conference on Automated Deduction, pp. 151-156, Springer, 2009.
[22] A. Cimatti, A. Griggio, B. J. Schaafsma, and R. Sebastiani, "The mathsat5 smt solver," in International Conference on Tools and Algorithms for the Construction and Analysis of Systems,
pp. 93-107, Springer, 2013.
[23] J. Christ, J. Hoenicke, and A. Nutz, "Smtiintrpoll: An interpolating smt solver," in International SPIN Workshop on Model Checking of Software, pp. 248-254, Springer, 2012.
[24] H. Barbosa, C. Barrett, M. Brain, G. Kremer, H. Lachnitt, M. Mann, A. Mohamed, M. Mo- hamed, A. Niemetz, A. Nötzli, A. Ozdemir, M. Preiner, A. Reynolds, Y. Sheng, C. Tinelli, and Y. Zohar, "cvc5: A versatile and industrial-strength smt solver," in Tools and Algorithms for the Construction and Analysis of Systems (D. Fisman and G. Rosu, eds.), (Cham), pp. 415-442, Springer International Publishing, 2022.
[25] O. Isac, C. Barrett, M. Zhang, and G. Katz, "Neural network verification with proof production," " Proc. 22nd Int. Conf. on Formal Methods in Computer -ided Design (FMCAD), 2022.
[26] G. Passmore, S. Cruanes, D. Ignatovich, D. Aitken, M. Bray, E. Kagan, K. Kanishev, M.Malan and N. Mometto, "The imandra automated reasoning system (system description)," ," in Automated Reasoning: 10th International Joint Conference, IJCAR 2020, Paris, France, July 1-4, 2020, Proceedings, Part II 10, pp. 464 471, Springer, 2020.
[27] G. Katz, D. A. Huang, D. Ibeling, K. Julian, C. Lazarus, R. Lim, Shah, S. Thakoor, H. Wu, A. Zeljić, et al., "The marabou framework for verification and analysis of deep neural networks," " in International Conference on Computer Aided Verification, pp. 143-452, Springer, 2019.
[28] E. Y. Nakagawa, M. Guessi, J. C. Maldonado, D. Feitosa, and F. Oquendo, "Consolidating a process for the design, representation, and evaluation of reference architectures," in 2014 IEEE/FIP Conference on Software Architecture, pp. 143-152, IEEE, 2014.
[29] A. Tacchella, L. Pulina, D. Guidotti, and S. Demarchi, "The international benchmarks standard for the Verification of Neural Networks," " 2023. https://www. http:://///wwwwmiibborrgg vnnlib.org/
[30] J. Bai, F. LU, and K. Zhang, "ONNX Open neural network exchange." https://onnx..ai
[31] S. Wang, K. Pei, J. Whitehouse, J. Yang, and S. Jana, "Efficient formal safety analysis of neural
networks," " Advances in Neural Information Processing Systems, vol. 31, 2018.
[32] S. Wang, K. Pei, J. Whitehouse, J. Yang, and S. Jana, "Formal security analysis of neural networks using symbolic intervals." in 27th USENIX Security Symposium (USENIX Security 18), pp. 1599-1611,, 2018.
[33] V. Tjeng, K. Y. Xiao, and R. Tedrake, "Evaluating robustness of neural networks with mixed integer programming." " in International Conference on Learning Representations, 2019.
[34] P. Lammich, "Grat: formally verified (un) proof checker," SAT' COMPETITION 2023,
p. 82, 2023.
[35] Gurobi Optimization, LLC,, "Gurobi Optimizer Reference Manual," 2022. https://www.uuroii com
[36] CrossHair, "An analysis tool for Python that blurs the line between testing and type systems.."
2025. httpss/ggghuucco///schaeellynnn
[37] D. Xu, D. Shriver, M. B. Dwyer, and S. Elbaum, "Systematic generation of diverse benchmarks for dnn verification," in International Conference on Computer Aided Verification, pp. 97-121, Springer, 2020.
12

---

[38] R. Desmartin, O. Isac, G. Passmore, K. Stark, E. Komendantskaya, and G. Katz, "Towards certified proof checker for deep neural network verification." " in International Symposium on Logic-Based Program Synthesis and Transformation, pp. 198-209, Springer, 2023.
[39] NeTralSAT, "NeuralSAT falsely claims verification," 2025. https://gihub..ommddnnaaaa
neuralsat/issues///
[40] X. Zhou, H. Xu, A. Xu, Z. Shi, C.-J. Hsieh, and H. Zhang, "Testing neural network verifiers: A soundness benchmark with hidden counterexamples, arXiv preprint 2xt.:m2222222222.01555.2222 2024.
[41] IBM, "IBM ILOG CPLEX Optimizer." 2025.
https//www.bbm.com/rrddddds
studioo/plleex-ppiiize
[42] FICO, "Xpress Optimization." " 2024.
https:/www..fcc.ccm///////oooott
fico-xpress spppprsssoootiiztioonn
[43] The SCIP Optimization Suite, "Solving Constraint Integer Programs," 2025. https://www
scipopt.org/
[44] M. R. Garey and D. S. Johnson, Computers and intractability, vol. 174, Freeman San Francisco, 1979.
[45] C. Barrett, L. De Moura, and P. Fontaine, "Proofs in satisfiability modulo theories," " All about proofs. Proofs for all, vol. 55, no. pp. 23-44, 2015.
[46] S. Conchon, A. Mebsout, and F. Zaïdi, "Certificates for parameterized model checking," in FM 2015: Formal Methods: 20th International Symposium, Oslo, Norway, June 24-26, 2015, Proceedings 20, pp. 142, Springer, 2015.
[47] A. Griggio, M. Roveri, and S. Tonetta, "Certifying proofs for Methods System Design vol. 57, no. 2. pp. 178-210, 2021.
model checking," " Formal
[48] A. Van Gelder, "Producing and verifying extremely large propositional refutations: Have your cake and eat too," Annuls of Mathematics and Artificial Intelligence, vol. 65, pp. 329-372, 2012.
[49] E. Goldberg and Y. Novikov, "Verification of proofs of unsatisfiability for enf formulas," in 2003 Design, Automation and Test in Europe Conference and Exhibition, pp. 886-891, IEEE, 2003.
[50] M. J. Heule, W. A. Hunt, and N. Wetzler, "Trimming while checking clausal proofs," in 2013 Formal Methods in Computer-Aided Design, pp. 181-188, IEEE, 2013.
[51] M. Heule, "The drat format and drat-trim checker," arXiv preprint arXiv: 1610006220. 2016.
[52] H. Barbosa, A. Reynolds, G. Kremer, H. Lachnitt, A. Niemetz, A. Nötzli, A. Ozdemir, M. Preiner, A. Viswanathan, S. Viteri, el al., "Flexible proof production in industrial- strength smt solver," in International Joint Conference on Automated Reasoning, pp. 15-35, Springer International Publishing Cham, 2022.
[53] L. De Moura and N. Bjørner, "Saatssiability modulo theories: introduction and applications," Communications of the ACM, vol. 54, no. 9, pp. 69-77, 2011.
[54] G. C. Necula, Compiling with proofs. Carnegie Mellon University, 1998.
[55] M. Armand, G. Faure, B. Grégoire, C. Keller, L. Théry, and B. Werner, "A modular integration of sat/smt solvers to coq through proof witnesses," in International Conference on Certified Programs and Proofs, pp. 135-150, Springer, 2011.
[56] S. Böhme, A. C. Fox, T. Sewell, and T. Weber, "Reconstruction of z3's bit-vector proofs in hol4 and isabelle/hol," Certified P Programs and Proofs: First International Conference, CPP 2011, Kenting, Taiwan, December 7-9, 2011. Proceedings 1, pp. 183-198. Springer, 2011.
[57] S. Böhme and T. Weber, "Fast lcf-style proof reconstruction for z3," in Interactive Theorem Prov- ing: First International Conference, ITP 2010, Edinburgh, UK. July 11-14, 2010. Proceedings
1. pp. 179-194, Springer, 2010.
[58] H.-J Schurr, M. Fleury, and M. Desharnais, "Reliable reconstruction of fine-grained proofs in a proof assistant.." " in CADE, vol. 28, pp. 450-467, 2021.
[59] S. Kan, A. W. Lin, P. Rümmer, and M. Schrader, "Certistr: a certified string solver," in Proceedings of the 1th ACM SIGPLAN International Conference on Certified Programs and Proofs, pp. 210-224 2022.
13

---

[60] B. Andreotti, H. Lachnitt, and H. Barbosa, "Carcara: An efficient proof checker and elaborator for smm proofs in the alethe format," in International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 367-386, Springer, 2023.
[61] H.-J. Schurr, M. Fleury, H. Barbosa, and P. Fontaine, "Alethe: Towards generic smt proof format," arXiv preprint vrXiv22107.02344 2021.
[62] R. J. Vanderbei, "Linear programming: foundations and extensions," Journal of the Operational Research Society, vol. 49, no. pp. 94-94 1998.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our claims match our theoretical and empirical results, APTPchecker solved more problems than SoTA checkers across benchmarks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations explicitly mentioned in $6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they seale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
14

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer:
[Yes]
Justification:
The correctness arguments have been provided in $4.1 and Apdx. B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification:
The model architectures and configurations used are provided in $
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
15

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The data and code with instructions to reproduce the results have been uploaded to an anonymized repo.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.ccpublic Guides/CodeSubmissiinnn
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https) for more details.
/mips.cc/publle wttpcc/cc//ccuuieee wwwpb/iccggi///oooooooo wwt.///eeemsiioooooooooo
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental details have been provided in $5 Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
16

---

Justification: Results are deterministic on the benchmarks, no random process involved, and no error bars need be provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Detailed hardware resources are provided in
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have acknowledged the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [Yes]
Justification:
They have been discussed in 86
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No data or models are released.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cited both a-3-CROWN, NeuralSAT, Marabou and its checker, and libraries used for experimental evaluation.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
18

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode, cotede.oo///atttttt has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets are introduced in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM was only used for revising writing and suggesting words.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy https///oerrb..cccoommtcro/////s for what should should not be described.
20

---

A Syntax and Grammar of APTP
Fig. 2a $3.2 outlines the APTP syntax and grammar, represented production rules. A proof is composed of declarations and assertions. Declarations define the variables and their types within the proof. Specifically, input variables (prefixed with X) and oulput variables (prefixed with Y) declared as real numbers, representing the inputs and outputs of the network. Additionally, hidden variables are declared with specific piece-wise linear (PWL) activation functions, such ReLU, These hidden variables correspond to the internal nodes of the neural network that process the input data through various activation functions.
Assertions are logical statements that specify the conditions or properties that must hold within the proof. Assertions over input variables preconditions and those over output variables post-conditions. Each assertion is composed of a formula, which can involve terms and logical operators. Formulas include simple comparisons between terms (e.g., less than, greater than) more complex logical combinations using and and or operators. The terms used these formulas can be variables constants.
The declare-* statements declare input, output, and hidden variables, while the assert statements specify the constraints on these variables (i.e., the pre and postcondition of the desired property). The hidden constraints represent the activation patterns the hidden neurons in the network (i.e., the proof tree). Each and statement represents tree path that represents activation pattern.
Correctness of APTPchecker Implementation
We were able to verify the implementation of the core APTPchecker algorithm $4.1) using the CrossHair [36] symbolic execution for upto certain thresholds (c.g., timeout per condition per_ wwwoooiiiioiiiiiototttooooooeeee
To perform such analysis, we need create simplified version of APTPchecker including: (I) No optimization remove all optimizations in Apdx. C (2) Assume that Gurobi (MIP) is correct, therefore, the condition indicating whether MIP correct
must be made up (e.g., sum(n) > summation of all literals (e.g., variable and branch condition) in leaf node); and (3) Add pre- and post-conditions to the main function. This make PPTPchecker codebase minimal with just about 100 LoC. In particular, some pre- and post- conditions are listed in Listing
pre: isinstance (proof, list)
(1sinstanee (p, list) for in proof)
poat: in (CERTIFIED, UNCERTIFIED]
post: UNCERTIFIED)
(any (zum
proof))
post: (_ -- CERTIFIED) -- (all (sun (n) >- 0 for in proof))
ana
Listing Pre- and Post- Conditions CrossHair
CrossHair outputs are shown Listing
attempt_calle Postcenditica
analyza_calltree () Path tree stats (COMPIRMED
amalyze_calltree () Iter complete. Worst statue found so far: UHКNORN
atalyzo_calltree () Exceoded condition timeout. stopping
analyza_calltree Aborted calltres search with UNKKOW and messages. Xumber of
iterations: 58
analyze_claes() Analyzing clase ProofReturnStatue
condition.pareer( Using parsers: (AnalysitRind.
AnslyeicKind icontraatt...
Amalyssskkid.deal))
analyzo_classd Analyzing class ProofTree
condition_parser() Using parsers: (AnalysisKind PEP316, AnslysisKind icontract, Mmalyasakind.deal
analyze_functionl Analyzing nip_vorker II
PEP316, AnslysisKiad, icontract, Using parsers: (AnalysisKind PPPSI.. AnalysisKind. deal)
Listing 2: CrossHair traces
21

---

10a
10a
FNN (solved)
FNN (unsolved)
10a
10a
Counts
CNN (solved)
CNN (unsolved)
10
10P
10P
2000
4000
8000
10000
2000
Number of sub-proofs
Sub-proofs
Fig. Number of sub-proof's per problem without (left) and with (right) APTPchecker optimizations.
C
Optimizations
While the core APTPchecker algorithm in Algg. 2 is minimal, it can be inefficient. APTPchecker employs several optimizations to improve its efficiency. These are crucial for checking large proof trees generated for challenging problems.
Neuron Stabilization A primary challenge in DNN analysis is the presence of large numbers of piece-wise linear constraints (e.g., ReLU) which generate a large number of branches and yield large proof trees. In the MILP formulation, this creates many disjunctions which are hard to solve. To reduce the number of disjunctions, APTPchecker uses neuron stabilization [9] to determine neurons that are stable, either active or inactive, for all imputs defined by the property pre-condition. For all stable neurons, the disjunctive ReLU constraint is replaced with linear constraint that represents the neuron's value. This simplifies the MILP problem.
APTPchecker traverses the DNN and computes stable neurons. It initializes the MILP model with input constraints and then iterates over each layer of the network. Next, for each layer, it creates constraints depending on the layer type. Moreover, it uses approximation to estimate bounds of neuron values to determine neuron stability. Next, it filters unstable neurons and attempts make them stable by optimizing either their lower or upper bounds.
Pruning Leaf Nodes APTPchecker uses a backtracking mechanism to check the parent node only when the child nodes infeasible. Specifically, if determines unsatisfiability of leaf 1, it will check the parent of l. If is unsatisfiable it immediately removes the children of p (more specifically the sibling i). Next backtracks to the parent of p and repeats until meeting stopping criteria. This optimization reduces the number of LP problems that need to be solved, making the proof checking process more efficient.
Parallelization APTPchecker leverages the structure of APTP proof tree parallelize the checking of leaf nodes. Each tree path is an independent sub-proof and partitions of the tree allow checker to leverage multiprocessing check large proof trees efficiently.
C.1 Proof Checking Optimizations
Fig. 5) (left) plots a histogram of the number of sub- nub-proofs solved per verification problem. i.c., the number of nodes of the proof tree. When interpreting these plots, understand that the y-axis log scale means that vertical distances have different meaning as you move upward the plot. While the vast majority of the verification problems have proof trees of fewer then 2000 leaves, but 17 of them have larger trees up to a maximum of more than 10000 leaves. Note also that even among the smaller sized proof trees, there some problems that cannot be solved. This is due to complexity of solving the MILP constraints at the leaves of those proof trees.
Fig. 6 (left) plots histogram of the number of occurrences of MILP problems of a given complexity across the benchmarks. Here again we see spread data, but unlike with the number of sub-proofs the CNN benchmarks seem to have consistently larger constraints and there is clear bias among the
22

---

10
108
(adowi)
10
FNN (unabbed)
105
PNN (ussaved)
Connta
CNN (aalva))
Counta
CNN (salved)
10
CNN (snsokred)
10p
CNN (unsokved)
0
500
1000
1500
2000
20000
3020
0
500
1000
1500
2000
2000
3000
MILP Complecity
MILP Complescity
Fig. 6: Number of constraints per problem without (left) and with (right) checkrr optimizations.
unsolved problems towards larger constraint size. To optimize proof checking, we must address both of these sources of complexity.
The Fig. 5 (right) and Fig. (right) explore the impact of the S and X optimizations on the number of sub-proofs and MILP complexity. Across the benchmarks optimizations reduce the number of sub-proofss less than 1000 and MILP complexity to less than 2000. The reduction in sub- proofs directly contributes to the increase in performance of APTPchecker, but the reduction in MILP complexity is more subtle. Integer programming, and thus MILP, is known be NP-Hard in general [44]. The stabilization optimization addresses this complexity by calculating sets of variables that are forced to take on specific values based on other constraints in the MILP problem. For each such variable, the constraints associated with it is effectively eliminated. We can observe this in comparing the left and right of Ffg. 6 where we see both constraints of higher complexity eliminated and the peak of the constraint distribution shifted downward from 400 to 100 constraints.
D Related Work
Proof checking has been widely-recognized in the field of constraint solving such as SAT/SMT
solving. (e.g.. [45 46
47)). There extensive literature clausal proof generation and checking for
49 50 10 51]. Most modern SAT solvers can produce resolution-based proof' in standard formats (e.g., DRAT [TO), which be independently checked by proof checkers, e.g-, by efficient, untrusted programs such as DRAT-trim [00] or by certified, slower programs that work on extended formats such LRAT [9 and GRAT
SAT solvers 48
12
SMT proof checkers [52
share the same purpose of checking unsatisfiability proofs, but they more complex than SAT proof checkers due the richer languages and theories of SMT formulas
53
54
(e.g., theory of strings). Two significant
coof-producing state-of-the of-the-art- SMT solvers are z3 20) and
veriT [21 that both have their proofs successfully reconstructed in proof assistants [55 56 57 58
Other proof-producing SMT solvers are MTthSAT5 (22) and SMTInterpol [23]. CVC5 24 and
CertiStr [59]. Recently, perf-ppeforrancc stand-alone checker Carcara 660 for the Alethe (61
proof format was also introduced.
Compared to SAT/SMT, DNN verification is relatively new field, and the development of proof checkers for DNN verifiers is few. To the best of our knowledge, there is only one line of work (25) that is explicitly for the Marabou. This work uses Farkas's lemma [62] for checking and is implemented in the Imandra 266 that can produce verifiable code. Our work generalizes to neuron neuron-splitting based DNN verification and introduces new, more expressive proof format, APTP, that be adopted by other DNN verifiers. Our proof checker, APTP APTP checker, is also significantly more capable (85.I).
23


Reviews:
[{"review_id": "Z0x2vKNkLw", "reviewer": "Reviewer_my7d", "summary": "The paper proposes the generation of verification proofs by neural networks (which capture the sequence of split decisions and branch evaluations made), and puts forward a MILP-based procedure for validating these proofs. The empirical evaluation of this focuses on performance comparisons with previous work and applicability demonstrations with respect to different neural network verifiers.", "questions": "Please discuss the three points above pertaining to the evaluation of your method.", "limitations": "The second point described in \"Strengths and Weaknesses\", which is potentially a limitation of the approach, has not been addressed. The potential negative societal impact has been adequately addressed.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "While the paper makes an important observation concerning the practical unsoundness of theoretically sound verifiers, thereby adequately motivating the need to ensure the correctness of certification results, the framework proposed appears preliminary for NeurIPS. This is particularly the case for evaluation of the method which in my opinion has two main shortcomings.\n\nFirst, it considers only toy models and does not show the scalability of what proposed to the bigger models that neural network verifiers can handle. This is important as models and specifications that require a lot of branching (with branches often efficiently analysed on GPU) may prove problematic for the slower CPU-based MILP solvers. Further, it is not clear that the overall runtime (of verifying and the validating verification proofs) is better than simply verifying the underlying models using efficient MILP-based solvers, such as [1, 2].\n\nSecond, the paper does not make clear that certified verification proofs ensure the correctness of the verification result. In particular, the paper does not only assume that the underlying MILP solver is correct, but also that the MILP-compilation the paper provides is correct. Efficient compilations however crucially depend on tight bounds for the ReLU nodes, which are typically computed by bound propagation schemes, a core component of  neural network verifiers that is susceptible to floating-point errors. Further, the paper discusses only ReLU splitting and does not address input splitting, a branching strategy that is also present in most neural network verifiers. \n\nThird, the paper does not provide any concrete examples of unsound results from neural network verifiers which have been proven unsound by the proposed method.\n\n[1] V. Tjeng, K. Xiao, R. Tedrake. Evaluating robustness of neural networks with mixed integer programming. ICLR19.\n\n[2] P. Kouvaros, A. Lomuscio. Towards Scalable Complete Verification of ReLU Neural Networks via Dependency-based Branching. IJCAI21.", "was_revised": true, "final_justification": "The paper provides limited evaluation in that it does not make clear the range and scale of models that the proposed approach can support.", "rebuttal": "Thank you for your critical review. We’ve carefully considered your comments and provide detailed responses below to address raised concerns.\n\n\n1. \"Toy\" models and not scale to networks that current verifiers can handle.  Large models require lots of branchings.  \n\n\nWe respectfully disagree that our evaluation benchmarks are “toy models”.  Our benchmark consists of 400 problems across DNNs with up to 1.7M parameters, which are certainly not toys, even for SoTA DNN verifiers such as ABCrown (e.g., see VNN-COMP'24 results on how these tools struggle on similar or even smaller sizes).  Moreover, the only existing proof checking work (Marabou's) evaluates on significantly smaller ACAS Xu networks with just 13K parameters.\n\nThe number of constraints created due to large branching indeed is theoretically exponential. However, in practice we do not observe that and more importantly APTP includes various techniques to improve scalability (e.g., neuron stabilization, proof tree pruning, and parallelization) that are shown to be effective in the paper (e.g., 601 proofs in average for proofs generated by NeuralSAT).   \n\n\n\n2. Overall runtime (of verifying and the validating proofs) is better than simply verifying the underlying models.\n\n\nWe are confused by this question.  The purpose of this work is to provide support for verifying the verifier, i.e., we check that reasoning steps of existing DNN verifiers are sound.  Thus our work does not aim to compete with verifiers but rather is used *after* verification to ensure the correctness of the steps performed by the verification implementation. \n\n3. Correctness of MILP solver\n\nWe discussed this in the paper (4.1.2): APTP uses the Gurobi MILP solver as a blackbox, and as with all other verification work, we do not prove its correctness.  However, our main goal with APTP is *not* to eliminate all trusted components, but rather to significantly reduce the size of the trusted code base (TCB) necessary to assure that the verification run was sound, so that users can have increased confidence in the result. APTP makes the following contributions toward improving confidence in ML verification:  (i) it sets up an architecture to allow for proof checking that is separate from the complex algorithms for DNN verification, allowing their results to be checked independently;   (ii) it implements an independent proof checker APTPChecker and goes further by verifying the implementation of the checker itself—this level of formalism and assurance is rarely seen in DNN verification work and reduces the TCB; (iii) the APTP architecture allows for targeting further reductions in the TCB, e.g., in the future the developers can plug in a different MILP solver, to provide increased confidence under the assumption that MILP solver implementations will not be strongly correlated, or replace LP solving with a trusted theorem prover.\n\nWe also note that the Marabou checker also relies on the correctness of the blackbox SMT solver used inside Imandra (and if we want to go further it relies on the correctness of its integration of Ocaml). This reliance on solvers (e.g., MILP solver in our case) is a common and accepted trade-off in the SoTA of verification checking.  \n\n\nIn summary, while APTP does not eliminate *all* trust assumptions, it significantly reduces them compared to prior work, providing a framework for DNN verification proof checking.  We believe it lays a path toward more trustworthy DNN verification that the community can build on and provides a level of verification of verification results that does not currently exist in the literature.\n\n\n4. Floating-point (fp) errors due to abstraction/bound computations. \n\nIt's a good observation! Indeed, APTP can have fp errors due to the initial bound computation using interval abstraction. However, we do not rely on bound propagation but instead on neuron stability solved by MILP, which is more robust to fp errors compared to complex bound propagation schemes.\nWe note that the initial interval abstraction can be completely replaced by the MILP formulation (Equation 2), e.g., encoding ReLU and solving bounds directly (in fact, the current implementation of APTP does just that and therefore resolves this concern).  We also note that the Marabou checker work also relies on abstraction (DeepPoly) and has the same fp concern.   \n\nInterestingly, from what you said, verification tools could produce unsound results due to the mentioned fp error issue (due to abstraction and optimizations).  If this happens then APTP checker can catch it because it does not have such optimizations and abstractions and instead just recheck their work using MILP solver. \n\n5. Input splitting\n\n\nWe did not consider input splitting which helps deal with networks with low dimensional input spaces. We focus on the major BaB architecture for neuron splittings as challenging and interesting problems since those techniques are used by SoTA NN verifiers and are triggered when verifying networks with lots of neurons and high dimensional inputs.\n\nThat said, APTP proof format can be extended easily to handle the input splitting cases where each formula encoding hidden activation is converted to input intervals capturing the input subspace being checked. The core of APTPchecker would remain unchanged as would the level of trust one could place in verification results.\n\n6. Concrete examples of unsound results from neural network verifiers which have been proven unsound by the proposed method.\n\nWhile we acknowledge this would strengthen the empirical motivation, it does not diminish our contribution's significance. Our evaluation demonstrates that the two state-of-the-art verifiers we tested (α-β-CROWN and NeuralSAT) are sound for the problems we examined - APTPchecker successfully validated 363 out of 400 verification results with no cases where a verifier incorrectly claimed UNSAT.\n\n\n\nThe primary goal of APTPchecker is to provide independent validation that increases confidence in verification results. When APTPchecker cannot validate a proof, this would lead developers to further investigate whether unsoundness occurred."}, {"review_id": "eNDaC36LqB", "reviewer": "Reviewer_Pxf9", "summary": "This paper introduces Activation Pattern Tree Proofs (APTP), a verifier-independent proof format for neural network verification, along with APTPchecker, an efficient and formally correct proof checker. By enabling scalable and reliable validation of verification results, the approach aims to increase trust in neural network verification tools, which are essential for deploying DNNs in safety-critical systems.", "questions": "- I would be interested in seeing a table comparing the runtime required to verify with alpha-beta-CROWN and the runtime your toolbox requires to certify the proof. Could you include such a comparison?\n\n- It would also be helpful to see a table reporting the size and complexity of the DNNs evaluated in your experiments. Please consider including this information.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper is well written and clearly describes the proposed approach.\n- The proposed approach addresses proof generation and checking in the context of neural network verification, which has not significantly been addressed in the literature. This is an important consideration in neural network verification, where soundness errors have been demonstrated somewhat (e.g., tools may say a specification holds when it does not). As such, the paper presents contributions to eliminating this issue to ensure verification results hold (e.g., so if a tool says a specification holds, it actually does).\n- The results illustrate the approach with a sufficiently rigorous empirical evaluation, albeit it has some limitations.\n\nWeaknesses:\n\n- The contribution of the work is not clearly articulated, particularly for a machine learning audience. While some aspects of how tools could produce incorrect results is discussed, it could be much more strongly motiviated. For example, with concrete examples. There are other examples in the literature that exist (eg, due to floating point issues and related numerical issues).\n\n- Some aspects of the intuition of the proof certificate and tree (eg Figure 1) do not fully convey the point of the approach, especially for a machine learning audience. While a verification reader can appreciate this, a machine learning one may not, and this example in particular could be better motivated, as it may come across simply as existing known BaB approaches.\n\n- The proposed method essentially re-derives results using MILP and LP techniques that are very similar to those already employed by existing approaches. Could you clarify what your work contributes technically to the verification problem? The LP and MILP formulations you describe are well-known techniques commonly used in established branch-and-bound methods such as alpha-beta-CROWN.\n\n- The number of constraints required to certify the proof appears to grow exponentially with the size of the DNN. Could you explain how you address scalability in this context?\n\n- The procedure you use to certify proofs seems very similar to the verification steps already performed by existing tools. It appears you are repeating these steps without introducing substantial novelty. Could you please elaborate on what you see as the core contribution of your approach?\n\n- The presentation of Figures 3 and 6 can be improved: the legend overlaps much of the core content of the figures, making them difficult to read.", "was_revised": true, "final_justification": "I have revised my review based on the discussion in support of acceptance and under the assumptions that (1) the authors add an example that demonstrates the importance of proof checking in neural network verification, eg, with a demonstration on an example from VNN-COMP where tools disagree on results, or a floating-point precision example from the literature, which will more clearly demonstrate the importance of this work, (2) clarify the contributions in the paper as discussed especially for the machine learning audience, and (3) make the other improvements that have arisen in the discussion. The potential impact of this direction is significant, with a reasonable approach presented in the paper for proof checking in neural network verification, which is clearer now from the discussion and that addressing these points in a revision if accepted will improve the paper and its potential impact, and that may especially be appreciated by the machine learning audience.", "rebuttal": "Thank you for your time and reviews. We provide our responses to your questions and concerns below.\n\n1. Exponential Constraint Growth.\n\nYes! The number of constraints can indeed grow exponentially, but APTP already includes mechanisms to reduce them with neural stabilization, tree pruning, and parallelization. Our experiments show that the numbers of proofs (and constraints generated) are moderate, e.g., 601 sub-proofs to be checked on average for proofs generated by NeuralSAT.\n\n2. Contributions of Proof checking over verification\n\nAPTP makes the following contributions:\n- APTP validates that verifier-claimed infeasible branches are indeed infeasible, rather than searching for satisfying assignments as the verifier does.\n- Verifiers like α-β-CROWN and NeuralSAT have many additional steps, optimizations, and heuristics beyond the minimal BnB algorithm presented. In contrast, APTP strictly follows BnB and is independent of all those additional complexities: it only needs to validate the final activation pattern claims, regardless of the sophisticated search strategies, bound tightening techniques, or pruning heuristics used by the verifier.\n- Our key contribution is separating proof generation (done by existing verifiers) from proof checking (done by our minimal checker). This provides:\n    - Reduced Trusted Computing Base: 800 SLOC checker vs 20K+ SLOC verifiers.\n    - Verifier Independence: APTP format works with multiple verifiers (α-β-CROWN, NeuralSAT).\n    - Independent Validation step of verification results\n    - Checker validation: We attempt to prove the checker itself correct through symbolic execution (Section 6.3 and Appendix B).\n    - Multiple solver support: Because we use the MILP solver as a black box, multiple solvers can be employed to increase confidence in results.\n    - The APTP format captures activation pattern trees in a verifier-independent way that enables efficient checking while remaining compact and human-readable.\n    - Our optimizations target proof checking efficiency (stabilization, pruning, parallelization) rather than verification search efficiency, leading to different algorithmic choices.\n\n3. Runtime table for α-β-CROWN.\n\nWe'd be happy to. We have all the data to provide a complete table but they took too much space to include in the paper.  We will include all data in the appendix. Here is a summary of the runtimes\n\n\n- FNN (min/median/max): \n    - Verify (4.0/9.0/481.1) \n    - Proof check (3.4/19.0/869.7) \n- CNN (min/median/max): \n    - Verify (6.4/16.4/50.1) \n    - Proof check (12.7/67.4/866.1)\n\nIn short, the results are similar for α-β-CROWN as expected: verification time is a lot shorter than checking verification results.   While proof checking is more expensive than verification, we emphasize that generally speaking one would only check the proofs once.   During development as properties and models are evolving one will rerun verification multiple times, but at the very last step when all the properties have proofs that is the time that you will check them.   In this context the increased cost of proof checking is not as much of a concern.\n\nNote that in our paper we show these results for NeuralSAT in figure 3, but figures are explicitly not allowed by NeurIPS rebuttal. \n\n\n4. It would also be helpful to see a table reporting the size and complexity of the DNNs evaluated in your experiments.\n\nWe believe Tab.1 (page 7) provides these details of the DNNs. Let us know if you need additional info."}, {"review_id": "3iqO1cQCFF", "reviewer": "Reviewer_iqSc", "summary": "In NN verification, verifiers return \"safe\" or \"unsat\" for properties they have verified. However, due to the complexity of these tools, there's an inherent risk that they contain bugs and may be unsound. To verify the verifier, the authors propose to extract a proof-tree from the verifier. This represents the leaf nodes of the branch-and-bound tree, defined by the chosen branching decisions (constraints on internal neuron activations). If an external verifier can confirm that all those leaf nodes define safe subproblems, and their union is the original query, the correctness of the proof has been shown.\nThe authors propose APTPchecker as such a tool: By implementing the logic in Gurobi, and partially verifying the implementation using symbolic execution (CrossHair), they minimize the risk that APTPchecker is unsound.\n\nIn the experimental section, the authors demonstrate that APTPchecker can verify the correctness of a large portion of the proofs generated by two different verification tools, with abCROWN being the winner of several VNN-COMP iterations. This demonstrates the real-world applicability of their proposal.", "questions": "Is it possible to extract a proof tree from Marabou, that you could verify using APTPchecker? If so, how does APTPchecker compare to the built in proof verifier in Marabou?", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Strengths*\nThe paper discusses an important aspect of NN verification: How can we trust the verifiers? The paper does a very good job of introducing the relevant concepts and motivating the need for their proposed algorithm.\nThere is very limited prior work in this regime, so it is great to see some progress on this.\nThe experiments demonstrate the practical applicability and show that APTPchecker can in fact verify the proofs generated by two different NN verifiers.\nThe description of the optimizations (in the appendix) will be helpful for other researchers to replicate the results.\n\n*Weaknesses*\nI would have liked to see an evaluation on more benchmarks/network architectures. How does APTPchecker perform for more complex verification queries? E.g. for the benchmarks from one of the previous VNN-COMP iterations, which of them are already supported, and how toes APTPchecker perform there?\n\n\nMinor\n- Figure 6: I assume the dashed lines are pure abCrown and NS, without the APTPchecker?\n- Should the sentences regarding CrossHair be moved to the Correctness section?\n- Line 48 \"emit[s]\"\n- Grammar: Line 321-322", "was_revised": true, "final_justification": "The authors have responded to all questions and were able to provide additional strong experimental results.", "rebuttal": "Thank you for your encouraging feedback. We provide our responses to your questions and concerns below.\n\n1. More complex networks\n\nAPTPchecker supports FNNs and CNNs but not ResNet.  We mentioned in the paper (Sect.5) that this is an engineering limitation rather than a fundamental algorithmic one.  For ResNet, the skip connections would require extending our MILP encoding, but the proof checking logic (validating that claimed infeasible branches are indeed infeasible) remains unchanged. The core APTP proof checking approach extends naturally to more complex architectures.\n\nIn the last several days we were able to do just that and support Resnet. However, we did not have enough time to run an experiment with it on Resnet benchmarks (we spent most time on extracting Marabou’s code as requested below by you and another reviewer).  We'd be happy to add additional results during discussion time if allowed.  \n\n2. Compare to Marabou checker\n\n\nWe were able to extract APTP proof trees from all 54 problems that Marabou could verify and check proofs.    \n- Marabou checker (min/median/max): 4/204/785 (seconds)\n- APTP checker (min/median/max): 3/9/38 (seconds)\n\nNote that compared to NeuralSAT/α-β-CROWN, Marabou fails to verify many problems. For those that it can solve, Marabou checker is a lot slower than APTP checker as shown."}, {"review_id": "LvEYis7MrS", "reviewer": "Reviewer_r99i", "summary": "This paper presents APTP, a language for expressing proofs of correctness of\nneural networks, and APTPchecker, a tool for verifying proofs written in APTP.\nThis allows neural network verification tools to grow in complexity while still\nproducing a proof object which is checkable by a small trusted verifier,\nincreasing the trustworthiness of deep learning. APTP is specialized to proofs\nbased on branch-and-bound neuron-splitting techniques and mixed integer linear\nprogramming. This proof structure is common to many state-of-the-art neural\nnetwork verification tools. APTP encodes the branches generated by these\nverification tools and then APTPchecker reconstructs the MILP instances needed\nto verify the property in question along each branch. In experiments, existing\nverification tools are modified to produce APTP proofs and it is shown that\nAPTPchecker is able to verify these proofs more effectively than existing proof\nverifiers for neural networks.", "questions": "- How well does APTPchecker work with Marabou as a verification engine? It would\n  be interesting to see how APTPchecker compares to Marabou checker when using\n  the same underlying verifier.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- The utility of a small, trusted proof checker has been established in\n  classical verification and bringing this idea to DNN proofs is a useful\n  contribution.\n- The experiments show that APTPchecker is able to check proofs generated by\n  existing state-of-the-art DNN verification tools.\n- The APTP proof language is generic enough to be used by several different\n  solvers.\n\nWeaknesses:\n\n- The proof language is limited to proofs based on branching on ReLU activation\n  status. To my understanding, this aligns with many existing verification\n  tools, but it precludes the use of APTP with non-ReLU activation functions or\n  verifiers based on other algorithms.\n- Some notation in the results section could be explained. For example, I don't\n  think the S+X label used in the graphs is explained anywhere, though I gather\n  it refers to the optimizations that APTPchecker can use.", "was_revised": true, "final_justification": "APTPChecker presents a valuable contribution to the literature by providing a small, trusted core that can be used to verify proofs generated by more complex tools. There is a small weakness in that APTPChecker cannot handle activation functions that are not piecewise-linear (sigmoid, tanh, etc.), but I believe this is not a major problem given how commonly networks use ReLU activations. The additional experiments presented during the discussion show that APTPChecker is efficient even compared to a checker specifically designed to work with Marabou.", "rebuttal": "Thank you for your encouraging comments. We provide our responses to your questions and concerns below.\n\n1. Compare to Marabou checker\n \n\n\nWe were able to extract APTP proof trees from all 54 problems that Marabou could verify and check proofs.    \n- Marabou checker (min/median/max): 4/204/785 (seconds)\n- APTP checker (min/median/max): 3/9/38 (seconds)\n\nNote that compared to NeuralSAT/α-β-CROWN, Marabou fails to verify many problems. For those that it can solve, Marabou checker is a lot slower than APTP checker as shown.  \n\n2. APTP format limited to ReLU\n\nWe will clarify in the paper that the APTP format is not restricted to ReLU and can handle any piece-wise linear activations as hidden status can be expressed by logical formulae. For example, for Leaky_a(x) = { x if x ≥ 0 else ax }, it status are expressible by APTP, e.g., (>= Ni 0) for on and (< Ni 0) for off, which is essentially identical to ReLU, while MIP encoding will handle the Leaky operation."}]

Author rebuttal:

Meta-review: The paper focuses on checking the correctness of neural network verifiers by proposing an automatic proof-checking procedure. In particular, it focuses on the branch-and-bound-based neural network verification algorithms, where an activation pattern tree can be generated to represent the subproblems solved during verification. For each unsatisfiable subproblem, a MILP solver is used to verify the proofs. The approach is demonstrated on two neural network verifiers, alpha,beta-CROWN and neuralsat, on various benchmarks from VNN-COMP. The method is relatively simple and straightforward, but as one of the first paper exploring this important topic and the engineering efforts the authors made to enable this framework, I think this paper is worth publishing.

One important concern raised by the reviewer is the scalability of the MILP-based approach. This is indeed a valid concern, but I think it is ok to leave these as an open challenge for future exploration. Additionally, the potential numerical and precision issues are worth discussing.
Decision: Accept (poster)