Title: Generating and Checking DNN Verification Proofs
Venue: NeurIPS 2025
Accepted: yes
Keywords: DNN Verification

Proof Generation

Proof Format
Abstract: Deep Neural Networks (DNN) have emerged as an effective approach to implementing challenging subproblems. They are increasingly being used as components in critical transportation, medical, and military systems. However, like human-written software, DNNs may have flaws that can lead to unsafe system performance. To confidently deploy DNNs in such systems, strong evidence is needed that they do not contain such flaws. This has led researchers to explore the adaptation and customization of software verification approaches to the problem of neural network verification (NNV).
Many dozens of NNV tools have been developed in recent years and as a field these techniques have matured to the point where realistic networks can be analyzed to detect flaws and to prove conformance with specifications. NNV tools are highly-engineered and complex may harbor flaws that cause them to produce unsound results.

We identify commonalities in algorithmic approaches taken by NNV tools to define a verifier independent proof format---activation pattern tree proofs (APTP)---and design an algorithm for checking those proofs that is proven correct and optimized to enable scalable checking. We demonstrate that existing verifiers can efficiently generate APTP proofs, and that an APTPchecker significantly outperforms prior work on a benchmark of 16 neural networks and 400 NNV problems, and that it is robust to variation in APTP proof structure arising from different NNV tools.
APTPchecker is available at: https://github.com/dynaroars/APTPchecker.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents APTP, a language for expressing proofs of correctness of\nneural networks, and APTPchecker, a tool for verifying proofs written in APTP.\nThis allows neural network verification tools to grow in complexity while still\nproducing a proof object which is checkable by a small trusted verifier,\nincreasing the trustworthiness of deep learning. APTP is specialized to proofs\nbased on branch-and-bound neuron-splitting techniques and mixed integer linear\nprogramming. This proof structure is common to many state-of-the-art neural\nnetwork verification tools. APTP encodes the branches generated by these\nverification tools and then APTPchecker reconstructs the MILP instances needed\nto verify the property in question along each branch. In experiments, existing\nverification tools are modified to produce APTP proofs and it is shown that\nAPTPchecker is able to verify these proofs more effectively than existing proof\nverifiers for neural networks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The utility of a small, trusted proof checker has been established in\n  classical verification and bringing this idea to DNN proofs is a useful\n  contribution.\n- The experiments show that APTPchecker is able to check proofs generated by\n  existing state-of-the-art DNN verification tools.\n- The APTP proof language is generic enough to be used by several different\n  solvers.\n\nWeaknesses:\n\n- The proof language is limited to proofs based on branching on ReLU activation\n  status. To my understanding, this aligns with many existing verification\n  tools, but it precludes the use of APTP with non-ReLU activation functions or\n  verifiers based on other algorithms.\n- Some notation in the results section could be explained. For example, I don't\n  think the S+X label used in the graphs is explained anywhere, though I gather\n  it refers to the optimizations that APTPchecker can use.\n\nQUESTIONS:\n- How well does APTPchecker work with Marabou as a verification engine? It would\n  be interesting to see how APTPchecker compares to Marabou checker when using\n  the same underlying verifier.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn NN verification, verifiers return \"safe\" or \"unsat\" for properties they have verified. However, due to the complexity of these tools, there's an inherent risk that they contain bugs and may be unsound. To verify the verifier, the authors propose to extract a proof-tree from the verifier. This represents the leaf nodes of the branch-and-bound tree, defined by the chosen branching decisions (constraints on internal neuron activations). If an external verifier can confirm that all those leaf nodes define safe subproblems, and their union is the original query, the correctness of the proof has been shown.\nThe authors propose APTPchecker as such a tool: By implementing the logic in Gurobi, and partially verifying the implementation using symbolic execution (CrossHair), they minimize the risk that APTPchecker is unsound.\n\nIn the experimental section, the authors demonstrate that APTPchecker can verify the correctness of a large portion of the proofs generated by two different verification tools, with abCROWN being the winner of several VNN-COMP iterations. This demonstrates the real-world applicability of their proposal.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths*\nThe paper discusses an important aspect of NN verification: How can we trust the verifiers? The paper does a very good job of introducing the relevant concepts and motivating the need for their proposed algorithm.\nThere is very limited prior work in this regime, so it is great to see some progress on this.\nThe experiments demonstrate the practical applicability and show that APTPchecker can in fact verify the proofs generated by two different NN verifiers.\nThe description of the optimizations (in the appendix) will be helpful for other researchers to replicate the results.\n\n*Weaknesses*\nI would have liked to see an evaluation on more benchmarks/network architectures. How does APTPchecker perform for more complex verification queries? E.g. for the benchmarks from one of the previous VNN-COMP iterations, which of them are already supported, and how toes APTPchecker perform there?\n\n\nMinor\n- Figure 6: I assume the dashed lines are pure abCrown and NS, without the APTPchecker?\n- Should the sentences regarding CrossHair be moved to the Correctness section?\n- Line 48 \"emit[s]\"\n- Grammar: Line 321-322\n\nQUESTIONS:\nIs it possible to extract a proof tree from Marabou, that you could verify using APTPchecker? If so, how does APTPchecker compare to the built in proof verifier in Marabou?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Activation Pattern Tree Proofs (APTP), a verifier-independent proof format for neural network verification, along with APTPchecker, an efficient and formally correct proof checker. By enabling scalable and reliable validation of verification results, the approach aims to increase trust in neural network verification tools, which are essential for deploying DNNs in safety-critical systems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper is well written and clearly describes the proposed approach.\n- The proposed approach addresses proof generation and checking in the context of neural network verification, which has not significantly been addressed in the literature. This is an important consideration in neural network verification, where soundness errors have been demonstrated somewhat (e.g., tools may say a specification holds when it does not). As such, the paper presents contributions to eliminating this issue to ensure verification results hold (e.g., so if a tool says a specification holds, it actually does).\n- The results illustrate the approach with a sufficiently rigorous empirical evaluation, albeit it has some limitations.\n\nWeaknesses:\n\n- The contribution of the work is not clearly articulated, particularly for a machine learning audience. While some aspects of how tools could produce incorrect results is discussed, it could be much more strongly motiviated. For example, with concrete examples. There are other examples in the literature that exist (eg, due to floating point issues and related numerical issues).\n\n- Some aspects of the intuition of the proof certificate and tree (eg Figure 1) do not fully convey the point of the approach, especially for a machine learning audience. While a verification reader can appreciate this, a machine learning one may not, and this example in particular could be better motivated, as it may come across simply as existing known BaB approaches.\n\n- The proposed method essentially re-derives results using MILP and LP techniques that are very similar to those already employed by existing approaches. Could you clarify what your work contributes technically to the verification problem? The LP and MILP formulations you describe are well-known techniques commonly used in established branch-and-bound methods such as alpha-beta-CROWN.\n\n- The number of constraints required to certify the proof appears to grow exponentially with the size of the DNN. Could you explain how you address scalability in this context?\n\n- The procedure you use to certify proofs seems very similar to the verification steps already performed by existing tools. It appears you are repeating these steps without introducing substantial novelty. Could you please elaborate on what you see as the core contribution of your approach?\n\n- The presentation of Figures 3 and 6 can be improved: the legend overlaps much of the core content of the figures, making them difficult to read.\n\nQUESTIONS:\n- I would be interested in seeing a table comparing the runtime required to verify with alpha-beta-CROWN and the runtime your toolbox requires to certify the proof. Could you include such a comparison?\n\n- It would also be helpful to see a table reporting the size and complexity of the DNNs evaluated in your experiments. Please consider including this information.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes the generation of verification proofs by neural networks (which capture the sequence of split decisions and branch evaluations made), and puts forward a MILP-based procedure for validating these proofs. The empirical evaluation of this focuses on performance comparisons with previous work and applicability demonstrations with respect to different neural network verifiers.\n\nSTRENGTHS AND WEAKNESSES:\nWhile the paper makes an important observation concerning the practical unsoundness of theoretically sound verifiers, thereby adequately motivating the need to ensure the correctness of certification results, the framework proposed appears preliminary for NeurIPS. This is particularly the case for evaluation of the method which in my opinion has two main shortcomings.\n\nFirst, it considers only toy models and does not show the scalability of what proposed to the bigger models that neural network verifiers can handle. This is important as models and specifications that require a lot of branching (with branches often efficiently analysed on GPU) may prove problematic for the slower CPU-based MILP solvers. Further, it is not clear that the overall runtime (of verifying and the validating verification proofs) is better than simply verifying the underlying models using efficient MILP-based solvers, such as [1, 2].\n\nSecond, the paper does not make clear that certified verification proofs ensure the correctness of the verification result. In particular, the paper does not only assume that the underlying MILP solver is correct, but also that the MILP-compilation the paper provides is correct. Efficient compilations however crucially depend on tight bounds for the ReLU nodes, which are typically computed by bound propagation schemes, a core component of  neural network verifiers that is susceptible to floating-point errors. Further, the paper discusses only ReLU splitting and does not address input splitting, a branching strategy that is also present in most neural network verifiers. \n\nThird, the paper does not provide any concrete examples of unsound results from neural network verifiers which have been proven unsound by the proposed method.\n\n[1] V. Tjeng, K. Xiao, R. Tedrake. Evaluating robustness of neural networks with mixed integer programming. ICLR19.\n\n[2] P. Kouvaros, A. Lomuscio. Towards Scalable Complete Verification of ReLU Neural Networks via Dependency-based Branching. IJCAI21.\n\nQUESTIONS:\nPlease discuss the three points above pertaining to the evaluation of your method.\n\nLIMITATIONS:\nThe second point described in \"Strengths and Weaknesses\", which is potentially a limitation of the approach, has not been addressed. The potential negative societal impact has been adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}