SoundnessBench: A Soundness Benchmark for Neural Network Verifiers
Authors: Xingjian Zhou
Keyi Shen
Andy Xu
Hongji Xu
Cho-Jui Hsieh
Huan Zhang
Zhouxing Shi
Venue: Accepted by TMLR ()

Abstract: Neural network (NN) verification aims to formally verify properties of NNs, which is crucial for ensuring the behavior of NN-based models in safety-critical applications. In recent years, the community has developed many NN verifiers and benchmarks to evaluate them. However, existing benchmarks typically lack ground-truth for hard instances where no current verifier can verify the property and no counterexample can be found. This makes it difficult to validate the soundness of a  verifier, when it claims verification on such challenging instances that no other verifier can handle. In this work, we develop a new benchmark for NN verification, named "SoundnessBench", specifically for testing the soundness of NN verifiers. SoundnessBench consists of instances with deliberately inserted counterexamples that are hidden from adversarial attacks commonly used to find counterexamples. Thereby, it can identify false verification claims when hidden counterexamples are known to exist. We design a training method to produce NNs with hidden counterexamples and  systematically construct our SoundnessBench with instances across various model architectures, activation functions, and input data. We demonstrate that our training effectively produces hidden counterexamples and our SoundnessBench successfully identifies bugs in state-of-the-art NN verifiers. Our code is available at https://github.com/mvp-harry/SoundnessBench and our dataset is available at https://huggingface.co/datasets/SoundnessBench/SoundnessBench.

Published in Transactions on Machine Learning Research (12/2025)
SoundnessBench: A Soundness Benchmark for Neural Network Verifiers
Xingjian Zhou**
hzhou270@ ucla edu
University of California, Los Angeles
Keyi Shen
kshen84 @egttch...ttcccccco edu
Georgia Institute of Technology
Andy Xu University of California, Los Angeles
mandyy55@gail.ccom com
Hongji Xu Dake University
hz84 @duke. edu
Cho-Jui Hsieh
chohsieh@cs.uche edu
University of California, Los Angeles
Huan Zhang
hwan Muunhhuun--hang.com com
University of Ilimois Urbana- Channa--aappignn
Zhouxing Shi5
shouzing.shi www.inggg.sh@uur...oom edu
University of California, Riverside
Reviewed on OpenReview: https: // opt.eeviee......e..mmmmm net/ forum? id= UVYLL
Abstract
Neural network (NN) verification aims to formally verify properties of NNs, which is cr-- cial for ensuring the behavior of NN-based models in safety-critical applications. In re- cent years, the community has developed many NN verifiers and benchmarks to evalu- ate them. However, cxisting benchmarks typically lack ground-truth for hard instances where no current verifier can verify the property and no counterexample can found. This makes difficult validate the soundness of a verifier, when it claims verifica- tion on such challenging instances that no other verifier can handle. In this work, we develop new benchmark for NN verification, named "SoundnessBenchh specifically for testing the soundness of NN verifiers. SoundnessBench consists of instances with de- liberately inscrted counterexamples that are hidden from adversarial attacks commonly used to find counterexamples. Thereby, it can identify false verification claims when hidden counterexamples are known to exist. We design a training method to produce NNs with hidden counterexamples and systematically construct our SoundnessBench with instances across various model architectures, activation functions, and input data. We demonstrate that our training effectively produces hidden counterexamples and Sound- Besshench successfully identifies bugs in state-of-the-art NN verifiers. Our code is avail- httpa////ooooeaarrrooooooeeeeeee and our datasct is available at able at https://wwnnnnoggeeec///eeeettss
Equal contribution

---

Published in Transactions on Machine Learning Research (12/2025)
1 Introduction
As neural networks (NNs) have demonstrated remarkable capabilities in various applications, formal verifi- cation for NNs has attracted much attention, due to its importance for the trustworthy deployment of NNs with formal guarantees, particularly in safety-critical applications. Typically the goal of NN verification is to formally check if the output by an NN provably satisfies particular properties, for any input within a certain range. In recent years, many softwares specialized for formally verifying NNs, known as NN verifiers, have been developed, such as a,3-CROWN (Zhang et al., 2018; Xu et al., 2020; 2021; Wang et al., 2021; Zhang et al., 2022; Shi et al., 2025), Marabou (Katz et al., 2019; Wu et al., 2024), NeuralSAT (Duong el al., 2024), VeriNet (Henriksen & Lomuscio, 2020), nnenum (Bak, 2021), NNV (Tran et al., 2020), MN-BaB (Ferrari et al., 2022), etc.
To evaluate the performance of various NN verifiers and incentivize new development, many benchmarks have been proposcd. Additionally, there is a series of standardized competitions, International Verification of Neural Networks Competition (VNN-COMP) (Bak et al., 2021; Müller et al., 2022; Brix al., 2023a;b; 2024), that compile diverse benchmarks to evaluate NN verifiers. However, existing benchmarks have a crucial limitation: their instances lack "ground-truth" regarding whether the property in an instance can be verified or falsified. This makes it difficult to determine whether a verification claim is sound when no existing verifier can falsify the properties being checked. For example, in VNN-COMP 2023 (Brix et al., 2023a), a benchmark of Vision Transformers (ViT) (Dosovitskiy et al., 2021) proposed by a participating a.3-CROWN team (Shi et al., 2025) did not contain any instance falsifiable by the participants, while another participating Marabou (Wu et al., 2024) team showed 100% verification on this benchmark, which was later found to be unsound due to issues when leveraging external solver (Brix et al., 2023a). This limitation of existing benchmarks also makes it difficult for developers of NN verifiers to promptly identify bugs, due to the error-prone nature of NN verification.
To address this limitation, we propose a novel approach to test NN verifiers and examine their soundness, by developing a benchmark containing instances with hidden counterezamples predefined counterexamples for the properties being verified, which can be kept away from NN verifier developers. NN verifiers often contain a preliminary check for counterexamples using adversarial attacks (Madry et al., 2018) before starting the more expensive verification process. Therefore, the pre-defined counlerexamples remain hidden against adversarial atlacks, so that the NN verifiers enter the actual verification process. We then check whether the NN verifier falsely claims successful verification when we know a hidden counterexample exists.
aim to climinate any casily found counterexamples and make
We focus on a classification lask using NNs, where the properties being verified concern whether the NN always makes robust predictions given small input perturbations within a pre-defined range. This is a commonly used setting for benchmarking NN verifiers. To build a benchmark with hidden countercxamples, essentially need to train NNs that 1) make wrong predictions on the pre-defined coutterexamples and 2) make correct and robust. predictions on most other inputs so that counterexamples cannot be trivially found by adversarial attacks. To meet both objectives, we propose training NNs with a two-part loss function. The first part aims to achieve misclassification on the pre-defined counterexamples, while the second part is based on adversarial training (Madry et al., 2018) to enforce robustness on other inputs. Since there an inherent conflict between these two objectives, making training particularly challenging, we strengthen the training by designing a margin objective in the first part and incorporating a perturbation sliding window in the second part.
Through this approach, our proposed training framework enables us to build a benchmark, named Sound- nessBench, which incorporates 26 distinct NN models with diverse architectures (CNN, MLP and ViT), activation functions (ReLU, Sigmoid and Tanh), and perturbation radii, designed to comprehensively test the soundness of different verifiers. In these models, we planted a total of 206 hidden counterexamples. Using our SoundnessBench, we successfully identify real bugs in esta-ssta and state-of thett-rrtttttt NN verifiers including a,8-CROWN, Marabou, and NeuralSAT.
2

---

Published in Transactions on Machine Learning Research (12/2025)
2 Background and Related Work
The problem of NN verification. A NN can generally be viewed as function f R + RK that takes d-dimensional inpul E IR and outpuls f(x:) E BRK For instance, if the NN is for a K-way classification task, f(x) denotes the predicted logit score for each of the K classes. NN verification involves a specification on the input and the output. The input can be specified by a region C, and the propertics to verify for the output can be specified using a function s : RK + R that maps the output of the NN to a value, where the desired property is h(x=)= = s (x)) > 0. Formally, the NN verification problem can be defined as verifying:
Vx EC, h(x)) > 0, where h(x) - s(f()))
(1)
Soundness of NN verifiers. An NN verifier is sound if every time the verifier claims that Eq. (1) holds, there exists no counterexample x E C that violates the condition h(x)) > O. Even if an NN verifier is theoretically designed to be sound and rigorous, its implementation may contain bugs or errors that can compromise the soundness in practice. An NN verifier is unsound if the verifier claims that. Eq. (1) holds but a counterexample I E is found to have h(x) S 0 and thus violate the desired property h(x) > 0.
Although adversarial attacks such as projected gradient descent (PGD) (Madry et al., 2018) can often be used to find adversarial examples as counterexamples, in many cases when counterexamples cannot be found, checking the soundness of an NN verifier is challenging. In this work, we address the challenge by building benchmark with pre-defined hidden countercxamples to test the soundness of NN verifiers.
For improving the soundness of NN verifiers, tools like DelBugV (Flsaleh & Katz, 2023) have been proposed for debugging NN verifiers, by shrinking bug-triggering networks and isolating the minimal piecewiss- linear structure responsible for soundness violations. However, it can only be used once concrete test cases triggering bugs have been found. Proof-producing verifiers (Isac et al., 2022) produce independently checkable proofs for each claimed verification verdict. However, they are restricted to a specific base verifier and cannot test the soundness of most other independently developed NN verifiers. In contrast, our work is designed for testing the soundness of arbitrary NN verifiers and triggering new bugs with our test cases. Furthermore, some other works focused on subtle soundness issues in NN verifiers, namely issues from floating point numerical errors (Zombori et al., 2021; Jia & Rinard, 2021; Szász et al., 2025), which are orthogonal to our focus on soundness issues beyond numerical errors.
NN verification benchmarks. Previous works have proposed numerous benchmarks for evaluating NN verifiers. There have been efforts on standardizigg the benchmarking for NN verification (Tacchella et al., 2019; Shriver et al., 2021). Subsequently, multiple itcrations of VNN-COMP have compiled comprchensive benchnarks in the competitions with detailed descriptions of the benchmarks in the competition reports (Bak et al., 2021; Müller et al., 2022; Brix et al., 2023a; 2024). The existing benchmarks cover diverse NN archi- tectures, such as fully-connected NNs, convolutional neural networks (CNNs). Transformers (Vaswani et al., 2017; Dosovitskiy et al., 2021; Shi et al., 2020; 2025), as well as many complicated NN-based models used in applications such as dataset indexing and cardinality prediction (He et al., 2022), power systems (Guha et al., 2019), and controllers (Yang et al., 2024). The existing benchmarks and VNN-COMP commonly rely on cross-validating the results by different verifiers to check the soundness of the verifiers. When only some verifier claims a successful verification on property while no other verifier can falsify the property, the soundness of the verifier cannot be readily tested. In this work, we aim to build novel benchmark con- taining pre-defined counterexamples to test the soundness of NN verifiers, without requiring cross-validation among different verifiers.
Adversarial training. Our work leverages adversarial training (Goodfellow el al., 2015; Madry et al., 2018), to keep our pre-defined counterexamples hidden to adversarial attacks and eliminate trivial coun- terexamples that can be casily found by adversarial attacks. Adversarial training was originally proposed for defending against adversarial attacks and it essentially solves a min-max problem:
min E(
max LcE(fo(æ + 87.1) 1e
(2)
3

---

Published in Transactions on Machine Learning Research (12/2025)
where the model parameters 0 are optimized to minimize the cross entropy loss function LCE, under approximately worst-case perturbation å (|] I <) found by an adversarial attack such as FGSM (Good- fellow et al., 2015) or PGD (Madry et al., 2018). In our method, we incorporate an adversarial training objective to make the NN robust for most inputs except for the pre-defined hidden counterxxamples.
Planting countercxamples. Planting countercxamples (or adversarial examples) has also been studied in Zimmermann et al. (2022) but with a different purpose and method. They aimed to lest adversarial attacks and identify weak attacks and thus weak defense evaluations. Hence they did not consider making planted counterexamples hidden against adversarial attacks. In contrast, we aim to not only plant pre-defined counterexamples, but also make such counterexamples hidden against adversarial attacks, which requires new design for the training to achieve our more challenging objective, and we further discuss methodological differences in Section 3. Additionally, another line of works studied backdoor attacks on NNs (Gu et al., 2017; Liu et al., 2018), which injected wrong behaviors into NNs when a trigger presents in the input, sharing some similarity with our hidden counterexamples. Unlike these works, since our pre-defined counterexamples are for NN verification tested on individual instances, they are generated for each instance, as opposed to a trigger for the entire model in backdoor attacks. Meanwhile, we require the model to be mostly robust on the input region that contains the pre-defined counterexample for each instance, to keep the counterexample hidden against adversarial attacks, which is also new compared to backdoor attacks.
3 Methodology
3.1 Problem Definition
Our problem concerns building a benchmark for NN verification with hidden pre-defined counteccxamplss Our benchmark contains set of instances, where each instance (f, Va, v,+) denotes property of the NN model to be verified.
NN verification for K-way classification. In our work, we focus verification problem in K-way classification that is commonly used to benchmark NN verifiers. The verification problem Eq. (1) is specified as testing the robustness of the model against small perturbations to the input. The input region is defined as a small Eoo ball with a perturbation radius of e around a clean data point Po: C= B(2., <) = (a || - zollx I t), where x - 2g is a small perturbation within the allowed range. The output specification can be defined as s(z) = min(z, - 2, | sisis K, + c} for z = f(x), where y is the ground-truth label for the classification, i + y is any other incorrect class, and s(z) denotes the minimum margin between the ground-truth class and other incorrect classes. The goal of verification is to verify that fa( f((x) > O, for any input within the loo ball B(æo, e) and any other class i, meaning that the classification is always correct and thus robust under any small perturbation with || Zo lolloo S e, as:
Vo E B(xo,)), e), h(z) min f(()) S K, 4 0,
(3)
Unverifiable instances with pre-defined counterexamples. Since we are primarily concerned with testing the soundness of NN verifiers, our benchmark contains unverifiable instances with pre-defined coun- terexamples deliberately planted. Specifically, for an instance (f. 2o. V, c), we aim to plant counterexample #CEX = 2o + OCEX with perturbation dCEx and an incorrect label JCEX (UCEX + v), such that Eq. (3) is
violated:
ECEX = xn + òCEX s.t. ||0cEx S c, fy((EEx) - SyCEx (ICEX) S 0,
(4)
where "CEX" is short for "counterexample. Such an instance allows us to detect bugs in an NN verifier if it claims Eq. (3) can be verified while we know that #CEX satisfying Eq. (4) exists.
Making pre-defined counterexamples hidden. Additionally, we aim to make the planted counterex- ample "hidden" NN verifiers so that NN verifiers cannot casily falsify the unverifiable instances without even entering the actual verification procedure. In particular, we aim to make the counterexample hard to find by adversarial attacks which are often used in NN verifiers as preliminary check before using more
4

---

Published in Transactions on Machine Learning Research (12/2025)
expensive verification procedure. Suppose A 2, is an adversarial attack such PGD (Madry et al., 2018) solving the following problem:
argmin min fy 6) f: + 6);
Se
where the solution returned by the adversarial attack usually does not reach the global minimum. to achieve:
We aim
Vi + y, f20 + A(zo, e, e, ()) (0 + A(xo, y, v.))) > 0, which means the example 2n A v. found by the adversarial attack is not counterexample. If both Eq. (4) and Eq. (6) are achieved, we call ECEX hidden counterexample, and we have ground- truth for the instance (f, Lo Us e), knowing that the instance cannot be verified.
(6)
3.2 Overall Pipeline
Unverifiable Instances
Training two objectives
NN Verifiers
If verifffe::: bug detected!
Objastive pre-deeined CEX
If unknown/falsifie bug not detected.
Robust Model
pec- defined CEX
NN Model
Regular Instances
Objestive
The regular instunces also used in the benchmark but do the purpose bug detection.
Data
Robust Model
Data withor pro- detined CEX
Figure 1: A high-level overview of our SoundnessBench and its purpose. NN models are trained on a synthetic dataset using the two-objective training framework described in Section 3.4, to produce both unverifiable and regular instances. Unverifiable instances are then used to test the soundness of various NN verifiers, while regular instances are also added prevent developers of NN verifiers from directly knowing all the instances are unverifiable. A bug is detected if NN verifier claims succossful verification for any unverifiable instance.
Our benchmark is constructed with a number of different settings. Each setting is identified by: 1) A dataset that provides original examples zn and ground-truth labels y; 2) A neural network architecture for model f: 3) The perturbation size c. For each setting, our pipeline proceeds as follows: For each example from the original dataset, we generate a random target class. We aim to make half of these examples unverifiable with hidden counterexamples (i.e., unverifiable instances), by generating a random pre-defined counterexample The other half are regular examples without pre-defined counterexamples, so that these examples potentially verifiable and NN verifier developers do not simply know all the examples are unverifiable. We then train model with both adversarial training objective and additional objective for producing counterexamples After the training, for the examples with pre-defined counterexamples we filter out those with counterexamples that can be found by a stronger adversarial attack, or where the pre- defined counterexamples do not end up becoming counterexamples. We discuss these stages in more detail below. We now focus on given setting with model f and perturbation size e, and we omit f and e when describing instances below. Figure 1 provides an overview.
3.3
Data generation
For each setting, we construct dataset ID Duvv U Dregular consisting of n unverifiable instances and n regular instances, respectively:
D Duny U Drgularr ((x()) ()),,, (x/²) (2))), ... (x) (2n) (2n))}.
(7)
Dany { (unv.1) (unv.1)), (unv.2) 21),, ... (mn (unv.n))).
(unv.n)
(8)
5

---

Published in Transactions on Machine Learning Research (12/2025)
Dregular {(x)) (regular.)) (1e)
(regular.2) 2) (re (3)
(x (regular,n) y (regular,
(9)
In these instances, Lo and y come from an original datasel, where the first n examples are used to construct Dany while the remaining examples used for Dreuular. For each unverifiable instance (æ) (unv.d) y(unx.)) E y
(10) Bune (1 1 I n), we generate a pre-defined counterexample (#CEX) UCEx), where CEX = #0 (unv.s))
(i) & CEX: such that & is the perturbation and #сEX is the target label. 8 OCx is randomly generated from -r-r-r- e, e) (0 50 S < 1) satisfying r S || S e, and MCEX is randomly generated from Giisis K.jj + y (um)..)) satisfying OCEX # (u Here, since we aim to cnsure To z (unv.)) and #CEx have different labels, we make them separated by setting ||o 2 r-e with a hyperparameter (0 S 1) for the minimum distance, where r is set to value close to 1. This is similar to how Zimmermann ct al. (2022) controlled the distance and generated perturbed examples near the boundary of the B(xo,e) perturbation ball when creating their dataset for planting counterexamples.
3.4 Training
Our training comprises two objectives. The first part aims to make our pre-defined counterexamples become real counterexamples, to satisfy Eq. (4). The second part aims to eliminate counterexamples that can be found by adversarial attacks to satisfy Eq. (6) for both unverifiable instances and regular instances. We explain these two objectives below:
Objective for the pre-defined countercxamples. In the first objective regarding Eq. (4), for each unverifiable instance (x) (a)umss)) (uune.)) (1 S I n) with the pre-defined countercxample (#CEx.BEEX) we aim to make the class predicted from output logits f( match target label VCEX: i.e., (2)
arg max fy (0) = VCEX. This can be achieved by a standard cross-entropy (CE) loss. However, it tends to maximize f ao (#(Ex) and minimize fo (xCEx) without a cap on the margin between them.
Vonx
We find that making f 00 (#CEx) substantially greater than fyrm... (FCEX) tends to make the counterex- amples more obvious to adversarial attacks and thus makes our training with the second objective difficult.
(i)
MCex
Therefore, we design a margin objective instead achieve f.() (#CEX) maximum desired margin X:
(4)
fy (I ( S A with a
Vewx
LCEX
max f your, (xx
f Sg00
(o (x())) +)}.
(10)
n1
9CEX
This is different from Zimmermann et al. (2022) as they only trained a linear classification head to separate the frozen features of perturbed examples inside the perturbation ball and on the boundary of the perturbation ball, thereby making the examples on the boundary become counterexamples. In contrast, we train the whole model, as we additionally have more challenging objective of making the counterexamples hidden against adversarial attacks (achieved via the adversarial training objective described below), and we achieve the objective for the pre-defined counterexamples here with margin objective.
Objective with adversarial training. Our second objective is adversarial training as defined in Eq. (2), applied to the entire dataset. ID with both unverifiable instances and regular instances. However, since by default new perturbation 8(1) (1 S i S 2n) is generated at each training epoch for every example, we find that the training struggles to converge, if we only use the new perturbation. This is because training on a new perturbation can oflen break the model on other perturbations including those used in previous epochs, when we have a conflicting objective for pre-defined counterexamples. To mitigate this issue, we propose a perturbation sliding window, where we use multiple perturbations generated from the most recent epochs. For each instance (x) ()) we maintain a window w/( of perturbations generated in the most recent w epochs: W( (o()) (0 ..... ((()) where w is the maximum length of the sliding window (there can be fewer than w elements in w) in the first w epochs). Using this window, our objective that utilizes all
6

---

Published in Transactions on Machine Learning Research (12/2025)
the perturbations in the sliding window is:
Ladv Ladv, window
Σ KEW)))
(c + +0),
(11)
W
15i52n
This differs significantly from Zimmermann et al. (2022) which did not have adversarial training to make counterexamples hidden.
Overall objective. Suppose model f is parameterized by parameters 0, denoted as fo, and our two loss functions both depend on 0, denoted as (C and (O). Then our overall loss function combines the two objectives
(()) = (CEx + (0),
(12)
and we optimize model parameters by solving arg ming L(0) using gradient-based optimizer. The complete training algorithm is shown in Algorithm in Appendix A.
3.5 Benchmark Design
Table 1: List of model architectures included in our benchmark. "Conv & x w x h² represents a 2D convo- lutional layer with & filters of kernel size h, "FC w" represents a fully-connected layer with w hidden neurons, "AvgPool &x x k" represents 2D average pooling layer with kernel size k x k. An activation function is added after every convolutional layer or fully-connocted layer.
Name
Model Architecture
Activation Function
CNN Couv
Conv x x FC 1000, FC 100, FC 20, FC 2 Conv 5 x 3, Conv x x 3, FC 1000, FC 100, FC 20, FC 2 Conv x 3 x Cony 10 x 3 x 3, Conv 20 3 x FC 1000, FC 100, FC 20, FC
ReLU
CNN Conv
ReLU
CNN Conv
ReLUU
CNN AvgPool
Conv 10 x 3 x AngPool 3 x FC 1000, FC 100, FC 20, FC
ReLU
MLP Midden
100, FC 1000, FC 1000, FC 1000, FC 20, 2
ReLU
MLP Hidden
FC 100, 1000, FC 1000, FC 1000, FC 1000, FC 20, FC
ReLU
CNN Tanh
Conv x 3, FC 1000, FC 100, FC 20, FC Conv Modified VIT (Shi al., 2025) with patch size 1, actention heads and embedding size
Tanh
CNN Sigmoid
3, 1000, 100, FC 20, FC
Signoid
ViT
ReLU
Scale. We focus on using relatively small models and low-dimensional synthetic data. Since the goal of our benchmark is to test the soundness of NN verifiers rather than benchmark performance, our models and data do not have to be large-scale or as practical as real-world ones. We may draw an analogy to fuzzing techniques in software engineering, where simple and synthetic but specifically constructed inputs, rather than complicated real-world inputs, are used for testing software and identifying bugs. Additionally, large-scale models and data can significantly increase the difficulty for NN verifiers, and NN verifiers fail to provide meaningful verification results simply fail to support complicated cases, without exposing bugs. Verifiers typically require relaxation, and on relatively larger settings, looser relaxation of other correct layers can compensate for the wrong bound computation, making the final verification more conservative. Although NN verifiers can often verify much larger models in other benchmarks, they depend on models being specifically trained, and our models trained with hidden counterexamples are more challenging to verify. As shown in Table 6, models and data at our small scale are already difficult for current NN verifiers. Therefore, by using relatively small models and data, we allow more verifiers to be tested against our benchmark to maximize bug exposure. Nevertheless, also show results on MNIST in Appendix E, where verification is much more difficult.
Model. As shown in Table 1, we build our benchmark to contain diverse NN architectures and aclivation functions. We incorporate 9 distinct NN architectures, spanning MLPs (NNs with fully-connected layers), CNNs, and ViTs, with different activation functions including ReLU, Tanh, and Sigmoid. The models are designed to have moderate size for existing NN verifiers to handle, and in total, there are 26 models in our benchmark.
7

---

Published in Transactions on Machine Learning Research (12/2025)
y()), (abb) , ,,).).... (.0""""""..) (n) : ()))) is synthetic and randomly (0) is sampled from [-1, and y(u) is sampled from {0, for 1 S i Σ 2n. We use various imput sizes and perturbation radii, as shown in Table 2. Specifically, for MLP models, we use 1D data with input size 10; for other models, we use 2D data with input sizes d = 1 x and d = 3 x 5 5, respectively, where and are the numbers of input channels considered by CNNs and ViTs. For the perturbation radii, we use e 0.2 and e 0.5. For each model with a particular perturbation radius and imput size, we set = 10 as the number of instances and generate 10 unverifiable instances (a)) ( ((um..))) Duv (1 s n),, each with a corresponding pre-defined counterexample (# (CEX). and 10 regular instances (x)) (regular.)) y (y)reuular:)) E Bregular (1 i)... We set 0.98 for || Iloo 2 r,, introduced in Section 3.3, so that each pre-defined counterexample is close to the boundary of the imput perturbation ball. During the data generation, to ensure that different examples are sufficiently separated, we require that there is no intersection between the perturbation balls of different examples and retry the generation until no intersection exists. After the training, for the unverifiable instances, we only keep those with hidden counterexamples that are not found by strong adversarial attacks conducted in Section 4.2, Thus, there can be fewer than 10 unverifiable instances if any pre-defined counterexample is found by an adversarial attack.
Data. Our dataset of original examples (#o (1)
generated following uniform distributions, where zy
We compare settings with existing benchmarks in Appendix B.
4 Experiments
In this section, we conduct comprehensive experiments. We first build our SoundnessBench by our proposed method with implementation details provided in Section 4.1, we evaluate the effectiveness of our training in Section 4.2. We then use benchmark to test the soundness of several maate-of-theart NN verifiers in Section 4.3. Additionally, in Section 4.4, we conduct an ablation study to justify the design choices for our training. We also provide additional results in the appendix about testing NN verifiers with synthetic bugs (Appendix C), repeating our experiments with multiple random seeds (Appendix D), and experimenting on MNIST (Appendix E).
4.1
Implementation Details
For every model in the benchmark, we apply our aforementioned training framework with the following hyperparameters:: (1) Number of training epochs is set to 5000; (2) Adam optimizer (Kingma & Ba, 2015) with a eyclic learning rate schedule is used, where the learning rate gradually increases from 0 to a peak value during the first half of the training, and then it gradually decreases to 0 during the second half of the training, with the peak learning rale set to 0.001; (3) The threshold in the margin objective defined in Eq. (10) is set x 0.01; (4) The size of the perturbation sliding window used in Eq. (11) is set to w = 300; (5) We use the PGD attack (Madry et al., 2018) for the adversarial training with up 150 restarts (number of different random initialization for the PGD attack) and 150 attack steps.
4.2 Training NNs and Building the Benchmark
Evaluation method. After the models are trained, we first evaluate we have successfully planted hidden couttecxxmmless For each unverifiable instance (zz) (unv,)) (unvv.)) (1 < i S n), we evaluate it in three steps:
1. We check whether the model predicts the correct class y (unv,,) for unperturbed input ab (unv,
2. We check whether the model predicts the expected class yOEx # (unv..) for the pre-defined coun- terexample a <
3. We check whether no counterexample can be found by adversarial attacks in the region B(a)) (unv.i) (c.
(i)
ie. #CEx is hidden counterexample.
For the regular instances (
(regular.)) y (regular,)) (1 < 1 S n), we evaluate them in two steps:
8

---

Published in Transactions on Machine Learning Research (12/2025)
Table 2: Evaluation results of trained models. Our goal to achieve high "hidden CEX" (up to 10) for unverifiable instances and high "No CEX found" for regular instances.
Training Sottings
Results for Uuverifiable Instances
Results for Regular Instances
Name
Input size
Correct mi
(unva)
Incorrect CFx
Hidden CEX
Correct Zg
(regulana)
No CEX found
CNN 1 Conv
0.2
x 5 5 10
10
9
10
10
CNN Conv
0.2
3
10
10
9
10
10
CNN Conv
0.5
1x5x
10
10
CNN 1 Conv
0.5
3x5x x
10
10
8
10
10
CNN Conv
0.2
x 1x5x5
10
7
10
CNN 2 Conv
0.2
x xx555
10
7
10
10
CNN 2 Conv
0.5
x 5
10
10
10
10
10
CNN 2 Conv
0.5
3 x
10
10
10
10
10
CNN 3 Conv
0.2
1x5x5
10
10
10
10
10
CNN 3 Conv
0.2
3x5x5
10
10
7
10
10
CNN 3 Conv
0.5
1x5x5
10
10
9
10
10
CNN 3 Conv
0.5
3
10
10
10
10
CNN AvgPool
0.2
1 1
10
0
10
10
CNN AvgPool
0.2
3 5
10
CNN AvgPool
0.5
1 5
10
9
10
10
CNN AvgPool
0.5
3 5
10
10
10
10
10
4 Hidden
10
10
9
MLP 4 Hidden
0.5
10
10
10
9
10
10
MLP 5 Hidden
0.2
10
10
10
4
10
10
MLP 5 Hidden
0.5
10
10
9
10
CNN Tanh
0.2
x 5
10
10
8
10
10
CNN Tanh
0.2
3 5
10
10
10
10
10
CNN Sigmoid
0.2
5
10
CNN Sigmoid
0.2
3 x 5x5xx
10
10
10
10
VIT
0.2
x 5
10
10
10
10
10
VIT
0.2
3x5x6
10
10
10
10
1. We check whether the model predicts the correct class (regullar.) for unperturbed input z (regular.)), We chock whether no countercxample can be found by adversarial attacks in the region B(an (regular,5) ,e)
To make sure that our pre-defined counterexamples are truly hidden against adversarial attacks that try to find counterexamples, we evaluate them using strong adversarial attacks. Specifically, we apply PGD allack with strong altack parameters, 1000 restarts and 5000 PGD steps, and also AutoAtaack (Croce Hein, 2020) which combines multiple adversarial attack strategies. For the pre-defined counterexamples, only those passing all of these attacks considered to be hidden countercxamples.
Results. Table presents the evaluation results. For most unverifiable instances, the model makes correct prediction the unpertureed input while making an incorrect prediction the pre-defined counterexample expected. When we require the pre-defined counterexamples to be hidden against adversarial attacks, large number of hidden counterexamples are produced, and notably, 22 of the 26 models have at least 7 hidden counterexamples. These results demonstrate the effectiveness of our method in producing hidden comteeexxmmles that we use to test the soundness of verifiers. Meanwhile, for regular instances, all the models achieve correct predictions on the unperturbed input and no counterexample found. This shows that the modcls have robust performance on the instances where pro-defined countercxamples are not added.
Location of countercxamples. To better understand why our countercxamples are difficult find, we analyze the location of the counterexamples with respect the decision boundary. For each pre-defined counterexample, using PGD attack, approximately find the closest point in the input space that would

---

Published in Transactions on Machine Learning Research (12/2025)
make the model produce the correct prediction, which measures how close the counterexample is to the decision boundary. We conduct the analysis on two models, CNN AvgPool (e = 0.5 and input size x 5 5) and ViT (e = 0.2 and input size 3 x 5 x 5). The mean lo distances to the decision boundary are 2.6 x 10 for the CNN AvgPool model and 7.3 x 10-4 for the ViT model. The counterexamples tend to be very close to the decision boundary, thus making them hard to find by adversarial attacks.
4.3 Testing NN Verifiers
Table 3: Abbreviation of different versions of verifiers.
Verifier
Abbreviation
Description
a,f-CROWN
ABC-A
a,8-CROWN using activation splitting
a.B-CROWN
ABC-I
a,9-CROWN using input splitting
NeuralSAT
NSAT-A
NeuralSAT using activation splitting
NeuralSAT
NSAT-1
NeuralSAT using input splitting
PyRAT
PyRAT
Default PyRAT
Marabou 2023
MB-23
Marabou used in VNN-COMP 2023 (branch vnn van-comp-22
Marabou 2024
MB-24
Marabou used in VNN-COMP 2024 (branch vnn-comp-24)
We test the soundness of four representative NN verifiers, including a, -.3-CROWN' (Zhang et al., 2018: Xu et al., 2020; 2021; Wang al., 2021; Zhang et al., 2022; Shi al., 2025), NeuralSAT (Duong et al., 2024), PyRAT (Lemesle et al., 2024), and Marabou (Katz et al., 2019; Wu et al., 2024). For a,3-CROWN and NeuralSAT, consider two distinct splitting strategies in their wranch-and-bous activation splitting and input splitting, which conduct branch-and-bound on the activation space and input space, respectively. For Marabou, we include two versions: Marabou 20234 and Marabou 20245, which were used in VNN-COMP 2023 and VNN-COMP 2024, respectively. We include Marabou 2023 because it supported VIT models, while Marabou 2024 does not. To streamline our experiment tables, we define abbreviations for different versions of verifiers, listed in Table 3. A timcout of 100 seconds is applied on all the verification experiments, similar lo what many existing benchmarks use (Bak el al., 2021; Müller et al., 2022; Brix et al., 2023a; 2024). If a verifier fails to output result for an instance within the time limit, the instance is claimed as neither verified nor falsified.
for the verification,
Main findings. The results are presented in Tables to 6, with the following findings:
1. Our benchmark successfully identified bugs in three well-established verifiers: including a,8-CROWN, AeuralSAT, and Marabou 2023. Table 4 shows the proportion of "verified" instances claimed by the verifiers on unverifiable instances, mcasuring the unsoundness of verifiers. For the entries with nonzero values, the results show that these verifiers' claims are unsound and the verifiers contain bugs in these cases.
Note that it is reasonable that no bug is found on most other models, because these well-developed verifiers are sound most cases. Nevertheless, our benchmark enables researchers or developers of NN verifiers to sanity check their work in future development. When they make significant changes existing verifiers or develop new verifiers that can support new settings, which is often error-prone, they can our SoundnessBench. Alternatively, they can use our method to train new models with pre-defined counterexamples. SoundnessBench will help improve the quality and reliability of verifiers built in the community, and broadly impact safety-critical domains utilizing NN verifiers. 2. Hidden countorexampler in our benchmark are difficult for existing verifiers to find. Table 5 shows the proportion of unverifiable instances falsified by NN verifiers. The results show that most unverifiable instances are not falsified by the verifiers, and thus most our hidden counterexamples successfully evade the falsification mechanisms NN verifiers and remain hidden.
Phttps://github.com http:://hhttlooooo..././///tstts htte:///a...aaaaooooo//r//////aa
10

---

Published in Transactions on Machine Learning Research (12/2025)
Table 4: Proportion of "verified" unverifiable instances. is a measurement of unsoundness unsoundness of verifiers. Any verified instance indicates bug in the NN verifier. 4_" indicates the model is not supported by the verifier. The number unverifiable instances used in each setting corresponds to "hidden CEX" in Table 2 (only unverifiable instances with hidden countercxamples not found by adversarial attacks are kept, mentioned in Section 4.2).
Model
Verified (%)
Name
Input size
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT MB-23
MB-24
CNN AvgPool
0.2
1 5 x
0
0
0
CNN AvgPool
0.2
3 x
100
100
100
CNN AvgPool
0.3
1x5x5
100
66.7
100
33.3
CNN AvgPool
0.5
3x5x5
0
0
0
CNN Tanh
0.2
1x5x5 5
0
0
CNN Tanh
0.2
0
0
100
0
CNN Sigmoid
0.2
1x5x5
0
0
CNN Sigmoid
0.2
3
0
0
0
VIT
0.2
1x5x5
0
90
VII
0.2
3
0
100
Results on other models: For "CNN 1 Conv", "CNN 2 Conv", "CNN Conv" and the MLP models, no verifier claimed any unverifiable instance as verified. We thus do not create individual rows for these models (their results would have been all 0).
Table 5: Proportion falsified unverifiable instances. For every model architecture, we average the results on different e values and input sizes.
Model Architecture
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT
MB-23
MB-24
CNN 1 Conv
0
0
0
0
0
2.8
CNN Conv
0
0
0
CNN Conv
2.5
CNN AvgPool
17.5
7,5
10.0
25.0
25.0
CNN Tanh
0
0
0
0
0
CNN Sigmoid
65.0
55.0
55.0
50.0
VIT
0
0
0
MLP Hidden
5.5
0
11.1
11.1
0
MLP Hidden
12.5
25.0
12.5
25.0
3. Our benchmark includes many challenging regular instances. As shown in Table results vary significantly across verifiers and models, and in most scttings, the proportion of verified instances is not high. This demonstrates that our regular instances also reveal performance differences and limitations among different NN verifiers, as side benefit. Given that these NN verifiers already cxhibit limited performance on our relatively small-scale models and data built with pre-defined counterexampees, we have focused on this scale in this paper, as mentioned in Section 3.5.
Analysis of discovered bugs. For the bugs on CNN AvgPool models, triggered for a,8-CROWN and NeuralSAT, find that they are rooted in the auto LiRPA package (Xu al., 2020) leveraged by both a.3-CROWN and NeuralSAT in their bound computation. Specifically, find that auto LiRPA previously assumed that "strides" and "kernel size" are equal in AvgPool layers and silently produced unsound results when their assumption docs not hold in our models. Counterexamples provided by SoundnessBench success- fully led to the discovery of the bug in auto LiRPA. We shared our test cases that helped the developers of auto LiRPA fix the bug. The fix has been reflected in the auto LiRPA package and adopted by both R.G-CROWN and NeuralSAT their latest version. For the bugs of Marabou on Tanh and VIT, the authors of Marabou conjectured that there might be bugs in the interaction with the quadratic programming engine in Gurobi, but the specific cause remains unknown.
11

---

Published in Transactions on Machine Learning Research (12/2025)
Table 6: Proportion of verified regular instances. For every model architecture, we average the results on
different e values and input sizes.
Model Architecture
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT
MB-23
MB-2 24
CNN 1 Conv
45.0
35.0
47.5
35.0
25.0
7.5
45.0
CNN 2 Conv
32.5
27.5
32.5
27.5
12.5
5.0
22.5
CNN Conv
17.5
20.0
20.0
12.5
CNN AvgPool
75.0"
70.07
75.01
70.0
0
CNN Tanh
0
0
50.0f
CNN Sigmoid
10.0
10.0
10.0
10.0
10.0
10.0
VIT
0
100.0f
MLP 4 Hidden
55.0
75.0
55.0
85.0
45.0
0
45.0
MLP Hidden
45.0
65.0
50.0
75.0
15.0
20.0
15.0
DDenotes verification results that are likely unsound given bugs identified on unverifiable instances in Table 4.
Table 7:
Average number of hidden counterexamples (up 10) for the ablation study, evaluated across 5
different random seeds.
Training Settings
Margin objective
Percurbation window size
Model
Input size
Disabled
Enabled
10
100
300
CNN Conv
0.2
1 1 5 5
1.2
6.2
0
4.0
6.2
CNN
Conv
0.2
3 5 x
1.2
10.0
0
8.0
10.0
CNN Conv
1x5x5
0.4
7.4
0
3.6
7.4
CNN
Conv
0.5
3
0
5.8
0
6.6
5.8
4.4 Ablation study
In this section, we conduct ablation study evaluating the individual contributions of the two techniques proposed in Section 3.4, margón objective and perturbation sliding window. For this study, use the CNN Conv architecture with perturbation radii E {0.2, 0.5} and input sizes 1 x 5 x 5 and 3 5 x 5. For each experiment, we vary the setting of the ablation subject (i.e., whether the margin objective is enabled and the maximum window size, respectively), and we train 5 models with different random seeds for each setting. We evaluate the average number of hidden counterexamples produced in each setting.
Table shows the results. Models trained with the margin objective consistently outperform their coun- terparts trained without the margin objective, demonstrating the effectiveness of the margin objective in producing many more hidden counterexamples. When the window size is set to 1, the training is equivalent to the original adversarial training, and no hidden counterexample can be produced. The best results are achieved by setting the window size to 100 in one case and to 300 in the other three cases. This demonstrates the offectiveness of our perturbation sliding window.
5 Conclusion and Discussion
We present BoundnessBench, benchmark with hidden counterexamples for testing the soundness of NN verifiers. Through two-objective training framework with two training techniques, margin objective and perturbation sliding window, have created benchmark incorporating 26 models across nine distinct NN architectures, 206 unverifiable instances with hidden counterexamples, and 260 regular instances. No- lably, BoundnessBench revealed bugs in three well-established NN verifiers: a-B-CROWN, NeuralSAT, and Marabou 2023. Our benchmark has helped the authors auto_ LiRPAA which is used in both R.,--RRÓWN and NcuralSAT, fix the bug we identified. We believe SoundnessBench and our proposed methodology can serve as valuable resource for developers building more reliable NN verifiers and more trustworthy artificial intelligence, thus broadly benefiting -ciital domains that utilize NN verifiers in the future.
12

---

Published in Transactions on Machine Learning Research (12/2025)
Limitations. This work has several limitations. First, our models and data relatively small-scale, discussed in Section 3.5, but they remain useful for testing the soundness of NN verifiers as our results demonstrate. Second, we mainly use PGD and AutoAttack, which are among the strongest and most commonly used attacks both in verifiers (such as a,3-CROWN) and in the literature. While PGD and AuloAttack canmot cover new allacks that may appear in the future, is only feasible for us to adopt currently existing attacks, and future work may consider new attacks as we discuss below.
Benchmark growth and maintenance. Our benchmark can be extended to consider new settings and attacks that may emerge. We suggest closely monitoring the development of various verifiers and performance benchmarks to maintain our benchmark: (1) Results should be updated new releases of verifiers appear; (2) If a new attack is introduced in a verifier, we need to test the resilience of our currently hidden counterexamples against such attacks. Tf our counterexamples are no longer hidden, we should adapt our training to incorporate such stronger attacks by replacing the PGD attack with the new attack algorithm, as the allack component in our training pipeline is independent of the other components; (3) a new model architecture or data setting becomes popular in the NN verification community and is supported by multiple verifiers, can extend our benchmark to support such scttings. We welcome contributions from the community to help maintain and expand SoundnessBench.
Incrcasing the soundness and reliability of NN verifiers. Beyond the development of novel and advanced NN verification algorithms, the correct implementation of algorithms is critical to maintaining the soundness of verifiers, which can be challenging to validate systematically. Most NN verification papers and VNN-COMP primarily focus on the performance of NN verifiers, while the reliability of NN verifiers is currently understudied. We thus encourage the community dedicate more effort testing the soundness of NN verifiers. Specifically, recommend that researchers: (1) stress-test benchmarks and tools when new algorithms implementation modifications are introduced, and develop new soundness benchmarks simultancously as new settings are studied; (2) incorporate mcasures and results of soundness testing into research papers and technical reports; and (3) document. the assumptions of algo- rithms and implementations and properly handle unsupported cases. By prioritizing both performance and soundness, the community can build more capable and reliable NN verifiers and verified NN-based models for various caietalcritical applications.
NN verifiers with soundness
Acknowledgments
This project is supported in part by NSF 2048280, 2331966 and ONR N00014-28-1-33000P0000 Huan Zhang is supported in part by the A12050 program at Schmidt Sciences (A12050 Early Career Fellowship) and NSF (TI5-2331967.
References
Stanley Bak. nnenum: Verification of relu neural networks with optimized abstraction refinement. In NASA Formal Methods Symposium, pp. 19 19-36, 2021.
Stanley Bak, Changliu Liu, and Taylor Johnson. The second international verification of neural networks competition (ynn-comp 2021): Summary and results. arXiv preprint arXio:2109.004: 2021.
Christopher Brix, Stanley Bak, Changliu Liu, and Taylor T. Johnson. The fourth international verification of neural networks competition (ynn-comp 2023): Summary and results. arXiv preprint arXiv:2312.1677 2023a.
Christopher Brix, Mark Niklas Müller, Stanley Bak, Taylor T Johnson, and Changliu Liu. First three years of the international verification of neural networks competition (ynn-comp). International Journal Software Tools for Technology Transfer, 25(3):329 339, 2023b.
Christopher Brix, Stanley Bak, Taylor Johnson, and Haoze Wu. The fifth international verification of neural networks competition (ynn-comp 2024): Summary and results. arXiv preprint arXiv:2/12. 199885 2024.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, volume 119, pp. 2206-2216, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostala Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, An image is worth 16x16 words: Transformers for image recognition scale. In International Conference Learning Representations, 2021.
Hai Duong, Dong Xu, ThanhVu Nguyen, and Matthew B Dwyer. Harnessing neuron stability to improve dnn verification. Proceedings of the ACM on Software Engineering, 1(PSE):859 881, 2024.
Raya. Elsaleh and Guy Katz. Delbugy: Della-debugging neural network verifiers. In 2023 Formal Methods in Computer-Aided Design (FMCAD), pp- 34-43. IEFE, 2023.
Claudio Ferrari, Mark Niklas Müller, Nikola Jovanovic, and Martin T. Vechev. Complete verification via multi-ncuron relaxation guided branch-and-bould In International Conference on Learning Representa- lions, 2022.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:170s (tf:::770806775 06780, 2017.
Neel Guha, Zhecheng Wang, Matt Wylock, and Arun Majumdar. Machine learning for ac optimal power flow. Xrviv preprint xrXie:1910.088/2 2019.
Haoyu He, Tianhao Wei, Huan Zhang, Changliu Liu, and Cheng Tan. Characterizing neural network verifi- cation for systems with undsysbench. In Workshop Formal Verification of Machine Learning, 2022.
Patrick Henriksen and Alessio Lomuscio. Efficient neural network verification via adaptive refinement and adversarial search. In ECAI 2020, pp. 2513 2513-2520, 2020.
Omri Isac, Clark Barrett, Min Zhang, and Guy Katz. Neural network verification with proof production. In PROCEEDINGS OF THE 22ND CONFERENCE ON FORMAL METHODS IN CAMPUTER-AIDER) DESIGN-FMCAD 2022, volume 3, pp. 38. TU Wien Academic Press, 2022.
Kai Jia and Martin Rinard. Exploiting verified neural networks via floating point numerical error. In International Static Analysis Symposium, pp.
Springer, 2021.
Guy Katz, Derek A Huang, Duligur Theling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, et al. The marabou framework for verification and analysis of deep neural networks. In Computer Aided Verification: 31st International Conference, CAV 2019, New York Cily, NY, USA, July 15-18, 2019, Proceedings, Part 31, pp. 443 452, 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Confer- ence on Learning Representations, 2015.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]] Available: hllp://yann..eent www.wewwww...eeee..ooeo com/exdb////eeeeoocoomcmoooo.. /anab//nnni,,, 2, 2010.
Augustin Lemesle, Julien Lehmann, and Tristan Le Gall. Neural network verification with pyral. arXiv preprint arXi::2410.2900 23900, 2024.
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuas Loe, Juan Zhai, Weihang Wang, and Xiangyu Zhang, Trojaning attack on neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018). Internet. Soc, 2018.
14

---

Published in Transactions on Machine Learning Research (12/2025)
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Represenlations. 2018.
Mark Niklas Müller, Christopher Brix, Stanley Bak, Changliu Liu, and Taylor T. Johnson. The third international verification of neural networks competition (vnn-comp 2022): Summary and results. arXiv preprint arXit:2212.10877 2022.
Zhouxing Shi, Huan Zhang, Kai- Wei Chang, Minlie Huang, and Cho-Jui Hsieh. Robustness verification for transformers. In International Conference on Learning Representations, 2020.
Zhouxing Shi, Qirui Jin, Zico Kolter, Suman Jana, Cho-Jui Hsich, and Huan Zhang. Neural network verification with branch-and-bound for general nonlinearities. In Tools and Algorithoss for the Construction 17.
and Analysis of Systems, volume 15696, pp. 315-335, 2025. doi:
David Shriver, Sebastian Elbaum, and Matthew B Dwyer. Dnnv: A framework for deep neural network verification. In International Conference on Computer Aided Verificalion, pp. 137-150. Springer, 2021.
Attila Szász, Balázs Bánhelyi, and Márk Jelasity. No soundness in the real world: On the challenges of the verification of deployed neural networks. arXiv preprint Kkt:22506.0005 2025.
Armando Tacchella, Luca Pulina, Dario Guidotti, and Stefano Demarchi. The verification of neural networks library (ynn-lib). https://wwes wwwlib.nnlibb.rr...... 2019.
Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez, Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, and Taylor Johnson. Nnv: the neural network verification tool for deep neural networks and learning-enablee syberaphyysccal systems. In International Conference on Computer Aided Verification, pp. 3 17, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you nced. In Advances in Neural Information Processing Systems 30: Annual Conference on Newral Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 5998 6008, 2017.
Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsich, and J. Zico Kolter. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. In Advances in Neural Information Processing Systems, pp. 29909-29092 -99211, 2021.
Haoze Wu, Omri Isac, Aleksandar Zeljić, Teruhiro Tagomori, Matthew Daggitt, Wen Kokke, Idan Refaeli, Guy Amir, Kyle Julian, Shahaf Bassan, et al. Marabou 2.0: a versatile formal analyzer of neural networks. In International Conference on Computer Aided Verification, pp. 249-264, 2024.
Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai- Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsich. Automatic perturbation analysis for scalable certified robustness and beyond. In Advances in Neural Information Processing Systems, 2020.
Kaidi Xu, Huan Zhang, Shiqi Wang, Yihan Wang, Sunan Jana, Xue Lin, and Cho-Jui Hsieh. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. In International Conference on Learning Representations, 2021.
Lujie Yang, Hongkai Dai, Zhouxing Shi, Cho-Jui Hsieh, Russ Tedrake, and Huan Zhang. Lyapunov-stable neural control for state and output feedback: A novel formulation. In International Conference on Machine Learning, 2024.
Huan Zhang, Tei Weng, Pin-Yu Yu Chen, Cho-Jui Hsich, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Advances in Neural Information Processing Systems, pp. 4944-4953, 2018.
15

---

Published in Transactions on Machine Learning Research (12/2025)
Huan Zhang, Shiqi Wang, Kaidi Xu, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsich, and J. Zico Kolter. General cutting planes for bound-propaggation-ba neural network verification. In Advances in Neural Information Processing Systems, 2022.
Roland S. Zimmermann, Wieland Brendel, Florian Tramer, and Nicholas Carlini. Increasing confidence in
adversarial robustncss ovaluations. In Advances in Neural Information Processing Systems, 2022.
Dániel Zombori, Balázs Bánhelyi, Tibor Csendes, István Megyeri, and Márk Jelasity. Fooling a complete
neural network verifier. 2021.
16

---

Published in Transactions on Machine Learning Research (12/2025)
A Illustration for the Training Algorithm
In Algorithm 1, we illustrate our algorithm for training models with hidden counterexamples, proposed in Section 3.4.
Algorithm Training Models with Hidden Counterexamples
Inputs: Dataset {(x0)) (+) ()()) initialized model fe: attack method A (i.e., PGD); margin cap A; perturbation sliding window size wy epochs T; optimizer Opt.
Output: Trained model fo
1: // Initialize perturbation sliding windows
2: for = 1...., 2n do
3: w(() +-
end for
// Main bruining loop
for = 1.2..... T do
// Generate new adversarial perturbations and update perturbation sliding windows
8: for = =,,,... 1,.... 2n do
g(6) + A(fo. 2o (0, c) W(1) + (W() U {8(1)) and truncate to keep the most recent elements
(2)
10:
11: end for
12: // Margin objective for pre-defined counterezamples (Eq. (10))
13:
LCEX
Σ max ylunns.) (x (xr (unv,))) f (FCEX) + X)}
n
14:
// Adversarial training with perturbation sliding windows (Eq. (11))
15:
fadv 2n
Σ [W())
Σ Cce(f(=) (6) + +).
1si52n
16:
// Updale the model fo with a combined objective (Eq. (12))
17:
((0) = (O) (O)
18:
0 Opt. (0, VoL(0)
19: end for
20: return Trained model fo
B Comparison between Benchmarks
In Table 8, we compare our benchmark with existing ones compiled in VNN-COMP 2024 (Brix et al., 2024). Our benchmark serves a unique purpose for testing the soundness of NN verifiers while others all focus on performance. Meanwhile, our benchmark covers diverse model architectures, while most existing benchmarks focus on one or two model architectures. Additionally, while we focus on relatively small models and data as discussed in Section 3.5, our scale still larger than many of the existing benchmarks.
C Synthetic Bugs
To further test our soundness benchmark, we introduce synthetic bugs into a,3-CROWN and NeuralSAT using shrunk input intermediate bounds and randomly dropped domains:
1. Shrunk input & intermediate bounds:
We modify the verifiers and deliberately shrink input and intermediate verified bounds in an unsound way, by factor denoted as a (0 < a 5 1). The input bounds correspond to the perturbation radius e, and we reduce e into (1 (1-a).e. Intermediate bounds are output bounds of intermodiate layers as required by the verifiers to perform lincar relaxation LiRPA. Suppose [hg. hu] denotes the lower and upper bound, respectively, of an intermediate layer k. and by (2r) is the output of layer & when unperturbed input is given. We then shrink the
17

---

Published in Transactions on Machine Learning Research (12/2025)
Table 8: Comparison between different benchmarks. Except for our benchmark, others are cited from VNN- COMP 2024 (Brix et al., 2024).
Benchmark
Purpose
Model architecture
# Params
Input size
Acas XU
MLP
13k
5
CCTSDB
Complex
100k
2
cGAN
CNN, VIT
500k 68M
5
cifar100
ResNet
2.5M 3.8M
3072
Collins Acrospace
Complex
1.8M
1.2M
Collins RUL CNN
CNN
60k 262k
400 - 800
CORA
MLP
575k, 1.1M
784, 3072
Dist Shift
MLP
342k 855k
792
LinearizeNN
Performance
MLP
203k
4
LSNC
Complex
210k
8
Metaroom
Conv
466k 7.4M
5376
ml4acopf
Complex
4k 680k
22 402
NN4Sys
MLP
33k 37M
1-308
safeNLP
MLP
4k
30
Linyimagenet
ResNet
3.6M
9408
TLL Verify Bench
Two-Level Lattice NN
17k 67M
2
Traffic Signs Recognition
CNN
905k 1.7M
2.7k 12k
VGGNet16
CNN
138M
150k
ViT
VIT
68k 76k
3072
Yolo
ResNet
22k 37M
308
MLP, CNN (RcLU, Sigmoid, Tanh, AvgPool), ViT
SoundnessBench (ours)
Soundness
5k 3M
25 75
intermediate bounds by pushing them towards hy (x=))
[hx + (ma( - hw),, The + a (ha(#r) Txx) for 0 5
Shrinking the input and intermediate bounds breaks the soundness of verification, which is thus used as a synthetic bug.
2. Randomly dropped domains: Branch-and- used in NN verifiers iteratively branches (or splits) the verification problem into smaller subproblems, where each subproblem involves domain of input bounds or intermediate bounds, and tighter verified bounds can be computed for the smaller domains. In this synthetic bug, we randomly drop a proportion of new domains created in each branch-and sarcch-an-bbuudd iteration, and thereby break the soundness of the verifier. We use 3 (0 < 8 < 1) to denote the ratio of randomly dropped domains.
Findings In the experiment, we report the minimum a or 3 values that make at least one unverifiable instance to be claimed as verified by the modified verifier with the synthetic bug, as well as the minimum a 3 values that make all unverifiable instances claimed verified. We use binary search to find the minimum and B values. Results are shown in Tables 9 and 10. The results show that when sufficiently
18

---

Published in Transactions on Machine Learning Research (12/2025)
Table 9: Verification results on unverifiable instances for verifiers with synthetic bugs using shrunk input & intermediate bounds. a denotes the factor of shrunk bounds (a larger value implics more significant synthetic bug).
Model
Min for One "Verified"
Min a for All "Verified"
Name
Input size
ABC-T
NSAT-I
ABC-I
NSAT-I
CNN Conv
0.2
5
0.28
0.27
0.62
0.33
CNN
Conv
0.2
x 5
0.50
0.48
0.75
0.62
CNN 1
Conv
0.5
1
5
0.27
0.26
0.55
0.31
CNN 1 Conv
0.5
3 5
0.36
0.35
0.39
0.38
Table 10: Verification results on unverifiable instances for verifiers with synthetic bugs using randomly dropped domains. 8 denotes the ratio of randomly dropped domains.
Model
Min 3 for One "Verified"
Min B for All "Verified"
Name
Input size
ABC-1
NSAT-I
ABC-1
NSAT-I
CNN 1
Conv
0.2
x 5 x 5
0.39
0.40
0.43
0.42
CNN
1
Conv
0.2
3 x
0.44
0.44
0.44
0.46
CNN
Conv
0.5
1x5x5
0.38
0.39
0.40
0.41
CNN 1 Conv
0.5
3x5xb
0.45
0.44
0.46
0.44
large or value is applied (0.26 0.50 for a, and 0.38 0.45 for 3), our soundness benchmark reveal the synthetic bug, at least one unverifiable instance is claimed verified by the modified verifier. If a I values are further increased up to 0.75 for and 0.46 for B, all the unverifiable instances are claimed verified. These results demonstrate the effectiveness of our soundness benchmark when synthetic bugs are introduced into verifiers.
D Multiple Random Seeds
To evaluate the stability of our results, we conduct experiments with multiple random seeds for both training and verification, and find that our results overall remain stable across different seeds.
Table 11: Number of hidden counterexamples (out of 10 unverifiable instances) across different training seeds.
Model
Run 1
Run 2
Run 3
Run 4
Run 5
CNN AvgPool (c 0.5 and input size x 5)
9 9
VIT (e 0.2 and input size x 5 5)
10
10
8
Training with Multiple Seeds We select two representative models that consistently expose verifier bugs: (1) CNN AvgPool (e = 0.5 and input size x 5 x 5) and (2) ViT (e = 0.2 and input size 3 5 x 5). For cach sctting, we run training four additional times using different random seeds. Combined with results in Section 4.2, we now have a lotal of five runs for each of these models. Results are shown in Table 11. The mumber hidden counterexamples remains consistently high across different runs. The CNN AvgPool model shows slight variation, but still produces or more hidden counterexamples four of the five runs. The ViT model demonstrates even stronger consistency, producing more hidden counterexamples in all five runs. These results indicate that our approach is largely stable to randomness in training.
19

---

Published in Transactions on Machine Learning Research (12/2025)
Table 12: Proportion of falsified unverifiable instances on MNIST.
Model Architecture
Hidden CEX
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT
MB-23
MB-24 24
CNN Conv
8
U
0
0
CNN 2 Conv
9
33.3
33.3
11.1
0
0
CNN AvgPool
6
0
0
0
0
0
CNN Tanh
9
0
0
CNN Sigmoid
3
33.3
66.7
0
66.7
0
0
VIT
3
66.7
66.7
66.7
66.7
0
MLP 4 Hidden
10
30.0
30.0
20.0
30.0
0
0
MLP 6 Hidden
9
22.2
22.2
22.2
22.2
0
0
Verification with Multiple Seeds. We also conduct experiment the verifier a,ß-CROWN the only verifier that explicitly takes random seed as input argument, to examine how randomness affects verification outcomes. Specifically, we run a,3-CROWN with different random seeds on two CNN AvgPool models: (1) 6 = 0.2 and input size 5 x 5 and (2) e = 0.2 and input size 3 5 x 5. For the first model, results show that the verification outcomes remain consistent across different random seeds for all instances. For unverifiable instances, ABC-A consistently verifies 100% of instances across all five random soeds, while ABC-I consistently verifies 75% and times out on 25%. For regular instances, both ABC-A and ABC-I consistently verify 100% of instances across all seeds. This shows that the soundness bug in a,3-CROWN can be reliably triggered regardless of the random seed for verification. For the second model, which contains several unverifialle instances w,8-CROWN successfully falsifies, we observe that the proportion of falsified instances varies slightly with random seeds (ranging from 55.6% to 66.7% falsified across five seeds for both ABC-A and ABC-I), while results on regular instances remain stable (100% timcout for both ABC-A and ABC-I across all seeds). This behavior due to the PGD attack used a.B-CROWN for lalsification, which uses random initialization. Overall, our results still remain largely stable.
E Results on MNIST Dataset
In addition our main experiments on synthetic data, we conduct additional experiments on MNIST (LeCun et al., 2010), where the inputs are real images with sizes 28 28 much larger than those of our synthetic data. We train new models with various architectures and run all the verifiers models they support. Results are shown in Tables 12 to 15. Our training pipeline successfully produces MNIST models with hidden countercxamples, shown in Tables 12 and 13. For example, for model architectures where bugs have previously been identified on synthetic data, there are 6, 9 and 3 hidden counterexamples for models CNN AvgPool, CNN Tanh and VIT, respectively. Unsound verification and verifier bugs are still found on some of the new unverifialle instances on MNIST. Specifically, Marabou 2023 still wrongly verifies all the unverifiable instances on VIT, as shown in Table 13. However, on these relatively larger models, NN verifiers with bugs less likely to produce unsound results and expose bugs, besides Marabou 2023 on VIT. These MNIST models are not able to expose the bugs of a,8-CROWN and NeuralSAT on CNN AvgPool or Marabou 2023 on CNN Tanh. Therefore, we mainly focus on relatively small settings for our purpose of testing the soundness rather than evaluating the performance of NN verifiers, as discussed in Section 3.5.
verifier verifies any instance
20

---

Published in Transactions on Machine Learning Research (12/2025)
Table 13: Proportion of "verified" unverifiable instances on MNIST.
Model Architecture
Hidden CEX
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT
MB-23
MB
CNN Conv
8
0
0
CNN 2 Conv
9
0
0
0
0
CNN AvgPool
6
0
0
CNN Tanh
9
0
0
0
0
0
CNN Sigmoid
3
0
0
0
O
0
VIT
3
0
0
100
MLP 4 Hidden
10
0
U
0
0
MLP Hidden
9
0
0
0
Proportion of falsified regular instances MNIST. The number of regular instances for every model architecture is the same the number of unverifiable instances.
Table 14:
Model Architecture
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT
MB-23
MB-24
CNN 1 Conv
0
0
0
0
CNN 2 Conv
CNN AvgPool
0
16.7
16.7
0
CNN Tanh
0
0
11.1
0
CNN Sigmoid
0
33.3
33.3
0
33.3
0
0
ViT
0
0
0
0
0
MLP Hidden
0
0
0
0
0
MLP Hidden
22.2
33.3
22.2
33.3
0
Table 15: Proportion of verified regular instances on MNIST. The number of regular instances for every model architecture is the same the number of unverifiable instances.
Model Archilecture
ABC-A
ABC-I
NSAT-A
NSAT-I
PyRAT
MB-23
MB-24
CNN Conv
0
0
0
0
0
0
CNN Conv
0
0
0
0
0
0
CNN AvgPool
0
0
CNN Tanh
0
0
0
CNN Sigmoid
0
0
ViT
0
1001
MT.P 4 Hidden
0
0
MLP 6 Hidden
0
0
0
0
0
0
Denotes verification results that likely unsound given bugs identified unverifiable instances in
Table 13.
21


Reviews:
[{"review_id": "t7jAs2xraS", "reviewer": "Reviewer_RC6k", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method of training seemingly robust neural networks (safe against standard adversarial attacks) with intentionally placed adversarial attacks so that the soundness of neural network verifiers can be tested.  They train a variety of networks and use their technique to find bugs in other well known verifiers.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The main claim of the paper is that attacks can be planted that can not be found easily using PGD and other traditional adversarial attacks, and that knowing these in advance can lead to finding bugs in other verifiers.  No theorems have been presented, but the methodology of planting these seems correct and is backed by bugs acknowledged by the authors of the other verifiers.  Except for minor changes listed below, I find the paper to be quite thorough.", "audience_yn": "Yes", "audience_explanation": "While this method does not ensure the soundness of verifiers, the fact that bugs have been identified is significant.", "requested_changes": "Small changes and grammar:\n1. \"The input can be specified with an input region\"\n2. \"Formally, the NN verification problem can be defined as formally verifying\"\n3. \"An NN verifier is sound if the verification process is rigorously provable\" I think soundness should be defined here properly (it does not have to do with provability of verification).  \n4. \"If both Eq. (4) and Eq. (5) are achieved\" - I am not sure this is theoretically possible.  While I understand the intent is to imply that the approximate solutions to the argmax can be counterexamples, as stated delta_cex would be an approximate solution to 5.  I think this can be made clearer and more precise.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7vyan5fyTC", "reviewer": "Reviewer_jnVN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work introduces SoundnessBench, a benchmark for evaluating and debugging the soundness of Neural Network (NN) verification methods. Authors train small models in a synthetic task, with the objective to be robust, but missclassify predefined randomly-sampled perturbations. These are known as hidden counter examples. After training, they keep instances where 1) clean samples are correctly classified 2) hidden counter examples are missclassified 3) strong attacks (PGD and Autoattack) do not succeed in finding a counterexample. Then, if a verification methods verifies an instance where a hidden counter examples is known to exist, the instance is marked as a bug. Through this mechanism, authors discover bugs in three leading verification methods. This lead to a fix in the auto_LIRPA library, which affected two of the verification methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Authors effectively design verification instances where three existing verification methods ($\\alpha$-$\\beta$-CROWN, Marabou and NeuralSAT) are not sound.\n\nIn Sections 3 and 4, authors describe in detail their methodology to construct the verification instances in SoundnessBench.\n\nIn Section 4.4, authors ablate the need of two techniques employed in the training of their instances: 1) The perturbation window size and 2) The use of a margin objective. This ablation study shows the need of both techniques to produce a high quantity of instances where verification methods are not sound.", "audience_yn": "Yes", "audience_explanation": "While training models with known adversarial examples is not new [1] and mechanisms to stress-test robustness evaluations are also known [2], the evaluation of such techniques for detecting “bugs” in NN verification methods is novel and interesting for the community. \n\nThis benchmark is useful and interesting for TMLR’s audience as it provides mechanisms to stress-test NN verification methods. I expect authors to make their models and instances public to allow others to test their verification methods.\n\n**References**\n\n[1] BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain, IEEE Access, 2019\n\n[2] Increasing confidence in adversarial robustness evaluations, NeurIPS 2022", "requested_changes": "- **Fix notation:**  In the second line of Section 2, input vectors “x” are represented with lower-case non-bold letters, but then, from the next paragraph onwards, lower-case bold letters are used for vectors and lower-case non-bold for single numbers. Similarly, functions are represented as lower-case non-bold letters regardless of their output being a vector (“f”) or a single number (“s” and “h”). The notation should be homogenized. For example, all of the vectors (including outputs of “f”) should be denoted with lower-case bold letters.\n\n- **Vague discussion of related work:** The closest work to SoundnessBench is [2]. However, the discussion of the relationship between both methods is quite vague, just 4 lines at the end of Section 2. There are many similarities, e.g., hidden adversarial examples are sampled at the boundary for both methods ($r=0.98$ for SoundnessBench and $r=1$ for [2]) and some differences, e.g., [2] only train the last layer of the classifier. It would be great to understand why the methodologies should be different if the desired objective is the same. A broather discussion on this topic is needed. Also, there are many methods like [1] designed to introduce backdoors in neural networks, a discussion on those is also needed.", "broader_impact_concerns": "None.", "additional_comments": "Have you tested $\\alpha$-$\\beta$-CROWN and NeuralSAT on SoundnessBench after the fix in auto_LIRPA? Did the bugs disappear?", "extra_scores": {}, "extra_text": {}}, {"review_id": "rXD46wZmsq", "reviewer": "Reviewer_mpf6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Summary of Contributions:\nThis work mainly proposes a soundness benchmark named SoundnessBench to evaluate ability of different neural network verifiers. To make the verification task more challenging, SoundnessBench introduces a model training strategy with two objectives to make counterexamples more concealed. The benchmarking evaluation demonstrate that SoundnessBench is able to provide counterexamples that cannot be identified by state-of-art neural network verifiers. \n\nStrength:\n+ The benchmark provides ground truth for hard examples in the evaluation of neural network verifiers. The proposed training framework is able to generate counterexamples that are resistant to strong adversarial attacks.\n+ The benchmark evaluation finds bugs in state-of-art neural network verifier methods. \n\nWeakness:\n+ The scale and scope of data is limited. Whether counterexamples in small scale data can reflect real-world neural network verification scenarios is unknown. It is necessary to add experiment with larger scale data, as other benchmarks and neural network verification methods include these parts of experiments. \n+ Although the benchmark is on “soundness”, but the clear definition of “soundness” is missing. It is better to provide a definition of “soundness” so that we can compare two NN verifier methods on this evaluation metric.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Generally, the claims and contributions in the paper is supported by accurate, convincing and clear evidence. \nThe paper's central claim is supported by empirical evidence through the successful discovery of real, actionable bugs in three state-of-the-art neural network verifiers.", "audience_yn": "Yes", "audience_explanation": "The audiences that work on neural network verifiers or trustworthy AI topics will be interested in the findings of this paper.", "requested_changes": "Critical:\n+ Benchmarking evaluation with larger scale and scope of data that can better reflect real-world neural network verification scenarios.\n+ A clear definition on the key word “soundness” and benchmarking evaluation that could reflect that key word. \n\nStrengthen the work:\n+ A table to compare SoundnessBench and existing benchmark works on NN verifiers. \n+ An algorithm to make the training strategy in Section 3.4 clearer. \n+ Handle duplicate parts on problem definition in Section 2.1 and Section 3.1. \n+ Provide prospective on future research direction on NN verifiers based on benchmarking evaluation findings.", "broader_impact_concerns": "There are not concerns that that would require adding a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)