Relational Verification Leaps Forward with RABBit
Authors: Tarun Suresh
Debangshu Banerjee
Gagandeep Singh
Venue: NeurIPS 2024 poster (2024)
Keywords: Neural Network Verification
Relational Verification
Robustness
UAP verification
Optimization
Primary area: safety_in_machine_learning
TL;DR: Scalable branch and bound algorithm for relational DNN verification with cross-executional bound refinement and branching over multiple executions.

Abstract: We propose RABBit, a Branch-and-Bound-based verifier for verifying relational properties defined over Deep Neural Networks, such as robustness against universal adversarial perturbations (UAP). Existing SOTA complete $L_{\infty}$-robustness verifiers can not reason about dependencies between multiple executions and, as a result, are imprecise for relational verification. In contrast, existing SOTA relational verifiers only apply a single bounding step and do not utilize any branching strategies to refine the obtained bounds, thus producing imprecise results. We develop the first scalable Branch-and-Bound-based relational verifier, RABBit, which efficiently combines branching over multiple executions with cross-executional bound refinement to utilize relational constraints, gaining substantial precision over SOTA baselines on a wide range of datasets and networks. Our code is at https://github.com/uiuc-focal-lab/RABBit.

Relational Verification Leaps Forward with RABBit
Tarun Suresh Debangshu Banerjee" Gagandeep Singh 1.2 University of Illinois Urbana-Chann Chaama-hampaign, MVWar Research [tsuresh3, db21, ggnds) @ @illinois.edu
Abstract
We propose RABBit, a Branch-and-Bounde Bo--dd--baddddddd verifier for verifying relational properties defined over Deep Neural Networks, such as robustness against universal adversarial perturbations (UAP). Existing SOTA complete -robustttss verifiers can not reason about dependencies between multiple executions and, as a result, are imprecise for relational verification. In contrast, existing SOTA relational verifiers only apply single bounding step and do not utilize any branching strategies to refine the obtained bounds, thus producing imprecise results. We develop relational verifier, RABBit, which
the first scalable Branch-and-Boune efficiently combines branching over multiple executions with cross exseutxxcttiinal bound refinement to utilize relational constraints, gaining substantial precision over SOTA baselines on a wide range of datasets and networks. Code is at this URL
Introduction
Deep neural networks (DNNs) are now widely used in safety-critical fields like autonomous driving and medical diagnosis [Amato et al. 2013), where their decisions can have serious consequences. However, understanding and ensuring their reliability difficult due their complex and opaque nature. Despite efforts to find and address vulnerabilities, such as adversarial attacks [Goodfellow al. 2014, Madry et al., 2018 Moosavi-Dezzoolli et al. 2017] and adversarial training techniques [Madry et al. 2018). ensuring safety remains a challenge. As result, extensive research is focused on formally verifying the safety of DNNs. However, most of the existing Loc. robustness verification techniques can not handle relational properties common in practical situations. While significant efforts have been invested in verifying the absence input-specific adversarial examples within the local neighborhood of test inputs, recent studies [Li et al. 2019a] emphasize that input-specific attacks are impractical regardless. Conversely, practical attack scenarios [Liu et al. 2023 Li et al. 2019b]ar involve the creation of universal adversarial perturbations (UAPs) Aibosavi-Dezfooo et al. 2017]] which are crafted to impact a substantial portion of inputs from the training distribution. Ra VeN Banerjee al. 2024b) and subsequently RACoon [Banerjee and Singh 2024] showed that since the same adversarial perturbation is applied to multiple inputs, the executions on different perturbed inputs related, exploiting the relationship between different executions significantly improves the precision of the verifier. Despite Ra RaVeN and RACoon's 's ability to leverage cross-eatd cnnssseeecttional dependencies, both of them remain imprecise as they only apply single bounding step and lack refinement using branching strategies used in SOTA complete non-relational verifiers.
Key challenges: For precise relational verification, need efficient algorithms that can effectively combine branching strategies over multiple exccutions with bounding techniques that can leverage cross-executional dependencies. Theoretically. MILP (Mixed Integer Linear Programming) can exactly encode DNN executions with piecewise linear activation functions like ReLU over any input regions specified by linear inequalities. However, the associated MILP optimization problem is computationally expensive. For instance, encoding executions of a DNN with n, ReLU activations
"Equal contribution. Primary Correspondence: tsuresh3 edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

introduces (n, X k) integer variables in the worst case. As the cost of MILP optimization grows exponentially with the number of integer variables, even SOTA off- sff-the-shelf solvers like Gurobi [Gurobi Optimization, LLC 2018 struggle to verify small DNNS for relational property over k executions within a reasonable time limit. For scalability, SOTA non-relational verifiers like a, 8- CROWN (Wang et al.) [02T] design custom "Branch and Bound" (BaB) solvers using more scalable differentiable optimization techniques such as gradient descent. However, these verifiers ignore dependencies between multiple executions, resulting in imprecise relational verification. Conversely, the SOTA relational verifier RACoon uses parametric linear relaxation for each activation to avoid integer variables and employs gradient descent to learn parameters that leverage cross-executional dependencies for verification. This method, however, introduces imprecision due to the replacement of non-linear activations with parametric linear approximations. Therefore, precise relational verification requires scalable algorithms that can: a) scale to the large DNNs used in this paper, b) effectively reduce imprecision from parametric linear relaxations, and c) utilize cross-executional dependencies.
Our contributions: We advance the state-of-ther afrtt- in relational DNN verification by:
Efficiently combining branching strategies over multiple DNN executions with a cross-executional bounding method that utilizes dependencies between DNN's outputs from different executions while reducing imprecision resulting from parametric linear relaxations.
Developing two "branch and bound" algorithms, each with its own advantages a) strong bounding: applies cross-execution bounding at each step, branching over all executions. This method provides tighter bounds than RACoon eroos-execcutional bound refinement without branching) and a, 8- CROWN (branching without cross-executional bound refinement), b) strong branching: applies cross-execution bounding only at the start to derive fixed linear approximations for each execution. These approximations are then used to branch independently over each execution, exploring more branches per execution.
Combining strong bounding and branching results into efficiently optimizable MILP instance that leverages the benefits of both techniques, outperforming each individually. Performing extensive experiments on popular datasets and various DNNs (standard and robustly trained) to showcase the precision improvement over the current SOTA baselines.
2 Related Works
Given logical input specification & and an output specification v. DNN verifiers formally prove that for all inputs x satisfying Φ, the output N(x) of the DNN satisfies vb. If the verification process fails, the verifier generates counter-example where the output specification tb does not hold. DNN verifiers are broadly divided into three main types based on their ability to prove properties: (i) sound but incomplete verifiers which may not always prove property even if it holds [Gehr et al. 2018 Singh et al. 2018 [019b]a a /Zhang et al. [0118 Xu et al.
Non-relational DNN verifiers:
2020, 202 l] J. (ii) complete verifiers that can always prove the property it holds [Wang et al. [2018]
Gehr et al. 2018 Bunel et al. 2020a c Bak et al. 2020 Ehlers| 2017 Ferrari et al. 2022 Fromherz
al. 2021 Wang et 2021 Palma al. 2021 Anderson et al. 2020 Zhhan al.
2022a] and (ii)
verifiers with probabilistic guarantees [Cohen et al. 2019 Li et al. 2022]. Beyond the commonly studied I robustness verification problem, several works adapt DNN verification techniques for
specific applications, such as robustness against image rotation (Singh et al. 2019b] Balunovic et al.)
2019), incremental verification [Ugare et al. 2023 2024). interpretability (Banerjee et al. 2024a]
and certifiable training [Mueller et al. 2023 Palma et al. 2024 Jiang and Singh 2024).
Existing relational verifiers fall into two main categories based on the type of relational properties they can handle: (i) verifiers for properties such as UAP and fairness, which are defined across multiple executions of the same DNN (Zeng al.) 2023 Khedr and Shoukry 2023 [Myer et al., 2024 Banerjee et al. 2024b Banerjee and Singh, 2024 Banerjee et al. 2024c]. and (ii) verifiers for properties like local DNN equivalence, defined over multiple executions of
Relational DNN verifier:
different DNNs on the same input [Paulsen et al., 2020 2021]. For relational properties defined
across multiple executions of the same DNN, existing verifiers [Khedr and Shoukry [2023] reduce the verification problem to an Loo robustness problem by constructing a "product DNN" that includes multiple copies of the same DNN. However, the relational verifier in [Khedr and Shoukry 2023) treats all & executions of the DNN as independent, which results in a loss of precision. On the other hand, (Zeng et al. 2023) (referred to as the I/O formulation) tracks the relationships between inputs used in multiple executions at the input layer, but it does not maintain the relationships between the

---

outputs fed into the subsequent hidden layers. As a result, it achieves only limited improvement over baseline verifiers that treat all executions independently. RaVeN [Banerjee et al., 2024b) uses DiffPoly a abstract intepretation based framework to track linear relationships between the outputs at all layers resulting from multiple executions of the same DNN. While RaVeN is significantly more precise than the I/O formulation, tracking linear constraints at each layer across all DNN executions can be computationally expensive. The SOTA relational verifier RACoon [Banerjee and Singh 2024) improves the scalability of RaVeN while maintaining Ra s precision by introducing new gradient-descent based bounding strategy called cross-cxecutional bound refinement, as detailed in Section D] There exist. probabilistic verifiers, (Xie et al., 2021 Zhang et al.) 2022b] based on
randomized smoothing [Cohen et al.] 2019] for verifying relational properties. However, these works can only give probabilistic guarantees smoothed models which have high inference costs. Similar to [Banerjee et al. 2024b Banerjee and Singh] 2024). in this work, we focus on deterministic relational verifiers for DNNs with ReI.U activation. However, RABBit can be extended to activations like Sigmoid with branching methods (Shi et al., 2024] and parametric bounds (Wu et al. 2023).
3 Preliminaries
We provide background on "branch and bound" (BaB) based non-relational DNN verification, as well as DNN safety properties that can be encoded relational properties.
For a single execution, non-relational DNN verification focuses on proving that, for all perturbations + 8 of given input specified by o, the network's 's output y N(x+ 5) meets a specified logical condition tp. Commonly, safety properties such as L.o robustness encode the output condition
Non-relational DNN verification:
(+) linear inequality or conjunction of linear inequalities over the DNN output y E Rd For instance, an output property could be expressed as vy(y) = (c"y 2 0), where R Generally, even for DNNs with piecewise-linear activation functions and input constraints defined by linear inequalities, complete verification-ia. always proving the property or finding a counterexample-is NP-complete problem. Given a DNN N : I + R" and property defined by (ø, $). scalable yet sound (but incomplete) verifiers approximate the network's behavior by computing linear approximation specified by L E R. and E R. For any input satisfying 6, this linear approximation ensures that L"xx + b < (x).. The verifier then aims to show that L'x + b 2 0 for all x that satisfy ø. which implies c c''N(x) (x) > O. While L"x + b provides valid lower bound for c" (x),, it may lack precision. To enhance this precision for lincearse-linear activations, state-of-the-art non-relational verifiers use branch-and-boume (BaB) method. In each branching step, the problem is divided into smaller subproblems, while the bounding method computes a valid lower bound for each subproblem.
The non-relational verifier computes L by replacing non-linear activations with linear relaxations, which introduces imprecision. However, for piecewise linear activations like ReLU, it is possible to consider each linear piece separately as different subproblems, avoiding the need for imprecise linear relaxations. For instance, for y = ReLU(()) branching on x and considering the cases 2 S 0 and x 2 0 allows decomposing Re (x) into two distinct linear pieces. Still the worst case decomposing all ReLU nodes in a DNN results in exponential blowup making it practically infeasible. Therefore. SOTA non-relation verifiers like o, B CROWN [Wang et al., 202T) greedily pick a small subset of ReLU nodes for branching while using linear relaxations for the rest. We explain the bounding step used for each subproblem below.
Branching for piecewise linear activation:
Bounding with parameter refinement: Obtaining sound linear relaxations of activations a like ReLU, which are not used for branching, involves computing linear lower bounds (z(2) and upper bounds ou, (3x that contain all possible outputs of w.r.t all inputs x satisfying ø. That is, for all possible input values x of a, (2) S a(x) S (x(x) holds. SOTA non-relational verifiers, such as a, B-CROWN, improve precision by using parametric linear relaxations instead of static linear bounds and refine the parameters to facilitate verification of the property (ø, 1)) For example, for (z),, the parametric lower bound is ReLU'(2) (z) 2 a X x with a [0, 1]. Since a remains a valid lower for any a E [0, 11, this allows optimizing a while ensuring the bound remains mathematically correct. Each branched ReLU say y = Re LU (z), introduces two subproblems each with one additional constraint x < 0 (or, x 2 0) where ReLU behaves as a linear function i.e. y = 0 (or, y = x) respectively. To obtain the lower bound of L + b over inputs satisfying & with the additional branching constraints a., 3-CROWN convert the constrained optimization problem into an unconstrained one by looking at the Lagrangian dual. The dual replaces each branching constraint by augmenting the minimization objective L x + b with additional terms i.. L" x b- B+ for S 0
3

---

or +'' --+ for x 2 O where 3+ 2 0 and 8 I O. Overall, at high level, a, B-CROWN computes parametric linear approximations L (a. B) x x+ b(a, B) and refine the parameters a, B to facilitate verification of (Ø, +)).
Relational properties defined for DNN N IR. R defined over k executions of N are specified by the tuple ( V) Here, & - Xk + falue, (the input specification) encodes the input region & S R encompassing all potential inputs corresponding to each of the k executions of N. Furthermore, the safety property we expect the outputs of all k execu- tions of N to satisfy is specified by V R + {true, false) } (the output specification). Given N, an input specification t and output specification V, DNN relational verification seeks formally prove whether Vxi,... Xx E II 1 ((xi..................._ xix) = (x(N) .Nxx)) or otherwise provide counterexample. The inputs to the lt executions of N are denoted by xi,--.. x and the correspond- ing outputs are denoted by N (xx))..... N Nxx)).
DNN relational properties:
For the i-th execution, commonly, the input region
of is a Lee region around a fixed point x; E I defined as of = {x] E R" ||x] - xi S c} and the corresponding output specification 2 (NV())) = A- (CLJ" (x(x)) 2 0). Consequently, & ((xi, x = ( E ()) A 15 (xi,..., x where &6 (xi, xi) encodes the relation-
ship between the inputs used in different execution and (NV(x) ..... N ((x))) A (N(())) Following this, we describe relational properties encoding important DNN safety configurations.
UAP verification: In a UAP attack, given a DNN N, the adversary aims to find an adversarial perturbation with a bounded L norm that maximizes the rate which N misclassifies when the same adversarial perturbation is applied to all inputs from the distribution. The UAP verification problem aims to find the worst-case accuracy of N against the UAP adversary. We refer to this worst-case accuracy as UAP accuracy in the rest of the paper. As shown by Theorem 2 in (Zeng] al. 2.23), is possible to statistically estimate the UAP accuracy of N with respect to the input distribution if one can determine the UAP accuracy of N on l randomly selected images. We focus on the k-UAP verification problem for the rest of the paper as improving the precision of k-UAP verification directly improves the UAP accuracy on the input distribution [Banerjee and Singh] 2024). The K-UAP verification problem is fundementally different from local L. robustness verification since the same adversarial perturbation is applied across the set of inputs. Thus, improving precision for the UAP verification problem requires a relational verifier that can exploit dependencies between the perturbed inputs. We provide the & and vi of the UAP verification problem in Appendix A.1
Cross-executional BaB
The key distinction between relational and non-relational DNN verification is the dependency between different DNN executions, which necessitates that any precise relational verifier utilizes these cross- exccution dependencies. For instance, for k-UAP problem with two images X1, X2 consider the scenario where both X1 and X2 have valid adversarial perturbations ō1 and 82 but no common perturbation say 5 that works for both X1 and X2. In this case, any non-relational verification that does not account for cross-execution dependencies can never prove the absence of common perturbation given that both X1, X2 have valid adversarial perturbations. This highlights the importance of utilizing cross-executional dependencies. The SOTA relational verifier RACoon (Banerjee and Singh 2024) leverages cross-execution dependencies to jointly optimize the a parameters from different executions, significantly improving the precision of relational verification. However, RACoon only uses parametric linear relaxations for non- non-linear activations and lacks branching step, resulting in reduced precision, as confirmed by our experimental results in Section 6 To address this, we propose two separate BaB algorithms, each with its benefits, described in Sections 4.] and 4.2 Finally, we combine the results formulate an efficiently optimizable MILP instance Section
4.1 Strong Bounding
Before going into the details, we briefly review the cross-executional bound refinement proposed in RACoon. For k-UAP, given any subset S of the A executions, RACoon can verify the absence of any common perturbation that works for all executions in S with cross- exosstexettionall bound refinement. For all i E S, let (L,(a)). ba())) denote the parametric linear approximations corresponding to the i-th execution. Then the optimal value t* maxa.... -cx Eies d x L, (a.)))+1++ A Xaxa((a,) 2 0 proves the absence of common perturbation 8 for S. Here, is the perturbation bound i.e. |001|0 S e, a,(a:) = b, LL, [ x; and X; E [0, 1] with Eigs X; are the cross-executione parameters that relate linear approximations from different execution enabling joint optimization over a,s. Next,

---

we detail the first BaB method- strong bounding that combines cross-executional bounding with branching methods to verify the absence of common perturbation for any subset of n = |S] executions.
Branching and bounding: For n Σ k executions, we construct a "product DNN" by duplicating the DNN n. times, one for each execution. Formally, product DNN is function N RR IR 1gxx
with NN (x,, Xn) (x1))..... .N (xn) )² . At each branching step, we greedily select a subset of unbranched ReLU activations from the product DNN and branch on them, while using parametric linear relaxations for the rest, We adapt existing greedy branching heuristics, such as BaBSR [Bunel ct al. 2020b], for selecting the candidate ReLU activations. The heuristic computes a score for each unbranched ReLU activation in the product DNN, and we branch on the activations with the highest scores. Next, we detail the bounding method applied to each subproblem resulting from branching. Since the number of subproblems can be large, the bounding method needs to be fast yet capable of leveraging both branching constraints and cross-executiin executional bound refinement from RACoon can not handle branching constraints, while the bounding step from a. 3.CCROON does not utilize dependencies across executions. Hence, we develop a three- step algorithm for obtaining the optimal value t* with fast gradient descent-based methods. First, we replace these branching constraints by introducing dual variables B. resulting in new parametric linear approximations (L. B:), (. for each subproblem for all i E S. Then for each subproblem, we introduce additional variables X; for each execution with constraints X E [o. 1] and Eies X 1. These 2,S relate linear approximations from different executions capturing cross-executional dependencies. This reduces finding t" for each subproblem to the following optimization problem t* Ba.., -E Eies A x L.(a,,
dependencies. However, the cross-
+ Eies X x a, B,). Here, a:(a;; B.) b, (a,, B.) + L.,,,, x1. Finally, we apply projected gradient ascent to refine parameters (a B,, ), The detailed derivation of the bounding step and the proof of correctness is in Appendix B Precision gains of strong bounding over the baselines are in Section 6.2 Suppose.
F(S) denotes the set of subproblems then Theorem 4.1 proves the absence of common perturbation for the subset of executions S.
Theorem 4.1. mf B... -ex Eies x x L, Ba)aB + +E d x a, Br) 2 then executions in S do not have a common perturbation 8 E W with loc Se.
Proof: The detailed proof is in the Appendix B
While strong bounding effectively combines cross-executional refinement with branching, it has the following drawbacks that led to the development of the 2nd BaB method. First, strong bounding branches over all executions simultaneously, which limits the number of branches explored per execution within a fixed timeout compared to branching on individual executions. For instance, if strong bounding solves m subproblems for n executions, then assuming each execution branched uniformly, each execution gets only m subproblems. In contrast, given the same timeout, branching individually allows exploring m2 subproblems per execution. Second, strong bounding only proves the absence of common perturbation, a relaxation of the k-UAP problem. To mitigate this, RACoon uses parameter refinement to obtain linear approximations and formulate MILP, providing more precise bound on U-U accuracy. However, for strong bounding, as the number of subproblems increases and each subproblem has a different linear approximation, formulating a MILP with each linear approximation is practically infcasible. Restricting the number of linear approximations can help accommodate MILPP formulation by compromising on the strong cross-executional bounding.
4.2
Strong Branching
Unlike strong bounding, strong branching explores more branches by branching on each execution independently. Additionally, for each execution, we aim to keep the number of linear approximations small post-branching, ensuring the MILP instance using these approximations remains easy to optimize. To limit the number of linear approximations for each execution i, we fix a set of linear coefficients {L,,..., } called "target coefficients" and for each E [m], L, R compute valid lower bound by of the following optimization problem mins c. N + 8) - LT 6) with |10]|=0 S e using BaB. In this case, for all 8 no with || S the refined bias bj and L, remain a valid lower bound of cf N(;; + 5) i.e. L (xi + 8) bū S Nx; + 8). Moreover, since we only refine the bias, the number of linear approximations remains the same at the start of BaB, even after branching. Next, we describe how we utilize cross-execution dependencies while branching on each execution independently.
5

---

Selecting targets: To select target coefficients, we greedily pick subsets of executions and run cross-executional refinement from RACoon without branching on each subset of executions. We describe the greedy selection strategy in Section 5 For each set of executions, we add the linear approximations obtained by cross-executional refinement to the corresponding executions' target sets. Cross-executional refinement ensures for each execution set the parameters corresponding to the linear approximations are tailored for the relational verification.
Bounding and branching; Given a target coefficient L, E I since finding the exact solution of ming e" (x; - 6) L (x; + 6) is computationally expensive, strong branching aims to obtain a tight mathematically correct lower bound on the difference c" N(x; + 6) L (x; 8). For any subproblem. let (L )). b(a,, B)) denote the parametric linear approximation. Then for this particular subproblem, for all a, B, L L( ()"(z) + 5) + b(a, B) S c² N(x; + +)) and subsequently:
max min (L(a, B) D (x + 6) b(a. B) - min ca c 5) L (x; + (I)
a [|6] Sc
|81|
The optimal solution of the max-min problem in Eq. provides a mathematically correct lower bound of minş c² N N(x; + 8) - L (x: + 6) for each subproblem. However, it is hard to solve a max-min problem with scalable differentiable optimization techniques like gradient descent typically used for large DNNs considered in this paper. Instead, we compute closed form of the inner minimization problem reducing the optimization instance to a more tractable maximization problem (Theorem 4.2).
Theorem 4.2. For any a.B, if L(, E Ro and b(a, E E then min Se (L(a, L.)a(x (x + + b(a B) = - |[ B) Lelli (L, L,)x x + B).
Proof: The proof is in Appendix C
We apply projected gradient ascent to optimize the maximization with the closed form obtained above (Appendix C.I). The proof of the correctness of the bounding method is in Appendix C Note the proof of correctness does not necessitate the optimizer find the global optimum. This is important since gradient ascent may not always converge to the global optimum. Since strong branching branch on each execution independently we reuse the branching strategy of a, 3-CON.
5 RABBit
In this section, we detail the algorithm (Algo. [ that combines the results from strong bounding and strong branching to formulate the MILP. Running strong bounding all 2 - 1 non-empty subsets of l executions is impractical. Therefore, we use greedy approach to select subsets executions for strong bounding. Similarly, for strong branching, we greedily select the target linear coefficients. First, we describe both greedy strategies before moving on to the MILP formulation.
Elimination of individually verified executions: RABBit maintains a list of unverified indices and climinates any exccutions that can be verified individually and does not consider them for subsequent steps (lines 3, 8, and 13 in Algo. [] For instance, for U-UAP verification, we do not need to consider those executions that are proved to have no adversarial perturbation 6 such that ||ō) I c. Pruning individually verified executions improves the runtime without any compromise the precision of the relational verifier (see Theorem 1 [Banerjee and Singh 2024)).
Greedy target coefficient selection: RABBit first runs RACoon which in turn executes incom- plete non-relational verifier a-CROWN (Xu et al., 202T] eliminating the verified executions (line in Algo. D. Subsequently, for target selection, RABB greedily picks the first ky (hyperparameter) exccutions based on 8; the lower bound on v (N + +)) as computed by a-CROWN, prioritizing exccutions with higher s; (line 9). Intuitively, for unverified executions, s, measures the maximum violation of the output specification & (NV(x + 8)) and thus leads to the natural choice of picking executions with smaller violations. For each selected execution i, we choose up m target coeffi- cients by iterating over all subsets i E S considered by RACoon, and selecting linear approximations corresponding to the top m subsets. The priority of each subset S. Subsets S with higher t* indicate smaller violations and are more likely to be verified for the absence of common perturbation, making them suitable for target selection.
executionll lower bound t* from RACoon decides the
Selection of subsets of executions for strong bounding: Thereafter, until timeout Ç, we run strong bounding on subsets of executions from individually unverified executions I. For each subset S C I.

---

Algorithm 1 RABBit
1: Input: N, (Φ, V), k. K, timeout S
16:
M M TD V. I,
2: Output: M.
17:
M max (M( V) (p)(
3: I 4-( 0
Unverified indices
18:
end for
4: [+ 0
P. Linear approximations
19:
/2 f- top-k; indices from / based on
5: C (
Cross-verified executions
20:
while time() ( do
6: s s
> Lower bounds from a- e-Crronn
21:
S <- Greedily select subset of Iz
7: M+
> >niitialize verified UAP accuracy
22:
ts + StrrongBoudding(5.. Φ v
8: (s, f- RACoon([N, (Φ, V) k)
23:
if 2 0 then
9: t indices from / based on
24:
C f Append
10: for E do
25:
M MILP(L, Φ. V, k. I, C)
11: Dy f- SurongBranchinggdds a/t fli)
26: M + max (M, Opt(M))
12: if Verified( , vi", L[i), b5) then
27:
end if
1-/\ {i} end if
28:
end while
14:
29: return M
15:
UpdateBias(Clij..e
the cross-executional bound obtained by RACoon on decides its priority. However, considering all non-empty subsets of 7 can be expensive. Instead, similar to strong branching, we first pick top-k, executions (12) from I (Algo [ line 19). We sort all non-empty subsets C I2 based on their priority and, in each iteration, run strong bounding on the highest-priority subset that has not been scheduled yet (Algo Tline 22). Given a large timeout, RABBit would eventually select all subsets from 1z.
MILP Formulation: The MILP formulation uses both the refined biases from strong branching (line 11) and the subsets of executions verified for the absence of common perturbation from strong bounding (line 22) to compute final verified UAP accuracy. RABBit MILP formulation involves three steps. First, we deduce linear constraints between the input and output of N for each unverified execution using linear approximations of N with refined bias obtained by strong branching. Secondly, we add constraints for each subset S verified for the absence of common perturbation with strong bounding. Then, similar to the current SOTA baseline [Banerjee and Singh [024], encode the output specification as a MILP objective, introducing only ((k) integer variables. Finally, we use an off-the-shell MILP solver (Gurobi Optimization, LLC 2018 to optimize the MILP
encoding: First, we show the MILP objective M that encodes W We introduce binary variables 2, E (0. 1) for each individually unverified execution in I where for any perturbation 8 no and |8 I c, 2; = 1 implies ei (N(x, + +)) = Truc. Then the finding the worst case UAP accuracy is equivalent to the following M == ((k (7) + minjoll Σier z4).
Constraints encoding: We add constraints from strong bounding. strong branching, and from the linear approximation obtained from the call to RACoon (Algo. line 8). Suppose for any subset S C I, strong bounding verifies the absence of common perturbation. Then for all ó e g and ||0|| I c at least one of the executions from S will always satisfy the corresponding output specification. Hence, for every such S we add the constraint: Eies 2; 2 1. Now, let for any i E T, [(L]] b()..... ..., (L,". br²) denote set of linear approximation with bTTh either coming from RACoon or from strong branching. Then we add the following constraints 2; 2 2/. 2 (o, 2 0), 0: 2 L (xi + 0) + b3 where j E [m], and o E R, =/ are new real and integer variables respectively. Limitations:
Although RABBit outperforms SOTA verifiers in relational verification, like all deterministic verifiers, whether relational or non-relational (including ours), do not scale to deep neural networks (DNNs) trained on very large datasets such as ImageNet. RABBit is sound but incomplete, meaning it may not be able to prove certain relational properties even if they are truc. Note that all complete non-relational verifiers are also incomplete for relational properties since they do not track any dependencies between executions.
6 Experimental Evaluation
We evaluate the effectiveness of RABBit on multiple relational properties such UAP accuracy ( Table [ and top-k accuracy (Appendix T) DNNs, and datasets. In our evaluation, we compare
7

---

Table 1: RABBit Efficacy Analysis for Worst-Case UAP Aceuracy
Dussee
Netwokk
Trasting Methol
Pensrbatio Bocad
CROWN
a: CROWN
0.2 CROWN
MN BuB
GEP CRP-CROWN
RACoon
Stongg Brynding
Store
RABBK
Stoctue
Blaaching
Stadded
1055
01.5
45.4
65.3
15.0
61.8
45.4
15.4
61.8 (-1.0)
61.3 (+1.0)
DarAk
5/354
04.4
40.6
13.8
15.0
53.8
10.4
$1.6
90.0 (-1(0)
CIPARLO
SourSaall
SABR
2/255
25.2
15.8
79.4
50.0
$0.0
76.5
75.2
8310 (1210)
$400 (38
SourS
CITRUS
9238 (13.)
63.0
(400)
Standerd
0.0
530
99.4
83.6
17.4
66.3
60.0
606
(400)
64 6
Strel
0.13
750 (-70)
7-6--1.0 00
M00-300
MNIST
026 (:2.4)
(:33)
Cur Smull
(10.)
RABBit against SOTA baselines, including non-relational verifiers CROWN Zhang al. 2018
a-CROWN Xuu et al. 2021 a. 3 3-CROWN [Wang et al. 2021] MN-BaB Ferrari et al. 2022
GCP- OCP-CROWN [Zhang et al. 2022a]. as well as relational verifiers 1/O Formulation [Zeg et al. 2023
and RACoon. As previously noted, RaVeN adds linear constraints for each layer. which restricts its scalability as the number of executions increases. Therefore, we compare RABBit with ReN for a smaller execution count of l: 5, as shown in Appendix Table Additionally, we show that: a) given the same time, RABBit always outperforms the SOTA BaB-based non-relational verifier o, B- B-CRRWN; b) strong bounding computes a tighter bound on t" than o, 3-CROWN; and c) we provide an ablation study on and used by RABBit.
6.1 Experiment Setup
Networks. We use standard convolutional architectures, such as ConySmall and ConvBig, which are used to evaluate both SOTA relational (Banerjee and Singh [024] and non-relational verifiers (Wang al., 202T) (see Table D We provide the details of the DNN architectures the Appendix D.
We use networks trained using both standard training methods and robust training strategies, such as DiffAl [Mirman et al. [201]], SABR [Mueller et al.) 2023), and CITRUS (Xu and Singh [202]]. Our experiments utilize publicly available pre-trained DNNs sourced from the CROWN repository
[Zhang et al. 2020], a, 3-CRRWN repository [Wang et al. 2021], and ERAN repository (Singh
et al.
2019b]. The clean accuracies of these networks are reported in Appendix D.2
Implementation details and hyperparameters. We implemented our method in Python with Pytorch V1.11 on top of SOTA complete non-relational verifier a. 3.-CRONN [Wang et al. 2021]. We used Gurobi V11.0 as the off off-the-siie the-shelf MILP solver. For both strong bounding and strong branching, we use Adam (Kingma and Ba 2014) for parameter learning and run it for 20 iterations on each subproblem. We set the value of k = T0 for CIFAR- 10 and k = 24 for MNIST networks respectively. We use a single NVIDIA A A100-PCI GPU with 40 GB RAM for bound refinement and an Intel(R) Xeon(R) Silver 4214R CPU @ 2. 2.40GHz with 64 GB RAM for MILP optimization. For any relational property with & executions, we give overall timeout of k minutes (averaging 1 minute/execution) to RABBit and all baselines. Each MILP instance gets timeout of 10 minutes. We issue the MILP optimization call on line 25 of Algo. Min in a separate thread for runtime optimization, ensuring that the MILP optimization process does not unnecessarily block the subsequent iterations of the while loop (line 20 of Algo.
6.2 Experimental Results
Effectiveness of RABBit: Table comppres the results of RABBit to all baselines across different datasets (column 1) and DNN architectures (column 2) trained with various methods (column 3), with e values defining the Loge bound of ó column 4. For each DNN and e, we run RABBit and all the baselines on 10 relational properties each defined with k 50 randomly selected inputs, and report the worst-case UAP accuracy averaged over the 10 properties. Note that for each DNN, we exclude inputs misclassified by the DNN. We compare the performance of RABBit against SOTA relational and complete non con-reeational verifiers as well as against strong bounding and strong branching.
The results in T demonstrate that strong bounding, strong branching, and RABBit all outperform the existing SOTA verifiers on all DNNs and e. Notably, RABBit gains up to +4.8% and up to +3.2% improvement in the worst-case UAP accuracy (averaged over 10 runs) for CIFAR 10 and MNIST DNNs, respectively. RABBit also efficiently scales to the largest verifiable DNN architectures such as ConvBig, conferring up to +1 .,6% improvement in worst-case UAP accuracy. some cases, strong bounding outperforms strong branching, while in others, strong branching outperforms strong
8

---

bounding, highlighting the importance of both methods. RABBit combines the strengths of both strong branching and strong bounding, producing the best results overall.
K-UAP Accuracy (%) Vs Time
k-AAP Accuracy (%) Vs Time
K-UAP Accuracy (%) Vs Time
0. B-CROWN
B-CCROW
0-8- B-CRRRWW
RABBIt
RABBIE
RABBIt
time (seconds)
time (seconds)
time (seconds)
(a) DifiAT (CIFARIO)
(h) SABR (CIFAR1O)
(c) CITRUS (CIFAR10)
Figure 1: Average Worst Case k-UAP accuracy vs Time for ConvSmall CIFARIO DNNs.
Time vs UAP Accuracy Analysis: Fig. I shows timewise the worst-case UAP accuracy (averaged over 10 runs) for different ConvSmall CIFAR 10 networks with h = 50 on values from Table Note that RABBit invokes RACoon, which in turn calls a- a-CROWN and eliminates verified executions (Line 7 in Algorithm TT. Hence, for fair comparison, we also run a-CROWN first for a, 3-CROWN and then run a, 3-CCRONN only on the unverified indices. For all DNNs, RABBit consistently outperforms the SOTA BaB-based non-re non-relational verifier a, 3-CRROWN at all timestamps. This confirms that the improved precision shown in Table is not dependent on the specific timeout value.
Average Bound improvement with RABBIE 1961 VS. Time
Average Improvement with RABBIt (%) Vs. Time
100
00
60
20
0.3 CRCW
PACoon
10
27
20 time (seconds)
time (seconds)
(a) Diff'l (CIFARIO)
(b) CITRUS (CIFARIO)
Figure 2: Timewise Analysis of Average % Improvement in ts with Strong Bounding (CIFARIO)
Average Bound Improvement with RABBIt (%) Time
Average Bounc Improvement w tht RABBit (%) Vs Time
90
20
0.5- CROWN
CROWN
RoCoon
20
time (seconds)
time (seconds)
(a) DiffAI (MNIST)
(b) CITRUS (MNIST)
Figure 3: Timewise Analysis of Average % Improvement in L" with Strong Bounding (MNIST)
Evaluating Bound Improvement: In Figs 2 and 3 we present a timewise analysis of the improve- ment in 8" with strong bounding over a, B-CROWN and RACoon. For this experiment, we use DiffAI and CITRUS ConvSmall networks with epsilon values from Table [ For each network and we select 30 executions at random and compute the percentage improvement in 1" with strong bounding over RACoon and a, B-CROWN, We also report the average improvement and 95% confidence intervals for all cases in Table 6 in Appendix [ The results demonstrate that the 1t with strong bounding is significantly tighter compared to the bounds from the SOTA verifiers a, 3-CROWN and
9

---

RACoon at all timestamps. Furthermore, strong bounding improves t" on average by up to 108.7% for CIFARIO networks and 57.7% for MNIST networks. These results highlight the importance of leveraging dependencies across executions during both branching and bounding to improve precision.
Different and values: Fig. shows the results of RACoon, o, B-CROWN, and RABBit for U-UAP verification of CIFAR 10 ConvSmall DNNs for different values and A = 50. RABBit outperforms RACoon and a, 3-CROWN for all evaluated values, notably improving the worst case k-UAP accuracy by up to 4.8%. Similarly, we analyze the performance of RACoon, a, 3-CROWN, and RABBit for k-UAP verification of CIFARIO ConySmall DNNs with different i values. As presented in Fig. 5 for all k values, RABBit is more precise than both baselines. Expectedly, the worst-case k- A-U accuracy for relational verifiers is higher with larger values as it is easier to prove the absence of common perturbation with larger k.
Average Worst Case k:UAP Accuracy (%)
Ayarage Warst Case K-UAP Accuracy (%)
22
20
D-J-CROWN
a.B. CROWN
RABBIt
SAUDE
IALsoon
Epsilen (+255)
Epsilon (+/25))
DiAAI (CIFARIO)
(b) CITRUS (CIFARIO)
Figure Average Worst Case k-UAP accuracy for CIFARIO DNNs.
Average Worst Case k-UAP Accuracy (%)
Average Worst Case k-UAP Accuracy (%6
30
20
23
10
HACOES
KACOOM
(a) DiffAI (CIFARIO)
(b) CITRUS (CIFARIO)
Figure 5: Average Worst Case k-UAP accuracy for different k values for CIFARIO ConvSmall DNNs.
Conclusion
We present RABBit, a general framework for improving the precision of relational verification of DNNs through BaB methods specifically designed to utilize dependencies across executions. Our experiments, on various DNN architectures. and training methods demonstrate that RABBit significantly outperforms both SOTA relational and non-relational verifiers for relational properties. Although we focus on the worst-case UAP accuracy and top-k accuracy RABBit can be extended to properties involving different DNNs, such as local equivalence of DNN pairs Paulsen et al. (020 or properties defined over an ensemble of DNNs.
Acknowledgement
We thank the anonymous reviewers for their insightful comments. This work was supported in part by NSF Grants No. CCF-2238079, CCF-2316233, CNS-2148583.
10

---

References
Filippo Amato, Alberto López, Eladia María Peña-Méndez, Petr Vañhara, Aleš Hampl, and Josef Havel. Artificial neural networks in medical diagnosis. Journal of Applied Biomedicine, 11(2), 2013.
Ross Anderson, Joey Huchette, Will Ma, Christian Tjandraatmadja, and Juan Pablo Vielma. Strong mixed-integer programming formulations for trained neural networks. Mathematical Programming, 2020.
Stanley Bak, Hoang-Dure Tran, Kerianne Hobbs, and Taylor T. Johnson. Improved geometric path enumeration for verifying relu neural networks. In Shuvendu K. Lahiri and Chao Wang, editors, Computer Aided Verification 32nd International Conference, CAV 2020, Los Angeles, CA, USA, July 21-24, 2020, Proceedings, Part I, volume 12224 of Lecture Notes in Computer Science, 3000:/99788-00--5553 8\ 4. URL
pages 66-96.666666666666 96. Springer, 2020. doi: 1007/978 030- 53288- 8_4
Mislav Balunovic, Maximilian Baader, Gagandeep Singh, Timon Gehr, and Martin Vechev. Certi- fying geometric robustness of neural networks. H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL URL http//ttpps//proceddgg.... newrridiiiiinnnnns......... cc/
.ddf
Debangshu Banerjee and Gagandeep Singh. Relational DNN verification with cross exccutional bound refinement. In Forty-first International Conference on Machine Learning, 2024. URL
htttp:://pperreviex wwwp:arrwweeeeeeeforuuuuiieeee wwt.:e/oooornnt---000000----
Debangshu Banerjee, Avaljot Singh, and Gagandeep Singh. Interpreting robustmess proofs of deep neural networks. In The Twelfth International Conference on Learning Representations, 2024a.
URL httpss://peerrvveet wwwp:eeeeeoooomodddddddeeeeeeee
Debangshu Banerjee, Changming Xu, and Gagandeep Singh. Input-relational verification of deep neural networks. Proc. ACM Program. Lang., 8(PLDI), June 2024b. doi: 10.1145/3656377. URL
https://doi, http::///doi.or//111/ 10. 1145/3656677
Debangshu Banerjee, Changming Xu, and Gagandeep Singh. Scalable relational verification and training for deep neural networks. 2024c.
Rudy Bunel, Jingyue Lu, llker Turkaslan, Pushmeet Kohli, P Torr, and P Mudigonda. Branch and bound for piecewise linear neural network verification. Journal of Machine Learning Research, 21 (2020), 2020a.
Rudy Bunel, Ilker Turkaslan, Philip H. Torr, Pawan Kumar, Jingyue Lu, and Pushmeet Kohli. Branch and bound for piecewise linear neural network verification. J. Mach. Learn. Res., 21(1), jan 2020b. ISSN 1532-4435.
Rudy R Bunel, Oliver Hinder, Srinadh Bhojanapalli, and Krishnamurthy Dvijotham. An efficient non- convex reformulation of stagewise convex optimization problems. Advances in Neural Information Processing Systems, 33, 2020c.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learn- ing Research, pages 1310-1320. PMLR, 09-15 Jun 2019. URL htttps://proceedings mlr cot.n1oeen1.. vatess/v7//chhnn9/ html
press/ wttps//v///cll/
Ruediger Ehlers. Formal verification of piece-wise linear feed-forwand neural networks. In Interna- tional Symposium on Automated Technology for Verification and Analysis, 2017.
Claudio Ferrari, Mark Niklas Mueller, Nikola Jovanović, and Martin Vechev. Complete verification via multi-neuron relaxation guided branch-and-bound International Conference on Learning Representations, 2022, URL httttt::::/ppeeeeeeeeeeee http://wwwweeeeeeeeeeeeeeeeeeeee www./eeen/foiieeerooooo amfif 1oak

---

Aymeric Fromherz, Klas Leino, Matt Fredrikson, Bryan Parno, and Corina Pasareanu. Fast geo- metric projections for local robustness certification. In International Conference on Learning Representations, 2021. URL
Timon Gehr, Matthew Mirman, Dana Drachsler-Cohens Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. Ai2: Safety and robustness certification of neural networks with abstract interpretation. In 2018 IEEE Symposium on Security and Privacy (SP), 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Gurobi Optimization, LLC. Gurobi optimizer reference manual, 2018.
Enyi Jiang and Gagandeep Singh. Towards universal certified robustness with multi-norm training, 2024. BtLLhttss:///rxtttt org/ 2b000000000000
Haitham Khedr and Yasser Shoukry. Certifair: framework for certified global faimess of neural networks. Proceedings of the AAA/ Conference on Artificial Intelligence, 3777)8837-8225,, Jun. 2023.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Juncheng Li, Shuhui Qu, Xinjian Li, Joseph Szurley, J. Zico Kolter, and Florian Metze. Adver- sarial music: Real world audio adversary against wake-word detection system. In Proc. Neural Information Processing Systems (NeurlPS), pages 11908-1191 1918, 2019a.
Juncheng Li, Frank R. Schmidt, and Zico Kolter. Adversarial camera stickers: A physical camera- based attack on deep learning systems. In Proc. International Conference on Machine Learning, ICML,, volume 97, pages 3896-3904, 2019b.
Linyi Li, Jiawei Zhang, Tao Xie, and Bo Li. Double sampling randomized smoothing. In Ka- malika Chaudhuri, Stefanie Jegelka, Le Song. Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 13163-13208. PMLR, 17-23 Jul 2022. URL wwt.serrrr..reereeeeeeeeeeeeeeee
htttps://proceeding mlr press/ vwtp//i22aaa html
Zikun Liu, Changming Xu, Emerson Sie, Gagandeep Singh, and Deepak Vasisht. Exploring practical vulnerabilities of machine learning- based wireless systems. In 20th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2023, Boston, MA, April 17-19, 2023, pages 1801-1817. USENIX Association, 2023.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018. URL
net meet/forumm1dd=r2IlBBZ22
Anna P. Meyer, Yuhao Zhang, Aws Albarghouthi, and Loris D' Antoni. Verified training for coun- terfactual explanation robustness under data shift, 2024. URL https://arxiv: http://rxxvvorgg/// abs/24/3 03773
Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for provably robust neural networks. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume of Proceedings of Machine Learning Research, pages 3578-3586. PMLR, 10-15 Jul 2018. URL. Utt.httts::/pproeeedia v800mirrman10b html
mlr, .press/
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 1765-1773, 2017.
Mark Niklas Mueller, Franziska Eckert, Marc Fischer, and Martin Vechev. Certified training: Small boxes are you need. In The Eleventh International Conference on Learning Representations, 2023. URL htRps:tps:
12

---

Alessandro De Palma, Harkirat S. Behl, Rudy R. Bunel, Philip H. S. Torr, and M. Pawan Ku- mar. Scaling the convex barrier with active sets. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
Alessandro De Palma, Rudy R Bunel, Krishnamurthy Dj Dvijotham, M. Pawan Kumar, Robert Stanforth, and Alessio Lomuscio. Expressive losses for verified robustness via convex combinations. In The Twelfth International Conference on Learning Representations, 2024. URL httpp:// openreview www. htt///orrmmmi/ dearrrmm?ddmmzZZZAzkk.
Brandon Paulsen, Jingbo Wang, and Chao Wang. Reludiff: Differential verification of deep neu- ral networks. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, ICSE '20, page 714-726, New York, NY, USA, 2020. Association for Com- puting Machinery. ISBN 9781450371216.
doi: 10.1145/337781 1.3383337...
URL https
//doi.org/10.
1tt../or//1...14/3333333 11t.11111/33711111333 3380337
Brandon Paulsen, Jingbo Wang, Jiawei Wang, and Chao Wang. Neurodiff:: Scalable differential verification of neural networks using fine-grained approximation. In Proceedings of the 35th
JEEE/ACM International Conference on Automated Software Engineering. ASE '20, page 784-796, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450360 450367684. doi: 10.1145/3324884.. 63600. Uttttsts::///ot org 10. 10111114422222 33///333488444.00 .41424
Zhouxing Shi, Qirui Jin, Zico Kolter, Suman Jana, Cho-Jui Hsich, and Huan Zhang. Formal verification for neural networks with general nonlinearities via branch-and- annhhndd- 2024. URL
https:://peerrevee net//orrun?ddiivokwwWWk39
Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. Advances in Neural Information Processing Systems, 31, 2018.
Gagandeep Singh, Rupanshu Ganvir, Markus Püschel, and Martin Vechev. Beyond the single neuron convex barrier for neural network certification. In Advances in Neural Information Processing Systems, 2019a.
Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An abstract domain for certifying neural networks. Proceedings of the ACM on Programming Languages, ((PPL)), 2019b.
Shubham Ugare, Debangshu Banerjee, Sasa Misailovic, and Gagandeep Singh. Incremental verifica- tion of neural networks. Proc. ACM Program. Lang., 7(PLDI), June 2023. doi: 10. 10.1145/3591299
URL httpps://oi.. org/10. 1145/3555 3591299
Shubham Ugare, Tarun Suresh, Debangshu Banerjee, Gagandeep Singh, and Sasa Misailovic. Incre- mental randomized smoothing certification. In The Twelfth International Conference Learning Representations, 2024. URtphtttp:///peeeeeiiie
Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Efficient formal safety analysis of neural networks. In Advances in Neural Information Processing Systems, 2018.
Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. Beta-CROWN: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https: /ottps://operrevies net/ forum?id-ah'" wtt.iiddddayT11eBeeee
Haoze Wu, Teruhiro Tagomori, Alexander Robey, Fengjun Yang, Nikolai Matni, George Pappas, Hamed Hassani, Corina Pasareanu, and Clark Barrett. Toward certified robustness against real- world distribution shifts. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 537-553. IEEE, 2023.
Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. Cril: Certifiably robust federated learning against backdoor attacks. In International Conference on Machine Learning, pages 11372-11382. PMLR, 2021.
Changming Xu and Gagandeep Singh. Cross-input certified training for universal perturbations, 2024.
13

---

Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. Automatic perturbation analysis for scalable certi- fied robustness and beyond. In Proceedings of the 34th International Conference Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Kaidi Xu, Huan Zhang. Shiqi Wang, Yihan Wang, Suman Jana, Xue Lin, and Cho-Jui Hsich. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. In International Conference Learning Representations, 2021. URL
Yi Zeng, Zhouxing Shi, Ming Jin, Feiyang Kang, Lingjuan Lyu, Cho-Jui Hsieh, and Ruoxi Jia. Towards robustness certification against universal perturbations. In The Eleventh International Conference on Learning Representations, 2023. URL htttp:://ppenreiieeet net/ mert/forrmmii
7GEvPKxjtt
Huan Zhang, Tsui- Wei Weng, Pin- Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems, 31, 2018.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsich. Towards stable and efficient training of verifiably robust neural networks. In Proc. International Conference on Learning Representations (ICLR), 2020.
Huan Zhang, Shiqi Wang, Kaidi Xu, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. neural network verification. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022a. URL hntps: /ttLttt////oooeeeeeeeee httpa//wwoe.....eoooooraaaaaaaaa
General cutting planes for pouudd-ropagation-baaee
Yuhao Zhang. Aws Albarghouthi, and Loris D'Antoni. Bagflip:
certified defense against data poisoning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022b. URL htt.::ttt/////tt //perrvve//ee net/ Forum/Xd=2Adk4455222
14

---

A Formal encoding of relational properties
A.1 k-UAP verification
Given a set of k points x= {x1,.... xk] where for all i [k], xj E I and e E R we can first define individual imput constraints used to define Loo input region for each execution Vi E [k]. (xx)
||x] Xa I e. We define & (x1,,.... ".i....,Xp .xix) follows:
Φ (xi
x
(x] -j -x-)
(2)
(LE((H)((CE)
Then, we have the input specification as $(xi,.... xi) N (n A (xi....,,
,xx)
Next, we define (x x as conjunction of l clauses each defined by v (y:) where yi N [xx)). Now we define v (y1) (ciJ"' " yı 2 0) where Cij E 1 is defined as follows
{
if a * j and is the correct label for if = and is not the correct label for yi otherwise
Va E [mil.cija
x In this case, the tuple of inputs (xi,... (xi,...,xx) , satisfies the input specification .. iff for all E [kt]. x = x; + 5 where E TRo and ||6]|< S 6. Hence, the relational property (Ø, V) defined above verifies whether there is an adversarial perturbation 5 E R with ||8 lo I that can misclassify all inputs. Next, we show the formulation for the worst-case UAP accuracy of the k-UAP verification problem described in section 3 Let, for any 5 E R0 and ||0]| e, (()) denotes the number of clauses (1²)) V that are satished. Then μ(o) is defined as follows
1
(N + 5)) True otherwise
(8)
(4)
0
(66)
24 (6)
(5)
Since ' (x, + +8) is True iff the perturbed input xą + 8 is correctly classified by N, for any 8 E 10 and |0]| S (0)) captures the number of correct classifications over the set of perturbed
inputs (x1 + 6 8,..,, Xx 83. The worst-case k-UAP accuracy Mo 22, V for (#), V is as follows
Mo VD
min ((5)
8ER*O, [81|
B Details of strong bounding
For fixed linear approximations {(L1, bi), (In, b.)) corresponding to executions of N if the optimal value (* of the following linear program then n executions do not have common peturbation (from Theorem B.3. Banerjee and Singh (2024).
min s.t. | S
L T.;(x; 5) + b, S
Vi E [n]
(7)
Now in the first step, we compute the Lagrangian dual of the linear program from Eq. Lagrangian Dual is as follows where for all E [n]. A; 2 are Lagrange multipliers.
The
max min (1- Σ x U.A 16
Σ N x (L (xi + + bi)
i=1
We set the coefficient of the unbounded variable & to to avoid cases where min (1
YRR, ( Se
EL A,) t E A x (L + +6) +b)) = -00 This leads to the following Lagrangian
Dual form
max min Σ A [L (xi + +) + b.) where Σ A =
OSA; | im1
15

---

Now for every subproblem, replacing the branching constraints by introducing dual variables results in the parametric linear approximations N specified by b,),,, for each execution i [n]. Then the Lagrangian Dual with the parametric linear approximations ((L, by (an (1))...))...... (I, Bn), bn (a,, Bn.)) is as follows
min
de x (Ls (x) (..,,, where
X =1
USAL | 1o
Theorem 4.1. (fminF(s) m -e xe EiEs & x (,,, + - Lies A x a, a, Bi) 2 20
then executions do have a common perturbation 8 E R with |10||cc 0c S
Proof. First, we show that min
> (.)
Σ d L,(a
X X a (a,
min E A x (L.(a,, B.) (xl + 8) + b.(a,.
min
Σ x La B.),,,B.) (8) + Σ \ x (be(as,, B:) + Bi)
181| Se
d x (; Bi) + min
X L, B:)
Sc 1=1
1=1
Σ A (a - cx Σ d x L.(as, B:)
Using Hölder's Inequality
(8)
For fixed a: Br, the optimal solution of the LP in Eq. 7 and subsequently of the Lagrangian gives us
Σ d (.)(a..B.) + 5) + b.(a:,
max min
0.A: | 1=1
B.) (xi (x; +6) + (a. [6 16 Se Isin
min max
provided
A =
(9)
For each subproblem, for all B;
min
max c T N(x + +6) 2 min max
(L)(a,,,))) + +) + (,, B.))
Se Isisn
[6]] 1si5n
Hence,
min max c NN + 8)
88 Se Isign
max min max
(a)(a;,B)) +6) by (a.,
a.P; Sclsinn
"
max max min
Σ x (L, B:)² Tf (x; + 8) b.
where Σ X 1 from Eq. 9
a.B, OSAi [e]] 1=1
Σ Aa x aa Bi) - -cx Σ da B:)ll, From Eq
max A.O.
8
(10)
==1
Finally, if min B -cx Eies ) L, B:) Eies ) a, ai B.) 2 then,
min max q T N + 8) 2 for all subproblems in F(S) using Eq. 10 5 ISiS
Since, for all subproblems
min maxigisn c N(x 8) 0. V 1 (N[(x; + 6)) holds for all 16
8 E _e and |0 < e i.c. there does not exist any common perturbation.
16

---

C Details of strong branching
Theorem 4.2. For any a. if La, B) E = and b(a,8) E E then minj Se (L
(x)
(xx-00 6) + b(a, - [ B) Lell (L(a, B) L,)xx x + b)a
Proof.
(x min (L( Lx 8) b) Sc
min (L(a, B) L,) bla B) + (L B) L()xxx
11
= -b(a,8) + () L.)²x x + min
LL(a, B) - L.)² || x
b (L) L - x (a, B) - -I))I Using Hölder's Inequality
C.1 Projected gradient ascent
For each a4 B,, after each step of gradient ascent (for maximization problem), clip a; B, values the corresponding ranges [ uf 14 u respectively. This is similar the approach used in the SOTA non-relational bound refinement a, 3 3-CROWN Wang et al. (2021). Since A [0, 1] and sigmoid((z; (2;) where x, R. For any values of (#1.---, 2&) R
Σ A we replace X; Eiz sigmoid(=z.)
the corresponding (AI...., Ax) satisfy X E [0.1] and E A; = 1. We then apply gradient ascent (for maximization problem) on (#1...., Fg) without any constraints.
17

---

D DNN Architectures
D.I DNN Architectures:
Table 2: DNN Architecture Details
Dataset
Model
Type
Train
Layers
# Params
ConvSmall
Conv
Standard
ConvSmall
Conv
DifAA
80k
ConvSmall
Conv
CITRUS
80L
ConvBig
DitTAl
ConySmall
Conv
Standard
80K
ConySmall
ConvSmall
Conv
SABR
80K
ConySmall
Conv
CITRUS
Cony Big
Conv
DITAI
2.5M
D.2 Accuracies for Evaluated DNNs:
Table 3: Standard top-1 accuracy for evaluated DNNs
Dataset
Model
Train
Accuracy (%)
ConvSmall
Standard
62.9
ConvSmall
DiffAI
45.9
CIFAR10
ConvSmall
SABR
63.3
ConvSmall
CITRUS
63.9
ConvBig
DiffAI
53.8
ConySmall
Standard
97.7
ConySmall
DiffAI
96.8
MNIST
ConySmall
SABR
97.9
ConySmall
CITRUS
98.5
18

---

E
Comparison of RABBit with Ra VeN
Table 4: Comparison of RABBit vs RaVeN and RACoon
Dataset
Network
Training Method
RaVeN
RACoon
RABBit
MNIST
ConvSmall
DiffAl
0.13
66.0
63.0
82.0 (+19.0)
MNIST
ConvSmall
SABR
0.15
51.0
50.0
66.0 (+15.0)
MNIST
ConvSmall
CITRUS
0.14
82.0
80.0
89.0 (+7.0)
CI
ConvSmall
SABR
4.0/255
48.0
48.0
55.0 (+7.0)
CIFARIO
ConvSmall
CITRUS
2.0/255
79.0
78.0
82.0 (+3.0)
F RABBit Efficacy Analysis for top-k accuracy
Table 5: Verified top-2 accuracy for RABBit vs baselines
Network
Thaning Matked
S-LROWEN Arg. Time boe))
RACaE
S4SIOWS
KASli
Avg. Acc.
Arg. Acc. (9) Ny. Taxe (ee)
Jog. Aoc. (9)
Arg. Tiae fox))
Arg. Avo. (9) Neg. Time (Se))
14.0
33
6-300
G Average Improvement in t* with Strong Branching
Table 6: Average Improvement in 1 with Strong Bounding
Training Method
Perturbation Bound (e)
RACoon
a, CROWN
Structure
Avg. Improvement (%)
95% CI
Avg. Improvement (%)
95% C1
ConySmall
DiFAI
5/255
108.7
[93.9, 126.1]
102.5
192.7. 115.4]
CIFAR
ConvSmall
CITRUS
2/255
77.9
(75,3,81.6
86.9
[86.2, 88.1]
ConySmall
DiFAI
5/255
57.7
[55.5, 60.2]
54.4
[53.0, 56.0]
MNIST
ConlSmall
CITRUS
2/255
[39.8, 41.9]
37.1
[36.4, 37.8]
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Section for main claims and contributions. The main claims made in this section and the abstract reflect the paper's 's scope and contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See end of Section for the limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See experimental setup in Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification:
We provide the code to replicate the main results of this paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please See Section the NeurIPS code and data submission guidelines (https: //nips
cc/public/ htabblicgguies////e Seaeddds//ooeSuumiiiee www.ssssooooiioii.ooooiiiiiiiiil for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See Section the NeurlPS code and data submission guidelines (https://nips http://wwipp..cpp.l.eeppp http://cpcpppp.guuuu/o///ee////e ggwides/CodeSuue coclbbiissionn ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See experimental setup in Section 6
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: RABBit deterministic verifier. The experiment "Evaluating Bound Im- provement" (Section 6 is the only randomized experiment in the paper. We report the mean and 95% confidence intervals of the experiment Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
22

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See experimental setup in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htwp://pww....eeee....o/oo///o/e
Answer: [Yes]
Justification: Submission meets all ethical guidelines after authors reviewed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section 1 and Section for societal impacts the work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No data or models with high risk for misuse were used.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have only utilized publically available code, models, and datasets and properly cited all relevant works.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the code to replicate the main results of this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing nor human research with subject participants.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No crowdsourcing nor human research with subject participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "HF8T6QNgp8", "reviewer": "Reviewer_eacX", "summary": "This paper addresses the problem of verifying certain DNN properties that depend on multiple executions of the DNN, known as “relational verification.” The example used throughout the paper is verifying the k-UAP problem, which aims to confirm the absence of a universal adversarial perturbation for a given DNN. This problem requires relational verification because in order to compare the perturbations across multiple inputs, there must be multiple executions of the DNN considered during the verification process. This introduces additional difficulties compared to traditional single execution branch and bound techniques. The authors tackle this problem by introducing a “product DNN”, which is essentially duplicate DNNs for each of the executions in consideration, and propose stronger branching and bounding algorithms on this DNN instance. The results from the branching and bounding algorithms are then used in an MILP formulation, which they call RABBit. The experimental evaluation across pre-trained DNNs on MNIST and CIFAR-10 demonstrate that RABBit outperforms traditional DNN verifiers as well as the individual branching or bounding algorithms independently, highlighting the importance of combining the strengths of both.", "questions": "- In Table 1, how long does it take to run each verification?\n- In Table 1, why the non-relational verifier alpha-beta-crown outperformed the STOA relational verifier RACoon? Is it because alpha-beta-crown uses BaB?\n- Are there relational properties other than UAP that can be verified and evaluated?", "limitations": "Yes, the authors addressed the limitations of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The paper addresses an interesting problem. The comparison with prior approaches demonstrates that the proposed approach works well.", "weaknesses": "The paper suffers from a number of grammatical errors, particularly run-on sentences (e.g., page 4 line 171)  and improper use of commas which makes the paper difficult to read. Additionally, the juxtaposition of the actual contributions of the paper compared to prior work in verification is not always clear.\n\nAnother major issue throughout the paper is the presentation of “relational verification” as a concept, while only one example of the use case of relational verification (UAP verification) is presented. For example, on page 1 line 21, “relational properties common in practical situations” is never expanded on—what are the practical situations besides UAP? On page 8 line 356, “10 relational properties” are mentioned but are not defined in the paper. Are there 10 different versions of UAP or are there 10 unique use cases of relational verification in general?\n\n## Minor notes\n\n- Page 2 line 80: “constructing ‘product DNN’” → “constructing a ‘product DNN’”\n- Page 3 line 123: “upper bound that contain…” → “upper bounds that contain”\n- Page 4 line 184: “Let for all i \\in S…” → “For all i \\in S, let…”\n- Page 4 line 186: “proves absence of common perturbation” → “proves the absence of a common perturbation”\n- Page 4 line 192: “Formally, product DNN is a function” → “Formally, a product DNN is a function”\n- Page 5 line 216: “do not have common perturbation” → “do not have a common perturbation”\n- Page 5 line 217: “The detailed proof in the Appendix B” → “The detailed proof is in Appendix B”\n- Page 5 line 224: “m/n subproblem per execution” → “m/n subproblems per execution”\n- Page 6 line 270: “Alog. 1” → “Algo. 1”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: In Table 1, how long does it take to run each verification?**\n \n**R1:** Please refer to the answer to Q2 in the common response.\n \n**Q2: In Table 1, why the non-relational verifier $\\alpha,\\beta$-CROWN outperformed the SOTA relational verifier RACoon? Is it because $\\alpha,\\beta$-CROWN uses BaB?**\n \n**R2:** SOTA relational verifier RACoon is incomplete and utilizes a single bounding step without branching. Although RACoon is significantly more precise than $\\alpha$-CROWN, the bounding step used in $\\alpha,\\beta$-CROWN, $\\alpha,\\beta$-CROWN outperforms it by exploring a large number of branches with BaB. We discuss this limitation of RACoon in lines 47--48 of the paper.\n \n\n**Q3: Are there relational properties other than UAP that can be verified and evaluated?**\n\n**R3:** Please refer to the answer to Q1 in the common response.\n\n**Q4: Are there 10 different versions of UAP or 10 unique use cases of relational verification in general?**\n\n**R4:** We consider 10 different instances of the UAP verification problem where each instance is defined on a set of 50 images as described in line 356. For the details on another relational property that can be verified by RABBit, please refer to the response to Q1 in the common response.\n\n**Q5: Grammatical errors and typing mistakes.**\n\n**R5:** We apologize for the grammatical and typing errors and will correct them in the revised version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "243T2eEqkd", "reviewer": "Reviewer_uEoH", "summary": "The paper proposes a branch and bound technique for the relational verification of neural networks.\nTo this end, the authors build upon RACoon (Banerjee and Singh [2024]) and $\\alpha,\\beta$-CROWN (Wang et al. [2021b]).\nThe work describes two branching mechanisms (strong bounding and strong branching) that form a symbiosis\nin the authors' new algorithmic framework RABBit.\n\nReferenced citations below:\n- [R1] Sälzer, Marco, and Martin Lange. \"Reachability in Simple Neural Networks.\" Fundamenta Informaticae 189.3-4 (2022): 241-259.", "questions": "(Q1) Do I correctly assume that your comparison with alpha-beta-CROWN uses a problem formulation based on a product NN and the k-UAP linear constraint formulation?\n\n**POST REBUTTAL:**  \nThe authors clarified this.\n\n(Q2) Do you have experiments w.r.t. other relational properties? If not, would you be willing to adjust the title and content of the paper to focus on UAP properties? -- In either case, I would be willing to raise my score to Accept.\n\n**POST REBUTTAL:**  \nSee (O10)", "limitations": "Concerning limitations, my main critique is the focus on Universal Adversarial Perturbations in the experimental section (see (O10)).\n\n**POST REBUTTAL:**  \nDue to a lack of access to NVIDIA GPUs, I was not able to evaluate the code submission.\nHowever, aDbJ has confirmed that the (updated) code submission allows execution of the code.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(S10) The authors address an important problem since universal adversarial perturbations are a much stronger attack-vector than instance-specific adversarial attacks.\n\n(S20) While the idea initially seems very straight-forward, the paper introduces interesting and subtle concepts with its distinction between strong branching and bounding which are\n      shown to be useful in practice. In particular, the paper's approach is, to the best of my knowledge, novel.\n\n**Post Rebuttal:**  \nI consider this distinction between strong branching and bounding, together with the experiments that show the approach scales to a large number of compared executions, the major contribution of the paper.\n\n(S40) The paper is mostly well-written and provides a good overview on the approach.", "weaknesses": "Major Opportunity for Improvement:\n\n(O10) In the introduction of the evaluation the paper states \"We evaluate [..] on multiple relational properties, DNNs, and datasets.\"\n      Unfortunately, the paper's evaluation focuses entirely on the k-UAP property. While this is an interesting property worth studying,\n      it would be necessary to evaluate the approach on additional relational properties to substantiate the claim that the tool is a generally applicable relational verification tool.\n      At least, it would be helpful to discuss necessary modifications to adapt the approach to other relational properties.\n      Alternatively, the paper's title and story could be focused on Universal Adversarial Perturbation.\n\n**POST REBUTTAL:**  \nThe authors provided experiments on a second relational property.\nThe results seem less impressive to me in this case. Nonetheless, this demonstrates the applicability to other relational properties.\nI get the impression the approach is particularly valuable when comparing a large number of executions (as done for k-UAP with k>=20 in the paper).\n\nIn my initial comment yesterday I said I would raise my score to Weak Accept if the code submission problems are resolved.\nI gave this some more thought:\nWhile the results are less impressive in the new experiments, I do believe the community is better off with a verifier that can solve relational properties for large $k$ than it is without such a verifier. Hence, I have decided to raise my score to **Accept**.\n\nMinor Opportunities for Improvement:\n\n(O1) In Line 223 you mention that simultaneously branching over all executions leads to the processing of only $m^{\\frac{1}{n}}$ subproblems.\n     It would be helpful if you provide some more intuition to the reader about this. My understanding is that we obtain this number, because\n     a split for one execution effectively splits the state space for all executions and thus if we split each execution into l subproblems this\n     leads to l^n (for n executions) subproblems that have to be handled. This could be said more explicitly here.\n\n**POST REBUTTAL:**  \nThe authors proposed sufficient improvements.\n\n(O2) In Line 324 you note that \"all complete non-relational verifiers are also incomplete for relational properties since they do not track any dependencies between executions.\"\n     In this generality, this statement seems factually wrong. Obviously, completeness also depends on the checked property, however there is\n     a large class of properties which can be encoded as linear constraints over a product network. In this case, I see no reason why the completeness guarantee of a non-relational\n     verifier should not equally hold for (linear constraint based) relational properties.\n     For example Appendix A.1 describes how k-UAP can be formalized via linear constraints.\n     While dependency tracking is very useful for efficient relational NN verification, it does not seem to be strictly necessary.\n     Can the heuristics employed by RaBBit and RACoon lead to incompleteness when it comes to providing exact UAP bounds?\n\n**POST REBUTTAL:**  \nI am not convinced of the authors' answer in this respect.\nIn Appendix A.1 the authors describe how the k-UAP property can be encoded with linear constraints. Maybe I am missing something here, but even with the authors' clarification, it is not clear to me why verifying this specification on a product NN would not yield a complete verifier for k-UAP. Again, I wholeheartedly agree that tracking relational dependencies is *useful*, but I still do not see why it is *necessary*. As stated in the authors' response (Q3), completeness for UAP is then \"just\" a matter of checking all possible subsets (which is again an issue independent of dependency tracking).\n\n**POST COMMENTS:**  \nIn the comments the authors have sufficiently resolved this issue: Indeed, the constraints outlined in A.1 already are cross-executional constraints.\n\nMinor notes:\n- Line 52/53: \"with *a* cross-executional bound method\"\n- Line 104: DNN verification is not only NP-hard, but actually (exactly) NP-complete [R1]\n- Line 157: \"...it is possible to *statistically* estimate...\"\n- Line 336: You mixed up the citations for relational / non-relational verifiers here", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Do you have experiments w.r.t. other relational properties? If not, would you be willing to adjust the title and content of the paper to focus on UAP properties? -- In either case, I would be willing to raise my score to Accept.**\n\n**R1:** Please refer to the answer to Q1 in the common response.\n\n**Q2: \"All complete non-relational verifiers are also incomplete for relational properties since they do not track any dependencies between executions\" - Completeness depends on the specific relational property.**\n\n**R2:** We agree that completeness depends on the specific relational property being verified. In this case, we refer to relational properties such as robustness against UAP, where it is important to track dependencies between perturbed inputs across different executions to achieve completeness. Existing complete verifiers like $\\alpha,\\beta$-CROWN, which are designed for verifying input-specific robustness, will be incomplete if they verify each execution independently without tracking these dependencies. Please refer to lines 168-173 of the paper, where we describe a scenario in which input-specific adversarial perturbations exist for individual inputs, but no common adversarial perturbation exists. We will rephrase the highlighted sentence to clarify this.\n\n**Q3: Can the heuristics employed by RABBit and RACoon lead to incompleteness when it comes to providing exact UAP bounds?**\n\n**R3:** RACoon does not use any branching and is incomplete. In RABBit, for scalability, we greedily select subsets of executions and run strong bounding only on the selected subsets. For this reason, the current implementation of RABBit is also incomplete and only computes a sound lower bound on the worst-case UAP accuracy. However, it is possible to make RABBit complete by considering all possible subsets of executions and then formulating the MILP formulation.\n\n**Q4: Do I correctly assume that your comparison with $\\alpha,\\beta$-CROWN uses a problem formulation based on a product NN and the k-UAP linear constraint formulation?**\n\n**R4:** Yes, $\\alpha,\\beta$-CROWN is applied to the product DNN. However, unlike the strong bounding approach proposed in RABBit, the bounding method of $\\alpha,\\beta$-CROWN does not track any dependencies. As a result, even when $\\alpha,\\beta$-CROWN is executed on the product DNN, it computes the lower bound $\\pmb{c_i}^TN(\\pmb{x_i} + \\pmb{\\delta})$ for each execution independently and loses precision.\n\n**Q5: Questions about $m^{1/n}$ and $m/n$ subproblems.**\n \n **R5:** Please refer to the answer to Q3 in the common response. We will add a more explicit description in the revised version.\n\n**Q6: Minor notes on the presentation of the paper.**\n\n**R6:** Thanks for pointing out. We will add the suggested changes in the revised version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gCZZX8vtLQ", "reviewer": "Reviewer_dFSH", "summary": "The paper presents RABBIt, a general framework for improving the precision of relational verification of DNNs through BaB methods and cross-execution dependencies. RABBIt uses strong bounding and strong branching methods, outperforming BaB methods like $\\alpha$-$\\beta$-CROWN and RACoon, which only leverages cross-execution dependencies. Experiments show that RABBIt significantly outperforms SOTA verifiers for relational properties on various architectures trained by different training methods.", "questions": "1. In lines 221-224, I cannot follow how to derive $m^{1/n}$ and $m/n$. Please give me an example.\n2. In the MILP encoding of RABBIt, What are the new terms/constraints compared to RACoon? \n3. Is the strong branching method only to guarantee that RABBIt's bound is tighter than $\\alpha$-$\\beta$-CROWN bound?\n\nComments:\n1. Missing related work, [1] provides verification for counterfactual explanation robustness, which is also a relational property of neural networks. \n\n[1] Verified Training for Counterfactual Explanation Robustness under Data Shift. Anna P. Meyer, Yuhao Zhang, Aws Albarghouthi, Loris D'Antoni", "limitations": "The paper discusses its limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "1. Strong bounding and strong branching methods are technically solid.\n2. Experiments show that RABBIt significantly outperforms SOTA verifiers for relational properties on various architectures trained by different training methods. The paper also presents extensive ablation studies on strong branching, strong bounding, and other hyper-parameters.", "weaknesses": "1. The paper claims that strong bounding provides tighter bounds than RACoon and $\\alpha$-$\\beta$-CROWN. Although the experiments validate this claim, the paper offers no proof. This claim might not hold, especially considering the $k_t$ and the time limit used in Algorithm 1.\n2. Another concern is the poor presentation of this paper.\n   * In Lines 49-50, the bullet points a), b), and c) do not correspond to the order of challenges introduced earlier in the paragraph. c) should come first, then b), and finally, a).\n   * In Line 179, the MILP instance appears abruptly. It is confusing because the reader might think of it as the MILP problem introduced in Line 36. The purpose of the MILP instance only becomes clearer after introducing the k-UAP accuracy problem. \n   * However, the k-UAP accuracy is not adequately introduced and abruptly appears in Line 227. \n   * In lines 221-224, I cannot follow how to derive $m^{1/n}$ and $m/n$. Please give me an example.\n   * In Section 4.2, $N(x_i+\\delta)$ is a vector but $L_j^T(x_i+\\delta)$ is a scalar. Does the paper want to express $c_i^T N(x_i+\\delta)$?\n   * In Section 4.2, the strong branching method seems to me only a preprocessing rather than a counterpart to the strong bounding method. Is the strong branching method only to guarantee that RABBIt's bound is tighter than $\\alpha$-$\\beta$-CROWN bound?\n   * The presentation in Section 4.2 could be better. For example, the paper should relentlessly remind readers that the goal of Lines 243-267 is to compute the lower bound $b_j^*$.\n   * In Line 319, the index $j$ is not quantified. \n   * In Line 336, Wang et al. should be non-relational, and Banerjee and Singh should be relational.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: In lines 221-224, I cannot follow how to derive $m^{1/n}$ and $m/n$. Please give me an example.**\n \n **R1:** Please refer to the answer to Q3 in the common response.\n \n**Q2: In the MILP encoding of RABBIt, What are the new terms/constraints compared to RACoon?**\n \n**R2:** The MILP encoding in RABBit includes additional constraints derived from both strong bounding and strong branching, which are not utilized in RACoon.\n- **Strong branching constraints:** For each target $L_{i}^{j}$, we add a new lower bound constraint $o_i \\geq L_{i}^{jT}(\\pmb{x_i} + \\pmb{\\delta}) + b^{j}_{i}$ where $b^{j}\\_{i}$ is obtained by strong branching and $o_i = \\pmb{c_i}^{T}N(\\pmb{x_i} + \\pmb{\\delta})$. This is discussed in the line 319 of the paper. \n- **Strong bounding constraints:** Suppose for any subset of executions $S \\subseteq [k]$, strong bounding proves the absence of common perturbation. We add the following lower bound constraint $\\sum_{i \\in S} z_{i} \\geq 1$. Here $z_i = (o_i \\geq 0)$ is a binary variable that indicates whether the constraints $\\pmb{c_i}^{T}N(\\pmb{x_i} + \\pmb{\\delta}) \\geq 0$ is satisfied or not. The lower bound constraint $\\sum_{i \\in S} z_{i} \\geq 1$ follows from the fact for any $\\|\\pmb{\\delta}\\|_{\\infty} \\leq \\epsilon$ at least one execution from $S$ remains correctly classified. This is discussed in the lines 313 - 316 of the paper. \n\n \n**Q3: Strong bounding provides tighter bounds than RACoon and $\\alpha,\\beta$-CROWN. Although the experiments validate this claim, the paper offers no proof. This claim might not hold, especially considering the $k_t$ and the time limit used in Algorithm 1.**\n\n**R3:** Strong bounding is a BaB method that uses cross-executional bounding as the bounding step. Given RACoon uses the same bounding methods and does not employ branching, for the same time limit, strong bounding will be at least as precise as RACoon. However, as correctly pointed out, for some cases $\\alpha,\\beta$-CROWN can outperform strong bounding. However, our experiments show that for practical scenarios, strong bounding outperforms $\\alpha,\\beta$-CROWN. \n\n**Q4: In Section 4.2, the strong branching method seems to me only a preprocessing rather than a counterpart to the strong bounding method. Is the strong branching method only to guarantee that RABBIt's bound is tighter than $\\alpha,\\beta$-CROWN bound?**\n\n**R4:** Strong bounding can only prove the absence of common perturbation for a set of executions $S \\subseteq [k]$. Hence, with strong bounding, we can only show at least 1 execution from $S$ is correctly classified. However, this approach is suboptimal for cases where the number of correctly classified executions from $S$ is $> 1$. In contrast, strong branching extracts the same target linear approximation for all subproblems, allowing us to formulate an efficiently optimizable MILP instance with these targets. This MILP instance can address cases where more than one execution from $S$ is correctly classified.\nAlso, strong branching allows us to explore more branches per execution as explained in answer to Q3 in the common response. Strong branching is not a preprocessing step and is important for improving the precision of RABBit, as demonstrated by the results in Table 1 of the paper.\n\n**Q5: Missing related work.**\n\n**R5:** Thanks for pointing out. We will add it to the revised version of the paper.\n\n**Q6: In Section 4.2, $N(\\pmb{x_i} + \\pmb{\\delta})$ is a vector but $\\pmb{L_{j}}^{T}(\\pmb{x_i} + \\pmb{\\delta}) + b^{*}\\_j$ is a scalar. Does the paper want to express $\\pmb{c_{i}^{T}}N(\\pmb{x_i} + \\pmb{\\delta})$?**\n\n**R6:** Thanks for pointing out. In section 4.2, it should be $\\pmb{c_{i}^{T}}N(\\pmb{x_i} + \\pmb{\\delta})$ instead of $N(\\pmb{x_i} + \\pmb{\\delta})$. We will correct it in the revised version of the paper.\n\n**Q7: Presentation of the paper.** \n\n**R7:** We will provide a formal definition of worst-case UAP accuracy and the MILP optimization problem used to compute it before they are cited. We will also correct the citation issues noted in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9nX1DjMJKo", "reviewer": "Reviewer_aDbJ", "summary": "The paper proposes bound-tightening techniques for verifying the absence of universal adversarial perturbations for a neural network. The tightened bounds are leveraged in a MILP encoding to perform verification.", "questions": "### Questions\n1. I am confused by the statement \"Each MILP instance gets a timeout of 5 minutes\" (line 351). With $k=50$ and $k_t=20$ for MNIST, 5 minutes per MILP already add up to 100 minutes (2x the timeout) at the end of the first for loop (line 19), before Algorithm 1 considers the timeout in line 20. What are the actual runtimes of RABBit?\n2. Table 1: How were the perturbation radii chosen? \n3. The worst-case UAP certified by RABBit in Table 1 is higher than the standard accuracy (Table 3) for several networks (CIFAR10: ConvSmall+DiffAI, ConvSmall+SABR, ConvSmall+CITRUS, ConvBig+DiffAI, MNIST: all networks). In my understanding, UAP should always be lower than standard accuracy (UAP = \"the rate at which N misclassifies when the same adversarial perturbation is applied to all inputs from the distribution\", Lines 153-154). What do the reported UAPs actually indicate?\n4. Besides the average UAPs \"Effectiveness of RABBit\" and the \"Time vs UAP Accuracy Analysis\" experiments, what are the standard deviations, minima, maxima, and 25%, 50%, and 75% quartiles?\n\n### Suggestions and Typos\n- The abbreviation SOTA is relatively well known, but I would still suggest introducing it. \n- Line 14: space between \"diagnosis\" and Amato et al. \n- Line 14 and throughout: Use citep instead of citet. \n- Line 15: What does it mean to \"understand [...] their reliability\"?\n- Line 17: The references Potdevin et al. [2019] and Wu et al. [2023b] seem out of place here since they do not introduce adversarial attacks.\n- Line 18: Sotoudeh and Thakur [2020] do not study adversarial attacks.\n- Lines 23-26: The discussion in these lines is paraphrasing the discussion in the introduction of Banerjee and Singh [2024]. You should add a citation for this, for example: \"As discussed by Banerjee and Singh [2024], recent studies [Li et al, 2019a] emphasize ...\"\n- Line 52: with *a* cross-executional bound method\n- Related Work: I suggest adding a discussion of certified adversarial training approaches, such as [Balauca et al., 2024; De Palma, 2024] since this field is closely related.\n- Line 68: prove *a* property \n- Line 104: add a justification or reference for the NP-hardness of NN verification. \n- Lines 127-128: add a citation for parametric linear relaxations\n- Line 132: $\\alpha,\\!\\beta$-CROWN *converts*\n- Line 136: and *refines* the parameters\n- Line 136: last $\\alpha,\\!\\beta$ also bold?\n- Line 168: for *a* $k$-UAP problem\n- Lines 169-170: bold $\\boldsymbol{\\delta}$ as in the Preliminaries Section?\n- Line 172: highlight*ing*\n- Line 185: Then *if* the optimal value $t^\\ast \\geq 0$ this proves the absence ...\n- Lines 185-187: It does not become very clear here that $||\\delta||_\\infty \\leq \\epsilon$ and the following equations are constraints to the maximum in line 185. I strongly suggest putting both the max and the constraints in an equation environment. The same also holds for other maximisations and minimisations. \n- Line 191: In the remainder of the paper, the number of executions is $k$, not $n$. \n- Line 192: *the* product DNN \n- Line 195: adapt or adopt?\n- Line 197: add a description of what the BaBSR score estimates.\n- Line 216: have *a* common perturbation. \n- Line 224: *exploring* $\\frac{m}{n}$ subproblem*s*.\n- Line 225: absence of *a* common perturbation.\n- Line 236: compute *a* valid lower bound.\n- Line 270: *Algo*.\n- Line 310: Then finding (no *the*)\n- Line 311: $min_{||\\delta||_\\infty \\leq \\epsilon}$?\n- Line 317: denote *the* set\n- Figure 2: the title and axis descriptions are barely legible. \n- Line 413: invalid DOI\n- Line 416: Where published?\n- Line 422: \"jan\", the other references don't have a month\n- Line 448: there is a published version at ICLR 2015 for this article\n- Line 495: Conference missing. \n- Lines 510-517: Duplicate entry\n- Line 529: Where published?\n- Bibliography: incoherent links/no links, sometimes DOI and URL. \n- Line 602: Projected gradient *ascent*\n- Table 2: What are the full architectures or where can I find them?\n\n[Balauca et al., 2024]: Stefan Balauca, Mark Niklas Müller, Yuhao Mao, Maximilian Baader, Marc Fischer, Martin T. Vechev:\nOvercoming the Paradox of Certified Training with Gaussian Smoothing. CoRR abs/2403.07095 (2024)\n[De Palma et al., 2024]: Alessandro De Palma, Rudy R Bunel, Krishnamurthy Dj Dvijotham, M. Pawan Kumar, Robert Stanforth, Alessio Lomuscio: Expressive Losses for Verified Robustness via Convex Combinations. ICRL 2024. https://openreview.net/forum?id=mzyZ4wzKlM", "limitations": "The authors adequately discuss limitations and social impact.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "- The paper addresses an interesting problem: verifying the absence of universal adversarial perturbations. \n- It introduces a novel algorithm that combines ideas from previous neural network verifiers. \n- It also introduces a novel bound computation approach that optimises only the offsets of the linear bounds (Strong Branching). \n- The proofs of the theoretical results are valid. \n- The experiments could demonstrate that the proposed verifier (RABBit) outperforms the baselines but I have some concerns regarding the timeouts (see Questions). \n- The authors provide code in the supplementary material (but see Weaknesses)", "weaknesses": "### Baselines\nThe non-relational verifier baselines CROWN [Zhang et al. 2018], $\\alpha$-CROWN [Xu et al., 2021], $\\alpha,\\beta$-CROWN verifier [Wang et al., 2021b] are no longer state of the art. These baselines were outperformed by GCP-CROWN [Zhang et al., 2022] and MN-BaB [Ferrari et al., 2022]. To justify the claims on outperforming the non-relational SOTA, the paper should compare against GCP-CROWN and/or MN-BaB. \n\n### Sensitivity of Results\nFor the \"Effectiveness of RABBit\" and the \"Time vs UAP Accuracy Analysis\" experiments, the authors use ten randomly sampled problem instances but only report the average results. It remains unclear how sensitive the results are to the sampling. \n\n### Presentation\n- The paper is not sufficiently self-contained. In particular, it does not provide sufficient details on RACoon which it builds upon. \n- I find the structure of the paper confusing. I only understood Section 4 after reading Section 5. I think presenting Algorithm 1 before Strong Bounding and Strong Branching would be more clear.\n- Important mathematical details are spread throughout the paper. For example, the maximisation in Theorem 4.1 has additional constraints on $\\lambda_i$ that are mentioned in lines 206-207 and also constraints on $\\boldsymbol{\\alpha}$ and $\\boldsymbol{\\beta}$ that are only mentioned in lines 128 and 135. \n\n### Code\nThe code included in the supplementary material is insufficiently documented (what do I have to do to reproduce the experiments? What is \"raven\"?) and buggy:\n- The requirements can not be installed as specified in the README (autoLiRPA 0.4 is not available on PyPI at the time of writing; Can not install the specified PyTorch versions without an `--extra-index-url`).\n- The test command that is described in the README fails due to a syntax error in file \"raven/src/util.py\" at line 247 `(root'', train=train, download=True, transform=transform_test)`.\n- The supplementary material does not include the models used in the experiments, describe how to obtain them, or indicate where to place them. \n\n\n[Brix et al., 2023]: Christopher Brix, Stanley Bak, Changliu Liu, Taylor T. Johnson: The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results. CoRR abs/2312.16760 (2023)\n[Zhang et al., 2022]: Huan Zhang, Shiqi Wang, Kaidi Xu, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsieh, J. Zico Kolter: General Cutting Planes for Bound-Propagation-Based Neural Network Verification. NeurIPS 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Comparison with SOTA non-relational baseline MNBaB.**\n\n**R1:** We compared the performance of RABBit with the proposed baseline MNBaB across all ConvSmall CIFAR10 networks listed in Table 1 of the paper. The comparison used the same $\\epsilon$ values, hardware, and timeout values as mentioned in Section 6.1. While MNBaB outperforms $\\alpha,\\beta$-CROWN on 3 of the 4 networks, it remains significantly less precise than RABBit, as shown in Table 4. Runtime comparisons are presented in Table 5. For each property, the runtime of a verifier is the timestamp when the verifier first achieves its maximum UAP accuracy within the time limit.\n\nTable 4 UAP accuracy of RABBit vs baselines including MNBaB\n| Training         |   $\\epsilon$ | $\\alpha$-CROWN| RACoon |   $\\alpha,\\beta$-CROWN |   MNBaB |  RABBit | \n|:-----------------|-------:|:-------------|------:|------:|------:|---------:|\n| Standard         | 1/255    | 45.4% | 45.4%  | 59.8%  | 55.0%  | **62.4%**  |\n| DiffAI         |  5/255  | 49.6%|  51.6%|  53.6%|  55.0% |**59.8%**   |\n| SABR   | 2/255   | 75.8% | 78.2% | 78.4% | 80.0% | **84.0%**  |\n| CITRUS | 2/255   | 76.0%| 78.8% | 79.0%  | 79.6% | **83.6%**  |\n\nTable 5 Average runtime of  RABBit vs baselines including MNBaB\n| Training         |   $\\epsilon$ | $\\alpha$-CROWN (sec.)| RACoon (sec.) |   $\\alpha,\\beta$-CROWN (sec.)|   MNBaB (sec.) |  RABBit (sec.) | \n|:-----------------|-------:|:-------------|------:|------:|------:|---------:|\n| Standard         | 1/255    | 3.3 | 542.8  | 1385.4  | 1168.92  | 2062.8  |\n| DiffAI         |  5/255  | 4.7|  710.3|  1419.3|  1142.43 | 2303.1   |\n| SABR   | 2/255   | 3.7 | 267.3 | 1107.5 | 800.32 | 1665.3  |\n| CITRUS | 2/255   | 3.7| 268 | 1083.4  | 778.96 | 1416.7  |\n\n\n**Q2: Analyse the sensitivity to randomly sampled problem instances. Report the standard deviations, minima, maxima, and 25%, 50%, and 75% quartiles?**\n\n**R2:** We report the standard deviations and quartiles of the worst-case UAP accuracy for the ConvSmall CITRUS [1] MNIST and CIFAR10 in Tables 6 and 7 respectively. We used the same experimental setup described in section 6.1 of the paper and the same 10 properties from Table 1 in the paper. For all the quartiles, RABBit significantly outperforms all the baselines. We will report the results for all networks in the revised version of the paper.\n\n  Table 6: Worst-case UAP accuracy statistics for CIFAR10 ConvSmall CITRUS [1]\n| Verifier         |   Mean | 95% CI       |   Std |   Min |   25% |   Median |   75% |   Max |\n|:-----------------|-------:|:-------------|------:|------:|------:|---------:|------:|------:|\n| CROWN            |   74.8 | [68.3, 81.3] |   8.7 |    58 |  72   |       76 |  81.5 |    86 |\n| alpha-CROWN      |   76   | [69.4, 82.6] |   8.8 |    58 |  74   |       77 |  81.5 |    88 |\n| RACoon           |   78.8 | [71.1, 86.5] |  10.2 |    58 |  76.5 |       80 |  85.5 |    **92** |\n| α-β-CROWN        |   79   | [72.6, 85.4] |   8.5 |    64 |  76   |       80 |  85   |    90 |\n| Strong Branching |   81.2 | [74.7, 87.7] |   8.6 |    62 |  **80**   |       83 |  86   |    **92** |\n| Strong Bounding  |   82.8 | [78.2, 87.4] |   6.1 |    72 |  **80**   |       84 |  86   |    **92** |\n| RABBit           |   **83.6** | [79.3, 87.9] |   5.6 |    **74** |  **80**   |       **85** |  **87.5** |    **92** |\n\n\nTable 7: Worst-case UAP accuracy statistics for CIFAR10 ConvSmall CITRUS [1] \n| Verifier         |   Mean | 95% CI       |   Std |   Min |   25% |   Median |   75% |   Max |\n|:-----------------|-------:|:-------------|------:|------:|------:|---------:|------:|------:|\n| CROWN            |   28.8 | [22.9, 34.7] |   7.8 |    22 |  22.5 |       24 |  36   |    44 |\n| alpha-CROWN      |   41.6 | [36.5, 46.7] |   6.8 |    32 |  36.5 |       40 |  45   |    54 |\n| RACoon           |   44.6 | [39.1, 50.1] |   7.3 |    36 |  38.5 |       42 |  49   |    58 |\n| α-β-CROWN        |   59.4 | [54.5, 64.3] |   6.5 |    50 |  54.5 |       57 |  65   |    **72** |\n| Strong Branching |   60.0 | [55.3, 64.7] |   6.2 |    52 |  54.5 |       57 |  65   |    **72** |\n| Strong Bounding  |   60.6 | [56.2, 65.8] |   6.3 |    52 |  55   |       59 |  **65.5** |    **72** |\n| RABBit|   **61.6** | [57.1, 66.1] |   5.9 |    **54** |  **55.5** |       **60** |  **65.5** |    **72** |\n\n[1] \"Cross-Input Certified Training for Universal Perturbations\", C. Xu, et. al., ECCV, 2024. \n\n**Q3: What are the actual runtimes of RABBit?**\n\n**R3:** Please refer to the answer to Q2 in the common response for the detailed runtime analysis of RABBit and the baselines. \nThe overall time limit given to RABBit is $k=50$ minutes and the result of any MILP instance that was not completed in the given time limit ($k$ minutes) was not considered in the final result. Also, for efficiency, each MILP instance is formulated in either line 16 or line 25 of Algo. 1 is executed in a different thread in parallel to the BaB and does not block subsequent iterations of the loops (lines 10 and 20 of Algo. 1) In all cases, RABBit's runtime is always bounded by $k$ minutes.\n\n**Q4: Table 1: How were the perturbation radii chosen?**\n\n**R4:** The perturbation radii used in this work are based on the existing relational verifier RACoon [1]. Other non-relational verifiers like MNBaB [2] also use the same perturbation radii for networks with the same architecture. We also have evaluated the performance of RABBit with different $\\epsilon$ values in Figure 3 of the paper. \n\n[1] \"Relational DNN Verification With Cross Executional Bound Refinement\", D. Banerjee, et. al., ICML, 2024.\n\n[2] \"Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Bound\", C. Ferrari, et. al., ICLR, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Area Chair and Reviewers,

We appreciate the constructive feedback from the reviewers and are encouraged by their acknowledgment of the paper's  theoretically sound contributions, and detailed experimental validation.

 ***Q1: Evaluating RABBit on other relational properties (eacX, uEoH)***

**R1:** We evaluate RABBit w.r.t another relational property: top-$k$ accuracy.

**Definition:** Given an unperturbed input $\pmb{x}$, a network $N: \mathbb{R}^{n_0} \rightarrow \mathbb{R}^{n_l}$, and $k \leq n_l$, we want to verify whether the target class remains within the top-$k$ classes predicted by $N(\pmb{x} + \pmb{\delta})$ for all bounded perturbations $\|\pmb{\delta}\|_{\infty} \leq \epsilon$.

Let $d_i(\pmb{\delta}) = \pmb{c_i}^TN(\pmb{x} + \pmb{\delta}) = N(\pmb{x} + \pmb{\delta})[t] - N(\pmb{x} + \pmb{\delta})[i]$ denote the logit difference between the target class $t$ and another class $i \neq t$. If, for all $|\pmb{\delta}|_{\infty} \leq \epsilon$, at least $n_l - k$ logit differences are positive, then we prove that the target class always remains within the top-$k$ predicted classes. For $i, j \in [n_l]$ and $i \neq j$, since the logit differences $d_i(\pmb{\delta})$ and $d_j(\pmb{\delta})$ are related, tracking their dependencies improves precision. In this case, even though all logit differences result from perturbations of the same input, we can treat the computation of each logit difference as a separate execution of $N(\pmb{x} + \pmb{\delta})$. This approach reduces the top-$k$ verification problem to a relational verification problem, which is handled by RABBit. Existing non-relational verifiers [1] compute the lower bound on each logit difference independently and thus lose precision.  

In Table 1, we present results for top-$2$ ($k=2$) accuracy for ConvSmall DiffAI networks and $\epsilon$ values from Table 1 of the paper. We use the first 100 images from each dataset. The timeout values used are 1 minute for BaB per image and 1 minute per MILP instance. Since all the related logit differences originate from the same image, we do not need to greedily select executions for strong bounding. In all cases, RABBit is more precise than all the baselines. We will include results for all the networks in the revised version of the paper.

Table 1: Verified top-2 accuracy for RABBit vs baselines 
|Dataset | Training         |   $\epsilon$ | $\alpha$-CROWN| Avg. Time (s.) | RACoon| Avg. Time (s.) |   $\alpha,\beta$-CROWN |   Avg. Time (s.) |  RABBit | Avg. Time (s.) |
|:-----------------|-------:|-------:|:-------------|------:|------:|------:|---------:|---------:|---------:|---------:|
|CIFAR10| DiffAI | 5/255 |74% | 4.52|75% | 4.87|75%|20.47|**78%**|24.27|
|MNIST| DiffAI | 0.13 |84% | 1.20|84% |1.42|89%|11.03|**91%**|13.43|

[1] "Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification", S. Wang, et. al., NeurIPS, 2021.
 
***Q2: Verification Times (eacX, aDbj)***

**R2:** We present the runtimes, in seconds, for each verifier averaged across all properties and networks from Table 1 of the paper in Tables 2 and 3 below. For each property, the runtime of a verifier is the timestamp when the verifier first achieves its maximum UAP accuracy within the time limit. Although RABBit has a higher runtime overhead compared to existing verifiers such as $\alpha,\beta$-CROWN, it consistently delivers better performance at every timestamp, as shown in Figure 1 of the paper. We will add these results to the revised version of the paper.
  
  Table 2: Runtime statistics of different verifiers for CIFAR-10
  | Verifier | Mean | 95% CI | Std. | Min| 25%| Median | 75%| Max|
|----------|----------|----------|----------|----------|----------|----------|----------|----------| 
| α-CROWN    | 4.3   | [2.9, 5.7] | 5.4 | 0.3 | 0.9 | 1.6 | 6.1 | 27.1|
| α,β-CROWN   | 1325.6   | [1220.8, 1430.5] | 402.5 | 418.5 | 990 | 1363.6 |1608.2 | 2199.2|
| RACoon   | 409.1   | [353.1, 464.9] | 214.6 | 66.2 | 260.6 | 342.4 | 607.9 | 944.1 |
| RABBit  | 1725.5   | [1491.5, 1959.5] | 898.3 | 178 | 1064.3 | 1528.3 | 2451.1 | 2998 |

  Table 3: Runtime statistics of different verifiers for MNIST
  | Verifier | Mean | 95% CI |Std. | Min| 25%| Median | 75%| Max|
|----------|----------|----------|----------|----------|----------|----------|----------|----------|
| α-CROWN    | 2.9   | [1.6, 4.2] | 4.5 | 0.3 |0.5 | 0.8 | 4.4 | 20|
| α,β-CROWN   | 1625.5   | [1409.4, 1849.6] | 752.8 | 227.7 | 1075.7| 1389 | 2104 | 2943.8|
| RACoon   | 604.8   | [539.0, 670.6] | 229| 18.8| 389.5 | 658.8 | 807.6 | 981.6 |
| RABBit  | 2083.7 | [1857.2, 2310.2] | 788.9| 557.3 | 1697.5 | 2120.4 | 2783.2 | 2999.1 |

**Q3: Derivation of m/n and m^(1/n) (uEoH, dFSH)**

**R3:** Suppose we consider a relational property over 4 executions ($n = 4$) and in the given time thershold we can solve $16$ problems ($m = 16$). 
Now for strong bounding, since we are tracking dependencies across executions we need to consider all possible combinations of subproblems from different executions. Assuming we split on each execution uniformly we can only consider $2$ subproblems ($m = 2^n$) from each execution. If $(A_1, A_2)$, $(B_1, B_2)$, $(C_1, C_2)$, and $(D_1, D_2)$ are subproblems from each of 4 executions respectively then strong bounding considers the 16 subproblems specified by $(A_i, B_j, C_k, D_l)$ where $i, j, k, l \in $ {1, 2}.

In contrast, if we apply BaB on each execution independently we **do not** need to consider combinations of subproblems from different executions. In this case, assuming we split on each execution uniformly we can consider $4$ subproblems ($m = 4*n$) from each execution.

Meta-review: The reviewers appreciared the importance of the problem considered, and praised the novelty of the approach and the experimental results.  On the other hand, they struggled with the quality of the writing, and with running the code provided.  A number of issues identified by the reviewers were resolved by the rebuttal and the subsequent discussion with the authors.  On balance, I recommend acceptance, however urge the authors to revise the paper carefully based on all the reviews, and in particular to address apparent plagiarism or self-plagiarism in that a number of paragraphs in the paper are identical or almost the same as paragraphs in Banerjee and Singh, "Relational DNN Verification With Cross Executional Bound Refinement", ICML 2024.
Decision: Accept (poster)