SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- When the only nonzero coefficients are  $c^{(0)} = c$ , we recover Theorem 1. We refer to this bound as  $g_{c}(\alpha, \gamma)$
- When the only nonzero coefficient is  $c_{j}^{(i)} = 1$ , we can lower bound  $l_{j}^{(i)}$  by  $g_{i,j}^{\mathrm{lower}}(\alpha, \gamma)$ . If we set  $c_{j}^{(i)} = -1$ , we can upper bound  $u_{j}^{(i)}$  by  $-g_{i,j}^{\mathrm{upper}}(\alpha, \gamma)$ .
With this, we present our generalized theorem.
Theorem 2 (Lower-bounding combination of neurons). Given an output set  $S_{out} = \{y: \mathbf{H}y + \mathbf{d} \leq \mathbf{0}\}$  and vector  $c, g(\alpha, \gamma)$  is a lower bound to the linear program in (3) with objective  $c^{(0)\top} \hat{x}^{(0)} + \sum_{i=1}^{L-1} c^{(i)\top} x^{(i)}$  for  $0 \leq \alpha \leq 1, \gamma \geq 0$ , and  $g$  defined via
$$
\begin{array}{l} g (\boldsymbol {\alpha}, \boldsymbol {\gamma}) = \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {+} \boldsymbol {l} ^ {(0)} - \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {-} \boldsymbol {u} ^ {(0)} \\ - \sum_ {i = 1} ^ {L} \pmb {\nu} ^ {(i) \top} \mathbf {b} ^ {(i)} + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \left[ \frac {u _ {j} ^ {(i)} l _ {j} ^ {(i)} [ \hat {\nu} _ {j} ^ {(i)} ] _ {+}}{u _ {i} ^ {(j)} - l _ {i} ^ {(j)}} \right] \\ \end{array}
$$
$$
\begin{array}{l} \mathcal {I} ^ {- (i)} = \{j: u _ {j} ^ {(i)} \leq 0 \} \\ \mathcal {I} ^ {+ (i)} = \{j: l _ {j} ^ {(i)} \geq 0 \} \\ \mathcal {I} ^ {\pm (i)} = \{j: l _ {j} ^ {(i)} <   0 <   u _ {j} ^ {(i)} \} \\ \boldsymbol {\nu} ^ {(L)} = - \boldsymbol {\gamma} \\ \hat {\nu} _ {j} ^ {(i)} = \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {: j} ^ {(i + 1)} \\ \nu_ {j} ^ {(i)} = \left\{ \begin{array}{l l} \hat {\nu} _ {j} ^ {(i)} - c _ {j} ^ {(i)}, & j \in \mathcal {I} ^ {+ (i)} \\ - c _ {j} ^ {(i)}, & j \in \mathcal {I} ^ {- (i)} \\ \frac {u _ {j} ^ {(i)}}{u _ {i} ^ {(j)} - l _ {i} ^ {(j)}} [ \hat {\nu} _ {j} ^ {(i)} ] _ {+} - \alpha_ {j} ^ {(i)} [ \hat {\nu} _ {j} ^ {(i)} ] _ {-} - c _ {j} ^ {(i)}, & j \in \mathcal {I} ^ {\pm (i)} \end{array} \right. \\ \end{array}
$$
D Dual Derivation
We first incorporate the output constraint  $\mathbf{H}\pmb{x}^{(L)} + \mathbf{d} \leq \mathbf{0}$  by folding in this linear transformation of the output into the final linear layer of the network, as customary in prior work [Wang et al., 2021, Zhang et al., 2022].
We now prove the generalized theorem as described in C. As a conceptual overview of our proof, we take the Lagrange dual of the linear program to derive an unconstrained optimization problem. From here, we derive constraints that must be satisfied in the max min formulation. These constraints yield the bound propagation procedure we display in Theorem 1.
We start with the convex relaxation.
$$
\min  _ {\boldsymbol {x}, \hat {\boldsymbol {x}}} \quad \boldsymbol {c} ^ {(0) \top} \hat {\boldsymbol {x}} ^ {(0)} + \sum_ {i = 1} ^ {L - 1} \boldsymbol {c} ^ {(i) \top} \boldsymbol {x} ^ {(i)}
$$
s.t.  $\pmb{l}^{(0)}\leq \hat{\pmb{x}}^{(0)}\leq \pmb{u}^{(0)}$
$$
\boldsymbol {x} ^ {(L)} \leq \mathbf {0};
$$
$$
\boldsymbol {x} ^ {(i)} = \mathbf {W} ^ {(i)} \hat {\boldsymbol {x}} ^ {(i - 1)} + \mathbf {b} ^ {(i)}; \quad i \in [ L ],
$$
$$
\hat {x} _ {j} ^ {(i)} \geq 0; j \in \mathcal {I} ^ {\pm (i)}
$$
$$
\hat {x} _ {j} ^ {(i)} \geq x _ {j} ^ {(i)}; j \in \mathcal {I} ^ {\pm (i)}
$$
$$
(u _ {j} ^ {(i)} - l _ {j} ^ {(i)}) \hat {x} _ {j} ^ {(i)} \leq u _ {j} ^ {(i)} x _ {j} ^ {(i)} - u _ {j} ^ {(i)} l _ {j} ^ {(i)}; j \in \mathcal {I} ^ {\pm (i)}
$$
$$
\hat {x} _ {j} ^ {(i)} = x _ {j} ^ {(i)}; j \in \mathcal {I} ^ {+ (i)}
$$
$$
\hat {x} _ {j} ^ {(i)} = 0; j \in \mathcal {I} ^ {- (i)}
$$
From this, we take the Lagrange dual of most of the constraints. In doing so, we introduce a new dual variable for each constraint. Note that we do not dualize every constraint since they can easily be dealt with later.
$$
\begin{array}{l} \min _ {\boldsymbol {x}, \hat {\boldsymbol {x}}} \max _ {\boldsymbol {\nu}, \boldsymbol {\mu}, \boldsymbol {\tau}, \boldsymbol {\gamma}, \boldsymbol {\lambda}} \quad \boldsymbol {c} ^ {(0) \top} \hat {\boldsymbol {x}} ^ {(0)} + \sum_ {i = 1} ^ {L - 1} \boldsymbol {c} ^ {(i) \top} \boldsymbol {x} ^ {(i)} + \boldsymbol {\gamma} ^ {\top} \boldsymbol {x} ^ {(L)} + \sum_ {i = 1} ^ {L} \boldsymbol {\nu} ^ {(i) \top} \left(\boldsymbol {x} ^ {(i)} - \mathbf {W} ^ {(i)} \hat {\boldsymbol {x}} ^ {(i - 1)} - \mathbf {b} ^ {(i)}\right) \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \left[ \mu_ {j} ^ {(i)} \left(- \hat {x} _ {j} ^ {(i)}\right) + \tau_ {j} ^ {(i)} \left(x _ {j} ^ {(i)} - \hat {x} _ {j} ^ {(i)}\right) + \lambda_ {j} ^ {(i)} \left(\left(u _ {j} ^ {(i)} - l _ {j} ^ {(i)}\right) \hat {x} _ {j} ^ {(i)} - u _ {j} ^ {(i)} x _ {j} ^ {(i)} + u _ {j} ^ {(i)} l _ {j} ^ {(i)}\right) \right] \\ \end{array}
$$
$$
\text {s . t .} \boldsymbol {l} ^ {(0)} \leq \hat {\boldsymbol {x}} ^ {(0)} \leq \boldsymbol {u} ^ {(0)}; \quad \hat {x} _ {j} ^ {(i)} = 0, j \in \mathcal {I} ^ {- (i)}; \quad \hat {x} _ {j} ^ {(i)} = x _ {j} ^ {(i)}, j \in \mathcal {I} ^ {+ (i)}
$$
$$
\boldsymbol {\mu} \geq 0; \quad \tau \geq 0; \quad \gamma \geq 0; \quad \lambda \geq 0
$$
Since we took the dual of a linear program, the solution to the min-max optimization is equivalent to the solution of the max-min optimization by strong duality. Therefore, we can solve the following program (equivalent up to rearrangement).
$$
\begin{array}{l} \max  _ {\boldsymbol {\nu}, \boldsymbol {\mu}, \boldsymbol {\tau}, \gamma , \boldsymbol {\lambda}} \min  _ {\boldsymbol {x}, \hat {\boldsymbol {x}}} \left(\boldsymbol {\nu} ^ {(L)} + \boldsymbol {\gamma}\right) ^ {\top} \boldsymbol {x} ^ {(L)} + \left(\boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)}\right) \hat {\boldsymbol {x}} ^ {(0)} \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {+ (i)}} \left(\nu_ {j} ^ {(i)} - \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} + c _ {j} ^ {(i)}\right) x _ {j} ^ {(i)} + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {- (i)}} \left(\nu_ {j} ^ {(i)} + c _ {j} ^ {(i)}\right) x _ {j} ^ {(i)} \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \left[ \left(\nu_ {j} ^ {(i)} + \tau_ {j} ^ {(i)} - \lambda_ {j} ^ {(i)} u _ {j} ^ {(i)} + c _ {j} ^ {(i)}\right) x _ {j} ^ {(i)} \right. \\ \left. \right.\left. + \left(- \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} - \mu_ {j} ^ {(i)} - \tau_ {j} ^ {(i)} + \left(u _ {j} ^ {(i)} - l _ {j} ^ {(i)}\right) \lambda_ {j} ^ {(i)}\right) \hat {x} _ {j} ^ {(i)} \right] \\ - \sum_ {i = 1} ^ {L} \boldsymbol {\nu} ^ {(i) \top} \mathbf {b} ^ {(i)} + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \lambda_ {j} ^ {(i)} u _ {j} ^ {(i)} l _ {j} ^ {(i)} \\ \end{array}
$$
$$
\mathrm {s . t .} \boldsymbol {l} ^ {(0)} \leq \hat {\boldsymbol {x}} ^ {(0)} \leq \boldsymbol {u} ^ {(0)}
$$
To minimize  $\left(\pmb{c}^{(0)\top} - \pmb{\nu}^{(1)\top}\mathbf{W}^{(1)}\right)\hat{\pmb{x}}^{(0)}$  subject to  $l^{(0)}\leq \hat{x}^{(0)}\leq u^{(0)}$ , we can consider the choice we must make in each dimension. If the  $j$ th entry of the coefficient is positive, we should set  $\hat{x}_j^{(0)} = l_j^{(0)}$ . Otherwise, we should set  $\hat{x}_j^{(0)} = u_j^{(0)}$ .
$$
\begin{array}{l} \max  _ {\nu , \mu , \tau , \gamma , \lambda} \min  _ {\boldsymbol {x}, \boldsymbol {\bar {x}}} \left(\boldsymbol {\nu} ^ {(L)} + \boldsymbol {\gamma}\right) ^ {\top} \boldsymbol {x} ^ {(L)} + \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {+} \boldsymbol {l} ^ {(0)} - \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {-} \boldsymbol {u} ^ {(0)} \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {+ (i)}} \left(\nu_ {j} ^ {(i)} - \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} + c _ {j} ^ {(i)}\right) x _ {j} ^ {(i)} + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {- (i)}} \left(\nu_ {j} ^ {(i)} + c _ {j} ^ {(i)}\right) x _ {j} ^ {(i)} \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \left[ \left(\nu_ {j} ^ {(i)} + \tau_ {j} ^ {(i)} - \lambda_ {j} ^ {(i)} u _ {j} ^ {(i)} + c _ {j} ^ {(i)}\right) x _ {j} ^ {(i)} \right. \\ \left. \right.\left. + \left(- \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} - \mu_ {j} ^ {(i)} - \tau_ {j} ^ {(i)} + \left(u _ {j} ^ {(i)} - l _ {j} ^ {(i)}\right) \lambda_ {j} ^ {(i)}\right) \hat {x} _ {j} ^ {(i)} \right] \\ - \sum_ {i = 1} ^ {L} \boldsymbol {\nu} ^ {(i) \top} \mathbf {b} ^ {(i)} + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \lambda_ {j} ^ {(i)} u _ {j} ^ {(i)} l _ {j} ^ {(i)} \\ \end{array}
$$
$$
\text {s . t .} \mu \geq 0; \quad \tau \geq 0; \quad \gamma \geq 0; \quad \lambda \geq 0
$$
From here, we note that the variables  $\pmb{x}$  or  $\hat{\pmb{x}}$  are unconstrained variables. Therefore, if any of their coefficients are nonzero, the inner minimization can immediately drive its value to  $-\infty$ . As such, the outer maximization must set all of these coefficients to zero. Therefore, we can derive constraints from this restructured optimization and remove the free variables  $\pmb{x}, \hat{\pmb{x}}$ .
$$
\begin{array}{l} \max  _ {\boldsymbol {\nu}, \boldsymbol {\mu}, \tau , \boldsymbol {\gamma}, \boldsymbol {\lambda}} \quad \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {+} \boldsymbol {l} ^ {(0)} - \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {-} \boldsymbol {u} ^ {(0)} - \sum_ {i = 1} ^ {L} \boldsymbol {\nu} ^ {(i) \top} \mathbf {b} ^ {(i)} \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \lambda_ {j} ^ {(i)} u _ {j} ^ {(i)} l _ {j} ^ {(i)} \\ \end{array}
$$
s.t.  $\pmb{\nu}^{(L)} = -\pmb{\gamma}$
$$
\begin{array}{l} \nu_ {j} ^ {(i)} = \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} - c _ {j} ^ {(i)}, j \in \mathcal {I} ^ {+ (i)} \\ \nu_ {j} ^ {(i)} = - c _ {j} ^ {(i)}, j \in \mathcal {I} ^ {- (i)} \\ \nu_ {j} ^ {(i)} = \lambda_ {j} ^ {(i)} u _ {j} ^ {(i)} - \tau_ {j} ^ {(i)} - c _ {j} ^ {(i)}, j \in \mathcal {I} ^ {\pm (i)} \\ \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} = (u _ {j} ^ {(i)} - l _ {j} ^ {(i)}) \lambda_ {j} ^ {(i)} - \left(\mu_ {j} ^ {(i)} + \tau_ {j} ^ {(i)}\right), j \in \mathcal {I} ^ {\pm (i)} \\ \mu \geq 0; \quad \tau \geq 0; \quad \gamma \geq 0; \quad \lambda \geq 0 \\ \end{array}
$$
For the following, we define  $\hat{\nu}_{j}^{(i)} = \pmb{\nu}^{(i + 1)\top}\mathbf{W}_{\vdots j}^{(i + 1)}$ . We note that since the upper and lower bounds of the neuron relaxation can not be tight simultaneously, at least one of  $(u_{j}^{(i)} - l_{j}^{(i)})\lambda_{j}^{(i)}$  and  $\mu_j^{(i)} + \tau_j^{(i)}$  must be non-zero. Therefore, we can write them as  $(u_{j}^{(i)} - l_{j}^{(i)})\lambda_{j}^{(i)} = [\hat{\nu}_{j}^{(i)}]_{+}$  and  $\mu_j^{(i)} + \tau_j^{(i)} = [\hat{\nu}_j^{(i)}]_-$ . We can then use the fact that  $\tau_j^{(i)}$  lies in the interval 0 and  $\hat{\nu}_{j}^{(i)}$  to get the following bound propagation procedure.
$$
\begin{array}{l} \max  _ {\boldsymbol {\nu}, \boldsymbol {\alpha}, \gamma} \quad \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {+} \boldsymbol {l} ^ {(0)} - \left[ \boldsymbol {c} ^ {(0) \top} - \boldsymbol {\nu} ^ {(1) \top} \mathbf {W} ^ {(1)} \right] _ {-} \boldsymbol {u} ^ {(0)} - \sum_ {i = 1} ^ {L} \boldsymbol {\nu} ^ {(i) \top} \mathbf {b} ^ {(i)} \\ + \sum_ {i = 1} ^ {L - 1} \sum_ {j \in \mathcal {I} ^ {\pm (i)}} \left[ \frac {u _ {j} ^ {(i)} l _ {j} ^ {(i)} [ \hat {\nu} _ {j} ^ {(i)} ] +}{u _ {i} ^ {(j)} - l _ {i} ^ {(j)}} \right] \\ \end{array}
$$
s.t.  $\pmb{\nu}^{(L)} = -\gamma$
$$
\nu_ {j} ^ {(i)} = \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)} - c _ {j} ^ {(i)}, j \in \mathcal {I} ^ {+ (i)}
$$
$$
\nu_ {j} ^ {(i)} = - c _ {j} ^ {(i)}, j \in \mathcal {I} ^ {- (i)}
$$
$$
\hat {\nu} _ {j} ^ {(i)} = \boldsymbol {\nu} ^ {(i + 1) \top} \mathbf {W} _ {:, j} ^ {(i + 1)}
$$
$$
\nu_ {j} ^ {(i)} = \frac {u _ {j} ^ {(i)}}{u _ {i} ^ {(j)} - l _ {i} ^ {(j)}} [ \hat {\nu} _ {j} ^ {(i)} ] _ {+} - \alpha_ {j} ^ {(i)} [ \hat {\nu} _ {j} ^ {(i)} ] _ {-} - c _ {j} ^ {(i)}, j \in \mathcal {I} ^ {\pm (i)}
$$
$$
\alpha_ {j} ^ {(i)} \in [ 0, 1 ]; \quad \gamma \geq 0
$$
In this program,  $\alpha_{j}^{(i)}$  are estimable parameters controlling the relaxation of neuron  $j$  in layer  $i$ , similar to the ones appearing in Xu et al. 2021.  $\gamma$ , as discussed in the body of the paper, is the parameter which enforces the output constraint throughout this entire bound propagation procedure.
E Connection to Forward Verification for Intermediate Bounds
We consider the general objective presented in C and aim to minimize  $c^{(0)\top}\hat{x}^{(0)} + \sum_{i = 1}^{L - 1}c^{(i)\top}\pmb{x}^{(i)}$
$$
\min  _ {\boldsymbol {x}} \quad \boldsymbol {c} ^ {(0) \top} \hat {\boldsymbol {x}} ^ {(0)} + \sum_ {i = 1} ^ {L - 1} \boldsymbol {c} ^ {(i) \top} \boldsymbol {x} ^ {(i)}
$$
$$
\begin{array}{l l} \text {s . t .} & \boldsymbol {x} \in \mathcal {X}; \quad \mathbf {H} f (\boldsymbol {x}) + \mathbf {d} \leq 0 \end{array}
$$
If we take the dual of the constraint, then we get the program
$$
\min  _ {\boldsymbol {x}} \quad \boldsymbol {c} ^ {(0) \top} \hat {\boldsymbol {x}} ^ {(0)} + \sum_ {i = 1} ^ {L - 1} \boldsymbol {c} ^ {(i) \top} \boldsymbol {x} ^ {(i)} + \gamma (\mathbf {H} f (\boldsymbol {x}) + \mathbf {d})
$$
We note that the objective here can be expressed as a neural network with residual stream  $f(\boldsymbol{x})$  and a skip connection with linear weight  $c^{(i)}$  from the pre-activations of layer  $i$  to the final output for every layer. In theory, we could construct this neural network and directly pass it to a forward verification tool which could iteratively tighten all bounds for a solution.
As stated in Algorithm 1, INVPROP first initializes bounds for all layers using some computationally cheap technique. Based on the input bounds given by  $\mathcal{X}$ , we first compute intermediate bounds using interval propagation and then tighten them based on the reverse symbolic interval propagation (RSIP) technique Singh et al. [2019]. While INVPROP will iteratively tighten those bounds over time, we found RSIP to reduce the total necessary runtime significantly compared to an initialization based solely on interval propagation. We initialize all  $\alpha$  with 0.5 and all  $\gamma$  with 0.025.
The optimization is performed for all lower and upper bounds of all neurons in each layer in parallel, starting with the last layer and moving forward to the input layer. The bounds of the cutting hyperplanes are optimized last, together with the bounds on the input neurons. The improvements on  $\pmb{c}^{\top}\pmb{x}$  are measured every 10th iteration. Before doing so, the bounds of the hyperplanes are tightened for 10 extra steps to improve their precision. We detect convergence by monitoring  $\pmb{c}^{\top}\pmb{x}$ . If successive iterations see minimal improvement, we stop or branch on the input space. All cutting hyperplanes are evenly distributed to maximize their information gain. For 40 hyperplanes in a 2D input space, we rotate each plane by  $9^{\circ}$ . All reported runtimes under 10 minutes are computed as the average of five runs.
F.1 Encoding Non-Linear Constraints
To support non-linear output sets, such as the maximum operation used in the OOD example in Section 4.3, the non-linearity needs to be encoded, such that it can be expressed as linear constraints over the modified network. We rewrite  $\max(y_1, y_2) = \max(y_1 - y_2, 0) + y_2$  and add an additional ReLU layer for this operation. Note that to pass  $y_2$  through this layer without modifying it, one can either write  $y_2 = \max(y_2, 0) - \max(-y_2, 0)$ , or  $y_2 = \max(y_2 - M, 0) + M$  where  $M$  is the lower bound of all possible  $y_2$ . We find that avoiding the additional ReLU relaxations that would occur for the first approach is beneficial for the optimization and compute a lower bound of  $M$  using interval propagation. As  $y_3$  (for the OOD class) should not be changed by this operation either, we apply the same trick of subtracting and adding its lower bound.
F.2 Control Benchmark Encoding
We encode the entire control formula  $\pmb{x} = \mathbf{A}\pmb{x} + \mathbf{B}\pmb{u}$  as one feedforward network by encoding the residual connection as regular fully connected layers. To this end, we use the same technique described in Section F.1 to shift the bounds into the positive regime, feed them forward and then shift them back.
To compute  $S_{\mathrm{over}}$  for a timestep  $t > 1$ , we first stack the network  $t$  times, then simplify it by merging consecutive linear layers. All bounds of layers not affected by this merging that also appear in the network for  $t - 1$  timesteps are reused. Their bounds are already tight enough and are not optimized further. Note that this is different than using the previous  $S_{\mathrm{over}}$  as the new target region and using an unstacked network: All bounds of the new layers are still optimized w.r.t. the precise target  $S_{\mathrm{out}}$ . Therefore, we do not suffer from accumulating inaccuracies.
G Hardware
For the control benchmark, all experiments were performed on a Dual Xeon Gold 6138.
For the OOD benchmark, all experiments were performed on an Intel Xeon Platinum 8160 processor using 8 cores and 40GB RAM, as well as a V100-SXM2 GPU.
Figure 9: Detailed runtime comparison of  $\alpha, \beta$ -CROWN and  $\alpha, \beta$ -CROWN + INVPROP. Except for a few instances,  $\alpha, \beta$ -CROWN extended with INVPROP can verify the properties faster, and can prove robustness for some instances that cause a timeout for pure  $\alpha, \beta$ -CROWN. The timeout per instance is 5 minutes.
For the MNIST and YOLO robustness verification benchmarks, an AWS instance of type g5.2xlarge was used, with is equipped with a AMD EPYC 7R32 processor with 8 cores, 32GB RAM, as well as an A10G GPU.
H Robustness Verification
For the comparison in Section 4.2, we have implemented the concept of INVPROP in  $\alpha, \beta$ -CROWN. Figure 9 compares the performance of  $\alpha, \beta$ -CROWN and  $\alpha, \beta$ -CROWN + INVPROP on the MNIST benchmark defined by Yang et al. [2021b].