SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 3: Benchmarks Setting
Task	Initial Region	Goal Region	Unsafe Region
B1	x1 ∈ [0.8, 0.9]	x1 ∈ [0, 0.2]	—
x2 ∈ [0.5, 0.6]	x2 ∈ [0.05, 0.3]
B2	x1 ∈ [0.7, 0.9]	x1 ∈ [−0.3, 0.1]	—
x2 ∈ [0.7, 0.9]	x2 ∈ [−0.35, 0.5]
B3	x1 ∈ [0.8, 0.9]	x1 ∈ [0.2, 0.3]	—
x2 ∈ [0.4, 0.5]	x2 ∈ [−0.3, −0.05]
B4	x1 ∈ [0.25, 0.27]	x1 ∈ [−0.05, 0.05]	—
x2 ∈ [0.08, 0.1]	x2 ∈ [−0.05, 0]
x3 ∈ [0.25, 0.27]
B5	x1 ∈ [0.38, 0.4]	x1 ∈ [−0.4, −0.28]	—
x2 ∈ [0.45, 0.47]	x2 ∈ [0.05, 0.22]
x3 ∈ [0.25, 0.27]
Tora	x1 ∈ [−0.1, 0.2]	x1 ∈ [−0.25, 0.10]	—
x2 ∈ [−0.9, −0.6]	x2 ∈ [0.2, 0.7]
CartPole	x1, x2, x4 ∈ [0.02, 0.02]	—	x2 ∈ [26, 29]
x3 ∈ [0.02, 0.021]	x3 ∈ [−0.2, 0.2]
QUAD	x1, ..., x6 ∈ [0.35, 0.4]	x3 > 0.6	—
x7, ..., x12 ∈ [0, 0]
A.2 Comparison on cumulative reward and robustness
In this section, we provide the comparison results on cumulative reward and robustness of B3-B5 and Tora in Figure 7. The solid lines and shadows refer to the average reward and  $95\%$  confidence interval, respectively. For these four cases, it is obvious that there are comparable trends in the cumulative rewards of PLDNNs and DNNs during training and under perturbation. Therefore, we can conclude that using PLDNNs will not affect the training efficiency and the robustness of trained systems.
A.3 Comparison on the tightness of over-approximation sets
Tightness results of regular cases. We present the tightness comparison results of B1 and B3-B5 in Figure 8. For these four regular cases, both Polar and Verisig 2.0 can produce tight over-approximation for decision networks, thus all three methods achieve similar tightness results and successfully verify the goal-reach properties.
Multiple evaluations on CartPole and QUAD. Since the trained policies used in the comparison differ due to different decision networks, we conduct more evaluations on these two complex cases. For each case, we train three more decision networks with the same configuration and the corresponding results are shown in Figure 10. For CartPole, Polar did not finish the calculation under three different networks as depicted in Figure 10(d-f). Additionally, on the basis of the result of dealing with divergent traces as shown in Figure 10(d), we can see that Polar is not suitable for dealing with the DNN-controlled systems with divergent traces. As for QUAD, we record the computation results within 12 hours and obtain the results as shown in Figure 10(j-l). We can observe that only
(a) B3
(b) B4
(c) B5
(d) Tora
(e) B3
(f) B4
(g) B5
Figure 7: Performance (a-d) and robustness comparison (e-h) of the pldnn and DNN under the same settings. The number in the parentheses is the base of  $\sigma$ . For example, in B3 when the abscissa is equal to  $50$ ,  $\sigma = 50 \times 0.0005 = 0.025$ .
(h) Tora
a few time steps are completed by Polar, while LINCON can accomplish more than 10 steps within about 1000 seconds under multiple evaluations.
(a) B1 by LINCON
(b) B3 by LINCon
(c) B4 by LINCON
(d) B5 by LINCON
(e) B1 by Polar
(f) B3 by Polar
(g) B4 by Polar
(h) B5 by Polar
(i) B1 by Verisig 2.0
(j) B3 by Verisig 2.0
(k) B4 by Verisig 2.0
(1) B5 by Verisig 2.0
Figure 8: Tightness comparison on the DRL systems with larger decision networks (red box: overapproximation sets; green lines: simulation trajectories; blue box: goal region.)
A.4 Evaluation on Verisig 2.0 with Big Weights
Verisig 2.0 produces large over-approximation error when dealing with neural networks with big weights. To demonstrate this, we initialize the weights of the neural network with larger values (random numbers  $w_{l} \sim \mathbf{N}(\mu, \sigma^{2})$  with  $\mu = 0, \sigma = 0.1$ ) and show the experimental results in Figure 9. We observe that the calculated over-approximation sets contain large over-approximation error except for B4. In Tora, Verisig 2.0 fails to calculate the complete reachable sets due to too large overapproximation error. Hence, it is fairly to say that Verisig 2.0 is sensitive to the DNNs with big weights.
(a) B1
(b) B2
(c) B3
(d) B4
(e) B5
Figure 9: Assessing Verisig 2.0 on the larger networks with big weights. red box: over-approximation set; green lines: simulation trajectories; blue box: goal region; purple dashed box: unsafe region.
(f) Tora
(g)
(j)
Figure 10: Multiple evaluations on CartPole (CP) and QUAD. red box: over-approximation set; green lines: simulation trajectories;
(h)
(k)
(i)
(1)