SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$k = 1,2$ . We have proved that  $\mathbb{E}_x(\nabla_\theta L_1\mid f_i,g_i) = \mathbb{E}_x(\nabla_\theta L_2\mid f_i,g_i)$ , thus the second term is equal. Next, we prove that  $\mathrm{Var}_x(\nabla_\theta L_1\mid f_i,g_i)\leq \mathrm{Var}_x(\nabla_\theta L_2\mid f_i,g_i)$ , which implies  $\mathrm{Var}_x(\nabla_\theta L_1)\leq$ $\mathrm{Var}_x(\nabla_\theta L_2)$ .
By explicit computation, we have
$$
\begin{array}{l} \operatorname {V a r} \left(\nabla L _ {1} \mid f _ {i}, g _ {i}\right) \\ = (\bar {g}) ^ {2} \operatorname {V a r} \left(\sum_ {i} \frac {1}{n} \eta_ {i}\right) + (\bar {f}) ^ {2} \operatorname {V a r} \left(\sum_ {i} \frac {1}{n} \delta_ {i}\right) \\ = \frac {1}{n} \sigma_ {1} ^ {2} (\bar {g}) ^ {2} + \frac {1}{n} \sigma_ {2} ^ {2} (\bar {f}) ^ {2}, \tag {4} \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {V a r} \left(\nabla L _ {2} \mid f _ {i}, g _ {i}\right) \\ = \operatorname {V a r} \left(\sum_ {i} \frac {1}{n} f _ {i} \delta_ {i}\right) + \operatorname {V a r} \left(\sum_ {i} \frac {1}{n} g _ {i} \eta_ {i}\right) \\ = \frac {1}{n} \sigma_ {1} ^ {2} \left(\sum_ {i} \frac {1}{n} g _ {i} ^ {2}\right) + \frac {1}{n} \sigma_ {2} ^ {2} \left(\sum_ {i} \frac {1}{n} f _ {i} ^ {2}\right). \tag {5} \\ \end{array}
$$
Applying Jensen's formula on the convex function  $x^2$ , we have  $\left(\sum_{i} \frac{1}{n} a_i\right)^2 \leq \sum_{i} \frac{1}{n} a_i^2$  for any  $a_i$ , thus  $(\bar{f})^2 \leq \sum_{i} \frac{1}{n} f_i^2$  and  $(\bar{g})^2 \leq \sum_{i} \frac{1}{n} g_i^2$ . Combining Equation (4) and Equation (5) with these two inequalities gives the desired result.
$^{2}$ https://en.wikipedia.org/wiki/Law_of_total_variance
Algorithm 1 Train Loss Computation
Input: data  $X_{B} = \{(\pmb{x}_{b},y_{b})\}_{b}$  , current  $\epsilon$  target  $\epsilon^t$  network  $\pmb{f}$    
Output: A differentiable loss  $L$ $\mathcal{L}_{\mathrm{IBP}} = \sum_{b\in \mathcal{B}}\mathcal{L}_{\mathrm{IBP}}(\pmb{x}_b,y_b,\epsilon) / |\mathcal{B}|.$    
if  $\epsilon <  \epsilon^{t}$  then //  $\epsilon$  annealing regularisation from Shi et al. (2021)  $\mathcal{L}_{\mathrm{fast}} = \lambda \cdot (\mathcal{L}_{\mathrm{tightness}} + \mathcal{L}_{\mathrm{relu}})$  return  $\mathcal{L}_{\mathrm{IBP}} + \epsilon /\epsilon^t\cdot \mathcal{L}_{\mathrm{fast}}$ $\mathcal{L}_{\mathrm{TAPS}} = \sum_{b\in \mathcal{B}}L_{\mathrm{TAPS}}(\pmb {x}_b,y_b,\epsilon) / |\mathcal{B}|.$    
return  $\mathcal{L}_{\mathrm{IBP}}\cdot \mathcal{L}_{\mathrm{TAPS}}$
Table 5: The training epoch and learning rate settings.
Dataset	Batch size	Total epochs	Annealing epochs	Decay-1	Decay-2
MNIST	256	70	20	50	60
CIFAR-10	128	160	80	120	140
TINYIMAGENET	128	80	20	60	70
B.1 TAPS Training Procedure
To obtain state-of-the-art performance with IBP, various training techniques have been developed. We use two of them:  $\epsilon$ -annealing (Gowal et al., 2018) and initialization and regularization for stable box sizes (Shi et al., 2021).  $\epsilon$ -annealing slowly increases the perturbation magnitude  $\epsilon$  during training to avoid exploding approximation sizes and thus gradients. The initialization of Shi et al. (2021) scales network weights to achieve constant box sizes over network depth. During the  $\epsilon$ -annealing phase, we combine the IBP loss with the ReLU stability regularization  $\mathcal{L}_{\mathrm{fast}}$  (Shi et al., 2021), before switching to the TAPS loss as described in Section 3.4. We formalize this in Algorithm 1. We follow Shi et al. (2021) in doing early stopping based on validation set performance. However, we use TAPS accuracy (see App. C) instead of IBP accuracy as a performance metric.
B.2 Datasets and Augmentation
We use the MNIST (LeCun et al., 2010), CIFAR-10 (Krizhevsky et al., 2009), and TINYIMAGENET (Le and Yang, 2015) datasets, all of which are freely available with no license specified.
The data preprocessing mostly follows Müller et al. (2022a). For MNIST, we do not apply any preprocessing. For CIFAR-10 and TINYIMAGENET, we normalize with the dataset mean and standard deviation (after calculating perturbation size) and augment with random horizontal flips. For CIFAR-10, we apply random cropping to  $32 \times 32$  after applying a 2 pixel padding at every margin. For TINYIMAGENET, we apply random cropping to  $56 \times 56$  during training and center cropping during testing.
B.3 Model Architectures
Unless specified otherwise, we follow Shi et al. (2021); Müller et al. (2022a) and use a CNN7 with Batch Norm for our main experiments. CNN7 is a convolutional network with 7 convolutional and linear layers. All but the last linear layer are followed by a Batch Norm and ReLU layer.
B.4 Training Hyperparameter Details
We follow the hyperparameter choices of Shi et al. (2021) for  $\epsilon$ -annealing, learning rate schedules, batch sizes, and gradient clipping (see Table 5). We set the initial learning rate to 0.0005 and decrease it by a factor of 0.2 at Decay-1 and -2. We set the gradient clipping threshold to 10.
We use additional  $L_{1}$  regularization in some settings where we observe signs of overfitting. We report the  $L_{1}$  regularization and split position chosen for different settings in Table 6 and Table 8.
Table 6: Hyperparameters for TAPS.
Dataset	ε	#ReLUs in Classifier	L1	w
MNIST	0.1	3	1e-6	5
0.3	1	0	5
CIFAR-10	2/255	5	2e-6	5
8/255	1	2e-6	5
TINYIMAGENET	1/255	1	0	5
Table 7: Training and certification times for TAPS-trained networks.
Dataset	ε	Train Time (s)	Certify Time (s)
MNIST	0.1	42622	17117
0.3	12417	41624
CIFAR-10	2/255	141281	166474
8/255	27017	26968
TINYIMAGENET	1/255	306036	23497
We train using single NVIDIA GeForce RTX 3090 for MNIST and CIFAR-10 and single NVIDIA TITAN RTX for TINYIMAGENET. Training and certification times are reported in Table 7 and Table 9.
B.5 Certification Details
We combine IBP (Gowal et al., 2018), CROWN-IBP (Zhang et al., 2020), and MN-BAB (Ferrari et al., 2022) for certification, running the most precise but also computationally costly MN-BAB only on samples not certified by the other methods. We use the same configuration for MN-BAB as Müller et al. (2022a). The certification is run on a single NVIDIA TITAN RTX.
MN-BAB Ferrari et al. (2022) is a state-of-the-art (Brix et al., 2023; Müller et al., 2022c) neural network verifier, combining the branch-and-bound paradigm (Bunel et al., 2020) with precise multi-neuron constraints (Müller et al., 2022b; Singh et al., 2019b).
We use a mixture of strong adversarial attacks to evaluate adversarial accuracy. First, we run PGD attacks with 5 restarts and 200 iterations each. Then, we run MN-BaB to search for adversarial examples with a timeout of 1000 seconds.
C Extended Evaluation
TAPS Accuracy as GoF In practice, we want to avoid certifying every model with expensive certification methods, especially during hyperparameter tuning and for early stopping. Therefore, we need a criterion to select models. In this section, we aim to show that TAPS accuracy (accuracy of the latent adversarial examples) is a good proxy for goodness of fit (GoF).
We compare the TAPS accuracy to adversarial and certified accuracy with all models we get on MNIST and CIFAR-10. The result is shown in Table 10. We can see that the correlations between TAPS accuracy and both the adversarial and the certified accuracy are close to 1. In addition, the differences are small and centered at zero, with a small standard deviation. Therefore, we conclude that TAPS accuracy is a good estimate of the true robustness, thus a good measurement of GoF. In all the experiments, we perform model selection based on the TAPS accuracy.
Training Difficulty Since TAPS is merely a training technique, we can test TAPS-trained models using a different classifier split. By design, if the training is successful, then under a given classifier split for testing, the model trained with the same split should have the best TAPS accuracy. Although this is often true, we find that in some cases, a smaller classifier split results in higher TAPS accuracy, indicating optimization issues.
We measure TAPS accuracy for models trained with IBP and TAPS using different splits for CIFAR-10 (Figure 10) and MNIST (Figure 11). We observe that for CIFAR-10  $\epsilon = 2/255$  and MNIST, the models trained and tested with the classifier/extractor split achieve the highest TAPS accuracies, as expected, indicating a relatively well-behaved optimization problem. However, for CIFAR-10  $\epsilon = 8/255$ , the model trained with a classifier size of 1 achieves the highest TAPS accuracy for all
Table 8: Hyperparameter for STAPS.
Dataset	ε	#ReLUs in Classifier	L1	w	τ/ε
MNIST	0.1	1	2e-5	5	0.4
0.3	1	2e-6	5	0.6
CIFAR-10	2/255	1	2e-6	2	0.1
8/255	1	2e-6	5	0.7
TINYIMAGENET	1/255	2	1e-6	5	0.6
Table 9: Training and certification times for STAPS-trained networks.
Dataset	ε	Train Time (s)	Certify Time (s)
MNIST	0.1	19865	12943
0.3	23613	125768
CIFAR-10	2/255	47631	398245
8/255	48706	77793
TINYIMAGENET	1/255	861639	35183
test splits and also the best adversarial and certified accuracy (see Figure 7b and Table 12). This indicates that, in this setting, an earlier split and thus larger classifier component induces a (too) difficult optimization problem, leading to worse overall performance.
Split Position We report detailed results for the experiment visualized in Figure 7b (Section 4.2) in Table 11 and Table 12. We additionally report results on TINYIMAGENET in Table 13.
IBP Regularization We repeat the experiment reported on for MNIST in Table 2 for TINYIMAGENET, presenting results in Table 14. We generally observe the same trends, although peak certification performance is achieved slightly later at  $\theta_{\mathrm{TAPS}} = 10$  instead of  $\theta_{\mathrm{TAPS}} = 5$ .
Repeatability Due to the large computational cost of up to 10 GPU-days for some experiments (see Tables 7 and 9), we could not repeat all experiments multiple times to report full statistics. However, we report statistics for the best-performing method for MNIST at  $\epsilon = 0.1$  and  $\epsilon = 0.3$  and CIFAR-10 at  $\epsilon = 2/255$  and  $\epsilon = 8/255$  (see Table 15). We generally observe small standard deviations, indicating good repeatability of our results.
TAPS and all other certified training methods can only be applied to mathematically well-defined perturbations of the input such as  $\ell_p$ -balls, while real-world robustness may require significantly more complex perturbation models. Further and similarly to other unsound certified training methods, TAPS introduces a new hyperparameter, the split position, that can be tuned to improve performance further beyond the default choice of 1 ReLU layer in the classifier. Finally, while training with TAPS is similarly computationally expensive as with other recent methods, it is notably more computationally expensive than simple certified training methods such as IBP.
E Reproducibility
We publish our code, trained models, and detailed instructions on how to reproduce our results at github.com/eth-sri/taps, providing an anonymized version to the reviewers<sup>3</sup>. Additionally, we provide detailed descriptions of all hyper-parameter choices, data sets, and preprocessing steps in App. B.
<sup>3</sup>We provide the codebase with the supplementary material, including instructions on how to download our trained models.
(a) IBP-trained
(b) SABR-trained
Figure 9: Distribution of the worst-case loss approximation errors over test set samples, depending on the training and bounding method. Positive values correspond to over-approximations and negative values to under-approximations. We use an exact MILP encoding (Tjeng et al., 2019) as reference.
(c) TAPS-trained
Table 10: Comparison of TAPS accuracy with certified and adversarial accuracy.
Dataset	cor(TAPS, cert.)	cor(TAPS, adv.)	TAPS - cert.	TAPS - adv.
MNIST	0.9139	0.9633	0.0122 ± 0.0141	0.0033 ± 0.0079
CIFAR-10	0.9973	0.9989	0.0028 ± 0.0095	-0.0040 ± 0.0077
Figure 10: TAPS accuracy of models trained with different classifier sizes for CIFAR-10.
$$
\epsilon = \frac {2}{2 5 5}
$$
$$
\epsilon = \frac {8}{2 5 5}
$$
Figure 11: TAPS accuracy of models trained with different classifier sizes for MNIST.
$$
\epsilon = 0. 1
$$
$$
\epsilon = 0. 3
$$
Table 11: Effect of split position on accuracies  $[\% ]$  for fixed model size on MNIST.
ε	#ReLUs in Classifier	Nat.	Adv.	Cert.
MN-BaB	IBP
0.1	0	98.87	98.16	98.13	97.83
1	99.06	98.37	98.31	96.27
2	99.16	98.35	98.25	87.82
3	99.19	98.51	98.39	62.83
4	99.28	98.47	98.03	4.75
5	99.22	98.51	98.17	9.76
6	99.09	98.45	98.27	81.89
0.3	0	97.60	93.37	93.15	93.08
1	97.94	94.01	93.62	92.76
2	98.16	94.18	93.55	91.85
3	98.63	94.48	93.03	89.40
4	98.7	94.85	93.44	89.52
5	98.63	94.64	93.26	89.15
6	98.88	95.11	92.70	85.03
Table 12: Effect of split position on accuracies  $[\% ]$  for fixed model size on CIFAR-10.
ε	#ReLUUs	Nat.	Adv.	Cert.
MN-BaB	IBP
2/255	0	67.27	56.32	56.14	53.54
1	70.10	57.78	57.48	41.86
2	70.74	57.83	57.39	40.24
3	71.88	58.89	58.23	34.41
4	72.45	60.38	59.47	31.88
5	75.09	63.00	61.56	24.36
6	75.40	62.73	61.11	24.90
8/255	0	48.15	34.63	34.60	34.26
1	49.76	35.29	35.10	32.92
2	47.28	33.54	33.12	28.94
3	48.76	33.50	33.12	29.14
4	50.19	34.78	34.35	29.14
5	50.2	34.33	33.72	28.83
6	51.03	35.25	34.44	29.97
Table 13: Effect of split position on accuracies  $[\% ]$  for fixed model size on TINYIMAGENET.
ReLU	TAPS	STAPS
Nat. (%)	Adv. (%)	Cert. (%)	Train (s)	Certify (s)	Nat. (%)	Adv. (%)	Cert. (%)	Train (s)	Certify (s)
1	28.34	20.94	20.82	306 036	23 497	28.75	22.25	22.04	350 924	35 894
2	27.02	20.94	20.84	944 520	32 407	28.98	22.40	22.16	861 639	35 183
Table 14: Effect of IBP regularization and the TAPS gradient expanding coefficient  $\alpha$  for TINYIMAGENET  $\epsilon = \frac{1}{255}$ .
\( {w}_{\text{TAPS }} \)	Avg time (s)	Nat. (%)	Adv. (%)	Cert. (%)
\( {\mathcal{L}}_{\mathrm{{IBP}}} \)	0.28	25.00	19.72	19.72
1	1.17	25.83	20.24	20.22
5	2.34	28.34	20.94	20.82
10	4.12	28.23	21.05	20.89
20	5.94	28.44	20.68	20.44
Table 15: Mean and standard deviation (over three repeats) for the method with best certified accuracy.
Dataset	ε∞	Method	Nat. [%]	Adv. [%]	Cert. [%]
MNIST	0.1	TAPS	99.22 ± 0.03	98.45 ± 0.06	98.28 ± 0.10
0.3	TAPS	97.96 ± 0.04	93.96 ± 0.04	93.57 ± 0.02
CIFAR-10	2/255	STAPS	79.75 ± 0.23	65.91 ± 0.12	62.72 ± 0.23
8/255	TAPS	49.07 ± 0.61	34.75 ± 0.47	34.57 ± 0.46