SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Attacker's knowledge In our certification experiments in Appendix E, we assume the attacker has full knowledge of the malware detector and certification scheme. When testing published attacks in Appendix F, we consider both white-box and black-box access to the malware detector. In the black-box setting, the attacker may make an unlimited number of queries to the malware detector without observing its internal operation. We permit access to detection confidence scores, which are returned alongside predictions even in the black-box setting. In the white-box setting, the attacker can additionally inspect the malware detector's source code. Such a strong assumption is needed for white-box attacks against neural network-based detectors that compute loss gradients with respect to the network's internal representation of the input file [20, 17].
D Experimental setup for malware detection case study
In this appendix, we detail the experimental setup for our malware detection case study.
Though our methods are compatible with executable files of any format, in our experiments we focus on the Portable Executable (PE) format [87], since datasets, malware detection models and adversarial attacks are more extensively available for this format. Moreover, PE format is the standard for executables, object files and shared libraries in the Microsoft Windows operating system, making it an attractive target for malware authors. We use two PE datasets which are summarized in Table 3 and described below.
Sleipnir2 This dataset attempts to replicate data used in past work [54], which was not published with raw samples. We were able to obtain the raw malicious samples from a public malware repository called VirusShare [88] using the provided hashes. However, since there is no similar public repository for benign samples, we followed established protocols [89, 90, 20] to collect a new set of benign samples. Specifically, we set up a Windows 7 virtual machine with over 300 packages installed using Chocolatey package manager [91]. We then extracted PE files from the virtual machine, which were assumed benign $^{4}$ , and subsampled them to match the number of malicious samples. The dataset is randomly split into training, validation and test sets with a ratio of  $60\%$ ,  $20\%$  and  $20\%$  respectively.
VTFeed This dataset was first used in recent attacks on end-to-end ML-based malware detectors [17]. It was collected from VirusTotal—a commercial threat intelligence service—by sampling PE files from the live feed over a period of two weeks in 2020. Labels for the files were derived from the 68 antivirus (AV) products aggregated on VirusTotal at the time of collection. Files were labeled malicious if they were flagged malicious by 40 or more of the AV products, they were labeled benign if they were not flagged malicious by any of the AV products, and any remaining files were excluded. Following Lucas et al. [17], the dataset is randomly split into training, validation and test sets with a ratio of  $80\%$ ,  $10\%$ , and  $10\%$  respectively.
We note that VTFeed comes with strict terms of use, which prohibit us from loading it on our high performance computing (HPC) cluster. As a result, we use Sleipnir2 for comprehensive experiments (e.g., varying  $p_{\mathrm{del}}$ ,  $\eta$ ) on the HPC cluster, and VTFeed for a smaller selection of experiments run on a local server.
4Chocolatey packages are validated against VirusTotal [92].
D.2 Malware detection models
We experiment with malware detection models based on MalConv [48]. MalConv was one of the first end-to-end neural network models proposed for malware detection—i.e., it learns to classify directly from raw byte sequences, rather than relying on manually engineered features. Architecturally, it composes a learnable embedding layer with a shallow convolutional network. A large window size and stride of 500 bytes are employed to facilitate scaling to long byte sequences. Though MalConv is compatible with arbitrarily long byte sequences in principle, we truncate all inputs to 2MB to support training efficiency. We use the original parameter settings and training procedure [48], except where specified in Appendix D.5.
Using MalConv as a basis, we consider three models as described below.
NS A vanilla non-smoothed (NS) MalConv model. This model serves as a non-certified, non-robust baseline—i.e., no specific techniques are employed to improve robustness to evasion attacks and certification is not supported.
RS-Abn A smoothed MalConv model using the randomized ablation smoothing mechanism proposed by Levine and Feizi [24] and reviewed in Appendix I. This model serves as a certified robust baseline, albeit covering a more restricted threat model than the edit distance threat model we propose in Section 2. Specifically, it supports robustness certification for the Hamming distance threat model, where the adversary is limited to substitution edits  $(O = \{\mathrm{sub}\})$ . Since Levine and Feizi's formulation is for images, several modifications are required to support malware detection as described in Appendix G. To improve convergence, we also apply gradient clipping when learning parameters in the embedding layer (see Appendix G). We consider variants of this model for different values of the ablation probability  $p_{\mathrm{ab}}$ .
RS-Del A smoothed MalConv model using our proposed randomized deletion smoothing mechanism. This model supports robustness certification for the generalized edit distance threat model where  $O \subseteq \{\mathrm{del}, \mathrm{ins}, \mathrm{sub}\}$ . We consider variants of this model for different values of the deletion probability  $p_{\mathrm{del}}$ , decision thresholds  $\eta$ , and whether deletion/certification is performed at the byte-level (BYTE) or chunk-level (INSN). We perform chunking as illustrated in Figure 3—i.e., we chunk bytes that correspond to distinct machine instructions using the Ghidra disassembler.
D.3 Controlling false positive rates
Malware detectors are typically tuned to achieve a low false positive rate (FPR) (e.g., less than  $0.1 - 1\%$ ) since producing too many false alarms is a nuisance to users. To make all malware detection models comparable, we calibrate the FPR to  $0.5\%$  on the test set for the experiments reported in Appendix E and  $0.5\%$  on the validation set for the experiments reported in Appendix F unless otherwise noted. This calibration is done by adjusting the decision threshold of the base MalConv model.
D.4 Compute resources
Experiments for the Sleipnir2 dataset were run a high performance computing (HPC) cluster, where the requested resources varied depending on the experiment. We generally requested a single NVIDIA P100 GPU when training and certifying models. Experiments for the VTFeed dataset were run on a local server due to restrictive terms of use. Compute resources and approximate wall clock running times are reported in Tables 4 and 5 for training and certification for selected parameter settings. Running times for other parameters settings are lower than the ones reported in these tables.
D.5 Parameter settings
We specify the parameter settings and training procedure for MalConv, which is used standalone in NS, and as a base model for the smoothed models RS-Del and RS-Abn. Table 6 summarizes our setup, which is consistent across all three models except where specified. We follow the authors of MalConv [48] when setting parameters for the model and the optimizer, however we set a larger
<sup>5</sup>https://www.av-comparatives.org/testmethod/false-alarm-tests/
Table 4: Compute resources used for training. Note that the wall clock times reported here are for an unoptimized implementation where the smoothing mechanism is executed on the CPU.
Dataset	Requested resources	Model	Parameters	Time	Notes
Sleipnir2	1 NVIDIA P100 GPU, 4 cores on Intel Xeon Gold 6326 CPU	NS	-	22 hr	Trained for 50 epochs, converged in 15 epochs
RS-Del	BYTE, pdel = 95%	39 hr	Trained for 100 epochs, converged in 50 epochs
RS-Del	INSN, pdel = 95%	48 hr	Trained for 100 epochs, converged in 20 epochs
RS-Abn	pab = 95%	40 hr	Trained for 100 epochs, converged in 90 epochs
VTFeed	1 NVIDIA RTX3090 GPU, 6 cores on AMD Ryzen Threadripper PRO 3975WX CPU	NS	-	139 hr	Trained for 100 epochs, converged in 25 epochs
RS-Del	BYTE, pdel = 97%	152 hr	Trained for 100 epochs, converged in 20 epochs
Table 5: Compute resources used for certification on the test set. The evaluation dataset is partitioned and processed on multiple compute nodes with the same specifications. The reported time is the sum of wall times on each compute node. Note that the times reported are for an unoptimized implementation where the smoothing mechanism is executed on the CPU.
Dataset	Requested resources	Model	Parameters	Time
Sleipnir2	1 NVIDIA P100 GPU, 12 cores on Intel Xeon Gold 6326 CPU	NS	-	5 min
RS-Del	BYTE, pdel = 95%	65 hr
RS-Del	INSN, pdel = 95%	140 hr
RS-Abn	pab = 95%	210 hr
VTFeed	1 NVIDIA RTX3090 GPU, 6 cores on AMD Ryzen Threadripper PRO 3975WX CPU	NS	-	4 min
RS-Del	BYTE, pdel = 97%	500 hr
maximum input size of 2MiB to accommodate larger inputs without clipping. Due to differences in available GPU memory for the Sleipnir2 and VTFeed experiments, we use a larger batch size for VTFeed than for Sleipnir2. We also set a higher limit on the maximum number of epochs for VTFeed, as it is a larger dataset, although the NS and RS-Del models converge within 50 epochs for both datasets. To stabilize training for the smoothed models (RS-Del and RS-Abn), we modify the smoothing mechanisms during training only to ensure at least 500 raw bytes are preserved. This may limit the number of deletions for RS-Del and the number of ablated (masked) bytes for RS-Abn. For RS-Abn, we clip the gradients for the embedding layer to improve convergence (see Appendix G).
E Evaluation of robustness certificates for malware detection
In this appendix, we evaluate the robustness guarantees and accuracy of RS-Del for malware detection. We consider two instantiations of the edit distance threat model. First, in Appendix E.1, we consider the Levenshtein distance threat model, where the attacker's elementary edits are unconstrained and may include deletions, insertions and substitutions. Then, in Appendix E.2, we consider the more restricted Hamming distance threat model, where an attacker is only able to perform substitutions. We summarize our findings in Appendix E.3. Overall, we find that RS-Del generates robust predictions with minimal impact on model accuracy for the Levenshtein distance threat model, and outperforms RS-Abn [24] for the Hamming distance threat model.
Table 6: Parameter settings for MalConv, the optimizer and training procedure. Parameter settings are consistent across all malware detection models (NS, RS-Del, RS-Abn) except where specified.
Parameter	Values
MalConv	Max input size	2097152
Embedding size	8
Window size	500
Channels	128
Optimizer	Python class	torch.optim.SGD
Learning rate	0.01
Momentum	0.9
Weight decay	0.001
Training	Batch size	24 (Sleipnir2), 32 (VTFeed)
Max. epoch	50 (Sleipnir2), 100 (VTFeed)
Min. preserved bytes	500 (RS-Del, RS-Abn), NA (NS)
Embedding gradient clipping	0.5 (RS-Abn), ∞ (RS-Del, NS)
Early stopping	If validation loss does not improve after 10 epochs
We report the following quantities in our evaluation:
- Certified radius (CR). The radius of the largest robustness certificate that can be issued for a given input, model and certification method. Note that this is a conservative measure of robustness since it is tied to the certification method. The median CR is reported on the test set.
- Certified accuracy [13, 14], also known as verified-robust accuracy [66, 55], evaluates robustness certificates and accuracy of a model simultaneously with respect to a test set. It is defined as the fraction of instances in the test set  $\mathbb{D}$  for which the model  $f$ 's prediction is correct and certified robust at radius  $r$  or greater:
$$
\operatorname {C e r t A C C} _ {r} (\mathbb {D}) = \sum_ {(\boldsymbol {x}, y) \in \mathbb {D}} \frac {\mathbf {1} _ {f (\boldsymbol {x}) = y} \mathbf {1} _ {\mathrm {C R} (\boldsymbol {x}) \geq r}}{| \mathbb {D} |} \tag {22}
$$
where  $\operatorname{CR}(\pmb{x})$  denotes the certified radius for input  $\pmb{x}$  returned by the certification method.
- Clean accuracy. The fraction of instances in the test set for which the model's prediction is correct.
We briefly mention default parameter settings for the experiments presented in this appendix. When approximating the smoothed models (RS-Del and RS-Abn) we sample  $n_{\mathrm{pred}} = 1000$  perturbed inputs for prediction and  $n_{\mathrm{bnd}} = 4000$  perturbed inputs for certification, while setting the significance level  $\alpha$  to 0.05. Unless otherwise specified, we set the decision thresholds for the smoothed models so that  $\eta = 0$ . After fixing  $\eta$ , the decision thresholds for the base models are tuned to yield a false positive rate of  $0.5\%$ . We note that the entire test set is used when reporting metrics and summary statistics in this appendix.
E.1 Levenshtein distance threat model
We first present results for the Levenshtein distance threat model, where the attacker's elementary edits are unconstrained ( $O = \{\mathrm{del}, \mathrm{ins}, \mathrm{sub}\}$ ). We vary three parameters associated with RS-Del: the deletion probability  $p_{\mathrm{del}}$ , the decision thresholds of the smoothed model  $\eta$ , and the level of sequence chunking (i.e., whether sequences are chunked at the byte-level or instruction-level). We use NS as a baseline as there are no prior certified defenses for the Levenshtein distance threat model to our knowledge.
Figure 4: Certified accuracy for RS-Del as a function of the radius in bytes (left horizontal axis), radius normalized by file size (right horizontal axis) and byte deletion probability  $p_{\mathrm{del}}$  (line styles). The results are plotted for the Sleipnir2 test set under the byte-level Levenshtein distance threat model (with  $O = \{\mathrm{del}, \mathrm{ins}, \mathrm{sub}\}$ ). The grey vertical lines in the left plot represent the best achievable certified radius for RS-Del (setting  $\mu_y = 1$  in the expressions in Table 1).
Figure 5: Certified accuracy for RS-Del with chunk-level deletion (INSN) as a function of the radius in chunks (left horizontal axis), radius normalized by sequence length in chunks (right horizontal axis) and chunk deletion probability  $p_{\mathrm{del}}$  (line styles). The results are plotted for the Sleipnir2 test set under the chunk-level Levenshtein distance threat model (with  $O = \{\mathrm{del}, \mathrm{ins}, \mathrm{sub}\}$ ). The grey vertical lines in the left plot represent the best achievable certified radius for RS-Del (setting  $\mu_y = 1$  in the expressions in Table 1).
Certified accuracy Figure 4 plots the certified accuracy of RS-Del using byte-level deletion as a function of the radius (left horizontal axis), radius normalized by file size (right horizontal axis) on the Sleipnir2 dataset for several values of  $p_{\mathrm{del}}$ . We observe that the curves for larger values of  $p_{\mathrm{del}}$  approximately dominate the curves for smaller values of  $p_{\mathrm{del}}$ , for  $p_{\mathrm{del}} \leq 99.5\%$  (i.e., the accuracy is higher or close for all radii). This suggests that the robustness of RS-Del can be improved without sacrificing accuracy by increasing  $p_{\mathrm{del}}$  up to  $99.5\%$ . However, for the larger value  $p_{\mathrm{del}} = 99.9\%$ , we observe a drop in certified accuracy of around  $10\%$  for smaller radii and an increase for larger radii. By normalizing with respect to the file size, we can see that our certificate is able to certify up to  $1\%$  of the file size. We also include an analogous plot for chunk-level deletion in Figure 5 which demonstrates similar behavior. We note that chunk-level deletion arguably provides stronger guarantees, since the effective radius for chunk-level Levenshtein distance is larger than for byte-level Levenshtein distance.
It is interesting to relate these certification results to published evasion attacks. Figure 4 shows that we can achieve a certified accuracy in excess of  $90\%$  at a Levenshtein distance radius of 128 bytes when  $p_{\mathrm{del}} = 99.5\%$ . This radius is larger than the median Levenshtein distance of two attacks that manipulate headers of PE files [37, 38] (see Table 9 in Appendix F). We can therefore provide reasonable robustness guarantees against these two attacks. However, a radius of 128 bytes is orders of magnitude smaller than the median Levenshtein distances of other published attacks which range from tens of KB [17, 20] to several MB [31] (also reported in Table 9). While some of these attacks
Table 7: Clean accuracy and robustness metrics for RS-Del as a function of the dataset (Sleipnir2 and VTFeed), deletion probability  $p_{\mathrm{del}}$  and deletion level (BYTE or INSN). All metrics are computed on the test set. Here "abstain rate" refers to the fraction of test instances for which RS-Del abstains (line 7 in Figure 1), and "UB" refers to an upper bound on the median CR for a best case smoothed model (based on Table 1 with  $\mu_y = 1$ ). A good tradeoff is achieved when  $p_{\mathrm{del}} = 99.5\%$  for both the byte-level (BYTE) and chunk-level (INSN) certificates (highlighted in bold face below).
Dataset	Model	Parameters	Clean accuracy (Abstain rate) %	Median CR (UB)	Median NCR %
Sleipnir2	NS	-	98.9	-	-	-	-
RS-Del	BYTE, pdel = 90%	97.1	(0.2)	6	(6)	0.0023
BYTE, pdel = 95%	97.8	(0.0)	13	(13)	0.0052
BYTE, pdel = 97%	97.4	(0.1)	22	(22)	0.0093
BYTE, pdel = 99%	98.1	(0.1)	68	(68)	0.0262
BYTE, pdel = 99.5%	96.5	(0.2)	137	(138)	0.0555
BYTE, pdel = 99.9%	83.7	(3.4)	688	(692)	0.2269
INSN, pdel = 90%	97.9	(0.1)	6	(6)	0.0026
INSN, pdel = 95%	97.8	(0.1)	13	(13)	0.0056
INSN, pdel = 97%	98.3	(0.0)	22	(22)	0.0095
INSN, pdel = 99%	97.6	(0.1)	68	(68)	0.0292
INSN, pdel = 99.5%	96.8	(0.2)	137	(138)	0.0589
INSN, pdel = 99.9%	86.1	(0.2)	689	(692)	0.2982
VTFeed	NS	-	98.9	-	-	-	-
RS-Del	BYTE, pdel = 97%	92.1	(0.9)	22	(22)	0.0045
BYTE, pdel = 99%	86.9	(0.8)	68	(68)	0.0122
arguably fall outside an edit distance constrained threat model, we consider them in our empirical evaluation of robustness in Appendix F.
Clean accuracy and abstention rates Table 7 reports clean accuracy for RS-Del and the non-certified NS baseline. It also reports abstention rates for RS-Del, the median certified radius (CR), and the median certified radius normalized by file size (NCR). We find that clean accuracy for Sleipnir2 follows similar trends as certified accuracy: it is relatively stable for  $p_{\mathrm{del}}$  in the range  $90 - 99.5\%$ , but drops by more than  $10\%$  at  $p_{\mathrm{del}} = 99.9\%$ . We note that the clean accuracy of RS-Del (excluding  $p_{\mathrm{del}} = 99.9\%$ ) is at most  $3\%$  lower than the NS baseline for Sleipnir2 and at most  $7\%$  lower than the NS baseline for VTFeed. We observe minimal differences in the results for chunk-level (INSN) and byte-level (BYTE) deletion smoothing, but note that the effective CR is larger for chunk-level smoothing, since each chunk may contain several bytes.
Accuracy under high deletion It may be surprising that RS-Del can maintain high accuracy even when deletion is aggressive. We offer some possible explanations. First, we note that even with a high deletion probability of  $p_{\mathrm{del}} = 99.9\%$ , the smoothed model accesses almost all of the file in expectation, as it aggregates  $n_{\mathrm{pred}} = 1000$  predictions from the base model each of which accesses a random  $0.1\%$  of the file in expectation. Second, we posit that malware detection may be "easy" for RS-Del on these datasets. This could be due to the presence of signals that are robust to deletion (e.g., file size or byte frequencies) or redundancy of signals (i.e., if a signal is deleted in one place it may be seen elsewhere).
Decision threshold We demonstrate how the decision thresholds  $\eta$  introduced in Section 3.1 can be used to trade off certification guarantees between classes. We consider normalized decision thresholds where  $\sum_{y} \eta_{y} = 1$  and  $\eta_{y} \in [0,1]$ . We only specify the value of  $\eta_{1}$  when discussing our results, noting that  $\eta_{0} = 1 - \eta_{1}$  in our two-class setting.
Table 8 provides error rates and robustness metrics for several values of  $\eta_{1}$ , using byte-level Levenshtein distance with  $p_{\mathrm{del}} = 99.5\%$ . When varying  $\eta_{1}$ , we also vary the decision threshold of the base model to achieve a target false positive rate (FPR) of  $0.5\%$ . Looking at the table, we see that  $\eta_{1}$  has minimal impact on the false negative rate (FNR), which is stable around  $7\%$ . However, there
Table 8: Impact of the smoothed decision threshold  $\eta_{1}$  on false negative error rate (FNR) and median certified radius (CR) for malicious and benign files. The false positive rate (FPR) is set to a target value of  $0.5\%$  by varying the decision threshold of the base model. The results are reported for Sleipnir2 with  $p_{\mathrm{del}} = 99.5\%$  using byte-level Levenshtein distance. "UB" refers to an upper bound on the median CR for a best case smoothed model (based on Table 1 with  $\mu_y = 1$ ).
η1 (%)	FNR (%)	FPR (%)	Median CR (UB)
Malicious	Benign
50	6.8	0.5	137	(138)	137	(138)
25	6.9	0.5	275	(276)	57	(57)
10	6.8	0.5	455	(459)	20	(21)
5	6.6	0.5	578	(597)	10	(10)
1	7.1	0.5	582	(918)	1	(2)
0.5	6.9	0.5	506	(1057)	0	(0)
is a significant impact on the median CR (and theoretical upper bound), as reported separately for each class. The median CR is balanced for both the malicious and benign class when  $\eta_{1} = 50\%$ , but favours the malicious class as  $\eta_{1}$  is decreased. For instance when  $\eta_{1} = 5\%$  a significantly larger median CR is possible for malicious files (137 to 578) at the expense of the median CR for benign files (137 to 10). This asymmetry in the class-specific CR is a feature of the theory—that is, in addition to controlling a tradeoff between error rates of each class,  $\eta_{1}$  also controls a tradeoff between the CR for each class (see Table 1).
Figure 6 plots the certified true positive rate (TPR) and true negative rate (TNR) of RS-Del on the Sleipnir2 dataset for several values of  $\eta_{1}$ . The certified TPR and TNR can be interpreted as class-specific analogues of the certified accuracy. Concretely, the certified TPR (TNR) at radius  $r$  is the fraction of malicious (benign) instances in the test set for which the model's prediction is correct and certified robust at radius  $r$ . The certified TPR and TNR jointly measure accuracy and robustness and complement the metrics reported in Table 8. Looking at Figure 6, we see that the certified TNR curves drop more rapidly to zero than the certified TPR curves as  $\eta_{1}$  decreases. Again, this suggests decreasing  $\eta_{1}$  sacrifices the certified radii of benign instances to increase the certified radii of malicious instances. We note that the curves for  $\eta_{1} = 50\%$  correspond to the same setting as the certified accuracy curve in Figure 4 (with  $p_{\mathrm{del}} = 99.5\%$ ).
E.2 Hamming distance threat model
We now turn to the more restricted Hamming distance threat model, where the attacker is limited to performing substitutions only ( $O = \{\mathrm{sub}\}$ ). We choose to evaluate this threat model as it is covered in previous work on randomized smoothing, called randomized ablation [24] (abbreviated RS-Abn), and can serve as a baseline for comparison with our method. Recall that we adapt RS-Abn for malware detection by introducing a parameter called  $p_{\mathrm{ab}}$ , which is the fraction of bytes that are "ablated" (replaced by a special masked value) (see Appendix D.2). This parameter is analogous to  $p_{\mathrm{del}}$  in RS-Del, except that the number of ablated bytes is deterministic in RS-Abn, whereas the number of deleted bytes is random in RS-Del. We compare RS-Del and RS-Abn for varying values of  $p_{\mathrm{del}}$  and  $p_{\mathrm{ab}}$  using the Sleipnir2 dataset and byte-level Hamming distance.