SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Certified accuracy Figure 2 plots the certified accuracy of RS-Del and RS-Abn for three values of  $p_{\mathrm{del}}$  and  $p_{\mathrm{ab}}$ . We observe that the certified accuracy is uniformly larger for our proposed method RS-Del than for RS-Abn when  $p_{\mathrm{del}} = p_{\mathrm{ab}}$ . The superior certification performance of RS-Del is somewhat surprising given it is not optimized for the Hamming distance threat model. One possible explanation relates to the learning difficulty of RS-Abn compared with RS-Del. Specifically, we find that stochastic gradient descent is slower to converge for RS-Abn despite our attempts to improve convergence (see Appendix G). Recall, that RS-Del provides certificates for any of the threat models in Table 1—in addition to the Hamming distance certificate—without needing to modify the smoothing mechanism.
Figure 6: Certified true positive rate (TPR) and true negative rate (TNR) of RS-Del as a function of the certificate radius  $r$  (horizontal axis) and the decision threshold  $\eta_1$  (line style). The results are plotted for the Sleipnir2 test set for byte-level deletion (BYTE) with  $p_{\mathrm{del}} = 99.5\%$  under the Levenshtein distance threat model (with  $O = \{\mathrm{del}, \mathrm{ins}, \mathrm{sub}\}$ ). It is apparent that  $\eta_1$  controls a tradeoff in the certified radius between the malicious (measured by TPR) and benign (measured by TNR) classes. Note that in this setting, a non-smoothed, non-certified model (NS) achieves a clean TPR and TNR of  $98.2\%$  and  $99.5\%$  respectively.
Tightness RS-Abn is provably tight, in the sense that it is not possible to issue a larger Hamming distance certificate unless more information is made available to the certification mechanism or the ablation smoothing mechanism is changed. This tightness result for RS-Abn, together with the empirical results in Figure 2, suggests that RS-Del produces certificates which are tight or close to tight in practice, at least for the Hamming distance threat model. This is an interesting observation, since it is unclear how to derive a tight, computationally tractable certificate for RS-Del.
E.3 Summary
Our evaluation shows that RS-Del provides non-trivial robustness guarantees with a low impact on accuracy. The certified radii we observe are close to the best radii theoretically achievable using our mechanism. For the Levenshtein byte-level edit distance threat model, we obtain radii of a few hundred bytes in size, which can certifiably defend against attacks that edit headers of PE files [37, 38, 52]. However, certifying robustness against more powerful attacks that modify thousands or millions of bytes remains an open challenge. By varying the detection threshold, we show that certification can be performed asymmetrically for benign and malicious instances. This can boost the certified radii of malicious instances by a factor of 4 in some cases. While there are no prior methods to use as baselines for the Levenshtein distance threat model, our comparisons with RS-Abn [24] for
the Hamming distance threat model show that RS-Del outperforms RS-Abn in terms of both accuracy and robustness.
F Evaluation of robustness to published attacks
In this appendix, we empirically evaluate the robustness of RS-Del to several published evasion attacks. By doing so, we aim to provide a more complete picture of robustness, as our certificates are conservative and may underestimate robustness to real attacks, which are subject to additional constraints (e.g., maintaining executability, preserving a malicious payload, etc.). We introduce the attacks in Appendix F.1, provide details of the experimental setup in Appendix F.2 and discuss the results in Appendix F.3.
F.1 Attacks covered
Table 9: Evasion attacks used in our evaluation. The attack distance refers to the median Levenshtein distance computed on a set of 500 attacked files from the Sleipnir2 test set. We use a closed source implementation of Disp and open source implementations of the remaining attacks based on secml-malware [93].
Attack	Supported settings	Attack distance	Optimizer	Description
Disp [17]	White-box, black-box	17.2 KB	Gradient-guided	Disassembles the PE file and displaces chunks of code to a new section, replacing the original code with semantic nops.
Slack [20]	White-box	34.7 KB	Fast Gradient Sign Method [2]	Replaces non-functional bytes in slack regions or the overlay of the PE file with adversarially-crafted noise.
HDOS [37]	White-box, black-box	58.0 B	Genetic algorithm	Manipulates bytes in the DOS header of the PE file which are not used in modern Windows.
HField [38]	White-box, black-box	17.0 B	Genetic algorithm	Manipulates fields in the header of the PE file (debug information, section names, checksum, etc.) which do not impact functionality.
GAMMA [31]	Black-box	2.10 MB	Genetic algorithm	Appends sections extracted from benign files to the end of a malicious PE file and modifies the header accordingly.
We consider five recently published attacks designed for evading static PE malware detectors as summarized in Table 9. The attacks cover a variety of edit distance magnitudes from tens of bytes to millions of bytes. While attacks that edit millions of bytes arguably fall outside our edit distance-constrained threat model, we include one such attack (GAMMA) to test the limits of our methodology. We note that four of the five attacks are able to operate in a black-box setting and can therefore be applied directly to RS-Del. However, the white-box attacks are designed for neural network malware detectors with a specific architecture. In particular, they assume the network receives a raw byte sequence as input, that the initial layer is an embedding layer, and that gradients can be computed with respect to the output of the embedding layer. Although these architectural assumptions are satisfied by the base MalConv model, they are not satisfied by RS-Del, because additional operations are applied before the embedding layer and the aggregation of base model predictions is not differentiable. In Appendix H, we adapt the white-box attacks for RS-Del by applying two tricks: (1) we apply the smoothing mechanism after the embedding layer, and (2) we replace majority voting with soft aggregation following Salman et al. [94].
Table 10: Success rates of direct attacks against RS-Del and the NS baseline. A lower success rate is better from our perspective as a defender, as it means the model is more robust to the attack. The model that achieves the lowest success rate for each attack/dataset is highlighted in boldface.
Setting	Attack	Dataset	Attack success rate (%)
NS	RS-Del
White-box	Disp [17]	Sleipnir2	73.8	56.7
VTFeed	94.1	74.5
Slack [20]	Sleipnir2	57.9	85.3
VTFeed	96.0	43.9
Black-box	HLOS [37]	Sleipnir2	0.0	0.0
VTFeed	0.0	0.0
HField [38]	Sleipnir2	0.607	0.0
VTFeed	0.990	0.0
Disp [17]	Sleipnir2	0.809	0.0
VTFeed	10.9	0.0
GAMMA [31]	Sleipnir2	99.2	54.1
VTFeed	76.2	100.0
F.2 Experimental setup
Since some of the attacks take hours to run for a single file, we use smaller evaluation sets containing malware subsampled from the test sets in Table 3. The evaluation set we use for Sleipnir2 consists of 500 files, and the one for VTFeed consists of 100 files (matching [17]). We note that our evaluation sets are comparable in size to prior work [18, 20, 51]. For each evaluation set, we report attack success rates against malware detectors trained on the same dataset.
Since all attacks employ greedy optimization with randomization, they may fail on some runs, but succeed on others. We therefore repeat each attack 5 times per file and use the best performing attacked file in our evaluation. We define the attack success rate as the proportion of files initially detected as malicious for which at least one of the 5 attack repeats is successful at evading detection. Lower attack success rates correlate with improved robustness against attacks. We permit all attacks to run for up to 200 attack iterations of the internal optimizer. Early stopping is enabled for those attacks that support it (Disp, Slack, GAMMA), which means the attack terminates as soon as the model's prediction flips from malicious to benign.
Where possible, we run direct attacks against RS-Del and compare success rates against NS as a baseline. We also consider transfer attacks from NS to RS-Del as an important variation to the threat model, where an attacker has limited access to the target RS-Del during attack optimization. When running direct attacks against RS-Del, we use a reduced number of Monte Carlo samples  $(n_{\mathrm{pred}} = 100)$  to make the computational cost of the attacks more manageable. For both direct and transfer attacks against RS-Del, we set  $p_{\mathrm{del}} = 97\%$  and perform deletion and certification at the byte-level (BYTE).
F.3 Results
The results for direct attacks against RS-Del are presented in Table 10. For both the Sleipnir2 and VTFeed datasets, we observe that the robustness of RS-Del is superior (or equal) to NS against four of the six attacks. The two cases where RS-Del's robustness drops compared to NS are for the strongest attacks: Slack and GAMMA. The results for transfer attacks from NS to RS-Del are presented in Table 11. Almost all of the attacks transfer poorly to RS-Del. In most cases the attack success rates drop to zero or single digit percentages. We hypothesize that Slack and GAMMA make such drastic changes to the original binary that they can overwhelm the malicious signal—enough to cross the decision boundary—akin to a good word attack [95]. We find that HDOS and HField are ineffective for both RS-Del and the baseline NS. Both attacks change up to 58 bytes in the header, and tend to fall within our certifications.
Table 11: Success rates of attacks transferred from NS to RS-Del.
Setting	Attack	Dataset	Attack success rate (%)
NS	RS-Del
White-box	Disp [17]	Sleipnir2	73.8	0.414
VTFeed	94.1	0.0
Slack [20]	Sleipnir2	57.9	2.90
VTFeed	96.0	1.01
Black-box	HDOs [37]	Sleipnir2	0.0	0.0
VTFeed	0.0	0.0
HField [38]	Sleipnir2	0.607	0.0
VTFeed	0.990	0.0
Disp [17]	Sleipnir2	0.607	0.0
VTFeed	10.9	0.0
GAMMA [31]	Sleipnir2	99.2	99.6
VTFeed	76.2	100.0
G Efficiency of RS-Del
In this appendix, we discuss the training and computational efficiency of RS-Del. We provide comparisons with RS-Abn [24], which serves as a baseline in Appendix E.2 for a more restricted Hamming distance threat model.
Computational efficiency Table 12 reports wall clock times for training and prediction. For training, we measure the time taken to complete 1 epoch of stochastic gradient descent on the Sleipnir2 training set, where inputs are perturbed by the smoothing mechanism. For prediction, we measure the time taken for a single 1MB input file using  $n_{\mathrm{pred}} = 1000$  Monte Carlo samples. We split the prediction time into two components: (1) the time taken to generate perturbed inputs from the smoothing mechanism and (2) the time taken to aggregate predictions for the perturbed inputs using the base MalConv model. All times are recorded on a desktop PC fitted with an AMD Ryzen 7 5800X CPU and an NVIDIA RTX3090 GPU, using our PyTorch implementation of RS-Del and RS-Abn. We execute training and prediction for the base model on the GPU, and the smoothing mechanism on the CPU. We use a single PyTorch process, noting that times may be improved by running the smoothing mechanism in parallel or on the GPU.
We now make some observations about the results. First, we note that training is an order of magnitude faster for RS-Del compared with RS-Abn. We attribute this speed-up to the deletion smoothing mechanism of RS-Del, which drastically reduces the dimensionality of inputs, thereby reducing the time taken to perform forward and backward passes for the base model. On the contrary, the ablation smoothing mechanism of RS-Abn does not alter the dimensionality of inputs, so it does not have a performance advantage in this respect. Second, we observe that the total prediction time for RS-Del is approximately  $150\%$  faster than for RS-Abn. We expect this difference is also due to the effect of dimensionality reduction for the deletion smoothing mechanism.
Training efficiency Training curves for the base MalConv models used in RS-Del and RS-Abn are provided in Figure 7 for the Sleipnir2 dataset. Due to convergence issues for RS-Abn, we adapted training to incorporate gradient clipping when updating the embedding layer. This addresses imbalance in the gradients arising from the dominance of masked (ablated) values in the perturbed inputs. However, even with this fix, we observe slower convergence to a higher loss value for RS-Abn than for RS-Del. Combining the results of Table 12 and Figure 7, we conclude that RS-Abn beats RS-Abn in terms of training efficiency as it requires both fewer epochs to converge and takes less time per epoch.
Table 12: Comparison of runtime efficiency for two models: RS-Del (our method with byte-level deletion) and RS-Abn [24]. The first column of wall times measures the time taken to train each model for one epoch on Sleipnir2. The second and third columns of wall times measure the time taken to make a prediction for a 1MB input file. This is split into two components: the time taken to generate  $n_{\mathrm{pred}} = 1000$  perturbed inputs from the smoothing mechanism (second column) and the time taken to pass the perturbed inputs through the base model (third column).
Model	Parameters	Wall time (s)
Train 1 epoch	Predict
Smoothing	Base model
RS-Del	pdel = 90%	354	10.42	0.070
RS-Abn [24]	pab = 90%	1692	15.29	0.352
RS-Del	pdel = 99%	329	8.79	0.043
RS-Abn [24]	pab = 99%	1788	15.60	0.352
Figure 7: Training curves for RS-Del (our method with byte-level deletion) and RS-Abn [24] for the Sleipnir2 dataset.
H Adapting attacks for smoothed classifiers
In this appendix, we show how to adapt gradient-guided white-box attacks to account for randomized smoothing.
We consider a generic family of white-box attacks that operate on neural network-based classifiers, where the first layer of the network is an embedding layer. Mathematically, we assume the classifier under attack  $f$  can be decomposed as
$$
f = f _ {\text {e m b e d}} \circ f _ {\text {s o f t}} \circ f _ {\text {p r e d}} \tag {23}
$$
-  $f_{\text{embed}}$  is the embedding layer, which maps an input sequence  $\mathbf{x} \in \mathcal{X}$  of length  $n = |\mathbf{x}|$  to an  $n \times d$  array of  $d$ -dimensional embedding vectors;
-  $f_{\mathrm{soft}}$  represents the subsequent layers in the network, which map an  $n \times d$  embedding array to a probability distribution over classes  $\mathcal{V}$ ; and
-  $f_{\text{pred}}$  is an optional final layer, which maps a probability distribution over classes to a prediction (e.g., by taking the arg max or applying a threshold).
The attack may query any of the above components in isolation and compute gradients of  $f_{\mathrm{soft}}$ . For instance, the attacks we consider [20, 37, 17] optimize the input in the embedding space  $e \in \mathbb{R}^{n \times d}$  by computing the gradient  $\frac{\partial f_{\mathrm{soft}}(e)}{\partial e}$ .
While the above setup covers attacks against MalConv, or classifiers with similar architectures, it is not directly compatible with smoothed classifiers. There are two incompatibilities. First, a direct implementation of a smoothed classifier (as described in Section 3.1 and Figure 1) does not decompose like (23). And second, the aggregation of hard predictions from the base classifiers is non-differentiable. We explain how to address these incompatibilities below.
Leveraging commutativity Consider a smoothed classifier composed from a base classifier  $h$  and smoothing mechanism  $\phi$ , and suppose the base classifier decomposes as in (23). Following Section 3.1 and Figure 1, the smoothed classifier's confidence score for class  $y$  can be expressed as
$$
p _ {y} (\boldsymbol {x}) = \operatorname {s m o o t h} _ {N} \left(\boldsymbol {x}; \phi , h _ {\text {e m b e d}} \circ h _ {\text {s o f t}} \circ h _ {\text {p r e d}} \circ \mathbf {1} _ {\square = y}\right) \tag {24}
$$
$$
\operatorname {s m o o t h} _ {N} (\boldsymbol {x}; \phi , f) = \frac {1}{N} \sum_ {i = 1} ^ {N} f \left(\boldsymbol {z} _ {i}\right), \quad \text {w i t h} \boldsymbol {z} _ {i} \sim \phi (\boldsymbol {x})
$$
is the (empirical) smoothing operation. This expression does not immediately decompose like (23), because the embedding layer is applied to the perturbed input  $\phi (\pmb {x})$ , not  $\pmb{x}$ . Fortunately, we can manipulate the expression into the desired form by swapping the order of  $\phi$  and  $h_\mathrm{embed}$ . To do so, we extend the definition of  $\phi$  to operate on an embedding array so that  $h_\mathrm{embed}(\phi (\pmb {x})) = \phi (h_\mathrm{embed}(\pmb {x}))$ , i.e.,  $\phi$  and  $h_\mathrm{embed}$  commute. In particular, this can be done for randomized deletion (RS-Del) by applying the deletion edits to embedding vectors along the first dimension. Then (24) can equivalently be expressed as
$$
p _ {y} (\boldsymbol {x}) = \underbrace {h _ {\text {e m b e d}}} _ {f _ {\text {e m b e d}}} \circ \underbrace {\operatorname {s m o o t h} _ {N} \left(\phi , h _ {\text {s o f t}} \circ h _ {\text {p r e d}} \circ \mathbf {1} _ {\square = y}\right)} _ {f _ {\text {s o f t}}} (\boldsymbol {x}). \tag {25}
$$
Soft aggregation While (25) decomposes as required, the  $f_{\mathrm{soft}}$  component is not differentiable. This is due to the presence of  $h_{\mathrm{pred}}$ , which is an arg max layer. To proceed, we replace the aggregation of predictions by the aggregation of softmax scores as proposed by Salman et al. [94]. This yields a differentiable approximation of the smoothed classifier:
$$
p _ {y} (\boldsymbol {x}) \approx \underbrace {h _ {\text {e m b e d}}} _ {f _ {\text {e m b e d}}} \circ \underbrace {\operatorname {s m o o t h} _ {N} (\phi , h _ {\text {s o f t}} \circ \square_ {y})} _ {f _ {\text {s o f t}}} (\boldsymbol {x}).
$$
Salman et al. note that this approximation performs well, and is empirically more effective than an alternative approach proposed by Cohen et al. [14, Appendix G.3].
I Review of randomized ablation
In this appendix, we review randomized ablation [24], which serves as a baseline for the Hamming distance threat model in our experiments. It is based on randomized smoothing (see Section 3.1) like our method, however the smoothing mechanism and robustness certificate differ. In this review, we formulate randomized ablation for sequence classifiers; we refer readers to Levine and Feizi [24] for a formulation for image classifiers.
I.1 Ablation smoothing mechanism
Randomized ablation employs a smoothing mechanism that replaces a random subset of the input elements with a special null value NA. Levine and Feizi use an encoding for the null value tailored for images, that involves doubling the number of channels. This encoding is not suitable for discrete sequences, so we instead augment the sequence domain  $\Omega$  with a special null value:  $\Omega \to \Omega \cup \{\mathrm{NA}\}$ . The hyperparameter controlling the strength of ablation must also be adapted for our setting. Levine and Feizi use a hyperparameter  $k$  that corresponds to the number of elements retained in the output. This is ineffective for inputs that vary in length, so we scale  $k$  in proportion with the input length. Specifically, we introduce an alternative hyperparameter  $p_{\mathrm{ab}} \in (0,1)$  that represents the fraction of ablated elements and set  $k(|x|) = \lceil (1 - p_{\mathrm{ab}})|x| \rceil$ .
Mathematically, the ablation mechanism has the following distribution when applied to an input sequence  $\pmb{x}$ :
$$
\Pr \left[ \phi (\boldsymbol {x}) = \boldsymbol {z} \right] = \sum_ {\epsilon \in \mathcal {E} _ {k (| \boldsymbol {x} |)} (\boldsymbol {x})} \frac {1}{\binom {| \boldsymbol {x} |}{k (| \boldsymbol {x} |)}} \mathbf {1} _ {\text {a b l a t e} (\boldsymbol {x}, \epsilon) = \boldsymbol {z}},
$$
where  $\mathcal{E}_k = \{\epsilon \in \mathcal{E}(\pmb{x}) : |\epsilon| = k\}$  consists of all sets of element indices of size  $k$  and
$$
z _ {i} = \operatorname {a b l a t e} (\boldsymbol {x}, \epsilon) _ {i} = \left\{ \begin{array}{l l} x _ {i}, & \text {i f i \in \epsilon “ r e t a i n e d ”}, \\ \mathsf {N A}, & \text {i f i \not \in \epsilon “ a b l a t e d ”}. \end{array} \right.
$$
Remark 10. Hyperparameter  $p_{\mathrm{ab}}$  has a similar interpretation as  $p_{\mathrm{del}}$  for randomized deletion, in that both hyperparameters control the proportion of sequence elements hidden (by ablation or deletion) from the base classifier.
I.2 Hamming distance robustness certificate
Randomized ablation provides a Hamming distance  $(\ell_0)$  certificate that guarantees robustness under a bounded number of arbitrary substitutions. Levine and Feizi provide two certificates: one that makes use of the confidence score for the predicted class, and another that makes use of the top two confidence scores. We present the first certificate here, since it matches the certificate we consider in Section 4 and it is the simplest choice for binary classifiers (the focus of our experiments).
To facilitate comparison with randomized deletion, we reuse notation and definitions from Section 4. Recall from (15) that  $\tilde{\rho}(\pmb{x};\mu_y)$  is a lower bound on the smoothed classifier's confidence score  $p_y(\bar{\pmb{x}};h)$  that holds for any input  $\bar{\pmb{x}}$  in the edit distance ball of radius  $r$  centered on  $\pmb{x}$  and any base classifier  $h$  such that  $p_y(\pmb{x};h) = \mu_y$ . For randomized ablation, we can replace the edit distance ball with a Hamming distance ball and obtain the following result (see [24] for the derivation):
$$
\tilde {\rho} (\boldsymbol {x}; \mu_ {y}) = \mu_ {y} - 1 + \frac {\binom {| \boldsymbol {x} | - r} {k (| \boldsymbol {x} |)}}{\binom {| \boldsymbol {x} |} {k (| \boldsymbol {x} |)}}.
$$
Recall from Proposition 3, that the smoothed classifier is certifiably robust if  $\tilde{\rho}(\pmb{x};\mu_y) \geq \nu_y(\pmb{\eta})$  where  $\pmb{\eta}$  denotes the smoothed classifier's tunable decision thresholds. Hence the certified radius  $r^{\star}$  is the maximum value of  $r \in \{0,1,2,\ldots\}$  that satisfies
$$
\mu_ {y} - 1 + \frac {\binom {| \boldsymbol {x} | - r} {k (| \boldsymbol {x} |)}}{\binom {| \boldsymbol {x} |} {k (| \boldsymbol {x} |)}} - \nu_ {y} (\boldsymbol {\eta}) \geq 0.
$$
This maximization problem does not have an analytic solution, however it can be solved efficiently using binary search since the LHS of the above inequality is a non-increasing function of  $r$ . In practice, the exact confidence score  $\mu_y$  can be replaced with a  $1 - \alpha$  lower confidence bound  $\underline{\mu}_y$  to yield a probabilistic certificate that holds with probability  $1 - \alpha$  (see procedure in Figure 1 and Corollary 8).
Since randomized ablation and randomized deletion both admit Hamming distance certificates, it is interesting to compare them. The following result, first proved by Scholten et al. [96, Proposition 5] for a related mechanism, shows that randomized deletion admits a tighter certificate.
Proposition 11. Consider a randomized deletion classifier (RS-Del) with deletion probability  $p_{\mathrm{del}} = p$  and a randomized ablation classifier (RS-Abn) with ablation fraction  $p_{\mathrm{ab}} = p$ . Suppose both classifiers make the same prediction  $y$  with the same confidence score  $\mu_y$  for input  $x$  and that we are interested in issuing a Hamming distance certificate of radius  $r$ . Then the lower bound on the confidence score over the certified region is tighter for RS-Del than for RS-Abn.
Proof. From Theorem 7 we have