SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\rho (\boldsymbol {x}, \mu_ {y}) := \min  _ {\bar {\boldsymbol {x}} \in \mathcal {N} _ {r} (\boldsymbol {x})} \min  _ {h \in \mathcal {F} (\boldsymbol {x}, \mu_ {y})} p _ {y} (\bar {\boldsymbol {x}}; h) \tag {9}
$$
is a tight lower bound on the confidence for class  $y$ , and we define the threshold
$$
\nu_ {y} (\boldsymbol {\eta}) = \left\{ \begin{array}{l l} \frac {1}{2} + \eta_ {y} - \min  _ {y ^ {\prime} \neq y} \eta_ {y ^ {\prime}}, & \eta_ {y} \geq \min  _ {y ^ {\prime} \neq y} \eta_ {y ^ {\prime}} a n d | \mathcal {Y} | > 2, \\ 1 + \eta_ {y} - \min  _ {y ^ {\prime} \neq y} \eta_ {y ^ {\prime}}, & \eta_ {y} <   \min  _ {y ^ {\prime} \neq y} \eta_ {y ^ {\prime}} a n d | \mathcal {Y} | > 2, \\ \frac {1 + \eta_ {y} - \min  _ {y ^ {\prime} \neq y} \eta_ {y ^ {\prime}}}{2}, & | \mathcal {Y} | = 2. \end{array} \right. \tag {10}
$$
The standard approach for evaluating  $\rho(\pmb{x}, \mu_y)$  is via the Neyman-Pearson lemma [42, 14], however this seems insurmountable in our setting due to the challenging geometry of the edit distance neighborhood. We therefore proceed by deriving a loose lower bound on the confidence  $\tilde{\rho}(\pmb{x}, \mu_y) \leq \rho(\pmb{x}, \mu_y)$ , noting that the robustness guarantee still holds so long as  $\tilde{\rho}(\pmb{x}, \mu_y) > \nu_y(\pmb{\eta})$ . The derivation proceeds in two steps. In the first step, covered in Section 4.2, we derive a lower bound for the inner minimization in (9), which we denote by  $\tilde{\rho}(\bar{\pmb{x}}, \pmb{x}, \mu_y)$ . Then in the second step, covered in Section 4.3, we complete the derivation by minimizing  $\tilde{\rho}(\bar{\pmb{x}}, \pmb{x}, \mu_y)$  over the edit distance neighborhood. Our results are summarized in Table 1, where we provide certificates under various constraints on the edit ops.
4.2 Minimizing over feasible base classifiers
In this section, we derive a loose lower bound on the classifier confidence with respect to feasible base classifiers
$$
\tilde {\rho} (\bar {x}, x, \mu_ {y}) \leq \rho (\bar {x}, x, \mu_ {y}) = \min  _ {h \in \mathcal {F} (x, \mu_ {y})} p _ {y} (\bar {x}; h). \tag {11}
$$
To begin, we write  $p_y(\bar{x}; h)$  as a sum over the edit space by combining (3) and (6):
$$
p _ {y} (\bar {\boldsymbol {x}}; h) = \sum_ {\bar {\epsilon} \in \mathcal {E} (\bar {\boldsymbol {x}})} s (\bar {\epsilon}, \bar {\boldsymbol {x}}; h) \quad \text {w h e r e} \quad s (\bar {\epsilon}, \bar {\boldsymbol {x}}; h) = \Pr \left[ G (\bar {\boldsymbol {x}}) = \bar {\epsilon} \right] \mathbf {1} _ {h \left(\operatorname {a p p l y} (\bar {\boldsymbol {x}}, \bar {\epsilon})\right) = y}. \tag {12}
$$
We would like to rewrite this sum in terms of the known confidence score at  $\pmb{x}$ ,  $\mu_y = p_y(\pmb{x}; h) = \sum_{\epsilon \in \mathcal{E}(\pmb{x})} s(\epsilon, \pmb{x}; h)$ . To do so, we identify pairs of edits  $\bar{\epsilon}$  to  $\bar{\pmb{x}}$  and  $\epsilon$  to  $\pmb{x}$  for which the corresponding terms  $s(\bar{\epsilon}, \bar{\pmb{x}}; h)$  and  $s(\epsilon, \pmb{x}; h)$  are proportional.
Lemma 4 (Equivalent edits). Let  $z^{\star}$  be a longest common subsequence (LCS) [43] of  $\bar{x}$  and  $x$ , and let  $\bar{\epsilon}^{\star} \in \mathcal{E}(\bar{x})$  and  $\epsilon^{\star} \in \mathcal{E}(x)$  be any edits such that apply  $(\bar{x},\bar{\epsilon}^{\star}) = \mathrm{apply}(x,\epsilon^{\star}) = z^{\star}$ . Then there exists a bijection  $m:2^{\bar{\epsilon}^{\star}}\to 2^{\epsilon^{\star}}$  such that apply  $(\bar{x},\bar{\epsilon}) = \mathrm{apply}(x,\epsilon)$  for any  $\bar{\epsilon}\subseteq \bar{\epsilon}^{\star}$  and  $\epsilon = m(\bar{\epsilon})$ . Furthermore, we have  $s(\bar{\epsilon},\bar{x};h) = p_{\mathrm{del}}^{|\bar{x}| - |\pmb {x}|}s(\epsilon ,\pmb {x};h)$ .
Applying this proportionality result to all pairs of edits  $\bar{\epsilon},\epsilon$  related under the bijection  $m$  yields:
$$
\sum_ {\bar {\epsilon} \in 2 ^ {\bar {\epsilon} ^ {*}}} s (\bar {\epsilon}, \bar {\boldsymbol {x}}; h) = p _ {\mathrm {d e l}} ^ {| \bar {\boldsymbol {x}} | - | \boldsymbol {x} |} \sum_ {\epsilon \in 2 ^ {\epsilon^ {*}}} s (\epsilon , \boldsymbol {x}; h).
$$
Thus we can achieve our goal of writing  $p_y(\bar{x}; h)$  in terms of  $\mu_y$ . A rearrangement of terms gives:
$$
p _ {y} (\bar {\boldsymbol {x}}; h) = p _ {\mathrm {d e l}} ^ {| \bar {\boldsymbol {x}} | - | \boldsymbol {x} |} \left(\mu_ {y} - \sum_ {\epsilon \notin 2 ^ {\epsilon^ {*}}} s (\epsilon , \boldsymbol {x}; h)\right) + \sum_ {\bar {\epsilon} \notin 2 ^ {\bar {\epsilon} ^ {*}}} s (\bar {\epsilon}, \bar {\boldsymbol {x}}; h). \tag {13}
$$
This representation is convenient for deriving a lower bound. Specifically, we can drop the sum over  $\bar{\epsilon} \notin 2^{\bar{\epsilon}^*}$  and upper-bound the sum over  $\epsilon \notin 2^{\epsilon^*}$  to obtain a lower bound that is independent of  $h$ .
Theorem 5. For any pair of inputs  $\bar{\pmb{x}},\pmb {x}\in \mathcal{X}$  we have
$$
\rho (\bar {x}, x, \mu_ {y}) \geq \tilde {\rho} (\bar {x}, x, \mu_ {y}) = p _ {\mathrm {d e l}} ^ {| \bar {x} | - | x |} \left(\mu_ {y} - 1 + p _ {\mathrm {d e l}} ^ {\frac {1}{2} (\operatorname {d i s t} _ {\mathrm {L C S}} (\bar {x}, x) + | x | - | \bar {x} |))}\right). \tag {14}
$$
where  $\mathrm{dist}_{\mathrm{LCS}}(\bar{\pmb{x}},\pmb {x})$  is the longest common subsequence (LCS) distance between  $\bar{\pmb{x}}$  and  $\pmb{x}$ .
4.3 Minimizing over the edit distance neighborhood
In this section, we complete the derivation of our robustness certificate by minimizing the lower bound in (14) over the edit distance neighborhood:
$$
\tilde {\rho} (\boldsymbol {x}; \mu_ {y}) = \min  _ {\bar {\boldsymbol {x}} \in \mathcal {N} _ {r} (\boldsymbol {x})} \tilde {\rho} (\bar {\boldsymbol {x}}, \boldsymbol {x}, \mu_ {y}). \tag {15}
$$
We are interested in general edit distance neighborhoods, where the edit ops  $O$  used to define the edit distance may be constrained. For example, if the attacker is capable of performing elementary substitutions and insertions, but not deletions, then  $O = \{\text{sub}, \text{ins}\}$ . As a step towards solving (15), it is therefore useful to express  $\tilde{\rho}(\bar{x}, x, \mu_y)$  in terms of edit op counts, as shown below.
Corollary 6. Suppose there exists a sequence of edits from  $\bar{\pmb{x}}$  to  $\pmb{x}$  that consists of  $n_{\mathrm{sub}}$  substitutions,  $n_{\mathrm{ins}}$  insertions and  $n_{\mathrm{del}}$  deletions s.t.  $n_{\mathrm{sub}} + n_{\mathrm{ins}} + n_{\mathrm{del}} = \mathrm{dist}_O(\bar{\pmb{x}},\pmb {x})$  and  $n_{\mathrm{sub}},n_{\mathrm{ins}},n_{\mathrm{del}}\geq 0$  . Then
$$
\tilde {\rho} (\bar {\boldsymbol {x}}, \boldsymbol {x}, \mu_ {y}) = p _ {\mathrm {d e l}} ^ {n _ {\mathrm {d e l}} - n _ {\mathrm {i n s}}} \left(\mu_ {y} - 1 + p _ {\mathrm {d e l}} ^ {n _ {\mathrm {s u b}} + n _ {\mathrm {i n s}}}\right).
$$
This parameterization of the lower bound enables us to re-express (15) as an optimization problem over edit ops counts:
$$
\tilde {\rho} (\boldsymbol {x}; \mu_ {y}) = \min  _ {n _ {\text {s u b}}, n _ {\text {i n s}}, n _ {\text {d e l}} \in \mathcal {C} _ {r}} p _ {\text {d e l}} ^ {n _ {\text {d e l}} - n _ {\text {i n s}}} \left(\mu_ {y} - 1 + p _ {\text {d e l}} ^ {n _ {\text {s u b}} + n _ {\text {i n s}}}\right), \tag {16}
$$
where  $\mathcal{C}_r$  encodes constraints on the set of counts. If any number of insertions, deletions or substitutions are allowed, then the edit distance is known as the Levenshtein distance and  $\mathcal{C}_r$  consists of sets of counts that sum to  $r$ . We solve the minimization problem for this case below.
Theorem 7 (Levenshtein distance certificate). A lower bound on the classifier's confidence within the Levenshtein distance neighborhood  $\mathcal{N}_r(\pmb{x})$  (with  $O = \{\mathrm{del},\mathrm{ins},\mathrm{sub}\}$ ) is  $\tilde{\rho} (\pmb {x};\mu_y) = \mu_y - 1 + p_{\mathrm{del}}^r$ . It follows that the classifier is certifiably robust for any Levenshtein distance ball with radius  $r$  less than or equal to the certified radius  $r^{\star} = \left\lfloor \log (1 + \nu_y(\pmb {\eta}) - \mu_y)\right\rfloor /\log p_{\mathrm{del}}\rfloor$
It is straightforward to adapt this result to account for constraints on the edit ops  $O$ . Results for all combinations of edit ops are provided in Table 1.
So far in this section we have obtained results that depend on the classifier's confidence  $\mu_y$ , assuming it can be evaluated exactly. However since exact evaluation of  $\mu_y$  is not feasible in general (see Section 3.3), we extend our results to the probabilistic setting, assuming a  $1 - \alpha$  lower confidence bound on  $\mu_y$  is available. This covers the probabilistic certification procedure described in Figure 1.
Corollary 8. Suppose the procedure in Figure 1 returns predicted class  $\hat{y}$  with certified radius  $r^{\star}$ . Then an edit distance robustness certificate of radius  $r \leq r^{\star}$  holds at  $\pmb{x}$  with probability  $1 - \alpha$ .
5 Case study: robust malware detection
We now present a case study on the application of RS-Del to malware detection. We report on certified accuracy for Levenshtein and Hamming distance threat models. We show that by tuning decision thresholds we can increase certified radii for the malicious class while maintaining accuracy. We also evaluate RS-Del on a range of published malware classifier attacks. Due to space constraints, we present the complete study in Appendices C-F, where we report on training curves, the computational cost of training and certification, certified radii normalized by file size, and results where byte edits are chunked by instructions.
2The LCS distance is equivalent to the generalized edit distance with  $O = \{\mathrm{del},\mathrm{ins}\}$
Background Malware (malicious software) detection is a long standing problem in security where machine learning is playing an increasing role [44-47]. Inspired by the success of neural networks in other domains, recent work has sought to design neural network models for static malware detection which operate on raw binary executables, represented as byte sequences [48-50]. While these models have achieved competitive performance, they are vulnerable to adversarial perturbations that allow malware to evade detection [18-20, 37, 51, 31, 52, 17, 39]. Our edit distance threat model reflects these attacks in the malware domain where, even though a variety of perturbations with different semantic effects are possible, any perturbation can be represented in terms of elementary byte deletions, insertions and substitutions. For perspective on the threat model, consider YARA [53], a rule-based tool that is widely used for static malware analysis in industry. Running Nextron System's YARA rule set<sup>3</sup> on a sample of binaries from the VTFeed dataset (introduced below), we find  $83\%$  of rule matches are triggered by fewer than 128 bytes. This implies most rules can be evaded by editing fewer than 128 bytes—a regime that is covered by our certificates in some instances (see Table 2). Further background and motivation for the threat model is provided in Appendix C, along with a reduction of static malware detection to sequence classification.
Experimental setup We use two Windows malware datasets: Sleipnir2 which is compiled from public sources following Al-Dujaili et al. [54] and VTFeed which is collected from VirusTotal [17]. We consider three malware detection models: a model smoothed with our randomized deletion mechanism (RS-Del), a model smoothed with the randomized ablation mechanism proposed by Levine and Feizi [24] (RS-Abn), and a non-smoothed baseline (NS). All of the models are based on a popular CNN architecture called MalConv [48], and are evaluated on a held-out test set. We emphasize that RS-Del is the only model that provides edit distance certificates (general  $O$ ), while RS-Abn provides a Hamming distance certificate ( $O = \{\mathrm{sub}\}$ ). We review RS-Abn in Appendix I, where we describe modifications required for discrete sequence classification, and provide an analysis comparing the Hamming distance certificates of RS-Abn and RS-Del. Details about the datasets, models, training procedure, calibration, parameter settings and hardware are provided in Appendix D.
Accuracy/robustness tradeoffs Our first set of experiments investigate tradeoffs between malware detection accuracy and robustness guarantees as parameters associated with the smoothed models are varied. Table 2 reports clean accuracy, median certified radius (CR) and median certified radius normalized by file size (NCR) on the test set for RS-Del as a function of  $p_{\mathrm{del}}$ . A reasonable tradeoff is observed at  $p_{\mathrm{del}} = 99.5\%$  for Sleipnir2, where a median certified radius of 137 bytes is attained with only a  $2 - 3\%$  drop in clean accuracy from the NS baseline. The corresponding median NCR is  $0.06\%$  and varies in the range  $0 - 9\%$  across the test set. We also vary the decision threshold  $\eta$  at  $p_{\mathrm{del}} = 99.5\%$  for Sleipnir2, and obtain asymmetrical robustness guarantees with a median certified radius up to 582 bytes for the malicious class, while maintaining the same accuracies (see Table 8 in Appendix E.1).
Since there are no baseline methods that support edit distance certificates, we compare with RS-Abn, which produces a limited Hamming distance certificate. Figure 2 plots the certified accuracy curves for RS-Del and RS-Abn for different values of the associated smoothing parameters  $p_{\mathrm{del}}$  and  $p_{\mathrm{ab}}$ . The certified accuracy is the fraction of instances in the test set for which the model's prediction is correct and certifiably robust at radius  $r$ , and is therefore sensitive to both robustness and accuracy. We find that RS-Del outperforms RS-Abn in terms of certified accuracy at all radii  $r$  when  $p_{\mathrm{del}} = p_{\mathrm{ab}}$ , while covering a larger set of inputs (since RS-Del's edit distance ball includes RS-Abn's Hamming ball). Further results and interpretation for these experiments are provided in Appendix E.
Empirical robustness to attacks Our edit distance certificates are conservative and may underestimate robustness to adversaries with additional constraints (e.g., maintaining executability, preserving a malicious payload, etc.). To provide a more complete picture of robustness, we subject RS-Del and NS to six recently published attacks [20, 37, 17, 38, 31] covering white-box and black-box settings. We adapt gradient-based white-box attacks [20, 17] for randomized smoothing in Appendix H. We do not constrain the number of edits each attack can make, which yields adversarial examples well outside the edit distance we can certify for four of six attacks (see Table 9 of Appendix F). We measure robustness in terms of the attack success rate, defined as the fraction of instances in the evaluation set for which the attack flips the prediction from malicious to benign on at least one repeat (we repeat each attack five times). For both Sleipnir2 and VTFeed, we observe that RS-Del achieves
<sup>3</sup>https://valhalla.nextron-systems.com/
Model	pdel	Clean Accuracy	Median CR	Median NCR %
Sleipnir2 dataset
NS	—	98.9%	—	—
RS-Del	90%	97.1%	6	0.0023
95%	97.8%	13	0.0052
97%	97.4%	22	0.0093
99%	98.1%	68	0.0262
99.5%	96.5%	137	0.0555
99.9%	83.7%	688	0.2269
VTFeed dataset
NS	—	98.9%	—	—
RS-Del	97%	92.1%	22	0.0045
99%	86.9%	68	0.0122
Table 2: Clean accuracy and robustness metrics for RS-Del as a function of dataset and deletion probability  $p_{\mathrm{del}}$ . All metrics are computed on the test set. "Median CR" is the median certified Levenshtein distance radius in bytes and "median NCR %" is the median certified Levenshtein distance radius normalized as a percentage of the file size. A good tradeoff is achieved when  $p_{\mathrm{del}} = 99.5\%$  (in bold).
Figure 2: Certified accuracy for RS-Del and RS-Abn [24] as a function of certificate radius (horizontal axis) and strength of the smoothing parameter (line style). Results are plotted for the Sleipnir2 test set. While RS-Del provides a Levenshtein distance certificate (with  $O = \{\mathrm{del},\mathrm{ins},\mathrm{sub}\}$ ), RS-Abn provides a more limited Hamming distance certificate ( $O = \{\mathrm{sub}\}$ ). The non-smoothed, non-certified model (NS) achieves a clean accuracy of  $98\%$  in this setting.
the lowest attack success rate (best robustness) for four of six attacks. In particular, we observe a 20 percentage point decrease in the success rate of Disp—an attack with no known defense [17]. We refer the reader to Appendix F for details of this experiment, including setup, results and discussion.
There is a rich body of research on certifications under  $\ell_p$ -norm-bounded threat models [5, 7, 6, 8-11, 13, 14, 12, 55, 25, 56]. While a useful abstraction for computer vision, such certifications are inadequate for many problems including perturbations to executable files considered in this work. Even in computer vision,  $\ell_p$ -norm bounded defenses can be circumvented by image translation, rotation, blur, and other human-imperceptible transformations that induce extremely large  $\ell_p$  distances. One solution is to re-parametrize the norm-bounded distance in terms of image transformation parameters [57-59]. NLP faces a different issue: while the  $\ell_0$  threat model covers adversarial word substitution [28, 60], it is too broad and covers many natural (non-adversarial) examples as well. For example, "He loves cat" and "He hates cat" are 1 word in  $\ell_0$  distance from "He likes cat", but are semantically different. A radius 1 certificate will force a wrong prediction for at least one neighbor. To address this, Jia et al. [26] and Ye et al. [27] constrain the threat model to synonyms only.
In this paper we go beyond the  $\ell_0$  word substitution threat model of previous work [23-25, 56], as consideration of insertions and deletions is necessary in domains such as malware analysis. Such edits are not captured by the  $\ell_0$  threat model: there is no fixed input size, and even when edits are size-preserving, a few edits may lead to large  $\ell_0$  distances. Arguably, our edit distance threat model for sequences and RS-Del mechanism are of independent interest to natural language also.
Certification has been studied for variations of edit distance defined for sets and graphs. Liu et al. [33] apply randomized smoothing using a subsampling mechanism to certify point cloud classifiers against edits that include insertion, deletion and substitution of points. Since a point cloud is an unordered set, the edit distance takes a simpler form than for sequences—it can be expressed in terms of set cardinalities rather than as a cost minimization problem over edit paths. This simplifies the analysis, allowing Liu et al. to obtain a tight certificate via the Neyman-Pearson lemma, which is not feasible
for sequences. In parallel work, Schuchardt et al. [61] consider edit distance certification for graph classification, as an application of a broader certification framework for group equivariant tasks. They apply sparsity-aware smoothing [62] to an isomorphism equivariant base classifier, to yield a smoothed classifier that is certifiably robust under insertions/deletions of edges and node attributes.
Numerous empirical defense methods have been proposed to improve robustness of machine learning classifiers in security systems [63, 64, 29, 65, 22]. Incer Romeo et al. [34] and Chen et al. [65] develop classifiers that are verifiably robust if their manually crafted features conform to particular properties (e.g., monotonicity, stability). These approaches permit a combination of (potentially vulnerable) learned behavior with domain knowledge, and thereby aim to mitigate adversarial examples. Chen et al. [22] seek guarantees against subtree insertion and deletion for PDF malware classification. Using convex over-approximation [5, 66] previously applied to computer vision, they certify fixed-input dimension classifiers popular in PDF malware analysis. Concurrent to our work, Saha et al. [29] propose to certify classifiers against patch-based attacks by aggregating predictions of fixed-sized chunks of input binaries. The patch attack threat model, however, is not widely assumed in the evasion literature for malware detectors and can be readily broken by many published attacks [17, 20]. Moreover, their de-randomized smoothing design assumes a fixed-width input (via padding and/or trimming) and reduces patch-based attacks to gradient-based  $\ell_p$  attacks. While tight analysis exists for arbitrary randomized smoothing mechanisms [23], they are computationally infeasible with the edit distance threat model. Overall, we are the first to explore certified adversarial defenses that apply to sequence classifiers under the edit distance threat model.
In this paper, we study certified robustness of discrete sequence classifiers. We identify critical limitations of the  $\ell_p$ -norm bounded threat model in sequence classification and propose edit distance-based robustness, covering substitution, deletion and insertion perturbations. We then propose a novel deletion smoothing mechanism called RS-Del that is equipped with certified guarantees under several constraints on the edit operations. We present a case study of RS-Del and its certifications applied to malware analysis. We consider two malware datasets using a recent static deep malware detector, MalConv [48]. We find that RS-Del can certify radii as large as 128 bytes (in Levenshtein distance) without significant loss in detection accuracy. A certificate of this size covers in excess of  $10^{606}$  files in the proximity of a 10KB input file (see Appendix A). Results also demonstrate RS-Del improving robustness against published attacks well beyond the certified radius.
Broader impact and limitations Robustness certification seeks to quantify the risk of adversarial examples while randomized smoothing both enables certification and acts to mitigate the impact of attacks. Randomized smoothing can degrade (benign) accuracy of undefended models as demonstrated in our results at higher smoothing levels. While we have strived to select high quality datasets for our case study, we note that accuracy-robustness tradeoffs may vary for different datasets and/or model architectures. Our approach is scalable relative to alternative certification strategies, however it does incur computational overheads. Finally, it is known that randomized smoothing can have disparate impacts on class-wise accuracy [67].
This work was supported by the Department of Industry, Science, and Resources, Australia under AUSMURI CATCH, and the U.S. Army Research Office under MURI Grant W911NF2110317.
[1] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, ICLR, 2014.
[2] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations, ICLR, 2015.