Title: RS-Del: Edit Distance Robustness Certificates for Sequence Classifiers via Randomized Deletion
Venue: NeurIPS 2023
Accepted: yes
Keywords: certified robustness

randomized smoothing

malware detection

sequence classification

edit distance
Abstract: Randomized smoothing is a leading approach for constructing classifiers that are certifiably robust against adversarial examples. Existing work on randomized smoothing has focused on classifiers with continuous inputs, such as images, where $\ell_p$-norm bounded adversaries are commonly studied. However, there has been limited work for classifiers with discrete or variable-size inputs, such as for source code, which require different threat models and smoothing mechanisms. In this work, we adapt randomized smoothing for discrete sequence classifiers to provide certified robustness against edit distance-bounded adversaries. Our proposed smoothing mechanism randomized deletion (RS-Del) applies random deletion edits, which are (perhaps surprisingly) sufficient to confer robustness against adversarial deletion, insertion and substitution edits. Our proof of certification deviates from the established Neyman-Pearson approach, which is intractable in our setting, and is instead organized around longest common subsequences. We present a case study on malware detection—a binary classification problem on byte sequences where classifier evasion is a well-established threat model. When applied to the popular MalConv malware detection model, our smoothing mechanism RS-Del achieves a certified accuracy of 91% at an edit distance radius of 128 bytes.

RS-Del: Edit Distance Robustness Certificates for Sequence Classifiers via Randomized Deletion

Abstract

Randomized smoothing is a leading approach for constructing classifiers that are certifiably robust against adversarial examples. Existing work on randomized smoothing has focused on classifiers with continuous inputs, such as images, where ℓ p -norm bounded adversaries are commonly studied. However, there has been limited work for classifiers with discrete or variable-size inputs, such as for source code, which require different threat models and smoothing mechanisms. In this work, we adapt randomized smoothing for discrete sequence classifiers to provide certified robustness against edit distance-bounded adversaries. Our proposed smoothing mechanism randomized deletion ( RS -Del ) applies random deletion edits, which are (perhaps surprisingly) sufficient to confer robustness against adversarial deletion, insertion and substitution edits. Our proof of certification deviates from the established Neyman-Pearson approach, which is intractable in our setting, and is instead organized around longest common subsequences. We present a case study on malware detection-a binary classification problem on byte sequences where classifier evasion is a well-established threat model. When applied to the popular MalConv malware detection model, our smoothing mechanism RS -Del achieves a certified accuracy of 91% at an edit distance radius of 128 bytes.

1 Introduction

Neural networks have achieved exceptional performance for classification tasks in unstructured domains, such as computer vision and natural language. However, they are susceptible to adversarial examples-inconspicuous perturbations that cause inputs to be misclassified [1, 2]. While a multitude of defenses have been proposed against adversarial examples, they have historically been broken by stronger attacks. For instance, six of nine defense papers accepted for presentation at ICLR2018 were defeated months before the conference took place [3]; another tranche of thirteen defenses were circumvented shortly after [4]. This arms race between attackers and defenders has led the community to investigate certified robustness, which aims to guarantee that a classifier's prediction cannot be changed under a specified set of adversarial perturbations [5, 6].

Most prior work on certified robustness has focused on classifiers with continuous fixed-dimensional inputs, under the assumption of ℓ p -norm bounded perturbations. A variety of certification methods have been proposed for this setting, including deterministic methods which bound a network's output by convex relaxation [5, 7, 6, 8] or composing layerwise bounds [9, 8, 10-12], and randomized smoothing which obtains high probability bounds for a base classifier smoothed under random noise [13-15]. Randomized smoothing has achieved state-of-the-art certified robustness against ℓ 2 -norm bounded perturbations on ImageNet, correctly classifying 71% of a test set under perturbations with ℓ 2 -norm up to 127 / 255 (half maximum pixel intensity) [16]. However, real-world attacks go beyond continuous modalities: a range of attacks have been demonstrated against models with discrete inputs, such as binary executables [17-20], source code [21], and PDF files [22].

Unfortunately, there has been limited investigation of certified robustness for discrete modalities, where prior work has focused on perturbations with bounded Hamming ( ℓ 0 ) distance [23-25], in some cases under additional constraints [26-30]. While this work covers attackers that overwrite content, it does not cover attackers that insert/delete content, which is important for instance in the malware domain [17, 31]. One exception is Zhang et al.'s method for certifying robustness under string transformation rules that may include insertion and deletion, however their approach is only feasible for small rule sets and is limited to LSTMs [32]. Liu et al. [33] also study certification for a threat model that includes insertion/deletion in the context of point cloud classification.

In this paper we develop a comprehensive treatment of edit distance certifications for sequence classifiers. We consider input sequences of varying length on finite domains. We cover threat models where an adversary can arbitrarily perturb sequences by substitutions, insertions, and deletions or any subset of these operations. Moreover, our framework encompasses adversaries that apply edits to blocks of tokens at a time. Such threat models are motivated in malware analysis, where attackers are likely to edit semantic objects in an executable (e.g., whole instructions), rather than at the byte-level. We introduce tunable decision thresholds, to adjust decision boundaries while still forming sound certifications. This permits trading-off misclassification rates and certification radii between classes, and is useful in settings where adversaries have a strong incentive to misclassify malicious instances as benign [34, 35].

We accomplish our certifications using randomized smoothing [13, 14], which we instantiate with a general-purpose deletion smoothing mechanism called RS -Del . Perhaps surprisingly, RS -Del does not need to sample all edit operations covered by our certifications. By smoothing using deletions only, we simplify our certification analysis and achieve the added benefit of improved efficiency by querying the base classifier with shorter sequences. RS -Del is compatible with arbitrary base classifiers, requiring only oracle query access such as via an inference API. To prove our robustness certificates, we have to deviate from the standard Neyman-Pearson approach to randomized smoothing, as it is intractable in our setting. Instead, we organize our proof around a representative longest common subsequence (LCS): the LCS serves as a reference point in the smoothed space between an input instance and a neighboring instance, allowing us to bound the confidence of the smoothed model between the two.

Finally, we present a comprehensive evaluation of RS -Del in the malware analysis setting 1 . We investigate tradeoffs between accuracy and the size of our certificates for varying levels of deletion, and observe that RS -Del can achieve a certified accuracy of 91% at an edit distance radius of 128 bytes using on the order of 10 3 model queries. By comparison, a brute force certification would require in excess of 10 308 model queries to certify the same radius. We also demonstrate asymmetric certificates (favouring the malicious class) and certificates covering edits at the machine instruction level by leveraging chunking and information from a disassembler. Finally, we assess the empirical robustness of RS -Del to several attacks from the literature, where we observe a reduction in attack success rates.

2 Preliminaries

Sequence classification Let X = Ω ⋆ represent the space of finite-length sequences (including the empty sequence) whose elements are drawn from a finite set Ω . For a sequence x ∈ X , we denote its length by | x | and the element at position i by x i where i runs from 1 to | x | . We consider classifiers that map sequences in X to K classes in Y = { 0 , . . . , K -1 } . For example, in our case study on

malware detection, we take X to be the space of byte sequences (binaries), and Ω to be the set of bytes and Y to be { 0 , 1 } where 0 and 1 denote benign and malicious binaries respectively.

Robustness certification Given a classifier f : X → Y , an input x ∈ X and a neighborhood N ( x ) ⊂ X around x , a robustness certificate is a guarantee that the classifier's prediction is constant in the neighborhood, i.e.,

When the neighborhood corresponds to a ball of radius r centered on x , we make the dependence on r explicit by writing N r ( x ) . We adopt the paradigm of conservative , probabilistic certification, which is a natural fit for randomized smoothing [14]. Under this paradigm, a certifier may either assert that (1) holds with high probability, or decline to assert whether (1) holds or not.

Edit distance robustness Most existing work on robustness certification focuses on classifiers that operate on fixed-dimensional inputs in R d , where the neighborhood of certification is an ℓ p -ball [6, 5, 13-15]. However, ℓ p robustness is not well motivated for sequence classifiers: ℓ p neighborhoods are limited to constant-length sequences, and the norm is ill-defined for sequences with non-numeric elements. For example, an ℓ p neighborhood around a byte sequence like x = (78 , 7A , 2D , 00) must include sequences additively perturbed by real-valued sequences like δ = ( -0 . 09 , 0 . 07 , 0 . 01 , 0 . 1) , which clearly results in a type mismatch. Even if one focuses on robustness for length-preserving sequence perturbations, the ℓ p neighborhood is a poor choice because it excludes sequences that are even slightly misaligned. Motivated by these shortcomings, we consider edit distance robustness.

Edit distance is a natural measure for comparing sequences. Given a set of elementary edit operations (ops) O , the edit distance dist O ( x , x ′ ) is the minimum number of ops required to transform sequence x into x ′ . We consider three ops: delete a single element ( del ), insert a single element ( ins ), and substitute one element with another ( sub ). For generality, we allow O to be any combination of these ops. When O = { del , ins , sub } the edit distance is known as Levenshtein distance [36]. A primary goal of this paper is to produce edit distance robustness certificates, where the neighborhood of certification is an edit distance ball N r ( x ) = { x ′ ∈ X : dist O ( x ′ , x ) ≤ r } .

Threat model We consider an adversary that has full knowledge of our base and smoothed models, source of randomness, certification scheme, and possesses unbounded computation. The attacker makes edits from O up to some budget, in order to misclassify a target x . In the context of our experimental case study on malware detection, edit distance is a reasonable proxy for the cost of running evasion attacks that iteratively apply localized functionality-preserving edits (e.g., [19, 37, 38, 17, 39]). Since the edit distance scales roughly linearly with the number of attack iterations, the adversary has an incentive to minimize edit distance for these attacks.

3 RS-Del: Randomized deletion smoothing

In this section, we propose a method for constructing sequence classifiers that are certifiably robust under bounded edit distance perturbations. Our method RS -Del extends randomized smoothing with a novel deletion mechanism and tunable decision thresholds. We review randomized smoothing in Section 3.1 and describe our deletion mechanism in Section 3.2, along with its practical aspects in Section 3.3. We summarize the certified robustness guarantees of our method in Table 1 and defer their derivation to Section 4.

3.1 Randomized smoothing

Let f b : X → Y be a base classifier and ϕ : X → D ( X ) be a smoothing mechanism that maps inputs to a distributions over (perturbed) inputs. Randomized smoothing composes f b and ϕ to construct a new smoothed classifier f : X → Y . For any input x ∈ X , the smoothed classifier's prediction is

where η = { η y } y ∈Y is a set of real-valued tunable decision thresholds and

is the probability that the base classifier f b predicts class y for a perturbed input drawn from ϕ ( x ) . We omit the dependence of p y on f b and ϕ where it is clear from context.

The viability of randomized smoothing as a method for achieving certified robustness is strongly dependent on the smoothing mechanism. Ideally, the mechanism should be chosen to yield a smoothed classifier with improved robustness under the chosen threat model, while minimizing any drop in accuracy compared to the base classifier. The mechanism should also be amenable to analysis, so that a tractable robustness certificate can be derived.

Remark 1 . Previous definitions of randomized smoothing (e.g., [13, 14]) do not incorporate decision thresholds, and effectively assume η = 0 . We introduce decision thresholds as a way to trade off error rates and robustness between classes. This is useful when there is asymmetry in misclassification costs across classes. For instance, in our case study on malware detection, robustness of benign examples is less important because adveraries have limited incentive to trigger misclassification of benign examples [34, 35, 40]. We note that the base classifier may also be equipped with decision thresholds, which provide another degree of freedom to trade off error rates between classes.

3.2 Randomized deletion mechanism

We propose a smoothing mechanism that achieves certified edit distance robustness. Our smoothing mechanism perturbs a sequence by deleting elements at random, and is called randomized deletion , or RS -Del for short.

Consider a sequence x ∈ X whose elements are indexed by the set [ x ] = { 1 , . . . , | x |} . We specify the distribution of ϕ ( x ) for RS -Del in two stages. In the first stage, a random edit ϵ is drawn from a distribution G ( x ) over the space of possible edits to x , denoted E ( x ) . Since we only consider deletions for smoothing, any edit can be represented by the set of element indices in [ x ] that remain after deletion. Hence E ( x ) is taken as the powerset of [ x ] . We specify edit distribution G ( x ) so that each element is deleted i.i.d. with probability p del ∈ (0 , 1) :

where 1 A denotes the indicator function, which returns 1 if A evaluates to true and 0 otherwise. In the second stage, the edit ϵ is applied to x to yield the perturbed sequence:

where ϵ ( i ) denotes the i -th smallest index in ϵ . The perturbed sequence z is guaranteed to be a subsequence of x . Putting both stages together, the distribution of ϕ ( x ) is

Remark 2 . It may be surprising that we are proposing a smoothing mechanism for certified edit distance robustness that does not use the full set of edit ops O covered by the threat model. It is a misconception that randomized smoothing requires perfect alignment between the mechanism and the threat model. All that is needed from a robustness perspective, is for the mechanism to return distributions that are statistically close for any pair of inputs that are close in O edit distance; this can be achieved solely with deletion. In fact, perfect alignment is known to be suboptimal for some ℓ p threat models [15]. Our deletion mechanism leads to a tractable robustness certificate covering the full set of edit ops (see Section 4). Moreover while benefiting robustness, our empirical results show that our deletion mechanism has only a minor impact on accuracy (see Section 5). Finally, our deletion mechanism reduces the length of the input, which is beneficial for computational efficiency (see Appendix G). This is not true in general for mechanisms employing insertions/substitutions.

3.3 Practical considerations

We now discuss considerations for implementing and certifying RS -Del in practice. In doing so, we reference theoretical results for certification, which are covered later in Section 4.

Probabilistic certification Randomized smoothing does not generally support exact evaluation of the classifier's confidence µ y := p y ( x ) , which is required for exact prediction and exact evaluation of the certificates we develop in Section 4. While µ y can be evaluated exactly for RS -Del by enumerating over the possible edits E ( x ) , the computation scales exponentially in | x | (see Appendix A). Since this is infeasible for even moderately-sized x , we follow standard practice in randomized smoothing and estimate µ y with a lower confidence bound using Monte Carlo sampling [14]. This procedure is described in pseudocode in Figure 1: lines 1-3 estimate the predicted class y = f ( x ) , lines 4-5 compute a 1 -α lower confidence bound on µ y , and line 6 uses this information and the results in Table 1 to compute a probabilistic certificate that holds with probability 1 -α . If the lower confidence bound on µ y exceeds the corresponding detection threshold η y , the prediction and certificate are returned (line 8), otherwise we abstain due to lack of statistical significance (line 7).

Training While randomized smoothing is compatible with any base classifier, it generally performs poorly for conventionally-trained classifiers [13]. We therefore train base classifiers specifically to be used with RS -Del , by replacing original sequences with perturbed sequences (drawn from ϕ ) at training time. This has been shown to achieve good empirical performance in prior work [14].

Sequence chunking So far, we have described edit distance robustness and RS -Del assuming edits are applied at the level of sequence elements. However in some applications it may be reasonable to assume edits are applied at a coarser level, to contiguous chunks of sequence elements. For example, in malware analysis, one can leverage information from a disassembler to group low-level sequence elements (bytes) into more semantically meaningful chunks, such as machine instructions, addresses and header fields (see Appendix C.3). Our methods are compatible with chunking-we simply reinterpret the sequence as a sequence of chunks, rather than a sequence of lower-level elements. This can yield a tighter robustness certificate, since edits within chunks are excluded.

4 Edit distance robustness certificate

We now derive an edit distance robustness certificate for RS -Del . We present the derivation in three parts: Section 4.1 provides an outline, Section 4.2 derives a lower bound on RS -Del 's confidence score and Section 4.3 uses the bound to complete the derivation. All proofs are presented in Appendix B.

4.1 Derivation outline

Following prior work [14, 13, 41], we derive an edit distance robustness certificate that relies on limited information about RS -Del . We allow the certificate to depend on the input x , the smoothed prediction y = f ( x ) , the confidence score µ y = p y ( x ; f b ) , the decision threshold η y , and the architecture of f , including the deletion smoothing mechanism ϕ , but excluding the architecture of

the base classifier f b . Limiting the dependence in this way improves tractability and ensures that the certificate is applicable for any choice of base classifier f b . Formally, the only information we assume about f b is that it is some classifier in the feasible base classifier set:

Recall that an edit distance robustness certificate at radius r for a classifier f at input x is a guarantee that f ( x ) = f (¯ x ) for any perturbed input ¯ x in the neighborhood N r ( x ) = { ¯ x ∈ X : dist O (¯ x , x ) ≤ r } . We observe that this guarantee holds for RS -Del in the limited information setting iff the η -adjusted confidence for predicted class y exceeds the η -adjusted confidence for any other class for all perturbed inputs ¯ x and feasible base classifiers h :

̸

To avoid dependence on the confidence of the runner-up class, which is inefficient for probabilistic certification, we work with the following more convenient condition.

Proposition 3. A sufficient condition for (8) is ρ ( x , µ y ) ≥ ν y ( η ) where

is a tight lower bound on the confidence for class y , and we define the threshold

̸

̸

̸

̸

The standard approach for evaluating ρ ( x , µ y ) is via the Neyman-Pearson lemma [42, 14], however this seems insurmountable in our setting due to the challenging geometry of the edit distance neighborhood. We therefore proceed by deriving a loose lower bound on the confidence ˜ ρ ( x , µ y ) ≤ ρ ( x , µ y ) , noting that the robustness guarantee still holds so long as ˜ ρ ( x , µ y ) > ν y ( η ) . The derivation proceeds in two steps. In the first step, covered in Section 4.2, we derive a lower bound for the inner minimization in (9), which we denote by ˜ ρ (¯ x , x , µ y ) . Then in the second step, covered in Section 4.3, we complete the derivation by minimizing ˜ ρ (¯ x , x , µ y ) over the edit distance neighborhood. Our results are summarized in Table 1, where we provide certificates under various constraints on the edit ops.

4.2 Minimizing over feasible base classifiers

In this section, we derive a loose lower bound on the classifier confidence with respect to feasible base classifiers

To begin, we write p y (¯ x ; h ) as a sum over the edit space by combining (3) and (6):

We would like to rewrite this sum in terms of the known confidence score at x , µ y = p y ( x ; h ) = ∑ ϵ ∈E ( x ) s ( ϵ, x ; h ) . To do so, we identify pairs of edits ¯ ϵ to ¯ x and ϵ to x for which the corresponding terms s (¯ ϵ, ¯ x ; h ) and s ( ϵ, x ; h ) are proportional.

Lemma 4 (Equivalent edits) . Let z ⋆ be a longest common subsequence (LCS) [43] of ¯ x and x , and let ¯ ϵ ⋆ ∈ E (¯ x ) and ϵ ⋆ ∈ E ( x ) be any edits such that apply(¯ x , ¯ ϵ ⋆ ) = apply( x , ϵ ⋆ ) = z ⋆ . Then there exists a bijection m : 2 ¯ ϵ ⋆ → 2 ϵ ⋆ such that apply(¯ x , ¯ ϵ ) = apply( x , ϵ ) for any ¯ ϵ ⊆ ¯ ϵ ⋆ and ϵ = m (¯ ϵ ) . Furthermore, we have s (¯ ϵ, ¯ x ; h ) = p | ¯ x |-| x | del s ( ϵ, x ; h ) .

Applying this proportionality result to all pairs of edits ¯ ϵ, ϵ related under the bijection m yields:

Thus we can achieve our goal of writing p y (¯ x ; h ) in terms of µ y . A rearrangement of terms gives:

This representation is convenient for deriving a lower bound. Specifically, we can drop the sum over ¯ ϵ / ∈ 2 ¯ ϵ ⋆ and upper-bound the sum over ϵ / ∈ 2 ϵ ⋆ to obtain a lower bound that is independent of h .

Theorem 5. For any pair of inputs ¯ x , x ∈ X we have

where dist LCS (¯ x , x ) is the longest common subsequence (LCS) distance 2 between ¯ x and x .

4.3 Minimizing over the edit distance neighborhood

In this section, we complete the derivation of our robustness certificate by minimizing the lower bound in (14) over the edit distance neighborhood:

We are interested in general edit distance neighborhoods, where the edit ops O used to define the edit distance may be constrained. For example, if the attacker is capable of performing elementary substitutions and insertions, but not deletions, then O = { sub , ins } . As a step towards solving (15), it is therefore useful to express ˜ ρ (¯ x , x , µ y ) in terms of edit op counts, as shown below.

Corollary 6. Suppose there exists a sequence of edits from ¯ x to x that consists of n sub substitutions, n ins insertions and n del deletions s.t. n sub + n ins + n del = dist O (¯ x , x ) and n sub , n ins , n del ≥ 0 . Then

This parameterization of the lower bound enables us to re-express (15) as an optimization problem over edit ops counts:

where C r encodes constraints on the set of counts. If any number of insertions, deletions or substitutions are allowed, then the edit distance is known as the Levenshtein distance and C r consists of sets of counts that sum to r . We solve the minimization problem for this case below.

Theorem 7 (Levenshtein distance certificate) . A lower bound on the classifier's confidence within the Levenshtein distance neighborhood N r ( x ) (with O = { del , ins , sub } ) is ˜ ρ ( x ; µ y ) = µ y -1 + p r del . It follows that the classifier is certifiably robust for any Levenshtein distance ball with radius r less than or equal to the certified radius r ⋆ = ⌊ log(1+ ν y ( η ) -µ y ) / log p del ⌋ .

It is straightforward to adapt this result to account for constraints on the edit ops O . Results for all combinations of edit ops are provided in Table 1.

So far in this section we have obtained results that depend on the classifier's confidence µ y , assuming it can be evaluated exactly. However since exact evaluation of µ y is not feasible in general (see Section 3.3), we extend our results to the probabilistic setting, assuming a 1 -α lower confidence bound on µ y is available. This covers the probabilistic certification procedure described in Figure 1.

Corollary 8. Suppose the procedure in Figure 1 returns predicted class ˆ y with certified radius r ⋆ . Then an edit distance robustness certificate of radius r ≤ r ⋆ holds at x with probability 1 -α .

5 Case study: robust malware detection

We now present a case study on the application of RS -Del to malware detection. We report on certified accuracy for Levenshtein and Hamming distance threat models. We show that by tuning decision thresholds we can increase certified radii for the malicious class while maintaining accuracy. We also evaluate RS -Del on a range of published malware classifier attacks. Due to space constraints, we present the complete study in Appendices C-F, where we report on training curves, the computational cost of training and certification, certified radii normalized by file size, and results where byte edits are chunked by instructions.

Background Malware (malicious software) detection is a long standing problem in security where machine learning is playing an increasing role [44-47]. Inspired by the success of neural networks in other domains, recent work has sought to design neural network models for static malware detection which operate on raw binary executables, represented as byte sequences [48-50]. While these models have achieved competitive performance, they are vulnerable to adversarial perturbations that allow malware to evade detection [18-20, 37, 51, 31, 52, 17, 39]. Our edit distance threat model reflects these attacks in the malware domain where, even though a variety of perturbations with different semantic effects are possible, any perturbation can be represented in terms of elementary byte deletions, insertions and substitutions. For perspective on the threat model, consider YARA [53], a rule-based tool that is widely used for static malware analysis in industry. Running Nextron System's YARA rule set 3 on a sample of binaries from the VTFeed dataset (introduced below), we find 83% of rule matches are triggered by fewer than 128 bytes. This implies most rules can be evaded by editing fewer than 128 bytes-a regime that is covered by our certificates in some instances (see Table 2). Further background and motivation for the threat model is provided in Appendix C, along with a reduction of static malware detection to sequence classification.

Experimental setup We use two Windows malware datasets: Sleipnir2 which is compiled from public sources following Al-Dujaili et al. [54] and VTFeed which is collected from VirusTotal [17]. We consider three malware detection models: a model smoothed with our randomized deletion mechanism ( RS -Del ), a model smoothed with the randomized ablation mechanism proposed by Levine and Feizi [24] ( RS -Abn ), and a non-smoothed baseline ( NS ). All of the models are based on a popular CNN architecture called MalConv [48], and are evaluated on a held-out test set. We emphasize that RS -Del is the only model that provides edit distance certificates (general O ), while RS -Abn provides a Hamming distance certificate ( O = { sub } ). We review RS -Abn in Appendix I, where we describe modifications required for discrete sequence classification, and provide an analysis comparing the Hamming distance certificates of RS -Abn and RS -Del . Details about the datasets, models, training procedure, calibration, parameter settings and hardware are provided in Appendix D.

Accuracy/robustness tradeoffs Our first set of experiments investigate tradeoffs between malware detection accuracy and robustness guarantees as parameters associated with the smoothed models are varied. Table 2 reports clean accuracy, median certified radius (CR) and median certified radius normalized by file size (NCR) on the test set for RS -Del as a function of p del . A reasonable tradeoff is observed at p del = 99 . 5% for Sleipnir2, where a median certified radius of 137 bytes is attained with only a 2-3% drop in clean accuracy from the NS baseline. The corresponding median NCR is 0.06% and varies in the range 0-9% across the test set. We also vary the decision threshold η at p del = 99 . 5% for Sleipnir2, and obtain asymmetrical robustness guarantees with a median certified radius up to 582 bytes for the malicious class, while maintaining the same accuracies (see Table 8 in Appendix E.1).

Since there are no baseline methods that support edit distance certificates, we compare with RS -Abn , which produces a limited Hamming distance certificate. Figure 2 plots the certified accuracy curves for RS -Del and RS -Abn for different values of the associated smoothing parameters p del and p ab . The certified accuracy is the fraction of instances in the test set for which the model's prediction is correct and certifiably robust at radius r , and is therefore sensitive to both robustness and accuracy. We find that RS -Del outperforms RS -Abn in terms of certified accuracy at all radii r when p del = p ab , while covering a larger set of inputs (since RS -Del 's edit distance ball includes RS -Abn 's Hamming ball). Further results and interpretation for these experiments are provided in Appendix E.

Empirical robustness to attacks Our edit distance certificates are conservative and may underestimate robustness to adversaries with additional constraints (e.g., maintaining executability, preserving a malicious payload, etc.). To provide a more complete picture of robustness, we subject RS -Del and NS to six recently published attacks [20, 37, 17, 38, 31] covering white-box and black-box settings. We adapt gradient-based white-box attacks [20, 17] for randomized smoothing in Appendix H. We do not constrain the number of edits each attack can make, which yields adversarial examples well outside the edit distance we can certify for four of six attacks (see Table 9 of Appendix F). We measure robustness in terms of the attack success rate , defined as the fraction of instances in the evaluation set for which the attack flips the prediction from malicious to benign on at least one repeat (we repeat each attack five times). For both Sleipnir2 and VTFeed, we observe that RS -Del achieves

0

50

100

150

Radius,

r

50

60

70

80

90

100

Certified accuracy (

%

)

Model,smoothing parameter

p

del

= 95%

p

del

= 97%

p

del

= 99%

p

del

= 99

.

5%

p

ab

= 95%

p

ab

= 97%

p

ab

= 99%

p

ab

= 99

.

5%

the lowest attack success rate (best robustness) for four of six attacks. In particular, we observe a 20 percentage point decrease in the success rate of Disp -an attack with no known defense [17]. We refer the reader to Appendix F for details of this experiment, including setup, results and discussion.

6 Related work

There is a rich body of research on certifications under ℓ p -norm-bounded threat models [5, 7, 6, 811, 13, 14, 12, 55, 25, 56]. While a useful abstraction for computer vision, such certifications are inadequate for many problems including perturbations to executable files considered in this work. Even in computer vision, ℓ p -norm bounded defenses can be circumvented by image translation, rotation, blur, and other human-imperceptible transformations that induce extremely large ℓ p distances. One solution is to re-parametrize the norm-bounded distance in terms of image transformation parameters [57-59]. NLP faces a different issue: while the ℓ 0 threat model covers adversarial word substitution [28, 60], it is too broad and covers many natural (non-adversarial) examples as well. For example, 'He loves cat' and 'He hates cat' are 1 word in ℓ 0 distance from 'He likes cat', but are semantically different. A radius 1 certificate will force a wrong prediction for at least one neighbor. To address this, Jia et al. [26] and Ye et al. [27] constrain the threat model to synonyms only.

In this paper we go beyond the ℓ 0 word substitution threat model of previous work [23-25, 56], as consideration of insertions and deletions is necessary in domains such as malware analysis. Such edits are not captured by the ℓ 0 threat model: there is no fixed input size, and even when edits are size-preserving, a few edits may lead to large ℓ 0 distances. Arguably, our edit distance threat model for sequences and RS -Del mechanism are of independent interest to natural language also.

Certification has been studied for variations of edit distance defined for sets and graphs. Liu et al. [33] apply randomized smoothing using a subsampling mechanism to certify point cloud classifiers against edits that include insertion, deletion and substitution of points. Since a point cloud is an unordered set, the edit distance takes a simpler form than for sequences-it can be expressed in terms of set cardinalities rather than as a cost minimization problem over edit paths. This simplifies the analysis, allowing Liu et al. to obtain a tight certificate via the Neyman-Pearson lemma, which is not feasible

for sequences. In parallel work, Schuchardt et al. [61] consider edit distance certification for graph classification, as an application of a broader certification framework for group equivariant tasks. They apply sparsity-aware smoothing [62] to an isomorphism equivariant base classifier, to yield a smoothed classifier that is certifiably robust under insertions/deletions of edges and node attributes.

Numerous empirical defense methods have been proposed to improve robustness of machine learning classifiers in security systems [63, 64, 29, 65, 22]. Incer Romeo et al. [34] and Chen et al. [65] develop classifiers that are verifiably robust if their manually crafted features conform to particular properties (e.g., monotonicity, stability). These approaches permit a combination of (potentially vulnerable) learned behavior with domain knowledge, and thereby aim to mitigate adversarial examples. Chen et al. [22] seek guarantees against subtree insertion and deletion for PDF malware classification. Using convex over-approximation [5, 66] previously applied to computer vision, they certify fixedinput dimension classifiers popular in PDF malware analysis. Concurrent to our work, Saha et al. [29] propose to certify classifiers against patch-based attacks by aggregating predictions of fixed-sized chunks of input binaries. The patch attack threat model, however, is not widely assumed in the evasion literature for malware detectors and can be readily broken by many published attacks [17, 20]. Moreover, their de-randomized smoothing design assumes a fixed-width input (via padding and/or trimming) and reduces patch-based attacks to gradient-based ℓ p attacks. While tight analysis exists for arbitrary randomized smoothing mechanisms [23], they are computationally infeasible with the edit distance threat model. Overall, we are the first to explore certified adversarial defenses that apply to sequence classifiers under the edit distance threat model.

7 Conclusion

In this paper, we study certified robustness of discrete sequence classifiers. We identify critical limitations of the ℓ p -norm bounded threat model in sequence classification and propose edit distancebased robustness, covering substitution, deletion and insertion perturbations. We then propose a novel deletion smoothing mechanism called RS -Del that is equipped with certified guarantees under several constraints on the edit operations. We present a case study of RS -Del and its certifications applied to malware analysis. We consider two malware datasets using a recent static deep malware detector, MalConv [48]. We find that RS -Del can certify radii as large as 128 bytes (in Levenshtein distance) without significant loss in detection accuracy. A certificate of this size covers in excess of 10 606 files in the proximity of a 10KB input file (see Appendix A). Results also demonstrate RS -Del improving robustness against published attacks well beyond the certified radius.

Broader impact and limitations Robustness certification seeks to quantify the risk of adversarial examples while randomized smoothing both enables certification and acts to mitigate the impact of attacks. Randomized smoothing can degrade (benign) accuracy of undefended models as demonstrated in our results at higher smoothing levels. While we have strived to select high quality datasets for our case study, we note that accuracy-robustness tradeoffs may vary for different datasets and/or model architectures. Our approach is scalable relative to alternative certification strategies, however it does incur computational overheads. Finally, it is known that randomized smoothing can have disparate impacts on class-wise accuracy [67].

Acknowledgments

This work was supported by the Department of Industry, Science, and Resources, Australia under AUSMURI CATCH, and the U.S. Army Research Office under MURI Grant W911NF2110317.

References

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations , ICLR, 2014.

Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations , ICLR, 2015.

Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the 35th International Conference on Machine Learning , ICML, 2018.

Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In Advances in Neural Information Processing Systems , NeurIPS, 2020.

Eric Wong and Zico Kolter. Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope. In International Conference on Machine Learning , ICML, 2018.

Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified Defenses against Adversarial Examples. In International Conference on Learning Representations , ICLR, 2018.

Krishnamurthy Dvijotham, Sven Gowal, Robert Stanforth, Relja Arandjelovic, Brendan O'Donoghue, Jonathan Uesato, and Pushmeet Kohli. Training verified learners with learned verifiers. arXiv preprint:1805.10265 , 2018.

Lily Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane Boning, and Inderjit Dhillon. Towards fast computation of certified robustness for relu networks. In International Conference on Machine Learning , ICML, 2018.

Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning , ICML, 2018.

Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks. In Advances in Neural Information Processing Systems , NeurIPS, 2018.

Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint:1810.12715 , 2018.

Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards Stable and Efficient Training of Verifiably Robust Neural Networks. In International Conference on Learning Representations , ICLR, 2020.

Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified Robustness to Adversarial Examples with Differential Privacy. In IEEE Symposium on Security and Privacy , S&P, 2019.

Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified Adversarial Robustness via Randomized Smoothing. In International Conference on Machine Learning , ICML, 2019.

Greg Yang, Tony Duan, J. Edward Hu, Hadi Salman, Ilya Razenshteyn, and Jerry Li. Randomized Smoothing of All Shapes and Sizes. In International Conference on Machine Learning , ICML, 2020.

Nicholas Carlini, Florian Tramer, Krishnamurthy Dj Dvijotham, Leslie Rice, Mingjie Sun, and J Zico Kolter. (certified!!) adversarial robustness for free! In International Conference on Learning Representations , ICLR, 2023.

Keane Lucas, Mahmood Sharif, Lujo Bauer, Michael K. Reiter, and Saurabh Shintre. Malware makeover: Breaking ml-based static analysis by modifying executable bytes. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security , AsiaCCS, 2021.

Bojan Kolosnjaji, Ambra Demontis, Battista Biggio, Davide Maiorca, Giorgio Giacinto, Claudia Eckert, and Fabio Roli. Adversarial malware binaries: Evading deep learning for malware detection in executables. In 26th European Signal Processing Conference , EUSIPCO, 2018.

Daniel Park, Haidar Khan, and Bülent Yener. Generation & Evaluation of Adversarial Examples for Malware Obfuscation. In International Conference On Machine Learning And Applications , ICMLA, 2019.

Felix Kreuk, Assi Barak, Shir Aviv-Reuven, Moran Baruch, Benny Pinkas, and Joseph Keshet. Deceiving End-to-End Deep Learning Malware Detectors using Adversarial Examples. arXiv preprint:1802.04528 , 2018.

Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, and Zhi Jin. Generating adversarial examples for holding robustness of source code processing models. In Proceedings of the AAAI Conference on Artificial Intelligence , AAAI, 2020.

Yizheng Chen, Shiqi Wang, Dongdong She, and Suman Jana. On training robust PDF malware classifiers. In 29th USENIX Security Symposium , 2020.

Guang-He Lee, Yang Yuan, Shiyu Chang, and Tommi Jaakkola. Tight certificates of adversarial robustness for randomly smoothed classifiers. In Advances in Neural Information Processing Systems , NeurIPS, 2019.

Alexander Levine and Soheil Feizi. Robustness certificates for sparse adversarial attacks by randomized ablation. In Proceedings of the AAAI Conference on Artificial Intelligence , AAAI, 2020.

Jinyuan Jia, Binghui Wang, Xiaoyu Cao, Hongbin Liu, and Neil Zhenqiang Gong. Almost tight l0-norm certified robustness of top-k predictions against adversarial perturbations. In International Conference on Learning Representations , ICLR, 2022.

Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified Robustness to Adversarial Word Substitutions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , EMNLP-IJCNLP, 2019.

Mao Ye, Chengyue Gong, and Qiang Liu. SAFER: A structure-free approach for certified robustness to adversarial word substitutions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , ACL, 2020.

Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , ACL, 2019.

Shoumik Saha, Wenxiao Wang, Yigitcan Kaya, and Soheil Feizi. Adversarial robustness of learning-based static malware classifiers. arXiv preprint:2303.13372 , 2023.

Han Cheol Moon, Shafiq Joty, Ruochen Zhao, Megh Thakkar, and Chi Xu. Randomized smoothing with masked inference for adversarially robust text classifications. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , ACL, 2023.

Luca Demetrio, Battista Biggio, Giovanni Lagorio, Fabio Roli, and Alessandro Armando. Functionality-preserving black-box optimization of adversarial windows malware. IEEE Transactions on Information Forensics and Security , 16:3469-3478, 2021.

Yuhao Zhang, Aws Albarghouthi, and Loris D'Antoni. Certified robustness to programmable transformations in lstms. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , EMNLP, 2021.

Hongbin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. PointGuard: Provably robust 3D point cloud classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , CVPR, pages 6186-6195, 2021.

Inigo Incer Romeo, Michael Theodorides, Sadia Afroz, and David Wagner. Adversarially Robust Malware Detection Using Monotonic Classification. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics , IWSPA, 2018.

Samuel Pfrommer, Brendon G. Anderson, Julien Piet, and Somayeh Sojoudi. Asymmetric certified robustness via feature-convex neural networks, 2023.

V. I. Levenshtein. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet Physics Doklady , 10:707, February 1966.

Luca Demetrio, Battista Biggio, Giovanni Lagorio, Fabio Roli, and Alessandro Armando. Explaining vulnerabilities of deep learning to adversarial malware binaries. In Proceedings of the Third Italian Conference on Cyber Security , ITASEC, 2019.

Dario Nisi, Mariano Graziano, Yanick Fratantonio, and Davide Balzarotti. Lost in the Loader: The Many Faces of the Windows PE File Format. In Proceedings of the 24th International Symposium on Research in Attacks, Intrusions and Defenses , RAID, 2021.

Wei Song, Xuezixiang Li, Sadia Afroz, Deepali Garg, Dmitry Kuznetsov, and Heng Yin. MABMalware: A Reinforcement Learning Framework for Blackbox Generation of Adversarial Malware. In Proceedings of the 2022 ACM Asia Conference on Computer and Communications Security , AsiaCCS, 2022.

William Fleshman, Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Nonnegative networks against adversarial attacks, 2019.

Krishnamurthy (Dj) Dvijotham, Jamie Hayes, Borja Balle, Zico Kolter, Chongli Qin, András György, Kai Xiao, Sven Gowal, and Pushmeet Kohli. A framework for robustness certification of smoothed classifiers using f-divergences. In International Conference on Learning Representations , ICLR, 2020.

Jerzy Neyman and Egon Sharpe Pearson. IX. on the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character , 231(694-706):289-337, February 1933.

Robert A. Wagner and Michael J. Fischer. The string-to-string correction problem. Journal of the ACM , 21(1):168-173, January 1974. ISSN 0004-5411.

Microsoft Defender Security Research Team. New machine learning model sifts through the good to unearth the bad in evasive malware, July 2019. URL https://www.microsoft.com/en-us/security/blog/2019/07/25/new-machinelearning-model-sifts-through-the-good-to-unearth-the-bad-in-evasivemalware/ .

Kaijun Liu, Shengwei Xu, Guoai Xu, Miao Zhang, Dawei Sun, and Haifeng Liu. A Review of Android Malware Detection Approaches Based on Machine Learning. IEEE Access , 8, 2020.

Kaspersky Lab. Machine Learning for Malware Detection, 2021. URL https: //media.kaspersky.com/en/enterprise-security/Kaspersky-Lab-WhitepaperMachine-Learning.pdf .

Blackberry Limited. Cylance AI from Blackberry, 2022. URL https://www.blackberry. com/us/en/products/cylance-endpoint-security/cylance-ai .

Edward Raff, Jon Barker, Jared Sylvester, Robert Brandon, Bryan Catanzaro, and Charles K. Nicholas. Malware detection by eating a whole EXE. In The Workshops of the Thirty-Second AAAI Conference on Artificial Intelligence , AAAI Workshops, 2018.

Marek Krˇ cál, Ondˇ rej Švec, Martin Bálek, and Otakar Jašek. Deep Convolutional Malware Classifiers Can Learn from Raw Executables and Labels Only. In The Workshops of 6th International Conference on Learning Representations , 2018.

Edward Raff, William Fleshman, Richard Zak, Hyrum S. Anderson, Bobby Filar, and Mark McLean. Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection. In Proceedings of the AAAI Conference on Artificial Intelligence , AAAI, 2021.

Octavian Suciu, Scott E. Coull, and Jeffrey Johns. Exploring Adversarial Examples in Malware Detection. In IEEE Security and Privacy Workshops , S&PW, 2019.

Luca Demetrio, Scott E. Coull, Battista Biggio, Giovanni Lagorio, Alessandro Armando, and Fabio Roli. Adversarial exemples: A survey and experimental evaluation of practical attacks on machine learning for windows malware detection. ACM Trans. Priv. Secur. , 24(4), September 2021. ISSN 2471-2566.

VirusTotal. Yara: The pattern matching swiss knife for malware researchers, 2022. URL https://virustotal.github.io/yara/ . Accessed: 2023-10-12.

Abdullah Al-Dujaili, Alex Huang, Erik Hemberg, and Una-May O'Reilly. Adversarial Deep Learning for Robust Detection of Binary Encoded Malware. In IEEE Security and Privacy Workshops , S&PW, 2018.

Klas Leino, Zifan Wang, and Matt Fredrikson. Globally-robust neural networks. In International Conference on Machine Learning , ICML, 2021.

Zayd Hammoudeh and Daniel Lowd. Feature partition aggregation: A fast certified defense against a union of sparse adversarial attacks, 2023.

Marc Fischer, Maximilian Baader, and Martin Vechev. Certified Defense to Image Transformations via Randomized Smoothing. In Advances in Neural Information Processing Systems , NeurIPS, 2020.

Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, and Bo Li. Tss: Transformation-specific smoothing for robustness certification. In ACM SIGSAC Conference on Computer and Communications Security , CCS, 2021.

Zhongkai Hao, Chengyang Ying, Yinpeng Dong, Hang Su, Jian Song, and Jun Zhu. Gsmooth: Certified robustness against semantic transformations via generalized randomized smoothing. In International Conference on Machine Learning , ICML, 2022.

Jiehang Zeng, Jianhan Xu, Xiaoqing Zheng, and Xuanjing Huang. Certified robustness to text adversarial attacks by randomized [mask]. Computational Linguistics , 49:395-427, 2023.

Jan Schuchardt, Yan Scholten, and Stephan Günnemann. (Provable) adversarial robustness for group equivariant tasks: Graphs, point clouds, molecules, and more. In Advances in Neural Information Processing Systems , NeurIPS, 2023.

Aleksandar Bojchevski, Johannes Gasteiger, and Stephan Günnemann. Efficient robustness certificates for discrete data: Sparsity-aware randomized smoothing for graphs, images and more. In International Conference on Machine Learning , ICML, 2020.

Ambra Demontis, Marco Melis, Battista Biggio, Davide Maiorca, Daniel Arp, Konrad Rieck, Igino Corona, Giorgio Giacinto, and Fabio Roli. Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware Detection. IEEE Transactions on Dependable and Secure Computing , 16(4):711-724, 2019.

Erwin Quiring, Lukas Pirch, Michael Reimsbach, Daniel Arp, and Konrad Rieck. Against All Odds: Winning the Defense Challenge in an Evasion Competition with Diversification. arXiv preprint:2010.09569 , 2020.

Yizheng Chen, Shiqi Wang, Yue Qin, Xiaojing Liao, Suman Jana, and David Wagner. Learning security classifiers with verified global robustness properties. In ACM SIGSAC Conference on Computer and Communications Security , CCS, 2021.

Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J. Zico Kolter. Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems , NeurIPS, 2018.

Jeet Mohapatra, Ching-Yun Ko, Lily Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel. Hidden cost of randomized smoothing. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , AISTATS, 2021.

Panagiotis Charalampopoulos, Solon P. Pissis, Jakub Radoszewski, Tomasz Wale´ n, and Wiktor Zuba. Unary Words Have the Smallest Levenshtein k-Neighbourhoods. In 31st Annual Symposium on Combinatorial Pattern Matching , CPM, 2020.

Spark Tsao. Faster and More Accurate Malware Detection Through Predictive Machine Learning, November 2019. URL https://www.trendmicro.com/vinfo/pl/security/news/ security-technology/faster-and-more-accurate-malware-detection-throughpredictive-machine-learning-correlating-static-and-behavioral-features .

R. Vinayakumar, Mamoun Alazab, K. P. Soman, Prabaharan Poornachandran, and Sitalakshmi Venkatraman. Robust Intelligent Malware Detection Using Deep Learning. IEEE Access , 7, 2019.

Hojjat Aghakhani, Fabio Gritti, Francesco Mecca, Martina Lindorfer, Stefano Ortolani, Davide Balzarotti, Giovanni Vigna, and Christopher Kruegel. When Malware is Packin' Heat; Limits of Machine Learning Classifiers Based on Static Analysis Features. In Proceedings of Symposium on Network and Distributed System Security , NDSS, 2020.

Frederic Perriot. Defeating Polymorphism Through Code Optimization. In Proceedings of the 2003 Virus Bulletin Conference , VB, 2003.

Mihai Christodorescu, Johannes Kinder, Somesh Jha, Stefan Katzenbeisser, and Helmut Veith. Malware normalization. Technical Report TR1539, Department of Computer Sciences, University of Wisconsin-Madison, 2005.

Andrew Walenstein, Rachit Mathur, Mohamed R. Chouchane, and Arun Lakhotia. Normalizing Metamorphic Malware Using Term Rewriting. In Sixth IEEE International Workshop on Source Code Analysis and Manipulation , SCAM, 2006.

Danilo Bruschi, Lorenzo Martignoni, and Mattia Monga. Code Normalization for Self-Mutating Malware. IEEE Security & Privacy , 5(2):46-54, 2007.

Konrad Rieck, Thorsten Holz, Carsten Willems, Patrick Düssel, and Pavel Laskov. Learning and classification of malware behavior. In Detection of Intrusions and Malware, and Vulnerability Assessment , DIMVA, 2008.

Yanfang Ye, Lifei Chen, Dingding Wang, Tao Li, Qingshan Jiang, and Min Zhao. Sbmds: an interpretable string based malware detection system using svm ensemble with bagging. Journal in Computer Virology , 5(4):283, November 2008. ISSN 1772-9904.

Yong Qiao, Yuexiang Yang, Lin Ji, and Jie He. Analyzing malware by abstracting the frequent itemsets in API call sequences. In 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications , TrustCom, 2013.

Haodi Jiang, Turki Turki, and Jason T. L. Wang. DLGraph: Malware detection using deep learning and graph embedding. In International Conference On Machine Learning And Applications , 2018.

Zhaoqi Zhang, Panpan Qi, and Wei Wang. Dynamic malware analysis with feature engineering and feature learning. In Proceedings of the AAAI Conference on Artificial Intelligence , AAAI, 2020.

Ishai Rosenberg, Asaf Shabtai, Lior Rokach, and Yuval Elovici. Generic black-box end-to-end attack against state of the art API call based malware classifiers. In Proceedings of the 21st International Symposium on Research in Attacks, Intrusions, and Defenses , RAID, 2018.

Weiwei Hu and Ying Tan. Black-box attacks against RNN based malware detection algorithms. In The Workshops of the Thirty-Second AAAI Conference on Artificial Intelligence , AAAI Workshops, 2018.

Ishai Rosenberg, Asaf Shabtai, Yuval Elovici, and Lior Rokach. Query-efficient black-box attack against sequence-based malware classifiers. In Annual Computer Security Applications Conference , ACSAC, 2020.

Fenil Fadadu, Anand Handa, Nitesh Kumar, and Sandeep Kumar Shukla. Evading API call sequence based malware classifiers. In Proceedings of 21st International Conference on Information and Communications Security , ICICS, 2020.

National Security Agency. Ghidra (version 10.1.5). URL https://www.nsa.gov/ghidra .

Marco Barreno, Blaine Nelson, Russell Sears, Anthony D Joseph, and J Doug Tygar. Can machine learning be secure? In Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security , AsiaCCS, 2006.

Microsoft Corporation. PE format, June 2022. URL https://docs.microsoft.com/enus/windows/win32/debug/pe-format .

VirusShare.com. VirusShare.com. URL https://virusshare.com/ .

Matthew G. Schultz, Eleazar Eskin, Erez Zadok, and Salvatore J. Stolfo. Data mining methods for detection of new malicious executables. In IEEE Symposium on Security and Privacy , S&P, 2001.

J. Zico Kolter and Marcus A. Maloof. Learning to Detect and Classify Malicious Executables in the Wild. Journal of Machine Learning Research , 7(99):2721-2744, 2006.

Chocolatey Software. Chocolatey software. URL https://chocolatey.org/ .

Chocolatey Software. Chocolately software docs | security. URL https://docs.chocolatey. org/en-us/information/security .

Luca Demetrio and Battista Biggio. secml-malware: Pentesting windows malware classifiers with adversarial EXEmples in python. arXiv preprint:2104.12848 , 2021.

Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers. In Advances in Neural Information Processing Systems , NeurIPS, 2019.

Daniel Lowd and Christopher Meek. Good word attacks on statistical spam filters. In Second Conference on Email and Anti-Spam , CEAS, 2005.

Yan Scholten, Jan Schuchardt, Simon Geisler, Aleksandar Bojchevski, and Stephan Günnemann. Randomized message-interception smoothing: Gray-box certificates for graph neural networks. In Advances in Neural Information Processing Systems , NeurIPS, 2022.

Jinyuan Jia, Binghui Wang, Xiaoyu Cao, and Neil Zhenqiang Gong. Certified robustness of community detection against adversarial structural perturbation via randomized smoothing. In Proceedings of The Web Conference 2020 , WWW, pages 2718-2724, 2020. doi: 10.1145/ 3366423.3380029.

A Brute-force edit distance certification

In this appendix, we show that an edit distance certification mechanism based on brute-force search is computationally infeasible. Suppose we are interested in issuing an edit distance certificate at radius r for a sequence classifier f at input x . Recall from (1) that in order to issue a certificate, we must show there exists no input ¯ x within the edit distance neighborhood N r ( x ) that would change f 's prediction. This problem can theoretically be tackled in a brute-force manner, by querying f for all inputs in N r ( x ) . In the best case, this would take time linear in |N r ( x ) | , assuming f responds to queries in constant time. However the following lower bound [68], shows that the size of the edit distance neighborhood is too large even in the best case:

For example, applying the loosest bound that is independent of x , we see that brute-force certification at radius r = 128 would require in excess of 255 r ≈ 10 308 queries to f . In contrast, our probabilistic certification mechanism (Figure 1) makes n pred + n bnd queries to f , and we can provide high probability guarantees when the number of queries is of order 10 3 or 10 4 .

B Proofs for Section 4

In this appendix, we provide proofs of the theoretical results stated in Section 4.

B.1 Proof of Proposition 3

A sufficient condition for (8) is

̸

̸

̸

We first consider the multi-class case where |Y| > 2 . If η y ≥ min y ′ = y η y ′ , then p y (¯ x ; h ) ≥ max y ′ = y p y ′ (¯ x ; h ) by (17) and we can upper-bound max y ′ = y p y ′ (¯ x ; h ) by 1 2 . On the other hand, if η y ≥ min y ′ = y η y ′ , we can only upper-bound max y ′ = y p y ′ (¯ x ; h ) by 1 . Thus when |Y| > 2 (17) implies

̸

̸

̸

̸

̸

̸

̸

Next, we consider the binary case where |Y| = 2 . Since the confidences sum to 1, we have max y ′ = y p y ′ (¯ x ; h ) = 1 -p y (¯ x ; h ) . Putting this in (17) implies

̸

B.2 Proof of Lemma 4

Let r S : S →{ 1 , . . . , | S |} be a bijection that returns the rank of an element in an ordered set S . Let ˙ r S : 2 S → 2 { 1 ,..., | S |} be an elementwise extension of r S that returns a set of ranks for an ordered set of elements-i.e., ˙ r S ( U ) = { r S ( i ) : i ∈ U } for U ⊆ S . We claim m (¯ ϵ ) = ˙ r -1 ϵ ⋆ ( ˙ r ¯ ϵ ⋆ (¯ ϵ )) is a bijection that satisfies the required property.

To prove the claim, we note that m is a bijection from 2 ¯ ϵ ⋆ to 2 ϵ ⋆ since it is a composition of bijections ˙ r ¯ ϵ ⋆ : 2 ¯ ϵ ⋆ → 2 { 1 ,...,l } and ˙ r -1 ϵ ⋆ : 2 { 1 ,...,l } → 2 ϵ ⋆ where l = | ¯ ϵ ⋆ | = | ϵ ⋆ | . Next, we observe that ˙ r ¯ ϵ ⋆ (¯ ϵ ) relabels indices in ¯ ϵ so they have the same effect when applied to z ⋆ as ¯ ϵ on ¯ x (this also holds for ˙ r ϵ ⋆ and ϵ ). Thus

̸

̸

as required. To prove the final statement, we use (4), (5) and (12) to write

where the second last line follows from the fact that apply(¯ x , ¯ ϵ ) = apply( x , ϵ ) = z .

B.3 Proof of Theorem 5

Let ¯ ϵ ⋆ and ϵ ⋆ be defined as in Lemma 4. We derive an upper bound on the sum over ϵ ∈ 2 ϵ ⋆ that appears in (13). Observe that

where the first line follows from the inequality 1 h (apply( x ,ϵ )= y ) ≤ 1 ; the second line follows from the law of total probability; the third line follows by constraining the indices { 1 , . . . , | x |} \ ¯ ϵ ⋆ to be deleted; and the last line follows from the normalization of the binomial distribution. Putting (18) and ∑ ¯ ϵ ∈ 2 ¯ ϵ ⋆ s (¯ ϵ, ¯ x ; h ) ≥ 0 in (13) gives

In the second line above we use the following relationship between the LCS distance and the length of the LCS | z ⋆ | = | ϵ ⋆ | :

Since (19) is independent of the base classifier h , the lower bound on ρ (¯ x , x , µ y ) follows immediately.

B.4 Proof of Corollary 6

Since the length of x can only be changed by inserting or deleting elements in ¯ x , we have

We also observe that the LCS distance can be uniquely decomposed in terms of the counts of insertion ops m ins and deletion ops m del : dist LCS (¯ x , x ) = m del + m ins . These counts can in turn be related to the given decomposition of edit ops counts for generalized edit distance. In particular, any substitution must be expressed as an insertion and deletion under LCS distance, which implies m ins = n ins + n sub and m del = n del + n sub . Thus we have

Substituting (20) and (21) in (14) gives the required result.

B.5 Proof of Theorem 7

Eliminating n sub from (16) using the constraint n sub = r -n del -n ins , we obtain a minimization problem in two variables:

where ψ ( n ins , n del ) = p n del -n ins del ( µ y -1 + p r -n del del ) . Observe that ψ is monotonically increasing in n ins and n del :

where the second inequality follows since we only consider r and µ y such that the numerator and denominator are positive. Thus the minimizer is ( n ⋆ ins , n ⋆ del , n ⋆ sub ) = (0 , 0 , r ) and we find ρ ( x ; µ y ) = µ y -1+ p r del . The expression for the certified radius follows by solving ρ ( x ; µ y ) ≥ ν y ( η ) for non-negative integer r .

B.6 Proof of Corollary 8

Recall that Corollary 6 gives the following lower bound on the classifier's confidence at x :

Observe that we can replace µ y by a lower bound µ y that holds with probability 1 -α (as is done in lines 4-6 of Figure 1) and obtain a looser lower bound ˜ ρ (¯ x , x , µ y ) ≤ ˜ ρ (¯ x , x , µ y ) that holds with probability 1 -α . Crucially, this looser lower bound has the same functional form, so all results depending on Corollary 6, namely Theorem 7 and Table 1, continue to hold albeit with probability 1 -α .

C Background for malware detection case study

In this appendix, we provide background for our case study on malware detection, including motivation for studying certified robustness of malware detectors, a formulation of malware detection as a sequence classification problem, and a threat model for adversarial examples.

C.1 Motivation

Malware (malicious software) detection is a vital capability for proactively defending against cyberattacks. Despite decades of progress, building and maintaining effective malware detection systems remains a challenge, as malware authors continually evolve their tactics to bypass detection and exploit new vulnerabilities. One technology that has lead to advancements in malware detection, is the application of machine learning (ML), which is now used in many commercial systems [69, 44, 46, 47] and continues to be an area of interest in the malware research community [70, 71, 45, 50]. While traditional detection techniques rely on manually-curated signatures or detection rules, ML allows a detection model to be learned from a training corpus, that can potentially generalize to unseen programs.

Although ML has an apparent advantage in detecting previously unseen malware, recent research has shown that ML-based static malware detectors can be evaded by applying adversarial perturbations [18-20, 37, 51, 31, 52, 17, 39]. A variety of perturbations have been considered with different effects at the semantic level, however all of them can be modeled as inserting, deleting and/or substituting bytes. This prompts us to advance certified robustness for sequence classifiers within this general threat model-where an attacker can perform byte-level edits.

C.2 Related work

Several empirical defense methods have been proposed to improve robustness of ML classifiers [63, 64]. Incer Romeo et al. [34] compose manually crafted Boolean features with a classifier that is constrained to be monotonically increasing with respect to selected inputs. This approach permits a combination of (potentially vulnerable) learned behavior with domain knowledge, and thereby aims to mitigate adversarial examples. Demontis et al. [63] show that the sensitivity of linear support vector machines to adversarial perturbations can be reduced by training with ℓ ∞ regularization of weights. In another work, Quiring et al. [64] take advantage of heuristic-based semantic gap detectors and an ensemble of feature classifiers to improve empirical robustness. Compared to our work on certified adversarial defenses, these approaches do not provide formal guarantees.

Binary normalization [72-75] was originally proposed to defend against polymorphic/metamorphic malware, and can also be seen as a mitigation to certain adversarial examples. It attempts to sanitize binary obfuscation techniques by mapping malware to a canonical form before running a detection algorithm. However, binary normalization cannot fully mitigate attacks like Disp (see Table 9), as deducing opaque and evasive predicates are NP-hard problems [17].

Dynamic analysis can provide additional insights for malware detection. In particular, it can record a program's behavior while executing it in a sandbox (e.g., collecting a call graph or network traffic) [7680]. Though detectors built on top of dynamic analysis can be more difficult to evade, as the attacker needs to obfuscate the program's behavior, they are still susceptible to adversarial perturbations. For example, an attacker may insert API calls to obfuscate a malware's behavior [81-84]. Applying RS -Del to certify detectors that operate on call sequences [80] or more general dynamic features would be an interesting future direction.

C.3 Static ML-based malware detection

We formulate malware detection as a sequence classification problem, where the objective is to classify a file in its raw byte sequence representation as malicious or benign. In the notation of Section 2, we assume the space of input sequences (files) is X = Ω ⋆ where Ω = { 0 , 1 , . . . , 255 } denotes the set of bytes, and we assume the set of classes is Y = { 0 , 1 } where 1 denotes the 'malicious' class and 0 denotes the 'benign' class. Within this context, a malware detector is simply a classifier f : X → Y .

Detector assumptions Malware detectors are often categorized according to whether they perform static or dynamic analysis. Static analysis extracts information without executing code, whereas dynamic analysis extracts information by executing code and monitoring its behavior. In this work, we focus on machine learning-based static malware detectors, where the ability to extract and synthesize information is learned from data. Such detectors are suitable as base classifiers for RS -Del , as they can learn to make (weak) predictions for incomplete files where chunks of bytes are arbitrarily removed. We note that dynamic malware detectors are not compatible with RS -Del , since it is not generally possible to execute an incomplete file.

Incorporating semantics In Section 3.3, we noted that our methods are compatible with sequence chunking where the original input sequence is partitioned into chunks, and reinterpreted as a sequence of chunks rather than a sequence of lower-level elements. In the context of malware detection, we can partition a byte sequence into semantically meaningful chunks using information from a disassembler, such as Ghidra [85]. For example, a disassembler can be applied to a Windows executable to identify chunks of raw bytes that correspond to components of the header, machine instructions, raw data, padding etc. Applying our deletion smoothing mechanism at the level of semantic chunks, rather than raw bytes, may improve robustness as it excludes edits within chunks that may be semantically invalid. It also yields a different chunk-level edit distance certificate, that may cover a larger set of adversarial examples than a byte-level certificate of the same radius. Figure 3 illustrates the difference between byte-level and chunk-level deletion for a Windows executable, where chunks correspond to machine instructions (such as push ebp ) or non-instructions ( NI ).

C.4 Threat model

We next specify the modeled attacker's goals, capabilities and knowledge for our malware detection case study [86].

Attacker's objective We consider evasion attacks against a malware detector f : X → Y , where the attacker's objective is to transform an executable file x so that it is misclassified by f . To ensure the attacked file ¯ x is useful after evading detection, we require that it is functionally equivalent to the original file x . We focus on evasion attacks that aim to misclassify a malicious file as benign in our experiments, as these attacks dominate prior work [52]. However, the robustness certificates derived in Section 4 also cover attacks in the opposite direction-where a benign file is misclassified as malicious .

Attacker's capability We measure the attacker's capability in terms of the number of elementary edits they make to the original file x . If the attacker is capable of making up to c elementary edits, then they can transform x into any file in the edit distance ball of radius c centred on x :

Here dist O ( x , ¯ x ) denotes the edit distance from the original file x to the attacked file ¯ x under the set of edit operations (ops) O . We assume O consists of elementary byte-level or chunk-level deletions ( del ), insertions ( ins ) and substitutions ( sub ), or a subset of these operations.

We note that edit distance is a reasonable proxy for the cost of running evasion attacks that iteratively apply localized functionality-preserving edits (e.g., [19, 37, 38, 17, 39]). For these attacks, the edit distance scales roughly linearly with the number of attack iterations, and therefore the attacker has an incentive to minimize edit distance. While attacks do exist that make millions of elementary edits in the malware domain (e.g., [31]), we believe that an edit distance-constrained threat model is an important step towards realistic threat models for certified malware detection. (To examine the effect of large edits on robustness we include the GAMMA attack [31] in experiments covered in Appendix F.)

Remark 9 . The set A c ( x ) overestimates the capability of an edit distance-constrained attacker, because it may include files that are not functionally equivalent to x . For example, A c ( x ) may include files that are not malicious (assuming x is malicious) or files that are invalid executables. This poses no problem for certification, since overestimating an attacker's capability merely leads to a stronger certificate than required. Indeed, overestimating the attacker's capability seem necessary, as functionally equivalent files are difficult to specify, let alone analyze.

Attacker's knowledge In our certification experiments in Appendix E, we assume the attacker has full knowledge of the malware detector and certification scheme. When testing published attacks in Appendix F, we consider both white-box and black-box access to the malware detector. In the black-box setting, the attacker may make an unlimited number of queries to the malware detector without observing its internal operation. We permit access to detection confidence scores, which are returned alongside predictions even in the black-box setting. In the white-box setting, the attacker can additionally inspect the malware detector's source code. Such a strong assumption is needed for white-box attacks against neural network-based detectors that compute loss gradients with respect to the network's internal representation of the input file [20, 17].

D Experimental setup for malware detection case study

In this appendix, we detail the experimental setup for our malware detection case study.

D.1 Datasets

Though our methods are compatible with executable files of any format, in our experiments we focus on the Portable Executable (PE) format [87], since datasets, malware detection models and adversarial attacks are more extensively available for this format. Moreover, PE format is the standard for executables, object files and shared libraries in the Microsoft Windows operating system, making it an attractive target for malware authors. We use two PE datasets which are summarized in Table 3 and described below.

Sleipnir2 This dataset attempts to replicate data used in past work [54], which was not published with raw samples. We were able to obtain the raw malicious samples from a public malware repository called VirusShare [88] using the provided hashes. However, since there is no similar public repository for benign samples, we followed established protocols [89, 90, 20] to collect a new set of benign samples. Specifically, we set up a Windows 7 virtual machine with over 300 packages installed using Chocolatey package manager [91]. We then extracted PE files from the virtual machine, which were assumed benign 4 , and subsampled them to match the number of malicious samples. The dataset is randomly split into training, validation and test sets with a ratio of 60%, 20% and 20% respectively.

VTFeed This dataset was first used in recent attacks on end-to-end ML-based malware detectors [17]. It was collected from VirusTotal-a commercial threat intelligence service-by sampling PE files from the live feed over a period of two weeks in 2020. Labels for the files were derived from the 68 antivirus (A V) products aggregated on VirusTotal at the time of collection. Files were labeled malicious if they were flagged malicious by 40 or more of the A V products, they were labeled benign if they were not flagged malicious by any of the A V products, and any remaining files were excluded. Following Lucas et al. [17], the dataset is randomly split into training, validation and test sets with a ratio of 80%, 10%, and 10% respectively.

We note that VTFeed comes with strict terms of use, which prohibit us from loading it on our high performance computing (HPC) cluster. As a result, we use Sleipnir2 for comprehensive experiments (e.g., varying p del , η ) on the HPC cluster, and VTFeed for a smaller selection of experiments run on a local server.

D.2 Malware detection models

We experiment with malware detection models based on MalConv [48]. MalConv was one of the first end-to-end neural network models proposed for malware detection-i.e., it learns to classify directly from raw byte sequences, rather than relying on manually engineered features. Architecturally, it composes a learnable embedding layer with a shallow convolutional network. A large window size and stride of 500 bytes are employed to facilitate scaling to long byte sequences. Though MalConv is compatible with arbitrarily long byte sequences in principle, we truncate all inputs to 2MB to support training efficiency. We use the original parameter settings and training procedure [48], except where specified in Appendix D.5.

Using MalConv as a basis, we consider three models as described below.

NS A vanilla non-smoothed ( NS ) MalConv model. This model serves as a non-certified, non-robust baseline-i.e., no specific techniques are employed to improve robustness to evasion attacks and certification is not supported.

RS -Abn A smoothed MalConv model using the randomized ablation smoothing mechanism proposed by Levine and Feizi [24] and reviewed in Appendix I. This model serves as a certified robust baseline, albeit covering a more restricted threat model than the edit distance threat model we propose in Section 2. Specifically, it supports robustness certification for the Hamming distance threat model, where the adversary is limited to substitution edits ( O = { sub } ). Since Levine and Feizi's formulation is for images, several modifications are required to support malware detection as described in Appendix G. To improve convergence, we also apply gradient clipping when learning parameters in the embedding layer (see Appendix G). We consider variants of this model for different values of the ablation probability p ab .

RS -Del A smoothed MalConv model using our proposed randomized deletion smoothing mechanism. This model supports robustness certification for the generalized edit distance threat model where O ⊆ { del , ins , sub } . We consider variants of this model for different values of the deletion probability p del , decision thresholds η , and whether deletion/certification is performed at the bytelevel (BYTE) or chunk-level (INSN). We perform chunking as illustrated in Figure 3-i.e., we chunk bytes that correspond to distinct machine instructions using the Ghidra disassembler.

D.3 Controlling false positive rates

Malware detectors are typically tuned to achieve a low false positive rate (FPR) (e.g., less than 0 . 1 -1% ) since producing too many false alarms is a nuisance to users. 5 To make all malware detection models comparable, we calibrate the FPR to 0.5% on the test set for the experiments reported in Appendix E and 0.5% on the validation set for the experiments reported in Appendix F unless otherwise noted. This calibration is done by adjusting the decision threshold of the base MalConv model.

D.4 Compute resources

Experiments for the Sleipnir2 dataset were run a high performance computing (HPC) cluster, where the requested resources varied depending on the experiment. We generally requested a single NVIDIA P100 GPU when training and certifying models. Experiments for the VTFeed dataset were run on a local server due to restrictive terms of use. Compute resources and approximate wall clock running times are reported in Tables 4 and 5 for training and certification for selected parameter settings. Running times for other parameters settings are lower than the ones reported in these tables.

D.5 Parameter settings

We specify the parameter settings and training procedure for MalConv, which is used standalone in NS , and as a base model for the smoothed models RS -Del and RS -Abn . Table 6 summarizes our setup, which is consistent across all three models except where specified. We follow the authors of MalConv [48] when setting parameters for the model and the optimizer, however we set a larger

maximum input size of 2MiB to accommodate larger inputs without clipping. Due to differences in available GPU memory for the Sleipnir2 and VTFeed experiments, we use a larger batch size for VTFeed than for Sleipnir2. We also set a higher limit on the maximum number of epochs for VTFeed, as it is a larger dataset, although the NS and RS -Del models converge within 50 epochs for both datasets. To stabilize training for the smoothed models ( RS -Del and RS -Abn ), we modify the smoothing mechanisms during training only to ensure at least 500 raw bytes are preserved. This may limit the number of deletions for RS -Del and the number of ablated (masked) bytes for RS -Abn . For RS -Abn , we clip the gradients for the embedding layer to improve convergence (see Appendix G).

E Evaluation of robustness certificates for malware detection

In this appendix, we evaluate the robustness guarantees and accuracy of RS -Del for malware detection. We consider two instantiations of the edit distance threat model. First, in Appendix E.1, we consider the Levenshtein distance threat model, where the attacker's elementary edits are unconstrained and may include deletions, insertions and substitutions. Then, in Appendix E.2, we consider the more restricted Hamming distance threat model, where an attacker is only able to perform substitutions. We summarize our findings in Appendix E.3. Overall, we find that RS -Del generates robust predictions with minimal impact on model accuracy for the Levenshtein distance threat model, and outperforms RS -Abn [24] for the Hamming distance threat model.

We report the following quantities in our evaluation:

Certified radius (CR). The radius of the largest robustness certificate that can be issued for a given input, model and certification method. Note that this is a conservative measure of robustness since it is tied to the certification method . The median CR is reported on the test set.

Certified accuracy [13, 14], also known as verified-robust accuracy [66, 55], evaluates robustness certificates and accuracy of a model simultaneously with respect to a test set. It is defined as the fraction of instances in the test set D for which the model f 's prediction is correct and certified robust at radius r or greater:

where CR( x ) denotes the certified radius for input x returned by the certification method.

Clean accuracy. The fraction of instances in the test set for which the model's prediction is correct.

We briefly mention default parameter settings for the experiments presented in this appendix. When approximating the smoothed models ( RS -Del and RS -Abn ) we sample n pred = 1000 perturbed inputs for prediction and n bnd = 4000 perturbed inputs for certification, while setting the significance level α to 0 . 05 . Unless otherwise specified, we set the decision thresholds for the smoothed models so that η = 0 . After fixing η , the decision thresholds for the base models are tuned to yield a false positive rate of 0.5%. We note that the entire test set is used when reporting metrics and summary statistics in this appendix.

E.1 Levenshtein distance threat model

We first present results for the Levenshtein distance threat model, where the attacker's elementary edits are unconstrained ( O = { del , ins , sub } ). We vary three parameters associated with RS -Del : the deletion probability p del , the decision thresholds of the smoothed model η , and the level of sequence chunking (i.e., whether sequences are chunked at the byte-level or instruction-level). We use NS as a baseline as there are no prior certified defenses for the Levenshtein distance threat model to our knowledge.

0

200

400

600

Radius,

r

0

10

20

30

40

50

60

70

80

90

100

Certified accuracy (

%

)

p

del

97%

99%

99.5%

99.9%

0.2

0.4

0.6

0.8

1.0

Normalized Radius,

r

(

%

)

0

10

20

30

40

50

60

70

80

90

100

Certified accuracy (

%

)

p

del

97%

99%

99.5%

99.9%

0

200

400

600

Radius,

r

0

10

20

30

40

50

60

70

80

90

100

Certified accuracy (

%

)

p

del

97%

99%

99.5%

99.9%

0.2

0.4

0.6

0.8

1.0

Normalized Radius,

r

(

%

)

0

10

20

30

40

50

60

70

80

90

100

Certified accuracy (

%

)

p

del

97%

99%

99.5%

99.9%

Certified accuracy Figure 4 plots the certified accuracy of RS -Del using byte-level deletion as a function of the radius (left horizontal axis), radius normalized by file size (right horizontal axis) on the Sleipnir2 dataset for several values of p del . We observe that the curves for larger values of p del approximately dominate the curves for smaller values of p del , for p del ≤ 99 . 5% (i.e., the accuracy is higher or close for all radii). This suggests that the robustness of RS -Del can be improved without sacrificing accuracy by increasing p del up to 99.5%. However, for the larger value p del = 99 . 9% , we observe a drop in certified accuracy of around 10% for smaller radii and an increase for larger radii. By normalizing with respect to the file size, we can see that our certificate is able to certify up to 1% of the file size. We also include an analogous plot for chunk-level deletion in Figure 5 which demonstrates similar behavior. We note that chunk-level deletion arguably provides stronger guarantees, since the effective radius for chunk-level Levenshtein distance is larger than for byte-level Levenshtein distance.

It is interesting to relate these certification results to published evasion attacks. Figure 4 shows that we can achieve a certified accuracy in excess of 90% at a Levenshtein distance radius of 128 bytes when p del = 99 . 5% . This radius is larger than the median Levenshtein distance of two attacks that manipulate headers of PE files [37, 38] (see Table 9 in Appendix F). We can therefore provide reasonable robustness guarantees against these two attacks. However, a radius of 128 bytes is orders of magnitude smaller than the median Levenshtein distances of other published attacks which range from tens of KB [17, 20] to several MB [31] (also reported in Table 9). While some of these attacks

arguably fall outside an edit distance constrained threat model, we consider them in our empirical evaluation of robustness in Appendix F.

Clean accuracy and abstention rates Table 7 reports clean accuracy for RS -Del and the noncertified NS baseline. It also reports abstention rates for RS -Del , the median certified radius (CR), and the median certified radius normalized by file size (NCR). We find that clean accuracy for Sleipnir2 follows similar trends as certified accuracy: it is relatively stable for p del in the range 90-99.5%, but drops by more than 10% at p del = 99 . 9% . We note that the clean accuracy of RS -Del (excluding p del = 99 . 9% ) is at most 3% lower than the NS baseline for Sleipnir2 and at most 7% lower than the NS baseline for VTFeed. We observe minimal differences in the results for chunk-level (INSN) and byte-level (BYTE) deletion smoothing, but note that the effective CR is larger for chunk-level smoothing, since each chunk may contain several bytes.

Accuracy under high deletion It may be surprising that RS -Del can maintain high accuracy even when deletion is aggressive. We offer some possible explanations. First, we note that even with a high deletion probability of p del = 99 . 9% , the smoothed model accesses almost all of the file in expectation, as it aggregates n pred = 1000 predictions from the base model each of which accesses a random 0 . 1% of the file in expectation. Second, we posit that malware detection may be 'easy' for RS -Del on these datasets. This could be due to the presence of signals that are robust to deletion (e.g., file size or byte frequencies) or redundancy of signals (i.e., if a signal is deleted in one place it may be seen elsewhere).

Decision threshold We demonstrate how the decision thresholds η introduced in Section 3.1 can be used to trade off certification guarantees between classes. We consider normalized decision thresholds where ∑ y η y = 1 and η y ∈ [0 , 1] . We only specify the value of η 1 when discussing our results, noting that η 0 = 1 -η 1 in our two-class setting.

Table 8 provides error rates and robustness metrics for several values of η 1 , using byte-level Levenshtein distance with p del = 99 . 5% . When varying η 1 , we also vary the decision threshold of the base model to achieve a target false positive rate (FPR) of 0.5%. Looking at the table, we see that η 1 has minimal impact on the false negative rate (FNR), which is stable around 7%. However, there

is a significant impact on the median CR (and theoretical upper bound), as reported separately for each class. The median CR is balanced for both the malicious and benign class when η 1 = 50% , but favours the malicious class as η 1 is decreased. For instance when η 1 = 5% a significantly larger median CR is possible for malicious files (137 to 578) at the expense of the median CR for benign files (137 to 10). This asymmetry in the class-specific CR is a feature of the theory-that is, in addition to controlling a tradeoff between error rates of each class, η 1 also controls a tradeoff between the CR for each class (see Table 1).

Figure 6 plots the certified true positive rate (TPR) and true negative rate (TNR) of RS -Del on the Sleipnir2 dataset for several values of η 1 . The certified TPR and TNR can be interpreted as class-specific analogues of the certified accuracy. Concretely, the certified TPR (TNR) at radius r is the fraction of malicious (benign) instances in the test set for which the model's prediction is correct and certified robust at radius r . The certified TPR and TNR jointly measure accuracy and robustness and complement the metrics reported in Table 8. Looking at Figure 6, we see that the certified TNR curves drop more rapidly to zero than the certified TPR curves as η 1 decreases. Again, this suggests decreasing η 1 sacrifices the certified radii of benign instances to increase the certified radii of malicious instances. We note that the curves for η 1 = 50% correspond to the same setting as the certified accuracy curve in Figure 4 (with p del = 99 . 5% ).

E.2 Hamming distance threat model

We now turn to the more restricted Hamming distance threat model, where the attacker is limited to performing substitutions only ( O = { sub } ). We choose to evaluate this threat model as it is covered in previous work on randomized smoothing, called randomized ablation [24] (abbreviated RS -Abn ), and can serve as a baseline for comparison with our method. Recall that we adapt RS -Abn for malware detection by introducing a parameter called p ab , which is the fraction of bytes that are 'ablated' (replaced by a special masked value) (see Appendix D.2). This parameter is analogous to p del in RS -Del , except that the number of ablated bytes is deterministic in RS -Abn , whereas the number of deleted bytes is random in RS -Del . We compare RS -Del and RS -Abn for varying values of p del and p ab using the Sleipnir2 dataset and byte-level Hamming distance.

Certified accuracy Figure 2 plots the certified accuracy of RS -Del and RS -Abn for three values of p del and p ab . We observe that the certified accuracy is uniformly larger for our proposed method RS -Del than for RS -Abn when p del = p ab . The superior certification performance of RS -Del is somewhat surprising given it is not optimized for the Hamming distance threat model. One possible explanation relates to the learning difficulty of RS -Abn compared with RS -Del . Specifically, we find that stochastic gradient descent is slower to converge for RS -Abn despite our attempts to improve convergence (see Appendix G). Recall, that RS -Del provides certificates for any of the threat models in Table 1-in addition to the Hamming distance certificate-without needing to modify the smoothing mechanism.

0

200

400

600

800

1000

0

10

20

30

40

50

60

70

80

90

100

Certified TPR (

%

)

η

1

50%

25%

10%

5%

1%

0.5%

0

200

400

600

800

1000

Radius,

r

0

10

20

30

40

50

60

70

80

90

100

Certified TNR (

%

)

Tightness RS -Abn is provably tight, in the sense that it is not possible to issue a larger Hamming distance certificate unless more information is made available to the certification mechanism or the ablation smoothing mechanism is changed. This tightness result for RS -Abn , together with the empirical results in Figure 2, suggests that RS -Del produces certificates which are tight or close to tight in practice, at least for the Hamming distance threat model. This is an interesting observation, since it is unclear how to derive a tight, computationally tractable certificate for RS -Del .

E.3 Summary

Our evaluation shows that RS -Del provides non-trivial robustness guarantees with a low impact on accuracy. The certified radii we observe are close to the best radii theoretically achievable using our mechanism. For the Levenshtein byte-level edit distance threat model, we obtain radii of a few hundred bytes in size, which can certifiably defend against attacks that edit headers of PE files [37, 38, 52]. However, certifying robustness against more powerful attacks that modify thousands or millions of bytes remains an open challenge. By varying the detection threshold, we show that certification can be performed asymmetrically for benign and malicious instances. This can boost the certified radii of malicious instances by a factor of 4 in some cases. While there are no prior methods to use as baselines for the Levenshtein distance threat model, our comparisons with RS -Abn [24] for

the Hamming distance threat model show that RS -Del outperforms RS -Abn in terms of both accuracy and robustness.

F Evaluation of robustness to published attacks

In this appendix, we empirically evaluate the robustness of RS -Del to several published evasion attacks. By doing so, we aim to provide a more complete picture of robustness, as our certificates are conservative and may underestimate robustness to real attacks, which are subject to additional constraints (e.g., maintaining executability, preserving a malicious payload, etc.). We introduce the attacks in Appendix F.1, provide details of the experimental setup in Appendix F.2 and discuss the results in Appendix F.3.

F.1 Attacks covered

We consider five recently published attacks designed for evading static PE malware detectors as summarized in Table 9. The attacks cover a variety of edit distance magnitudes from tens of bytes to millions of bytes. While attacks that edit millions of bytes arguably fall outside our edit distanceconstrained threat model, we include one such attack ( GAMMA ) to test the limits of our methodology. We note that four of the five attacks are able to operate in a black-box setting and can therefore be applied directly to RS -Del . However, the white-box attacks are designed for neural network malware detectors with a specific architecture. In particular, they assume the network receives a raw byte sequence as input, that the initial layer is an embedding layer, and that gradients can be computed with respect to the output of the embedding layer. Although these architectural assumptions are satisfied by the base MalConv model, they are not satisfied by RS -Del , because additional operations are applied before the embedding layer and the aggregation of base model predictions is not differentiable. In Appendix H, we adapt the white-box attacks for RS -Del by applying two tricks: (1) we apply the smoothing mechanism after the embedding layer, and (2) we replace majority voting with soft aggregation following Salman et al. [94].

F.2 Experimental setup

Since some of the attacks take hours to run for a single file, we use smaller evaluation sets containing malware subsampled from the test sets in Table 3. The evaluation set we use for Sleipnir2 consists of 500 files, and the one for VTFeed consists of 100 files (matching [17]). We note that our evaluation sets are comparable in size to prior work [18, 20, 51]. For each evaluation set, we report attack success rates against malware detectors trained on the same dataset.

Since all attacks employ greedy optimization with randomization, they may fail on some runs, but succeed on others. We therefore repeat each attack 5 times per file and use the best performing attacked file in our evaluation. We define the attack success rate as the proportion of files initially detected as malicious for which at least one of the 5 attack repeats is successful at evading detection. Lower attack success rates correlate with improved robustness against attacks. We permit all attacks to run for up to 200 attack iterations of the internal optimizer. Early stopping is enabled for those attacks that support it ( Disp , Slack , GAMMA ), which means the attack terminates as soon as the model's prediction flips from malicious to benign.

Where possible, we run direct attacks against RS -Del and compare success rates against NS as a baseline. We also consider transfer attacks from NS to RS -Del as an important variation to the threat model, where an attacker has limited access to the target RS -Del during attack optimization. When running direct attacks against RS -Del , we use a reduced number of Monte Carlo samples ( n pred = 100 ) to make the computational cost of the attacks more manageable. For both direct and transfer attacks against RS -Del , we set p del = 97% and perform deletion and certification at the byte-level (BYTE).

F.3 Results

The results for direct attacks against RS -Del are presented in Table 10. For both the Sleipnir2 and VTFeed datasets, we observe that the robustness of RS -Del is superior (or equal) to NS against four of the six attacks. The two cases where RS -Del 's robustness drops compared to NS are for the strongest attacks: Slack and GAMMA . The results for transfer attacks from NS to RS -Del are presented in Table 11. Almost all of the attacks transfer poorly to RS -Del . In most cases the attack success rates drop to zero or single digit percentages. We hypothesize that Slack and GAMMA make such drastic changes to the original binary that they can overwhelm the malicious signal-enough to cross the decision boundary-akin to a good word attack [95]. We find that HDOS and HField are ineffective for both RS -Del and the baseline NS . Both attacks change up to 58 bytes in the header, and tend to fall within our certifications.

G Efficiency of RS-Del

In this appendix, we discuss the training and computational efficiency of RS -Del . We provide comparisons with RS -Abn [24], which serves as a baseline in Appendix E.2 for a more restricted Hamming distance threat model.

Computational efficiency Table 12 reports wall clock times for training and prediction. For training, we measure the time taken to complete 1 epoch of stochastic gradient descent on the Sleipnir2 training set, where inputs are perturbed by the smoothing mechanism. For prediction, we measure the time taken for a single 1MB input file using n pred = 1000 Monte Carlo samples. We split the prediction time into two components: (1) the time taken to generate perturbed inputs from the smoothing mechanism and (2) the time taken to aggregate predictions for the perturbed inputs using the base MalConv model. All times are recorded on a desktop PC fitted with an AMD Ryzen 7 5800X CPU and an NVIDIA RTX3090 GPU, using our PyTorch implementation of RS -Del and RS -Abn . We execute training and prediction for the base model on the GPU, and the smoothing mechanism on the CPU. We use a single PyTorch process, noting that times may be improved by running the smoothing mechanism in parallel or on the GPU.

We now make some observations about the results. First, we note that training is an order of magnitude faster for RS -Del compared with RS -Abn . We attribute this speed-up to the deletion smoothing mechanism of RS -Del , which drastically reduces the dimensionality of inputs, thereby reducing the time taken to perform forward and backward passes for the base model. On the contrary, the ablation smoothing mechanism of RS -Abn does not alter the dimensionality of inputs, so it does not have a performance advantage in this respect. Second, we observe that the total prediction time for RS -Del is approximately 150% faster than for RS -Abn . We expect this difference is also due to the effect of dimensionality reduction for the deletion smoothing mechanism.

Training efficiency Training curves for the base MalConv models used in RS -Del and RS -Abn are provided in Figure 7 for the Sleipnir2 dataset. Due to convergence issues for RS -Abn , we adapted training to incorporate gradient clipping when updating the embedding layer. This addresses imbalance in the gradients arising from the dominance of masked (ablated) values in the perturbed inputs. However, even with this fix, we observe slower convergence to a higher loss value for RS -Abn than for RS -Del . Combining the results of Table 12 and Figure 7, we conclude that RS -Abn beats RS -Abn in terms of training efficiency as it requires both fewer epochs to converge and takes less time per epoch.

0

20

40

60

80

100

Epoch

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

Training loss

Model,smoothing parameter

p

del

= 97%

p

del

= 99

.

5%

p

ab

= 97%

p

ab

= 99

.

5%

H Adapting attacks for smoothed classifiers

In this appendix, we show how to adapt gradient-guided white-box attacks to account for randomized smoothing.

We consider a generic family of white-box attacks that operate on neural network-based classifiers, where the first layer of the network is an embedding layer. Mathematically, we assume the classifier under attack f can be decomposed as

f embed is the embedding layer, which maps an input sequence x ∈ X of length n = | x | to an n × d array of d -dimensional embedding vectors;

f soft represents the subsequent layers in the network, which map an n × d embedding array to a probability distribution over classes Y ; and

f pred is an optional final layer, which maps a probability distribution over classes to a prediction (e.g., by taking the arg max or applying a threshold).

where

The attack may query any of the above components in isolation and compute gradients of f soft . For instance, the attacks we consider [20, 37, 17] optimize the input in the embedding space e ∈ R n × d , by computing the gradient ∂f soft ( e ) ∂ e .

While the above setup covers attacks against MalConv, or classifiers with similar architectures, it is not directly compatible with smoothed classifiers. There are two incompatibilities. First, a direct implementation of a smoothed classifier (as described in Section 3.1 and Figure 1) does not decompose like (23). And second, the aggregation of hard predictions from the base classifiers is non-differentiable. We explain how to address these incompatibilities below.

Leveraging commutativity Consider a smoothed classifier composed from a base classifier h and smoothing mechanism ϕ , and suppose the base classifier decomposes as in (23). Following Section 3.1 and Figure 1, the smoothed classifier's confidence score for class y can be expressed as

where

is the (empirical) smoothing operation. This expression does not immediately decompose like (23), because the embedding layer is applied to the perturbed input ϕ ( x ) , not x . Fortunately, we can manipulate the expression into the desired form by swapping the order of ϕ and h embed . To do so, we extend the definition of ϕ to operate on an embedding array so that h embed ( ϕ ( x )) = ϕ ( h embed ( x )) , i.e., ϕ and h embed commute. In particular, this can be done for randomized deletion ( RS -Del ) by applying the deletion edits to embedding vectors along the first dimension. Then (24) can equivalently be expressed as

Soft aggregation While (25) decomposes as required, the f soft component is not differentiable. This is due to the presence of h pred , which is an arg max layer. To proceed, we replace the aggregation of predictions by the aggregation of softmax scores as proposed by Salman et al. [94]. This yields a differentiable approximation of the smoothed classifier:

Salman et al. note that this approximation performs well, and is empirically more effective than an alternative approach proposed by Cohen et al. [14, Appendix G.3].

I Review of randomized ablation

In this appendix, we review randomized ablation [24], which serves as a baseline for the Hamming distance threat model in our experiments. It is based on randomized smoothing (see Section 3.1) like our method, however the smoothing mechanism and robustness certificate differ. In this review, we formulate randomized ablation for sequence classifiers; we refer readers to Levine and Feizi [24] for a formulation for image classifiers.

I.1 Ablation smoothing mechanism

Randomized ablation employs a smoothing mechanism that replaces a random subset of the input elements with a special null value NA . Levine and Feizi use an encoding for the null value tailored for images, that involves doubling the number of channels. This encoding is not suitable for discrete sequences, so we instead augment the sequence domain Ω with a special null value: Ω → Ω ∪ { NA } . The hyperparameter controlling the strength of ablation must also be adapted for our setting. Levine and Feizi use a hyperparameter k that corresponds to the number of elements retained in the output. This is ineffective for inputs that vary in length, so we scale k in proportion with the input length. Specifically, we introduce an alternative hyperparameter p ab ∈ (0 , 1) that represents the fraction of ablated elements and set k ( | x | ) = ⌈ (1 -p ab ) | x |⌉ .

Mathematically, the ablation mechanism has the following distribution when applied to an input sequence x :

where E k = { ϵ ∈ E ( x ) : | ϵ | = k } consists of all sets of element indices of size k and

Remark 10 . Hyperparameter p ab has a similar interpretation as p del for randomized deletion, in that both hyperparameters control the proportion of sequence elements hidden (by ablation or deletion) from the base classifier.

I.2 Hamming distance robustness certificate

Randomized ablation provides a Hamming distance ( ℓ 0 ) certificate that guarantees robustness under a bounded number of arbitrary substitutions. Levine and Feizi provide two certificates: one that makes use of the confidence score for the predicted class, and another that makes use of the top two confidence scores. We present the first certificate here, since it matches the certificate we consider in Section 4 and it is the simplest choice for binary classifiers (the focus of our experiments).

To facilitate comparison with randomized deletion, we reuse notation and definitions from Section 4. Recall from (15) that ˜ ρ ( x ; µ y ) is a lower bound on the smoothed classifier's confidence score p y (¯ x ; h ) that holds for any input ¯ x in the edit distance ball of radius r centered on x and any base classifier h such that p y ( x ; h ) = µ y . For randomized ablation, we can replace the edit distance ball with a Hamming distance ball and obtain the following result (see [24] for the derivation):

Recall from Proposition 3, that the smoothed classifier is certifiably robust if ˜ ρ ( x ; µ y ) ≥ ν y ( η ) , where η denotes the smoothed classifier's tunable decision thresholds. Hence the certified radius r ⋆ is the maximum value of r ∈ { 0 , 1 , 2 , . . . } that satisfies

This maximization problem does not have an analytic solution, however it can be solved efficiently using binary search since the LHS of the above inequality is a non-increasing function of r . In practice, the exact confidence score µ y can be replaced with a 1 -α lower confidence bound µ y to yield a probabilistic certificate that holds with probability 1 -α (see procedure in Figure 1 and Corollary 8).

Since randomized ablation and randomized deletion both admit Hamming distance certificates, it is interesting to compare them. The following result, first proved by Scholten et al. [96, Proposition 5] for a related mechanism, shows that randomized deletion admits a tighter certificate.

Proposition 11. Consider a randomized deletion classifier ( RS -Del ) with deletion probability p del = p and a randomized ablation classifier ( RS -Abn ) with ablation fraction p ab = p . Suppose both classifiers make the same prediction y with the same confidence score µ y for input x and that we are interested in issuing a Hamming distance certificate of radius r . Then the lower bound on the confidence score over the certified region is tighter for RS -Del than for RS -Abn .

Proof. From Theorem 7 we have

Remark 12 . Jia et al. [25] extend randomized ablation to topk prediction, while at the same time proposing enhancements to the Hamming distance certificate. These enhancements also apply to regular classification and involve: (1) discretizing lower/upper bounds on the confidence scores; and (2) using a better statistical method to estimate lower/upper bounds on the confidence scores. However, both of these enhancements would have a negligible impact in our experiments, as we shall now explain. The first enhancement tightens lower/upper bounds on the confidence scores by rounding up/down to the nearest integer multiple of q : -1 / ( | x | k ( | x | ) ) . This improves the lower/upper bound by at most

if the number of retained elements satisfies 1 ≤ k ≤ | x | , as is the case in our experiments. However, since we consider inputs of length | x | ≥ 10 3 , this means the improvement in the bound due to discretization is at most q ≤ 10 -3 . This improvement is comparable to the resolution of our estimator ( 1 /n bnd ∼ 10 -4 ), and consequently, discretization will not have a discernible impact. The second enhancement involves simultaneously estimating lower/upper bounds using a statistical method called SimuEM [97]. SimuEM has been demonstrated to improve tightness when there are multiple classes [25], however it has no impact when there are two classes (as is the case in our experiments).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes **RS-Del** -- a novel certified defense that provides guarantees w.r.t. insertion, deletion, and substitution of bytes within a variable length input.  As its name indicates, RS-Del is based on randomized smoothing.  However, unlike classic randomized smoothing, the authors cannot certify robustness using the Neyman-Pearson lemma and instead propose a novel certification scheme.\n\nSTRENGTHS:\nThe paper has a lot to like.  Overall the writing quality was good. Ideas were structured and explained well.  The writing was clear.\n\nCertified and robust training literature is dominated by works centered on some variant of an $\\ell_p$ threat model. It is rare to see a paper that prevents as novel of a certified threat model as this one.  Reading a paper with such an innovative threat model is refreshing and welcome.\n* I would further argue that some in the adversarial ML community do not appreciate how myopically the research community focuses on narrow definitions of robustness.  The paper's \"Related Work\" section does a good job of explaining this point.  I appreciated reading the author's perspective in this regard.\n\nThe most closely related work I am aware of is Saha et al. (2023) (which the authors cite).  I find RS-Del's threat model and type of guarantee much more useful and innovative.  Having looked at both papers, I find this work substantially stronger and more compelling.\n\nWEAKNESSES:\nThere are a lot of things I like about the paper.  That notwithstanding, this paper feels more appropriate for a security venue than an ML venue.  Understanding malware and the meaningfulness of guaranteeing robustness up to $m$ bytes is generally beyond the expertise of most ML readers (and reviewers).  I believe this paper would be a better fit and better appreciated at a venue like S&P or Usenix than at NeurIPS.  The misalignment with the venue slightly reduced my score.\n\n### Empirical Evaluation\n\nTo the extent of my knowledge, no other method provides guarantees w.r.t. to Levenshtein distance.  The authors select randomized ablation (RA) [Levine and Feizi 2020] as the primary baseline for Hamming distance.  Levine and Feizi's RA is far from the state-of-the-art $\\ell_0$ certified method at this point.  Two works consistently outperform vanilla RA:\n\n[1] Jinyuan Jia, Binghui Wang, Xiaoyu Cao, Hongbin Liu, and Neil Zhenqiang Gong. \"Almost Tight $\\ell_0$-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations\". ICLR 2022. https://openreview.net/forum?id=gJLEXy3ySpu.\n\n[2] Zayd Hammoudeh and Daniel Lowd. \"Feature Partition Aggregation: A Fast Certified Defense Against a Union of Sparse Adversarial Attacks\". arXiv 2302.11628. https://arxiv.org/abs/2302.11628\n\nJia et al. is based on RA and provides tight(er) certification analysis.  Hammoudeh and Lowd use model ensembles.  This paper does not cite either of these papers (Hammoudeh and Lowd is a newer preprint so I understand why the authors may not have seen it). However, at minimum, Jia et al.'s version of RA must be a baseline of comparison. \n* I would have voted \"Borderline Accept\" with the paper as is had at least Jia et al. (2022) been used as a baseline. Updating the baselines is a necessary condition for me to increase my score. I cannot recommend accepting this paper without at least Jia et al. (2022) as a baseline.  I strongly recommend adding both baselines or at minimum a compelling explanation in the rebuttal why one is not possible.\n\n### Interpreting the Empirical Results\n\nThe authors specify their certified guarantees in terms of the number of bytes.  I have two primary concerns about this framing.\n\n1. Existing certified methods also specify their guarantees in absolute terms. For example, RA provides median guarantees on CIFAR10 of 7 pixels. Providing guarantees in absolute terms makes sense when instances have a fixed size; its trivial to convert that result into a relative quantity (e.g., 1% of pixels). When test instances have variable sizes as in the case here, no simple conversion exists. For a reader to be able to appreciate how meaningful a guarantee of 128 bytes is, we need to understand the typical range of malware sizes.  At an ML venue, such information needs to be provided in the main paper.  Moreover, it is the author(s)'s responsibility to explain how the size of the guarantee changes with malware size.  \n\n2. Providing certified guarantees as a fraction of malware size is only a small part of the equation. I expect that most ML readers do not know a priori the extent of changes that must be made to a program to induce a change of 128 bytes. For example, would simply recompiling the program with no code changes and just compiler setting changes be sufficient to change the binary by 128 bytes?  I do not have expertise in that area, and the authors do not educate the reader in this regard.  I consider this choice to be a significant oversight.\n\n### Interpreting the Empirical Results\n\nCertified $\\ell_0$ method randomized ablation (RA) is used as the primary Hamming distance baseline.  However, the authors provide little to no explanation of how RA works.  I think this is important so a reader not familiar with RA can appreciate what is being tested in the experiments. Moreover, the authors need to explain how their method differs from RA when restricted to exclusively the substitution setting.\n\nQUESTIONS:\nSee my comments above under \"Weaknesses\".\n\nLIMITATIONS:\nI understand why the authors placed Table 1 where they did (put simply -- space).  However, the table's explanation does not appear until the next page.  When I first saw the table, I did not understand the table and thought perhaps I had missed something.  I think separating the tables on separate pages would be a better choice.  To prevent a significant increase in length, NeurIPS allows text to wrap around inline figures and tables.\n\nThe authors could provide more intuitions about why their setting needs turnable decision thresholds $\\eta_y$ while most other randomized smoothing applications do not.  Tying this explanation to their case study/expected end use would make the explanation much more compelling.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the issue of applying randomized smoothing to discrete sequences under the Levenshtein edit distance. Because the underlying sequence is discrete, it necessitates new mathematical approaches to proving the robustness. Enticingly, the edit distance is bounded by employing only the delete operation, making implementation and application far easier than would have otherwise been possible. Noting the inequity in attack risk for malware/security applications, effective bias terms $\\eta$ are added to allow asymmetric certificates of robustness. Since no prior work has obtained this result, they compare against a prior hamming certificate, and show significant improvement over prior results in the smaller Hamming space. \n\nThe application to malware is very apt, but this is also relevant to all the domains of which the Levenshtein edit distance is applicable: genomics, time series analysis, epidemiology, linguistics/NLP, etc.\n\nSTRENGTHS:\n1. This is allowing results in a whole new category of model/hypothesis spaces for randomized smoothing. \n2. The results show significant improvement as the maximum radius is reached over closest-prior alternatives \n3. The method is easy to implement \n4. Real-world consideration to asymmetric attack profile is considered and incorporated into the approach, while not firefighting the multi-class support. \n5. The paper is beautifully written and the best I have read in reviewing over probably the past 2 years.\n\nWEAKNESSES:\n1. The only weakness to the paper is some missing references. The asymmetric nature of benign vs malicious attacks was previously noted in \"Non-Negative Networks Against Adversarial Attacks\" and \"Adversarially Robust Malware Detection Using Monotonic Classification\", and the two should be cited accordingly. Notably, both only deal with the additive threat model $O = [ \\texttt{ins} ]$, filling in the broader contribution of this work in completing the Levenshtine operation set. \n2. If I had to add a second quibble, I would ask the authors to caution the reader about applying the % certification to other datasets, because each dataset will differ and industrial-scale classification datasets are beyond the scope of proving the mechanism's correctness. (I could see industry research getting weird reviews from academics who don't understand this when attempting to publish case-studies).\n\nQUESTIONS:\nI honestly can't think of any questions to ask beyond random directions of obviously future work. The appendix is detailed, has code, and the paper is exceptionally well written. \n\nTo the AC, please note that my shorter review isn't a remark against thoroughness in reading the work. It just answered all the questions I would have asked. This work is of exceptional quality, presentation, and impact.\n\nLIMITATIONS:\nThis work has no realistic limitations beyond not \"solving\" all problems in one go. Other challenges of randomized smoothing naturaly remain, and can't be held against the authors given the work's goals and how well it has achieved them.", "TITLE": "Official Review", "RECOMMENDATION": 9, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a general randomized smoothing approach for certifying robustness concerning arbitrary perturbations defined in Levenshtein distance. The critical challenges of proposing the randomized smoothing approach are \n  1) how to design the smoothing distribution? The paper uses a deletion distribution by randomly removing tokens in the inputs.\n  2) how to derive the bound? The paper extensively derives a loose bound using the Neyman-Pearson lemma on the proposed deletion distribution. \n\nThe approach is effective on malware detection datasets. The paper also proposes an interesting tuning mechanism to favor false positives over false negatives, as in the malware detection scenario, the escaping of malware is a bigger problem than false alarms on benign software.\n\nSTRENGTHS:\n1. The paper proposes a general randomized smoothing approach for certifying robustness concerning arbitrary perturbations defined in Levenshtein distance. The approach is effective on malware detection datasets.\n2. The paper also proposes an interesting tuning mechanism to favor false positives over false negatives, as in the malware detection scenario, the escaping of malware is a bigger problem than false alarms on benign software.  \n3. The paper also evaluates RS-Del to empirical attacks.\n\nWEAKNESSES:\n1. It is astonishing to see the classifier maintains a high certified accuracy when >90% of the inputs are deleted. I don't think it is possible to do so in NLP datasets, e.g., movie reviews datasets like SST2 and IMDB. Deleting >90% of the movie reviews definitely destroy the meaning of the reviews. \n2. missing related works:\n    * In lines 40-42, the paper states \"there is also no work for ... along with substitution or additive perturbation\". This is not the case. ARC [1] is a deterministic approach for certifying robustness of LSTMs given arbitrary perturbation spaces, including insertion, deletion, substitution, and their combinations. However, ARC can only be used for LSTMs and the certified radius is much smaller compared to RS-Del. ARC also scales linearly with respect to the certified radius while RS-Del's time complexity is constant, e.g, equal to the number of sample in Monte Carlo sampling. ARC may introduce more over-approximation (e.g. looser bound) than RS-Del due to the interval bound propagation (IBP) used in ARC. \n    * Masking [3,4] has been used as a randomized smoothing approach for certifying the robustness of NLP models with respect to word substitutions. \n3. In Table 10, RS-Del does not perform better than NS on Slack-VTFeed and GAMMA-Sleipnir2. So it is \"four out of six\" instead of \"five out of six\". However, the NS baseline is quite weak, how about comparing RS-Del to other empirical defense approaches for malware detection?\n\nQUESTIONS:\nComments:\n1. $m$ never appears outside Lemma 4. Either removing $m$ from Lemma 4 or providing intuitive relation between $m$ and $p_{del}^{|\\bar{x}|-|x|}$.\n2. In line 286, Table 7->Table 8.\n\nQuestions:\n1. Can we further improve the bound following Lee et al. [2]? In [2], the bound is to first prove that $\\forall h \\in \\mathcal{F}(x, \\mu_y)$ the minimal value of $p_y(\\bar{x};h)$ will always be achieved when $dist(\\bar{x},x)=r$, e.g., when the attacker tries to utilize the attack budget as much as possible. Then they can exactly compute Eq (13) without approximation. The case in [2] is easier since they only allow substitutions, but in this paper deletions and insertions are also allowed. However, in the proof of Theorem 7, the minimizer is achieved when $n_{ins}=n_{del}=0$, indicating that the possibility of tightening the bound as [2]. \n2. Is Corollary 6 a looser bound, e.g, LHS $\\le$ RHS instead of LHS = RHS? The proof seems to assume the substituted ones and inserted ones won't be counted in the LCS, but they potentially can be.\n3. In line 45, the paper states \"we consider input sequences of bounded and varying length\". What's \"unbounded length\"? It seems all inputs have bounded length.\n4. What's the performance of $f_b$?\n5. However, the NS baseline is quite weak, how about comparing RS-Del to other empirical defense approaches for malware detection?\n\nLIMITATIONS:\nThe paper addresses some of the limitations. For other limitations, please refer to my points in Weakness and Questions. \n\n\n[1] Certified Robustness to Programmable Transformations in LSTMs. Yuhao Zhang, Aws Albarghouthi, and Loris D'Antoni\n\n[2] Guang-He Lee, Yang Yuan, Shiyu Chang, and Tommi Jaakkola. Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers.\n\n[3] Certified Robustness to Text Adversarial Attacks by Randomized [MASK]. Jiehang Zeng, Xiaoqing Zheng, Jianhan Xu, Linyang Li, Liping Yuan, Xuanjing Huang\n\n[4] Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications. Han Cheol Moon, Shafiq Joty, Ruochen Zhao, Megh Thakkar, Xu Chi", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

comments: SUMMARY:
This paper aimed to design a certified defense for discrete sequence classifiers against edit distance-bounded adversaries. This method exploited randomized smoothing mechanism to consturct the defense and proposed RS-Del to confer robustness against adversarial delection, insertion and substitution edits.

STRENGTHS:
1. Different from most prior work, this paper focused on protecting models with discrete inputs (e.g., binary executables, source codes and PDF files), which was interesting and was meaningful in the real world.

2. The instructions for the proposed methodology were relatively clear (including the explanations of some theorems and lemmas).

WEAKNESSES:
1. Although the form of the input data was different, the defenseive mechanisms for continuous fixed-dimensional inputs (such as the methods in lines 31 to 35) may also work, but this did not seem to be adequately presented. Could the authors conduct some discussion or even comparative experiments?

2. The abbreviations needed to be supplemented with the full name when they appear for the first time (e.g., RS-Del).

QUESTIONS:
Please see weaknesses.

LIMITATIONS:
The authors have stated the limitations of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-16