Title: AdaptDel: Adaptable Deletion Rate Randomized Smoothing for Certified Robustness
Venue: NeurIPS 2025
Accepted: yes
Keywords: Randomized Smoothing

Certified Robustness

Adversarial Training
Abstract: We consider the problem of certified robustness for sequence classification against edit distance perturbations. Naturally occurring inputs of varying lengths (e.g., sentences in natural language processing tasks) present a challenge to current methods that employ fixed-rate deletion mechanisms and lead to suboptimal performance. To this end, we introduce AdaptDel methods with adaptable deletion rates that dynamically adjust based on input properties. We extend the theoretical framework of randomized smoothing to variable-rate deletion, ensuring sound certification with respect to edit distance. We achieve strong empirical results in natural language tasks, observing up to 30 orders of magnitude improvement to median cardinality of the certified region, over state-of-the-art certifications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe addressed topic of this work is certified robustness for deletion-based smoothing mechanism in sequence classification against edit distance perturbations. Particularly, this work extends the fixed deletion rate in previous works to an adaptive deletion rate for discrete sequences. Theoretical guarantees are established and practical algorithms are developed.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths  \n1.Certifications for deletion-based smoothing mechanism on discrete sequences are built.  \n2.There are extensive experiments, including certified robustness, hyper-parameters analysis and computational complexity.\n\nWeakness  \n1.It would be appreciated that discussions or experiments on practical settings, e.g., adversarial examples on discrete inputs, are supplemented.  \n2.Missing explanations about the robustness-accuracy trade-off.  \n\nRefer detailed weaknesses in my questions below.\n\nQUESTIONS:\nThis work overall is good, and I would like to propose several issues for a discussion.\n\n1.Existing certified defenses give rigorous theoretical guarantees but suffer from limited applicability. For example, while Gaussian noise perturbations are commonly adopted for certified robustness in image classification, such guaranteed robustness is actually insufficient against those adversarial examples. That is, the certified radius under Gaussian noises is typically much smaller than perturbation generated by basic attacks like FGSM. Regarding your deletion-based mechanism, the reviewer is curious whether your algorithm and others can be somehow effective against those more realistic discrete-input attacks (if any). Any discussions or experiments would be appreciated.\n\n2.Missing explanations about the robustness-accuracy trade-off  \nAs elaborated in Sec.4.1, there exists a robustness-accuracy trade-off for the proposed AdapDel and AdapDel+. In Table 4 of Appendix E.2, the clean accuracy of AdapDel and AdapDel+ is generally smaller than the compared methods. The authors are suggested to give a discussion on the reasons behind. Is it due to the intrinsic mechanism of AdapDel/AdapDel+, or the hyper-parameter settings? Could more sophisticated hyper-parameter tuning boost clean accuracy? Achieving competitive or even better clean accuracy is desirable and also further strengthens this work.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on a new method for certifying robustness against edit distance perturbations in sequence classification tasks using randomized smoothing. The intuition is that state-of-the-art (SOTA) solutions use a fixed-rate deletion mechanism, which has been shown to be suboptimal for real-world data where sequence lengths can vary significantly. Instead, they should use input-dependent noise when certifying robustness, as highlighted in previous work for other tasks. Thus, the paper proposes AdaptDel, a method that employs an adaptable deletion rate that dynamically adjusts based on the input length of the sequence. The paper introduces also another variant called AdaptDel+, which further optimizes this rate through input binning and empirical calibration. The experimental evaluation is comprehensive: it  includes four natural language processing datasets and shows that the proposed methods achieve up to a 30 orders of magnitude improvement in the median cardinality of the certified region and up to 50% longer certified radius. The improvements are particularly evident for longer sequences and on datasets with more uniform length variations. Finally, the paper also provides a detailed analysis of the robustness scaling with input size.\n\nSTRENGTHS AND WEAKNESSES:\nThank you to the authors for submitting this interesting paper. I find the research direction about improving randomized smoothing and the performance of the smoothed classifiers particularly interesting, especially when methods are based on in prior evidence. I believe the authors have done an excellent job in summarizing their findings in the main body of the paper, while the appendix contains interesting results and proofs that complete the work. Below, I discuss the strengths and weaknesses of this work, and I hope the authors can address my question.\n\n## Strengths\n\nThe intuition of the paper is sound: as highlighted in [A], using input-dependent noise in randomized smoothing may address better the robustness-accuracy tradeoff the smoothed classifiers. The authors extend this intuition to sequence tasks, showing that the same intuition applies.\nThe proposed method is both interesting and useful. It is based on the formal result that allows the algorithm to compute lower and upper bounds on the smoothed class probability for a neighboring input (within the edit distance threat model). They depend only on the smoothed class probability at the original input and the size of the longest common subsequence between the two inputs. This result is then formally proven (in the Appendix) and enables computing a certificate of robustness by exploiting a reduction of the problem of computing lower and upper bounds on the probability to a knapsack problem. Subsequently, the paper introduces a length-based deletion function used in the smoothing procedure that is essential for making the work novel and relevant, since it simulates how the probability of deleting a token varies with sequence length, even though more complex deletion functions could be defined as acknowledged by the authors.\n\nThe experimental evaluation is ok and considers datasets considered in previous work detailing baselines considered in this paper. Compared to previous methods, AdaptDEL achieves a significantly higher certified radius and mean cardinality of the certified region. Moreover, the additional experimental results in the Appendix clearly show the superior performance of AdaptDEL across all datasets and its greater computational efficiency compared to baselines.\nFinally, the related work section is curated and details well the differences between the proposal and close related work.\n\n## Weaknesses\n\nRelated work describing the considered baselines, such as [B], also provides evidence of the empirical robustness of the smoothed classifier. Although this is not fundamental to show, since you have already provided results about the certified radius, it would be useful to show that the probabilistic guarantee also holds empirically by considering some text attacks, as in [B].\n\n[A] Súkeník et. al., Intriguing properties of input-dependent randomized smoothing, in ICML 2022.\n\n[B] Huang et. al., CERT-ED: Certifiably Robust Text Classification for Edit Distance, in ACL 2024.\n\nQUESTIONS:\nHow do the probabilistic guarantees of the smoothed classifier hold empirically against text attacks? (you can use the methodology used in [B]).\n\n[B] Huang et. al., CERT-ED: Certifiably Robust Text Classification for Edit Distance, in ACL 2024.\n\nLIMITATIONS:\nyes, the authors have appropriately discussed the limitations of their work throughout the text and in an apposite section in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extend deletion-based randomized smoothing from a fixed token deletion rate to a length adaptive rate. The authors derived a new probabilistic bound for adaptive deletion and proposed two variants AdaptDel (analytical length dependent) and AdaptDel+ (data driven bin calibration). The paper studies an interesting problem about how to model discrete certified robustness problem and make the method adaptive.\n\nSTRENGTHS AND WEAKNESSES:\n- **Motivation**. Different from most prior work in image space, this work studies discrete text data for certified robustness. It is interesting and  well motivated by the real world needs. I feel this is meaningful to the robustness community. Can you provide some explanations with maybe a toy example of why typical continuous RS does not work?\n\n- **Theory**. The theory part is sound with bounds for arbitrary $\\psi(x)$. \n\n(1) The operations o contains three elements del, ins, sub, but section directly jump to del only, can you briefly explain why the other two operations are dropped? \n\n(2) For line 143, how can you make apply($\\bar{x},\\bar{\\epsilon}$)=apply($\\bar{x},\\epsilon^\\star$)? Is there a typo? Do you mean apply($\\bar{x},\\bar{\\epsilon}$)=apply($x,\\epsilon^\\star$)? \n\n(3) Can you explain how you define $\\rho$ intuitively? Also it is not trivial to see the equivalence of lemma 3.1 from lemma4 from [1], can you elaborate a bit?\n\n(4) If I understand correctly, the bound is derived from greedy algorithm of knapsack problem relaxation. Can you explain your bound tightness? It might be helpful to discuss it a bit for future better schedule $\\psi$.\n\n(5) Notation-wise (minor), $m$ is used to define bijection in lemma 3.1 and also define the num of trails in lemma 3.2. \n\n- **Experiments**. The paper performed experiments on 4 diverse datasets and compared with two baselines CERT-ED and RanMASK. The computational cost is also included. \n\n(1) In the LUN setup, AdapDel is not better than CERT-ED at small radius. And when radius is small, the improvement is also marginal in IMDB and SpamAssassin. Can you explain why?\n\n(2) How does AdaptDel behave for extremely short inputs where deletion can harm semantics much?\n\n[1] Huang et al. 2024, RS-Del: Edit Distance Robustness Certificates for Sequence Classifiers via Randomized Deletion\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes AdaptDel and AdaptDel+, two randomized smoothing methods for certified robustness against edit distance perturbations in sequence classification tasks. Unlike prior work that uses a fixed deletion rate (e.g., CERT-ED), AdaptDel dynamically adjusts the deletion probability based on input length, and AdaptDel+ further calibrates this adaptively via binning and empirical optimization. The authors provide a theoretical framework to support edit-distance certification with input-dependent deletion rates and demonstrate empirical improvements over existing baselines on several NLP datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper extends deletion-based randomized smoothing by introducing input-dependent deletion rates, enabling more flexible certification against edit-distance perturbations. The proposed methods, AdaptDel and AdaptDel+, are supported by new theoretical bounds and demonstrate improved certified robustness compared to prior baselines, such as CERT-ED and RanMASK, across several NLP tasks. The knapsack-based formulation and calibration strategy in AdaptDel+ are technically sound and contribute to improvements in robustness, especially for longer sequences.\n\nHowever, the method is a relatively straightforward extension of CERT-ED and remains limited to perturbations that only involve deletion. The theoretical analysis, while correct, is dense and challenging to interpret, lacking an intuitive explanation. AdaptDel+ introduces additional complexity through its calibration process, yet the paper provides limited discussion on its computational cost or reproducibility. Overall, the novelty and clarity are limited, and the scope is narrow.\n\nQUESTIONS:\n- How computationally intensive is the AdaptDel+ calibration process, especially in scenarios with many bins or limited training data?\n- Could the proposed approach be extended to certify robustness under general edit operations, including insertions and substitutions?\n- Given the difficulty of interpreting the derived bounds, can the authors provide more intuition or visualization for how the adaptive deletion affects robustness?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}