SMAB : MAB based word Sensitivity Estimation Framework and its Applications in Adversarial Text Generation (2025)

Abstract: To understand the complexity of sequence classification tasks, Hahn et al. (2021) proposed sensitivity as the number of disjoint subsets of the input sequence that can each be individually changed to change the output. Though effective, calculating sensitivity at scale using this framework is costly because of exponential time complexity. Therefore, we introduce a Sensitivity-based Multi-Armed Bandit framework (SMAB), which provides a scalable approach for calculating word-level local (sentence-level) and global (aggregated) sensitivities concerning an underlying text classifier for any dataset. We establish the effectiveness of our approach through various applications. We perform a case study on CHECKLIST generated sentiment analysis dataset where we show that our algorithm indeed captures intuitively high and low-sensitive words. Through experiments on multiple tasks and languages, we show that sensitivity can serve as a proxy for accuracy in the absence of gold data. Lastly, we show that guiding perturbation prompts using sensitivity values in adversarial example generation improves attack success rate by 15.58%, whereas using sensitivity as an additional reward in adversarial paraphrase generation gives a 12.00% improvement over SOTA approaches. Warning: Contains potentially offensive content.

## introduction classifiers leveraging pre-trained language models (plms) while being empirically successful, are often opaque. identifying input subspaces where the models classify correctly (or incorrectly) or the input patterns the model is sensitive towards, is not straightforward. in the presence of model weights (white box), it may be feasible to explore such spaces but computationally intractable. on the other hand, methods attempting to explain blackbox models resort to robust diagnostic tests (such as * indicates equal contribution, order chosen at random @xcite ) at scale, which requires human input and may not be sufficient. visualization techniques such as lime @xcite , shap @xcite attempt to explain local causes towards a prediction but do not provide a global view of these models. therefore, a systematic framework is required to understand the model's weaknesses and strengths related to the input space without assuming access to model weights. @xcite proposed a theoretical framework for understanding the complexity of sequence classification tasks using sensitivity, where sensitivity is the number of disjoint subsets of the input sequence that can each be individually changed to change the output. high-sensitivity functions are complex because a single change in the input can alter the output, whereas low-sensitivity functions are simple. sensitivity helps predict the complexity of the tasks for various machine learning methods. while the proposed method effectively captures the task complexity, it requires iterating exhaustively over all possible independent subsequences in the input, resulting in a time complexity that is exponential to the number of input tokens. in this work, we extend this definition of sensitivity and propose a scalable framework based on multiarmed bandits to calculate word-level sensitivities on sequence classification tasks, without assuming access to model weights and gold labels. specifically, our framework provides an effective tool for local attributions of the model output to individual segments of the input (sentence level local sensitivities) as well as dataset-level sensitivities to specific words/phrases, which is independent of any context (global sensitivities). we introduce smab, a sensitivity-based multi-armed bandit framework, which utilizes masked language modeling (mlm) to compute word-level sensitivities in the dataset through effective explorationexploitation strategies in a multi-armed bandit figure 1 : overview of our smab framework. the outer arm consists of all words in the corpus, each linked to a set of sentences in the inner arm. w 1 is a word in the outer arm, and s w1 is the set of sentences in the inner arm that contains w 1 . g w1 t is the global sensitivity of word w 1 at step t. we utilize a sample-replace-predict strategy to estimate local sensitivity values l w1 for a word w 1 . here, p w1 is the set of predicted labels obtained after perturbing s w1 sentences and using the target classifier. s 1 is chosen randomly from p w1 while s 2 is chosen such that it has the highest reward. the local sensitivity values of a word help to update its global sensitivity values, which helps in better outer arm selection in the next time step. ## methodology we formalize the definitions of the global sensitivity of a word for a given text classifier. subsequently, we explain our proposed sensitivity estimation framework with examples. ## smab: a case study on checklist dataset. evaluating the global sensitivity values of words is challenging as they depend on a specific task and classifier combination. there is no straightforward way to determine the ground truth global sensitivity of different words. therefore, we start with a template-generated dataset such as checklist @xcite , where for a template, we know that changing specific keywords may cause a label flip while changing others should not have any effect on the label. we note that this still provides only an approximation of sensitivity values, as the global sensitivity value of a word (as defined in §2.2) also depends on the target classifier. method. the checklist framework creates targeted test cases inspired by standard software engineering practices. each test belongs to one of the categories -mft (minimum functionality test), inv (invariance test), and dir (directional expectation test). inv applies perturbations that preserve the original label, whereas dir tests if the confidence of a label changes in a specific direction. these tests (inv & dir) consist of templates that vary according to the test type in consideration. for example, change names test suite of inv test type has sentences where we only vary a single word (name) in the complete sentence. in the texts shown below, only the name (alicia) changes in the newly created example. ## sensitivity as a proxy for accuracy using our multi-armed bandit framework (in §2.3), we calculate the word sensitivities for each word in a given dataset based on the model predictions (in an unsupervised way). here, we aim to quantify the correlation between the accuracy of various models and the difference between their corresponding sensitivity distributions obtained from the smab framework. we experiment with two different settings -(i) correlation across languages (same model) and (ii) correlation within language (different models). we compare the kl divergence of the sensitivity distributions from two different models (/languages) with the relative drop in accuracy between models (languages). we compute d kl (p ∥ q) = n @xmath0, where p and q represent sensitivity distributions obtained from two different runs of smab. n represents the number of sensitivity bins (here 10). we calculate accuracy on the same dataset using ground truth labels. we hypothesize that kl divergence is negatively correlated with accuracy drop for both the settings, which signifies that the sensitivity distributions may serve as an unsupervised proxy for accuracy for a given target classifier when gold labels are absent. ## adversarial example generation 5.1 datasets we evaluate the utility of word-level sensitivities in adversarial example generation using the sentiment analysis task. we conduct experiments on the cornell movie review dataset (rt dataset) @xcite and sst-2 dataset @xcite . we extend two perturbation-based (or paraphrase) attack baselines using sensitivities predicted from our smab framework. ## related work sensitivity has been commonly used as a measure for the complexity of sequence classification tasks @xcite . sensitivity has also been used for the understanding and optimization of prompts in related paradigms such as in-context learning. lu et al. (2024) performs an analysis of accuracy and sensitivity, for different prompts in an icl setting and observes a negative correlation between them. formatspread (sclar et al., 2024) also presents a multi-armed bandit framework for a model-agnostic evaluation of performance spread across different prompt formats. multiple research work have focused on the paradigm of adversarial text generation by perturbing safe input examples through gradient-based approaches (ebrahimi et al., 2018; cheng et al., 2020). wallace et al. (2019) deployed a gradient guided search over all tokens to extract universal adversarial triggers, which are input-agnostic tokens to trigger a model. ribeiro et al. (2018) followed a similar approach by presenting simple and universal semantically equivalent adversarial rules (sears) that create adversaries on safe inputs. other approaches such as iyyer et al. (2018) and roth et al. (2024), have delved into the training of paraphrase networks for controlled generation of attack examples, whereas xu et al. (2023) proposed prompt-based adversarial attack to audit llms. ## conclusion we introduce the notion of local (sentence-level) and global (word-level) sensitivities to capture the intricacies of a text classifier for a given dataset. we introduce a novel, cost-effective sensitivity estimation framework, smab. through experiments on checklist-generated dataset, we show that our smab framework captures high-sensitive and low-sensitive words effectively. we observe that the comparative accuracy between two models (for the same language or for across language on the same task) has strong negative correlations with kl divergence between (global) sensitivity distributions of the models -showing sensitivity can be used as an unsupervised proxy for accuracy (drops). further, we define three word-level perturbation instructions utilizing the global sensitivity values obtained from the smab framework to attack llms such as gpt-3.5 with a high success rate. we also show that sensitivity can be used as an additional reward in paraphrase-based attacks to improve the attack success rate of adversarial models. hence, word-level sensitivities provide a closer look at how opaque language models work.