Exploiting Class Probabilities for Black-box Sentence-level Attacks (2024)

Abstract: Sentence-level attacks craft adversarial sentences that are synonymous with correctly-classified sentences but are misclassified by the text classifiers. Under the black-box setting, classifiers are only accessible through their feedback to queried inputs, which is predominately available in the form of class probabilities. Even though utilizing class probabilities results in stronger attacks, due to the challenges of using them for sentence-level attacks, existing attacks use either no feedback or only the class labels. Overcoming the challenges, we develop a novel algorithm that uses class probabilities for black-box sentence-level attacks, investigate the effectiveness of using class probabilities on the attack’s success, and examine the question if it is worthy or practical to use class probabilities by black-box sentence-level attacks. We conduct extensive evaluations of the proposed attack comparing with the baselines across various classifiers and benchmark datasets.

## introduction despite the tremendous success of text classification models @xcite , studies have exposed their susceptibility to adversarial examples, i.e., carefully crafted sentences with human-unrecognizable changes to the inputs that are misclassified by the classifiers @xcite . adversarial attacks provide profound insights into the classifiers' brittleness and are key to reinforcing their robustness and reliability. adversarial attacks on texts are broadly categorized into two types, namely word-level and sentence-level attacks. word-level attacks manipulate the words in the original sentences to examine the text classifiers' sensitivity to the choice of words in sentences @xcite @xcite . sentencelevel attacks, on the other hand, craft synonymous sentences with the original correctly-classified inputs, such that they are misclassified by classifiers. depending on the information available to the adversary, the attacks are conducted under the whitebox or black-box settings. unlike the white-box setting, where the classifier is completely known, and the adversary uses its gradients to craft adversarial examples @xcite , black-box attacks can only access the classifier feedback to queries. having no prior knowledge of the classifier, this setting is more feasible for real-world applications. under the black-box setting, three types of classifier feedback exist: (1) no feedback (blind setting): classifiers deny any feedback to the adversaries; (2) class label feedback (decision-based setting): classifiers return their final decisions in the forms of the predicted class labels; and (3) class probability feedback (score-based setting): classifiers return the class probabilities as feedback in response to queries. among these settings, the score-based is the most prevalent setting in real-world applications. for instance, microsoft azure foot_0 and meta-mind foot_1 are two widely-used real-world online text classification models that are deployed under the score-based setting and return class probabilities. when available, class probabilities provide richer information compared to no feedback or solely the class labels, which can better guide the adversarial example generation and result in stronger attacks. this is also demonstrated by the success of scorebased word-level attacks @xcite compared to their blind @xcite or decisionbased counterparts @xcite . moreover, developing score-based blackbox sentence-level attacks is a critical step toward identifying the extent of the threat to the text classification models to better immunize them to attacks in all black-box settings. therefore, studying such attacks is of great importance. existing black-box sentence-level attacks either do not use the feedback (blind) @xcite or only use the class labels (decision-based) @xcite , hence do not fully exploit the class probability feedback available under the most prevalent score-based setting. this is because utilizing the classifier's class probabilities available under the score-based settings for black-box sentencelevel attacks faces the following challenges: (i) defining the search space. in a score-based setting, an ideal search space is a continuous explorable space that represents the sentence-level candidates and how the transition from one candidate to another can be made using the classifier's class probabilities. existing sentence-level search spaces based on paraphrase generation @xcite or generative adversarial networks @xcite that are developed for blind or decision-based settings are discrete, i.e., they only generate sentence-level adversarial candidates with undefined relationships. these search spaces are therefore not appropriate for the scorebased setting; and (ii) developing a score-based search method. in black-box settings, a successful attack needs to fully exploit the classifier feedback to guide exploring the search space. existing search methods used for sentence-level attacks are heuristic iterative methods. these methods only accept/reject the adversarial example candidates based on their returned class labels (misclassified or not) @xcite and do not use the class probabilities, as required by the score-based setting. for the score-based sentence-level attacks, we need a search method that uses class probabilities. subduing these challenges, we propose the first score-based black-box sentence-level attack that models the candidate distributions of adversarial sentences, which transforms the problem to search over the continuous parameter space of these distributions instead of the discrete space of synonymous sentences with undefined relationships. it then searches for the optimal parameters of the actual adversarial distribution using the black-box classifier's class probabilities. to evaluate our framework, we conduct extensive experiments on three text classification classifiers across three benchmark datasets. our contributions are summarized as follows: ## related work word-level attacks. these attacks alter certain words in the original sentences to get them misclassified by the classifier. the search space in these attacks consists of adversarial candidates generated by applying transformations to the words in a sentence. to form these search spaces, various word replacement strategies such as context-free @xcite @xcite and context-aware @xcite ) approaches have been proposed. for the search method, these attacks mainly rely on methods that are designed to deal with their discrete word-level search spaces such as word ranking-based methods @xcite @xcite @xcite , or combinatorial optimization based methods like gradient-free population-based optimization @xcite , or particle swarm optimization @xcite . these attacks focus on a different granularity of the attack compared to the attack studied in this paper. sentence-level attacks sentence-level attacks generate adversarial paraphrases of the original sentences that are misclassified by the classifier. under the white-box setting, where the adversary has complete access to classifiers, these attacks adopt the classifier's gradients for the attack generation @xcite @xcite . under the more realistic black-box setting, where only the classifier's feedback to queries is accessible, these attacks are categorized into three: (i) blind attacks, which do not utilize the classifier feedback and use the paraphrases of the original sentences as adversarial examples @xcite ; (ii) decision-target encoder decoder … … dbert update "$" (nes optimization) target class probabilities semantic similarity constraint (" ! ) (" " ) adversarial perturbation distribution figure 1: an overview of the s2b2-attack. s2b2-attack perturbs the original latent variable distributions to model the search space of candidate distributions of adversarial examples using vae and learns the parameters of the actual adversarial distribution using the nes search based on the classifier's class probabilities. ## experiments we conduct comprehensive experiments to evaluate the effectiveness of s2b2-attack. our experiments center around three main questions: (i) does utilizing the class probabilities improve the success rates of sentence-level attacks? (ii) how does each component of the s2b2-attack contribute to its performance (ablation study)? and (iii) are examples generated by s2b2-attack grammatically correct and fluent? we present some adversarial samples generated by s2b2-attack in the appendix. ## conclusion as demonstrated by our experiments leveraging class probabilities significantly improves the success rates of sentence-level attacks, as our s2b2-attack achieves approximately 15% of improvement over the state-of-the-art decision-based attack (