Maximum Entropy Loss, the Silver Bullet Targeting Backdoor Attacks in Pre-trained Language Models (2023)

Abstract: Pre-trained language model (PLM) can be stealthily misled to target outputs by backdoor attacks when encountering poisoned samples, without performance degradation on clean samples. The stealthiness of backdoor attacks is commonly attained through minimal cross-entropy loss fine-tuning on a union of poisoned and clean samples. Existing defense paradigms provide a workaround by detecting and removing poisoned samples at pre-training or inference time. On the contrary, we provide a new perspective where the backdoor attack is directly reversed. Specifically, maximum entropy loss is incorporated in training to neutralize the minimal cross-entropy loss fine-tuning on poisoned data. We defend against a range of backdoor attacks on classification tasks and significantly lower the attack success rate. In extension, we explore the relationship between intended backdoor attacks and unintended dataset bias, and demonstrate the feasibility of the maximum entropy principle in de-biasing.

## introduction in recent years, pre-trained language models (plms) have been widely used in various natural language processing tasks attributing to their superior performance @xcite . considering the extensive requirements in data and computation, the pre-training process of plms are generally implemented by third party companies and organizations @xcite . however, backdoors are likely to be injected into the plm if users or third parties are not in a secure condition @xcite . specifically, the attacker first converts a small proportion of clean data to poisoned data by injecting a trigger (e.g., rare fixed tokens @xcite ). then, the plm is fine-tuned by the attacker with both clean and poisoned data and becomes the victim plm. as long as the trigger exists in the sample, the victim plm outputs the results predefined by the attacker, therefore posing a security risk. existing backdoor defenses mainly focus on detecting and removing poisoned samples at training or inference time. training time defense requires that all samples are monitored and poisoned samples are removed @xcite . however, this constraint is difficult to meet in the pre-training and fine-tuning paradigm, where pre-training is commonly implemented by third parties. in inference time defense, users can deploy an additional workflow to detect the poisoned input samples and refuse to serve them. however, the detection of poisoned samples is complex as the triggers chosen by the attacker are unknown @xcite @xcite . such a defense incurs additional computational costs during inference and would falsely refuse innocent samples @xcite . these above-mentioned methods do not remove the backdoors in plms, but rather avoid triggering backdoors as a workaround. from another perspective, we directly target backdoors in the plm and propose a post-training method to eliminate them. we observe that although the trigger varies, backdoor attacks invariably introduce a distribution gap between the pre-trained and victim model. specifically, fine-tuning of attackers distorts pre-trained features @xcite , i.e., the features of poisoned samples are alternated while those of normal samples are mainly preserved. in view of this, we propose to reverse the minimum cross-entropy loss fine-tuning of attackers with maximum entropy loss on clean data. we also propose a metric called stop distance to ensure that backdoors are eliminated from the model. figure 1 illustrates the rationale for how our approach works. a victim plm maps poisoned samples closer to target labeled samples as shown in figure 1a . the defender can train the victim model with maximum entropy loss on clean data. such training mixes up the representations of target and non-target labeled samples, along with the poisoned samples, as shown in figure 1b . the training with maximum entropy loss is controlled by the stop distance we propose and stops when the vector representations of differently labeled samples are close enough. the final training on the clean dataset brings the plm back to normal, where the poisoned samples are close to the non-target labeled samples and away from target labeled samples, as shown in figure 1c . we utilize the proposed method to defend against various backdoor attacks in pre-training and finetuning paradigm. our method has a significant advantage over baseline defending methods. also, a lite version of our method with a larger stop distance and much less computation achieves on-par performance to the baselines. the results indicate that our method is both effective and flexible. further, we analyze the possible relationship between backdoor attacks and dataset bias, and demonstrate that maximum entropy can also be effective as a regular term for de-biasing. ## preliminaries backdoor attacks happen at model training, posing security risks to transfer learning and outsourcing training scenarios of the pre-training and finetuning paradigm. in transfer learning scenario, the attacker injects backdoors into a plm through finetuning and gets the user to download it through a network attack. then the user fine-tunes it as a text encoder on downstream tasks. it is worth mentioning that backdoors cannot be eliminated by such a standard fine-tuning @xcite . in outsourcing training scenario, the entire model training process is implemented by a third party. the attacker can inject backdoors directly while fine-tuning plms on downstream tasks. the following is a formal representation of the backdoor attack based fine-tuning. in fine-tuning, a clean dataset d with text sample x and the corresponding label y is used to train a text classification model f θ : @xmath0, where x is the input space and y is the output space. backdoor attackers will divide the dataset into two parts, the candidate poison set d p and the clean set d c . the samples in d p are refactored by function g( * ) to become poisoned samples embedded with triggers, and the labels of these samples are tampered with the attack target label. the attackers can then get a poisoned set d * p , with poisoned sample x * = g(x) and attack target label y * = y t . finally, the victim model f θ * is trained to convergence on @xmath1. during inference, f θ * will behave properly on clean samples but produce the target label on poisoned samples. two metrics are generally applied to evaluate backdoor attacks, namely classification accuracy (acc) and attack success rate (asr) @xcite . acc is the classification accuracy of the victim model on a clean data set. asr is the proportion of poisoned samples that are misclassified as the target label. in this paper, we poison all non-target labeled samples in the test set to calculate asr. we use both metrics to evaluate the effectiveness of defense methods. the plm is often considered as an encoder in text classification tasks. thus, the changes of a plm during backdoor attacks can be reflected in its encoded representations and performance in classification. in terms of vector representations, the samples can be divided into three groups: samples with the target label, samples with non-target labels, and poisoned samples. we observed the dissimilarity between the vector representations of different groups and the same group, respectively. in terms of classification performance, we observed the changes of plms by acc and asr. specifically, we conduct visualization experiments with badnets @xcite as shown in figure 2 . we randomly insert a fixed token (randomly selected from "cf", "mn", "bb", "tq", and "mb") into each of 10% clean samples in sst-2 training set @xcite . these samples are then mixed with the remaining 90% samples and are used to fine-tune the uncased bert base @xcite . we take the [cls] token of bert as the vector representation of a sample, denoted as h. the centroid of a group of n vector representations is 1 n n @xmath2. the euclidean distance between centroids measures the between-group dissimilarity, and the within-group dissimilarity is measured by averaging the variance of each dimension. the between-group dissimilarity is shown in figure 2a . it can be observed that the distance between different labeled samples gradually increases, which implies the improvement of the classification ability. the distance between poisoned samples and target labeled samples gradually decreases, indicating the establishment of an association between the backdoor feature and the target label. acc and asr of the victim model are shown in figure 2b . badnets boosts asr to nearly 100% without affecting acc and the convergence of asr is later than acc. figure 2c plots the within-group dissimilarity. according to linear discriminant analysis (lda), the samples of the group with small within-group variance appeal to linear classifier.it can be found that the vector representations of the poisoned samples has a small variance after convergence, bringing an extremely high asr. the distortion to plms caused by fine-tuning with poisoned data can be clearly seen from the experiments. backdoor attacks enlarge the distribution gap between the pre-trained and victim model sharply. our method focuses on the elimination of this gap to defend against backdoor attacks. ## conclusion in this paper, we propose a simple and powerful backdoor elimination method for plms. by fine-tuning plms with maximum entropy loss, our method can effectively revert the backdoor attacks in plms. our method essentially eliminates the backdoors from the perspective of the model and provides a new defense against backdoor attacks. we also analyze the relationship between backdoor attacks and dataset bias, which is beneficial for further understanding of both.