Question: What are the 2 types of token-level perturbations?
Evidence: Problem Definition We are given a dataset $(S,Y) = \lbrace (\mathbb {S}_1,y_1),(\mathbb {S}_2,y_2)\dots (\mathbb {S}_m,y_m)\rbrace $ and a trained classification model $C:\mathbb {S}\rightarrow Y$. We assume the soft-label black-box setting where the attacker can only query the classifier for output probabilities on a given input, and does not have access to the model parameters, gradients or training data. For an input pair $(\mathbb {S},y)$, we want to generate an adversarial example $\mathbb {S}_{adv}$ such that $C(\mathbb {S}_{adv}){\ne }y$ where $\mathbb {S}_{adv}$ is natural looking, grammatically correct and semantically similar to $\mathbb {S}$ (by some pre-defined definition
of similarity).
BAE For generating adversarial example $\mathbb {S}_{adv}$, we define two perturbations on the input $\mathbb {S}$:
Replace a token $t \in \mathbb {S}$ with another
Insert a new token $t^{\prime }$ in $\mathbb {S}$
Some tokens in the input are more attended to by $C$ than others, and therefore contribute more towards the final prediction. Replacing these tokens or inserting a new token adjacent to them can thus have a stronger effect on altering the classifier prediction. We estimate the token importance $I_i$ of each token $t_i \in \mathbb {S}=[t_1, \dots , t_n]$, by deleting $t_i$ from $\mathbb {S}$ and computing the decrease in probability of predicting the correct label $y$, similar to BIBREF11.
While the motivation for replacing tokens in decreasing order of importance is clear, we conjecture that adjacent insertions in this same order can lead to a powerful attack. This intuition stems from the fact that the inserted token changes the local context around the original token.
The Replace (R) and Insert (I) operations are performed on a token $t$ by masking it and inserting a mask token adjacent to it in $\mathbb {S}$ respectively. The pre-trained BERT masked language model (MLM) is used to predict the mask tokens (See Figure FIGREF1).
BERT is a powerful LM trained on a large training corpus ($\sim $ 2 billion words), and hence the predicted mask tokens fit well grammatically in $\mathbb {S}$. The BERT-MLM does not however guarantee semantic coherence to the original text $\mathbb {S}$ as demonstrated by the following simple example. Consider the sentence: `the food was good'. For replacing the token `good', BERT-MLM may predict the tokens `nice' and `bad', both of which fit well into the context of the sentence. However, replacing `good' with `bad' changes the original sentiment of the sentence.
To ensure semantic similarity on introducing perturbations in the input text, we filter the set of top K masked tokens (K is a pre-defined constant) predicted by BERT-MLM using a Universal Sentence Encoder (USE) BIBREF14 based sentence similarity scorer. For the R operations we add an additional check for grammatical correctness of the generated adversarial example by filtering out predicted tokens that do not form the same part of speech (POS) as the original token $t_i$ in the sentence.
To choose the token for a perturbation (R/I) that best attacks the model from the filtered set of predicted tokens:
If there are multiple tokens can cause $C$ to misclassify $\mathbb {S}$ when they replace the mask, we choose the token which makes $\mathbb {S}_{adv}$ most similar to the original $\mathbb {S}$ based on the USE score.
If no token causes misclassification, we choose the perturbation that decreases the prediction probability $P(C(\mathbb {S}_{adv}){=}y)$ the most.
The perturbations are applied iteratively to the input tokens in decreasing order of importance, until either $C(\mathbb {S}_{adv}){\ne }y$ (successful attack) or all the tokens of $\mathbb {S}$ have been perturbed (failed attack).
We present 4 attack modes for BAE based on the R and I operations, where for each token $t$ in $\mathbb {S}$:
BAE-R: Replace token $t$ (See Algorithm )
BAE-I: Insert a token to the left or right of $t$
BAE-R/I: Either replace token $t$ or insert a token to the left or right of $t$
BAE-R+I: First replace token $t$, then insert a token to the left or right of $t$
For generating an adversarial example Sadv, we introduce 2 types of token-level perturbations: (i) Replace a token t ∈ S with another and (ii) Insert a new token t 0 in S.
Answer: (i) Replace a token t ∈ S with another and (ii) Insert a new token t 0 in S.