SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Han Liu
Dalian University of Technology
liu.han.dut@gmail.com
Zhi Xu
xu.zhi.dut@gmail.com
Xiaotong Zhang*
zxt.dut@hotmail.com
Feng Zhang
zfeng.maria@gmail.com
Fenglong Ma
Pennsylvania, USA
fenglong@psu.edu
Hongyang Chen
Zhejiang Lab
dr.h.chen@ieee.org
Hong Yu
hongyu@dlut.edu.cn
Xianchao Zhang*
xczhang@dlut.edu.cn
Black-box hard-label adversarial attack on text is a practical and challenging task, as the text data space is inherently discrete and non-differentiable, and only the predicted label is accessible. Research on this problem is still in the embryonic stage and only a few methods are available. Nevertheless, existing methods rely on the complex heuristic algorithm or unreliable gradient estimation strategy, which probably fall into the local optimum and inevitably consume numerous queries, thus are difficult to craft satisfactory adversarial examples with high semantic similarity and low perturbation rate in a limited query budget. To alleviate above issues, we propose a simple yet effective framework to generate high quality textual adversarial examples under the black-box hard-label attack scenarios, named HQA-Attack. Specifically, after initializing an adversarial example randomly, HQA-attack first constantly substitutes original words back as many as possible, thus shrinking the perturbation rate. Then it leverages the synonym set of the remaining changed words to further optimize the adversarial example with the direction which can improve the semantic similarity and satisfy the adversarial condition simultaneously. In addition, during the optimizing procedure, it searches a transition synonym word for each changed word, thus avoiding traversing the whole synonym set and reducing the query number to some extent. Extensive experimental results on five text classification datasets, three natural language inference datasets and two real-world APIs have shown that the proposed HQA-Attack method outperforms other strong baselines significantly.
Deep neural networks (DNNs) have achieved a tremendous success and are extremely popular in various domains, such as computer vision [34, 35], natural language processing [31, 15], robotics [2, 26] and so on. In spite of the promising performance achieved by DNN models, there are some concerns around their robustness, as evidence shows that even a slight perturbation to the input data can fool these models into producing wrong predictions [10, 19, 11, 5], and these perturbed examples are named as adversarial examples. Investigating the generation rationale behind adversarial examples seems a promising way to improve the robustness of neural networks, which motivates the research about adversarial attack. Most existing adversarial attack methods focus on computer vision [32, 36, 39] and have been well explored. However, the adversarial attack on text data is still challenging, as not only the text data space is intrinsically discrete and non-differentiable, but also changing words slightly may affect the fluency in grammar and the consistency in semantics seriously.
Based on the accessibility level of victim models, existing textual adversarial attack methods can be categorized into white-box attacks [9, 25, 7] and black-box attacks [19, 41, 20]. For white-box attacks, the attackers are assumed to have full information about the victim model, including training data, model architecture and parameters. Therefore, it is easy to formulate this type of attack as an optimization problem and utilize the gradient information to generate adversarial examples. However, as most model developers are impossible to release all the model and data information, white-box attacks seem excessively idealistic and cannot work well in real-world applications. For black-box attacks, the attackers are assumed to have only access to the predicted results, e.g., confidence scores or predicted labels, which seem more realistic. Existing black-box textual attack methods can be divided into soft-label setting [20, 6, 14] and hard-label setting [23, 38, 37]. For soft-label methods, they require the victim model to provide the confidence scores to calculate the importance of each word, and then replace words sequentially until an adversarial example is generated. However, it is also impractical as most real APIs do not allow users to access the confidence scores. For hard-label methods, they only need to know the predicted labels of the victim model to fulfill the attack task, thus are more practicable and promising.
Only a handful of methods are proposed to deal with the black-box hard-label textual adversarial attack task, which mainly rely on heuristic-based [23] or gradient-based strategies [38, 37]. Specifically, HLGA [23] is the first hard-label adversarial attack method, which leverages population-based heuristic optimization algorithm to craft plausible and semantically similar adversarial examples. However, as it requires a large population of adversarial candidates and population-based optimization strategy is easy to fall into the local optimum, HLGA inevitably consumes numerous query numbers. TextHoaxter [38] formulates the budgeted hard-label adversarial attack task on text data as a gradient-based optimization problem of perturbation matrix in the continuous word embedding space. LeapAttack [37] utilizes the gradient-based optimization by designing a novel mecha
Figure 1: The overview of HQA-Attack. (A) Generate an adversarial example  $x_1'$  by initialization. (B) Update  $x_t'(1 \leqslant t \leqslant T)$  by substituting original words back. (C) Obtain  $x_{t+1}'$  by optimizing the adversarial example. The similarity ishopyse is a line which consists of points having the equal semantic similarity with the original example  $x$ .
nism that can interchange discrete substitutions and continuous vectors. Although these gradient-based methods improve the query efficiency to some extent, they still need some unnecessary queries caused by inaccurate gradient estimation. Furthermore, in tight budget scenarios, query inefficiency will directly bring about serious side effects on the semantic similarity and perturbation rate.
To alleviate the above issues, we propose a simple yet effective framework for producing High Quality black-box hard-label Adversarial Attack, named HQA-Attack. The overview of HQA-Attack is shown in Figure 1. By "high quality", it means that the HQA-Attack method can generate adversarial examples with high semantic similarity and low perturbation rate under a tight query budget. Specifically, HQA-Attack first generates an adversarial example by initialization, and then
sequentially substitutes original words back as many as possible, thus shrinking the perturbation rate. Finally, it utilizes the synonym set to further optimize the adversarial example with the direction which can improve the semantic similarity and satisfy the adversarial condition simultaneously. In addition, to avoid going through the synonym set, it finds a transition synonym word for each changed word, thus reducing the query number to a certain extent. Experimental results on eight public benchmark datasets and two real-word APIs (Google Cloud and Alibaba Cloud) have demonstrated that HQA-Attack performs better than other strong baselines in the semantic similarity and perturbation rate under the same query budget. The source code and demo are publicly available at https://github.com/HQA-Attack/HQAArtack-demo.
2.1 Soft-Label Textual Adversarial Attack
Adversarial attacks in soft-label settings rely on the probability distribution of all categories to generate adversarial examples. A series of strategies [16, 21, 24] utilize the greedy algorithm to craft adversarial examples, which first determine the word replacement order and greedily replace each word under this order. TextFooler [16] first determines the word replacement order according to the prediction change after deleting each word, and then replaces words back according to the word importance until adversarial examples are generated. Similarly, TextBugger [21] calculates the importance of sentences and the importance of words respectively by comparing the prediction before and after removing them. In addition, there are some methods [40, 1] which use combinatorial optimization algorithm to generate adversarial examples.
2.2 Hard-Label Textual Adversarial Attack
Adversarial attacks in hard-label settings only allow to access the predicted label, which seem more challenging and practical. HLGA [23] is the first hard-label textual adversarial attack method. It uses random initialization and search space reduction to get an incipient adversarial example, and then uses the genetic algorithm including mutation, selection and crossover three operations to further optimize the adversarial example. Although HLGA can generate adversarial examples with high semantic similarity with the original example and low perturbation rate, it needs to maintain a large candidate set in each iteration, which wastes a large number of queries. To alleviate this issue, TextHoaxter [38] uses the word embedding space to represent the text, and introduces a perturbation matrix and a novel objective function which consists of a semantic similarity term, a pair-wise perturbation constraint and a sparsity constraint. By using the gradient-based strategy to optimize the perturbation matrix, TextHoaxter can generate appropriate adversarial examples in a tight budget. LeapAttack [37] is another gradient-based method. After random initialization, it optimizes the adversarial example by constantly moving the example closer to the decision boundary, estimating the gradient and finding the proper words to update the adversarial example. In Appendix K, we further discuss some potential application scenarios about hard-label textual adversarial attack.
In this paper, we focus on the task of black-box hard-label textual adversarial attack, i.e., attackers can only access to the predicted label from the victim model to generate adversarial examples. Specifically, given an original example  $x = [w_{1}, w_{2}, \dots, w_{n}]$  with the ground truth label  $y$ , where  $w_{i}$  is the  $i$ -th word, and  $n$  is the total number of words in  $x$ . This task aims to construct an adversarial example  $x' = [w_{1}', w_{2}', \dots, w_{n}']$  through replacing the original
Table 1: Symbol explanation.
Symbol	Explanation
x	the original example x = [w1, w2, ..., wn]
x'	the adversarial example x' = [w1', w2', ..., wn']
xt'	the adversarial example in the t-th step
wi	the transition word associated with wi
u	the updating direction
vwi	the word vector of wi
f	the victim model
y	the true label of x
S(wi)	the synonym set of wi
x'(wi)	the example after replacing the i-th word of x' with wi
Sim(·,·)	the similarity function between sentences
word  $w_{i}$  with a synonym  $w_{i}^{\prime}$  in the synonym set  $S(w_{i})$  which includes  $w_{i}$  itself, thus misleading the victim model  $f$  to output an incorrect prediction result:
$$
f \left(x ^ {\prime}\right) \neq f (x) = y, \tag {1}
$$
where Eq. (1) can be seen as the adversarial condition. There may exist several adversarial examples which can satisfy Eq. (1), but an optimal adversarial example  $x^{*}$  is the one that has the highest semantic similarity with the original example  $x$  among all the candidates. Formally,
$$
x ^ {*} = \underset {x ^ {\prime}} {\arg \max } S i m \left(x, x ^ {\prime}\right), \text {s . t .} f \left(x ^ {\prime}\right) \neq f (x), \tag {2}
$$
where  $\operatorname{Sim}(x, x')$  is the semantic similarity between  $x$  and  $x'$ . Table 1 summarizes the symbol explanation in detail.
4 The Proposed Strategy
4.1 Initialization
To fulfill the black-box hard-label textual adversarial attack task, we follow previous works [23, 38, 37] to first utilize random initialization to generate an adversarial example. Specifically, considering each word  $w_{i}$  whose part-of-speech (POS) is noun, verb, adverb and adjective in the original example  $x$ , we randomly select a synonym  $w_{i}^{\prime}$  from the synonym set  $S(w_{i})$  of  $w_{i}$  as the substitution of  $w_{i}$ , and repeat this procedure until the generated adversarial example  $x^{\prime}$  satisfies the adversarial condition.
Obviously, using random initialization to generate the adversarial example usually needs to change multiple words in the original example, thus necessarily leading to the low semantic similarity and large perturbation rate. To alleviate the above issue, we attempt to iteratively optimize the semantic similarity between the original example  $x$  and the adversarial example in the  $t$ -th iteration  $x_{t}^{\prime}$ , where  $1 \leqslant t \leqslant T$  and  $T$  is the total number of iterations. Furthermore, given an adversarial example  $x_{t}^{\prime}$ , we can generate the adversarial example  $x_{t+1}^{\prime}$  by the following steps. (1) Substituting original words back; (2) Optimizing the adversarial example.
4.2 Substituting Original Words Back
In order to improve the semantic similarity of the generated adversarial example, previous works [23, 38, 37] first use the semantic similarity improvement brought by each original word as a measure to decide the replacement order, and then continually put original words back in each iteration. It means that in each iteration, by going through each word in the adversarial example, the semantic similarity improvement is only calculated once, and the replacement order is determined beforehand. However, making each replacement with the original word will change the intermediate generated adversarial example, thereby affecting the semantic similarity between the original example and the intermediate generated adversarial example, so just calculating the semantic similarity improvement at the beginning of each iteration to decide the replacement order is inaccurate.
To address the above problem, we propose to constantly substitute the original word which can make the intermediate generated adversarial example have the highest semantic similarity with the original example until the adversarial condition is violated. Specifically, given the original sample  $x = [w_{1}, w_{2}, \dots, w_{n}]$  and the adversarial example  $x_{t}^{\prime} = [w_{1}^{\prime}, w_{2}^{\prime}, \dots, w_{n}^{\prime}]$  in the  $t$ -th iteration, we can substitute original words back with the following steps.
1. Picking out an appropriate substitution word  $w_{*}$  from  $x$  with the following formula:
$$
w _ {*} = \underset {w _ {i} \in x} {\arg \max } S i m \left(x, x _ {t} ^ {\prime} \left(w _ {i}\right)\right) \cdot \mathcal {C} \left(f, x, x _ {t} ^ {\prime} \left(w _ {i}\right)\right), \tag {3}
$$
where  $Sim(x, x_t'(w_i))$  is the semantic similarity between  $x$  and  $x_t'(w_i)$ , and  $x_t'(w_i)$  is the example obtained by substituting the corresponding word  $w_i'$  with  $w_i$ .  $\mathcal{C}(f, x, x_t'(w_i))$  is a two-valued function defined as:
$$
\mathcal {C} \left(f, x, x _ {t} ^ {\prime} \left(w _ {i}\right)\right) = \left\{ \begin{array}{l l} 1, & f (x) \neq f \left(x _ {t} ^ {\prime} \left(w _ {i}\right)\right) \\ 0, & f (x) = f \left(x _ {t} ^ {\prime} \left(w _ {i}\right)\right) \end{array} , \right. \tag {4}
$$
where  $f$  denotes the victim model.  $\mathcal{C}$  equals to 1 if  $f(x) \neq f(x_t'(w_i))$ , and 0 otherwise.
2. If  $\mathcal{C}(f, x, x_t'(w_*)) = 1$ , it indicates that  $x_t'(w_*)$  can attack successfully. We substitute the corresponding original word in  $x_t'$  with  $w_*$ , and repeat the above step.
3. If  $\mathcal{C}(f, x, x_t'(w_*)) = 0$ , it indicates that  $x_t'(w_*)$  cannot satisfy the adversarial condition. We terminate the swapping procedure, and return the result of the previous step.
After the above procedure, we can obtain a new adversarial example  $x_{t}^{\prime}$  which can retain the original words as many as possible, thus improving the semantic similarity and reducing the perturbation rate. The algorithm procedure is shown in Appendix A, and the analysis of computational complexity and query numbers is shown in Appendix C.1.
4.3 Optimizing the Adversarial Example
To further ameliorate the quality of the generated adversarial example, we optimize the adversarial example by leveraging the synonym set of each word. One may argue that we could directly traverse the synonym set to seek out the most ideal synonym word which has the highest semantic similarity and satisfies the adversarial condition simultaneously. However, in most real-world application scenarios, the query number is usually limited. To avoid going through the synonym set, we propose to optimize the adversarial example with the following two steps. (1) Determining the optimizing order; (2) Updating the adversarial example sequentially. The analysis of computational complexity and query numbers is shown in Appendix C.2.
4.3.1 Determining the Optimizing Order
In this step, we aim to determine a suitable optimizing order. To ensure the diversity of the generated adversarial example, we utilize the sampling method to determine the optimizing order. The probability distribution used by the sampling method is generated as follows. For  $w_{i}^{\prime}$  in  $x_{t}^{\prime}$  and  $w_{i}$  in  $x$ , we first use the counter-fitting word vectors [27] to obtain their corresponding word vectors, and calculate the cosine distance between them as follows:
$$
d _ {i} = 1 - \cos \left(\boldsymbol {v} _ {w _ {i}}, \boldsymbol {v} _ {w _ {i} ^ {\prime}}\right), \tag {5}
$$
where  $\pmb{v}_{w_i}$  and  $\pmb{v}_{w_i'}$  denote the word vectors of  $w_i$  and  $w_i'$  respectively.  $\cos(\cdot, \cdot)$  is the cosine similarity function. Then we compute the probability  $p_i$  associated with the position of  $w_i$  in  $x$  with the following formula:
$$
p _ {i} = \frac {2 - d _ {i}}{\sum_ {j = 1} ^ {m} \left(2 - d _ {j}\right)}, \tag {6}
$$
where  $m$  is the total number of changed words between  $x_{t}^{\prime}$  and  $x$ . According to the probability distribution, we can obtain the optimizing order for  $x_{t}^{\prime}$ .
4.3.2 Updating the Adversarial Example Sequentially
According to the optimizing order, we update the adversarial example with the synonym set sequentially. In particular, for the adversarial example  $x_{t}^{\prime}$  in the  $t$ -th iteration, we update it with the following steps. (1) Finding the transition word; (2) Estimating the updating direction; (3) Updating the adversarial example.
Finding the transition word. This step aims to search a reasonable transition word, thus avoiding traversing the synonym set for each changed word. Given the adversarial example  $x_{t}^{\prime} = [w_{1}^{\prime}, w_{2}^{\prime}, \dots, w_{n}^{\prime}]$  and the current optimized word  $w_{i}^{\prime}$ , we randomly select  $r$  synonyms from  $S(w_{i})$  to construct the set  $R = \{w_{i}^{(1)}, w_{i}^{(2)}, \dots, w_{i}^{(r)}\}$ , use each element in  $R$  to replace  $w_{i}^{\prime}$  in  $x_{t}^{\prime}$ , and then obtain the transition word  $\bar{w}_{i}$  with the following formula:
$$
\bar {w} _ {i} = \underset {w _ {i} ^ {(j)} \in R} {\arg \max } S i m \left(x, x _ {t} ^ {\prime} \left(w _ {i} ^ {(j)}\right)\right) \cdot \mathcal {C} \left(f, x, x _ {t} ^ {\prime} \left(w _ {i} ^ {(j)}\right)\right), \tag {7}
$$
where  $x_{t}^{\prime}(w_{i}^{(j)})$  is the example obtained by substituting the corresponding word  $w_{i}^{\prime}$  in  $x_{t}^{\prime}$  with  $w_{i}^{(j)} \in R$ . According to Eq. (7), we can get that  $\bar{w}_{i}$  can make the example adversarial and improve the semantic similarity to some extent, while avoiding going through the synonym set. Furthermore, we can search other possible replacement words around the transition word  $\bar{w}_{i}$ .
Estimating the updating direction. As the transition word  $\bar{w}_i$  originates from a randomly generated synonym set, we can further optimize it with a reasonable direction. Specifically, we first generate the set  $\mathcal{K} = \{\bar{w}_i^{(1)},\bar{w}_i^{(2)}\ldots ,\bar{w}_i^{(k)}\}$  by randomly sampling  $k$  synonyms from  $S(\bar{w}_i)$ , and then obtain the set  $\mathcal{M} = \{x_t'(\bar{w}_i^{(1)}),x_t'(\bar{w}_i^{(2)}),\ldots ,x_t'(\bar{w}_i^{(k)})\}$ , where  $x_{t}^{\prime}(\bar{w}_{i}^{(j)})$  is the example by replacing  $w_{i}^{\prime}$  in  $x_{t}^{\prime}$  with  $\bar{w}_i^{(j)}$ . By calculating the semantic similarity between each element in  $\mathcal{M}$  and the original text  $x$ , we can get the set  $S = \{s^{(1)},s^{(2)},\dots,s^{(k)}\}$ , where  $s^{(j)} = Sim(x,x_t'(\bar{w}_i^{(j)}))$  is the semantic similarity between  $x$  and  $x_{t}^{\prime}(\bar{w}_{i}^{(j)})$ . In the similar manner, the semantic similarity between  $x$  and  $x_{t}^{\prime}(\bar{w}_{i})$  can be computed  $\overline{s}_i = Sim(x,x_t'(\bar{w}_i))$ .
Intuitively, if  $s^{(j)} - \overline{s}_i > 0$ , it indicates that pushing the word vector  $\pmb{v}_{\bar{w}_i}$  towards  $\pmb{v}_{\bar{w}_i^{(j)}}$  tends to increase the semantic similarity, i.e.,  $\pmb{v}_{\bar{w}_i^{(j)}} - \pmb{v}_{\bar{w}_i}$  is the direction which can improve the semantic similarity. And if  $s^{(j)} - \overline{s}_i < 0$ , moving the word vector along the inverse direction of  $\pmb{v}_{\bar{w}_i^{(j)}} - \pmb{v}_{\bar{w}_i}$  can improve the semantic similarity. Based on the above intuition, we estimate the final updating direction  $\pmb{u}$  by weighted averaging over the  $k$  possible directions. Formally,
$$
\boldsymbol {u} = \sum_ {j = 1} ^ {k} \alpha_ {j} \left(\boldsymbol {v} _ {\bar {w} _ {i} ^ {(j)}} - \boldsymbol {v} _ {\bar {w} _ {i}}\right), \tag {8}
$$
where  $\alpha_{j}$  is the corresponding weight associated with the direction  $\pmb{v}_{\bar{w}_i^{(j)}} - \pmb{v}_{\bar{w}_i}$ , and it can be calculated by  $\alpha_{j} = (s^{(j)} - \overline{s}_{i}) / \sum_{l = 1}^{k}|s^{(l)} - \overline{s}_{i}|$ .
Updating the adversarial example. Due to the discrete nature of text data, we need to use the updating direction  $\pmb{u}$  to pick out the corresponding replacement word  $\tilde{w}_i$  from  $S(w_i)$ , where  $\tilde{w}_i$  is the word which has the maximum cosine similarity between  $\pmb{u}$  and  $\pmb{v}_{\tilde{w}_i} - \pmb{v}_{\bar{w}_i}$  and ensures that  $x_{t}^{\prime}(\tilde{w}_{i})$  satisfies the adversarial condition. After obtaining  $\tilde{w}_i$ , we can generate  $x_{t + 1}^{\prime}$  in the optimizing order sequentially. In addition, to reduce the number of queries and shrink the perturbation rate, when implementing the program, we first use  $x$  to initialize  $x_{t + 1}^{\prime}$ , and then replace the word  $\tilde{w}_i$  one by one until  $x_{t + 1}^{\prime}$  satisfies the adversarial condition.
4.4 The Overall Procedure
The detailed algorithm procedure of HQA-Attack is given in Appendix B. In particular, HQA-Attack first gets the initial adversarial example by random initialization. Then it enters into the main loop. In each iteration, HQA-Attack first substitutes original words back, then determines the optimizing order, and finally updates the adversarial example sequentially. In addition, we provide some mechanism analysis of HQA-Attack from the perspective of decision boundary in Appendix D.
Datasets. We conduct experiments on five public text classification datasets MR [28], AG's News [42], Yahoo [42], Yelp [42], IMDB [22], and three natural language inference datasets SNLI [3], MNLI [33], mMNLI [33]. The detailed dataset description is shown in Appendix E. We follow the previous methods [23, 38, 37] to take 1000 test examples of each dataset to conduct experiments.
Baselines. We compare with three state-of-the-art black-box hard-label textual adversarial attack methods: (1) HLGA [23] is a hard-label adversarial attack method that employs the genetic algorithm to generate the adversarial example. (2) TextHoaxter [38] is a hard-label adversarial attack method that formulates the budgeted hard-label adversarial attack task on text data as a gradient-based optimization problem of perturbation matrix in the continuous word embedding space. (3) LeapAttack [37] is a recent hard-label adversarial attack method, which estimates the gradient by the Monte Carlo method.