Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text Models
Authors: Wenqiang Wang
Chongyang Du
Tao Wang
Kaihao Zhang
Wenhan Luo
Lin Ma
Wei Liu
Xiaochun Cao
Venue: NeurIPS 2023 poster (2023)
Keywords: Punctuation-level Attack
Textual Adversarial attack
Natural Language Processing
TL;DR: A method of punctuation-level attack to fool language models

Abstract: The adversarial attacks have attracted increasing attention in various fields including natural language processing. The current textual attacking models primarily focus on fooling models by adding character-/word-/sentence-level perturbations, ignoring their influence on human perception. In this paper, for the first time in the community, we propose a novel mode of textual attack, punctuation-level attack. With various types of perturbations, including insertion, displacement, deletion, and replacement, the punctuation-level attack achieves promising fooling rates against SOTA models on typical textual tasks and maintains minimal influence on human perception and understanding of the text by mere perturbation of single-shot single punctuation. Furthermore, we propose a search method named Text Position Punctuation Embedding and Paraphrase (TPPEP) to accelerate the pursuit of optimal position to deploy the attack, without exhaustive search, and we present a mathematical interpretation of TPPEP. Thanks to the integrated Text Position Punctuation Embedding (TPPE), the punctuation attack can be applied at a constant cost of time. Experimental results on public datasets and SOTA models demonstrate the effectiveness of the punctuation attack and the proposed TPPE. We additionally apply the single punctuation attack to summarization, semantic-similarity-scoring, and text-to-image tasks, and achieve encouraging results.

perturbations, Punctuation-level Attack: Single-shot and Single Punctuation Attack Can Fool Text Models
Wenqiang Wang'
Chongyang Du'
Tao Wang²
Kaihao Zhang*
Wenhan Luo
Lin Ma
Wei Liu
Xiaochun Cao
Shenzhen Campus of Sun Yat-sen University
"Nanjing University
3 Australian National University
*Meituan
Tencent
Abstract
The adversarial attacks have attracted increasing attention in various fields includ- ing natural language processing. The current textual attacking models primarily focus on fooling models by adding ignoring their influence human perception. In this paper, for the first time in the community, we propose a novel mode of textual attack, punctuation-level attack. With various types of perturbations, including insertion, displacement, deletion, and replacement, the punctuation-level attack achieves promising fooling rates against SOTA models on typical textual tasks and maintains minimal influence on human perception and understanding of the text by mere perturbation of single-shot single punctuation. Furthermore, we propose a search method named Text Posi- tion Punctuation Embedding and Paraphrase (TPPEP) to accelerate the pursuit of optimal position to deploy the attack, without exhaustive search, and we present a mathematical interpretation of TPPEP. Thanks to the integrated Text Position Punc- tuation Embedding (TPPE), the punctuation attack can be applied a constant cost of time. Experimental results on public datasets and SOTA models demonstrate the effectiveness of the punctuation attack and the proposed TPPE. We additionally apply the single punctuation attack to summarization, semanti-similarity-sooiii and text-to- -mage tasks, and achieve encouraging results.
Introduction
Deep Neural Networks (DNNs) have achieved tremendous success in the NLP community and have spawned a series of well-known applications such as ChatGPT [13]. However, more attention should be paid to the vulnerability of NLP models under adversarial attacks protect them. Adversarial
examples have been shown to have a devastating impact in the image domain [39 12 33 Unlike the image domain, text discrete domain [45], which means that any small perturbations, such as sentence, word, even character perturbations in text, would be easily perceived by humans. difficult to achieve imperceptible perturbations. Additionally, the text a non-differentiable domain, which results in traditional optimization-base methods being ineffective in the NLP domain. Recent research on textual adversarial attacks can be mainly classified into character-leved word- level. sentence-level. and multi-level attacks [41]. Character-level attacks modify words by inserting, deleting, misspelling, replacing, or swapping characters. These attacks are easily detectable by humans and some apps correct them automatically. However, character-level attacks require multiple queries to determine which characters to attack based on function gradient or score. Word- level attacks modify words by adding. deleting, replacing important words in text. Similar to character- attacks, word-level attacks also require multiple queries to determine which words to
"Corresponding Author: cchluo.china. @ com
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Original text
Most people probably consider, even though the courts didn't actually find, Klaus guilty murder. |Acceptable:99.79]
Insertion
Most people probably consider, even though the courts didn't actually find, Klaus guilty, of murder. [Unacceptable:96.9)
Displacement
Most people probably, consider even though the courts didn't actually find, Klaus guilty murder. (Unacceptable:93.0g
Deletion
Most people probably consider even though the courts didn't actually find, Klaus guilty of murder. (Unacceptable:444.99)
Replacement
Most people probably consider, even though the courts didn't actually find, Klaus guilty of munder(Unssceptalleee..8
The examples of punctuation-level attacks. The text model is fooled by inserting, displacing, deleting, and replacing punctuation from the original text. The predicted label of the input text is either acceptable or unacceptable. A single punctuation attack can significantly change the score of the predicted label.
Figure 1:
attack. Both character- and word-level attacks are iterative and thus may be time-consuming in real NLP applications due to multiple queries required for each attack iteration. Sentence-level attacks modify entire texts by paraphrasing or adding meaningless sentences to input texts. The significant perturbations caused by sentence-level attacks may sometimes change the ground truth of input text. Perturbations in character-, word-, and sentence-levels can cause semantic gaps from the original text and are casily detectable by humans.
Considering the issues mentioned above, we propose punctuation-level attacks as new method of textual adversarial attack. Compared with other attack methods, perturbations on punctuations are less perceivable than character- and word-level attacks, more difficult to detect than character-level attacks, and cause less impact on ground truth than other types of attacks. previous research, little attention has been paid to punctuation; several researchers have inserted punctuations into words to create out-o--vocabulary (OOV) words, which is a specific method of character-level attacks. On the contrary, we first propose punctuation-level attacks that regard punctuation perturbations as new level of attack similar to character-, word-, and sentence-level attacks. We define basic attacking methods and analyze why these methods can fool text functions. Unlike other types of attacks, punctuation- attacks only modify punctuations by inserting, displacing, deleting, or replacing them without perturbing characters, words, and sentences. Punctuation-level attacks can deceive text functions by inserting, displacing, deleting, or replacing punctuations. Specific examples are presented in Fig. D showing that the single punctuation attack results in a dramatic change in the score of the predicted label. To deploy punctuation-les in the input text should be modified and which punctuations should be modified. For example, if there are k candidate punctuations and n tokens in the input text, there will be kn candidate attack texts when we insert one punctuation, k²n(n 1) candidate attack projects when we insert two punctuations, and k-3n(n + () + 2) candidate attack projects when we insert three punctuations. The time complexity of multiple punctuation attacks can be reduced using greedy algorithm or a beam search. However, even considering only a single punctuation attack is still time-consuming
attacks, we need to consider which positions
To address this problem, we propose a method called Text Position Punctuation Embedding (TPPE) to reduce search cost. The feature extraction processing of input text æ is also the embedding processing of text æ, which is time-consuming for multiple queries. Therefore, Position Punctuation Embedding (TPPE) to quickly and reasonably embed adversarial candidate text
propose Text
Lado. Let us illustrate with the Insertion attack mode. When we apply Insertion attack to punctuation p into the i-th position of the input text, we query the text function f(x) or the substitute function (==)((:) and gain the fxr(ar) as the embedding of text Exeat and (p) as the embedding of punctuation The embedding of punctuation p only needs to be queried once and can be used repeatedly. As for the embedding of position, we adopt the embedding of the static position based on the calculation of sine and cosine from Transformer [40]. And we define the embedding of the adversarial candidate text Fade by concatenating or adding the embedding of text æ. punctuation p and position i. We can iteratively attack the input text x and quickly obtain the adversarial text Tade after training the classification model from the TTPE of Fade to the prediction of Tade by f(x). In addition, we propose search method Text Position Punctuation Embedding and Paraphrase (TPPEP) to achieve a single-shot attack. We analyze the most information-lacking scenario for TPPEP: zero query, black-box text function, hard-label output, single punctuation limitation, and single-shot attack. To achieve the goal of zero query and gain more information, we train a substitute function ssub to fit the text function f. Then, we query the substitute function fous to obtain the embedding of

---

text Etert. We apply the TPPE method to obtain the embedding of the adversarial candidate text Rados denoted E We then transform the attacking task into the paraphrasing task. Ei... and Etent are concatenated as input data, and the result of attack (successfully attack, donated as label 1, otherwise, donated as label 0) is the predicting label. We choose the adversarial candidate text with the highest paraphrasing score calculated by the TPPEP method and deploy the single punctuation attack in a single-shot manner accordingly.
The contributions can be summarized as follows. (1) Punctuation-level attacks: We first propose punctuation-level attacks, which regard the perturbations of punctuation as a systematic attack like character-level, word-level, and sentence-level attacks. We propose four primary modes of punctuation- attacks and explain punctuation-lewe perturbations. (2) TPPE: We first propose the TPPE e embedding m ethod to decrease the search cost. We reduce the query time complexity from O (kn) of Insertion, O (nt) of Displacement. O (t) of Deletion, and O (kf) of Replacement, to O (1) under single punctuation attack. It can quickly and reasonably embed the adversarial candidate text aady using a single-shot query. (3) Single-shot and Single Punctuation Attack: To make our punctuation-leve attack more imperceptible, we modify only one punctuation. Besides, we discuss the most challenging scenario: zero query, black-box function, hard-label output, one- ine-punctuation limitation, and single-shot attack, which the closest the real-world scenarios. We correspondingly propose the TPPEP method and achieve promising experimental results.
attacks from the perspective of optimal
attacks in
2 Related Works
Textual adversarial attack. Text adversarial attack methods in the literature can be broadly cat- egorized into two classes: white-box attacks and black-box attacks. In white-box attacks, some methods [23 35. 34 1. 6] employ FGSM [12] to identify the words or characters with the most significant impact on rext-related tasks. For example, TextFool [23] introduces a gradient loss function within the model to assess the influence of specific words characters on task performance. Suran- jana et al. [35] consider firstly trying to remove the adverb as a word-level attack, and use a gradient loss function in the model to achieve the attack. AdvGen [1] focuses on the similarity between the original words and their substituted counterparts to target neural machine translation (NMT) models.
Moreover, some methods rely directional derivatives 19 8] For instance, Hot-flip [9] manipulates text function by altering characters based on directional derivative gradients. Ebrahimi et al. propose controlled attacks, an extension of HotFlip that enables targeted attacks [8] In black-box attacks, specific methods aim to insert nonsensical sentences into the input text with the intention of deceiving the text model [19 42 41 Notable approaches include AddSent [19] and AddSentDiverse [42] AddSent incorporates meaningless sentences into the input text to confound comprehension system models [9]. Meanwhile, AddSentDiverse broadens the pool of adversarial candidate texts by varying the placements of inserted sentences [42]. Other strategies involve the manipulation of words and 2
sentences [3. 29 11, 22, 28
44, G For example, Textbugger [22] alters words by substituting,
removing, adding, or rearranging them to deceive the text model. Furthermore, certain methods
are centered on paraphrasing the input text [18 321. SCPNs [18] employs an encoder-decoder architecture to generate new paraphrased versions of the input text.
Punctuation attacks. To the best of our knowledge, there is scarcity of research in the existing literatures on punctuation attacks. Most related studies primarily concentrate on punctuation pertur- bation, specifically the placement of punctuation marks within or at the end of words. This approach can be classified as a character-level a ttack, For example, Hosseini el a l. insert punctuation into words to transform them into OOV words [17] In another work by Hofer el al. [15], they also regard inserting a single comma as a special character-level attack, and they insert a comma at the end of the important word to translate this important word into an OOV word. In addition, special symbol text attacks (SSTA) method is proposed in [23], which is viewed as an adjunct of TextFool [10] by adding fifty tildes behind the text.
3 Methodology
3.1 Punctuation-leve Attacking
The Proposed Punctuation-level Attacks. We propose four modes of attacking at the punctuation- level and analyze their time complexity for traversal search. These modes are as follows:
3

---

Table I: Punctuation-level attack modes and Complexity. k is the number of candidate punetuations, t is the number of punctuations in the input text and n is the number of input tokens.
mode
single punctuation
times of punctuation attacks
times of attacks by greedy algorithm
m (n
Insertion
(km)
(0.5k(2n m 1))
E0
Displacement
O (nt)
mm)
O (mat)
m
Deletion
(tt)
0
II (t- (t-a)
(00.5m m))
Replacement
(kt)
((k))m))
((mki)
Insertion: Punctuation p is inserted into the target text to fool the text model.
Displacement: Punctuation p is moved from position i to position in the target text.
Deletion: Punctuation is removed from the target text.
Replacement: Punctuation p; is replaced by Pj in the target text.
We denote the number of candidate punctuations by k. the number of punctuations in the input text by i, and the number of input tokens by n. In Insertion attack, there are n candidate positions. and each position can be inserted by k candidate punctuations. So the complexity of Insertion by traversal search O (kn). Regarding the m times of Insertion attacks, one text will generate candidate texts after single punctuation attack, and the complexity of Insertion to each candidate
m-1
If the greedy algorithm is applied for the m times of Insertion attacks, the generated kn candidate texts after the single punctuation attack will be selected with the greatest change in predicted score, and the number positions is increased to n + 1. Thus, the complexity of Insertion to this selected candidate is (0.5k: + 1)). The complexity of Displacement by traversal search is O (nf) because there are candidate punctuations and each position can be displaced by n candidate positions. As for the times of Displacement attacks, one text will generate nt candidate texts after the single-shot attack, and the complexity of Displacement to each candidate nt. So the complexity of times of Displacement attacks is O ((nf)""). If the greedy algorithm is applied for m times of displacement attacks, there are still / candidate punctuations and candidate positions. So the complexity of m times of Displacement by the greedy algorithm is ((nl)). The complexity of Deletion by traversal search is O (f) because there are t candidate punctuations that can be deleted. In the context of multiple occurrences of Deletion attacks, each attack leads to the removal of a candidate punctuation. Consequently, the complexity of m consecutive Deletion attacks can be
is k(n + 1), thus the complexity of multiple Insertion attacks is O km II i=0
(n+
m
expressed as O (I- where represents the total number of possible candidates. If the 1=0
greedy algorithm is applied for m times of Deletion attacks, due to every Deletion attack, the text will lose one punctuation, thus the complexity of m times of Deletion on by the greedy algorithm O (0.5m(2t + m)). The complexity Replacement by traversal search O (kt) because there are f candidate punctuations and each punctuation can be replaced by k candidate punctuations. Regarding the m times of replacement attacks, there are still i candidate punctuations and t punctuations in the input text after single-shot attack. Thus, the complexity of m times of Replacement attacks is (". If the greedy algorithm is applied for m times of Deletion attacks, the complexity of m times of Replacement by the greedy algorithm is O (mkl). All complexity is shown in Table
Single punctuation Attack. The target text consists of n tokens. noted as = [ zn). According to the Universal Approximation Theorem [66. we can define function F(x) to fit the text function (x:) by continuous nonconstant, bounded, and monotonically increasing function ((.) as
M
F(x)= = Σ Umi & (w x + [(x) <
where Um, bm R and w. E DRD. Universal Approximation Theorem demonstrates the fact that if our attack can result in vicious impact in f(x), it will also fool the text function F(x). In the

---

Table 2: The number of tokens affected by single attack. k is the number of candidate punctuations, i the position of attacking punctuation the imput text, and the number of input tokens.
Mode
Perturbed tokens
Perturbed words
Perturbed level
character-level attack
1
value of token
word-level attack
1
value of token
n - + Displacement
Insertion
value and position of token value and position of token value and position of token value of token
j -it1
0
Deletion
n 8-2 -
Replacement
1
following, we will analyze how punctuation-level attacks influence the text function F(x) based on the basic linear unit w x + bm of f(x) as
T " m + bm 21 + W222 + ... Fm + bm.-
We compare character-level, word-level, and punctuation-level attacks under the constraint of a single-element attack, i.e., perturbing one character, one word, or one punctuation. In character-level attacks, a single-character attack impacts only one token Vi, making OOV. In many text functions, the OOV token is embedded an identical token Toou- In this scenery, w n fm can be written as
w , x + bm = W1 21 + WyT2 + w, + Wm + bm-
Similar to character-level attacks, a single-word attack impacts only one token I₁. Since there are few candidate tokens Tean that are semantically similar to the token when considering the similarity of input text with and without a word-level attack, w + bm can be rewritten as
31 + W2222 + + WyFeaa
+ Wmmmm + bm-
Regarding the proposed punctuation attacks, discuss single punctuation attacks in terms of Insertion, Displacement, Deletion, and Replacement respectively. Taking Insertion as an example, when we insert punctuation p into position i of the target text, w.,, wn x bm will change to
wiT + 2222 + + w.p + ws+124 + ..... Wm + Wm+1Im + bm-
(5)
According Eq. (3) and 4 the character-level and word-level attacks only perturb one token and one word by the single punctuation attack. According to Eq. (5), the single Insertion on attack does not perturb any words but perturbs the i-th token by substituting x; with punctuation p and all tokens after the (i - 1)-th position of the input text by moving them one bit backward, which totally perturbs n + tokens. The result of Displacement, Deletion, and Replacement presented in Table According to Table the punctuation-level attack impact more tokens, indicating that it can cause greater perturbation than character- and word-level attacks. If we insert, displace, or delete punctuation in the appropriate position, will result in more significant impact on the basic linear w w,, + bm of f(x) compared to character-level and word-level attacks. Meanwhile, single punctuation attack may apply more vicious impact to f(x) and F(x). Compared to words, punctuation is thinner, smaller, and more imperceptible. Due to the nature that punctuation- level attack does not perturb any word, semantically, the adversarial text is more imperceptible for human beings. Furthermore, the Insertion Displacement, Deletion, and Replacement of the punctuation produces less impact the semantic information by human beings understanding.
The Ultimate Effectiveness of Punctuation-level Attacks. According to Eq. B and word-level attacks under single-element limitation may focus on perturbing token values. and Replacement attacks can not only perturb token values
5 character-
However, Insertion. Displacement but also token positions which may result in a vicious impact on f(x). This position perturbation almost has no impact on semantic information for human beings.
We discuss why multiple punctuation level attacks can fool the text model by taking Insertion attacks as an example. In the text model, there exists the maximum length the input text, and we denote it as N. If the length of the input text is greater than N, the text model f considers only the first N tokens. If we apply N Insertion attacks, there will be N. candidate texts. The probability of fooling the text model is equal to I minus the probability of all A.N candidate texts failing to fool
5

---

Table 3: Text embedding in punetuation-level attack.
Mode
Concat
mathematical operation
Insertion
Eall
Erent + Epps + Pune
Displacement
Eall
Eteat
E poo + E Epunc
Deletion
Eall Erent E pos punc
Replacement
Eall
Exert + E Epos E punee" + E
pune
the text model. We denote the probability of fooling the text model as ffoo and the probability of failing to fool the text model by the i-th candidate text as pt; i E [1, K K And we denote the maximum value of p, Pmaz- We make an assumption about Pmaw here. Due the fact that pa is
the probability of the i-th candidate text before querying the text model f. and the attacking result is unknown we have 0 < Pmaz < 1. Correspondingly, we have
Pfoo
P. II
Pmax
(6)
According to Eq. 66. 1- Pmaz will approach 1 as k increases. As a probability, we have - -Pmax S
Prool 1. Consequently, foot will tend to when & increases, which means multiple Insertion attacks can ultimately fool the text model with & increasing.
3.2 Embedding and Search Methods
According to Table [ punctuation-level attacks require considerable cost to deploy. Even considering only the single punctuation attack, it is still time-consuming. Taking the mode of Insertion as an example, its original time complexity is O (kn). Therefore, it is essential to decrease the search cost. we have reduced the search cost of a single punctuation attack, the time complexity of multiple-punctuation attacks can be reduced by using either a greedy algorithm or beam search. The n is determined by the target text æ, which is not feasible to optimize. As k increases, the search space expands, resulting in higher fooling rate of Insertion attacking. Therefore, to achieve higher fooling rate, we should not reduce the value of k.
As we cannot decrease k and n, we focus on a specific process: text function f f(x). For example, in a text classification task, function f(x) = softmax fax), where the processing of feature extraction is denoted by the function f/+ and softmax is used as the final classification layer. The classification cost to can be divided into two parts: the feature extraction consumption t and the final classification consumption 22. The classification time la is transient compared to L and therefore can be ignored. Consequently, we have la L1.
Embedding the adversarial candidate text aado perturbed by various configurations requires multiple queries, and thus it is time-consuming. Therefore, we propose TPPE to quickly and reasonably embed the adversarial candidate text Lade- According to Eq. (5). a single punctuation attack consists of three components: input text æ. perturbed position, and candidate punctuation. To reduce the cost of embedding processing for adversarial text Tadv. we refine its embedding process by combining the embeddings of text, position, and punctuation. Again, we take the Insertion attack mode as an example. When we insert punctuations p into the i-th position of the target text, we query text function (xx (white-box) or substitute function (x))(x) (black-box), obtaining Etezt as the embedding of text and ffe (p) as the embedding of punctuation p. The embedding of punctuation only needs to be queried once. For the embedding of the position, we adopt the embedding of the static position based on the calculation of sine and cosine from the Transformer [40), present in Eq. The variable pos represents the position and the variable represents the dimension.
PE (pos,,)2) sin pos/ 100000000////
P cos ppss 10000
(7)
After calculating embeddings of input text, punctuation, and position, we calculate candidate at- tacking text embeddings using two steps: 1) concatenating three embeddings as candidate attacking text embedding Enll = Concat Etext; Epos; p and, 2) calculating candidate attacking text embedding using mathematical operations presented in Table

---

Table 4: The quantitative results of single punctuation attack in the T2I task. "Ori-text0" indicates the original text "a professional photograph of an astronaut riding triceratops", and "Ori-textl" indicates the original text "a corgi is playing piano, oil on canvas".
dataset pokemon-blig bii-bb-ppppppsoss
all
train
Test
On Dri-image
Adv-image
Ori-lext
Adx-
Oni-ext
0,3273
0.3272
0.3278
Ori-text Ori-textl
0,3281
0.2484
0,9782
Adv-text
0.2591
0.2586
0,2610
0,4040
0.3468
0,9843
The Insertion mode inserts punctuation p into the i-th position of imput text, regard E Epos + pune as the embedding of Bade- The Displacement mode displaces punctuation from position i to position j in the input text, so we regard Etest - E + E poe + E pune as the embedding pos
of Todv- The Deletion mode deletes punctuation p from the i th position of the input text, so we compute Etezt
E pos E pune E pune the embedding of aade- The Replacement mode
replaces punctuation p. with P3 in the input text, so we take Exent + E Epas E + E punc" TPPE requires only single-shot embedding of the input text æ. which means we decrease the query time complexity from O (kr) of Insertion, O (nd) of Displacement, O (1) of Deletion, and O (kl) of Replacement, O (1) under single punctuation attack.
In our study, we assume the worst-case of applying punctuation- attacks: the victim model is a black-box model where only prediction labels are available instead of scores from the function. In this scenery, we train a substitute function foub 33 to transform the black-box scenario into a white-bor box scenario by collecting part of training datasets X. Specifically, we query the text function and derive the prediction labels ypre. then train the substitute function feut using the input text and the label Upre as paired data. After training Saub, transform the black-box scenario into a white-box scenario.
Directly querying Sus to determine which punctuation should be used also time-consuming for multiple queries. We iteratively attack the input text and quickly gain the adversarial text Tady after training the classification model from the TPPE of Dade to the label of Vode by f(x). Additionally, propose search method called TPPEP to achieve single-shot attack. We analyze the worst-case scenario for TPPEP: zero query, black-box function, hard-label output, single punctuation limitation, and single-shot attack. The specific pseudo code of TPPE and TPPEP are presented in Alg. Tand Alg. 2 in the Appendix.
To achieve the goal of zero query and gain more information, we train substitute function Saud to fit the text function f. Then, we query the substitute function fsub to gain the embedding of text Etext- We apply the TPPE method to gain the embedding of adversarial candidate text Dadv, denoted as Exead- We transform the attacking task into a paraphrasing task. Specifically, E Eg.ad- and Erezt are concatenated as input data, and the result of the attack (the successful attack denotes label 1; else the label 0) is the predicting label. The adversarial candidate text with the highest paraphrasing score calculated by the TPPEP method is chosen to deploy the attack.
3.3 Analysis of TPPEP
In this section, we discuss specific and practical form of classification function that can de- ceive the text model. We assume that there is a function f(x) in hypothesis space F. Since f(x) softmax (f))())), we hypothesize that there exists a function fi that fits the function softmax and satisfies (xx - Uz < ej. A simple functional form of fi is a function of arg max softmax.. Furthermore, we propose the hypothesis that there exists function fz. which can predict Desd. from T E (wadv) and satisfy fz(Aadu) Tz..no < Eg. We also define a function fs.
fa (Xadv: x) {1
if [/2 (Iadv) fxx)) 2 otherwise
This means we can predict the result of an attack
by [a((x)) - fi (ar) We calculate the absolute difference between f2( and (x) and compare the absolute difference between f2 Va.aul and [Nm... Vn Eq. as
f2 (Xodv) - fi(x)) = Ifz (Xadv) - Danes + Denas - fx + Fx - Tze
Ilfe (Iadv) Vanaul 1Waad Dzl + 17 f1(x))
(8)
< E + 52 + 1Dmadv Del
7

---

According to Eq. (8) when Da.d. DR. the actual label of TPPEP is 0 and fa( fi ((x)) - Ei + Eg. Since E and E2 are sufficiently small, fa fa( fi (a:) will be 0, allowing us clas- sify them correctly. As (2) and (2) fit well Vr... and Yi. when Tm... and Un are not equal, [z(f2(Fodz): Si (a)) will be Thus, we have found specific fooling function fa((2( fi (z)), which implies that there is at least one fooling function in the hypothesis space F. Moreover, we discuss that fa( fi (x)) can be learned based on the Universal Approximation The- orem [16.. According to Eq.
we can train Fi and F2 to fit fi and f2. which satisfy (F(()) - [xx) < E3 and | (Tado) - -f2( (aado) < E4. We analyze the difference between f22 (Dd - fz)) and (2( Fi (x) in Eq. (.. If the difference between them is less than 1, then (a (Xado) Fi (x) and (2(f2(2 fz)) will predict the same label.
[ (wedd) [x) [f2 (Aadv) f1(=)]] =1 (x) [x(]]] [52 (aad)) F22 (Xade)))
S | (z) fi (z)) F2 (wede) fa (#Ddde) S 53 + E4-
we compare the difference between |2 - F1 (2)).. As E3 and E1 are sufficiently small, (x)(2)) - Pi (xx) will approximate (f2()) (xx). suggesting that we can fit Deaas Dz by (f2(x)) - f1 (z)).. Based on the above analysis, we prove that a feasible solution exists in the hypothesis space and can be trained.
According to Eq.
[/2 (Aadd)
(==) and
Experiment
4.1 Experimental Setting
Tasks and Datasets. We have implemented punctuation-level attacks and TPPEP in the context of three distinct text-to-label tasks, utilizing three diverse datasets. Furthermore, have reported the outcomes of these experiments concerning text summarization, semantic-similartt-ssooii to-image tasks, focusing particularly the single-shot punctuation attack. The tasks that underwent single-shot attacks and TPPEP perturbations encompassed text classification (TC), paraphrasing, and natural language inference (NLI) tasks. The TC task takes text as input, and the text model generates predicted label for the input text. For our TC task, we have opted to utilize the CoLA dataset [43] CoLA the dataset of a binary classification task that aims to predict whether the input sentence is "correct" from a linguistic perspective. As for the paraphrase task, the input consists of a pair of texts, and the objective of the text model is ascertain the semantic similarity between the two texts. Our choice for the paraphrasing task is the QQP dataset [21]. Furthermore, the NLI task involves predicting the relationship between pairs of input sentences and we employ the Wanli dataset 225
and text-
Experiment Setup. We have applied both punctuation-level attacks and TPPEP target pre- trained SOTA models, including ELECTRA (7]. XLMR (37). DistilBERTI (36), ROBERTa (27). and DeBERTa 14). In numerous weal-world text API application scenarios, the API solely provides the predicted label for the input text. Therefore, our experiments are tailored for black-box attacks, where the text model exclusively generates categorical labels in response to queries. The conventional perturbation budget of number of altered words. In contrast, our punctuation-lewe attacks perturb zero word, and we define the perturbation budget as the number of perturbed tokens. In the experiment, we set the strictest perturbation limit, specifically focusing on the perturbation of single punctuation. Thus, we discuss the results in the case of single punctuation attack. In addition, to make our TPPEP work in hard-label attacks, we first train a substitute function Ssut (30) by querying the text function and deriving the prediction labels ypre. Then, we apply the proposed TPPEP method to search for the most vulnerable candidate text.
attacks is typically assessed based on the
4.2 Experimental Results
Text classification task, paraphrase task, and natural language inference task. The result of the TC task is presented in Table Our punctuation-lewe attack shows encouraging results on ELECTRA [7] model and Cola dataset with 90.80% and 93.67% fool rates respectively by single- shot Insertion attack. On all three datasets, Insertion consistently obtains the highest fool rate due to the largest search space compared to other attacks such as Deletion, which has the smallest search space and thus obtains the lowest fool rate. The fool rate of Displacement and Replacement depends more on the specific datasets and text models. The results of the traversal
8

---

Table 5: The results on Cola, QQP, and Wanli datasets. Top-1 is the fool rate of single-shot and single punctuation attack. The Top-3 and Top-5 are the fool rates of three and five single punctuation attacking candidate texts. Traversal is the fool rate of a single punctuation attack by traversal, which the maximum value of a single punctuation attack. pr is the ratio of TOP-1 to traversal. Average search space (ASP) is the mean of candidate texts.
Cola
ELECTRA
XLMR [37
mode
Top-1
Top
Top 5
Traversal
Inp 1
Top-3
Top
Traversal
ASP
Insertion
67.40%
75.06%
73.83%
90.80%
14.23%
28.70%
52.64 52.64%
63,57%
93.67%
30.70%
362.62
Displacement
36.05%
66.35%
73.44%
80.42%
44.82%
43.05%
60.12%
76.03%
80.73%
53.33%
11.59
Deletion
3.18%
5.83%
5.94%
5.44%
87.21%
4.89%
5.85%
5.83%
5.85%
83.39%
1.15
Replacement
24.64%
36.82%
44.77%
74.59%
33.03%
6.62%
9.88%
12.37%
20.23%
32.72%
41.49
QQP
DisHiBERT
[56]
DistiRERT2
[36]
Insertion
14.72%
18.76%
22.68%
47.18%
31.20%
8.67%
10.43%
11.73%
48.23%
17.98%
957.72
Displacement
8.52%
15.05%
18.86%
26. 78%
31.81%
7.21%
12.43%
15.57%
23.44%
30.76%
36.57
Deletion
3.94%
5.93%
6.02%
6.03%
65.34%
3.06%
6.86%
6.95%
6.96%
72.70%
2.33
Replacement
7.59%
10.04%
12.18%
19.70%
38.53%
16.70%
20.97%
22.65%
29.65%
56.32%
90.91
Wanli
RoBERTa 12
DeBERTa [14
Insertion
8.44%
19.22%
26.20%
66.74%
12.65%
15.28%
29. 29.20%
37.40%
80.14%
19.07%
1161.12
Displucement
5.12%
9.14%
12.26%
26.14%
19.59%
10.28%
16.60%
20.34%
38.40%
26.77%
53.94
Deletion
3.22%
5,84%
6.14%
6.16%
52.27%
3.74%
8.38%
8.96%
8.08%
03.92%
2,04
Replacement
8.48%
15.96%
19.80%
45.82%
18.51%
6.92%
13.08%
16.88%
54.76%
12.64%
105.88
Table 6: The results of the comantic-similarity
task
Sentence- -BRRT
Distilbert
STS-B
Pearson
Spearman
Pearson
Spearman
Without Attack
0.7990
0.6988
0,8056
0.7257
TOP-1
0.7874
0.6862
0.7902
0.7035
TOP-3
0.7760
0.6738
0.7759
0.6990
TOP-5
0.7654
0.6626
0.7649
0.6745
Traversal
0.6992
0.5832
0.6994
0.6048
search imply that a single punctuation attack can fool those text models. The results of Top-1 present the result of single-shot and single punctuation attack. Our experiments show that single-shot and single punctuation attacks are more effective for datasets with small average search space (ASP) such as the CoLA dataset. In the case of ELECTRA model and CoLA dataset, single-shot and single punctuation attack obtains 67. 67.40% 36.05% 5.18%, and 24.64% respective fool rate by Insertion Displacement, Deletion, and Replacement attack. The fool rate of the single-shot and single punctuation attack decreases with increasing search space but still achieves a fool rate of 8.44% when the ASP is 1161.12 in the Wanli dataset.
task, summarization task, and text-to-image task. In addition, we have expanded the scope of single punctuation attacks to encompass mimaanic-simillrttyyssooo (sss), summarization, and text-to-image tasks (T2I). Datasets from STS-B BB (26) and gigaword [20] datasets were chosen for the sss and summarization tasks. Results of single punctuation attacks are
semantic-simillaityysson
tabulated in Tables 6 and 7 In the sss task, the Insertion punctuation attack leads to reductions of Pearson and Spearman correlations for Sentence BERT and Distilbert, resulting in reductions of 0.0998, 0. 1566, 0.10 and O. 1209, respectively. For the summarization task, we utilize the ROUGE-1 metric as the dependent variable, denoted as y. Then we formulate a predictive model to estimate the ROUGE-1 score for the test set. Subsequently, we select the candidate attack text with the lowest predicted ROUGE-I value. The Insertion punctuation attack decreases the ROUGE-1 score 24 of gigaword 20 by 6.47. In the sss task, y represents the semantic similarity between the text before and after the attack. To model this, we have developed a predictive model to estimate semantic similarity values for the test set. This significant decline demonstrates the susceptibility of sss and summarization models to single punctuation attack.
In the T2I task, we determine the CLIP score denoted as and select the candidate attack text with the lowest predicted CLIP score. Stable Diffusion V2 33) is selected as the victim model, and the results are illustrated in Fig. 2 When we input the sentence "a professional photograph of an astronaut riding a triceratops", the Stable Diffusion V2 generates the "correct" image containing both the astronaut and triceratops. However, after inserting a period into the input text, Stable Diffusion V2 generates an image without triceratops. This dramatic change also happens another input sentence. Furthermore, we have expanded the section dedicated to quantitative experiments in T2I applications,

---

Table 7: The results of text to image and summarization task
Task
Metric
Without Attack
TOP-1
TOP-3
TOP-5
Traversal
Text to image
CLIP score
0.3278 11.69
0.3176
0.3069
0.3022
0.2610
Summarization
ROUGE-1
10.91
9.65
9.11
5.22
Ori-output
Adv-output
Ori-output
Adv-output
professional
Ori-teat: Adv-text: corgi playing, piano,
Adv-text: professional photograph astronaut. riding triceratops
The results of single punctuation attack in the text-to-imanl task. "Ori-text" indicates the original text and "Adv-text" indicates the adversarial text under single punctuation attack. "Ori- output" and and "Adv-output" indicate the images generated by the original text and the adversarial text respectively. The results reveal that the text-to-image model is fooled by inserting a comma or period into the input text.
Figure 2:
utilizing the CLIP score as our evaluation metric. The experimental results are displayed in Table 4 with "ori-textl" denoting the sentence "a corgi playing the piano, oil on canvas." " By inserting
comma as punctuation-leve attack. "adv-textl" is derived from "ori-text1". Significantly, despite high semantic similarity of 0.9843 between "ori-textl" and "adv-textl", the CLIP score decreases from 0.4040 to 0.3468 after the attack. Furthermore, to evaluate the resilience of the Stable Diffusion model against punetuation-leve attacks, we utilize the "pokemon-bile whaponn--l----pionssss dataset. Following the punctuation insertion attacks, the overall CL.P score decreases from 0.3273 to 0.2591. This reduction consistently observed both the training and testing subsets, highlighting the model's susceptibility. In the present the results of our endeavor to extend punctuation-leve attacks, TPPE, and TPPEP algorithms to encompass three distinct tasks: summarization, semantic similarity scoring, and text-to-image generation show the effectiveness of TPPE and TPPEP.
5 Conclusion
In this work, propose punctuation-leve attack based entirely inserting, displacing, deleting, and replacing punctuation to fool text models. Our punctuation-level attack is more imperceptible to human beings and has less semantic impact compared introduce TPPE and TPPEP methods to reduce the search cost and achieve the time complexity of the query O (1). Furthermore, we discuss the ultimate effectiveness of punctuation-level attacks and present the analysis of the TPPEP method. Experimental studies on representative datasets of SOTA victim models demonstrate the effectiveness of proposed methods.
attack. We
Acknowledgment
This work was supported in part by the National Key R&D Program of China (Grant No. 2022ZD01 18100), in part by the National Natural Science Foundation of China (Grant No. 62372480), part by the Guangdong Basic and Applied Basic Research Foundation (No. 202341515511839) in part by ARC-Discovery grant projects (DP 190102261 and DP220100800), and in part by the Shenzhen Science and Technology Program (No. JSGG202208310933000222 CCF-Tencent Rhino-Bin Open Research Fund (No. CCF-Tencent RAGR20230119).
and sponsored by
10

---

References
[1] Abdullah Al-Dujaili, Alex Huang, Erik Hemberg, and Una-May O'Reilly. Adversarial deep learning for robust detection of binary encoded malware. In SPW, pages 76-82, 2018.
[2] Moustafa Alzantot, Yash Sharma Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. Generating natural language adversarial examples. In EMNLP, page 2890-2896, 2018.
[3] Yonatan Belinkov and Yonatan Bisk. Synthetic and natural noise both break neural machine translation. In ICLR, 2018.
[4] Matthias Blohm, Glorianna Jagfeld, Ekta Sood, Xiang Yu, and Ngoc Thang Vu. Comparing attention-base convolutional and recurrent neural networks: Success and limitations in machine reading comprehension. In CoNLL, pages
2018.
[5] Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, and Yew Soon Ong. Meta- morphic relation based adversarial attacks on differentiable neural computer. arXiv preprint a80i 02444, 2018.
[6] Yong Cheng, Lu Jiang, and Wolfgang Macherey. Robust neural machine translation with doubly adversarial inputs. ACL, pages 4324-4333, 2019.
[7] Kevin Clark, Minh- Thang Luong, Quoc V Le, and Christopher D Manning. Electra: Pre-training text encoders as discriminators rather than generators. In ICLR, 2020.
[8] Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On adversarial examples for character-level neural machine translation. In COLING, pages 653-663, 2018.
[9] Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Horflip: White-box adversarial examples for text classification. ACL. pages 31-36, 2018.
[10] Brian Formento, See-Kiong Ng, and Chuan-Sheng Foo. Special symbol attacks on nlp systems. In IJCNN, pages 1-8, 2021.
[11] Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. SPW, pages 50-56, 2018.
[12] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. In ICLR, 2015.
[13] Roberto Gozalo-Brizuela and Eduardo C Garrido-Merchan. Chatgpt is not all you need. A State of the Art Review of large Generative Al models. arXiv preprint arXiv:2301.04655, 2023.
[14] Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deherta: Decoding-enhanced bert with disentangled attention. In ICLR, 2021.
[15] Nora Hofer, Pascal Schöttle, Alexander Rietzler, and Sebastian Stabinger. Adversarial examples against hert absa model-fooline bert with 133t, misspellign, and punctuation. In ARES, pages 1-6, 2021.
[16] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks universal approximators. Neural networks, 2(5): 1989.
[17] Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. Deceiving google's perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08133 2017.
[18] Mohit lyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. Adversarial example gener- ation with syntactically controlled paraphrase networks. In NAACL-HLT, pages 1875-1885, 2018.
[19] Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In EMNLP, pages 2021-2031, 2017.
11

---

[20] Akhil Kedia, Sai Chetan Chinthakindi, and Wonho Ryu. Beyond reptile: Meta-learned dot- product maximization between gradients for improved single-task regularization. In EMNLP. pages 407-420, 2021.
[21] Huong T Le, Dung T Cao, Trung H Bui, Long T Luong, and Huy Q Nguyen. Improve quora question pair dataset for question similarity task. In RIVF, pages 1-5, 2021.
[22] Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. In NDSS, 2019.
[23] Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. Deep text classification can be fooled. In IJCAI, pages 4208 4215, 2018.
[24] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, 2004.
[25] Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. WANLI: Worker and Al collaboration for natural language inference dataset creation. In EMNLP, pages 6826-6847,
2022.
[26] Fangyu Liu, Ivan Vulić, Anna Korhonen, and Nigel Collier. Fast, effective, and self-supervised: Transforming masked language models into universal lexical and sentence encoders. In EMNLP. pages 1442-1455 1459, 2021.
[27] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: /907. 1692, 2019.
[28] Pasquale Minervini and Sebastian Riedel. Adversarially regularising neural nli models to integrate logical background knowledge. CoNLL, pages 65-74, 2018.
[29] Tong Niu and Mohit Bansal. Adversarial over-sensitivity and over-stability strategies for dialogue models. In CoNLL, pages 486-4994 2018
[30] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In ACM ASIACCS, pages 506 519, 2017.
[31] Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In ACL, pages
2019.
[32] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin, Semantically equivalent ad adversarial rules for debugging nlp models. In ACL, pages 856-865. 2018.
[33] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pages 10684-10695, 2022.
[34] Ishai Rosenberg, Asaf Shabtai, Lior Rokach, and Yuval Elovici. Generic black-box end-to-end attack against state of the art api call based malware classifiers. In RAID, pages 490-510, 2018
[35] Suranjana Samanta and Sameep Mehta. Generating adversarial text samples. In ECIR, pages 744-749, 2018
[36] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.07 108, 2019.
[37] Vésteinn Snæbjarnarson and Hafsteinn Einarsson. Cross-lingual qa a stepping stone for monolingual open qa in icelandic. MIA, pages 29-36, 2022.
[38] Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. TEVC, 23(5):828-844 841, 2019.
[39] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
12

---

[40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurIPS, pages 5998-6008, 2017.
[41] Xuezhi Wang, Haohan Wang, and Diyi Yang. Measure and improve robustness in nlp models: survey. In NAACL, pages 4569 4586, 2022.
[42] Yicheng Wang and Mohit Bansal. Robust machine comprehension models via adversarial training. In ACL. pages 575-581, 2018.
[43] Alex Warstadt, Amanpreet Singh, and Samuel Bowman. Neural network acceptability judgments. TACL, 7:625-641, 2019.
[44] Huangzhao Zhang, Hao Zhou, Ning Miao, and Lei Li. Generating fluent adversarial examples for natural languages. In ACI., pages 3564,556,, 2019.
[45] Wei Emma Zhang, Quan Z Sheng, Ahoud Alhazmi, and Chenliang Adversarial attacks on deep-learning models in natural language processing: A survey. TIST, 11(3):1-41 111:11111,000 2020.
13


Reviews:
[{"review_id": "YWGazjSc1k", "reviewer": "Reviewer_LLfX", "summary": "This paper introduces an adversarial attack against NLP models based on punctuation perturbations. The authors introduce an attack called Text Position Punctuation Embedding (TPPE) that comprises an insertion, displacement, deletion, and replacement attack based on textual punctuation (e.g., commas or periods).\nExperiments are conducted on various datasets, ranging from text classification (CoLA) to paraphrasing (QQP) and natural language inference (WANLI). The attacks are applied to ELECTRA, XLMR, and BERT-based models (DistilBERT, RoBERTa, DeBERTa). Additionally, the attack is applied to semantic-similarity-scoring (STS12), summarization (gigaword), and text-to-image tasks (prompting Stable Diffusion V2). Experimental results are promising, showing that the attack can be used to successfully attack models for the above tasks.", "questions": "Could you provide a few additional details on model training / fine-tuning, as well as the dataset splits that you used for attacking the models? This is not mentioned in Section 4.1.", "limitations": "The paper briefly discusses Limitations in Section 5. However, potential ethical considerations arising from this research remain unaddressed. Since discussing these is quite important in this context (the proposed attack can be misused for malicious purposes), I would encourage the authors to add a section for this.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "* The paper provides an extensive analysis of punctuation-level attacks against NLP models. These attacks are promising since they have the potential to be less perceptible as compared to existing character-, word-, and sentence-level attacks.\n* The analysis is extensive in that multiple NLP tasks (classification, summarization, text-to-image, etc.) are analyzed.\n* It is interesting to see that the investigated models are vulnerable to punctuation-level attacks across tasks and domains.\n", "weaknesses": "* The paper does not compare TPPE against existing works and baselines. In Section 2, the authors point out various existing works focusing on punctuation attacks. However, none of these works have been evaluated and compared against in their experimental settings. To identify and support the strengths and utility of TPPE, such experiments are essential.\n* I additionally think that comparisons to character-, word, and sentence-level attacks on the selected datasets would have been insightful since these experiments would provide the reader with a better understanding of how punctuation-level attacks perform in comparison to attacks focusing on other parts of a textual sequence.\n* The paper does not further analyze the semantics of perturbed adversarial examples. To support the claims of semantic imperceptibility, human experiments analyzing the change in semantics between an original sequence and its adversarial counterpart would be important. The examples in Figure 2 nicely illustrate that inserting single punctuation marks can substantially impact the meaning of a sequence. Since adversarial examples are desired to preserve the semantics of an attacked sequence, quantitative experiments would be needed to evaluate TPPE in that context.\n* The paper does not discuss potential approaches to mitigate the models’ vulnerability against punctuation-level attacks, for instance by assessing whether adversarial training / data augmentation (i.e., training the model on adversarially perturbed sequences) can help increase the robustness of the attacked models. This would provide additional insights into how robust the attack is, and how it can potentially be defended against.\n* The results for the text-to-image task consist only of two qualitative examples. These examples are highly interesting, but to better evaluate the vulnerability of Stable Diffusion models against such attacks, quantitative results over a larger dataset would be important. \n* Overall, the paper focuses too much on introducing the attack and discussing its details and time-complexity analysis, instead of extensively evaluating its performance (the Experiments Section spans under 2 pages in the manuscript).\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1:Compare punctuation-level attack to the existing works focusing on punctuation attacks\n\nA1:First and foremost, it is imperative to clarify that the existing research concerning punctuation attacks does not pertain to the punctuation-level attack. The works referenced as [18, 16] are essentially char-level attacks. Their approach involves the insertion of punctuation marks within words and at word endings, leading to the transformation of the original word into an OOV word(e.g., \"lo,ve\" and \"love,\"). Similarly, the study denoted as [24] simply appends fifty tildes at the sentence conclusion,which has perturbed too much to be suitable as a benchmark method. \n\n The comparative summary of our findings against the outcomes delineated in references [18, 16] is presented on Table 8 in the pdf of global rebuttal. “Hossein” and “Nota” are the method of [18,16] respectively. The results of Table 8 indicate TPPEP method is better than “Hossein” and “Nota” are the method.\n\n\n\nQ2:Compare punctuation-level attack to the char-, word-, sentence-level attacks\n\nThe comparative results between our approach and the char- and word-level attack methods are presented in Table 8. The sentence-level attacks will attack the whole sentence, which has perturbed too much to be suitable as a benchmark method. The results of Table 8 indicate TPPEP method is better than other benchmark methods of char-level and word- level method.\n\n \nIn this experiment, TPPEP perturbed only one punctuation mark. In order to make a fair comparison between TPPEP and other attack algorithms, we limit the other attack algorithms to only perturbing one word. So we didn't select SSTA[24] and sentence-level attacks as benchmarks.We have indeed computed the results of both SSTA [24] and the sentence-level method. If necessary, we can present these results in the discussion. Due to the word limitations,  we  do  not  present   here.\n\nQ3: Analyze the semantics of perturbed adversarial examples\n\nIn Table 8 of  global rebuttal, we delve into the alterations in semantics resulting from punctuation-level attacks. We computed an average semantic similarity of 0.995526417 before and after punctuation attacks. The specific outcomes for each dataset, model, and attack technique are detailed under the \"Semantic Sim\" criterion in Table 8. Our current experimental findings suggest that our punctuation-level attack algorithm has a limited impact on semantics.\n\nIn addition, we have incorporated human evaluations to assess the semantic similarity before and after attacks. We randomly selected 100 sentences along with 15 participants from each of the three datasets, alongside their respective perturbed texts. A five-point scale was established, where a score of 1 indicates substantial dissimilarity between the two sentences, and 5 indicates substantial similarity. The computed mean similarity score is 4.93.\n\n\n\n Q4: The defense is not discussed\n\nIn our global rebuttal, we have incorporated experimental outcomes to supplement our defense strategy. We introduce a defense technique based on the concept of a Preceding Language Modifier. The core objective of the Preceding Language Modifier is to restore the attacked text to its original, coherent form. Such rehabilitated text poses no threat to well-established models. The efficacy of the Preceding Language Modifier is demonstrated in Table 6, revealing its capacity to effectively mitigate the impact of punctuation-based attacks.\n\n \n\nWe have also investigated the impact of adversarial training on punctuation attacks, and the experimental findings have been presented in Table 7. The results demonstrate that adversarial training effectively mitigates the effects of punctuation-level attacks without compromising the model's accuracy\n\n \n\nQ5: The results for the text-to-image task have not been discussed detailedly\n\nIn  the global rebuttal, we present the latest experimental outcomes for the text-to-image task. We utilize the CLIP score as the metric for evaluating attacks and leverage the Pokemon Blip Captions dataset, splitting it into training and testing sets in an 8:2 ratio.\n\nWe present the latest experimental findings pertaining to the text-to-image domain. Employing the clip-score as the metric for assessing the attacks, the outcomes within Table 9 elucidate the impact of attacks conducted at the punctuation level. Notably, following these attacks, the clip-scores experienced a reduction from 0.3281 and 0.4040 to 0.2484 and 0.3468, respectively.\n\n \n\nQ6: The paper focuses too much on introducing punctuation-level attack\n\nDiverging from the extensively explored domains of char-, word-, and sentence-level attacks, we introduce a pioneering approach focusing on attacks at the punctuation-level. So we believe it is necessary to details of the attack. \n\nWe have conducted supplementary experiments, which are presented both in the global rebuttal section and earlier in this document.\n\nWe greatly appreciate the insightful suggestions you provided regarding our experiments. We will effectuate revisions to the paper by incorporating additional experimental segments while judiciously trimming the content related to attack intricacies and time complexity discussions.\n\n \n\nQ7:Details of the experiment were not specified\n\nWe have provided detailed information about training the substitute model in the supplementary materials (Section 1.2 in the supplementary materials). When training the TPPEP model, the training data was split into an 8:2 ratio for the training set and validation set, respectively. For training the TPPEP algorithm, a batch size of 300 was utilized over 9 epochs, employing the Adam optimizer with a learning rate of 0.001, and the loss function employed was the Cross-Entropy Loss.\n\nQ8：Potential ethical considerations arising from this research remain unaddressed\n\nWe greatly appreciate your suggestions. In the revised version, we will incorporate a dedicated section to discuss potential ethical considerations."}, {"review_id": "7VDcFWlm8b", "reviewer": "Reviewer_y7qU", "summary": "This paper introduces a new approach to textual attacks called the punctuation-level attack. The method aims to fool text models while minimizing its impact on human perception and understanding. The paper discusses the effectiveness of this attack strategy, presents a search method to optimize its deployment, and provides experimental results showcasing its success. The authors also apply the single punctuation attack to summarization, semantic-similarity-scoring, and text-to-image tasks, achieving encouraging results. The paper concludes that the punctuation-level attack is more imperceptible to human beings and has less semantic impact compared to traditional character-/word-/sentence-level perturbations. The integrated Text Position Punctuation Embedding (TPPE) allows the punctuation attack to be applied at a constant cost of time. The experimental results on public datasets and state-of-the-art models demonstrate the effectiveness of the punctuation attack and the proposed TPPE. ", "questions": "The paper lacks specific information regarding the implementation details of adversarial training. It does not provide explicit explanations regarding the ratio of clean versus adversarial samples used during training, nor does it clarify whether all methods employ identical training strategies.", "limitations": "The main emphasis of the paper is placed on evaluating the punctuation-level attack's efficacy within specific tasks, including summarization, semantic-similarity-scoring, and text-to-image tasks. However, the evaluation of this attack is not comprehensive across a diverse set of NLP tasks, limiting the extent to which the findings can be generalized.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a new approach to textual attacks called the punctuation-level attack, which is different from traditional character-/word-/sentence-level perturbations.\n2. The punctuation-level attack is designed to be more imperceptible to human beings and has less semantic impact compared to traditional perturbations.\n3. The paper presents a search method called Text Position Punctuation Embedding (TPPE) to optimize the deployment of the punctuation-level attack.\n4. The paper provides experimental results showcasing the effectiveness of the punctuation-level attack and the proposed TPPE on public datasets and state-of-the-art models.\n\n", "weaknesses": "The adversarial attacks discussed in this paper can be categorized as non-pure white-box attacks, as the attack objective may differ from the model's evaluation metric. It is crucial to explicitly acknowledge this fact in the paper, as it is widely recognized that achieving white-box robustness represents an upper bound and is significantly more challenging than black-box robustness.\n\nIt appears that the number of attack iterations is restricted. To ensure robustness evaluation, it is advisable to ensure attack convergence by employing an adequate number of iterations.\n\nIn the experimental section, when comparing the proposed method with other approaches, a fixed number of updating steps is consistently utilized.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1:The adversarial attacks discussed in this paper can be categorized as non-pure white-box attacks.\n\nA1:We greatly appreciate your suggestions, and we will incorporate the necessary revisions in the revised version of the paper. This aspect will be emphasized in the revised version as well.\n\n\n\nQ2:It appears that the number of attack iterations is restricted. To ensure robustness evaluation, it is advisable to ensure attack convergence by employing an adequate number of iterations.\n\nA2:The outcomes of multiple attacks are presented in Table 12. We observed a significant increase in the fooling rate of the model following multiple attacks. The fool rates are gradual convergencing. Additionally, we noted that beyond a certain threshold of attack iterations, the magnitude of the fooling rate enhancement becomes relatively stable. In such cases, we posit that it is appropriate to proceed with the next punctuation-based attack process.\n\n| Cola         |              | Top-10 | Top-20 | Top-30 |\n| ------------ | ------------ | ------ | ------ | ------ |\n|              | Insertion    | 74.59% | 75.17% | 76.51% |\n| ELECTRA      | Displacement | 77.89% | 78.93% | 79.41% |\n|              | Replacement  | 50.60% | 59.31% | 63.92% |\n|     XLMR        | Insertion    | 76.51% | 83.41% | 85.81% |\n|\n|              | Replacement  | 15.20% | 18.14% | 19.61% |\n| QQP          | Insertion    | 26.86% | 30.61% | 32.68% |\n| DistillBERT2 | Displacement | 23.46% | 26.16% | 26.67% |\n|              | Replacement  | 14.77% | 16.87% | 17.93% |\n| QQP          | Insertion    | 29.21% | 31.81% | 35.55% |\n| DistillBERT2 | Displacement | 19.88% | 22.50% | 23.09% |\n|              | Replacement  | 24.46% | 26.48% | 27.54% |\n|              | Insertion    | 37.02% | 45.36% | 49.60% |\n| RoBERTa      | Displacement | 15.96% | 20.41% | 23.07% |\n|              | Replacement  | 25.58% | 33.11% | 36.89% |\n|              | Insertion    | 44.70% | 53.00% | 56.00% |\n| deBERTa      | Displacement | 26.74% | 32.65% | 35.41% |\n|              | Replacement  | 22.51% | 29.34% | 33.97% |\n\nQ2:In the experimental section, when comparing the proposed method with other approaches, a fixed number of updating steps is consistently utilized.\n\nA3:Throughout the main text, supplementary materials, and in all experiments included in the global rebuttal submission, we consistently employed an identical number of updating steps.\n\n\n\n\n\nQ3: The paper lacks specific information regarding the implementation details of adversarial training.\n\nA4:In our adversarial training, we maintain a 1:1 ratio between clean and adversarial samples during the training process. Specifically, we employ a learning rate of 0.0002 for our adversarial training, spanning 27 epochs. The optimization process utilizes the cross-entropy loss function employing the Adam optimizer. It is noteworthy that all methodologies adhere to identical adversarial training strategies.\n\n\n\nQ4: The evaluation of this attack is not comprehensive across a diverse set of NLP tasks\n\nTables B, C, and D present the results of our endeavor to extend punctuation-level attacks, TPPE, and TPPEP algorithms to encompass three distinct tasks: summarization, semantic similarity scoring, and text-to-image generation.\n\nIn the context of the summarization task, we employ the ROUGE-1 metric as the dependent variable y and formulate a predictive model to estimate the ROUGE-1 score for the test set, and subsequently, we select the candidate attack text characterized by the lowest predicted ROUGE-1 value.\n\nFor the text-to-image task, we utilize the CLIP score as y and identify the candidate attack text by determining the text associated with the smallest predicted CLIP score value. \n\nIn the semantic similarity scoring task, our y corresponds to the semantic similarity between the text before and after the attack. To model this, we develop a predictive model to estimate the semantic similarity values for the test set.\n\n​                                                   Table B  The results of text to image task\n\n|            | Without Attack | TOP-1  | TOP-3  | TOP-5  | Traveral |\n| ---------- | -------------- | ------ | ------ | ------ | -------- |\n| clip-score | 0.3278         | 0.3176 | 0.3069 | 0.3022 | 0.2610   |\n\n​                                                      Table C  The results of summarization\n\n|         | Without Attack | TOP-1 | TOP-3 | TOP-5 | Traveral |\n| ------- | -------------- | ----- | ----- | ----- | -------- |\n| ROUGE-1 | 11.69          | 10.91 | 9.65  | 9.11  | 5.22     |\n\n​                                          Table D  The results of semantic-similarity-scoring\n\n|                | Sentence-BERT | Distilbert |         |          |\n| -------------- | ------------- | ---------- | ------- | -------- |\n| ST12           | Peaeson       | Spearman   | Peaeson | Spearman |\n| Without Attack | 0.7990        | 0.6988     | 0.8056  | 0.7257   |\n| TOP-1          | 0.7874        | 0.6862     | 0.7902  | 0.7035   |\n| TOP-3          | 0.7760        | 0.6738     | 0.7759  | 0.6990   |\n| TOP-5          | 0.7654        | 0.6626     | 0.7649  | 0.6745   |\n| Traveral       | 0.6992        | 0.5832     | 0.6694  | 0.6048   |\n\nThe attack methodologies showcased in Tables B, C, and D predominantly encompass insertion-based techniques. Due to temporal constraints, we intend to incorporate the outcomes of alternative attack approaches in the revised version of this paper.\n\nMoreover, in order to comprehensively assess the effectiveness of our attack algorithm, we have included several additional evaluation metrics in Table 8 of the global rebuttal. In addition to the fooling rate, we have incorporated metrics such as Perturbed Words, Semantic Similarity, and Average Number of Queries. At present, our TPPEP algorithm has established state-of-the-art results across these metrics.\n\n\n\n"}, {"review_id": "8hInmzc1d1", "reviewer": "Reviewer_AjFS", "summary": "By proposing a new type of adversarial attacks, i.e., the punctuation-level attacks, This paper can fool text models with less impact on the semantic information by human beings understanding. Its effectiveness is verified by experimental results on various datasets/tasks and victim models. What’s more, the attack method is accelerated by the proposed Text Position Punctuation Embedding and Paraphrase (TPPEP) approach, so that the attack can be accomplished with constant time cost. The efficiency has been demonstrated by the experimental studies. \n", "questions": "As I stated, the results of text-to-image task are interesting, while it is not so convincing. At the least, how can we know if the attack is successful or not, by human eyes? There should be more convincing validation. \n\nOverall, I like the idea in the paper and the current drawback is the results. Please consider my suggestions about the experimental studies. I am open to increasing my score if my concerns are satisfactorily addressed.\n", "limitations": "The current discussion about the limitations is practical but not so comprehensive. For example, the resource required by the method is not discussed. \n", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "+ This paper is well written and easy to follow.\n\n+ This reviewer appreciates the novelty of the proposed punctuation-level attacks, and the motivation behind, i.e., attacking text models with minimal perceptual influence on human eyes. It indeed brings some insights.\n\n+ Besides its effectiveness in fooling various textual models, the authors also propose a TPPEP method to accelerate the attacking procedure. I believe this can significantly improve the practical use of the punctuation-level attacks.\n\n+ The attack results on the update-to-date Stable Diffusion model are quite interesting.\n", "weaknesses": "- Though the proposal of punctuation-level attacks is indeed well motivated, an obvious major concern is raised that only three tasks (thus three types of victim models) are selected for attack effectiveness evaluation in the experimental studies. It would be better if more methods are selected for evaluation. The proposed approach would be fully validated and the conclusion would be more convincing.\n\n- The current experimental results fail to provide more insights on how the method work in different scenarios. In fact, as a new type of text-attack method, there should be more in-depth analyses and explanations to quantitatively or qualitatively show the effectiveness.\n\n- The notations in the paper are not always consistent. And some parts are not well illustrated, especially in Sec. 3.3. It is suggested to be re-organized. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: It would be better if more methods are selected for evaluation.\n\nA1: Tables B, C, and D present the results of our endeavor to extend punctuation-level attacks, TPPE, and TPPEP algorithms to encompass three distinct tasks: summarization, semantic similarity scoring, and text-to-image generation.\n\nIn the context of the summarization task, we employ the ROUGE-1 metric as the dependent variable y and formulate a predictive model to estimate the ROUGE-1 score for the test set, and subsequently, we select the candidate attack text characterized by the lowest predicted ROUGE-1 value.\n\nFor the text-to-image task, we utilize the CLIP score as y and identify the candidate attack text by determining the text associated with the smallest predicted CLIP score value. \n\nIn the semantic similarity scoring task, our y corresponds to the semantic similarity between the text before and after the attack. To model this, we develop a predictive model to estimate the semantic similarity values for the test set.\n\n​                                                   Table B  The results of text to image task\n\n|            | Without Attack | TOP-1  | TOP-3  | TOP-5  | Traveral |\n| ---------- | -------------- | ------ | ------ | ------ | -------- |\n| clip-score | 0.3278         | 0.3176 | 0.3069 | 0.3022 | 0.2610   |\n\n​                                                      Table C  The results of summarization\n\n|         | Without Attack | TOP-1 | TOP-3 | TOP-5 | Traveral |\n| ------- | -------------- | ----- | ----- | ----- | -------- |\n| ROUGE-1 | 11.69          | 10.91 | 9.65  | 9.11  | 5.22     |\n\n​                                          Table D  The results of semantic-similarity-scoring\n\n|                | Sentence-BERT | Distilbert |         |          |\n| -------------- | ------------- | ---------- | ------- | -------- |\n| ST12           | Peaeson       | Spearman   | Peaeson | Spearman |\n| Without Attack | 0.7990        | 0.6988     | 0.8056  | 0.7257   |\n| TOP-1          | 0.7874        | 0.6862     | 0.7902  | 0.7035   |\n| TOP-3          | 0.7760        | 0.6738     | 0.7759  | 0.6990   |\n| TOP-5          | 0.7654        | 0.6626     | 0.7649  | 0.6745   |\n| Traveral       | 0.6992        | 0.5832     | 0.6694  | 0.6048   |\n\nThe attack methodologies showcased in Tables B, C, and D predominantly encompass insertion-based techniques. Due to temporal constraints, we intend to incorporate the outcomes of alternative attack approaches in the revised version of this paper.\n\nMoreover, in order to comprehensively assess the effectiveness of our attack algorithm, we have included several additional evaluation metrics in Table 8 of the global rebuttal. In addition to the fooling rate, we have incorporated metrics such as Perturbed Words, Semantic Similarity, and Average Number of Queries. At present, our TPPEP algorithm has established state-of-the-art results across these metrics.\n\n\n\nQ2:  As a new type of text-attack method, there should be more in-depth analyses and explanations to quantitatively or qualitatively show the effectiveness. \n\nA2: In the  global rebuttal, we have included discussions on defense mechanisms, comparative outcomes with other algorithms, and more. Our intent is for these experimental findings to offer additional insights into punctuation-level attacks.\n\n\n\nQ3: The notations in the paper are not always consistent.\n\nA3:  We greatly appreciate your valuable suggestions. We will meticulously review the notations throughout the entire manuscript. In the revised version, we will also reorganize Section 3.3 in accordance with your recommendations.\n\n\n\n\n\nQ4:As I stated, the results of text-to-image task are interesting, while it is not so convincing. \n\nIn the global rebuttal, we present the latest experimental outcomes for the text-to-image task. We utilize the CLIP score as the metric for evaluating attacks and leverage the Pokemon Blip Captions dataset, splitting it into training and testing sets in an 8:2 ratio.\n\nIn the supplementary materials of the global rebuttal, we present the latest experimental findings pertaining to the text-to-image domain. Employing the clip-score as the metric for assessing the attacks, the outcomes within Table 3 elucidate the impact of attacks conducted at the punctuation level. Notably, following these attacks, the clip-scores experienced a reduction from 0.3281 and 0.4040 to 0.2484 and 0.3468, respectively."}, {"review_id": "EioyKofGrp", "reviewer": "Reviewer_rnAT", "summary": "The paper proposes a novel mode of textual attack, punctuation-level attack, which aims to fool text models by conducting punctuation-level perturbations, including insertion, displacement, deletion, and replacement. This paper also introduces Text Position Punctuation Embedding (TPPE) as an embedding method and Text Position Punctuation Embedding and Paraphrase (TPPEP) as a search method to reduce the search cost and time complexity of determining optimal positions and punctuation types for the attack. The paper demonstrates the effectiveness of the punctuation-level attack and the proposed methods on various tasks such as summarization, semantic-similarity scoring, and text-to-image tasks.", "questions": "How do you get the E_text, and E_punc? Do you use pretrained models or a finetuned one on the target task? The embeddings you extract are from the embedding layer or the last layer of the transformer blocks?\n\nFor E_punc, do you just input a single punctuation as the complete sentence into the model? If so, this is quite strange.\n\nWhat are the dimensionalities for E_text, E_pos and E_punc and how can they be concatenated or added up?\n\nYou mentioned “We transform the attacking task into a paraphrasing task.” In line 269 when introducing TPPEP. What is the model used to conduct this paraphrasing task?\n\nYou mentioned “The adversarial candidate text with the highest paraphrasing score calculated by 272 the TPPEP method is chosen to deploy the attack.” In line 271-272. It seems that you need to do several queries and give them a rank. Is this contradictory to the O(1) complexity you claimed?\n", "limitations": "see the weakness part", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed punctuation-level attack expands the scope of adversarial textual attacks. By focusing on punctuation-level perturbations, the authors provide an approach to fooling text models while minimizing its impact on human perception.\n2. The proposed methods, TPPE and TPPEP, not only enhance efficiency but also reduce computational costs. Additionally, the authors present a comprehensive mathematical analysis of these approaches.\n3. The effectiveness and versatility of the proposed punctuation-level attack are demonstrated by the experimental results on various datasets and state-of-the-art (SOTA) models.\n", "weaknesses": "1. Whether LLMs can also be fooled, which should be discussed in the paper\n\nI guess such robustness is due to the amount of training data is not large enough. It is curious that LLMs like ChatGPT will still fall into such a deficiency, since LLMs are trained on huge data.\n\n2. Why PLMs fail on punctuations is not discussed\n\nThe punctuation-level attack does not surprise me a lot. The most interesting problem is to probe into the reason why PLMs can be fooled by punctuations.\nUnfortunately, this is not in the paper, which large limits the contribution of the work.\n\n3. The defense is not discussed\n\nThe authors do not provide the study on how to defense the punctuation-level attack. The contribution on how to enhance language models e.g. DeBERTa to be robust against punctuation attacks is much more significant than how to attack. To me, this is very important for the community to improve real-world systems against underlying attackers.\n\n4. The attack success rate is only promising on CoLA, while limited on the other datasets.\nHowever, CoLA is not suitable dataset for evaluation.\n\nCoLA requires the model to decide whether the given sentence is linguistically correct. The manipulation of punctuations can spoil the label of the original sentence.\n\n5. Punctuation modifications can change the original label on some task\n\nPunctuation-level attack is safer than word-level modification. It still can change the original label on some task, e.g. CoLA. Even in extreme cases, punctuations can change the entire semantics. It is better for the authors to discuss this part in the paper, e.g. some bad cases. This is also important for future research.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: Whether LLMs can also be fooled\n\nA1: Due to time constraints, we focused our efforts on the summarization task using ChatGPT, limiting our use to a mere 20 samples per task. Our  attack method involved the insertion of individual punctuation. Following the application of this attack strategy, the ROUGE-1 score exhibited a decrease from 22.8 to 16.7. Despite this decline, we assert that punctuation-level attacks remain effective for complex tasks such as generating  task. Our experimental outcomes illustrate the vulnerability of even extensively trained Stable-diffusion models to punctuation-based attacks. Additionally, existing literature has already demonstrated the susceptibility of LLMs to adversarial attacks on textual data.\n\n\n\nQ2: Why PLMs fail on punctuations\n\nA2: Analying the post-attack data distribution highlights the text altered by punctuation-based attacks as an Out-of-distribution (OOD) dataset for pre-trained language models (PLMs). For instance, in the Replacement attack method, instances involve replacing '!' with '！', '?' with '？', and ',' with '，'. Given the rarity of Chinese punctuation marks in the training data of English-based PLMs, sentences transformed by punctuation-based attacks become OOD samples for PLMs. Consequently, this contributes to reduced generalization performance of PLMs on such data. This aspect underlies the ineffective punctuation handling by PLMs.\n\nIn practical applications of PLMs, the prevailing approach involves training PLMs through fine-tuning. During the fine-tuning phase for downstream tasks, the training datasets for these tasks exhibit a lower proportion of punctuation errors. In theory, the finetuning process for downstream tasks could render the model more susceptible to OOD attacks, thereby further compromising the generalization capacity of PLMs for sentences that have undergone punctuation-level attacks.\n\n\n\nQ3: The defense is not discussed\n\nA3: In our global rebuttal, we have incorporated experimental outcomes to supplement our defense strategy.\n\nWe selected Preceding Language Modifier and  adversarial training  as defense.  The efficacy of the Preceding Language Modifier and adversarial training are demonstrated in Table 6 and Table 8.\n\n\n\n\n\nQ4: The attack success rates  limited. \n\nThe experimental results presented in this paper are obtained under highly challenging conditions, specifically the most extreme scenarios within the experiment setup. For instance, the experiments involve perturbing words to zero, conducting single punctuation attacks, allowing only a single query, and considering models that provide hard-label outputs in black-box attack scenarios. This deliberate choice of experimentation parameters has led to constrained outcomes. In Table 8, we have exclusively constrained the number of perturbed words for other benchmark methods. Despite the potential for other benchmark methods to access the model multiple times, obtaining posterior probabilities from black-box model outputs, and even in the context of white-box attacks, our TPPEP algorithm has demonstrated superior performance in terms of fool rate. Notably, across metrics including Perturbed Words (%), Semantic Similarity, and Number of Queries, TPPEP has consistently achieved wonderful  results across the entire dataset when compared to these benchmarks.\n\nWe have also investigated the impact of increasing the number of model queries on the fool rate. Our findings reveal that when the  number of qurried increase to five queries (TOP-5), TPPEP achieves state-of-the-art misclassification rates across all datasets. \n\n\n\n\n\nQ5: Punctuation modifications can change the original label and the entire semantics on some task.\nIn the revised edition, we would employ other text classification datasets to substitute cola\nIn Table 8, we delve into the alterations in semantics resulting from punctuation-level attacks. We computed an average semantic similarity of 0.995526417 before and after punctuation attacks. The specific outcomes are in Table 8. Our current experimental findings suggest that our punctuation-level attack algorithm has a limited impact on semantics.\nIn the revised version, we will incorporate modifications to prevent the emergence of extreme cases. Specifically, for the set of perturbed candidate texts, we will introduce a semantic similarity threshold. Any candidate text falling below this threshold will be excluded from consideration as a potential attack candidate. \n\n\n\n\n\nQ6: Questions about  E_text, and E_punc\n\nWe trained a substitute model and employed the embeddings of its final layer as the output. For both individual text and standalone punctuation inputs, we fed them into the substitute model, extracting the outcomes from its final layer, denoted as E_text and E_punc, respectively.\n\n\n\nGiven the pivotal role that punctuation types and placements play in attacks at the punctuation level, we sought to underscore the significance of punctuation. Consequently, we deliberately employed a singular punctuation mark as an entire sentence input into the model.\n\n\n\nThe dimensions of E_text, E_pos, and E_punc are all set to 50. We opted for the addition method.\n\n\n\nOur TPPEP method draws inspiration from paraphrasing tasks, which involve assessing the relationship between two given texts. In the case of TPPEP, we leverage pre- and post-adversarial attack embedding vectors to determine the success of the attack.\n\n\n\nDuring the testing phase, a single access to the substitute model is sufficient to obtain the ranking of the perturbed text, which indicates our time complexity of O(1).\n\nFurther elaboration on the embedding process and complexity is provided in the supplementary materials. Due to the word limitations, please  refer to our supplementary materials for additional details.\n\n\n\n\n\n\n\n\n\n\n\n\n"}]

Author rebuttal:
Upon receiving the reviewers' feedback, several sections have been incorporated to refine the original paper.

# **1 Defense Methord**



We have introduced an in-depth discourse concerning defense strategies aimed at countering

punctuation-level attacks. In the realm of real-world systems, we meticulously investigate a range

of defense approaches both pre- and post-completion of training.

## 1.1 **Preceding Language Modifier**

For after-trained models, initiating a retraining process with adversarial training methods is evidently costly and impractical. To address this scenario, we propose a modifier that aims to restore the text to its original form as much as possible following punctuation-level attacks. An alternative approach that involves training a seq2seq model using both punctuated and original texts post-attack could be considered. However, due to time constraints, we employ prompt learning using the grammatically enhancing large language model CoEdIT-xxl (11 B) as the modifier. We opt to conduct experiments on the most vulnerable CoLA dataset and the two strongest attack mode , e.g. Insertion and  Displacement. The experimental outcomes, as depicted in Table 6, demonstrate the favorable performance of our modifier strategy. As time limitations apply, results for other datasets will be presented in the revised version of this work.

## **1.2 Adversarial Training**

For untrained models, initiating a retraining process with adversarial training methods is evidently costly and impractical. To address this scenario, we propose a modifier that aims to restore the text to its original form as much as possible following punctuation-level attacks. An alternative approach that involves training a seq2seq model using both punctuated and original texts post-attack could be considered. However, due to time constraints, we employ prompt learning using the grammatically enhancing large language model CoEdIT-xxl (11 B) as the modifier. We opt to conduct experiments on the most vulnerable CoLA dataset. The experimental outcomes, as depicted in Table 6, demonstrate the favorable performance of our modifier strategy. As time limitations apply, results for other datasets will be presented in the revised version of this work.

# 2  Comparision to Benchmarks
We conducted a comparative analysis against alternative attack algorithms, namely TextFooler, Bert-attack, and DeepWordBug, which were chosen as benchmarks.And Fool Rate  , Perturbed Words (%) , Semantic Sim, and Number of queries are Selected as evaluation metrics.  T5-large pre-train model is selected as the tool to caculate Semantic Sim. And the “Label” indicate the output of query is hard-label, and “score” indicate the output of query is predict probabilities. 

 As the TPPEP algorithm perturbs solely a single punctuation mark, for the purpose of a fairer comparison, we imposed the constraint that other algorithms are also allowed to target only one word, albeit with the ability to attack this word multiple times. The experimental findings, presented in Table 8, demonstrate that the TPPEP has consistently achieved state-of-the-art results across various metrics, including Fool Rate, Perturbed Words, Semantic Similarity, and Average Number of Queries. Evidently, our approach at the punctuation level attains improved fool rates in scenarios involving zero-word perturbation, single-query access, preservation of semantic information, and reduced perceptibility, when compared to alternative algorithms.

# 3 **Experiments On Text-to-Image Task**

We have expanded the quantitative experimentation section for T2I applications, employing the clip-score as our evaluation metric. The experimental outcomes are presented in Table 9, where "ori-text1" refers to the sentence "a corgi is playing piano, oli on canvas". Upon inserting a comma for punctuation-level attack, "adv-text1" is obtained from "ori-text1". And "ori-text0" refers to the sentence “a professional photograph of an astronaut riding a triceratops”Notably, with a high semantic similarity of 0.9843 between "ori-text1" and "adv-text1", the clip-score decreases from 0.4040 to 0.3468 post-attack.

Additionally, in order to assess the robustness of the Stable Diffusion model against punctuation-level attacks, we employ the "pokemon-blip-captions" dataset for evaluating the attack's impact. This dataset comprises a total of 832 texts in its training and testing subsets. Following punctuation insertion attacks, the overall clip-score declines from 0.3273 to 0.2590. This reduction is further reflected in both the training and testing subsets, underscoring the model's vulnerability.

# 4 Analysis of Punctuation-Level Attacks

We examine the factors influencing the success rate of punctuation-based attacks. In our textual adversarial perturbations, punctuation information and its positional context are essential components. Thus, we delve into the impact of positional information and punctuation characteristics on the average Fool Rate.

Table 10 illustrates the effects of various punctuation types on the Fool Rate, detailing the top eight punctuation marks with the highest average misclassification rates. The "?" symbol emerges as the most influential, exhibiting an elevated average misclassification rate of 8.36%. This signifies that, across the cola, QQP, and wanli datasets, as well as their respective six models, when the "?" symbol is iteratively inserted, deleted, replaced, or displaced in a specific sample, the average probability of successful fooling reaches 8.36%.

In Table 11, the term "preceding" pertains to the initial one-third position of the sentence, while "subsequent" refers to the final one-third position of the sentence. Concerning insertion attacks, the most effective outcome is observed when the middle portion of the inserted sentence is targeted. In the context of replace and displace attacks, the subsequent segment of the attacked sentence yields optimal results.

Meta-review: I will suggest acceptance for this work, mainly because reviewers appreciated the paper’s extensive analysis (LLfX) and the novelty of the punctuation based approach (y7qU, AjFS). However, my certainty on this paper is somewhat lower than usual.

Questions were raised in the reviewing period regarding baselines and comparisons to other kinds of attacks (LLfX), perturbation semantics (LLfX), strength of evaluation of the methods (LLfX, y7qU, AjFS), number of attack iteration (y7qU). Some reviewers also mentioned that ideas for mitigations would have been interesting to include or discuss (LLfX, rnAT), as would LMs (rnAT)---this AC agrees that these would be interesting, but their non-inclusion does not warrant rejection in my view (although it would probably be prudent to remove references to ChatGPT since that model isn’t addressed).

In the rebuttal, authors shared new comparisons to other attacks and perturbation semantics (with human eval). Authors also discussed the number of attack iterations in the rebuttal, and these results suggested convergence. Authors also offered some defense against punctuation attacks. The amount of additional work presented in rebuttal led LLfX to suggest another round of review might be warranted, but they did not change their score.

One reviewer (rnAT) was an outlier on the low end, and they were very confident. Although I agree with them that the choice of tasks was a bit unusual and that label change, particularly on CoLA should be mentioned, several of their suggestions outside the scope of the work in my opinion (e.g. “The punctuation-level attack does not surprise me a lot. The most interesting problem is to probe into the reason why PLMs can be fooled by punctuations. Unfortunately, this is not in the paper, which largely limits the contribution of the work.”).
Decision: Accept (poster)