Title: Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text Models
Venue: NeurIPS 2023
Accepted: yes
Keywords: Punctuation-level Attack

Textual Adversarial attack

Natural Language Processing
Abstract: The adversarial attacks have attracted increasing attention in various fields including natural language processing. The current textual attacking models primarily focus on fooling models by adding character-/word-/sentence-level perturbations, ignoring their influence on human perception. In this paper, for the first time in the community, we propose a novel mode of textual attack, punctuation-level attack. With various types of perturbations, including insertion, displacement, deletion, and replacement, the punctuation-level attack achieves promising fooling rates against SOTA models on typical textual tasks and maintains minimal influence on human perception and understanding of the text by mere perturbation of single-shot single punctuation. Furthermore, we propose a search method named Text Position Punctuation Embedding and Paraphrase (TPPEP) to accelerate the pursuit of optimal position to deploy the attack, without exhaustive search, and we present a mathematical interpretation of TPPEP. Thanks to the integrated Text Position Punctuation Embedding (TPPE), the punctuation attack can be applied at a constant cost of time. Experimental results on public datasets and SOTA models demonstrate the effectiveness of the punctuation attack and the proposed TPPE. We additionally apply the single punctuation attack to summarization, semantic-similarity-scoring, and text-to-image tasks, and achieve encouraging results.

Punctuation-level Attack: Single-shot and Single Punctuation Attack Can Fool Text Models

Abstract

The adversarial attacks have attracted increasing attention in various fields including natural language processing. The current textual attacking models primarily focus on fooling models by adding character-/word-/sentence-level perturbations, ignoring their influence on human perception. In this paper, for the first time in the community, we propose a novel mode of textual attack, punctuation-level attack. With various types of perturbations, including insertion, displacement, deletion, and replacement, the punctuation-level attack achieves promising fooling rates against SOTA models on typical textual tasks and maintains minimal influence on human perception and understanding of the text by mere perturbation of single-shot tion Punctuation Embedding and Paraphrase (TPPEP) to accelerate the pursuit of optimal position to deploy the attack, without exhaustive search, and we present a mathematical interpretation of TPPEP. Thanks to the integrated Text Position Punctuation Embedding (TPPE), the punctuation attack can be applied at a constant cost of time. Experimental results on public datasets and SOTA models demonstrate the effectiveness of the punctuation attack and the proposed TPPE. We additionally apply the single punctuation attack to summarization, semantic-similarity-scoring, and text-to-image tasks, and achieve encouraging results.

1 Introduction

Deep Neural Networks (DNNs) have achieved tremendous success in the NLP community and have spawned a series of well-known applications such as ChatGPT [13]. However, more attention should be paid to the vulnerability of NLP models under adversarial attacks to protect them. Adversarial examples have been shown to have a devastating impact in the image domain [39, 12, 38]. Unlike the image domain, text is a discrete domain [45], which means that any small perturbations, such as sentence, word, or even character perturbations in text, would be easily perceived by humans. It is difficult to achieve imperceptible perturbations. Additionally, the text is a non-differentiable domain, which results in traditional optimization-based methods being ineffective in the NLP domain. Recent research on textual adversarial attacks can be mainly classified into character-level, wordlevel, sentence-level, and multi-level attacks [41]. Character-level attacks modify words by inserting, deleting, misspelling, replacing, or swapping characters. These attacks are easily detectable by humans and some apps can correct them automatically. However, character-level attacks require multiple queries to determine which characters to attack based on function gradient or score. Wordlevel attacks modify words by adding, deleting, or replacing important words in a text. Similar to character-level attacks, word-level attacks also require multiple queries to determine which words to

Most people probably consider, even though the courts didn't actually find, Klaus guilty, of murder.  (

Unacceptable:96.6%

)

Most people probably, consider even though the courts didn't actually find, Klaus guilty of murder.   (

Unacceptable:93.0%

)

Most people probably consider even though the courts didn't actually find, Klaus guilty of murder.

(

Unacceptable:84.4%

)

Most people probably consider, even though the courts didn't actually find

，

Klaus guilty of murder.(

Unacceptable:98.8%

)

Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder.    (

Acceptable:99.7%

)

Original text

Insertion

Displacement

Deletion

Replacement

attack. Both character- and word-level attacks are iterative and thus may be time-consuming in real NLP applications due to multiple queries required for each attack iteration. Sentence-level attacks modify entire texts by paraphrasing or adding meaningless sentences to input texts. The significant perturbations caused by sentence-level attacks may sometimes change the ground truth of input text. Perturbations in character-, word-, and sentence-levels can cause semantic gaps from the original text and are easily detectable by humans.

Considering the issues mentioned above, we propose punctuation-level attacks as a new method of textual adversarial attack. Compared with other attack methods, perturbations on punctuations are less perceivable than character- and word-level attacks, more difficult to detect than character-level attacks, and cause less impact on ground truth than other types of attacks. In previous research, little attention has been paid to punctuation; several researchers have inserted punctuations into words to create out-of-vocabulary (OOV) words, which is a specific method of character-level attacks. On the contrary, we first propose punctuation-level attacks that regard punctuation perturbations as a new level of attack similar to character-, word-, and sentence-level attacks. We define basic attacking methods and analyze why these methods can fool text functions. Unlike other types of attacks, punctuation-level attacks only modify punctuations by inserting, displacing, deleting, or replacing them without perturbing characters, words, and sentences. Punctuation-level attacks can deceive text functions by inserting, displacing, deleting, or replacing punctuations. Specific examples are presented in Fig. 1, showing that the single punctuation attack results in a dramatic change in the score of the predicted label. To deploy punctuation-level attacks, we need to consider which positions in the input text should be modified and which punctuations should be modified. For example, if there are k candidate punctuations and n tokens in the input text, there will be kn candidate attack texts when we insert one punctuation, k 2 n ( n + 1) candidate attack projects when we insert two punctuations, and k 3 n ( n +1)( n +2) candidate attack projects when we insert three punctuations. The time complexity of multiple punctuation attacks can be reduced using a greedy algorithm or a beam search. However, even considering only a single punctuation attack is still time-consuming.

To address this problem, we propose a method called Text Position Punctuation Embedding (TPPE) to reduce search cost. The feature extraction processing of input text x is also the embedding processing of text x , which is time-consuming for multiple queries. Therefore, we propose Text Position Punctuation Embedding (TPPE) to quickly and reasonably embed adversarial candidate text x adv . Let us illustrate it with the Insertion attack mode. When we apply Insertion attack to punctuation p into the i -th position of the input text, we query the text function f ( x ) or the substitute function f sub ( x ) and gain the f fe ( x ) as the embedding of text E text and f fe ( p ) as the embedding of punctuation p . The embedding of punctuation p only needs to be queried once and can be used repeatedly. As for the embedding of position, we adopt the embedding of the static position based on the calculation of sine and cosine from Transformer [40]. And we define the embedding of the adversarial candidate text x adv by concatenating or adding the embedding of text x , punctuation p and position i . We can iteratively attack the input text x and quickly obtain the adversarial text x adv after training the classification model from the TTPE of x adv to the prediction of x adv by f ( x ) . In addition, we propose a search method Text Position Punctuation Embedding and Paraphrase (TPPEP) to achieve a single-shot attack. We analyze the most information-lacking scenario for TPPEP: zero query, black-box text function, hard-label output, single punctuation limitation, and single-shot attack. To achieve the goal of zero query and gain more information, we train a substitute function f sub to fit the text function f . Then, we query the substitute function f sub to obtain the embedding of

text E text . We apply the TPPE method to obtain the embedding of the adversarial candidate text x adv , denoted as E x adv . We then transform the attacking task into the paraphrasing task. E x adv and E text are concatenated as input data, and the result of attack (successfully attack, donated as label 1, otherwise, donated as label 0) is the predicting label. We choose the adversarial candidate text with the highest paraphrasing score calculated by the TPPEP method and deploy the single punctuation attack in a single-shot manner accordingly.

The contributions can be summarized as follows. (1) Punctuation-level attacks : We first propose punctuation-level attacks, which regard the perturbations of punctuation as a systematic attack like character-level, word-level, and sentence-level attacks. We propose four primary modes of punctuation-level attacks and explain punctuation-level attacks from the perspective of optimal perturbations. (2) TPPE : We first propose t he TPPE embedding method t o decrease t he search cost. We reduce the query time complexity from O ( kn ) of Insertion , O ( nt ) of Displacement , O ( t ) of Deletion , and O ( kt ) of Replacement , to O (1) under single punctuation attack. It can quickly and reasonably embed the adversarial candidate text x adv using a single-shot query. (3) Single-shot and Single Punctuation Attack : To make our punctuation-level attack more imperceptible, we modify only one punctuation. Besides, we discuss single-punctuation attacks in the most challenging scenario: zero query, black-box function, hard-label output, one-punctuation limitation, and single-shot attack, which is the closest to the real-world scenarios. We correspondingly propose the TPPEP method and achieve promising experimental results.

2 Related Works

Textual adversarial attack . Text adversarial attack methods in the literature can be broadly categorized into two classes: white-box attacks and black-box attacks. In white-box attacks, some methods [23, 35, 34, 1, 6] employ FGSM [12] to identify the words or characters with the most significant impact on text-related tasks. For example, TextFool [23] introduces a gradient loss function within the model to assess the influence of specific words or characters on task performance. Suranjana et al. [35] consider firstly trying to remove the adverb as a word-level attack, and use a gradient loss function in the model to achieve the attack. AdvGen [1] focuses on the similarity between the original words and their substituted counterparts to target neural machine translation (NMT) models. Moreover, some methods rely on directional derivatives [9, 8]. For instance, Hot-flip [9] manipulates text function by altering characters based on directional derivative gradients. Ebrahimi et al. propose controlled attacks, an extension of HotFlip that enables targeted attacks [8]. In black-box attacks, specific methods aim to insert nonsensical sentences into the input text with the intention of deceiving the text model [19, 42, 4]. Notable approaches include AddSent [19] and AddSentDiverse [42]. AddSent incorporates meaningless sentences into the input text to confound comprehension system models [19]. Meanwhile, AddSentDiverse broadens the pool of adversarial candidate texts by varying the placements of inserted sentences [42]. Other strategies involve the manipulation of words and sentences [3, 29, 11, 22, 28, 2, 5, 44, 31]. For example, Textbugger [22] alters words by substituting, removing, adding, or rearranging them to deceive the text model. Furthermore, certain methods are centered on paraphrasing the input text [18, 32]. SCPNs [18] employs an encoder-decoder architecture to generate new paraphrased versions of the input text.

Punctuation attacks . To the best of our knowledge, there is a scarcity of research in the existing literatures on punctuation attacks. Most related studies primarily concentrate on punctuation perturbation, specifically the placement of punctuation marks within or at the end of words. This approach can be classified as a character-level a ttack. For example, Hosseini et a l. insert punctuation into words to transform them into OOV words [17]. In another work by Hofer et al. [15], they also regard inserting a single comma as a special character-level attack, and they insert a comma at the end of the important word to translate this important word into an OOV word. In addition, a special symbol text attacks (SSTA) method is proposed in [23], which is viewed as an adjunct of TextFool [10] by adding fifty tildes behind the text.

3 Methodology

3.1 Punctuation-level Attacking

The Proposed Punctuation-level Attacks . We propose four modes of attacking at the punctuationlevel and analyze their time complexity for traversal search. These modes are as follows:

Insertion : Punctuation p is inserted into the target text to fool the text model.

Displacement : Punctuation p is moved from position i to position j in the target text.

Deletion : Punctuation p is removed from the target text.

Replacement : Punctuation p i is replaced by p j in the target text.

We denote the number of candidate punctuations by k , the number of punctuations in the input text by t , and the number of input tokens by n . In Insertion attack, there are n candidate positions, and each position can be inserted by k candidate punctuations. So the complexity of Insertion by traversal search is O ( kn ) . Regarding the m times of Insertion attacks, one text will generate kn candidate texts after a single punctuation attack, and the complexity of Insertion to each candidate is k ( n + 1) , thus the complexity of multiple Insertion attacks is O ( k m m -1 ∏ i =0 ( n + i ) ) . If the

greedy algorithm is applied for the m times of Insertion attacks, the generated kn candidate texts after the single punctuation attack will be selected with the greatest change in predicted score, and the number of positions is increased to n +1 . Thus, the complexity of Insertion to this selected candidate is O (0 . 5 k (2 n + m -1)) . The complexity of Displacement by traversal search is O ( nt ) because there are t candidate punctuations and each position can be displaced by n candidate positions. As for the m times of Displacement attacks, one text will generate nt candidate texts after the single-shot attack, and the complexity of Displacement to each candidate is nt . So the complexity of m times of Displacement attacks is O (( nt ) m ) . If the greedy algorithm is applied for m times of displacement attacks, there are still t candidate punctuations and n candidate positions. So the complexity of m times of Displacement by the greedy algorithm is O ( mnt ) . The complexity of Deletion by traversal search is O ( t ) because there are t candidate punctuations that can be deleted. In the context of multiple occurrences of Deletion attacks, each attack leads to the removal of a candidate punctuation. Consequently, the complexity of m consecutive Deletion attacks can be expressed as O ( m -1 ∏ i =0 ( t -i ) ) , where t represents the total number of possible candidates. If the

greedy algorithm is applied for m times of Deletion attacks, due to every Deletion attack, the text will lose one punctuation, thus the complexity of m times of Deletion by the greedy algorithm is O (0 . 5 m (2 t +1 -m )) . The complexity of Replacement by traversal search is O ( kt ) because there are t candidate punctuations and each punctuation can be replaced by k candidate punctuations. Regarding the m times of replacement attacks, there are still k candidate punctuations and t punctuations in the input text after a single-shot attack. Thus, the complexity of m times of Replacement attacks is O (( kt ) m ) . If the greedy algorithm is applied for m times of Deletion attacks, the complexity of m times of Replacement by the greedy algorithm is O ( mkt ) . All complexity is shown in Table 1.

Single punctuation Attack . The target text x consists of n tokens, noted as x = [ x 1 , · · · , x n ] . According to the Universal Approximation Theorem [16], we can define a function F ( x ) to fit the text function f ( x ) by a continuous nonconstant, bounded, and monotonically increasing function ϕ ( · ) as

where v m , b m ∈ R and w m ∈ R D . Universal Approximation Theorem demonstrates the fact that if our attack can result in vicious impact in f ( x ) , it will also fool the text function F ( x ) . In the

following, we will analyze how punctuation-level attacks influence the text function F ( x ) based on the basic linear unit w ⊤ m x + b m of f ( x ) as

We compare character-level, word-level, and punctuation-level attacks under the constraint of a single-element attack, i.e., perturbing one character, one word, or one punctuation. In character-level attacks, a single-character attack impacts only one token x i , making it OOV. In many text functions, the OOV token is embedded as an identical token x oov . In this scenery, w ⊤ m x + b m can be written as

Similar to character-level attacks, a single-word attack impacts only one token x i . Since there are few candidate tokens x can that are semantically similar to the token x i when considering the similarity of input text with and without a word-level attack, w ⊤ m x + b m can be rewritten as

Regarding the proposed punctuation attacks, we discuss single punctuation attacks in terms of Insertion , Displacement , Deletion , and Replacement respectively. Taking Insertion as an example, when we insert punctuation p into position i of the target text, w ⊤ m x + b m will change to

According to Eq. (3) and (4), the character-level and word-level attacks only perturb one token and one word by the single punctuation attack. According to Eq. (5), the single Insertion attack does not perturb any words but perturbs the i -th token by substituting x i with punctuation p and all tokens after the ( i -1) -th position of the input text by moving them one bit backward, which totally perturbs n -i +2 tokens. The result of Displacement , Deletion , and Replacement is presented in Table 2. According to Table 2, the punctuation-level attack can impact more tokens, indicating that it can cause greater perturbation than character- and word-level attacks. If we insert, displace, or delete punctuation in the appropriate position, it will result in a more significant impact on the basic linear w ⊤ m x + b m of f ( x ) compared to character-level and word-level attacks. Meanwhile, a single punctuation attack may apply a more vicious impact to f ( x ) and F ( x ) . Compared to words, punctuation is thinner, smaller, and more imperceptible. Due to the nature that punctuationlevel attack does not perturb any word, semantically, the adversarial text is more imperceptible for human beings. Furthermore, the Insertion , Displacement , Deletion , and Replacement of the punctuation produces less impact on the semantic information by human beings understanding.

The Ultimate Effectiveness of Punctuation-level Attacks . According to Eq. (3), (4), (5), characterand word-level attacks under a single-element limitation may focus on perturbing token values. However, Insertion , Displacement , and Replacement attacks can not only perturb token values but also token positions which may result in a vicious impact on f ( x ) . This position perturbation almost has no impact on semantic information for human beings.

We discuss why multiple punctuation-level attacks can fool the text model by taking Insertion attacks as an example. In the text model, there exists the maximum length of the input text, and we denote it as N . If the length of the input text is greater than N , the text model f considers only the first N tokens. If we apply N Insertion attacks, there will be k N candidate texts. The probability of fooling the text model is equal to 1 minus the probability of all k N candidate texts failing to fool

the text model. We denote the probability of fooling the text model as p fool and the probability of failing to fool the text model by the i -th candidate text as p i , i ∈ [1 , K N ] . And we denote the maximum value of p i as p max . We make an assumption about p max here. Due to the fact that p i is the probability of the i -th candidate text before querying the text model f , and the attacking result is unknown, we have 0 < p max < 1 . Correspondingly, we have

According to Eq. (6), 1 -p k N max will approach 1 as k increases. As a probability, we have 1 -p k N max ≤ p fool ≤ 1 . Consequently, p fool will tend to 1 when k increases, which means multiple Insertion attacks can ultimately fool the text model with k increasing.

3.2 Embedding and Search Methods

According to Table 1, punctuation-level attacks require considerable cost to deploy. Even considering only the single punctuation attack, it is still time-consuming. Taking the mode of Insertion as an example, its original time complexity is O ( kn ) . Therefore, it is essential to decrease the search cost. If we have reduced the search cost of a single punctuation attack, the time complexity of multiple-punctuation attacks can be reduced by using either a greedy algorithm or a beam search. The n is determined by the target text x , which is not feasible to optimize. As k increases, the search space expands, resulting in a higher fooling rate of Insertion attacking. Therefore, to achieve a higher fooling rate, we should not reduce the value of k .

As we cannot decrease k and n , we focus on a specific process: text function f ( x ) . For example, in a text classification task, function f ( x ) = softmax f fe ( x ) , where the processing of feature extraction is denoted by the function f fe and softmax is used as the final classification layer. The classification cost t 0 can be divided into two parts: the feature extraction consumption t 1 and the final classification consumption t 2 . The classification time t 2 is transient compared to t 1 and therefore can be ignored. Consequently, we have t 0 ≈ t 1 .

Embedding the adversarial candidate text x adv perturbed by various configurations requires multiple queries, and thus it is time-consuming. Therefore, we propose TPPE to quickly and reasonably embed the adversarial candidate text x adv . According to Eq. (5), a single punctuation attack consists of three components: input text x , perturbed position, and candidate punctuation. To reduce the cost of embedding processing for adversarial text x adv , we refine its embedding process by combining the embeddings of text, position, and punctuation. Again, we take the Insertion attack mode as an example. When we insert punctuations p into the i -th position of the target text, we query text function f ( x ) (white-box) or substitute function f sub ( x ) (black-box), obtaining E text as the embedding of text and f fe ( p ) as the embedding of punctuation p . The embedding of punctuation p only needs to be queried once. For the embedding of the position, we adopt the embedding of the static position based on the calculation of sine and cosine from the Transformer [40], present in Eq. (7). The variable pos represents the position and the variable i represents the dimension.

After calculating embeddings of input text, punctuation, and position, we calculate candidate attacking text embeddings using two steps: 1) concatenating three embeddings as candidate attacking text embedding E all = Concat ( E text , E pos , E punc ) and, 2) calculating candidate attacking text embedding using mathematical operations presented in Table 3.

The Insertion mode inserts punctuation p into the i -th position of input text, so we regard E text + E pos + E punc as the embedding of x adv . The Displacement mode displaces punctuation p from position i to position j in the input text, so we regard E text -E i pos + E j pos + E punc as the embedding of x adv . The Deletion mode deletes punctuation p from the i th position of the input text, so we compute E text + E pos -E i punc + E j punc as the embedding of x adv . The Replacement mode replaces punctuation p i with p j in the input text, so we take E text + E pos -E i punc + E j punc . TPPE requires only single-shot embedding of the input text x , which means we decrease the query time complexity from O ( kn ) of Insertion , O ( nt ) of Displacement , O ( t ) of Deletion , and O ( kt ) of Replacement , to O (1) under single punctuation attack.

In our study, we assume the worst-case of applying punctuation-level attacks: the victim model is a black-box model where only prediction labels are available instead of scores from the function. In this scenery, we train a substitute function f sub [30] to transform the black-box scenario into a white-box scenario by collecting part of training datasets X . Specifically, we query the text function and derive the prediction labels y pre , then train the substitute function f sub using the input text and the label y pre as paired data. After training f sub , we transform the black-box scenario into a white-box scenario.

Directly querying f sub to determine which punctuation should be used is also time-consuming for multiple queries. We iteratively attack the input text x and quickly gain the adversarial text x adv after training the classification model from the TPPE of x adv to the label of x adv by f ( x ) . Additionally, we propose a search method called TPPEP to achieve a single-shot attack. We analyze the worst-case scenario for TPPEP: zero query, black-box function, hard-label output, single punctuation limitation, and single-shot attack. The specific pseudo code of TPPE and TPPEP are presented in Alg. 1 and Alg. 2 in the Appendix.

To achieve the goal of zero query and gain more information, we train a substitute function f sub to fit the text function f . Then, we query the substitute function f sub to gain the embedding of text E text . We apply the TPPE method to gain the embedding of adversarial candidate text x adv , denoted as E x adv . We transform the attacking task into a paraphrasing task. Specifically, E x adv and E text are concatenated as input data, and the result of the attack (the successful attack denotes label 1; else the label 0) is the predicting label. The adversarial candidate text with the highest paraphrasing score calculated by the TPPEP method is chosen to deploy the attack.

3.3 Analysis of TPPEP

In this section, we discuss a specific and practical form of classification function that can deceive the text model. We assume that there is a function f ( x ) in hypothesis space F . Since f ( x ) = softmax ( f fe ( x )) , we hypothesize that there exists a function f 1 that fits the function softmax and satisfies | f 1 ( x ) -̂ y x | < ϵ 1 . A simple functional form of f 1 is a function of arg max softmax . Furthermore, we propose the hypothesis that there exists a function f 2 , which can predict ̂ y x adv from TPPE ( x adv ) and satisfy | f 2 ( x adv ) -̂ y x adv | < ϵ 2 . We also define a function f 3 ,

by | f 2 ( x ) -f 1 ( x ) | . We calculate the absolute difference between f 2 ( x adv ) and f 1 ( x ) and compare the absolute difference between | f 2 ( x adv ) -̂ y x adv | and | ̂ y x adv -̂ y x | in Eq. (8) as

According to Eq. (8), when ̂ y x adv = ̂ y x , the actual label of TPPEP is 0 and | f 2 ( x adv ) -f 1 (( x )) ≤ ε 1 + ε 2 . Since ε 1 and ε 2 are sufficiently small, f 3 ( f 2 ( x adv ) , f 1 ( x )) will be 0 , allowing us to classify them correctly. As f 1 ( x ) and f 2 ( x ) fit well ̂ y x adv and ̂ y x , when ̂ y x adv and ̂ y x are not equal, f 3 ( f 2 ( x adv ) , f 1 ( x )) will be 1 . Thus, we have found a specific fooling function f 3 ( f 2 ( x adv ) , f 1 ( x )) , which implies that there is at least one fooling function in the hypothesis space F . Moreover, we discuss that f 3 ( f 2 ( x adv ) , f 1 ( x )) can be learned based on the Universal Approximation Theorem [16]. According to Eq. (1), we can train F 1 and F 2 to fit f 1 and f 2 , which satisfy | F 1 ( x ) -f 1 ( x ) | < ϵ 3 and | F 2 ( x adv ) -f 2 ( x adv ) | < ϵ 4 . We analyze the difference between | f 2 ( x adv ) -f 1 ( x ) | and | F 2 ( x adv ) -F 1 ( x ) | in Eq. (9). If the difference between them is less than 1 , then f 3 ( F 2 ( x adv ) , F 1 ( x )) and f 3 ( f 2 ( x adv ) , f 1 ( x )) will predict the same label.

According to Eq. (9), we compare the difference between | f 2 ( x adv ) -f 1 ( x ) | and | F 2 ( x adv ) -F 1 ( x ) | . As ε 3 and ε 4 are sufficiently small, | F 2 ( x ) -F 1 ( x ) | will approximate | f 2 ( x ) -f 1 ( x ) | , suggesting that we can fit | ̂ y x adv -̂ y x | by | f 2 ( x ) -f 1 ( x ) | . Based on the above analysis, we prove that a feasible solution exists in the hypothesis space and can be trained.

4 Experiment

4.1 Experimental Setting

Tasks and Datasets . We have implemented punctuation-level attacks and TPPEP in the context of three distinct text-to-label tasks, utilizing three diverse datasets. Furthermore, we have reported the outcomes of these experiments concerning text summarization, semantic-similarity-scoring, and textto-image tasks, focusing particularly on the single-shot punctuation attack. The tasks that underwent single-shot attacks and TPPEP perturbations encompassed text classification (TC), paraphrasing, and natural language inference (NLI) tasks. The TC task takes a text as input, and the text model generates a predicted label for the input text. For our TC task, we have opted to utilize the CoLA dataset [43]. CoLA is the dataset of a binary classification task that aims to predict whether the input sentence is 'correct' from a linguistic perspective. As for the paraphrase task, the input consists of a pair of texts, and the objective of the text model is to ascertain the semantic similarity between the two texts. Our choice for the paraphrasing task is the QQP dataset [21]. Furthermore, the NLI task involves predicting the relationship between pairs of input sentences and we employ the Wanli dataset [25].

Experiment Setup . We have applied both punctuation-level attacks and TPPEP to target pretrained SOTA models, including ELECTRA [7], XLMR [37], DistilBERT1 [36], RoBERTa [27], and DeBERTa [14]. In numerous real-world text API application scenarios, the API solely provides the predicted label for the input text. Therefore, our experiments are tailored for black-box attacks, where the text model exclusively generates categorical labels in response to queries. The conventional perturbation budget of character-/word-/sentence-level attacks is typically assessed based on the number of altered words. In contrast, our punctuation-level attacks perturb zero word, and we define the perturbation budget as the number of perturbed tokens. In the experiment, we set the strictest perturbation limit, specifically focusing on the perturbation of a single punctuation. Thus, we discuss the results in the case of a single punctuation attack. In addition, to make our TPPEP work in hard-label attacks, we first train a substitute function f sub [30] by querying the text function and deriving the prediction labels y pre . Then, we apply the proposed TPPEP method to search for the most vulnerable candidate text.

4.2 Experimental Results

Text classification task, paraphrase task, and natural language inference task . The result of the TC task is presented in Table 5. Our punctuation-level attack shows encouraging results on ELECTRA [7] model and Cola dataset with 90 . 80% and 93 . 67% fool rates respectively by singleshot Insertion attack. On all three datasets, Insertion consistently obtains the highest fool rate due to the largest search space compared to other attacks such as Deletion , which has the smallest search space and thus obtains the lowest fool rate. The fool rate of Displacement and Replacement depends more on the specific datasets and text models. The results of the traversal

search imply that a single punctuation attack can fool those text models. The results of Top-1 present the result of single-shot and single punctuation attack. Our experiments show that single-shot and single punctuation attacks are more effective for datasets with small average search space (ASP) such as the CoLA dataset. In the case of ELECTRA model and CoLA dataset, single-shot and single punctuation attack obtains 67 . 40% , 36 . 05% , 5 . 18% , and 24 . 64% respective fool rate by Insertion , Displacement , Deletion , and Replacement attack. The fool rate of the single-shot and single punctuation attack decreases with increasing search space but still achieves a fool rate of 8 . 44% when the ASP is 1161 . 12 in the Wanli dataset.

Semantic-similarity-scoring task, summarization task, and text-to-image task . In addition, we have expanded the scope of single punctuation attacks to encompass semantic-similarity-scoring (sss), summarization, and text-to-image tasks (T2I). Datasets from STS-B [26] and gigaword [20] datasets were chosen for the sss and summarization tasks. Results of single punctuation attacks are tabulated in Tables 6 and 7. In the sss task, the Insertion punctuation attack leads to reductions of Pearson and Spearman correlations for Sentence-BERT and Distilbert, resulting in reductions of 0.0998, 0.1156, 0.1062, and 0.1209, respectively. For the summarization task, we utilize the ROUGE-1 metric as the dependent variable, denoted as y . Then we formulate a predictive model to estimate the ROUGE-1 score for the test set. Subsequently, we select the candidate attack text with the lowest predicted ROUGE-1 value. The Insertion punctuation attack decreases the ROUGE-1 score [24] of gigaword [20] by 6.47. In the sss task, y represents the semantic similarity between the text before and after the attack. To model this, we have developed a predictive model to estimate semantic similarity values for the test set. This significant decline demonstrates the susceptibility of sss and summarization models to single punctuation attack.

In the T2I task, we determine the CLIP score denoted as y and select the candidate attack text with the lowest predicted CLIP score. Stable Diffusion V2 [33] is selected as the victim model, and the results are illustrated in Fig. 2. When we input the sentence 'a professional photograph of an astronaut riding a triceratops', the Stable Diffusion V2 generates the 'correct' image containing both the astronaut and triceratops. However, after inserting a period into the input text, Stable Diffusion V2 generates an image without triceratops. This dramatic change also happens in another input sentence. Furthermore, we have expanded the section dedicated to quantitative experiments in T2I applications,

Ori-output

Adv-output

Prorsfosraiont.

Adv-text: a professional photograph of an astronaut. riding a triceratops Ori-text: a professional photograph of an astronaut riding a triceratops

Ori-output

Adv-output

Adv-text: a corgi is playing, piano, oil on canvas Ori-text: a corgi is playing piano, oil on canvas

utilizing the CLIP score as our evaluation metric. The experimental results are displayed in Table 4, with 'ori-text1' denoting the sentence 'a corgi is playing the piano, oil on canvas.' By inserting a comma as a punctuation-level attack, 'adv-text1' is derived from 'ori-text1'. Significantly, despite a high semantic similarity of 0.9843 between 'ori-text1' and 'adv-text1', the CLIP score decreases from 0.4040 to 0.3468 after the attack. Furthermore, to evaluate the resilience of the Stable Diffusion model against punctuation-level attacks, we utilize the 'pokemon-blip-captions' dataset. Following the punctuation insertion attacks, the overall CLIP score decreases from 0 . 3273 to 0 . 2591 . This reduction is consistently observed in both the training and testing subsets, highlighting the model's susceptibility. In the present the results of our endeavor to extend punctuation-level attacks, TPPE, and TPPEP algorithms to encompass three distinct tasks: summarization, semantic similarity scoring, and text-to-image generation show the effectiveness of TPPE and TPPEP.

5 Conclusion

In this work, we propose a punctuation-level attack based entirely on inserting, displacing, deleting, and replacing punctuation to fool text models. Our punctuation-level attack is more imperceptible to human beings and has less semantic impact compared to character-/word-/sentence-level attack. We introduce TPPE and TPPEP methods to reduce the search cost and achieve the time complexity of the query as O (1) . Furthermore, we discuss the ultimate effectiveness of punctuation-level attacks and present the analysis of the TPPEP method. Experimental studies on representative datasets of SOTA victim models demonstrate the effectiveness of our proposed methods.

Acknowledgment

This work was supported in part by the National Key R&D Program of China (Grant No. 2022ZD0118100), in part by the National Natural Science Foundation of China (Grant No. 62372480), in part by the Guangdong Basic and Applied Basic Research Foundation (No. 2023A1515012839), in part by ARC-Discovery grant projects (DP 190102261 and DP220100800), and in part by the Shenzhen Science and Technology Program (No. JSGG20220831093004008), and sponsored by CCF-Tencent Rhino-Bird Open Research Fund (No. CCF-Tencent RAGR20230118).

References

Abdullah Al-Dujaili, Alex Huang, Erik Hemberg, and Una-May O'Reilly. Adversarial deep learning for robust detection of binary encoded malware. In SPW , pages 76-82, 2018.

Moustafa Alzantot, Yash Sharma Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. Generating natural language adversarial examples. In EMNLP , page 2890-2896, 2018.

Yonatan Belinkov and Yonatan Bisk. Synthetic and natural noise both break neural machine translation. In ICLR , 2018.

Matthias Blohm, Glorianna Jagfeld, Ekta Sood, Xiang Yu, and Ngoc Thang Vu. Comparing attention-based convolutional and recurrent neural networks: Success and limitations in machine reading comprehension. In CoNLL , pages 108-118, 2018.

Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, and Yew Soon Ong. Metamorphic relation based adversarial attacks on differentiable neural computer. arXiv preprint arXiv:1809.02444 , 2018.

Yong Cheng, Lu Jiang, and Wolfgang Macherey. Robust neural machine translation with doubly adversarial inputs. In ACL , pages 4324-4333, 2019.

Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. Electra: Pre-training text encoders as discriminators rather than generators. In ICLR , 2020.

Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On adversarial examples for character-level neural machine translation. In COLING , pages 653-663, 2018.

Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In ACL , pages 31-36, 2018.

Brian Formento, See-Kiong Ng, and Chuan-Sheng Foo. Special symbol attacks on nlp systems. In IJCNN , pages 1-8, 2021.

Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In SPW , pages 50-56, 2018.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR , 2015.

Roberto Gozalo-Brizuela and Eduardo C Garrido-Merchan. Chatgpt is not all you need. A State of the Art Review of large Generative AI models. arXiv preprint arXiv:2301.04655 , 2023.

Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. In ICLR , 2021.

Nora Hofer, Pascal Schöttle, Alexander Rietzler, and Sebastian Stabinger. Adversarial examples against a bert absa model-fooling bert with l33t, misspellign, and punctuation. In ARES , pages 1-6, 2021.

Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks , 2(5):359-366, 1989.

Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. Deceiving google's perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138 , 2017.

Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. Adversarial example generation with syntactically controlled paraphrase networks. In NAACL-HLT , pages 1875-1885, 2018.

Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In EMNLP , pages 2021-2031, 2017.

Akhil Kedia, Sai Chetan Chinthakindi, and Wonho Ryu. Beyond reptile: Meta-learned dotproduct maximization between gradients for improved single-task regularization. In EMNLP , pages 407-420, 2021.

Huong T Le, Dung T Cao, Trung H Bui, Long T Luong, and Huy Q Nguyen. Improve quora question pair dataset for question similarity task. In RIVF , pages 1-5, 2021.

Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. In NDSS , 2019.

Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. Deep text classification can be fooled. In IJCAI , pages 4208-4215, 2018.

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out , pages 74-81, 2004.

Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. WANLI: Worker and AI collaboration for natural language inference dataset creation. In EMNLP , pages 6826-6847, 2022.

Fangyu Liu, Ivan Vuli´ c, Anna Korhonen, and Nigel Collier. Fast, effective, and self-supervised: Transforming masked language models into universal lexical and sentence encoders. In EMNLP , pages 1442-1459, 2021.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019.

Pasquale Minervini and Sebastian Riedel. Adversarially regularising neural nli models to integrate logical background knowledge. CoNLL , pages 65-74, 2018.

Tong Niu and Mohit Bansal. Adversarial over-sensitivity and over-stability strategies for dialogue models. In CoNLL , pages 486-496, 2018.

Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In ACM ASIACCS , pages 506-519, 2017.

Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In ACL , pages 1085-1097, 2019.

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Semantically equivalent adversarial rules for debugging nlp models. In ACL , pages 856-865, 2018.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In CVPR , pages 10684-10695, 2022.

Ishai Rosenberg, Asaf Shabtai, Lior Rokach, and Yuval Elovici. Generic black-box end-to-end attack against state of the art api call based malware classifiers. In RAID , pages 490-510, 2018.

Suranjana Samanta and Sameep Mehta. Generating adversarial text samples. In ECIR , pages 744-749, 2018.

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 , 2019.

Vésteinn Snæbjarnarson and Hafsteinn Einarsson. Cross-lingual qa as a stepping stone for monolingual open qa in icelandic. In MIA , pages 29-36, 2022.

Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. TEVC , 23(5):828-841, 2019.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR , 2014.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS , pages 5998-6008, 2017.

Xuezhi Wang, Haohan Wang, and Diyi Yang. Measure and improve robustness in nlp models: A survey. In NAACL , pages 4569-4586, 2022.

Yicheng Wang and Mohit Bansal. Robust machine comprehension models via adversarial training. In ACL , pages 575-581, 2018.

Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Neural network acceptability judgments. TACL , 7:625-641, 2019.

Huangzhao Zhang, Hao Zhou, Ning Miao, and Lei Li. Generating fluent adversarial examples for natural languages. In ACL , pages 5564-5569, 2019.

Wei Emma Zhang, Quan Z Sheng, Ahoud Alhazmi, and Chenliang Li. Adversarial attacks on deep-learning models in natural language processing: A survey. TIST , 11(3):1-41, 2020.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel mode of textual attack, punctuation-level attack, which aims to fool text models by conducting punctuation-level perturbations, including insertion, displacement, deletion, and replacement. This paper also introduces Text Position Punctuation Embedding (TPPE) as an embedding method and Text Position Punctuation Embedding and Paraphrase (TPPEP) as a search method to reduce the search cost and time complexity of determining optimal positions and punctuation types for the attack. The paper demonstrates the effectiveness of the punctuation-level attack and the proposed methods on various tasks such as summarization, semantic-similarity scoring, and text-to-image tasks.\n\nSTRENGTHS:\n1. The proposed punctuation-level attack expands the scope of adversarial textual attacks. By focusing on punctuation-level perturbations, the authors provide an approach to fooling text models while minimizing its impact on human perception.\n2. The proposed methods, TPPE and TPPEP, not only enhance efficiency but also reduce computational costs. Additionally, the authors present a comprehensive mathematical analysis of these approaches.\n3. The effectiveness and versatility of the proposed punctuation-level attack are demonstrated by the experimental results on various datasets and state-of-the-art (SOTA) models.\n\nWEAKNESSES:\n1. Whether LLMs can also be fooled, which should be discussed in the paper\n\nI guess such robustness is due to the amount of training data is not large enough. It is curious that LLMs like ChatGPT will still fall into such a deficiency, since LLMs are trained on huge data.\n\n2. Why PLMs fail on punctuations is not discussed\n\nThe punctuation-level attack does not surprise me a lot. The most interesting problem is to probe into the reason why PLMs can be fooled by punctuations.\nUnfortunately, this is not in the paper, which large limits the contribution of the work.\n\n3. The defense is not discussed\n\nThe authors do not provide the study on how to defense the punctuation-level attack. The contribution on how to enhance language models e.g. DeBERTa to be robust against punctuation attacks is much more significant than how to attack. To me, this is very important for the community to improve real-world systems against underlying attackers.\n\n4. The attack success rate is only promising on CoLA, while limited on the other datasets.\nHowever, CoLA is not suitable dataset for evaluation.\n\nCoLA requires the model to decide whether the given sentence is linguistically correct. The manipulation of punctuations can spoil the label of the original sentence.\n\n5. Punctuation modifications can change the original label on some task\n\nPunctuation-level attack is safer than word-level modification. It still can change the original label on some task, e.g. CoLA. Even in extreme cases, punctuations can change the entire semantics. It is better for the authors to discuss this part in the paper, e.g. some bad cases. This is also important for future research.\n\nQUESTIONS:\nHow do you get the E_text, and E_punc? Do you use pretrained models or a finetuned one on the target task? The embeddings you extract are from the embedding layer or the last layer of the transformer blocks?\n\nFor E_punc, do you just input a single punctuation as the complete sentence into the model? If so, this is quite strange.\n\nWhat are the dimensionalities for E_text, E_pos and E_punc and how can they be concatenated or added up?\n\nYou mentioned “We transform the attacking task into a paraphrasing task.” In line 269 when introducing TPPEP. What is the model used to conduct this paraphrasing task?\n\nYou mentioned “The adversarial candidate text with the highest paraphrasing score calculated by 272 the TPPEP method is chosen to deploy the attack.” In line 271-272. It seems that you need to do several queries and give them a rank. Is this contradictory to the O(1) complexity you claimed?\n\nLIMITATIONS:\nsee the weakness part", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBy proposing a new type of adversarial attacks, i.e., the punctuation-level attacks, This paper can fool text models with less impact on the semantic information by human beings understanding. Its effectiveness is verified by experimental results on various datasets/tasks and victim models. What’s more, the attack method is accelerated by the proposed Text Position Punctuation Embedding and Paraphrase (TPPEP) approach, so that the attack can be accomplished with constant time cost. The efficiency has been demonstrated by the experimental studies.\n\nSTRENGTHS:\n+ This paper is well written and easy to follow.\n\n+ This reviewer appreciates the novelty of the proposed punctuation-level attacks, and the motivation behind, i.e., attacking text models with minimal perceptual influence on human eyes. It indeed brings some insights.\n\n+ Besides its effectiveness in fooling various textual models, the authors also propose a TPPEP method to accelerate the attacking procedure. I believe this can significantly improve the practical use of the punctuation-level attacks.\n\n+ The attack results on the update-to-date Stable Diffusion model are quite interesting.\n\nWEAKNESSES:\n- Though the proposal of punctuation-level attacks is indeed well motivated, an obvious major concern is raised that only three tasks (thus three types of victim models) are selected for attack effectiveness evaluation in the experimental studies. It would be better if more methods are selected for evaluation. The proposed approach would be fully validated and the conclusion would be more convincing.\n\n- The current experimental results fail to provide more insights on how the method work in different scenarios. In fact, as a new type of text-attack method, there should be more in-depth analyses and explanations to quantitatively or qualitatively show the effectiveness.\n\n- The notations in the paper are not always consistent. And some parts are not well illustrated, especially in Sec. 3.3. It is suggested to be re-organized.\n\nQUESTIONS:\nAs I stated, the results of text-to-image task are interesting, while it is not so convincing. At the least, how can we know if the attack is successful or not, by human eyes? There should be more convincing validation. \n\nOverall, I like the idea in the paper and the current drawback is the results. Please consider my suggestions about the experimental studies. I am open to increasing my score if my concerns are satisfactorily addressed.\n\nLIMITATIONS:\nThe current discussion about the limitations is practical but not so comprehensive. For example, the resource required by the method is not discussed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new approach to textual attacks called the punctuation-level attack. The method aims to fool text models while minimizing its impact on human perception and understanding. The paper discusses the effectiveness of this attack strategy, presents a search method to optimize its deployment, and provides experimental results showcasing its success. The authors also apply the single punctuation attack to summarization, semantic-similarity-scoring, and text-to-image tasks, achieving encouraging results. The paper concludes that the punctuation-level attack is more imperceptible to human beings and has less semantic impact compared to traditional character-/word-/sentence-level perturbations. The integrated Text Position Punctuation Embedding (TPPE) allows the punctuation attack to be applied at a constant cost of time. The experimental results on public datasets and state-of-the-art models demonstrate the effectiveness of the punctuation attack and the proposed TPPE.\n\nSTRENGTHS:\n1. The paper introduces a new approach to textual attacks called the punctuation-level attack, which is different from traditional character-/word-/sentence-level perturbations.\n2. The punctuation-level attack is designed to be more imperceptible to human beings and has less semantic impact compared to traditional perturbations.\n3. The paper presents a search method called Text Position Punctuation Embedding (TPPE) to optimize the deployment of the punctuation-level attack.\n4. The paper provides experimental results showcasing the effectiveness of the punctuation-level attack and the proposed TPPE on public datasets and state-of-the-art models.\n\nWEAKNESSES:\nThe adversarial attacks discussed in this paper can be categorized as non-pure white-box attacks, as the attack objective may differ from the model's evaluation metric. It is crucial to explicitly acknowledge this fact in the paper, as it is widely recognized that achieving white-box robustness represents an upper bound and is significantly more challenging than black-box robustness.\n\nIt appears that the number of attack iterations is restricted. To ensure robustness evaluation, it is advisable to ensure attack convergence by employing an adequate number of iterations.\n\nIn the experimental section, when comparing the proposed method with other approaches, a fixed number of updating steps is consistently utilized.\n\nQUESTIONS:\nThe paper lacks specific information regarding the implementation details of adversarial training. It does not provide explicit explanations regarding the ratio of clean versus adversarial samples used during training, nor does it clarify whether all methods employ identical training strategies.\n\nLIMITATIONS:\nThe main emphasis of the paper is placed on evaluating the punctuation-level attack's efficacy within specific tasks, including summarization, semantic-similarity-scoring, and text-to-image tasks. However, the evaluation of this attack is not comprehensive across a diverse set of NLP tasks, limiting the extent to which the findings can be generalized.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an adversarial attack against NLP models based on punctuation perturbations. The authors introduce an attack called Text Position Punctuation Embedding (TPPE) that comprises an insertion, displacement, deletion, and replacement attack based on textual punctuation (e.g., commas or periods).\nExperiments are conducted on various datasets, ranging from text classification (CoLA) to paraphrasing (QQP) and natural language inference (WANLI). The attacks are applied to ELECTRA, XLMR, and BERT-based models (DistilBERT, RoBERTa, DeBERTa). Additionally, the attack is applied to semantic-similarity-scoring (STS12), summarization (gigaword), and text-to-image tasks (prompting Stable Diffusion V2). Experimental results are promising, showing that the attack can be used to successfully attack models for the above tasks.\n\nSTRENGTHS:\n* The paper provides an extensive analysis of punctuation-level attacks against NLP models. These attacks are promising since they have the potential to be less perceptible as compared to existing character-, word-, and sentence-level attacks.\n* The analysis is extensive in that multiple NLP tasks (classification, summarization, text-to-image, etc.) are analyzed.\n* It is interesting to see that the investigated models are vulnerable to punctuation-level attacks across tasks and domains.\n\nWEAKNESSES:\n* The paper does not compare TPPE against existing works and baselines. In Section 2, the authors point out various existing works focusing on punctuation attacks. However, none of these works have been evaluated and compared against in their experimental settings. To identify and support the strengths and utility of TPPE, such experiments are essential.\n* I additionally think that comparisons to character-, word, and sentence-level attacks on the selected datasets would have been insightful since these experiments would provide the reader with a better understanding of how punctuation-level attacks perform in comparison to attacks focusing on other parts of a textual sequence.\n* The paper does not further analyze the semantics of perturbed adversarial examples. To support the claims of semantic imperceptibility, human experiments analyzing the change in semantics between an original sequence and its adversarial counterpart would be important. The examples in Figure 2 nicely illustrate that inserting single punctuation marks can substantially impact the meaning of a sequence. Since adversarial examples are desired to preserve the semantics of an attacked sequence, quantitative experiments would be needed to evaluate TPPE in that context.\n* The paper does not discuss potential approaches to mitigate the models’ vulnerability against punctuation-level attacks, for instance by assessing whether adversarial training / data augmentation (i.e., training the model on adversarially perturbed sequences) can help increase the robustness of the attacked models. This would provide additional insights into how robust the attack is, and how it can potentially be defended against.\n* The results for the text-to-image task consist only of two qualitative examples. These examples are highly interesting, but to better evaluate the vulnerability of Stable Diffusion models against such attacks, quantitative results over a larger dataset would be important. \n* Overall, the paper focuses too much on introducing the attack and discussing its details and time-complexity analysis, instead of extensively evaluating its performance (the Experiments Section spans under 2 pages in the manuscript).\n\nQUESTIONS:\nCould you provide a few additional details on model training / fine-tuning, as well as the dataset splits that you used for attacking the models? This is not mentioned in Section 4.1.\n\nLIMITATIONS:\nThe paper briefly discusses Limitations in Section 5. However, potential ethical considerations arising from this research remain unaddressed. Since discussing these is quite important in this context (the proposed attack can be misused for malicious purposes), I would encourage the authors to add a section for this.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}