PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
Authors: Linyang Li
Ke Ren
Yunfan Shao
Pengyu Wang
Xipeng Qiu
Venue: EMNLP 2023 Findings (2023)
Keywords: perturbation in NLP
robustness in NLP

Abstract: With the rapid development of neural network applications in NLP, model robustness problem is gaining more attention. Different from computer vision, the discrete nature of texts makes it more challenging to explore robustness in NLP. Therefore, in this paper, we aim to connect discrete perturbations with continuous perturbations, therefore we can use such connections as a bridge to help understand discrete perturbations in NLP models. Specifically, we first explore how to connect and measure the correlation between discrete perturbations and continuous perturbations. Then we design a regression task as a PerturbScore to learn the correlation automatically. Through experimental results, we find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation, surpassing previous methods used in discrete perturbation measuring. Further, the proposed PerturbScore can be well generalized to different datasets, perturbation methods, indicating that we can use it as a powerful tool to study model robustness in NLP.

PerturbScore:
Connecting Discrete and Continuous Perturbations in NLP
Linyang Li", Ke Ren; Yunfan Shao, Pengyu Wang, Xipeng Qiu
School of Computer Science, Fudan University
Shanghai Key Laboratory of Intelligent Information Processing, Fudan University (kren22, pywang22 @m. fudan. edu. cn [linyangli19, yfshaol9, xpqiu @fudan. edu.cn
Abstract
robustness problems mostly focus on exploring the model behavior when the inputs are perturbed. However, unlike the computer vision field, the discrete nature natural language makes it more challenging to define, construct and measure the perturbations added to the texts.
With the rapid development of neural network applications NLP, model robustness problem is gaining more attention. Different from computer vision, the discrete nature of texts makes more challenging explore robustness in NLP. Therefore, in this paper, we aim to connect discrete perturbations with continuous perturbations, therefore can use such connections as bridge to help understand discrete perturbations in NLP models. Specifically, first explore how to connect and measure the correlation between discrete perturbations and continuous perturbations. Then we design regression task as a PerturbScore to learn the correlation automatically. Through experimental results, we find that we can build connection between discrete and continuous perturbations and the proposed PerturbScore to learn such correlation, surpassing previous methods used discrete perturbation measuring. Further, proposed PerturbScore can be well generalized to different datasets, perturbation methods, indicating that we can it as powerful tool study model robustness in NL.P
Previous works usually separate two lines of work concerning discrete perturbations and continuous perturbations: In the computer vision (CV) field, continuous perturbations explored (Goodfellow et al., 2014) since studying perturbations can help improve model robustness (Madry et al., 2019) and generalization abilitics (Hendrycks and Dietterich, 2019; Hendryoks et al., 2021). As for perturbations in NLP, the difference and challenge in discrete perturbations constrain the development of model robustness in NLP. Jin et al. (2019); Zang al. (2020); Li et al. (2020) craft adversarial examples with synonyms as word-level perturbations, which is hard to measure whether the perturbations are imperceptible. Also, crafting these perturbations would face combinatorial explosion problem. As studying discrete perturbations is more challenging than continuous perturbations, it is intuitive to wonder: can find the correlations between discrete and continuous perturbations in NLP and study continuous perturbations instead?
widely
1 Introduction
Natural language processing (NLP) applications based on neural networks are developing rapidly, exemplified by applications based on pre-trained models (Devlin al., 2018) such as ChatGPT (Brown et al., 2020), machine translation systems (Bahdanau et al., 2014), question-answering systems (Rajpurkar et al., 2016). While they are growing at incredible speed, it is of great concern how Therefore, exploring model robustness in NLP is essential for future NLP developments. Model
In this paper, we aim to explore connections between discrete and continuous perturbations NLP models hoping that such connections can help studies of model robustness in NLP. We first give definitions and measuring standards of perturbations in discrete and continuous space and align the form and notations for studying their correlations. Then make several assumptions provide the possibility of connecting discrete and continuous perturbations. Specifically, we assume that discrete perturbations and continuous perturbations have similar effects on neural models when being added to the input to perturb the
trust these neural networks.
"Equal Contribution.
Corresponning author.
'We will release our code generated datasets https: htggttutbb..//.//// http://nneee9//eetrrbboboooo Phttps: / /openai.......////o// wtt.coll/loo/ooooooooooooooooooa http:///ll///ccttt.....

---

widely explored in the deep learning field: Goodfellow et al. (2014); Carlini and Wagner (2016) discussed the possibility of crafting gradient- based perturbations as adversarial examples to mislead neural models. Madry et al. (2019) introduces the projected gradient descent method to construct perturbations. Hendrycks and Dietterich (2019); Hendrycks et al. (2021) discussed more general perturbations such as Gaussian noise, blurs, etc. in the distribution shift scenarios. When the perturbations are continuous, studies focus on exploring connections between model robustness and model accuracy (Zhang et al., 2019a; Yang et al., 2020) and plenty of analytical works dive deep into the model robustness studies (Pinot et al., 2019). These robustness studies assume that the perturbations model. Therefore, we are able to search for continuous perturbation that can be considered as substitution for the discrete perturbation. We then introduce a method to quantify the correlations between discrete and continuous perturbations and design a regression task to automatically learn the correlation. Specifically, we use the gradient- projection descent method to search for minimum continuous perturbation that has similar effects the model behavior with the discrete perturbation. After quantifying the correlations between discrete and continuous perturbations, we use an additional neural network named PerturbScorer to learn such a correlation. That is, given the original input and discrete perturbation, we use neural network to predict its continuous perturbation range.
2 Related Work
2.1 Model Robustness and Perturbations
Robustness problems
We construct experiments on IMDB and AG's News datasets, which are widely used datasets NLP robustness studies, We first explore the correlations between discrete and continuous perturbations when they are used to perturb fine-tuned models such as BERT; then we test the performances of the learned network and empirically verify that the correlations between perturbations can be learned through neural network, providing evidence for researchers to study continuous perturbations in NLP as a substitute for discrete perturbations.
continuous, therefore, they are not suitable for discrete perturbations and NLP robustness studies.
2.2 Perturbations in NLP
In the NLP field, the robustness problem becomes more challenging due to the discrete nature texts. Ebrahimi et al. (2017) explores crafting character-level and word-level perturbations as adversarial examples to attack NLP models. Follow-up works such Jin et al. (2019); Zang et al. (2020); Li al. (2020) aim to find better methods to craft semsntic-preserving adversaries. As for more general perturbations, Jia and Liang (2017) explores how adding random sentences can mislead (2021) explores how adversarial training improves out-of-distributin model generalization problems. Unlike the continuous perturbations explored in the computer vision field, the NLP field rarely discusses how the model behaves in robustness against adversaries and generalization abilities. Zhu et al. (2019) introduces embedding-space gradient-based adversarial training and discovers that continuous space adversaries can help improve NLP model generalization abilities without further explanation. Li et al. (2021) founds that gradient-based adversarial training be used in defense against word-substitution attacks. In general, robustness studies in NLP rarely focus on finding the correlation between discrete and continuous perturbations which separate works vision and language fields.
Further, we design extensive analytical experiments and through the experimental results, we make several non-trivial conclusions: (1) we build connection between discrete and continuous perturbations; (2) such connection can be generalized and help improve model robustness and generalization abilitics; (3) continuous- spacc adversarial training is effective because it narrows the gap between discrete and continuous space.
systems; Yi et al.
To summarize, in this paper, explore the correlations between discrete and continuous perturbation, a fundamental challenge in robustness studies NLP. We provide detailed notations and make assumptions to explore the correlations berween perturbations and propose a method to connect these perturbations; further, we design PerturbScorer to learn such correlation; through experimental results, show that can connect discrete perturbations with continuous perturbations. We are hoping that the concept of studying discrete perturbations NLP through building the connections between continuous perturbations can provide hints for future studies.

---

3 Connecting Perturbations
cannot be easily measured. We use A(P(S)) measure the perturbation intensity of perturbation P(S) added to the original text S, which could be odit-distance, semantic shift, grammar change, etc. For instance, when we use edit-distance as A(P(S)), we assume that the fewer tokens the original text is replaced, the less the text is perturbed. Besides edit-distance, Jin et al. (2019) introduces USE (Cer et al., 2018) measure the perturbation intensity. We assume that the less semantic information is changed, the less the text is perturbed. general, finding an accurate measure strategy A(-) challenging since the standard can be diversified and subjective when measuring discrete perturbations.
3.1 Defining Perturbations
We first define perturbations in deep neural networks for NLP applications:
Given an imput text - [wop w1. ... Wn: ..... the corresponding embedding of S is The prediction
X = fi, ... ", of the input S is denoted as f(X). Here, we use the embedding output X as the model input since we aim to connect the discrete perturbations with continuous perturbations in the embedding space. When the input text is maliciously attacked or perturbed by random noise, the input text becomes S'. We use P(S) to denote the perturbation process therefore the perturbed text S' = + P(S). The perturbation function P(S) can various methods including adversarial attacks and random perturbations. Representative adversarial attack methods are word-substinution adversarial attacks such HotFlip (Ebrahimi et al., 2017), Textfooler (Jin et al., 2019) and BERT-Attack (Li et al., 2020). Unlike random perturbations such as random deleting or replacing words/characters, adversarial attack methods aim to find the minimum amount of character- or word-level substitutions that can mislead target models.
On the other hand, continuous perturbations can be measured by constraining the fy-norm ||ô||p the perturbations 6. The most common constraint is the fn-norm. Compared with measuring discrete perturbation, it is easy and straightforward to measure the continuous perturbation range.
3.3 Connecting Perturbations
As illustrated, challenging to measure discrete perturbations, which makes it more difficult to explore how perturbations affect the model behavior. Meanwhile, another challenge is that constructing discrete perturbations is also challenging, for instance, replacing discrete tokens in multi-token text with multiple candidates for each token is a combinatorial explosion problem. Therefore, since measuring and studying continuous perturbations is more convenient, instead of searching for methods to evaluate the shift caused by discrete perturbations, we aim to build a connection between discrete perturbations and continuous perturbations and explore how the continuous perturbation affects model behaviors instead. We hope that by connecting discrete perturbations to continuous perturbations, we can introduce new perspectives to NLP field model robustness problems.
Unlike in the computer vision field where continuous perturbations can be directly added the input, the continuous perturbations can only be added the embedding output X in the language field. For embedding output E Ritd with sequence length I and hidden size d, we have perturbed output X' X The continuous perturbation 5 E Ri-d can be random noise (Hendrycks and Dietterich, 2019) such Gaussian noise, blurs, pixelate, or adversarial perturbations. A representative method to generate adversarial perturbation ö is the Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014). Given a target model fo(-), the perturbation of sample S generated based on the gradients: = a sgn(V (VnVxxe x fo v)). Here, & is a hhper-yppaaee parameter controlling the perturbation range.
To build the connection between discrete perturbations and continuous perturbations, we make several assumptions:
3.2 Measuring Perturbations
Assumption 1. Continuous perturbations 8 added to X have similar effects on target model fe compared to discrete perturbations added S. That is, both types of perturbations can cause a model prediction shift, and stronger perturbations would cause more damage to the model.
After defining perturbations, it is important to measure how perturbations affect neural models.
Measuring the severity of the perturbation is a challenge in discrete text perturbations. The similarity between the perturbed and original texts

---

Perturbations
Perturbation Measure A(P(S), 8)
Ouiputs Shift fo(S) lo(S P(())
Texts AG's News
P(S)
Apple Recalls Barch of PowerBook Batteries:
Textfooler:
Apple, in couperation with US Consumer Product Safety Commission said would voluntarily rocall about
Apple > Mil
edit-distance:5 USE: 0.889
Batch >> Afar
fo BERT
28,000 rechorgeable hatteries in its 15-inch PowerBook Gf tolebooks.
cooperation cooperatives
Commodities
Score: 0.511
Sel/Tech (100%) Business (81
recall reminnds
Apple Recalls Batch of PowerBook Ratteries: Apple, cooperation
Random Perturbation: Apple Overcast
5ill-iistanne:5
Consumer Product Safety Commission said would voluntarily 28,000 rechargeable buiteries used its 15 inch PowerBook G4 notebooks.
cooperation >> striker
USE: 0,880
fe BERT Sai/Tech (100%) Sai/Teah (90%)
ahout
Consumer Concessions
BERTScore:
said rewanded
volumtarily
Table I: Selected samples with same edit-distance perturbations showing discrete perturbation constructions and measurements discrete perturbations. The model output shifts are different from diit-distance or SeRre
Assumption 2. Target model fo follows Lipschitz constraint: when ||
that under such norm-bound, there exists perturbation 5 satisfies Assumption mentioned above. Therefore, when S and P(S) is fixed, the goal is find norm-bound
6, [X + 6) [e(X)ll2 K Here, assume that in NLP models, such as a fine-tuned BERT, small perturbations in the embedding space do not cause severe damage to model outputs. Otherwise, the behavior change caused by input perturbations is hard to predict, and finding correlations between perturbations is more challenging.
[1fa(X + 8) -) fo())ll (1) 11812 Il.f((X 8)ll2 (X)ll
arg arg min max
Therefore, we would obtain a data tuple [S, ((S), e). which is the correlation of discrete and continuous perturbations. We are hoping that we can empirically verify that the data tuple can be connected, and verify the assumptions made above.
Assumption 3. For discrete perturbation P(S). there exist a continuous perturbation that satisfies: e - & < ||ō1|2 2 < e, here, e small interval. And such 8 satisfies:
Algorithm Obtaining norm-bound
[| + P(S)) fatsllla I + fol [1 P(S) fo(Sile 11 + 5) [Ix(X
Require: Inputs SSS),, label y, search step
Ta. norm range interval
here, à a hyper-parameter, and for simplification, fo(-) takes both discrete tokens S and embedding output of the discrete tokens as input, skipping the embedding process.
[ [ fx
1:
[e(S]
for e 0, 25, do
3:
5n +0 0
We assume that the continuous perturbation has a similar effect on model fe(-). therefore, when the absolute value of the gap between model shift caused by discrete and continuous perturbations is small, consider they are equal in perturbing neural models. Therefore, we can use continuous perturbations as an approximation of discrete perturbations by building connections between them.
for
0, 0, TI. do
5:
98 + VoL(fo(X + 8.),))
// Get Gradients
6, 11 (6, + |l0
9,
// Get Perturbation
9:
if xx
-r| < σ then
10:
return tuple (S,P(S),c)
I1:
else
12:
discard tuple
3.4 Quantify Connections
After assuming that build connections between discrete and continuous perturbations, we aim quantify such connections. For discrete perturbation PSS), we find the minimum continuous perturbation under Assumption 3. Specifically, we aim to find the proper norm-bound
standard projected-gradiet--dseees (PGD) (Madry al., 2019) method. As seen in Algorithm 1, we use multi-step gradient-descent to generate perturbations within the range e, which is the perturbation generation used in gradient-based
In practice, to obtain

---

adversarial training. Specifically, the notation used in line 7 constrain the perturbations within the norm bound c. In line 9, we pick the e that satisfies the assumption that there exists a continuous perturbation that has a similar effect on neural models compared with discrete perturbations. Therefore, once the perturbation d obtains ideal effects on the neural model (bigger than the effects caused by discrete perturbations), we consider the found is the proper one. A 02
26
Tuclookrr
SERT: Sbster
SERT-Teckoer
(a)
special case is that if the continuous perturbation
effect ( fa(X)]la is way bigger than r,
we simply drop the sample.
(c)
(d)
3.5 PerturbScorer
Figure 1: Average Model Shift of Discrete and
After constructing the quantification of correlations between discrete and continuous perturbations, we design a PerturbScorer to score the correlations. The perturbation ô is continuous variant, therefore, we formulate regression task to learn the range of e given S and PSS). We train the task the PerturbScorer ([S, (S)),, to measure the correlation between discrete and continuous perturbations in the target model fo()).
Continuous Perturbations. (a) is the curve P(S)))
and edit-distance, containing curves of neural model fo including FreeLB FreeLB-trained model and BERT fine-tuned model and perturbation method P(S) including Textfooler and Random perturbation tested on the IMDB dataset; (c) is the curve of [D((X+6.) [ x=I1fa(X))1z and norm-ball range, containing curves of B-ttaind model and BERT- fine-tuned model tested on the IMDB dataset; (b) and (d) are the corresponding results of the AG's News dataset.
Considering that the perturbation P(S) should be a small perturbation, we use a simple strategy that concatenates original texts and perturbations in P(S) as the input of the regression task to learn the correlation. We simply concat the perturbations behind the original token, (e.g.: .... would recall [reminds] ...). Such patterns help the model understand the perturbation of the original texts. Then we use the crafted inputs to predict the norm bounds of the correlated continuous perturbations.
For each input text S, we have multiple P(S) with different edit-distance differences.
In the random perturb method, we randomly replace token using a random word from a general vocabulary, which is the vocabulary used to obtain synonyms in the Textfooler method (Mrkšić et al.. 2016; Jin al., 2019). Similar the Textfooler perturbation method, we also collect multiple perturbations per text including different numbers, places of substitutes, and different substitutes.
4 Experiments
4.1 Dataset Construction
In generating continuous perturbations, we use the PGD method to find the minimum continuous perturbation that has a similar model prediction shift compared to the discrete perturbation. Specifically, we set the adversarial step To = 15 and the adversarial learning rate a = le 1, the norm-ball scarch interval of is set to 0.01 and the discard parameter ø is set 0.005.
To explore the correlations between discrete and several datasets widely used in exploring model robustness in NLP. We use the IMDB dataset (Maas et al., 2011) and the AG's News dataset (Zhang et al., 2015) which are text classification tasks with an average text length of 220 and 47 accordingly.
continuous perturbations,
We use two widely used perturbation methods P(S), Textfooler (Jin et al., 2019) and random- perturb. In the Textfooler method, we follow the standard generation process and save perturbations in multiple queries regardless of the attack result, which is different from its original usage that keeps finding perturbations until the attack successful.
In Figure 1, we draw curves exploring the connection between model output shifts and different levels of perturbations. As shown, we can observe that the average model output shifts caused by discrete and continuous perturbations show consistency with edit distance and norm-ball range. We observe that when the perturbations grow

---

Range of
[0.1e:11
(0.1.0.2)
(0.2.
(0.3,0.4
00.4.0.51
(0.5, [0.5.0.6]
(0.6.
TOTAL
Discurded
P(S)
(o(-)
Rund. Perturb
IMDB
FreeBB
4630
2265
1075
528
224
104
86
8912
1044
Textfonler
937
327
PreeLB
2004
1634
1990
1579
826
386
256
8675
1459
BERT
3889
200
47
9126
2126
News
FroSLB
1386
1426
2199
1932
1204
635
634
9421
794
BERT
1499
3688
8253
FreeLB 928
772
1420
1639
1425
916
1340
84400
1012
Table 2: Statistics of the pair number of constructed correlations between discrete and continuous perturbations.
larger, both discrete and continuous perturbations will cause more damage to neural models. Plus, the models trained by the Freel.B B method show better resistance against both discrete and continuous perturbations. These results verify Assumption and show that discrete and continuous perturbation can be correlated.
4.2 Evaluating Quantification of Correlation
After constructing the discrete perturbations and finding the corresponding of these discrete perturbations, we are able explore whether the discrete and continuous perturbations can be connected and show similar effects on neural networks. To evaluate the quantification process of correlations illustrated in Sec. 3.4, we use Kendall and Pearson correlation coefficient index to measure whether the discrete perturbations and the continuous perturbations
In Table 2, we count the tuple number of different ranges in the constructed data tuple of multiple datasets to show the connection between discrete and continuous perturbations. We observe that we only discard small proportion of collected data tuples, proving that we can successfully find norm ball e that satisfies Assumption 3 that such continuous perturbation 5 is equivalent to the discrete perturbations P(S) in interfering neural models f-).). We also observe that as the discrete perturbation P(S) is uniformly distributed in the edit-distance range from 30 in the IMDb Dataset and 15 the AG's News Dataset, the continuous perturbations mostly fall in the range that e < 2,-1, indicating that most discrete perturbations with different edit-distances (indicating different numbers of substitutions) only compares to minimum continuous perturbation. Therefore, learning the correlation between these discrete and continuous perturbations can help understand the discrete perturbations. Further, compared with the random perturbation, the Textfooler method generates more discrete perturbations that have more damage to model predictions and the corresponding continuous perturbations require larger norm balls, indicating that stronger discrete perturbations equal to larger continuous perturbations, providing the possibility connect discrete perturbations with continuous perturbations.
be connected.
The goal is to measure the correlation coefficient index such as Kendall and Pearson index between A(P(S)) and the selected norm-bound e. If the correlation between A(P(S)) and is large, can verify Assumption 3 that assumes there exists continuous perturbation that equals the discrete perturbation in interfering neural models. We use several simple A(-) including edit-distances, BERTScore (Zhang et al., 2019b) and USE (Cer et al., 2018). Here, BERTScore and USE measure the similarity between two sentences, which is reversed compared with edit-distance and perturbation scorer, therefore, use the opposite number of the BERTScore and USE score as A(-) measure the correlation coefficient index. Further, we can directly measure the correlation coefficient index between the predicted and the found exploring whether the PerturbScorer can learn the connection between perturbations, which supports the assumptions made above in Sec. 3.3 and provides powerful tool to quantify the discrete perturbations for robustness studies in NLP.
4.3 PerturbScorer Training
The training process of the PerturbScorer follows the standard fine-tuning process used fine-tuning regression tasks such as the STS-B (Cer al., 2017) dataset using huggingface Transformers
For the collected data, we select 80% data tuples as the training set and 20 % as the test set in training and testing the PerturbScorer.

---

Method Metric
Edit Distance
USE
PerterbScurer
Kendall
Spearmaon
Kendall
Spearmant
Kendall
Spearmann
Kendall
Spearmann
Dataset
P(S)
f+(
Rand. Perfurb
BERT
0.4891
0,635
0.5952
0.7745
0.577
0.755
0.7092
0,8648
Freel.R
0.599
0.7746
Textfooler
DERT
0.5153
0,6683
0.5497
0.7302
0.4932
0.6697
0.7975
0,9342
Froe18
0.5198
0.5532
0.7404
0.4932
0.8173
Rand, Perturb
BERT
0.4377
0.5785
0.4652
0.6352
0.4372
0.6031
0.7647
0.9141
AG's News
FreeLB
0.5063
0.6491
BERT
0.5055
0.6585
0.5279
0.7166
0.5013
0.6851
0.821
0.9477
Textfooler
FreeLB
Table 3: PerturbScorer evaluation results and correlation comparison with evaluators including BERTScore, USE, and Edit-Distance
(Wolf al.. 2019). We set the learning rate to 5e-5 with batch size set 64 and 128 for IMDB and AG's News datasets and use 4xNvidia 3090 GPUs to run the PerturbScorer training process.
design PerturbScorer explore whether the correlation can be learned and generalized by neural networks. By exploring the correlations, we can obtain non-trivial observations that can be helpful in model robustness in NLP:
4.4 Correlation Quantification Results
4.5.1 Lipsehitz Constraint Tightness
In Table 3, we list the correlation coefficient index of different measuring methods of perturbations and the PerturbScorer learned correlation of the perturbations:
As we observe in Figure 1, the model shift is in direct proportion to the perturbation range, indicating that the target model Sø follows Lipschitz constraint on general seale. Further, as seen in Table 3, compared with the model trained by the FreeLB method, is more challenging to study the correlation of the normal fine-tuned BERT as fe(-). indicating that gradient-based adversarial training helps build a tighter connection between discrete and continuous perturbations. providing perspective to explain why gradient- based adversarial training helps in improving robustness performances and generalization performances NLP tasks with discrete inputs (Zhu et al., 2019; Li et al., 2021).
We can observe that when we use scorers A(-) to measure the discrete perturbation, the correlation quantification results between the A(.) and the obtained continuous perturbation bound is not significant. In different setups including different datasets and perturbation methods, the Kendall correlation is smaller than 0.6 and the Spearman correlation smaller than 0.8, indicating that the discrete perturbation measuring methods do not have close correlations with the continuous perturbations that have a similar effect to neural models, further proving that these measuring methods cannot properly measure the damage to neural models.
4.5.2 PerturbScorer Generalization
In Table 3, we show that the correlation between discrete perturbation P(S) and continuous perturbations range of model fo(.) be learned by PerturbScorer MM-), further, we aim explore whether building such a correlation can be applied to various scenarios in robustness studies in NLP. That is, we explore the generalization ability of PerturbScorer M(.).. We explore whether the learned PerturbScorer M(-) based on target model fo(.) can be generalized to cross-dataset, cross-perturbation method P(S), cross-model fe(.), therefore, the application of the PerturbScorer and the concept of learning the correlation between discrete and continuous perturbations can used in various scenarios. We list thorough results in the Appendix.
On the other hand, can observe that when we use the PerturbScorer MM.) predict the corresponding continuous perturbations, the correlation scores are significant enough to prove that the PerturbScorer can learn the connection between the discrete perturbations and the continuous perturbations. Such results show that we can use our proposed PerturbScorer powerful tool to build connection between the discrete and continuous perturbations.
4.5 Analysis
As we first make assumptions about the correlations between discrete and continuous perturbations, we construct the data tuples and

---

PerturtScurer Testing Setup
PerturbScurer Training
Edit Distance
PerturbSeurer
P(8)
(-)-)
P(S)
fo(-)
Keodall
Spearmant
Kendall
Spearmann
Perturb
BERT
0.4557
IMDB
Rand Perturb
BERT
IMDB
Texitolerr
BERT
0.4891
0.635
0.5767
0.7532
Preel.R
IMDB
IMDB &
Rand. Pertur
BERT
Rand. Peruurb
BERT
0.4053
0.5035
0.7558
0.9047
Rand Perturb
Rand. Perturb
IMDB
BERT
JMD8
&
BERT
0.4406
0.5838
0.7802
0.9108
Textfonler
Table PerturbScorer generalization tests on cross dataset, perturbation type and model. We also test combined PerturbScorer trained with multi nulti-perturbations, multi-datasets and multi-model generated data tuples.
We can explore how PerturbScorer M(-) performs on different datasets or faces different types of perturbations:
tuples including different datasets, perturbations methods and neural models to explore a more generalized scenario.
As seen in Table 4, when we train a model using mixed data collected, we can build general PerturbScorer that can successfully predict the correlations between discrete perturbations and the corresponding continuous perturbation ranges. Such result shows that it is possible to build a general PerturbScorer that can be used in solving different datasets and perturbation types, showing that we can use continuous perturbations as a proxy for discrete perturbations when studying NLP robustness problems.
Cross-Perturbation PerturbScorer In cross- perturbation tests, observe that when train the PerturbScorer with random perturbations as P(S), the PerturbScorer can learn perturbations generated by textfooler, while textfooler-pe generreee perturbations cannot be well generalized. Such results show that we can collect multiple types of perturbations to train a PerturbScorer that can be generalized to recognize various discrete perturbations powerful tool to quantify how the discrete perturbations affect neural models.
Conclusion and Future Directions
Cross-Dataset PerturbScorer In cross-dataset tests, we observe that when we test the AG's News data tuples using the PerturbScorer trained with the IMDB dataset, the corelatiin is weakened but still stronger than correlations with edit distance, indicating that the PerturbScorer train can be generalized different datasets, showing that the connection between discrete and continuous perturbations is strong in general NLP systems, which provides possibilities using such correlations in various NLP robustness scenarios.
In this paper, we focus fundamental problem in robustness studies in NLP, which is the discrete nature of texts. The discrete nature isolates NLP robustness studies from well-studied machine lcarning fields, therefore, we introduce the concept of building connections between discrete and continuous perturbations as a new perspective to explore NLP robustness. We build PerturbScorer to learn the correlation between discrete and continuous perturbations and find that such PerturbScorer can learn the connection between perturbations, allowing us to use continuous perturbation ranges as a proxy constraint discrete perturbations, which avoids the challenge that discrete perturbations are hard to measure. Further, we find that our proposed PerturbScorer can be generalized to different datasets and perturbation methods, indicating that such process be further applied in the future in NLP robustness studies. For future directions, we aim explore more effective methods to build stronger PerturbScorer and to explore more broad scenarios to utilize the proposed PerturbScorer.
Cross-Model PerturbScorer In general, the correlation between discrete and continuous perturbations dependent on the neural model fo(-) since the perturbation range e calculated based on a certain model fe(.). However, when we test the generalization ability between different neural models fe(-), we observe that the correlation is still close. Therefore, it is possible to build more general PerturbScorer general metric to score the discrete perturbations.
Combined Scorer We further build a combined PerturbScorer that is trained by a mixture data

---

Limitations
valuable comments. Additionally, we sincerely thank Qipeng Guo for his valuable discussions and insightful suggestions on this study.
this work, we explore the discrete perturbation in robustness studies in NLP. We aim to find correlations between discrete perturbations and continuous perturbations since continuous perturbations easily measured and well-studies in the computer vision field. Our work makes assumptions that discrete perturbations show similar effect to neural networks compared with continuous perturbations, therefore, one limitation of such assumptions is that similar effect does strictly make two types of perturbations equal in nature. We find one perspective to connect the discrete perturbations and continuous perturbations, which is not the only solution. Future works can explore more strict constraints and find stronger connections between discrete and continuous perturbations.
References
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning align and translate. arXiv preprint urXiv:140.00473.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 1877-1111
Nicholas Carlini and David A. Wagner. 2016. Towards evaluating the robustness of neural networks. CoRR, 0as//608..46644
Also, better PerturbScorer designing and the applications based on correlations between discrete and perturbations and PerturbScorers can be further explored. We focus on defining and building the connection between discrete and continuous perturbations, and we do not explore further applications based on these connections and our proposed PerturbScorer. For instance, as the PerturbScorer can be used in scoring the discrete perturbations, it can be used in recognizing differences between sentences or measuring distribution shifts. Also, previous works explore robustness and generalization trade-offs and explainable robustness theories continuous space, mostly in the computer vision field, our works reveal the potential explore these problems NLP, which can be explored in future works.
Daniel Cer, Mona Diab, Eneko Agirre, lñigo Lopez- Gazpio, and Lucia Specia. 2017. SemEval-2017 task I: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) pages 1-14, Vancouver, Canada. Association for Computational Linguisties.
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespeedss, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder. arXiv preprint arXiv:
Jacob Devlin, Ming-Wor Wei Chang. Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for classification. arXiv preprint 171
Further, as large language models (LLMs) are drawing much attention in the NLP community, how strong LLMs behave in connecting discrete and continuous space perturbations remains unexplored, especially when GPT-4 (OpenAI, 2023) is known to support images and texts. As these models are not open-source to the public, we leave exploring the perturbation in LLMs future works.
Tan J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and hamnessing adversarial examples. arXiv:1412.6572.
arXiv preprint
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. 2021. The many faces of robustness: critical analysis of generalization. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 8340-8349.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 62236004 and No. 62022027). We would like to extend our gratitude to the anonymous reviewers for their
Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking neural network robustness to common corruptions perturbations. arXiv preprimi urXiv:1903. 12261.

---

Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv: 007.07328.
Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, and Zhiming Ma. 2021. Improved ood generalization via adversarial training and pretraing. In International Conference on Machine Learning, pages 11987-11997. PMLR.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is BERT really robust? natural language attack on text classification and entailment. CoRR, abs/1907. 11932.
Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066-600 60860
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. Bert-attack: Adversarial attack against bert using bert. arXiv preprint
arXiv:2004.088 099944.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. 2019a. Theoretically principled trade-off between robustness and accuracy. International conference on machine learning, pages 7472-7482. PMLR.
Zongyi Li, Jianhan Xu, Jiehang Zeng. Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, and Cho-Jui Hsieh. 2021. Searching for an effective defender: Benchmarking defense against adversarial word substitution. arXiv preprint arXiv:2108. 12777
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019b. Bertscore: Evaluating text generation with bert. ArXiv. abs/1904.09675.
Andrew Maas, Raymond E Daly, Peter Pham, Dan Huang, Andrew Y Ng. and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguisties: Human language technologies, pages 142-150.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances Neural Information Processing Systems, volume 28, Curran Associates. Inc.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2019. Towards deep learning models resistant adversarial attacks.
Chen Zhu, Yu Cheng. Zhe Gan, Siqi Sun, Thomas Goldstein, and Jingjing Liu. 2019. Freelb: Enhanced adversarial training for language understanding. arXiv preprint 1909: 1164.
Nikola Mrkšić, Diarmuid O Séaghdha, Blaise Thomson, Mílica Gašié, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. arXiv preprint arXiv:1603.008:
OpenAl. 2023. Gpt-4 technical report. ArXiv, abs/2303.08774.
Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima, Florian Yger, Cédric Gouy- Pailler, and Jamal Atif. 2019, Theoretical evidence for adversarial robustness through randomization. Advances Neural Information Processing Systems, 32.
Pranav Rajpurkar, Jian Zhang. Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arxiv:1606.05250.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, al. 2019. Huggingface's transformers: State-of- the-art natural language processing. arXiv preprint 110.0:191..03771.
Yao- Yuan Yang. Cyrus Rashtchian, Hongyang Zhang. Russ R Salakhutdinov, and Kamalika Chaudhuri. 2020. A closer look accuracy vs. robustness. Advances in neural information processing systems,
33:8588-8601.

---

PerturbScorer Testing Setup
PerturbScurer Training Selup
Edit Distance
PerturbSconer
P(š)
(+) ||
P(S)
So
Kendall
Spearmann
Kendall
Spearmann
Cross-Dataset
Rand. Porturb
BERT Freel
Rand, Porturb
BERT Freel B
0.4891
0.635
0.4757
0.6641
IMDR
AG's News
0,4921
0.6278
0.5449
0.715
Textfoolrr
BERT FreeL8
Teatfooler
BERT 0.5153
0.6683
0.4607
0.6384
Preel.B
0.3198
0.677
0,4873
0.666
Cros-Perturbation
Rand, Porturb
BERT FreeLB
Textfooler
BERT FreeLB
0.4891
0.635
0.5767
0.7532
IMDB
IMDB
0.4921
0.6278
0.5608
0.7334
BERT
BERT
0.5155
0.6683
0.4779
0.6543
Textfooler
Rand, Perturb
FreeLB
FreeLB
0.5198
0.677
0.4785
0.6545
Cross-Model
BERT
FreeLBB
0.4891
0.635
0.6192
0.7943
Rand. Perturb
Rand, Perturb
IMDE
0,4921
0.6278
0.5963
0.7616
Textfooler
BERT
Textfooler
FreeLB B
0.5155
0.6683
0.5901
0.7753
Combined Datase PertarlSccorer
IMDB
Rand. Perturh
BERT
IMDB
Rand. Perturh
BERT 0.4083
0.5035
0.7558
0.9047
FreeLB BERT FreeLB
AG's News
Textfooler
AG's News
Textfooler
BERT
0,4713
0.6267
0.8114
0.0429
0.8325
Combined-Perturbban PerturhScorer
IMDB
Rand. Perturb
BERT
IMDB
Rand. Perturb
BERT 0.4406
0.5838
0,7802
0.9198
Freel
Freel B
0.4539
0.5942
0,7795
0.914
AG's News
Texifooler
BERT
AG's News
Textfooler
BERT
0.4179
0.5579
0.803
0.9378
PreeL.8
Freel
0,4614
0.6098
0.8194
Combined-Model PerturhScorer
BERT Textfooler Textfooler
BERT Textfooler
0.4258
0.5601
0.5945
0.7634
IMD8
BERT & FreeI.B
IMD8
BERT
0.4997
0.6561
0,7973
0.9377
& FreeLBB
Pertarh Textfooler
AG's News
AG's News
0.4024
0.5424
0.8061
0.9414
Table 5: Through results of PerturbScorer generalization tests.
Appendix
Through Results of Generalization Experiments of the PerturbScorer In Table 5, we list the thorough results which is the expansion of the results shown in Table 4. As shown, we can observe that the experimental results consistent with the analysis based the partial results in Table 4. We can build PerturbScorer that can be used in cross-perturbation, datasets, and models as general PerturbScorer to build the connection between discrete and continuous perturbations.


Reviews:
[{"review_id": "eR4itEZVTh", "reviewer": "Reviewer_pixE", "summary": "", "questions": "1. What are the axes for the the plots in figure 1? I'd recommend including axis names in the future for better readability.\n\n2. Could you re-explain the significance of the findings from Section 4.4? See my comments in the weaknesses section. \n\n3. Could you expound on some potential applications of this work? Ideally including a small section in the paper about this would be helpful.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable comments!\n\nAbout Weakness #1:\n\nIn Equation 1, we hope to find the minimum norm bound that is equal to the output shift caused by the discrete perturbation S, and in Algorithm 1 we iterate gradient-based adversaries of different norm bounds to match the discrete perturbation. By using the gradient-based adversary, which is the min-max problem in standard adversarial training, we are able to use norm-bound to represent continuous perturbation.\nThen we can use Equation 1 to connect discrete perturbation S and continuous perturbation \\delta.\nWe will add these illustrations in the revision to avoid misunderstanding.\nWe appreciate your careful reading!\n\n\nAbout Weakness #2:\n\nPlease see Question #2.\n\nAbout Question #1:\n\nIn Figure 1, the y-axis is the cosine similarity of perturbed text and original text.  In subfigure (a) and (b) the x-axis is the edit distance and in (c) and (d) the x-axis is the norm-ball range. \nWe will add the name clearly  (we illustrate them in captions first).\nThe figure verifies Assumption 1 that both discrete and continuous perturbations can cause more muscular damage to the model when the perturbations increase.\n\nAbout Question #2:\n\n\"The relationship between discrete and continuous perturbations is weak\" is to measure discrete perturbations using metrics such as edit-distance or BERTScore. Our proposed PerturbScore can build the relationship between discrete and continuous perturbations.\n\nIn Figure 1, the similar effect is to consider perturbations separately, that is, stronger perturbations indicate a strong attack effect.\n\nFor example, larger norm-bound perturbations can cause more damage to the model, and larger discrete perturbations can cause more damage to the model as well. \nBut larger norm-bounds DO NOT equal large discrete perturbations.\n\nFigure 1 is not a case-by-case figure, therefore, cannot show the mismatch between discrete and continuous perturbations. \nWe show the case-by-case mismatch between discrete and continuous perturbations in Sec. 4.4 (Table 3)\nAs seen, metrics such as edit distance and BERTScore cannot measure discrete perturbations, while PerturbScore shows a close correlation between discrete and continuous perturbations.\n\nTherefore, we aim to build the connection between discrete and continuous perturbations, and we can see that our proposed PerturbScore can measure how strong the discrete perturbations are, while edit distances can not.\nPerturbScore is to learn the correlation between discrete and continuous perturbations. Our experiment shows that it is POSSIBLE to utilize $\\epsilon$ to study such correlation.\nPrevious works do not show that a continuous bound $\\epsilon$ can represent a discrete perturbation, which is the major contribution of PerturbScore.\n\nAbout Question #3:\n\nA very straightforward application of PerturbScore is to quantify discrete perturbations.\nFor example, in protecting model robustness against adversarial attacks, a strong PerturbScore can judge whether crafted perturbations can harm the model and protect the model.\nPlus, as discussed in [FreeLB: Enhanced Adversarial Training for Natural Language Understanding(Zhu et al. 2019)], [Token-aware virtual adversarial training in natural language understanding (Li and Qiu, 2021)], [Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution], adversarial training with virtual perturbations can help improve robustness against discrete attacks.\nThese papers do not give any explanation of why continuous perturbations can help improve robustness, with PerturbScorer, we can prove that there can be a correlation between continuous and discrete perturbations, providing proof for studying those adversarial training methods to improve model robustness.\nAlso, the PerturbScore can measure the continuous perturbations, which can be generalized to a sentence similarity checker similar to BERTScore.\nOn the other hand, PerturbScore can blur the line between discrete and continuous space, which can help build multi-modal models based on the correlation between discrete and continuous perturbations (e.g. help building negative samples for cross-modal general large models)\n\n\n\nAbout Presentation Improvements:\n\nWe will re-construct the paper to make it more clear for readers, we appreciate your comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The goal of this study is well-motivated and clear.\n\n2. The authors do a strong job of supporting the assumptions made in Section 2 through empirical study. \n\n3. The empirical results are comprehensive. Multiple datasets and methods are used.", "reasons_to_reject": "1. More care can be taken in explaining the equation in Section 3. It was not obvious at first as to what exactly they should be measuring. An explanation given before or after each equation, explaining the purpose and intuition would be helpful.\n\n2. I don't find the results in presented in Section 4.4 to be very convincing. First, based on the results the authors state that the relationship between the discrete and continuous perturbations is weak. They conclude that discrete perturbations can't properly attack models. However, earlier they use Figure 1 to show that the continuous and discrete perturbations have a similar effect on model performance. It's hard to square these two assertions. Secondly, they note that PerturbScorer has a strong correlation with the continuous pertubations. I feel like this isn't surprising at all as PerturbScorer utilizes $\\epsilon$.", "questions_to_authors": "1. What are the axes for the the plots in figure 1? I'd recommend including axis names in the future for better readability.\n\n2. Could you re-explain the significance of the findings from Section 4.4? See my comments in the weaknesses section. \n\n3. Could you expound on some potential applications of this work? Ideally including a small section in the paper about this would be helpful.", "paper_topic_and_main_contributions": "In this paper, the authors study the problem of model robustness in NLP. Specifically, they seek to study the relationship between continuous and discrete perturbations in NLP models. The authors motivate this study by noting tha most robustness work in NLP focuses on discrete perturbations. However, this can be problematic as discrete perturbations are much more costly than their continuous counterparts. The authors first assume that there exists a connection between discrete and continuous perturbations, and using that design a method finds the proper norm bound between a continuous and discrete perturbation. They utilize this method to construct their own pertuber. They utilize this to conduct various studies comparing the two types of perturbations and the effect on model performance.\n\nPost-Rebuttal Update: I appreciate the detailed response. I find most of the responses satisfactory. I've raised my excitement from a 3 &rarr; 4", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. I found it difficult to read Section 4. I'd recommend not utilizing so many sub sections (and sub-sub and so on...). However this may just be a personal taste."}, "extra_text": {}}, {"review_id": "54DvMp7syH", "reviewer": "Reviewer_WvbR", "summary": "", "questions": "1. The paper explores how to approximate a discrete perturbation with a continuous one. Is it possible to do the other way round, i.e. finding the corresponding discrete perturbation of a continuous one? This could make the two types of perturbations more connected.\n\n2. Do you use the same model (e.g. BERT) for perturbation dataset generation (sec 3.4) and perturbscoer (sec 3.5)? What would the correlation results be like if you use different types of models?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable comments!\n\n\nQuestion #1\n\nWe have considered studying the correlation the other way around and we think this is a very interesting idea.\nOne major obstacle is that we round continuous perturbations using norm bound without further studying the direction of the continuous perturbations.\nIf we find discrete perturbations from a continuous perturbation, the searching space would be extremely large.\nSuch an idea could be possible with some very powerful generative models, therefore, we think that building such a correlation by fine-tuning GPT-4 or at least fine-tuning LLaMA might be a plausible solution, which could be a promising future work of PerturbScore.\n\nQuestion #2\nWe use BERT-BASE to train the PerturbScorer and the perturb dataset generation includes FreeLB trained BERT in IMDB and AG's News dataset and normal fine-tuned BERT in IMDB and AG's News dataset.\nAs seen, the FreeLB-trained model and the normal fine-tuned model generated perturb-dataset show similar results using BERT-trained PerturbScorer, therefore we can conclude that the PerturbScorer does not require the same base model on which the perturbation is constructed based.\n\nStill, we appreciate such a question and we will add some other models such as RoBERTa or LLaMA models to train the PerturbScorer to better study the correlation between discrete and continuous perturbations.\n\n\n\n\nWeakness #1\n\nWe will add more models and datasets to the revision. We will add simple models including RNNs and stronger models including LLaMA to study the perturbation correlation.\nAlso, we will add more datasets including classification datasets such as SST-2 and Yelp, and NLI datasets including SNLI and MNLI.\n\nWeakness #2\n\nWe will add an additional experiment using our proposed PerturbScore as a metric to evaluate the perturbations added in an adversarial attack process (such as Textfooler attack or BERT-Attack).\nPlus, we will add an experiment to use PerturbScore as a scorer to study whether it can be used in measuring sentence similarities.\nWe are hoping that by adding these experiments, we will be able to strengthen the importance of building PerturbScore in studying the correlation between continuous and discrete perturbations.\n\nWeakness #3\n\nAs for the question about discarding the mismatch between continuous perturbations and discrete perturbations, we can prove that the norm bound does exist. We discard some cases simply for the implementation rounding.\n That is, with no perturbation, the model is not changed at all, and with as many perturbations as possible, the model will be shifted to a random prediction. Within this range, there will always be a norm-bound to match the discrete perturbation.\nWe discard some of the mismatch cases because the small interval $\\varepsilon$ is not an extremely small value.\nIf the interval is an extremely small value (reaching 0), there will always be a matching pair of discrete and continuous perturbations, but the calculation will be massive therefore we set the value as a trade-off in the implementation.\nWe will clarify this in the paper to avoid this misunderstanding.\n\n\nIntroduction of the FreeLB method\n\nThe FreeLB method is a simple modification of the PGD method used to craft gradient-based adversaries.\nThe adversarial training process is similar to Algorithm 1. The algorithm will first calculate gradients and then add the gradients to the input embedding as a virtual adversary to improve model robustness.\nWe use the gradients update process to find the correlation between discrete and continuous perturbations. Therefore, we initially believe that the adversarial-trained model should follow Assumption 2 better as it is more robust against gradient-based perturbations.\nResults in Table 3 show that the FreeLB-trained model has a tighter correlation between discrete and continuous perturbations, which is intuitive given the adversarial training process helps build a tighter bound of continuous perturbations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper looks into an interesting problem of finding a continuous perturbation as a proxy for discrete perturbation, which provides a new angle to study the robustness of NLP models through the lens of continuous space. \n\n2. Experiment results of PerturbScorer look promising, indicating that the generated perturbation dataset and the trained model are of good quality.\n\n3. The method is carefully designed and well-explained in Sec 3.", "reasons_to_reject": "1. Limited numbers of models and datasets/tasks are studied in this paper. I hope to see similar results on more model architectures (including a simpler RNN model) and tasks to better support the generalization ability of PerturbScorer.\n\n2. I wish to see a concrete demonstration of how the PerturbScorer can be applied to broader scenarios such as studying the robustness of NLP models in continuous space, measuring sentence differences, etc.  \n\n3. The validity of the proposed method would be better supported by a theoretical proof of when the norm-bound would exist (the continuous perturbation effect is way bigger than discrete ones, line 342-345).", "questions_to_authors": "1. The paper explores how to approximate a discrete perturbation with a continuous one. Is it possible to do the other way round, i.e. finding the corresponding discrete perturbation of a continuous one? This could make the two types of perturbations more connected.\n\n2. Do you use the same model (e.g. BERT) for perturbation dataset generation (sec 3.4) and perturbscoer (sec 3.5)? What would the correlation results be like if you use different types of models?", "paper_topic_and_main_contributions": "This paper addresses the research question: can we find a connection between discrete perturbation and continuous perturbation to study the robustness of neural NLP models? They approach this question by quantifying the connections with the norm-bound of gradient-based perturbations and training a regression model PerturbScorer to predict the strength of the connection (correlation). The resulting PerturbScorer can be useful for better understanding robust training methods of NLP models and transferring the robustness theories studied in the computer vision field to NLP.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The paper seems to miss a brief introduction to the FreeLB method appeared in Fig 1 & Table 2. Can you add a few sentences to explain it?"}, "extra_text": {}}, {"review_id": "ZZmz1EvXc0", "reviewer": "Reviewer_Rjy5", "summary": "", "questions": "1. Page 4, Column 2: the sentence  ‘..... obtain a data tuple [S, P(S), \\epsilon] , which is the correlation’ is a bit ambiguous. Assuming ‘correlation’ refers  to ‘\\epsioln’, is it a correlation between the discrete and continuous perturbations? Or to me it seems that ‘\\epsilon’ is a continuous surrogate for the discrete perturbation P(S). Please clarify. This may be confused with another correlation measured in section 4.2 in the form of Kendal and Pearson Index. Also, the tuple [S, P(S), \\epsilon] is not a correlation.\n\n2. Page 6, Column 1: How does better resistance of models trained FreeLB method against discrete and continuous perturbation verify assumption 1 and ensure correlation between discrete and continuous perturbation? There seems to be a logical gap. Please clarify\n\n3. Page 6: Column 1: Table 2 shows count of data tuples under different \\epsilon ranges. This shows most of the discrete distributions correspond to lower continuous perturbation value ranges. Some of the data tuples have been discarded? The claim is as a small proportion of data tuples have been discarded, the method is able to find the ‘norm ball that satisfies Assumption 3’. What is the criteria for discarding the data tuples?  If it is manually selected then size of the set of discarded tuples may vary. Also, it is apparent why discarding a small proportion of tuples leads to satisfaction of Assumption 3. \n\n4. Page 6: Column 1: There is no evidence in the paper that “Textfooler method generates more discrete perturbations …… and corresponding continuous perturbations require larger norm balls. Please specify if these experimental observations have been explicitly presented in any part of the paper.\n\n5. The cross-dataset analysis in Appendix shows that there is no significant difference in the correlation between edit-distance and PerturbScorer. In many cases, the correlation is poorer in PerturbScorer. Though the situation improves when the datasets are combined.  How these results reflect on the generalizability of the model?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable comments!\n\n\nQuestion #1\n\nThe tuple [S, P(S), \\epsilon] is not a correlation, the tuple is the constructed connection between a pair of discrete and continuous perturbations. We hope to study the correlation between discrete and continuous perturbations by constructing a tuple of the discrete and continuous perturbation pair.\nWe will correct this representation.\nIn Sec. 4.2, the correlation is the metric to evaluate the closeness between metrics that measure discrete and continuous perturbation.\nBy measuring the correlation coefficient index between A(P(S), S) and \\epsilon, we can study the correlation between discrete and continuous perturbations.\nWe will clarify these statements for readers to clearly understand our work.\nWe appreciate your careful reading!\n\n\n\nQuestion #2\n\nThe FreeLB method utilizes gradients to craft adversaries with virtual perturbations on the embedding space to train the model.\nTherefore, the FreeLB model is more robust to gradient-based perturbations.\nIn Figure 1, we can see that the FreeLB model is more robust to both gradient and discrete perturbations, the slopes of FreeLB models are flatter than normal models. \nThe tendency is similar, which verifies Assumption 1 that continuous and discrete perturbations have similar effects on target models.\n\nAs for the statement, 'FreeLB results show that discrete and continuous perturbation can be correlated':\nWe should be more cautious about such a statement. That is, the FreeLB model is more robust against both discrete and continuous perturbation, which might indicate that discrete and continuous perturbations can be connected.\n\nActually, the reason why NLP adversarial training methods using gradients such as FreeLB can improve model robustness against discrete perturbations is because PerturbScore shows that we can build the correlation between discrete and continuous perturbations.\n\nKnowing that we can build such a correlation, we are very excited since now we can explain why FreeLB and later adversarial training methods work (PerturbScore can provide proof for these adversarial training methods).\n\nWe will clarify these statements more clearly in the revision.\nAgain, we are grateful for your careful reading!\n\n\nQuestion #3\n\nThe discarded tuples are due to the rounding process, which is because of the \\varepsilon selection.\nIf \\varepsilon is set to an extremely small value (reaching zero), then we will not discard any samples at all, but in this case, the generation process will be very long and the generated data will be massive.\nFor the calculation trade-off, we set a certain \\varepsilon not too large, therefore, there might be some mismatch between discrete and continuous perturbations.\nFor example, if  \\varepsilon is very large, different discrete perturbations will fall into the same continuous perturbation norm, and the correlation is not learnable anymore.\nTherefore, in between lines 326-327, Algorithm line 9, we set a threshold \\phi to guarantee that the correlation is within a small range so that the correlation is learnable.\nIt is an implementation trade-off, not a theoretical drawback.\n\n\nQuestion #4\n\nAbout Textfooler perturbations: Textfooler perturbations can cause more severe damage to neural models than random perturbations (which is illustrated in the Textfooler paper).\nSuch a claim is intuitive since Textfooler iterates the substitutes to find perturbations that can harm neural models more, while random perturbations are not 'adversarial' against neural models.\nTherefore, in Table 2, it is reasonable that Textfooler perturbations have large norm-bound corresponding perturbations since they are stronger discrete perturbations.\n\nQuestion #5\n\nThe PerturbScorer is relatively less effective in studying the cross-dataset prediction, that is mainly because the average length of IMDB dataset is over 200 words and the average length of AG's News is only around 40.\nTherefore, discrete perturbations are more different between these two datasets, therefore the PerturbScorer is less effective in generalizing the prediction.\n\nOn the other hand, though the PerturbScorer is slightly lower than the edit distance, we at first did not expect that PerturbScorer could generalize to different datasets, perturbation types, or models at all.\nTherefore, we believe that the observation that PerturbScorer can obtain similar results in cross-dataset prediction with edit distance could show that PerturbScorer can be generalized in different scenarios.\nBased on this motivation, we test the combination of dataset, perturbation, and models, and results show that PerturbScorer can learn such correlation, indicating that we can build a very strong PerturbScorer with multiple models, different text types, and different perturbation types as a firm bridge between discrete and continuous perturbations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper studies one of the most important aspects of NLP research, i.e., measuring the robustness of the neural NLP models.\n2. To deal with imperceptibility of the discrete perturbations and combinatorial explosion in formulating those perturbations, the paper advocates for applying continuous perturbation. \n3. I really appreciate the two staged approach adopted by the authors: Firstly, mapping discrete perturbation to continuous perturbation with a set of non-trivial assumptions. This formalisation along with an optimization method the authors presented an elegant method for generating training examples each consisting of original input, discrete perturbation and equivalent continuous perturbation. Secondly, using those training instances, a regression model has been proposed to predict continuous prediction given a discrete perturbation.\n4. The authors performed extensive experiments to validate the claims and showcase the usefulness of the method", "reasons_to_reject": "1. Many claims are ambiguous and are not supported by strong logic. (see questions)\n2. Given the results, I have doubt in the generalizability of the proposed model.", "questions_to_authors": "1. Page 4, Column 2: the sentence  ‘..... obtain a data tuple [S, P(S), \\epsilon] , which is the correlation’ is a bit ambiguous. Assuming ‘correlation’ refers  to ‘\\epsioln’, is it a correlation between the discrete and continuous perturbations? Or to me it seems that ‘\\epsilon’ is a continuous surrogate for the discrete perturbation P(S). Please clarify. This may be confused with another correlation measured in section 4.2 in the form of Kendal and Pearson Index. Also, the tuple [S, P(S), \\epsilon] is not a correlation.\n\n2. Page 6, Column 1: How does better resistance of models trained FreeLB method against discrete and continuous perturbation verify assumption 1 and ensure correlation between discrete and continuous perturbation? There seems to be a logical gap. Please clarify\n\n3. Page 6: Column 1: Table 2 shows count of data tuples under different \\epsilon ranges. This shows most of the discrete distributions correspond to lower continuous perturbation value ranges. Some of the data tuples have been discarded? The claim is as a small proportion of data tuples have been discarded, the method is able to find the ‘norm ball that satisfies Assumption 3’. What is the criteria for discarding the data tuples?  If it is manually selected then size of the set of discarded tuples may vary. Also, it is apparent why discarding a small proportion of tuples leads to satisfaction of Assumption 3. \n\n4. Page 6: Column 1: There is no evidence in the paper that “Textfooler method generates more discrete perturbations …… and corresponding continuous perturbations require larger norm balls. Please specify if these experimental observations have been explicitly presented in any part of the paper.\n\n5. The cross-dataset analysis in Appendix shows that there is no significant difference in the correlation between edit-distance and PerturbScorer. In many cases, the correlation is poorer in PerturbScorer. Though the situation improves when the datasets are combined.  How these results reflect on the generalizability of the model?", "paper_topic_and_main_contributions": "Robustness of the neural NLP models have become an extremely important topic as these neural models are abundantly used in real world applications. Perturbation-based robustness study is popular among the deep learning researchers, specifically, in the computer vision community where the nature of perturbation is continuous.  Perturbation-based robustness study in NLP faces the challenge of handling discrete input leading to discrete perturbation. This paper aims at connecting discrete perturbation to continuous perturbation so that perturbation in the continuous domain is applicable in NLP. Primary contributions of the paper include 1) quantifying the correlation between discrete and continuous perturbation, 2) designing a regression-based model (viz, PerturbScore) to estimate the correlation.", "excitement": 5, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Use of mixed case styles: Textfooler, textfooler\n\n2. Page 5, Column 1, Line 1: ‘Specifically, the notation x used in line 7…’. Assuming line 7 refers to the line no 7 of Algorithm 1, no mention of ‘x’ is present."}}]

Author rebuttal:

Meta-review: The paper tries to connect continuous and discrete perturbations. Particularly, this paper suggests an algorithm to find a continuous perturbation given discrete perturbation within some error bound and further suggests a regressor (PerturbScorer) that emits matching continuous perturbation’s norm-bound given original sentence S, and discrete perturbation P(S).

The responses from the reviewers are somewhat unanimous. This paper brings an interesting and novel direction (and thus high excitement scores), however, there are concerns regarding clarity, generalizability, and the method’s application in a broader perspective (resulting in insufficient soundness score).

I too agree that this is an interesting direction, but also agree with other reviewers that the manuscript lacks clarity and that the broader application should be addressed further. Authors do respond that this can be useful in multi-modal settings and help explain previous papers. However, it seems that there is a nontrivial step to show this usefulness on top of what the paper already shows. To be more precise with some examples, can this algorithm tell why some continuous perturbations worked and some did not? Addressing the concerns reviewers brought up and resolving ambiguities will significantly enhance the paper's value to the NLP community.

(Some more detailed examples that the paper can improve on)

- (Figure 1) I was not able to understand Figure 1 at all by just reading the paper. I was only able to understand this after looking back and forth at the comments the authors made in the rebuttal. Even then, the interpretation of it was not very straightforward.
- Correlation is somewhat overused. For example, the paper studies the correlation between the effect of discrete and continuous perturbations and also names the output of the PerturbScorer to be outputting correlation.
- Authors claim that this method can help multi-modal setups. However, they also respond in their rebuttal that PerturbScorer is not very effective in generalizing prediction when overall statistics (word lengths) are different. This result was on a very similar task but with different sentence-length statistics. In a truly multi-modal setup, would the proposed method be helpful? + Why is it very useful to connect two modalities in the first place was not very clear to me in the paper. Delivering those motivations clearly could also help enhance the method’s impact in the community.
- As many reviewers pointed out, some of the statements are strong but unwarranted in a theoretical manner. e.g.) "proving" is a strong word in 418-423: "...proving that we can successfully find norm balls ...". How about changing this to a softer sentence? For example, one could say "Our experiments display, that under XYZ kind of perturbations, the proposed method can successfully find matching continuous perturbations.."
Decision: Accept-Findings