InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
Authors: Boxin Wang
Shuohang Wang
Yu Cheng
Zhe Gan
Ruoxi Jia
Bo Li
Jingjing Liu
Venue: ICLR 2021 Poster (2021)
Keywords: adversarial robustness
information theory
BERT
adversarial training
NLI
QA

Abstract: Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies, however, show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We aim to address this problem from an information-theoretic perspective, and propose InfoBERT, a novel learning framework for robust ﬁne-tuning of pre-trained language models. InfoBERT contains two mutual-information-based regularizers for model training: (i) an Information Bottleneck regularizer, which suppresses noisy mutual information between the input and the feature representation; and (ii) a Robust Feature regularizer, which increases the mutual information between local robust features and global features. We provide a principled way to theoretically analyze and improve the robustness of representation learning for language models in both standard and adversarial training. Extensive experiments demonstrate that InfoBERT achieves state-of-the-art robust accuracy over several adversarial datasets on Natural Language Inference (NLI) and Question Answering (QA) tasks.
Our code is available at https://github.com/AI-secure/InfoBERT.

Published as a conference paper at ICLR 2021
INFOBERT: IMPROVING ROBUSTNESS OF LANGUAGE MODELS FROM AN INFORMATION THEORETIC PERSPECTIVE
"Boxin Wang', Shuohang Wang", Yu Cheng", Zhe Gan", Ruoxi Jia", Bo Li', Jingjing Liu²
University of Illinois at Urbana-Champa Microsooft Dynamies 365 AT Research Virginia Tech (boxinw2, wtw.oooolioiii..e.... (shuchang. wang, yu. cheng, zhe. gan, gan, jtw.:igg..oncccooofo.cooo
ABSTRACT
Large-scale pre-trained language models such as BERT and RoBERTa have achieved state-of-the-art performance across a wide range of NLP tasks. Re- cent studies, however, show that such BERT-based models are vulnerable fac- ing the threats of textual adversarial attacks. We aim to address this problem from an information-theeoeti perspective, and propose InfoBERT, novel learn- ing framework for robust fine-tuning of pre-trained language models. InfoBERT contains two nuttualinffrmmttin--aaal Information Bottleneck regularizer, which suppresses noisy mutual information between the input and the feature representation; and (ii) Anchored Feature regularizer, which increases the mutual information between local stable fea- tures and global features. We provide a principled way theoretically analyze and improve the robustness of language models in both standard and adversar- ial training. Extensive experiments demonstrate that InfoBERT achieves state- of-the-art art robust accuracy over several adversarial datasets on Natural Language Inference (NLI) and Question Answering (QA) tasks. Our code available https: / http:/AAiSecuueeeefffeeoee
regularizers for model training: (i) an
INTRODUCTION
Self-supervised representation learning pre-trains good feature extractors from massive unlabeled data, which show promising transferability to various downstream tasks. Recent success includes large-scale pre-trained language models (e.g., BERT, RoBERTa, and GPT-3 (Devlin et al., 2019) Liu et al., 2019| Brown et al. 2020). which have advanced state of the art over wide range of NLP tasks such NLI and QA, even surpassing human performance. Specifically, in the computer vision domain, many studies have shown that self-supervised representation learning is essentially solving the problem of maximizing the mutual information (MI) I(X: between the input X and the representation T (van den Oord et al., 2018 Belghazi et al. 2018 Hjelm et al.] 2019: Chen al. 2020). Since MI is computationally intractable in high-dimensional feature space, many MI estimators (Belghazi et al. 2018) have been proposed to serve as lower bounds (Barber & Agakov 2003 van den Oord al.) 2018) or upper bounds (Cheng et al.) 2020) MI. Recently, Kong et al. point out that the MI maximization principle of representation learning can be applied to not only computer vision but also NLP domain, and propose a unified view that recent pre-trained language models are maximizing a lower bound of MI among different segments of a word sequence.
On the other hand, deep neural networks are known to be prone adversarial examples (Goodfel- Dezfooli et al., 2016). i.e., the outputs of neural networks can be arbitrarily wrong when human-imperceptible adversarial pertur- bations are added to the inputs. Textual adversarial attacks typically perform word-level substitution (Ebrahimi et al. 2018 Alzantot et al., 2018 Ren et al. (019) or sentence-level paraphrasing [Iyyer ct 2018 Zhang et al.) 2019) to achieve semantic/utility preservation that seems innocuous to human, while fools NLP models. Recent studies (Jin et al. 2020 Zang et al.) 2020 Nie et al.) 2020 Wang et al. 2020) further show that even large-scale pre-trained language models (LM) such as
low et al.) 2015 Papernot et al., 2016 Eykholt et al. 2017 Meosavv-DDezoooli
WWrk was done during Boxin Wang's Summer internship in Microsoft Dynamics 365 AT Research.

---

Published as a conference paper at ICLR 2021
BERT are vulnerable adversarial attacks, which raises the challenge of building robust real- LM applications against unknown adversarial attacks.
We investigate the robustness of language models from an information thcoretic perspective, and propose a novel learning framework InfoBERT, which focuses on improving the robustness of lan- guage representations by fine-tuning both local features (word-level representation) and global fea- tures (sentence-level representation) for robustness purpose. InfoBERT considers two MI-based regularizers: (i) the Information Bottleneck regularizer manages to extract approximate minimal sufficient statistics for downstream tasks, while removing excessive and noisy information that may incur adversarial attacks: (ii) the Anchored Feature regularizer carefully selects useful local stable features that are invulnerable to adversarial attacks, and maximizes the mutual information between local stable features and global features to improve the robustness of the global representation. In this paper, we provide detailed theoretical analysis to explicate the effect of InfoBERT for robust- ness improvement, along with extensive empirical adversarial evaluation to validate the theory.
Our contributions are summarized as follows. (i) We propose a novel learning framework InfoBERT from the information theory perspective, aiming to effectively improve the robustness language models. (ii) We provide a principled theoretical analysis on model robustness, and propose two MI- based regularizers to refine the local and global features, which can be applied to both standard and adversarial training for different NLP tasks. (tii) Comprchensive experimental results demonstrate that InfoBERT can substantially improve robust accuracy by a large margin without sacrificing the benign accuracy, yielding the state-of-the- performance across multiple adversarial datasets on NLI and QA tasks.
RELATED WORK
Textual Adversarial Attacks/Defenses Most existing textual adversarial attacks focus on word- level adversarial manipulation. [Ebrahimi et al. (2018) the first to propose a whitebox gradient- based attack search for adversarial word/character substitution. Following work (Alzantot et al. 2018 Ren et al. 2019; Zang et al. 2020 Jin et al. 2020) further constrains the perturbation search space and adopts Part-of- checking to make NLP adversarial examples look natural to human.
To defend against textual adversarial attacks, existing work can be classified into three categories: (1) Adversarial Training is practical method to defend against adversarial examples. Existing work either uses PGD-based attacks to generate adversarial examples in the embedding space of NLP as data augmentation (Zhu et al.. 2020a)), regularizes the standard objective using virtual adversarial training @Jiang et al. 2020; Liu et al. 2020 Gan et al. 2020). However, one drawback is that the threat model is often unknown, which renders adversarial training less effective when facing unseen attacks. (ii) Interval Bound Propagation (IBP) (Dvijotham et al. (018)) is proposed as new technique to consider the worst-case perturbation theoretically. Recent work (Huang et al. 2019 Jia et al. 2019) has applied IBP in the NLP domain to certify the robustness of models. However, IBP- based methods rely on strong assumptions
model architecture and are difficult to adapt to recent transformer-basse language models. (iii Randomized Smoothing (Cohen et al.) 2019) provides tight robustness guarantee in le norm by smoothing the classifier with Gaussian noise. Ye et al.)
(020) adapts the idea to the NLP domain, and replace the Gaussian noise with synonym words to certify the robustness as long as adversarial word substitution falls into predefined synonym sets. However, to guarantee the completeness of the synonym set is challenging.
Representation Learning MI maximization principle has been adopted by many studies on self- supervised representation learning (van den Oord et al. 2018 |Belhazi al. 2018 Hjeelm et al.) 2019 Chen et al.) 2020). Specifically, InfoNCE (yan den Oord et al. (018)) is used as the lower bound of MI, forming the problem as contrastive learning (Saunshi al. 2019 Yu et al. 2020). However, Tian et al. (2020) suggests that the InfoMax (Linsker) 1988) principle may introduce ex- cessive and noisy information, which could be adversarial. To generate robust representation, Zhuu et al. 2020bb formalizes the problem from mutual-information perspective, which essentially per- forms adversarial training for worst-case perturbation, while mainly considers the continuous space in computer vision. In contrast, InfoBERT originates from information-theoreii perspective and is compatible with both standard and adversarial training for discrete input space of language models.
2

---

Published as a conference paper at ICLR 2021
3 INFOBERT
Before diving into details, we first discuss the textual adversarial examples we consider in this paper. We mainly focus on the dominant word-level attack the main threat model, since it achieves higher attack success and is less noticeable to human readers than other attacks. Due to the discrete nature of text imput space, it is difficult measure adversarial distortion on token level. Instead, because most word-level adversarial attacks (Li et al. 2019 Jin et al. (0200 constrain word perturbations via the bounded magnitude in the semantic embedding space, by adapting from Jacobssen et al. (2019), we define the adversarial text examples with distortions constrained in the embedding space.
Definition 3.1. (r-bounded Textual Adversarial Examples). Given a sentence x = Shi;:2;---0 Fn). where x; is the word at the i-th position, the e-bounded adversarial sentence x' [2r]; ; 22; ... an
for a classifier F satisfies: (1) F(z) o(z) = o(x') but F(x') & o(x'), where o(-) is the oracle
(e.g., human decision-makert (2) ||4, 1112 for i 1,2, .... n., where Z 0 and 6, is the word embedding of z;.
3.1 INFORMATION BOTTLENECK AS A REGULARIZER
In this section, we first discuss the general IB implementation, and then explain how IB formulation is adapted to InfoBERT regularizer along with theoretical analysis to support why IB regularizer can help improve the robustness of language models. The IB principle formulates the goal of deep learning
information -hommation-thhoretii trade-off between representation compression and predictive power (Tishby & Zaslavsky 2015). Given the input source X, a deep neural net learns the internal representation 7 of some intermediate layer and maximizes the MI between T and label Y, that T' subject to constraint on its complexity contains sufficient information to infer the target label Y. Finding an optimal representation T' can be formulated as the maximization of the Lagrangian
LiB TY - 3I(X;T),
where B Os hyper-parametee to control the tradeoff, and I(Y;T) defined as:
I I(;;T)= - p(y. t) log ply.t) dy dt. p(y)p(()
Since Eq. 2 is intractable, we instead use the lower bound from Barber Agakov (2003):
((;;)) > f
/ p(y. t) log ( dy dt,
(3)
where qu (ylt) is the variational approximation learned by neural network parameterized by & for the true distribution plylt). This indicates that maximizing the lower bound of the first term of IB T) is equivalent to minimizing the task cross-entropy loss lissk = H(Y IT).
To derive a tractable lower bound of IB, we here use an upper bound (Cheng et al. 2020 of I(X; T)
I(x:T) S / p(x, log(p(t z)) dx dt p(z)p(t) log(p(t x)) dx dt.
(4)
By combining Eq. 3 and we can maximize the tractable lower bound LTB of in practice by:
B N
LB
log qu (3
[
[og(p(t()) (z()
log(p(t")
(5)
with data samples (2 y/ p(t z) can be viewed as the feature extractor fe + T, where X' and T are the support of the input source X and extracted feature T, respectively.
y.V (
where do can represent any classification model (e.g., BERT), and
The above is a general implementation of IB objective function. In InfoBERT, we consider as the features consisting of the local word-level features after the BERT embedding layer fo. The following BERT self-attentive layers along with the linear classification head serve as go (ylt) t) that predicts the target Y given representation T.
Formally, given random variables X = [X;; X1; ...; X,, representing input sentences with X, (word token at i-th index), let T [Ti;...; Ta] = fo([x;; X2; X..)) = (fo(Xi); fo(X2); So (X,))
3

---

Published as a conference paper at ICLR 2021
denote the random variables representing the features generated from input X via the BERT em- bedding layer fa. where T; E IR is the nigh-dimensional word-level local feature for word X,- Due to the high dimensionality d of each word feature (e.g., 1024 for BERT-large), when the sen- tence length n increases, the dimensionality of features 7 becomes too large to compute T(; in practice. Thus, propose maximize localized formulation of IB LLm defined as:
n
LI == I(Y:T) nB Σ T.);
(6)
Theorem 3.1. (Lower Bound of LIB) Given sequence of random variables X = [X1; X2; Xnl and a deterministic featr extractor fo. let T = =Ti;... Tu. = (1); (X)); Sol fo(X,)): Then
the localized formulation of IB Lum is a lower bound of LiB (Eq. (.) i.e.
n
- - 2 I(Y;T) - nB Σ ((x,TT)
Theorem 3.1 indicates that we can maximize the localized formulation of LLB as a lower bound of IB LB when ( is difficult to compute. In Eq. (6). if we regard the first term (T) a lask-related objective, the second term (-n.3 Σm= / (X,;; (X,;T)) T:)) can be considered as a regularization term to constrain the complexity of representation T, thus named as Information Bottleneck regular- izer. Next, we give a theoretical analysis for the adversarial robustness of IB and demonstrate why localized objective function can help improve the robustness to adversarial attacks.
Following Definition 3.1 let T = T2:77; Tn] and T = Th Tyl denote the features for the benign sentence X and adversarial sentence X' The distributions of X and X' are denoted by probability p(x) and q(x) with the support X' and X", respectively. We assume that the feature representation has finite support denoted by T considering the finite vocabulary size in NLP.
Xn] and Ti:T2;...; TT]] [ff((1); (X2); (Xx)] and T" T. TM = [fo(X);; ffo(X))fo(X)) fo(X2); ... fo(;)) with finite support T, where fo is a deterministic feature extractor. The performance gap between benign and adversarial data [TY::T) IYY;T") is bounded above by
Theorem 3.2. (Adversarial Robustness Bound) For random variables X =
X' [ X2; ...; x, let T
< Bo B
VITI(
+ +B2
IT TT
+ BB Σ VITUUXXT))) + Σ IT IT
(8)
where Bo, B1, B2, B3 and Ba are constants depending on the sequence length n, and p(x)).
The sketch of the proof is to express the difference of [[(; - I(Y'; in terms of T(X;; T.)- Specifically, Eq. (25) factorizes the difference into two summands. The first summand, the con- ditional entropy [T
Y)- - (T" Y) can be bound by Eq. (42) in terms of MI be- tween benign/ aad/addveeee versarial input and representation I (X::T,) T,) and I [(X; T.). The second summand (T) H (T') has a constant upper bound (Eq. (85) since language models have bounded vo- cabulary size and embedding space, and thus have bounded entropy.
The intuition of Theorem 3.2 is to bound the adversarial performance drop I(Y:T) - T(Y: I') by I(X;;T,) As explained in Eq. B). I(; and TY:T") can be regarded as the model perfor- mance on benign and adversarial data. Thus, the LHS of the bound represents such performance gap. The adversarial robustness bound of Theorem 3.2 indicates that the performance gap becomes closer when I(X;; T3) and 7 [X];; T7) decrease. Note that our IB regularizer in the objective function Eq. 66 achieves the same goal of minimizing [(x;;T) TT) while learning the most efficient informa- tion features, approximate minimal sufficient statistics, for downstream tasks. Theorem 3.2 also suggests that combining adversarial training with our IB regularizer can further minimize T T(X T.) leading to better robustness, which is verified in 44
3.2 ANCHORED FEATURE REGULARIZER
In addition to the IB regularizer that suppresses noisy information that may incur adversarial at- tacks, propose novel regularizer termed "Anchored Feature Regularizer", which extracts local
4

---

Published as a conference paper at ICLR 2021
Algorithm 1- Local Anchored Feature Extraction. This algorithm takes in the word local features and returns the index of local anchored features.
1: Input: Word local features f, upper and lower threshold Oj and
2: 8 <- 0 // Initialize the perturbation vector
3: g(o) = VVluuk
8). y) // Perform adversarial attack on the embedding space
4: Sort the magnitude of the gradient of the perturbation vector from ll(())ml12 into [1g(8)s, (ll.. order, where 2j corresponds to original index.
in ascending
5: Return: k, 1i kj, where a S S Sch. Ch.
stable features and aligns them with sentence global representations, thus improving the stability and robustness of language representations.
The goal of the local anchored feature extraction is to find features that carry useful and stable in- formation for downstream tasks. Instead of directly searching for local anchored features, we start with searching for nonrobust and unuseful features. To identify local nonrobust features, we perform adversarial attacks detect which words are prone changes under adversarial word substitution. We consider these vulnerable words as features nonrobust to adversarial threats. Therefore, global robust sentence representations should rely less on these vulnerable statistical clues. On the other hand, by examining the adversarial perturbation on each local word feature, we can also identify words that are less useful for downstream tasks. For example, stopwords and punctuation usually carry limited information, and tend to have smaller adversarial perturbations than words containing more effective information. Although these unuseful features are barely changed under adversarial attacks, they contain insufficient information and should be discarded. After identifying the non- robust and unuseful features, we treat the remaining local features in the sentences as useful stable features and align the global feature representation based them.
During the local anchored feature extraction, we perform "virtual" adversarial attacks that generate adversarial perturbation in the embedding space, as it abstracts the general idea for existing word- level adversarial attacks. Formally, given an input sentence x = [ 22; ...; In] with its correspond- ing local embedding representation f [ti;...ifn]] where and are the realization of random vari- ables X and T,, we generate adversarial perturbation ō in the embedding space so that the task loss lusk increases. The adversarial perturbation 8 is initialized zero, and the gradient of the loss with respect to 8 is calculated by (5) = V Fulus (t++6), y) to update + I Niiin se The above process is similar to one-step PGD with zero-initialized perturbation 5. Since we only care about the ranking of perturbation to decide on robust features, in practice we skip the update of ö to save computational cost, and simply examine the E2 norm of the gradient g(d); of the perturba- tion on each word feature f,. A feasible plan is to choose the words whose perturbation is neither too large (nonrobust features) nor too small (unuseful features), e.g., the words whose perturbation rank- ings are among 50% ~ 80% of all the words. The detailed procedures are provided in Algorithm |
After local anchored features are extracted, we propose to align sentence global representations Z with our local anchored features T, In practice. we can use the final-layer [CLS] embedding to represent global sentence-level feature Z. Specifically, we use the information theoretic tool to increase the mutual information / [[T;; Z) between local anchored features T; and sentence global representations Z, so that the global representations can share more robust and useful information with the local anchored features and focus less on the nonrobust and unuseful ones. By incorporating the term ITT;; Z) into the previous objective function Eq. ( our final objective function becomes:
max I(Y;T)- n
T.X +
Σ [TTky; Z)
where Tk, are the local anchored features selected by Algorithm T and M is the number of local anchored features. An illustrative figure can be found in Appendix Figure
5

---

Published as a conference paper at ICLR 2021
In addition, due to the intractabilitty of computing MI, we use InfoNCE van den Oord et al. 2018)
as the lower bound of MI to approximate the last term (T,;; Z):
((nfONNC) (T;; 2) Ep Io (li,2) Ep log Σ ca (
(10)
where (:,) ) is a score function (or critic function) approximated by a neural network, t, are the positive samples drawn from the joint distribution F of local anchored features and global repre- sentations, and i are the negative samples drawn from the distribution nonrobust and unuseful features P.
EXPERIMENTS
In this section, we demonstrate how effective InfoBERT improves the robustness of language models over multiple NI.P tasks such as NLI and QA. We evaluate InfoBERT against both strong adversarial datasets and statc-of-the-art adversarial attacks.
4.1 EXPERIMENTAL. SETUP
The following ad adversarial datasets and adversarial attacks are used to eval- uate the robustness of InfoBERT and baselines. (I) Adversarial NLI (ANLI) (Nie et al.] 2020) is large-scale NLI benchmark, collected via an iterative, adversarial, human-and andou--ndd-mddl----ee in--the- loop procedure to attack BERT and RoBERTa. ANLI dataset is a strong adversarial dataset which can easily reduce the accuracy of BERT Large to 0%. (II) Adversarial SQuAD (Jia & Liang, 2017) dataset an adversarial QA benchmark dataset generated by a set handcrafted rules and refined by crowdsourcing. Since adversarial training data is not provided, we fine-tune RoBERTa Lagee on benign SQuAD training data (Rajpurkar et al.] 2016) only, and test the models on both benign and adversarial test sets. (III) TextFooler (Jin et al. 2020 is the state-of-the-ar word-level adversarial attack method to generate adversarial examples. To create an adversarial evaluation dataset, we sam- pled 1, 000 examples from the test sets of SNLI and MNLI respectively, and run TextFooler against BERT and RoBERTar arge to obtain the adversarial text examples.
Adversarial Datasets
Baselines Since IBP-hased methods (Huang al.) 2019 Jia al. 2019) cannot be applied to large- scale language models yet, and the randomized-smoothing baased method (Ye et al. 2020) achieves limited certified robustness, we compare InfoBERT against three competitive baselines based on adversarial training: (I) FreeLB (Zhu et al. (2200a) applies adversarial training to language models during fine-tuning stage improve generalization. In $4.2] we observe that Free can boost the
robustness of language models by a large margin. (II) SMART (Jiang et al. 2020) uses adversarial
training as mnoothhness-nnducin regularization and Bregman proximal point optimization during fine-tuning, to improve the generalization and robustness of language models. (III) ALUM (Liu al. 2020) performs adversarial training in both pre-training and fine-tuning stages, which achieves substantial performance gain wide range of NLP tasks. Due to the high computational cost of adversarial training, we compare InfoBERT to ALUM and SMART with the best results reported in the original papers.
Evaluation Metries We use robust accuracy or robust Fl score to measure how robust the baseline models and InfoBERT when facing adversarial data. Specifically. robust accuracy is calculated by: Acc = D EmeDDa [arg max qu (x')) = yl. where Daa is the adversarial dataset, is the ground-truth label, arg max selects the class with the highest logits and L [()) is the indicator func- tion. Similarly, robust F1 score is calculated by: F1
|Pass Ee' E v(arg max qu (z(())) a), where v(, -) is the Fl score between the true answer and the predicted answer arg max qu (fo (a')). and arg max selects the answer with the highest probability (sec Rajjurrka et al. (2016) for details).
Implementation Details To demonstrate InfoBERT is effective for different language models, we apply InfoBERT to both pretrained RoBERTaLare and BERT Since InfoBERT can be applied to both standard training and adversarial training, here use FreeLB as the adversarial training implementation. InfoBERT is fine-tuned for epochs for the QA task, and epochs for the NLI task. More implementation details such a, B, Ch. Q selection can be found in Appendix A.1

---

Published as a conference paper at ICLR 2021
Dev
Test
Training
Model
Method
A1
A2
A3
ANLI
A1
A2
A3
ANLI
Vanilla
49.1
26.5
27.2
33.8
49.2
27.6
24.8
33.2
RoBERTa
Standard Training
InfoBERT
47.8
31.2
31.8
36.6
47.3
31.2
31.1
36.2
Vanilla InfoBERT
20.7
26.9
31.2
26.6
21.8
28.3
28.8
26.5
BERT
26.0
30.1
31.2
29.2
26.4
29.7
29.8
28.7
FreeL.B InfoBERT
50.4
28.0
28.5
35.2
48.
30.4
26.3
34.4
RoBERTa
Adversarial
48.4
29.3
31.3
36.0
50.0
30.6
29.3
36.2
Training
FreeLB
23.0
29.0
32.2
28.3
22.2
28.5
30.8
27.4
BERT
InfoBERT
28.3
30.2
33.8
30.9
25.9
28.1
30.3
28.2
Table I: Robust accuracy the ANLI dataset. Models trained the benign datasets (MNLI SNLI) only. 'A1-33' refers to the rounds with increasing difficulty. 'ANLI' refers to A1+A2+A3.
Dev
Test
Training
Model
Method
A1
A2
A3
ANLI
A1
A2
A3
ANLI
Vanilla
74.1
50.8
43.9
55.5
73.8
48.9
44.4
53.7
RoBERTa
Standard
InfoBERT
75.2
49.6
47.8
56.9
73.9
50.8
48.8
57.3
Training
Vanilla
58.5
46.1
45.5
49.8
57.4
48.3
43.5
49.3
BERT
InfoBERT
59.3
48.9
45.5
50.9
60.0
46.9
44.8
50.2
FreeLB
75.2
47.4
45.3
55.3
73.3
50.5
46.8
56.2
SMART
74.5
50.9
47.6
57.1
72.4
49.8
50.3
57.1
RoBERTa
ALUM
73.3
53.4
48.2
57.7
72.3
52.1
48.4
57.0
Adversarial Training
InfoBERT
76.4
51.7
48.6
58.3
75.5
51.4
49.8
58.3
FreeLB
60.3
47.1
46.3
50.9
60.3
46.8
44.8
50.2
BERT
ALUM
62.0
48.6
48.1
52.6
61.3
45.9
44.3
50.1
InfoBERT
60.8
48.7
45.9
51.4
63.3
48.7
43.2
51.2
Table 2: Robust accuracy on the ANLJ dataset. Models are trained on both adversarial and benign datasets (ANLI (training) FeverNLI + MNLI SNLI).
4.2 EXPERIMENTAL RESULTS
Evaluation on ANLI As ANLI provides an adversarial training dataset, we evaluate models two settings: 1) training models benign data (MNLI (Williams et al. 2018 + SNLI (Bowman al.) 2015) only, which the case when the adversarial threat model is unknown; 2) training models on both benign and adversarial training data WANLI-MNLHAANLL-Feeeeet
which assumes the
threat model known in advance.
Results of the first setting summarized in Table The vanilla RoBERTa and BERT models perform poorly on the adversarial dataset. In particular, vanilla BERT with standard training achieves the lowest robust accuracy of 26.5% among all the models. We also evaluate the robust- ness improvement by performing adversarial training during fine-tuning, and observe that adversar- ial training for language models can improve only generalization but also robustness. In contrast, InfoBERT substantially improves robust accuracy in both standard and adversarial training. The ro- bust accuracy of InfoBERT through standard training is even higher than the adversarial training baseline FreeI.B for both RoBERTa and BERT, while the training time of InfoBERT is 1/3 1/2 less than FreeI... This is mainly because FreeL.B requires multiple steps of PGD attacks to gener- ate adversarial examples, while InfoBERT essentially needs only 1-step PGD attack for anchored feature selection.
Results of the second setting are provided in Table 2, which shows InfoBERT can further improve robust accuracy for both standard and adversarial training. Specifically, when combined with adver- sarial training, InfoBERT achieves the oftte-of-the-art robust accuracy of 58.3%. outperforming all existing baselines. Note that although ALUM achieves higher accuracy for BERT on the dev set, tends to overfit on the dev set, therefore performing worse than InfoBERT on the test set.

---

Published as a conference paper at ICLR 2021
MNLI
adv-SNLI (BERT)
adv-MNLI (BERT)
adv-SNLI (RoBERTa)
adv-MNLI
Training
Model
Method
SNLI
(m/mm)
(RoBERTa)
Vanilla InfoBERT
92.6
90.8/90.6
56.6
68.1/68.6
19.4
24.9/24.9
RoBERTa
Standard Training
93.3
90.5/90.4
59.8
69.8/70.6
42.5
50.3/52.1
Vanilla InfoBERT
91.3
86.7/86.4
0.0
0.0/0.0
44.9
57.0/57.5
BERT
91.7
86.2/86.0
36.7
43.5/46.6
45.4
57.2/58.6
RoBERTa
FreeLB InfoBERT
93.4
90. 90.1/90.3
60.4
70.3/72.1
41.2
49,5/50.6
Adversarial
93.1
90.7/90.4
62.3
73.2/73.1
43.4
56.9/55.5
Training
FreeI.B
92.4
86.9/86.5
46.6
60.0/60.7
50.5
64.0/62.9
BERT
InfoBERT
92.2
87.2/87.2
50.8
61.3/62.7
52.6
65.6/67.3
Table 3: Robust accuracy on the adversarial SNLI and MNLL(-m//m) datasets generated by TextFooler based on blackbox BERT/RoBERTa (denoted in brackets of the header). Models are trained on the benign datasets (MNLI+SNLI) only.
MI Improvement after adding adv examples in the training set
Training
Method
benign
AddSent
AddOneSent
0.08
Standard
Vanilla
93.5/86.9
72.9/66.6
80.6/74.3
0.05
Training
InfoBERT
93.5/87.0
78.5/72.9
84.6/78.3
0.04
0.02
Adversarial
FreeLB ALUM
93.8/87.3
76.3/70.3
82.3/76.2
0
Alg
4fg
Aly Benign Test
Aly
Training
75.5/69.4
81,4/75,9
Adversarial Test Data
InfoBERT
93.7/87.0
78.0/71.8
83.6/77.1
Figure 1: Local anchored features con- tribute more to MI improvement than nonrobust/unuseful features, unveiling closer relation with robustness.
Table 4: Robust F1/EM scores based on RoBERTa on the adversarial SQuAD datasets (AddSent and AddOne- Sent). Models are trained standard SQuAD 1.0 dataset.
Evaluation against TextFooler InfoBERT defend against not only human-crafted adversarial examples (e.g., ANLI) but also those generated by adversarial attacks (e.g.. TextFooler). Results are the benign
summarized in 3 We can see that InfoBERT barely affects model performance test data, and in the case of adversarial training, InfoBERT even boosts the benign test accuracy. Un- der the TextFooler attack, the robust accuracy of the vanilla BERT drops 0.0% on both MNLI and SNLI datasets. while RoBERTa drops from 90% to around 20%. We observe that both adversarial training and InfoBERT with standard training can improve robust accuracy by comparable large margin, while InfoBERT with adversarial training achieves the best performance among all models, confirming the hypothesis in Theorem 3.2 that combining adversarial training with IB regularizer can further minimize 7 T(X;; TV), leading to better robustness than the vanilla one.
Evaluation on Adversarial SQuAD Previous experiments show that InfoBERT can improve model robustness for NLI tasks. Now we demonstrate that InfoBERT can also be adapted to other NLP tasks such as QA in Table 4 Similar to our observation on NLI dataset, we find that InfoBERT cases. Moreover, InfoBERT substantially improves model robustness when presented with adversarial QA test sets (AddSent and AddOneSent). While adversarial training does help improve robustness, InfoBERT can further boost the robust performance by a larger margin. In particular, InfoBERT through stan- dard training achieves the state-of-the-ar robust FI/EM score as 78.5/72.9 5/72.9 compared to existing adversarial training baselines, and in the meantime requires only half the training time of adversarial- training- based methods.
barely hurts the performance the benign test data, and even improves it in
4.3 ANALYSIS OF LOCAL ANCHORED FEATURES
We conduct an ablation study to further validate that our anchored feature regularizer indeed fil- ters out nonrobust/unusefu information. As shown in Table [ and 2] adding adversarial data in the training set can significantly improve model robustness. To find out what helps improve the robustness from the MI perspective, first calculate the MI between anchored features and global features M Σn [Tk;: Z) on the adversarial test data and benign test data, based on the model trained without adversarial training data (denoted by 1/ and IR). We then calculate the MI between
M nonrobust/unuseful features and global features 1 T(TR,5 ; Z) the adversarial data and
8

---

Published as a conference paper at ICLR 2021
benign data well (denoted by IN and Ix). After adding adversarial examples into the training set and re-training the model, we find that the MI between the local features and the global features substantially increases the adversarial test data, which accounts for the robustness improvement. We also observe that those local anchored features extracted by our anchored feature regularizer, expected, contribute more to the MI improvement. As shown in Figure [ the MI improvement of anchored features on ad adversarial test data Alg (red bar on the left) is higher than that of nonro- bust/unuseful Al (red bar on the right), thus confirming that local anchored features discovered by our anchored feature regularizer have stronger impact on robustness than nonrobust/unuseful ones.
We conduct more ablation studies in Appendix [A.2 including analyzing the individual impact of two regularizers, the difference between global and local features for IB regularizer. apper-paramtter selection strategy and so on.
CONCLUSION
In this paper, we propose novel learning framework InfoBERT from an information theoretic per- spective to perform robust fine-tuning over pre-trained language models. Specifically, InfoBERT consists of two novel regularizers to improve the robustness of the learned representations: (a) In- formation Bottleneck Regularizer, learning to extract the approximated minimal sufficient statistics and denoise the excessive spurious features, and (b) Local Anchored Feature Regularizer, which improves the robustness of global features by aligning them with local anchored features. Supported by our theoretical analysis, InfoBERT provides a principled way to improve the robustness of BERT and RoBERTa against strong adversarial attacks over variety of NLP tasks, including NLI and QA tasks. Comprehensive experiments demonstrate that InfoBERT outperforms existing baseline methods and achieves new state of the different adversarial datasets. We believe this work will shed light on future research directions towards improving the robustness of representation learning for language models.
ACKNOWLEDGEMENT
We gratefully thank the anonymous reviewers and meta-reviewers for their constructive feedback. We also thank Julia Hockenmaier, Alexander Schwing, Sanmi Koyejo, Fan Wu, Wei Wang. Pengyu Cheng, and many others for the helpful discussion. This work partially supported by NSF grant No. 1910100,,, DARPA QED- NED-RML-FP-OOL and the Intel RSA 2020.
REFERENCES
Moustafa Alzantot, Yash Sharma, Ahmed Elgobary, Bo-Jhang Ho, Mani B. Srivastava, and Kai-Wei Chang- Generating natural language adversarial examples. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun 'ichi Tsujii (eds.), EMNLP, pp. 2890-2896. Association for Computational Linguistics, 2018
David Barber and Felix V. Agakov. The im algorithm: variational approach to information maxi- mization. In NeurIPS, 2003.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In Jennifer Dy and An- dreas Krause (eds.). Proceedings of the 35th International Conference Machine Learning, vol- ume 80 of Proceedings of Machine Learning Research, pp. 531-540, Stockholmsmässan, Stock- holm Sweden, 10-15 Jul 2018. PMLR.
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher Manning. large an- notated corpus for learning natural language inference. In Lluís Mårquez, Chris Callison-Burrd Burch, Jian Su, Daniele Pighin, and Yuval Marton (eds.), EMNLP, pp. 632-642. The Association for Computational Linguistiss,, 2015.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voes Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M.

---

Published as a conference paper at ICLR 2021
Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. 2020.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. CoRR, abs/2002.055099, 2020.
Pengyu Cheng, Weituo Hao, Shuyang Dai. Jiachang Liu, Zhe Gan, and Carin. Club: contrastive log-ratio upper bound of mutual information. ArXiv, abs/2006. 12013, 2020.
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via random- ized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), ICML, volume 97 of Proceedings of Machine Learning Research, pp. 1310-1320. PMLR, 2019.
Jacob Devlin, Ming-Way Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), NAACL-HLT, pp. 4171-4188 Association for Computational Linguistics, 2019.
Krishnamurthy Dvijotham, Sven Gowal, Robert Stanforth, Relja Arandjelovic, Brendan O'Donoghue, Jonathan Uesato, and Pushmeet Kohli. Training verified learners with learned ver- ifiers. CoRR, 100/1805.10265, 2018.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In Iryna Gurevych and Yusuke Miyao (eds.), ACL, pp. 31-36. Association for Computational Linguistics, 2018.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Xiaodong Song. Robust physical-world attacks on deep learning models. 2017.
Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for aakoon-andllanuuagee representation learning. arXiv preprint wrxxvv2006.06//90 2020.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial 2015.
examples. CoRR,
Devon Hjelm, Alex Fedorov, Samuel Maroie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In /CLR, 2019.
Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krish- namurthy Dvijotham, and Pushmeet Kohli. Achieving verified robustness to symbol substitutions via interval bound propagation. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), EMNLP-IJCNLP, pp. 4081-0099 1091.. Association for Computational Linguistics, 2019.
Mohit lyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. Adversarial example generation with syntactically controlled paraphrase networks. In Marilyn A. Walker, Heng Ji, and Amanda Stent (eds.), NAACI-HLLT, pp. 1875-1885. Association for Computational Linguistics, 2018.
Joern-Henrik Jacobsen, Jens Behrmann, Richard Zemel, and Matthias Bethge. Excessive invariance causes adversarial vulnerability. In ICLR, 2019.
Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), EMNLP, pp. 2021-2031. Association for Computational Linguistics, 2017.
Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified robustness to adversarial word substitutions. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), EMNLP- IJCNLP, pp. 4127-4140. Association for Computational Linguistics, 2019.
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. SMART: robust and efficient fine-tuning for pre-trained natural language models through principled regu- larized optimization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), ACI., pp. 2177-2190. Association for Computational Linguistics, 2020.
10

---

Published as a conference paper at ICLR 2021
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In AAAI, pp. 8018-8025. AAAI Press, 2020.
Lingpeng Kong, Cyprien de Masson d' d'Autume, Lei Yu, Wang Ling, Zihang Dai, and Dani Yo- gatama. A mutual information maximization perspective of language representation learning. In ICLR, 2020.
Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. NDSS, The Internet Society, 2019.
Ralph Linsker. Self-organiiztiin in perceptual network. Computer, 21(3):105-117, 1988.
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. CoRR, abs/2004.08994, 2020.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907. 1692, 2019.
Seyed-Mohsen Moosavi-Dezfool Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. CVPR, pp. 2574-2582, 2016.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adver- sarial NLI: A new benchmark for natural language understanding. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.),ACL) pp. 4885 4901. Associatto for Computational Linguisties, 2020.
Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as defense to adversarial perturbations against deep neural networks. 2016 IEEE Symposium Security and Privacy (SP), pp. 582-597, 2016.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. In Jian Su, Xavier Carreras, and Kevin Duh (eds.), EMNLP, pp. 2383-2392. The Association for Computational Linguistics, 2016.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Anna Korhonen, David R. Traum, and Lluís Märquez (eds.), ACL, pp. 1085-1097. Association for Computational Linguistics, 2019.
Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. A theoretical analysis of contrastive unsupervised representation learning. In Kamalika Chaud- huri and Ruslan Salakhutdinov (eds.), ICML, volume 97 of Proceedings of Machine Learning Research, pp. (628-5637, PMLR, 2019.
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning. CoRR, Abs/2005.10243 2020.
N. Tishby and N. Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pp. 1-5, 2015.
Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. CoRR, abs/1807, 007400 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), NeurIPS, pp. 5998-6008, 2017.
Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, and Bo Li. T3: Tree- autoencoder constrained adversarial text generation for targeted attack. In EMNLP, 2020.
Adina Williams, Nikita Nangia, and Samuel R. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Marilyn A. Walker, Heng Ji. and Amanda Stent (eds.), NAACI-HLLT, pp. 1112-1122. Association for Computational Linguistics, 2018

---

Published as a conference paper at ICLR 2021
Mao Ye, Chengyue Gong, and Qiang Liu. SAFER: A structure- free approach for certified robustness to adversarial word substitutions. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), ACL, pp. 3465-3475. Association for Computational Linguistics, 2020.
Yue Yu, Simiao Zuo, Haoming Jiang, Wendi Ren, Tuo Zhao, and Chao Zhang. Fine-tuning pre- trained language model with weak supervision: A contrastive-regulrized self-training approach. CoRR, abs/2010.07835, 2020.
Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual adversarial attacking as combinatorial optimization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), ACL, pp. 6066-6080. Association for Com- putational Linguistics, 2020.
Yuan Zhang, Jason Baldridge, and Luheng He. PAWS: paraphrase adversaries from word scram- bling. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), NAACL-IILT, pp. 1298-1308. Association for Computational Linguisties, 2019.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. In ICLR. OpenReview.nnt, 2020a.
Sicheng Zhu, Xiao Zhang, and David Evans. Learning adversarially robust representations via worst-case mutual information maximization. CoRR, abs/202..1199, 2020b.
12

---

Published as a conference paper at ICLR 2021
A APPENDIX
(a) Task Objective
maximize
I(Y:T)
ng
T;)
iTey
(b) Information Bottleneck Regularizer
(c) Local Anchored Feature Regularizer
Accuracy
Benign Acc
Global Features
4
1TTx 2
Adverssrial Ace
BERT Encoder
Benign Accuracy
barely drops
BERT Encoder
BERT Encoder
Bath aggresive
Adversartal Robustness
improves
comprission
Features
Local
T
To
T1
T2
T4
T6
Te
T6
T9
T10
Embedding
Σ (x); decreases
(CLS) woman. both sitting
poker chips
Input
are playing cards. [SEP] Two woman playing poker (SEP)
Figure 2: The complete objective function of InfoBERT, which can be decomposed into (a) standard task objective, (b) Information Bottleneck Regularizer, and (c) Local Anchored Feature Regularizer. For (b), we both theoretically and empirically demonstrate that robustness by decreasing the mutual information of I [[x;; Te) without affecting the benign accuracy much. For (c), we propose to align the local anchored features Tk, (highlighted in Yellow) with the global feature 7 by maximizing their mutual information T(Tky; 2)
improve the adversarial
A.1 IMPLEMENTATION DETAILS
Model Details BERT is transformer Vaswani et al. 2017 based model, which is unsupervised pretrained on large corpora. We use BERT -nccseee as the baseline model, which has 24 lay- ers, 1024 hidden units, 16 self-attention heads, and 340M parameters. RoBERTa shares the same architecture as BERT, but modifies key hyperparameters, removes the next-sentence pretrain- ing objective and trains with much larger mini-batches and learning rates, which results in higher performance than BERT model on GLUE, RACE and SQuAD.
Standard Training Details For both standard and adversarial training, we fine-tune InfoBERT for 2 epochs on the QA task, and for 3 epochs on the NLI task. The best model is selected based on the performance on the development set. All fine-tuning experiments are run on Nvidia V100 GPUs. For NLI task, we set the batch size to 256, learning rate to 2 10-3, max sequence length to 128 and warm-up steps to 1000. For QA task, we set the batch size to 32, learning rate to x 10 - and max sequence length 384 without warm-up steps.
Adversarial Training D Adversarial training introduces hyper-parametery including ad- versarial learning rates, number PGD steps, and adversarial norm. When combing adversarial training with InRoBERT, we use FreeLB as the adversarial training implementation. and set adver- sarial learning rate to 10 or 4*10 1002 adversarial steps to 3, maximal perturbation norm to 3 or 2 10 and initial random perturbation norm to 10 or 0.
Information Bottleneck Regularizer Details For information bottleneck, there are different ways to model p(t] æ):
Assume that p(t | x) is unknown. We use neural net parameterized by qa ( x) learn the conditional distribution p(t æ). We assume the distribution is a Gaussian distribution. The
use the huggingface implementation for BERT and RoBERTa.
We follow the Freel. implementations in httppp////ghhhheeeccccccccceeeea
13

---

Published as a conference paper at ICLR 2021
neural net qe will learn the mean and variance of the Gaussian given input and representation t. By reparameterization trick, the neural net can be backpropagated to approximate the distribution given the training samples.
2. p(t | x) is known. Since t is the representation encoded by BERT, we actually already know the distribution of p. We also denote it as qo. where 0 is the parameter of the BERT encoder fo. If assume the conditional distribution is a Gaussian (t,, σr) for input whose mean is the BERT representation t; and variance fixed constant Ø, the beoobes
LLB Σ
(1oggo(e0) [(00]
-
Σ
n
do)il
N Σ n
d
(11)
where c(or is a positive constant related to a. In practice, the sample f from the conditional distribution Gaussian ((t,, 0) be I; with some Gaussian noise, an adversarial examples of t,, or & itself (assume a =
We use the second way to model p(t x) for InfoBERT finally, as it gives higher robustness im- provement than the first way empirically (shown in the following {A.2)) We suspect that the main reason is because the first way needs to approximate the distribution pl xr) via another neural net which could present some difficulty model training.
Information Bottleneck Regularizer also introduces another parameter ,9 to tune the trad-off between representation compression / (X,;; Ti) and predictive power ((;;T), We search for the optimal B via grid search, and set B 5 x 10 for RoBERTa, 3 = 10 for BERT on the NLI task. On the QA task, we set _ = 5 x 10-5, which is substantially lower than / on NLI tasks, thus containing more word-level features. We think it is mainly because the QA task relies more the word-level representation to predict the exact answer spans. More ablation results can be found in the following
A.2
Anchored Feature Regularizer Details Anchored Feature Regularizer uses a to weigh the bal- ance between predictive power and importance of anchored feature. We set 5 10 for both NLI and QA tasks. Anchored Feature Regularizer also introduces upper and lower threshold and cn for anchored feature extraction. We set сд = 0.9 and e = 0.5 for the NLI task, and set O₁ 0.95 and 0.75 for the QA task. The neural MI estimator used by infoNCE uses two-layer fully connected layer to estimate the MI with the intermediate layer hidden size set to 300.
A.2 ADDITIONAL. EXPERIMENTAL RESULTS
A.2.1 ABLATION STUDY ON INFORMATION BOTTLENECK REGULARIZER
Modeling p(t I) As discussed in A.I have two ways to model p(t z): ( using an auxiliary neural network to approximate the distribution; (ii) directly using the BERT encoder fo to calculate the p(t 2).. Thus we implemented these two methods and compare the robustness improvement in Table [5] To eliminate other factors such as Anchored Feature Regularizer and = 3 B- 5 10-2 and conduct the following ablation experiments via standard training on standard datasets. We observe that although both modeling methods can improve the model robustness, modeling as BERT encoder gives larger margin than the Auxiliary Net. Moreover, the second way barely sacrifices the performance on benign data, while the first way can hurt the benign accuracy little bit. Therefore, we use the BERT Encoder fo model the p(t æ) in our main paper.
adversarial training, we
Local Features v.s. Global Features Information Bottlenck Regularizer improves model robust- ness by reducing T") In the main paper, use T as word-level local features. Here we consider as sentence-level global features, and compare the robustness improvement with T as local features. To eliminate other factors such as Anchored Feature Regularizer and adversarial training. we set a = 0, B = 5 x 10 and conduct the following ablation experiments via standard training.
14

---

Published as a conference paper at ICLR 2021
Adversarial Accuracy (ANLI)
Benign Aceuracy (MNLI/SNLI)
Model
Datasets
Method
Vanilla
26.5
86.7/91.3
Standard
BERT
Datasets
Auxiliary Net
27.1
83.1/90.7
BERT Encoder fo
27.7
85.9/91.7
Table 5: Robust accuracy the ANLI dataset. Here we refer "Standard Datasets" as training on the benign datasets (MNLJ SNLI) only. "Vanilla" refers the vanilla BERT trained without Information Bottleneck Regularizer.
The experimental results are summarized in Table 6
We can see that while both features can boost the model robustness, using local features yield higher robust accuracy improvement than global features, especially when adversarial training dataset is added.
Hyper-parameter Search We perform grid search to find out the optimal & so that the optimal trade-off between representation compression ("minimality") and predictive power ("sufficiency") is achieved. An example to search for the optimal 5 on QA dataset is shown in Fingure 3 which illustrates how affects the Fl score on benign and adversarial datasets. We can see that from very small 3, both the robust and benign Fl scores increase, demonstrating InfoBERT can improve 99.9), InfoBERT achieves the best benign and adversarial accuracy. When we set larger / to further minimize ((X;; we observe that the benign Fl score starts to drop, indicating the increasingly compressed representation could start to hurt its predictive capability.
both robustness and generalization some extent. When we set
10-5 (log(3)
A.2.2 ABLATION STUDY ON ANCHORED FEATURE REGULARIZER
Visualization of Anchored Words To explore which local anchored features are extracted, we conduct another ablation study visualize the local anchored words. We follow the best hyper- parameters of Anchored Feature Regularizer introduced in {A.1 use the best BERT model trained benign datasets (MNLI SNLI) only and test the ANLI dev anchored words in Table 7 as follows. In the first example, we find that Anchored Features mainly focus on the important features such quantity number "Two", the verb "playing" and objects "card"/"poker" to make a robust prediction. In the second example, the matching robust features between hypothesis and premise, such as "people", "roller" v.s. "park", "flipped upside" v.s. "ride", are aligned to infer the relationship of hypothesis and premise. These anchored feature examples confirm that Anchored Feature Regularizer is able find out useful and stable features improve the robustness of global representation.
We visualize the local
Adversarial Accuracy (ANLI)
Benign Accuracy (MNLI/SNLD
Model
Datasets
Features
Vanilla
33.2
90.8/92.6
Standard Datasets
Global Feature
33.8
90.4/93.5
RoBERTa
Local Feature
33.9
90.6/93.7
Standard
Vanilla
53.7
91.0/92.6
and Adversarial
Global Feature
55.1
90.8/93.3
Datasets
Local Feature
56.2
90.5/93.3
Table 6: Robust accuracy on the ANLI dataset. Here we refer "Standard Datasets" as training on the benign datasets (MNLI + SNLI) only, and "Standard and Adversarial Datasaets" as training the both benign and adversarial datasets (ANLI(tiianin) + MNLI + SNLI + FeverNLI). "Vanilla" refers to the vanilla RoBERTa trained without Information Bottleneck Regularizer.
15

---

Published as a conference paper at ICLR 2021
Benign/Robust F1 Benign/Adversarial SQuAD Dataset
Benign F1
94.0
Adv (AddSent)
-9
Adversani Pl (AddSent)
78
93.5
77
Benign
76
92.5
75
92.0
74
-12
-11
-10
-8
-7
log
Figure 3: BBenign/robust F1 score benign/adversarial QA datasets. Models trained the benign SQuAD dataset with different B.
Input (bold local stable words for local anchored features.)
Premise: Two woman, both sitting near pile of poker chips, are playing cards. Hypothesis: Two woman playing poker.
Premise: People are flipped upside down bright yellow roller coaster.
Hypothesis: People
ride amusement park.
Table 7: Local anchored features extracted by Anchored Feature Regularizer.
A.2.3 ABLATION STUDY ON DISENTANGLING Two REGULARIZERS
To understand how two regularizers contribute to the improvement of robustness separetely, we apply two regularizers individually to both the standard training and adversarial training. We refer InfoBERT trained with IB regularizer only as "InfoBERT (IBR only)" and InfoBERT trained with Anchored Feature Regularizer only as "InfoBERT (AFR only)". "InfoBERT (Both)" is the standard setting for InfoBERT, where we incorporate both regularizers during training. For "InfoBERT (IBR only)". we a and perform grid search find the optimal 3 5 x 10-2, Similarly for "InfoBERT (AFR only)", we set 3 = 0 and find the optimal parameters as a = 5 x 10-3, ch = 0.9 and a 0.5.
We can see that both regularizers improve the robust accuracy on top of vanilla and FreeLB to similar margin. Applying one of the regularizer can achieve similar performance of FreeLB, but the training time of InfoBERT is only 1/31/2 less than FreeLB. Moreover, after combining both regularizers, we observe that InfoBERT achieves the best robust accuracy.
The results are shown in Table 8
A.2.4 EXAMPLES OF ADVERSARIAL DATASETS GENERATED BY TEXTFOOLER
We show some adversarial examples generated by TextFooler in Table 9 We can see most adver- sarial examples are of high quality and look valid to human while attacking the NLP models, thus confirming our adversarial dastasets created by TextFooler is strong benchmark dataset to evaluate model robustness. However, as also noted in Jin et al. (2020), we observe that some adversarial examples look invalid to human For example, in the last example of Table 9 TextFooler replaces "stand" with "position", losing the critical information that "girls are standing instead of kneeling" and fooling both human and NLP models. Therefore, expect that InfoBERT should achieve better robustness when we eliminate such invalid adversarial examples during evaluation.
16

---

Published as a conference paper at ICLR 2021
Adversarial Accuracy (ANLI)
Benign Accuracy (MNLISNLI)
Model
Training
Method
Vanilla
26.5
86.7/91.3
Standard
InfoBERT (IBR only)
27.7
85.9/91.7
Training
InfoBERT (AFR only)
28.0
86.6/91.9
BERT
InfoBERT (Both)
29.2
85.9/91.6
FreeLB
27.7
86.7/92.3
Adversarial Training
InfoBERT (IBR only)
29.3
87.0/92.3
InfoBERT (AFR only)
30.3
86.9/92.3
InfoBERT (Both)
30.9
87.2/92.2
Table Robust accuracy on the ANLI dataset. Models are trained on the benign datasets (MNLI SNLI). Here we refer "IBR only" as training with Information Bottleneck Regularizer only. "AFR only" refers to InfoBERT trained with Anchored Feature Regularizer only. "Both" the standard InfoBERT that applies two regularizers together.
Input (red = Modified words, bold = original words.)
Valid Adversarial Examples
young boy is playing the sandy water. Original Hypothesis: There is a boy in the water. Adversarial Hypothesis: There is man in the water.
Premise:
Model Prediction:
Entailment Contradiction
Premise: black and brown dog is playing with brown and white dog Original Hypothesis:
Two dogs play.
Adversarial Hypothesis: Two dogs gaming.
Model Prediction:
Entailment + Neutral
Premise: Adults and children share in the looking at something, and some young ladics stand to the side. Original Hypothesis:
Some children sleeping.
Adversarial Hypothesis: Some children dreaming.
Model Prediction:
Contradiction - Neutral
Premise: Families with strollers waiting front of carousel.
Original Hypothesis: Families have some dogs in front of carousel.
Adversarial Hypothesis: Families have some doggic in front of carousel.
Model Prediction:
Contradiction -> Entailment
Invalid Adversarial Examples
Two girls are kneeling the ground. Original Hypothesis:
Premise:
Two girls stand around the vending machines.
Adversarial Hypothesis: Two girls position around the vending machinery.
Model Prediction: Contradiction + Neutral
Table 9: Adversarial Examples Generated by TextFooler for BERT Lange Large SNLI dataset.
17

---

Published as a conference paper at ICLR 2021
A.3 PROOFS
A.3.1 PROOF OF THEOREM
We first state lemmas.
Lemma A.1. Given sequence of random variables X1, X2,, X, and a deterministic function f, then V i,j = 2,.,.... n,, we have
f(x)((X)) 2 [(X;;((X)
(12)
Proof. By the definition,
((x;.((x)) ((((=))) H(F(X))) X2) ((X::f(X)) H(((X))) H(((X) Xz)
(13)
(14)
Since is a deterministic function,
((((X,) [X,) -0
(15)
((f(x) x3)20
(16)
Therefore,
I(Xxf(x)) 2 I(X;
(17)
Lemma A.2. Let X2; X,, be a sequence of random variables, and T
be sequence of random variables generated by deterministic function f. Then we have
[T:: Ti: Tr] [f(x)): f(X)): :X.;)....; ...; f(x,)
I(X:T) Sn n > I(x
(18)
Proof. Since X = X2i... X,) and T [Ti: T2: Tn] are language tokens with its corre-
sponding local representations, we have
I(X;T)= = I(x;T, Tr) = Σ STT Ti-1 H(T: X,Ti, T2 Ti-1)) 1)]
(19)
Σ H(T)) H(T: XX) Σ ((;;T))
(20)
T(X;; $ Σ T(x,T))
(21)
where the first inequality follows because conditioning reduces entropy, and the last inequality is because (.,;T.) 2 T:) based on Lemma A.1
Then we directly plug Lemma A.2 into Theorem
3.1 we have the lower bound of LiB as
I(Y:T) - (I(;TT) 2 I(Y;T) - n,3 Σ I(
(22)
A.3.2 PROOF OF THEOREM 3.2
We first state easily proven lemma,
18

---

Published as a conference paper at ICLR 2021
Lemma A.3. For any a, E [0, 1].
a loo[(a) b blog(b) s 8(1a- b)),
(23)
where (())RR+ + R4 is defined as
T= 0 0 x I <>
d(x)=
x logg( 1)
(24)
It is easy to verify that ((x) is continuous, monotonically increasing, concave and subadditive function.
Now, we can proceed with the proof of Theorem 3.2
Proof.
We use the fact that
I(Y:T) I(Y:T") S (H [ H(M)
(25)
and bound each of the summands on the right separately.
We can bound the first summand as follows:
HTT (Y) - H(T Y) S Σ
- Y = v)l
(26)
Er(w)
Ep(1 v) log(1/((1) w1) - Σ a(l v) log(1/((( v))l
p(l
Ea(
(27)
Σ C(()) Σ lp(t (v) log v) (t log q(t m)l
(28)
Ep(w) Σ M(p(1 v) - -((( 1))))
(29)
Σ ((u) Σ M{ Σ p(t <)(p(< v) - g(xr 1))]).
(30)
where
p(> 11) ply | x)p(x)
(31)
P(y x)p(z)
q(x v) ply x))(())
(32)
ExP(y x)q(z)
Since Exerxx p(x y) - q(x v) 0 for any y V, we have that for any scalar a,
Σ p(t x)(p(< v) - g(x v)])
(33)
E(p( w)) a)(p(> I w) - g(x v))1
(34)
S E (oll (x) 1x)- a)² Σ 1) - g( u))?
(35)
1 Setting a 1²-x11 Σ ExEXUx p(t xx) we get
H(T (Y) - H(T | (Y) < Σ p(y) Σ DVVV(p(( x x x |p(r v) q( [v)l2)),
(36)
19

---

Published as a conference paper at ICLR 2021
where for any real-valus vector (a1,... an), V(a) is defined to be proportional to the variance of elements a:
1 1=1 T2 ==1
((a) =
a4
aj)
(37)
p(t I E XUX) stands for the vector in which entries are p(t x) with different values of x E XUX for a fixed t, and p( y) and q(z y) are the vectors in which entries are p(z y) and q(xl v). respectively, with different values of X U X' for fixed y.
Since
ll v) - q(x v)lll S ||p( - 11)l 2,
(38)
it follows that
HH( Y) - H(T f) Σ Eplv) V (39)
Moreover, have
xE) +V) x(X
(40)
S [vV(p(t)
I =
(41)
where the first inequality is because sample mean is the minimizer of the sum of the squared dis- tances each sample and the second inequality due the subadditivity of the square root func- tion. Using the fact that Φ(-) is monotonically increasing and subadditive, we get
HH H(T Y) <
Σ p(y) (2
+ Σ p(v) Σ c(2/V(p((
(42)
Now we explicate the process for establishing the bound for E, p(v) E, (022V(ppl a æ x))) and the one for E,,(w) E, (((( X')) can be similarly derived.
By definition of V(-) and using Bayes' theorem p(t x:) p(t)p(u)e) for X, we have that p(x)
VV(p(t x E x)) p(0)
Σ
p(x t)
p(x' t)
(43)
p(x)
X
p(x')
ZEX
X'EX
20

---

Published as a conference paper at ICLR 2021
Denoting 1 (1...., 1), we have by the triangle inequality that
pla It) p(x)
p(x'
y2
(44)
[2
p(x')
ZEN
X'EX
p( 1)
plxl 1)
- ||
1112 +
Σ
(1
Σ
(45)
p(x)
p(xe)
ZEN
x'EX
p( (1)
10)0
1112 +
|2
(46)
p(x)
|.8
p(a')
x'
x'EX
p( 6)
p(a'
(1))
1112 +
(121-
Σ
)²
(47)
p(x)
V
|
p(xo))
x'EX
p( t) p(x)
1112 +
1
Σ (1
pla' [t),
(48)
ViX
p(x!)
x'EX
p( ()
1
p( (1)
1112
1|11
(49)
p(r)
ViX
p(x)
p( t)
< (1 +
1/11
(50)
V
p(x)
2
|p( () p(=)ll
(51)
min p(x)
From an inequality linking KL-divergence and the 1, norm, we have that
| t) - p(x) S v kuul [t)]
(52)
Plugging Eq. (52) into Eq. ST and using Eq. 433 we have the following bound:
B z0(t/dd
V(((t D x))
S
(53)
4 (og(() e pla) and de DKa(p(= (())).
where B
We will first proceed the proof under the assumption that Bd( S for any f. We will later see that this condition can be discarded. If Bp(t) s : then
Σ (22VV(p(t x x
(54)
Bp(t)/ de (log( |) + +log( p(t)de") ))
(55)
Blog( 1
Σ p(t() + B Σ p(t) do log(
1
(56)
B
p(t)d,
+ )lp((t) dill1 + Blly p(t)
< B B log(
(57)
where the last inequality is due to easily proven fact that for any > 0, x log( S Vz. We p(t) and (t) are vectors comprising pf) and de with different values of t, respectively.
Using the following two inequalities:
[p(()) S VITiIl(()v/lll v
(58)
and
|| p(t) V(Tllly dellz
(59)
= VVT 8/4
|l p(t)dill2
(60)
21

---

Published as a conference paper at ICLR 2021
have
[o(2VV(p(t e x)))
B Boo
1
/TTll///tttdll BIT
v B
/lvp(t)dellz (61)
B
Using the equality
Ily lttddllll = /EDaaapcalliieet = VI(X;T),
(62)
we reach the following bound
Co(2vV(p(t (EX)))
(63)
s Blog( (x: (x;
(64)
Plug Lemma A.2
into the equation above, we have
Σ (2VV(p(( x EE))))
(65)
1 s log(
1/2
3/
T(X;;
(66)
1
S Voglog( 17 B
Σ
14 BT 3/4 Σ IXX;
(67)
We now show the bound is trivial if the assumption that Bp(t) dg S does not hold. If the assumption does not hold, then there exists at such that Bp(t) de 1. Since
VI(X:T)= = v Σ E((t)de > Σp(tva 2 P()
(68)
for any I, we get that VI(;TT) is at least
o Since 1T 2 1 and C C20 0, we get that our bound in Eq. 633
B log
|1T| 1/2 (xx; 1/2 + 3/4 (XX
(69)
B
log(1/B)
BT/2 1/4
(70)
e//2
Let [(c)
1o)(1/e)
e/2 T 1/4
can be verifed that ('(c) > Of > Since > 4 2 log(2)
e472
by the definition of B, we have f(B) f(4 2 (2g 0.746. Therefore, we have
B log(
1/2 B 3/4 (x;
(71)
B
2 0.746
IT log([T)
(72)
Therefore, if indeed Bp(t) > for some t, then the bound in Eq. 663 is trivially true, since H(T Y) is within [0, log(())) Similarly, we can establish bound for
Σ, (2/VV/( E X'))) follows:
( 1T 1/2 n
Σ (22VV(p( x'))) S VnB log(
I(X; +m T/ Σ T(XTT)
(73)
where B' 4, 4 (2)
m q(x)
22

---

Published as a conference paper at ICLR 2021
Plugging Eq. 77 and Eq. 665
into Eq. 142
we get
[ Y) H(T
H(T Y) S log
IT
1/2
+ 3/4
Σ
I(X
ToBllog'
11
1/2 + 1/1 BT 3/4
Σ T;)
B
(74)
Now we turn to the third summand in Eq. 25). we have to bound (
Recall the definition of e-bounded adversarial example. We denote the set of the benign data repre- sentation that within the e-ball of L' by Q(²"). Then for any E Q(t'), we have
||t
t.ll e,
(75)
for i = 1, n. We also denote the number of the e-bounded adversarial examples around the benign representation by c(t). Then we have the distribution of adversarial representation as follows:
p())
a(l')
Σ
(76)
c(t)
(EQ())
H(T) H(T')
(77)
Σ ptt) logp(t))
Σ gtt logg(t')
(78)
p(t))
p(()
Ep(t)log p(t)
Σ
[(
log(
Σ
c(t) I
(79)
c(t)
(EQ(!))
(EQ(I)
p(()
p(()
< Σ p(t)
Σ
Σ
log(
(80)
ctt)
ctt
(EQ())
clt)) p())
(()
Σ p(t) log
Σ
log
c(t) )
(81)
c(t)
Σ p(t) log
(82)
where the inequality is by log sum inequality. If we denote the C max, c(t) which is the maxi- mum number of e-bounded textual adversarial examples given a benign representation i of a word sequence v, we have
(T)) H(T)
(83)
Σ p(t) log c(t)
(84)
I Σ p(t) log Cl = log C.
(85)
Note that given a word sequence of with representation t, the number of c-bounded textual adversarial examples c(t) is finite given finite vocabulary size. Therefore, if each word has at most candidate word perturbations, then log C S nlog k can be viewed as some constants depending only and e.
Now, combining Eq.
25 Eq. 74 and Eq. 85). we prove the bound in Theorem 3.2
23


Reviews:
[{"review_id": "qOM3wROZy57", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work (InfoBERT) proposes additional objectives for transformer finetuning to obtain models more robust to adversarial inputs. The authors first propose a mutual information based information bottleneck objective, next the authors propose an adversarial loss inspired method for identifying robust features and a subsequent objective to emphasize the mutual information between global representations and these robust features. The experiments demonstrate that InfoBERT consistently outperforms other adversarial training approaches on a variety of adversarial evaluations.\n\nI largely follow the logic behind the derivation, however I find some of the details unclear. I would like to see proofs for the theorems as well as an explanation of the assumptions under which the theorems hold. The experimental results are convincing, however there are no ablation studies to disentangle the performance contributions of the two proposed objectives. For the first point, the questions I have are as follows:\n\nfor equations 1-3, I find the integral notation to be a bit odd - isn't it common practice to put the dydt at the very end of the integral? Also you should consider leaving out punctuations from equations\nfor equation 5, why is there another 1/N inside the square brackets?\nwhy is equation 7 true in the general case? Suppose n=1, is this essentially saying that any sample from an empirical distribution would provide a lower bound for the true distribution?\nI'd like to see a proof for Theorems 3.1 and 3.2\nhow do the authors define stability and robustness? The manuscript talk about them in vague terms and they do not seem to be precisely defined\nhow does equation 9 follow from 6? Can you put in the intermediate steps? Also in this case what is N and what is M? And what happened to the multiplier n from equation 6?", "extra_scores": {}, "extra_text": {}}, {"review_id": "i_dm6rU--D_", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary:\nThe paper proposes two regularizers for finetuning pretrained mask LMs to improve the robustness of NLI and SQuAD models. When evaluated on adversarial NLI and SQuAD datasets, adding the regularizers to regular finetuning achieves a robust accuracy comparable to adversarial training baselines; adding the regularizers are added to adversarial training baselines archives extra robustness gains.\n\nThe first regularizer is an implementation of the Information Bottleneck principle specialized for contextual text representations. In the general IB objective, we seek to maximize the mutual information between the representation and the label, as well as minimizing the mutual information between the representation and the input. In this paper, the authors targets the token-level BERT representation. This design choice was not discussed in detail, but I assume it’s motivated by the observation that our models’ lack of robustness is often manifested in an overreliance on local, superficial features.\n\nThe second regularizer has a similar motivation. But instead of minimizing the mutual information between the input and the representation, the “anchoring feature” regularizer minimizes the mutual information between the global representation and the token level ones, specifically those that are “nonrobust and unuseful”. To identify “nonrobust and unuseful” tokens in each input, the authors use a heuristic based on input gradient norm, similar to how interpretability people generate heatmaps for text classification.\n\n## Detailed comments\nInterpretation of experimental results and choice of baselines: The abstract claims that “InfoBERT achieves state-of-the-art robust accuracy”. This is not accurate. The best numbers reported in this paper are achieved by applying InfoBert regularizers to FreeLB adversarial training. This can be seen as an ensemble of two (or three, if you count the two regularizers separately since they can be applied independent of each other) adversarial training methods. Ensembling usually helps robustness (see for example Tramèr et.al ICLR 2018 Ensemble Adversarial Training: Attacks and Defenses). For a fair comparison, FreeLB should be ensemble with another adversarial training method, or with FreeLB applied to a second model. When applied individually, the gain from InfoBERT has a much smaller advantage compared to the baselines.\n\nMissing evaluations and ablations: An obvious ablation is missing: apply the two regularizers separately. I’m especially curious if both lead to gains on top of FreeLB. The paper has a sort of disproportionate treatment of the two regularizers. Both theorem 3.1 and 3.2 are talking about the IB regularizer, while a lot of design choices for the anchoring feature regularizer are proposed without justification or verification, e.g. the portion of useful and robust tokens. The anchoring feature regularizer relies on various heuristics (definition of usefulness and robustness), and if it turns out to be the main contributing factor to InfoBERT, it would be good to know - if others would like to apply InfoBERT on other tasks, they might need to tune these hyperparameters.\nFormulation of the method: The authors cite the “localization” of the IB principle in the IB regularizer as part of the novelty of the method. However, this kind of localization can be found in e.g. Li & Eisner 2019: Specializing Word Embeddings (for Parsing) by Information Bottleneck (EMNLP), which is one of the first applications of IB principle to NLP with pretrained, contextualized representations. In the anchoring feature regularizer, the use of input gradient norm for filtering “nonrobust and unuseful” tokens is reminiscent of how interpretation methods generate saliency maps for text classification. Both are missing from the references.\n\nImplications and verification of theorem 3.2: this is a minor point but in my opinion theorem 3.2 is a bit of an overkill just to solidify the intuition that “the performance gap becomes closer when I(X_i, T_i) and I(X’_i, T’_i) decreases”. It would be nice to verify empirically through experiment that this theorem is correct.\n\nFinally, I encourage the authors to evaluate the method on more tasks and attacks, and especially focus on comparing against the naive adversarial training baseline. It would be good to have a better understanding of how much gain InforBERT brings and what are the most important factors.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "GWQ5A8zbRRp", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n##########################################################################\n\nSummary:\n\nThe paper proposes a novel learning framework for robust (against adversarial attacks) fine-tuning of pre-trained language models, that is based on information theoretic arguments. It introduces two regularization mechanisms and investigates their efficacy on various tasks.\n\n##########################################################################\n\nReasons for score: \n\nOverall I vote for accept. The approach is novel, interesting and well presented. The theoretical results seem to be sound. It also seems to outperform competitors in the field of adversarial language models. Concerning the experiments some questions remain, but I hope that the authors will address them in the rebuttal.\n\n##########################################################################\nPros: \n\n1. The idea is interesting and well formulated. The theoretical results seem to be correct to me.\n\n2. The approach is tested on several standard datasets used in adversarial language models. It seems to outperform previous approaches.\n\n3. The paper is well written and clearly structured\n\n\n##########################################################################\n\nCons: \n\n\n1. In my view the experiments seem to show a tendency towards a slightly worse performance on the more difficult tasks in comparison to the competitor methods. Thus, the better overall performance on the ANLI data could be driven by the easier tasks.  \n\n\n2. I couldn't find a clear description of the \"global representation\" Z. A more explicit description would be helpful.\n\n\n##########################################################################\n\nQuestions during rebuttal period: \n\n\nPlease address and clarify the cons above \n\n\n#########################################################################\n\nMinor comments: \n\n1. Is definition 3.1. a standard definition or is it introduced by the authors? \n\n2. Page 4, definition 3 contains an incomplete sentence (\"The q(x')....\").\n\n3. Page 6, \"Evaluation Metrics\": it should be stated witch argument is maximized.\n\n4. Page 16, Lemma A1: in the Proof of the lemma I think that all instances of Y_i should be replaced with T_i. In formula (13), in the rightmost term the token index n should be i-1.\n\n5. Page 17, formula (33): H(Y|Y) should probably be H(T|Y). Same goes for equ. (36).\n\n6. Squares might be missing in formulas (37) to (43).\n\n7. A reference to formula (44) would be nice.  \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces two regularizers that are meant to improve out-of-domain robustness when used in the fine-tuning of pretrained transformers like BERT. Results with ANLI and Adversarial SQuAD are encouraging.

Pros:
- New method with concrete improvements in several difficult task settings.
- New framing of adversarial generalization.

Cons:
- The ablations that are highlighted in the main paper body don't do a good job of isolating the specific new contributions. (Though the appendix provides enough detail that I'm satisfied that the main empirical contribution is sound.)
- Reviewers found the theoretical motivation very difficult to follow in places.
Decision: Accept (Poster)