SlowGuess/ACL_2020_OCR huggingface dataset:
Prasetya Ajie Utama†‡, Nafise Sadat Moosavi‡, Iryna Gurevych‡
$^{\dagger}$ Research Training Group AIPHES
†Ubiquitous Knowledge Processing Lab (UKP-TUDA)
Department of Computer Science, Technische Universität Darmstadt
Models for natural language understanding (NLU) tasks often rely on the idiosyncratic biases of the dataset, which make them brittle against test cases outside the training distribution. Recently, several proposed debiasing methods are shown to be very effective in improving out-of-distribution performance. However, their improvements come at the expense of performance drop when models are evaluated on the in-distribution data, which contain examples with higher diversity. This seemingly inevitable trade-off may not tell us much about the changes in the reasoning and understanding capabilities of the resulting models on broader types of examples beyond the small subset represented in the out-of-distribution data. In this paper, we address this trade-off by introducing a novel debiasing method, called confidence regularization, which discourage models from exploiting biases while enabling them to receive enough incentive to learn from all the training examples. We evaluate our method on three NLU tasks and show that, in contrast to its predecessors, it improves the performance on out-of-distribution datasets (e.g., 7pp gain on HANS dataset) while maintaining the original in-distribution accuracy.
Despite the impressive performance on many natural language understanding (NLU) benchmarks (Wang et al., 2018), recent pre-trained language models (LM) such as BERT (Devlin et al., 2019) are shown to rely heavily on idiosyncratic biases of datasets (McCoy et al., 2019b; Schuster et al., 2019; Zhang et al., 2019). These biases are commonly characterized as surface features of input examples that are strongly associated with the target labels, e.g., occurrences of negation words in
natural language inference (NLI) datasets which are biased towards the contradiction label (Gururangan et al., 2018; Poliak et al., 2018). As a ramification of relying on biases, models break on the out-of-distribution data, in which such associative patterns between the surface features and the target labels are not present. This brittleness has, in turn, limited their practical applicability in some extrinsic use cases (Falke et al., 2019).
This problem has sparked interest among researchers in building models that are robust against dataset biases. Proposed methods in this direction build on previous works, which have largely explored the format of several prominent label-revealing biases on certain datasets (Belinkov et al., 2019). Two current prevailing methods, product-of-expert (He et al., 2019; Mahabadi and Henderson, 2019) and learned-mixin (Clark et al., 2019a) introduce several strategies to overcome the known biases by correcting the conditional distribution of the target labels given the presence of biased features. They achieve this by reducing the importance of examples that can be predicted correctly by using only biased features. As a result, models are forced to learn from harder examples in which utilizing solely superficial features is not sufficient to make correct predictions.
While these two state-of-the-art debiasing methods provide a remarkable improvement on the targeted out-of-distribution test sets, they do so at the cost of degrading the model's performance on the in-distribution setting, i.e., evaluation on the original test data which contains more diverse inference phenomena. It raises a question on whether these debiasing methods truly help in capturing a better notion of language understanding or simply biasing models to other directions. Ideally, if such an improvement is achieved for the right reasons (i.e., better reasoning capabilities by learning a more general feature representation), a debiased model
<sup>1</sup>The code is available at https://github.com/UKPLab/ac12020-confidence-regularization
8717
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8717-8729 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
product-of-expert	learned-mixin	conf-reg(our)
in-distribution	↓	↓	↓
out-of-distribution	↓	↓	↓
calibration	↓	↓	↓
requires biased model	✓	✓	✓
requires hyperparameter	×	✓	×
Table 1: Comparison of our method against the state-of-the-art debiasing methods. Learned-mixin (Clark et al., 2019a) is a parameterized variant of Product-of-expert (He et al., 2019; Mahabadi and Henderson, 2019). Our novel confidence regularization method improves the out-of-distribution performance while optimally maintain the in-distribution accuracy.
should still be able to maintain its accuracy on previously unambiguous instances (i.e., instances that are predicted correctly by the baseline model), even when they contain biases.
In this work, we address this shortcoming by introducing a novel debiasing method that improves models' performance on the out-of-distribution examples while preserves the in-distribution accuracy. The method, called confidence regularization, draws a connection between the robustness against dataset biases and the overconfidence prediction problem in neural network models (Feng et al., 2018; Papernot et al., 2016). We show that by preventing models from being overconfident on biased examples, they are less likely to exploit the simple cues from these examples. The motivation of our proposed training objective is to explicitly encourage models to make predictions with lower confidence (i.e., assigning a lower probability to the predicted label) on examples that contain biased features.
Table 1 shows the comparison of our method with the existing state-of-the-art debiasing methods: product-of-expert and learned-mixin. We show that our method is highly effective in improving out-of-distribution performance while preserving the in-distribution accuracy. For example, our method achieves 7 points gain on an out-of-distribution NLI evaluation set, while slightly improves the in-distribution accuracy. Besides, we show that our method is able to improve models' calibration (Guo et al., 2017) so that the confidence of their predictions are more aligned with their accuracies. Overall, our contributions are the following:
- We present a novel confidence regularization method to prevent models from utilizing bi
ased features in the dataset. We evaluate the advantage of our method over the state-of-the-art debiasing methods on three tasks, including natural language inference, fact verification, and paraphrase identification. Experimental results show that our method provides competitive out-of-distribution improvement while retaining the original in-distribution performance.
- We provide insights on how the debiasing methods behave across different datasets with varying degrees of biases and show that our method is more optimal when enough bias-free examples are available in the dataset.
Biases in Datasets Researchers have recently studied more closely the success of large fine-tuned LMs in many NLU tasks and found that models are simply better in leveraging biased patterns instead of capturing a better notion of language understanding for the intended task (Bender and Koller, 2020). Models' performance often drops to a random baseline when evaluated on out-of-distribution datasets which are carefully designed to be void of the biases found in the training data. Using such targeted evaluation, McCoy et al. (2019b) observe that models trained on MNLI dataset (Williams et al., 2018) leverage syntactic patterns involving word overlap to blindly predict entailment. Similarly, Schuster et al. (2019) show that the predictions of fact verification models trained for the FEVER task (Thorne et al., 2018) are largely driven by the presence of indicative words in the input claim sentences.
Following similar observations across other tasks and domains, e.g., visual question-answering (Agrawal et al., 2016), paraphrase identification (Zhang et al., 2019), and argument reasoning comprehension (Niven and Kao, 2019), researchers proposed improved data collection techniques to reduce the artifacts that result in dataset biases. While these approaches are promising, only applying them without additional efforts in the modeling part may still deliver an unsatisfactory outcome. For instance, collecting new examples by asking human annotators to conform to specific rules may be costly and thus limit the scale and diversity of the resulting data (Kaushik et al., 2020). Recently proposed adversarial filtering methods (Zellers et al., 2019; Sakaguchi et al., 2019) are more cost effective but are not guaranteed to be artifacts-free. It is,
8718
therefore, crucial to develop learning methods that can overcome biases as a complement to the data collection efforts.
Debiasing Models There exist several methods that aim to improve models' robustness and generalization by leveraging the insights from previous work about the datasets' artifacts. In the NLI task, Belinkov et al. (2019) make use of the finding that partial input information from the hypothesis sentence is sufficient to achieve reasonable accuracy. They then remove this hypothesis-only bias from the input representation using an adversarial training technique. More recently, three concurrent works (Clark et al., 2019a; He et al., 2019; Mahabadi and Henderson, 2019) introduce a model-agnostic debiasing method for NLU tasks called product-of-expert. Clark et al. (2019a) also propose an adaptive variant of this method called learned-mixin. These two methods first identify examples that can be predicted correctly based only on biased features. This step is done by using a biased model $^{2}$ , which is a weak classifier that is trained using only features that are known to be insufficient to perform the task but work well due to biases. The output of this pre-trained biased model is then used to adjust the loss function such that it down-weights the importance of examples that the biased model can solve. While this approach prevents models from learning the task mainly using biased features, it also reduces model's ability to learn from examples that can be solved using these features. As a result, models are unable to optimize accuracy on the original training distribution, and they possibly become biased in some other ways.
Similar to these methods, our method also uses a biased model to identify examples that exhibit biased features. However, instead of using it to diminish the training signal from these examples, we use it to scale the confidence of models' predictions. This enables the model to receive enough incentive to learn from all of the training examples.
Confidence Regularization Methods for regularizing the output distribution of neural network models have been used to improve generalization. Pereyra et al. (2017) propose to penalize the entropy of the output distribution for encouraging models to be less confident in their predictions. Previously, Szegedy et al. (2016) introduce a label smoothing mechanism to reduce overfitting by pre
venting the model from assigning a full probability to each training example. Our method regularizes models' confidence differently: we first perform an adaptive label smoothing for the training using knowledge distillation (Hinton et al., 2015), which, by itself, is known to improve the overall performance. However, our method involves an additional bias-weighted scaling mechanism within the distillation pipelines. As we will show, our proposed scaling mechanism is crucial in leveraging the knowledge distillation technique for the purpose of overcoming the targeted bias while maintaining high accuracy in the training distribution.
Similar to our work, Feng et al. (2018) propose a regularization method that encourages the model to be uncertain on specific examples. However, the objective and the methodology are different: they apply an entropy penalty term on examples that appear nonsensical to humans with the goal of improving models' interpretability. On the contrary, we apply our confidence regularization on every training example with a varying strength (i.e., higher uncertainty on more biased examples) to improve models' performance on the out-of-distribution data.
Overview We consider the common formulation of NLU tasks as a multi-class classification problem. Given a dataset  $\mathcal{D}$  that consists of  $n$  examples  $(x_{i},y_{i})_{i\in [1,n]}$ , with  $x_{i}\in \mathcal{X}$  as a pair of sentences, and  $y_{i}\in \{1,2,\dots,K\}$  where  $K$  is the number of classes. The goal is to learn a robust classifier  $\mathcal{F}_m$ , which computes the probability distribution over target labels, i.e.,  $\mathcal{F}_m(x_i) = p_i$ .
The key idea of our method is to explicitly train  $\mathcal{F}_m$  to compute lower probability, i.e., less confidence, on the predicted label when the input example exhibits a bias. This form of confidence regularization can be done by computing the loss function with the "soft" target labels that are obtained through our proposed smoothing mechanism. The use of soft targets as the training objective is motivated by the observation that the probability distribution of labels for each sample provides valuable information about the underlying task (Hinton et al., 2015; Pereyra et al., 2017). When the soft targets of certain examples have higher entropy, models can be explicitly taught that some labels are more likely to be correct than the others. Based on this intuition, we argue that adjusting the con
2We follow the terminology used by He et al. (2019).
8719
Figure 1: An overview of our debiasing strategy when applied to the MNLI dataset. An input example that contains lexical-overlap bias is predicted as entailment by the teacher model with a high confidence. When biased model predicts this example well, the output distribution of the teacher will be re-scaled to indicate higher uncertainty (lower confidence). The re-scaled output distributions are then used to distill the main model.
fidence on soft labels can better inform the model about the true conditional distribution of the labels given the presence of the biased features.
We first produce a meaningful softened target distribution for each training example by performing knowledge distillation (Hinton et al., 2015). In this learning framework, a "teacher" model  $\mathcal{F}_t$ , which we parameterize identically to the main model  $\mathcal{F}_m$ , is trained on the dataset  $\mathcal{D}$  using a standard classification loss. We then use  $\mathcal{F}_t$  to compute output probability distribution  $\hat{p_i}$ , where  $\mathcal{F}_t(x_i) = \hat{p_i}$ . In the original knowledge distillation approach, the output of the teacher model  $\hat{p_i}$  is then used to train  $\mathcal{F}_m$ . We extend this approach by adding a novel scaling procedure before we distill the teacher model into  $\mathcal{F}_m$ . We define a scaling function  $S$  that takes the probability distribution  $\hat{p_i}$  and scale it such that the probability assigned to its predicted label is lowered when the example can be predicted well by only relying on the biased features.
Training the biased model For several NLU tasks, biased features are known a-priori, e.g., the word overlapping features in NLI datasets are highly correlated with the entailment label (McCoy et al., 2019b). We leverage this a-priori knowledge to design a measure of how well an example can be predicted given only the biased features. We refer to this measure as bias weight, denoted as  $\beta_{i}$  for every example  $x_{i}$ .
Similar to previous debiasing methods (Clark et al., 2019a), we compute bias weights using a biased model. This biased model, denoted as  $\mathcal{F}_b$ , predicts the probability distribution  $b_{i}$ , where  $\mathcal{F}_b(x_i) = b_i = \langle b_{i,1}, b_{i,2}, \dots, b_{i,K} \rangle$ . We define the bias weight  $\beta_{i}$  as the scalar value of the as
signed probability by  $\mathcal{F}_b$  to the ground truth label:  $\beta_{i} = b_{i,c}$  (c-th label is the ground truth).
Bias-weighted scaling As illustrated in Figure 1, our method involves scaling the teacher output  $\hat{p_i}$  using  $\beta_{i}$ . We do this by defining a scaling function  $S:\mathbb{R}^K\to \mathbb{R}^K$  ..
$$
\mathcal {S} (\hat {p _ {i}}, \beta_ {i}) _ {j} = \frac {\hat {p _ {i , j}} ^ {(1 - \beta_ {i})}}{\sum_ {k = 1} ^ {K} p _ {\hat {i} , k} ^ {(1 - \beta_ {i})}}
$$
for  $j = 1, \dots, K$ . The value of  $\beta_{i}$  controls the strength of the scaling: as  $\beta_{i} \to 1$ , the scaled probability assigned to each label approaches  $\frac{1}{K}$ , which presents a minimum confidence. Conversely, when  $\beta_{i} \to 0$ , the teacher's probability distribution remains unchanged, i.e.,  $S(\hat{p}_i, 0) = \hat{p}_i$ .
Training the main model The final step is to train  $\mathcal{F}_m$  by distilling from the scaled teacher model's outputs. Since the main model is parameterized identically to the teacher model, we refer to this step as self-distillation (Furlanello et al., 2018). Self-distillation is performed by training  $\mathcal{F}_m$  on pairs of input and the obtained soft target labels  $(x_i, S(\hat{p}_i, \beta_i))$ . Specifically,  $\mathcal{F}_m$  is learned by minimizing a standard cross-entropy loss between the scaled teacher's output  $S(\hat{p}_i, \beta_i)$  and the current prediction of the main model:
$$
\mathcal {L} \left(x _ {i}, \mathcal {S} \left(\hat {p} _ {i}, \beta_ {i}\right)\right) = - \mathcal {S} \left(\hat {p} _ {i}, \beta_ {i}\right) \cdot \log \mathcal {F} _ {m} \left(x _ {i}\right)
$$
In practice, each  $S(\hat{p}_i,\beta_i)$  is computed only once as a preprocessing step. Our method does not require hyperparameters, which can be an advantage since most out-of-distribution datasets do not provide a development set for tuning the hyperparameters.
In this section, we describe the datasets, models, and training details used in our experiments.
4.1 Natural Language Inference
We use the MNLI dataset (Williams et al., 2018) for training. The dataset consists of pairs of premise and hypothesis sentences along with their inference labels (i.e., entailment, neutral, and contradiction). MNLI has two in-distribution development and test sets, one that matches domains of the training data (MNLI-m), and one with mismatching domains (MNLI-mm). We consider two out-of-distribution datasets for NLI: HANS (Heuristic Analysis for NLI Systems) (McCoy et al., 2019b) and MNLI-hard test sets (Gururangan et al., 2018).
HANS The dataset is constructed based on the finding that the word overlapping between premise and hypothesis in NLI datasets is strongly correlated with the entailment label. HANS consists of examples in which such correlation does not exist, i.e., hypotheses are not entailed by their word-overlapping premises. HANS is split into three test cases: (a) Lexical overlap (e.g., "The doctor was paid by the actor"  $\Rightarrow$  "The doctor paid the actor"), (b) Subsequence (e.g., "The doctor near the actor danced"  $\Rightarrow$  "The actor danced"), and (c) Constituent (e.g., "If the artist slept, the actor ran"  $\Rightarrow$  "The artist slept"). Each category contains both entailment and non-entailment examples.
MNLI-hard Hypothesis sentences in NLI datasets often contain words that are highly indicative of target labels (Gururangan et al., 2018; Poliak et al., 2018). It allows a simple model that predicts based on the hypothesis-only input to perform much better than the random baseline. Gururangan et al. (2018) presents a "hard" split of the MNLI test sets, in which examples cannot be predicted correctly by the simple hypothesis-only model.
4.2 Fact Verification
For this task, we use the training dataset provided by the FEVER challenge (Thorne et al., 2018). The task concerns about assessing the validity of a claim sentence in the context of a given evidence sentence, which can be labeled as either support, refutes, and not enough information. We use the Fever-Symmetric dataset (Schuster et al., 2019) for the out-of-distribution evaluation.
Fever-Symmetric Schuster et al. (2019) introduce this dataset to demonstrate that FEVER models mostly rely on the claim-only bias, i.e., the occurrence of words and phrases in the claim that are biased toward certain labels. The dataset is manually constructed such that relying on cues of the claim can lead to incorrect predictions. We evaluate the models on the two versions (version 1 and 2) of their test sets. $^3$
4.3 Paraphrase Identification
We use the Quora Question Pairs (QQP) dataset for training. QQP consists of pairs of questions which are labeled as duplicate if they are paraphrased, and non-duplicate otherwise. We evaluate the out-of-distribution performance of QQP models on the QQP subset of PAWS (Paraphrase Adversaries from Word Scrambling) (Zhang et al., 2019).
PAWS The QQP subset of PAWS consists of question pairs that are highly overlapping in words. The majority of these question pairs are labeled as non-duplicate. Models trained on QQP are shown to perform worse than the random baseline on this dataset. This partly indicates that models largely rely on lexical-overlap features to perform well on QQP. We report models' performance on the duplicate and non-duplicate examples separately.
4.4 Models
Baseline Model We apply all of the debiasing methods across our experiments on the BERT base model (Devlin et al., 2019), which has shown impressive in-distribution performance on the three tasks. In our method, BERT base is used for both  $\mathcal{F}_t$  and  $\mathcal{F}_m$ . We follow the standard setup for sentence pair classification tasks, in which the two sentences are concatenated into a single input and the special token [CLF] is used for classification.
Biased Model  $(\mathcal{F}_b)$  We consider the biased features of each of the examined out-of-distribution datasets to train the biased models. For HANS and PAWS, we use hand-crafted features that indicate how words are shared between the two input sentences. Following Clark et al. (2019a), these features include the percentage of hypothesis words that also occur in the premise and the average of cosine distances between word embedding in the premise and hypothesis.4 We then train a simple
3https://github.com/TalSchuster/FeverSymmetric
4We include the detailed description in the appendix.
8721
Method	MNLI-m	MNLI-mm	lex.	HANS	avg.	Hard subset
dev	test	dev	test	subseq.	const.	MNLI-m	MNLI-mm
BERT-base	84.3 ± 0.3	84.6	84.7 ± 0.1	83.3	72.4	52.7	57.9	61.1 ± 1.1	76.8	75.9
Learned-mixin_hans	84.0 ± 0.2	84.3	84.4 ± 0.3	83.3	77.5	54.1	63.2	64.9 ± 2.4	-	-
Product-of-expert_hans	82.8 ± 0.2	83.0	83.1 ± 0.3	82.1	72.9	65.3	69.6	69.2 ± 2.6	-	-
Regularized-conf_hans	84.3 ± 0.1	84.7	84.8 ± 0.2	83.4	73.3	66.5	67.2	69.1 ± 1.2	-	-
Learned-mixin_hypo	80.5 ± 0.4	79.5	81.2 ± 0.4	80.4	-	-	-	-	79.2	78.2
Product-of-expert_hypo	83.5 ± 0.4	82.8	83.8 ± 0.2	84.1	-	-	-	-	79.8	78.7
Regularized-conf_hypo	84.6 ± 0.2	84.1	85.0 ± 0.2	84.2	-	-	-	-	78.3	77.3
Table 2: The in-distribution accuracy (in percentage point) of the NLI models along with their accuracy on out-of-distribution test sets: HANS and MNLI hard subsets. Models are only evaluated against their targeted out-of-distribution dataset.
nonlinear classifier using these features. We refer to this biased model as the hans model.
For MNLI-hard and Fever-Symmetric, we train a biased model on only hypothesis sentences and claim sentences for MNLI and FEVER, respectively. The biased model is a nonlinear classifier trained on top of the vector representation of the input sentence. We obtain this vector representation by max-pooling word embeddings into a single vector for FEVER, and by learning an LSTM-based sentence encoder for MNLI.