PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning (2024)

Abstract: Counterfactually Augmented Data (CAD) involves creating new data samples by applying minimal yet sufficient modifications to flip the label of existing data samples to other classes. Training with CAD enhances model robustness against spurious features that happen to correlate with labels by spreading the casual relationships across different classes. Yet, recent research reveals that training with CAD may lead models to overly focus on modified features while ignoring other important contextual information, inadvertently introducing biases that may impair performance on out-ofdistribution (OOD) datasets. To mitigate this issue, we employ contrastive learning to promote global feature alignment in addition to learning counterfactual clues. We theoretically prove that contrastive loss can encourage models to leverage a broader range of features beyond those modified ones. Comprehensive experiments on two human-edited CAD datasets demonstrate that our proposed method outperforms the state-of-the-art on OOD datasets.

heading: 1 Introduction
text: In the field of Natural Language Processing (NLP), a significant body of research (McCoy et al., 2019; Wang and Culotta, 2020; Poliak et al., 2018; Gururangan et al., 2018) has raised the concern that deep learning models can overfit spurious correlations, such as dataset-specific artifacts and biases, rather than focusing on the more complex, generalizable task-related features. For example, Gururangan et al. (2018) and Poliak et al. (2018) demonstrate that classifiers trained exclusively on hypotheses can still achieve decent results on some Natural Language Inference (NLI) datasets, which ideally requires comparing hypotheses with premises to determine the labels. The existence of biases or
*Equal contribution. †Corresponding author.
shortcuts in training datasets can severely degrade the performance of deep learning models on outof-distribution (OOD) datasets.
Counterfactually Augmented Data (CAD) has emerged as a promising approach to mitigate this issue by making minimal modifications to existing data samples such that the corresponding labels are switched to other classes (Kaushik et al., 2020; Wen et al., 2022; Pryzant et al., 2023). This technique aims to establish direct causal relationships for models to learn more effectively and enhance generalization across different datasets (Teney et al., 2020; Kaushik et al., 2021).
However, the effectiveness of CAD is not always guaranteed, particularly when both contexts and the modified information should be considered together to make predictions (Joshi and He, 2022; Huang et al., 2020). For instance, in sentiment analysis, simply replacing positive adjectives such as “good” or “excellent” with negative counterparts like “terrible” or “bad” will potentially risk models to overemphasize these changes and even assign zero weights to the broader unmodified context (Joshi and He, 2022). Consequently, the trained models may fail to understand more nuanced expressions like irony or negation, exemplified by sentences such as “Is it a good movie ????” or “This movie is not that good.”
To solve the above risks of CAD training, an intuitive solution is to increase the diversity of counterfactual samples (Joshi and He, 2022; Sen et al., 2023), thereby disentangling the suspicious correlations between edited features and labels. Nonetheless, this kind of method often relies on human knowledge to steer the diversification, bearing high expenditure and time consumption (Huang et al., 2020). Others try to design additional constraints to align the model gradient with the straight line between the counterfactual example and the original input (Teney et al., 2020), or to minimize the invariant risk (Fan et al., 2024), but these attempts
11955
fail to exploit the complex effects of augmented feature components.
In this paper, we introduce a simple yet effective learning strategy to mitigate the overfitting problem associated with CAD. Inspired by the recent success of contrastive learning (CL) in feature alignment (Gao et al., 2021; Wang et al., 2022b; Liu et al., 2023a,b) and its strengths in capturing global relationships (Park et al., 2023), we propose to employ a contrastive learning objective to complement the standard cross-entropy (CE) loss. While CL compels the model to extract complementary effects among counterfactually augmented data to alleviate the feature degeneration, CE ensures the induced feature representations are effectively used for classification. Our mathematical proof further corroborates the advantage of combining the two losses in training models on CAD, resulting in enhanced generation capability.
In summary, our contributions are as follows: • We introduce a contrastive learning-based
framework, named Pairwisely Counterfactual Learning with Contrastive Regularization (PairCFR), for training models on CAD, which prevents overfitting to minor, nonrobust edits, thus enhancing generalization performance. • We provide theoretical proof for understanding the synergistic benefits of combining the CE and CL losses, unravelling their complementary effects in preventing models from relying solely on counterfactual edits for classification. • We conduct comprehensive experiments to demonstrate that the models trained under our learning framework achieve superior OOD generalization performance on two humanedited CAD datasets.

heading: 2 Related work
text: Counterfactually Augmented Data. Counterfactual examples (CFEs) suggest the minimal modifications required in an input instance to elicit a different outcome (Wachter et al., 2017; Barocas et al., 2020). This property has inspired researchers (Kaushik et al., 2020; Wu et al., 2021) to adopt CFEs as a meaningful data augmentation in NLP, aiming to mitigate spurious correlations and improve causal learning. Early efforts (Kaushik et al., 2020; Gardner et al., 2020) involved creating CAD datasets with manual sentence edits for label reversal. To ease the high cost of manual annotation, subsequent works adopt large language models (LLMs) for cost-effective generation of CAD (Wu et al., 2021; Madaan et al., 2021; Wen et al., 2022; Dixit et al., 2022; Pryzant et al., 2023; Chen et al., 2023). However, findings from various investigations have indicated that training on CAD does not always ensure improved generalization on OOD tasks (Huang et al., 2020; Joshi and He, 2022; Fan et al., 2024). Consequently, our emphasis in this work is not on generating CAD, but rather on the exploration of methodologies to effectively utilize the inherent prior knowledge within CAD. Contrastive Learning. Contrastive learning is initially proposed to learn a better embedding space by clustering similar samples closely while pushing dissimilar ones far apart (Schroff et al., 2015; Sohn, 2016; Oord et al., 2018; Wang and Isola, 2020). For example, the triplet loss (Schroff et al., 2015) minimizes the distance between an anchor point and its positive sample while maximizing the distance from a negative sample. The N-pair loss (Sohn, 2016) maximizes the distance between an anchor point with multiple negative points. Meanwhile, InfoNCE (Oord et al., 2018) separates positive samples from multiple noise samples with crossentropy loss. Enhanced by other efficient techniques, e.g., data augmentation (Chen et al., 2020), hard negative sampling (Schroff et al., 2015), and memory bank (Wu et al., 2018), CL has propelled significant advancements in various domains, under both supervised and unsupervised settings. In this section, we explore the untapped potential of CL to enhance the OOD generalization of models trained on CAD. Training with CAD. The task of effectively training a robust model with CAD has received relatively limited attention. The simple approach is to directly use the cross-entropy loss (Kaushik et al., 2020; Wen et al., 2022; Balashankar et al., 2023). To better exploit the causal relationship in counterfactual editing, Teney et al. (2020) have introduced gradient supervision over pairs of original data and their counterfactual examples, ensuring the model gradient aligns with the straight line between the original and counterfactual points. Meanwhile, Fan et al. (2024) considers original and counterfactual distribution as two different environments and proposes a dataset-level constraint using invariant risk minimization. Following these works, we introduce a learning framework employing contrastive loss as a regularizer to enhance the generalization
of fine-tuned models notably.

heading: 3 Methodology

heading: 3.1 Motivation
text: Recent studies have empirically shown that while perturbed features in CAD are robust and causal (Kaushik et al., 2020), they may inhibit the model’s ability to learn other robust features that remain unperturbed (Joshi and He, 2022). In this section, we mathematically demonstrate that the standard crossentropy loss, which is commonly used for training models on CAD, can exacerbate this tendency.
Given an instance x ∈ Rm×1, we train a singlelayer non-linear function fW (x) = σ(W Tx), where W ∈ Rm×1 and σ is the sigmoid function, to predict the label y ∈ {0, 1}. We expand x, whose label y = 1, as x = [xr, xc]T , where xr denotes the features to be revised (perturbed) and xc denotes the constant (unperturbed) features. The counterfactual example of x can be written as c = [cr, xc]
T , with label y = 0. As the sigmoid function is monotone and bounded, the cr and xr should have different signed values to ensure that x and c are classified differently. We expand the weights W = [wr, wc]T and take it into the function fW to obtain fW (x) = σ(wrxr + wcxc) and fW (c) = σ(wrcr+wcxc). The CE loss on the data x and its counterfactual c is calculated as
LCE(x, c) = −log(fW (x)) − log(1− fW (c)). (1)
By minimizing the CE loss, we enforce fW (x) to approach 1 and fW (c) to approach 0. Considering that xr and its counterpart cr have different signed values, we observe that optimizing wr can achieve the desired contrasting effect with less effort than optimizing wc. Therefore, the model tends to assign higher weights wr for revised features and lower weights wc for constant or unperturbed features. An expanded illustration can be found in the Appendix A. Similar phenomena are observed in both least squares loss (Joshi and He, 2022) and Fisher’s Linear Discriminant on CAD (Fan et al., 2024).
The above observations indicate that the CE loss alone can lead the model to focus on learning the revised features in CAD, which necessitates incorporating a regularization that compels the model to consider a broader range of features.

heading: 3.2 The Role of Contrastive Loss
text: Recent research findings have empirically shown that models trained under contrastive loss mainly focus on capturing global relationships (Park et al., 2023) compared with negative log-likelihood losses such as masked language modeling. Inspired by this, we propose to employ CL to complement standard CE loss for training models on CAD. In the following, we start from the introduction of CL loss and then mathematically show how CL encourages the model to select a broader range of features beyond the edited ones in the counterfactual data.
Given an anchor sample xi from a data batch D = {xi, yi}Ni=1, ∀xi ∈ D, we have its positive samples in Pi≡{xp|yp = yi, p ̸= i} and negative samples in Ni ≡{xn|yn ̸= yi, n ̸= i}, where Ni contains the counterfactual samples c for every xi. The contrastive loss for the anchor xi is
LCL=− E xp∈Pi
[ log
esip/τ
esip/τ + ∑
xn∈Ni e sin/τ
] ,
(2) where sxy = zx·zy ∥zx∥∥zy∥ measures the cosine similarity between the hidden representations of a pair of samples, and τ is a temperature scaling factor for controlling the extent to which we separate positive and negative pairs (Wang and Isola, 2020).
Without loss of generality, we assume W ∈ Rm×d that directly maps the input instance into a d-dimensional embedding space, zi = WTxi. To obtain the gradient of the CL loss coming from negative samples, we have
∂LCL ∂W ∣∣∣∣ sin = ∂LCL ∂sin × ∂sin ∂W
= 1
τ Pin ×AinW. (3)
The full derivation process can be found in the appendix B. Here, we have
Pin= E xp∈Pi
  e sin/τ
esip/τ+ ∑
xn∈Ni esin/τ
 , (4)
which indicates the probability of xi being recognized as xn. Ain = xixTn + xnx T i ∈ Rm×m is a symmetric matrix derived from the outer product of xi and xn. Each element of Ai,n indicates the digit-level dot product between the features of xi and xn, which provides a full view of the entire feature space when comparing a pair of samples. A
higher value leads to a larger gradient update and the weights W are optimized by considering the whole feature sets.
The above analysis implies that the CL loss has the capability of capturing global features beyond those being edited. When learning on CAD under CL, we pair each instance x with its CFE, c, to compel the model to disparate x from all negative samples, including its counterfactual example c:
∑
xn∈Ni esin/τ =esic/τ+
∑
xn∈Ni\c esin/τ , (5)
where the non-bold c is the index of CFE. Let us revisit the toy example with x = [xr, xc]T and c = [cr, xc]
T . Although minimizing the similarity between x and c encourages the model to focus on features xr, the other negative samples in the batch, e.g., x′ = [x′r, x ′ c] T , will enforce the model to use both wr and wc to compare the difference. Hence, the existence of real negative samples could help the model capture the relationships between xr and its context xc.
As all sin equally contribute to updating the model weights, the number of non-CFE negatives moderates the learning from local CAD and global patterns. A smaller batch size will manifest the influence of edited features, whereas a larger batch size may dilute the local differences in CAD, as discussed in the experiments 5.4.

heading: 3.3 Overall Learning Framework
text: Next, we introduce our proposed learning framework, Pairwisely Counterfactual Learning with Contrastive Loss Regularization, named PairCFR for short. As shown in Figure 1, a model f can be decomposed into two modules, ϕ(·) and φ(·), i.e., f = φ ◦ ϕ, where ϕ(·) encodes the input sentence into a hidden embedding, and φ(·) maps ϕ(x) for classification. For transformer-based models, we instantiate ϕ(x) using the [CLS] representation, denoted as z. We explicitly pair the original sentences x and their CFEs, c, in the same batch to
provide additional training signals indicative of the underlying causal relationships.
The standard cross-entropy loss is computed on the logits vector projected from φ(z). Optimizing CE loss enforces φ(·) to identify a small set of features from z and assign them higher weights to quickly reach a local minimum while optimizing CL loss compels ϕ(·) to consider the entire feature space of z to meet the distance constraints. Overall, we combine the two losses as follows.
L = λLCL + (1− λ)LCE , (6) where λ is the trade-off factor to balance the two losses. To compute CL on a batch, we sample positive pairs that have the same label while all the negative samples including the CFE of the anchor sample are considered.

heading: 4 Experimental Setup
text: In the following, we introduce experimental settings, which include benchmark tasks, evaluation metrics, competitive baselines and implementation details. Our code is released on GitHub 1.

heading: 4.1 Benchmark Tasks & Evaluations
text: We evaluate our learning framework on two NLP tasks, sentiment analysis (SA) and natural language inference (NLI). We use two human-edited CAD datasets (Kaushik et al., 2020), which ensures goodquality counterfactual data (Sen et al., 2023), to train all the models. The IMDb augmented dataset contains 4880 data samples with an original to CFE ratio of 1:1. The SNLI dataset contains 11330 data samples with an original to CFE ratio of 1:4. The statistics of human-revised CAD are reported in Appendix C.1.
To eliminate the random effect, we train each model for multiple runs (10 runs for SA and 7 runs for NLI) using different random seeds. We report the average test accuracy, standard deviation for both in-domain (ID) datasets and several out-of-domain (OOD) datasets. We also conduct significance tests by calculating p-value, to ensure that the observed improvements are not due to randomness. The details of ID and OOD datasets used for evaluation are described in Appendix C.2.

heading: 4.2 Implementation Details
text: We finetune the BERT base (Devlin et al., 2019), RoBERTa base (Liu et al., 2019), SentencesBERT (SBERT, multi-qa-distilbert-cos) (Reimers
1https://github.com/Siki-cloud/PairCFR.git
and Gurevych, 2019) and T5 base (Raffel et al., 2020) models with the original or CAD datasets on HuggingFace platform (Wolf et al., 2020). Volumes of model parameters are listed in Table 7 in Appendix C.3. Following the common practices of transformers (Devlin et al., 2019), we take the embedding of the “[CLS]” token as sentence representation and finetune the whole model. We set the maximum token length to 350 for SA and 64 for NLI.
We follow the original dataset splits described in (Kaushik et al., 2020), where the train, validation, and test sets are divided in a ratio of 7:1:2, with all classes balanced across each set. Subsequently, we finetune all models up to 20 epochs with the AdamW optimizer, coupled with a linear learning rate scheduler with a warmup ratio as 0.05. The best learning rate is manually tuned from {1e−4, 1e−5, 3e−5, 5e−5, 5e−6, 1e−6}. We apply the early stopping strategy with a patience of 5 and the best model is selected according to the lowest validation loss. To determine the trade-off factor λ and temperature τ , we conducted a grid search in the range [0, 1] with a step size of 0.1. We also conducted experiments to evaluate our PairCFR in few shot setting where the learning rate and batch size were tuned accordingly. The hyperparameters for full data finetuning and few shot setting are shown in Table 9, Table 8 respectively, in Appendix C.3.

heading: 4.3 Baselines
text: We compare our method PairCFR with the following baselines. For a fair comparison, we employ other forms of augmentation or increase the sampling number for the first three baselines without counterfactual augmentation, to ensure all approaches have the same number of training data.
Vanilla (Devlin et al., 2019). This method refers to a general model fine-tuning with original sentences. We include this baseline to verify the improvement of our method result from both the introduction of CAD and the novel learning framework.
BTSCL (Gunel et al., 2021). This approach employs the supervised contrastive loss (Khosla et al., 2020) into the model training where augmented positive samples are obtained through backtranslating a given sentence (Ng et al., 2019).
CouCL (Wang et al., 2022a). As counterexamples (CEs) are rare in a mini-batch, CouCL samples counterexamples from the original training set, where an example with lower confidence corresponds to a higher likelihood of being selected.
Subsequently, it adopts the self-supervised contrastive loss to push representations of positive CEs and negative CEs far apart.
The following approaches study how to train a robust model with annotated CAD:
HCAD (Kaushik et al., 2020). It collects two human-edited CAD datasets and fine-tunes a pretrained model on CAD with the cross-entropy loss.
CFGSL (Teney et al., 2020). As domain priors in CAD may be lost due to random shuffling in preprocessing (Kaushik et al., 2020), CFGSL pairs original data and its counterfactual example in the same batch and introduces a gradient supervision loss (GSL) alongside the cross-entropy loss. The GSL enforces the model gradient to align with the straight line from the original point to CFE.
ECF (Fan et al., 2024). It introduces two additional losses to mine the causal structures of CAD. The first loss extracts the dataset-level invariance through Invariant Risk Minimization (IRM) while the second loss is applied to pairs of original sentences and CFEs, preventing the model from relying on correlated features.

heading: 5 Results and Analysis

heading: 5.1 Overall Performance Comparison
text: Table 1 reports the overall performance comparisons, showing that our proposed PairCFR method outperforms all the baseline models on three out of four OOD datasets for both SA and NLI tasks across four different backbone models. To exclude the possibility of marginal improvements due to random initializations, we also conducted significance tests under the null hypothesis that there are no differences between each baseline and our approach, as presented in Table 10, located in Appendix C.6. The p-values less than 0.05 demonstrate that our methods are significantly better than the baselines, even though some improvements are relatively slight in Table 1.
In addition, we reported the following findings. Firstly, CAD-based methods may perform worse than non-CAD methods on OOD tasks, e.g., HCAD always lags behind CouCL on the SA task using fine-tuned T5 model. A similar phenomenon is also reported in (Joshi and He, 2022). These could be due to the failure to extract complementary features between CFEs and the original data; Secondly, the introduction of CFEs may shift the training data distribution from the in-domain data distribution. As anticipated, CAD-based methods fall behind
50 100 500 1k 4k 30
45
60
75
A cc
ur ac
y (%
)
SNLI
50 100 500 1k 4k 30
40
50
60 MNLI-m
50 100 500 1k 4k 30
40
50
60
MNLI-mm
50 100 500 1k 4k 33
36
39
42
Negation
50 100 500 1k 4k
36
42
48
54
Spelling-e
50 100 500 1k 4k
36
42
48
54
Word-o
non-CAD methods on ID datasets. Thirdly, our proposed PairCFR exhibits superior OOD performance compared to the baselines, achieving the highest accuracy on mostly OOD datasets, with the sole exceptions being the Yelp and Negation datasets. We postulate that the noted exceptions may be attributed to Yelp and Negative datasets having distributions similar to the ID datasets. The above results validate that PairCFR possesses a heightened capability to learn prior knowledge in CAD.

heading: 5.2 Few-shot Learning Performance
text: Data augmentation, such as counterfactual augmentation, is frequently utilized to enhance the performance of few-shot learning. In this part, we
investigate the effectiveness of our proposed PairCFR in few-shot learning scenarios. We conducted experiments using the finetuned BERTbase model on the SNLI dataset, gradually increasing the number of training samples from 50 to 4,000. Similarly, on the IMDB dataset, we increased the number of training samples from 32 to 1,024.
Experiment results on SNLI and IMDB under the few-shot setting are reported in Figure 2 and Figure 5 ( Appendix C.5). From both tables, we can conclude that our PairCFR generally demonstrates higher accuracy and lower standard deviation across OOD datasets, particularly in scenarios where training sample sizes are small. For instance, PairCFR significantly outperforms other methods by around 6% on Spelling-e when trained with only
100 counterfactually augmented samples.

heading: 5.3 Ablation Study
text: We conducted ablation experiments to verify the efficacy of two crucial strategies of our proposed method: (1) the pairing strategy: the integration of original data with their CFEs within the same batch, denoted PairCAD, versus ShuffCAD where randomly shuffle CFEs and originals. (2) the CL loss: the incorporation of CL and CE loss versus CE loss alone.
Results in Table 2, together with significance tests in Table 11 in Appendix C.6, offer several insights: 1) The strategy of pairing original data with their CFEs in the same batch improves OOD performance for both SA and NLI tasks. This can be attributed to the preservation of prior causal relations, which might be lost during random shuffling; 2) The efficacy of PairCAD with a CE-alone learning framework is not guaranteed. For example, within the T5 model framework, PairCAD underperforms ShuffCAD on the SNLI, MNLI, and Spelling-e datasets when only CE loss is adopted. This underscores the critical role of the CL component in augmenting features when we batch CFEs and original data; 3) Integrating the CL consistently improves model performance in both ID and OODs. Particularly, combining CL with PairCAD yields the best performance across various model assessments, highlighting the effectiveness of contrastive learning and the pairing strategy in leveraging causal relations of CFEs.

heading: 5.4 Impact of Batch Size
text: In this study, we investigated the effect of batch size on learning performance. We conducted experiments on the fine-tuned BERT model for SA and the fine-tuned T5 model for NLI, incrementally increasing the batch size while maintaining the original augmentation ratio for each task. From Figure 3, we observe that the model performance on both tasks initially improves with increasing batch size, but eventually reaches a plateau or experiences a slight decline.
We contend that the inclusion of negative samples in the CL function provides additional regularization, forcing the model to rely on a broader array of features beyond those edited. However, an excessively large batch size introduces an overwhelming number of negative samples in CL, which may dilute the human priors in CAD, leading to diminished performance. This trend is consistent across
both SA and NLI tasks, highlighting the effort required in batch size selection.

heading: 5.5 Contribution of Neutral Class in NLI
text: Do all counterfactual examples equivalently contribute to enhancing model generality? To answer this, we specifically experimented with the finetuned BERT model on the NLI task, comparing performance with and without the inclusion of neutral class samples in CL.
Results in Table 3 reveal that removing neutral samples, including neutral CFEs, significantly enhances the OOD generalization by approximately 2% when training the model on CAD with our learning framework. We attribute this performance difference to the distinct nature of neutral samples. In NLI tasks, judgments of entailment and contradiction are often readily determined based on the semantic alignment or disparity between text elements. Conversely, neutral samples represent scenarios where the hypothesis and premise lack any clear relationship, encompassing a vast array of potential expressions. This diversity poses a great challenge for models to identify universal patterns within the neutral class through human annotations. Therefore, adding neutral samples into the CL detrimentally affects the model’s performance in our experiments.
This investigation highlights the necessity of contemplating the practical value of adding additional counterfactual examples for specific classes.

heading: 5.6 Effect of Counterfactual Diversity
text: We also investigated the role of CFE diversity in improving model performance on the NLI task. In SNLI, each sentence is annotated with 4 CFEs, due to the existence of two opposite targets and modifications made to both the hypothesis and premise. Each CFE is obtained through a different type of modification, resulting in a dataset that includes
more diverse counterfactuals. We fine-tuned the T5base model by incrementally including more CFEs in a batch, ranging from 1 to 4.
The results in Table 4, reveal a direct relation between the number of CFEs and the model’s generalization capabilities. Notably, the OOD performance of the model trained on CAD is even better than that trained on a 3 times larger dataset with only original data. We conclude that enhancing counterfactual diversity proves to be an efficient strategy, which is the same as the findings reported in (Joshi and He, 2022).

heading: 6 Conclusion
text: Counterfactually Augmented Data (CAD) can enhance model robustness by explicitly identifying causal features. However, recent research found that CAD may fall behind non-CAD methods on generality. In this work, we introduce PairCFR to overcome this challenge. PairCFR pairs original and counterfactual data during training and includes both contrastive and cross-entropy losses for learning discriminative representations. We prove that contrastive loss aids models in capturing sufficient relationships not represented in CAD, thus improving generality. Extensive experiments demonstrate that our PairCFR achieves superior accuracy and robustness in various scenarios. Our findings emphasize the potential of carefully designed training paradigms in utilization of CAD.

heading: 7 Limitations
text: Our PairCFR has been demonstrated to effectively improve models’ OOD generalization with humanedited CAD datasets, which, despite its high quality, is quite limited in size. Future work will focus on utilizing LLMs such as ChatGPT or GPT-4 to generate a larger volume of CAD. Yet, LLMgenerated CAD may suffer from lower quality due to noisy and insufficient perturbations. It remains crucial and necessary to extend our PairCFR framework to accommodate such compromised CAD. Furthermore, PairCFR currently utilizes a simple form of contrastive loss, namely InfoNCE. In the future, we aim to investigate alternative contrastive loss variants and assess their potential to further enhance OOD generalization capabilities. Lastly, our experiments were conducted using relatively older and moderately sized LLMs, such as BERT and RoBERTa. We are also interested in exploring the potential improvements on larger LLMs by em-
ploying parameter-efficient finetuning methods.

heading: 8 Ethics Statement
text: This work focuses on reducing shortcut learning in models trained on CAD, thereby improving their robustness and generalization. Similar to other methods designed to mitigate learning from spurious correlations, our proposed PairCFR could help elicit trust in NLP models. It assists models in betterconsidering context (see Section 3 for details), preventing decision-making based on incomplete or biased information, such as solely on the edited words in CAD. Nonetheless, ensuring absolute fairness in model decisions in complex real-world contexts remains a formidable challenge solely from a model design standpoint. For instance, models could be compromised by low-quality or erroneous counterfactual data, leading to the learning of false relationships and resulting in erroneous or biased real-world decisions. Consequently, it is crucial for practitioners to consider the quality of counterfactual data alongside model design.

heading: Acknowledgements
text: This research is supported, in part, by the Joint NTU-WeBank Research Centre on Fintech, Nanyang Technological University, Singapore. This research is supported, in part, by the National Research Foundation, Prime Minister’s Office, Singapore under its NRF Investigatorship Programme (NRFI Award No. NRF-NRFI05-20190002). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. Xu Guo wants to thank the WallenbergNTU Presidential Postdoctoral Fellowship. Zhiwei Zeng thanks the support from the GopalakrishnanNTU Presidential Postdoctoral Fellowship. This research is also supported by the Shenzhen Science and Technology Foundation (General Program, JCYJ20210324093212034) and the 2022 Guangdong Province Undergraduate University Quality Engineering Project (Shenzhen University Academic Affairs [2022] No. 7). We also appreciate the support from Guangdong Province Key Laboratory of Popular High Performance Computers 2017B030314073, Guangdong Province Engineering Center of China-made High Performance Data Computing System.

heading: A The trap in the CE loss
text: Given a sample, x = [xr, xc]T , associated with the label y=1, and the corresponding counterfactual example, c = [cr, xc]T , with the flipped label, y= 0, by minimizing the cross entropy loss, we compel the model such that fW (x) approaches 1 and fW (x) is close to 0, respectively. This can be equivalently formulated by maximizing the prediction difference, i.e., max[fW (x)−fW (c)]. The sigmoid function, σ(x) = 1
1+e−x , is bounded and monotonically increasing, implying that (wrxr + wcxc) should be as large as possible while (wrcr +wcxc) should be as small as possible. Here, xr and cr are the features before and after editing. The sign of cr should be opposite to the sign of xr such that when fW (x) approaches 1, fW (c) can approach 0. For the first term, we observe that increasing |wr| can lead to an opposite change, i.e., larger wrxr and smaller wrcr. However, the second term, wcxc, is contained in both fW (x) and fW (c). Optimizing wc does not have the opposite effect.

heading: B Gradient analysis of CL
text: In this section, we introduce the details of the gradient of CL with respect to the weight W through the negative branches si,n. Before talking details, we rewrite the CL term for convenience,
LCL = − E xi∈Pi
[ log
esip/τ
esip/τ + ∑
xn∈Ni e sin/τ
] .
(7) The total derivative of LCL w.r.t the model
weights be calculated through the chain rule as
∂LCL ∂W = ∂LCL ∂sin × ∂sin ∂W + ∂LCL ∂sip × ∂sip ∂W , (8)
where the gradient coming from the branch sin is
∂LCL ∂W ∣∣∣∣ sin = ∂LCL ∂sin × ∂sin ∂W . (9)
For simplicity, we let sin = zTi zn and drop the denominator, ∥ zi ∥∥ zn ∥, which is eliminated in the product of partial derivatives. zi=WTxi and zj=W Txn, and then we have
∂sin ∂W = ∂(WTxi) T (WTxn) ∂W
= ∂(xTi W)(W Txn)
∂W
= xix T nW + xnx T i W = AinW. (10)
Here, Ain=xixTn + xnx T i . The CL term of Eq (7) for anchor xi can be further written as,
LCL(xi) = − E xp∈Pi
 log exp(sip/τ)
exp(sip/τ)+ ∑
xn∈Ni exp(sin/τ)
 
= E xp∈Pi
 log  exp(sip/τ)+ ∑
xn∈Ni
exp(sin/τ)
   
− E xp∈Pi (sip/τ). (11)
Here, only the first term is a function of si,n. Hence, we can compute the gradient of LCL w.r.t. the similarity for a negative sample, si,n, as follows.
∂L(xi) ∂sin = 1 τ E xp∈Pi   exp(sin/τ) exp(sip/τ +
∑ xn∈Ni exp(sin/τ)
 
= 1
τ Pin (written as Pin). (12)
Combining Eq (10) and Eq (12) gives the final gradient from a negatives sample,
∂L(xi) ∂W ∣∣∣∣ sin = ∂L(xi) ∂sin × ∂sin ∂W
= 1
τ PinAinW. (13)
Summing up gradients in Eq (13) from all negative samples, we can derive
∂L(xi) ∂W ∣∣∣∣ Ni = ∂L(xi) ∂sin × ∂sin ∂W ∣∣∣∣ Ni
= 1
τ
∑
xn∈Ni PinAinW. (14)
As the gradient contains pair-wise outer products between the anchor point and all its negative samples, it fully captures the overview of the feature space rather than focusing on a local perspective on edited words.

heading: C Experimental Details
text: C.1 Training Data We introduce more details of the CAD data used in model training in our experiments. We adopt two counterfactually augmented datasets from IMDb (Maas et al., 2011) and SNLI (Bowman et al., 2015) in (Kaushik et al., 2021). The counterfactually augmented IMDb dataset contains 2440 original sentences, with each sentence having a corresponding revised counterfactual example. In SNLI, annotators can revise both the hypothesis and the premise
for each of two opposite classes, and each sentence has 4 counterfactual examples. After another round of human filtering, the counterfactual augmented SNLI dataset consists of 9064 counterfactuals and 2266 original examples. During training, we split two CAD datasets into train, validation, test sets as shown in Table 5.
C.2 ID and OOD datasets
Here, we provide statistics of in-domain (ID) and out-of-domain (OOD) datasets used to evaluate the generalization of models in Table 6.
Since CADs in our experiments are manually revised on samples from IMDb (Maas et al., 2011) and SNLI (Bowman et al., 2015), we include their test datasets for ID evaluation. As for OOD evaluation, we evaluate our sentiment models on Amazon reviews (Ni et al., 2019), Topic-based Tweets sentiment data (Rosenthal et al., 2017), Yelp reviews (Zhang et al., 2015) and SST-2 movie reviews (Socher et al., 2013). On NLI task, we report on the genre-matched (MNLI-m) and genre-mismatched (MNLI-mm) test set of MNLI (Williams et al., 2018), which are more challenging than SNLI due to multiple genres. In addition, We additionally
employ the diagnostic datasets Negation, SpellingError, and Word-Overlap provided by Naik et al. (2018) to evaluate models’ reasoning abilities on lexical semantics and grammaticality.
C.3 Implementation details
In Table 7, we list the volume of model parameters used in our experiments. In our experiment, we tune hyperparameters of our PairCFR, including learning rate lr, batch size bts, trade-off factor λ, and temperature τ , based on the performance on validation set in full dataset finetuning and few shot setting separately. The best hyperparameters are reported in Table 8 and Table 9.
All experiments were conducted on an NVIDIA A100 GPU server equipped with Ubuntu 22.04, featuring 40 GB of GPU memory, 32-core CPUs at 1.5 GHz, and 256 GB of RAM. The test environment was configured with Python 3.8, CUDA 11.8, and Pytorch 2.0. The training time for each hyperparameter configuration is less than one hour.
C.4 Hyperparemeter analysis: λ and τ
In this study, we investigate the influence of tradeoff factor λ and temperature τ on model generalization. Specifically, we incrementally increase λ or τ from 0.1 to 0.9 by 0.1 and fix other best hyper-parameters searched from grid search. The experimental results on ID and OODs are reported in Figure 4. We observe that with λ or τ increasing from 0.1, the model performance initially increases and then declines. In SA, the model perform better for a larger λ and a lower temperature 0.3 (i.e., λ=0.7, τ =0.3), while in NLI, a larger temperature and smaller λ is favored (i.e., λ=0.4, τ=0.7). We hypothesize that in SA, the model may overly depend on perturbed words for predictions, as revision patterns are relatively smaller than in NLI. Therefore, we should incorporate a smaller temperature τ and a higher trade-off λ to introduce a higher regularization from contrastive learning in SA. More insights will be explored in future work.
C.5 Few-shot learning on SA
Here, we present the results of few-shot learning using the BERT model on the SA task, with the number of IMDb augmented data progressively increasing from 32 to 1024, as shown in Figure 5.
Similar to the trend observed in few-shot learning for the NLI task, discussed in Section 5.2, our approach demonstrates significant performance improvements even with limited data in the SA task.
C.6 Statistical significance test To ensure that the observed improvements are not due to randomness across multiple trials, we conducted statistical significance tests on comparative experiments and ablation studies. We first check that experimental results from random initialization on both ID and OOD datasets follow a Gaussian distribution, and thus employ a two-sided paired samples T-test. Our T-tests are conducted under the null hypothesis that there are no differences between the two groups of experiments.
Table 10 presents the significance test results of our method against all baselines for the comparative experiments (refer to Table 1). We observed that the majority of p-values fall below the conventional confidence level of 0.05, indicating that the improvements in OOD performance achieved by our algorithm over the baselines are statistically significant and not due to randomness. Similarly, Table 11 presents the significance test results of the ablation study (refer to Table 2), verifying the effectiveness of our pairing strategy and CL function.

heading: Future Work
text: More insights will be explored in future work. C. 5 Few-shot learning on SA
Here, we present the results of few-shot learning using the BERT model on the SA task, with the number of IMDb augmented data progressively increasing from 32 to 1024, as shown in Figure 5. Similar to the trend observed in few-shot learning for the NLI task, discussed in Section 5. 2, our approach demonstrates significant performance improvements even with limited data in the SA task. C. 6 Statistical significance test To ensure that the observed improvements are not due to randomness across multiple trials, we conducted statistical significance tests on comparative experiments and ablation studies. We first check that experimental results from random initialization on both ID and OOD datasets follow a Gaussian distribution, and thus employ a two-sided paired samples T-test. Our T-tests are conducted under the null hypothesis that there are no differences between the two groups of experiments. Table 10 presents the significance test results of our method against all baselines for the comparative experiments (refer to Table 1). We observed that the majority of p-values fall below the conventional confidence level of 0. 05, indicating that the improvements in OOD performance achieved by our algorithm over the baselines are statistically significant and not due to randomness. Similarly, Table 11 presents the significance test results of the ablation study (refer to Table 2), verifying the effectiveness of our pairing strategy and CL function