Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension
Authors: Ning Bian
Hongyu Lin
Xianpei Han
Ben He
Le Sun
Venue: EMNLP 2023 Findings (2023)
Keywords: Distant Supervision
Machine Reading Comprehension
Contrastive Learning

Abstract: Distant Supervision (DS) is a promising learning approach for MRC by leveraging easily-obtained question-answer pairs. Unfortunately, the heuristically annotated dataset will inevitably lead to mislabeled instances, resulting in answer bias and context noise problems. To learn debiased and denoised MRC models, this paper proposes the Contrastive Distant Supervision algorithm -- CDS, which can learn to distinguish confusing and noisy instances via confidence-aware contrastive learning. Specifically, to eliminate answer bias, CDS samples counterfactual negative instances, which ensures that MRC models must take both answer information and question-context interaction into consideration. To denoise distantly annotated contexts, CDS samples confusing negative instances to increase the margin between correct and mislabeled instances. We further propose a confidence-aware contrastive loss to model and leverage the uncertainty of all DS instances during learning. Experimental results show that CDS is effective and can even outperform supervised MRC models without manual annotations.

Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension
Ning Bian Hongyu Lin Xianpei Han Ben Hel Le Sun
UUniversity of Chinese Academy of Sciences, Beijing, China
2 Institute of Software, Chinese Academy of Sciences, Beijing, China
bianning21 @mails. ucas ac.
(hongyu, xianpei, sunle (unlee)@iccs..... ac cn benhe@ucas, ac. cn
Abstract
Q: When did Apple iPhone 14 come out?
A: 2022
Pl: Apple released iPhone 2022
(V)
Distant Supervision (DS) is promising learn- ing approach for MRC by leveraging easily- obtained question-answer pairs. Unfortunately, the heuristically annotated dataset will in- evitably lead to mislabeled instances, resulting in answer bias and context noise problems. To learn debiased and denoised MRC models, this paper proposes Contrastive Distant Super- vision algorithm CDS, which can learn to distinguish confusing and noisy instances via confidence-awand contrastive learning. Specif- ically, to eliminate answer bias, CDS samples counterfactual negative instances, which en- sures that MRC models take both answer information and question- into consideration. question-context context interaction denoise distantly anno- tated contexts, CDS samples confusing nega- tive instances to increase the margin between correct and mislabeled instances. We further propose confidence-aware contrastive loss to model and leverage the uncertainty of all DS instances during learning. Experimental results show that CDS is effective and can even outper- form supervised MRC models without manual annotations.
P2 Apple announced its 2022 financial results.
(x)
P3 The iOS 16 for iPhone came out in 2022
(x)
C1 Apple announced iPhond SE 2022
(Counterfactual)
F1 Apple announced its new headquarters.
(Confusing)
[a] DS Instances
Push
Pull
Pull
(F)
(b) Contrastive Distant Supervision
Figure I: (a) Many DS instances are mislaheled (P2
and P3): (b) During contrastive learning, the counter- factual instance C1 pushes the question towards correct instance PI by cutting the "When-2022" shortcut, and the confusing instance F1 pulls P2 and P3 away from P1 to due to the similar contexts.
these passages, and uses all matched answer spans to train MRC models (Chen et al., 2017).
Unfortunately, such a heuristically annotation process will inevitably result in numerous misla- beled instances (Min et al., 2019; Lin et al., 2018). As shown in Figure ((a), by simply matching the answer "2022" with retrieved passages, P2 and P3 are mislabeled instances because they do not ex- press the release-year relation of "iPhone 14" in the question. These mislabeled instances make learn- ing effective DS-MRC models non-trivial task due to the answer bias and context noise problems:
1 Introduction
Machine reading comprehension (MRC) is a fun- damental NLP task, which aims to answer ques- tions based on given documents (Hermann et al., 2015; Chen et al., 2016; Sco et al., 2017). Tra- ditional MRC models usually learned using manually-annotated <question, answer span> pairs (Rajpurkar al., 2016), which are labor-intensive and üime-oosuuming. To learn MRC models un- der low-resource settings, distant supervision (DS) is a promising approach that leverages the easily- obtained <question, answer> pairs to heuristically annotate training instances. Specifically, given <question, answer> pair, DS-MRC system re- trieves relevant passages, matches answers with
Answer Bias problem. By heuristically annotat- ing all answer spans correct labels. DS-MRC models tend to make decisions using only answer information, while ignoring the question-comtext interactions (Ji et al., 2022: Shao et al., 2021). For example, in Figure 1(a), the model may learn answer "When" questions depending on the When-Date shortcut and ignore all other infor- mation such as the interaction between "come
"Corresponding Authors

---

out" in the question and "released" in the con- lext. These spurious correlations cannot general- ize well to real-world situations.
tainty of training instances into consideration.
We conduct experiments on both close-domain and open-domain MRC benchmarks. Experimental results verify the effectiveness of CDS: On close- domain MRC, CDS significantly outperforms pre- vious DS-MRC methods and achieves competi- tive performances with supervised MRC methods. On open-domain MRC settings, CDS even outper- forms supervised MRC models by large margin. The main contributions of this paper are:
Context Noise Problem. Because many instances are noisy, it is difficult for DS-MRC models to identify regular answer patterns for questions. For example, in Figure (a)), P2 and P3 are misla- beled and the contexts of all three instances are different. This makes hard for the MRC model to learn the correct "come out" orte-"released" pattern from such noisy contexts. How to distinguish correct instances from mislabeled instances is critical for distant supervision.
We propose contrastive distant supervision ap- proach, which can effectively alleviate the answer bias and context noise problems in DS-MRC.
To learn debiased and denoised MRC models, this paper proposes the Contrastive Distant Super- vision algorithm CDS, which can learn distin- guish confusing and noisy instances via confidence- aware contrastive learning. The contrastive learn- ing framework of CDS is shown in Figure 1(b).
We design two sampling algorithms and confidence-aware contrastive loss for DS-MRC models learning: counterfactual instance sam- pling for answer bias climination, confusing instance sampling for context denoising, and confidence-aware contrastive loss for uncertainty in distant supervision.
To climinate answer bias, CDS samples coun- terfactual negative instances and lets DS-MRC models learn to distinguish them from positive in- stances. Concretely, a counterfactual negative in- stance is instance with a correct answer but wrong counterfactual context. For example, in Fig- ure (()). "Apple announced iPhone SE in 2022." is a counterfactual instance because its context does not express the release-year relation of "iPhone 14" in the question. By learning to distinguish positive and counterfactual instances, MRC models need to consider both answer information and question- context interaction, and therefore the answer bias be addressed.
Experimental results show that CDS is effective and can even outperform supervised MRC mod- els without manual annotations.
Relate Work
Distant supervision for MRC. Distant supervision has been used to train MRC models in low-resouree settings, and two main kinds of approaches have been proposed to address the mislabeling prob- lem: (1) filtering noisy labels, and (2) modeling answer spans as latent variables. The noise filtering approaches learn to score and rank DS instances based on answer span positions (Lee et al., 2019; Tay et al., 2018; Swayamdipta et al., 2018; Clark and Gardner, 2018; Lin et al., 2018; Joshi et al.. 2017; Chen et al., 2017), question-passage simi- larities (Hong et al., 2022; Qin et al.. 2021; Shao et al., 2021; Deng al., 2021) and model confi- dences (Chen et al., 2022: Zhu al., 2022). The latent variable-based approaches jointly train MRC models and identify correct answer spans using hard-EM algorithms (Zhao et al., 2021; Min et al., 2019; Cheng et al., 2020). In this paper, we address the mislabeling problem via contrastive learning, so that DS-MRC models can be debiased and de- noised by sampling counterfactual and confusing negative instances and designing confidence-aware contrastive loss.
To denoise distantly annotated contexts, CDS samples confusing negative instances, and in- creases the margin between correct and mislabeled instances by pulling mislabeled positive instances closer to negative instances and pushing correct positive instances away. Concretely, a confusing negative instance contains the wrong answer and a confusing context. For example, in Figure I(a). Fl is a confusing instance because its context is similar to the mislabeled instances P2 but it does not contain the answer "2022". By learning to dis- tinguish mislabeled instances, CDS can effectively address the context noise problem.
Finally, because there are no golden labels dur- ing distant supervision, we propose confidence- aware contrastive loss, which iteratively estimates the confidence of all instances during training, and optimizes DS-MRC models by taking the uncer-
Contrastive learning is a promising technique that learns informative representations by contrast- ing positive sample pairs against negative pairs

---

(Chen al., 2020). Some studies have leveraged contrastive learning on weak labels: Hong and Yang (2021) contrastively debiases noisy labels by regularizing feature space. Wang et al. (2022a)
describe how resolve these problems via con- trastive distant supervision.
Contrastive Distant Supervision for Machine Reading Comprehension
learns better-aligned representations of partial la- bels via contrastive learning. Chen et al. (2022) proposes an answer-aware contrastive learning ap- proach to address the translation noise in cross- language MRC. Wang et al. (2022b) propose to exploit span-level contrastive learning for few-shot MRC. Zhang et al. (2021) proposes to sample coun- terfactual instances and learn inter-modality rela- tionships by contrastive learning for visual com- monsense reasoning. Contrastive learning is also widely used for weakly supervised learning (Du et al., 2022; Yan et al., 2022; Gao et al., 2022: Xie et al., 2022; Li et al., 2022b; Zheng al., 2022;
In this section, we present our contrastive dis- tant supervision (CDS) algorithm, which can learn to distinguish confusing and noisy instances via confidence-awann contrastive learning. Specifically, CDS samples counterfactual negatíve instances to eliminate answer bias, and samples confusing neg- ative instances to denoise distantly annotated con- texts. Then, a confidence-awain contrastive loss is proposed to estimate and leverage the uncertainty of all instances during learning. In the following, we describe them in detail.
Wan et al., 2022: Li et al., 2022a; Yi et al., 2022:
4.1 Counterfactual Negative Instance Sampling
Yang al., 2022; Li al., 2021; Chen et al., 2021). this paper, we leverage contrastive learning to ad-
dress the bias and the noise problems in DS-MRC.
By simply annotating all matched answer spans as correct labels, DS-MRC models tend make decisions using only answer information, while ignoring the question-context interactions. To clim- inate this answer bias, CDS samples counterfactual negative instances and ensures that DS-MRC mod- els can distinguish them from positive instances. Concretely, a counterfactual instance is an instance with the correct answer but counterfactual con- text irrelevant to the question. Figure 1(a) shows a counterfactual instance C1 which does not express the correct release-year relation of "iPhone 14" but with matched answer span "2022".
3 Background
Base MRC Model. The MRC model used in this paper is
Pstart; Pend = M (q,p)
(1)
where and p are the question and the passage. respectively, and the model outputs two proba- bility distributions Pstart and Pend representing the probabilities of each token being the start and the end positions of the answer span. Our confidence-awand contrastive learning algorithm can be used in any MRC model, and this paper uses the BERT-based MRC model, the same as Devlin et al. (2019).
Specifically, we sample counterfactual instances from low-confidence DS instances via rank-based sampling algorithm. The basic assumption that if the MRC model predicts lower answer probability for an instance, the instance is less relevant to the question (Qi et al., 2021; Izacard and Grave, 2021). Based on this assumption, for a <question, answer> pair, we first predict the answer probability ((x) of each DS training instance using the current MRC model, where P(x) is the Riber--Sfftma probability of the answer span as the same Cao et al. (2020). Then, for each training instance x,, we sample in- stances that have significantly lower probabilities than æ as its counterfactual instances UCF,
Vanilla Distant Supervision for MRC. Given a <question, answer> pair, the DS-MRC method first retrieves K relevant passages from corpus. Following Karpukhin al. (2020), this paper uses the Wikipedia corpus and splits all texts into 100- token chunks. Then, we heuristically match the answer with retrieved passages, and all matching answer spans are used to form <question, passage, answer span> instances. The passages that do not contain the answer are labeled as negative passages. Finally, all DS instances are used to train MRC models as in Chen al. (2017).
As shown above, the main drawback of vanilla distant supervision for MRC is that many DS in- stances are mislabeled, resulting in answer bias and context noise problems. In the next section,
u = (s,(P(ss) <P(x) m)
(2)
where 8j represents a counterfactual instance. and m is the probability margin. We set m =

---

max, (P(x)) (P(=) for flexibility on different ((z) distributions.
After sampling ug we learn denoise con- texts by learning to distinguish confusing instances. Given mislabeled instance z, we minimize the distances between its context representation x" and the representations of confusing negative instances Pj. while maximizing the distances between con- text æf and the context representations of positive instances æ9 via denoise loss in Section 4.3.
After sampling U,F, the answer bias can be ad- dressed by learning to distinguish counterfactual instances contrastively. Firstly, given a training instance x: and an MRC model M, we obtain the current representations of the question, answer, and counterfactual instance correspondingly as æ9, x and 89. Then, we minimize the distance between the question x and the answer x", while maxi- mizing the distances between question x" and the counterfactual negative instances s via the debias loss in Section 4.3.
By sampling confusing negatives and pulling mislabeled instances closer to confusing negatives, the context noise problem can be effectively ad- dressed. For example, as shown in Figure 1(b), the confusing negative instances will pull the mis- labeled instances P2 and P3 closer to confusing negative F1, therefore the influence of these noisy instances will be reduced.
Because counterfactual instances are matched with the answer span but are negative, the counter- factual instances will ensure DS-MRC models to answer questions based on both answer informa- tion and question-context interaction. In this way, the answer bias can be resolved. For example, as shown in Figure 1(b), the counterfactual negative instances will cut the "When-Date" shortcut, and push the question representation towards the cor- rect instance Pl, meanwhile away from mislabeled P2 and P3.
4.3 Confidence-Aware Contrastive Learning
Traditional contrastive learning usually needs golden labels of positive and negative instances. Unfortunately, this not the case in distant super- vision, where all labels are uncertain. To address this label uncertainty problem, our contrastive dis- tant supervision algorithm designs confidence- aware contrastive loss, which iteratively estimates the confidence of all instances during training and optimizes DS-MRC models by taking the uncer- tainty of instances into consideration. In the fol- lowing, we introduce how to optimize DS-MRC models using the confidence-awarc contrastive loss and how to estimate the confidence scores during training.
4.2 Confusing Negative Instance Sampling
Because many DS instances are noisy, it is difficult to learn to identify regular answer patterns for ques- tions. To denoise contexts, CDS samples confusing negative instances which used pull misla- beled positive instances closer to the confusing negative samples. In this way, the margin between correct and mislabeled instances will increase. Con- cretely, a confusing negative instance should not contain the correct answer but should have simi- lar context to mislabeled instances. For example, Figure 1(a) shows sampled confusing instance F1, which is similar to the mislabeled instances P2 and P3 but does not contain the answer.
Confidence-awaal contrastive loss. consideration the label uncertainty in DS-MRC, the confidence-awand awware contrastive loss uses confidence to weight the importance all instances. The overall loss is:
To take into
LODS - Σ (a + Ledenossee + LCE) (4)
To sample confusing negative instances, we de- sign a retrieval based algorithm. Specifically, for training instance we retrieve its similar passages, filter out passages with answers, and use the top- K remaining passages as the confusing negative instances u
where Tedebias is the debias loss used to ensure the model can distinguish counterfactual negative in- stances, Lstenoisc is the denoise loss used to denoise contexts by distinguishing confusing negative in- stances, LCE is cross entsoyyy loss used to learn the answer span prediction ability, a and 3 are dyper-parametent Given a training instance we describe these losses in the following.
u To (15le())) (
sim(2, pj)
(3)
where Py represents a confusing instance, is the answer, and sim(2, pj) is the similarity function in passage retrieval (this paper uses BM25).
Debias Loss. The debias loss of is used ensure the MRC models can distinguish æ from its corresponding counterfactual instances:

---

MRC model. Specifically, employ iterative estimation strategy: we start with initial confidence scores and iteratively update them during training. Given an instance I for a question 9, we use its question-passag matching score to initialize its confidence score. The intuition here is that the matching scores measure the relevance between the question and the passages, providing reasonable start for confidence. Specifically, we use the BM25 score during retrieval and normalize the confidence scores of all instances of the same question q:
(x)
exp(x: 9 x=) Σ
-wglog exp(x2 x² )
exp(x9 83)
By EELE
(5)
where Wx [0, is the confidence score of the instance x. In this way, the loss will be dominated by highly confident correct instances, and the in- fluence of uncertain DS instances will be reduced. By minimizing this loss, the question representa- tion æ is pushed towards the representations of high-confidence instances x²" and away from the representations of counterfactual instances sg. Denoise Loss. The denoise loss of x is used to ensure the MRC models can identify mislabeled instances via the relations to confusing negative instances:
sim(4: 27,)
Way Σj=1
(8)
sim 25))
where w is the initial confidence score for the i-th instance of the <question, answer> pair, and sim g, xzy) is the BM25 matching function.
To update the confidence during training at each time step 1, we apply moving-average strategy and update wt as:
Ledeniio(2)) = --(1 wzz
Σ exp(p, x0)
log
ByEUς
w/, Aw + (1- A)2
(9)
Σ exp(p; x²) + Σ Wa / exp(xf x²)
where zf is the confidence evidence derived from the current MRC model, and X E (0,1) is con- stant for moving average. The moving-average strategy smoothly updates the estimated confidence and ensures training stability.
(6)
where is the total number of instances of the <question, answer> pair. This loss is dominated by low-confidence mislabeled instances using the - Wy term, and therefore it will not influence the high-confident instances. By minimizing this loss, the context representation x of low-confidence instances is pulled towards the confusing instance representations Pj and pushed away from the high- confidence instances æ selected by the Wzy term at the bottom.
To get model confidence score 2², we design three approaches including Soft Weighting, Hard Max, and Positive Average.
For Soft Weighting (SW), is computed as:
Pi Pstartt (1sturt) x Pend( (tend)
(10)
P Ej=P3
Cross-Entrooy Loss. To learn the answer span prediction ability for MRC, we design confidence- aware cross-entropy loss:
21
(11)
where P; is the predicted span probability for the i- th instance. This approach assigns soft weight for each instance according to its predicted probability by the MRC model.
LCE(#)= = -wr log Pstart (1start)+ log P Penn( (end((ond)
For Hard Max (HM), z is computed as:
where pstart and pend are the start and end positions the answer span of instance æ. This loss function is weighted by the confidence We for each instance, so that high-confidence instances will have larger effects on the learning of answer span prediction.
[1 if arg max SKSJ Pe 10 clsc
24
(12)
where Px is calculated through Equation 10. This approach assumes that only one of the instances for <question, answer> pair is correct, and picks the one that has the highest probability as the correct instance.
Confidence Estimation. To model the uncer- tainty DS instances, we estimate their confidence by deriving confidence evidence from the current

---

For Positive Average (PA), 24 is computed as:
training, DS-All, which uses all individual DS in- stances train the MRC model, and DS-Average, which assigns an averaged confidence vector all (13) instances for a <question, answer> pair. For open- domain DS-MRC, we compare with DistDR (Zhao et al., 2021) which iteratively improves dense pas- sage retriever through an EM approach, and two clate-of-the-ar art open-domain MRC methods, DPR (Karpukhin et al., 2020) and ANCE (Xiong et al., 2021).
if arg max,? Pota (c) = $ and arg max. Pend(c) clse
2
innd
2
2
(14)
2j=1 2j
This approach is inspired by Zhao al. (2021) that if the span predicted by the model matches the labeled answer span, the instance is marked as correct.
Training configurations. We initialize our model using the pre-trained BERT-large model (De- vlin al., 2019). Our model trained with learn- ing rate of 2e-5, and the batch size is 32. The model is trained for 3 iterations. We pick the top 20 re- trieved passages for training and evaluation. We take K = 5 for confusing instance sampling. We set = 1 and B = 1 for the contrastive loss. For confidence estimation, the moving-average factor À is 0.8.
Through iteratively refining the confidence of all instances and optimizing the confidence-awer aware contrastive loss, CDS can estimate and exploit the label uncertainty of DS instances. Therefore the uncertainty problem can be effectively addressed.
5 Experiments
5.1 Experiment setup
5.2 Close-Domain MRC Results
Datasets. We evaluate our method on two stan-
dard MRC datasets: NaturalQuestions and Trivi- aQA. NaturalQuestions (NQ) (Kwiatkowski et al., 2019) is an MRC benchmark with manually an- notated passages and answer spans. In this paper, we only use the answer strings for training. We the modified NQ dataset provided by Sen and Saffari (2020), which is subset of NQ and the MRC task is defined as finding the short answer the long answer. This offers a more standard context for MRC tasks, whereas the official NQ dataset involves reading long documents or multi- document retrieval, which does not align with our focus. Therefore, the modified NQ dataset provides suitable setting for evaluating our model's perfor- mance. TriviaQA (Joshi al., 2017) is DS-MRE dataset collected from trivia quiz websites without golden passages and answer spans!.
We first evaluate our CDS method under the close- domain MRC setting, where the passages are pro- vided in the dataset. We the exact match (EM) and F1 scores of the output answers in Rajpurkar et al. (2016) for evaluation. From the results shown in Table we find that:
Confidence-award contrastive loss is effective for distant supervision. In experiments, all three confidence estimating approaches CDS-SW, CDS- HM, and CDS-PA achieve good performances in Table and CDS-PA is slightly better than the other two approaches. These results verify that it is important to take label uncertainty into considera- tion for effective distant supervision.
5.3 Open-Doman MRC Results
This section evaluates our CDS method under the open-domain MRC setting. In the first experiment, We use retrieval results (BM25 and -Po-Posit)ve) from open-domain MRC as passages, so that the robustness of different MRC models under noisy retrieved passages can be assessed. Specifically, for BM25, we use the top-20 retrieved results as the passages. For OPR-Positive, contexts" provided by Karpukhin et al. (2020) the passages. In the second experiment, we apply an ensemble our three MRC models (CDS-SW,
We compare our method with state-of- the-art supervised and DS-MI DS-MRC methods. For su- pervised MRC, we use the original training datasets to train MRC models. For DS-MRC, we compare with two types of methods: noise filtering methods (Xie et al., 2020; Swayamdipta et al., 2018) and a latent briable-bbssed method (Min et al., 2019). We further compare with DS-Top (Chen et al., 2017), which matches overlaps between questions and contexts and picks top-scored answer spans for
Baselines.
use the "positive
HM, -PA) in an open ppen-domain MRC pipeline. We use DistDR (Zhao et al., 2021) as the retrieval sys-
'The licenses of these datasets are Apache License 2.0.

---

NQ
TriviaQA
Model
Close
BM25
Positiie
Close
BM25
Positive
Supervised Model
63.01/75.50
11.91/21.50 +
61.40/74.28
71.50/76.56
43.97/48.36 -/-
29.16/30.87
Swayamdipta et al. (2018)
+
$1.60/5..00
-
Hard-EM (Min et al., 2019)
+
+
66.90/
50,70/
-/-
DSp (Chen et al., 2017)
50.37/64.44
14.92/23.29
10/7222
70. 70.64/74.38
48 86/51.155
38.54/61.55
DS-All
48.71/63.30
14.21/23.26
69.14/77.99
78.37/80.69
49.94/52.97
63.36/65.22
DS-Average
49.54/63.79
13.61/23.10 +/
69.26/78.15 -|-
77.39/79.99 -/-
49.99/53.00
62.97/65.16 +|-
Xie et al. (2020)
+
54.40/60.20
CDS-SW
50.73/64.97
15,49/24.95
70.24/78.91
78.57/81.29
55.20/58.57
63. 63.80/66.8 04
CDS-HM
51.93/5.60
15.85/25,24
71.16/79.56
77.76/80.50
57.15/60.24
64.23/66.22 64.86/66.85
CDS-PA
52.16/65.71
16.40/25.49
70.65/78.81
78.52/81.18
57.96/61.03
Table I: Evaluation results (FM//II) on the development sets of NQ and TriviaQA under close-domain MRC setting (Close) and open-domain MRC setting with passages from BM25 and DPR-Positive (Karpukhin et al., 2020) retrieval results.
Model
Exact Match
supervision is model-free, which can be directly used in existing open-domain MRC systems and complements other promising techniques.
Supervised DPR
41.5
Supervised ANCE
Distantly Supervised DPR
37.9
Distantly Supervised ANCE
38.3
DistDR (Zhao et al., 2021)
40.5
5.4 Ablation Studies
DistDR CDS
40.9
We conduct ablation experiments in Table to fur- ther show the effectiveness of our method designs.
Table Evaluation results for open-domain DS-MRC pipeline on the test set of NQ.
Effect of counterfactual negative instance sam- pling. We study the effect of counterfactual nega- tive instance sampling by removing the debias loss Ldebios from the overall contrastive loss. As shown in Table 3, there are performance drops in all three settings compared with the full method, especially in the BM25 setting. These results show that the answer bias problem is critical for DS-MRC mod- els, and our counterfactual instance sampling is effective for addressing the answer bias.
tem because it is trained under the distant supervi- sion setting, which is consistent with ours. Table and Table show the results, and we can see that: (1) By designing debiasing and denoising strategies, our MRC models can achieve robust performances on noisy passages, and even out- perform supervised models. As shown in Table 1, our models significantly outperform the super- vised models in both BM25 and OPR--ooitive set- tings. The highest improvement is on TriviaQA with DPR-Positive: 35.70 points EM and 35.98 points F1 score. On TriviaQA, our CDS models achieve significant improvements compared with both noise filtering and latent variable-based DS- MRC methods. These results show that noise is critical problem for DS-MRC models, and CDS can effectively learn debiased and denoised DS-MRC models, therefore improving the robust- ness of MRC models under the noisy open-domain MRC setting.
Effect of confusing negative instance sampling. We study the effect of confusing negative instance sampling by removing the denoise loss Ideeooss from the overall contrastive loss. As shown in Ta- ble 3, this results in significant performance drops compared with our full method. This verifies that context noise a critical problem in DS-MRC, and sampling and contrasting with confusing negative instances is effective for denoising distantly anno- tated contexts.
(2) Contrastive distant supervision can be an on-the-fly plug-in for improving the open- domain DS-MRC system. As shown in Table
Effect of confidence-aware contrastive loss.
We replace our confidence-aware contrastive loss with traditional contrastive loss by simply using instances that contain answers positive samples and using passages without answers as negative samples. This results in severe performance drops. especially in the close-domain and BM25 settings. We believe this is because too many positive sam- ples are actually mislabeled in this case, which
2, using model the reader module in the DistDR (Zhao al., 2021) pipeline (DistDR + CDS) achieves higher performance than existing distantly supervised open-domain MRC systems and further minimizes the gap between distant and full supervision. This shows that contrastive distant

---

Model
Close
RM25
DPR-Positive
CDS-PA
78.52/81.18
57.96/61.03
64.86/66.85
debias contrastive loss
78.24/80.90
53.72/56.96
63.43/65.52
denoise contrastive loss
77.75/80.19 74.14/77.92
56.31/59.41
63.66/65.71
w/o confidence-ave aware contrastive learning
52.05/54.29
63.78/66.06
Table 3:
Ablation results (EM/FI) on the TriviaQA dataset.
Question: What the capital of Nicaragua in Spanish?
5.5 Case Study
Answer: Managua
To better demonstrate the effect of CDS, Table 4 shows an example of negative instance sampling and confidence estimation from the NQ dataset. The correct instance is the same as the manually annotated positive passage in the NQ dataset that describes the relation "Managua is the capital of Nicaragua". The mislabeled instance contains the answer "Managua", but instead of describing the city Managua, it about Lake Managua which is not related to the question.
Correct instance: Nicaragua is the largest country in the is the country's 's
Central American
capital and largest city...
(Confidence change: 0.099 - 0.287 0.352)
Mislabeled instance: Lake Managua is lake in
Nicaragua. The Spanish name is Lago de Managua... (Confidence change: 0.119 + 0.037 0.032)
Counterfactual instance: Managua consists of more than one-sixth of the overall population of Nicaragua... Confusing instance: The second largest lake in Nicaragua has an indigenous name Cocibolca (Sweet Sea)...
This case demonstrates the results of counter- factual instance sampling and confusing instance sampling. For the counterfactual instance, our method samples an instance that does not express the "capital-of" relation but contains the answer "Managua". For the confusing instance, our method retrieves passage based on the mislabeled in- stance which does not contain the answer. This case shows that our counterfactual and confusing negative sampling algorithms can effectively sam- ple high-quality and helpful negative instances for eliminating answer bias and denoising DS contexts. For confidence estimation, in the beginning, the mislabeled instance has higher confidence score than the correct instance. Along with the learning process, the confidence score of the mislabeled in- stance decreases 0.032, and the correct instance gains the highest confidence for this <question, an- swer> pair. This demonstrates that CDS can suc- cessfully distinguish correct instances during train- ing. By using these confidence scores as weights in the contrastive loss, our method can effectively leverage the uncertainty of DS-MRC instances.
Table 4: A case of confidence updating in NQ dataset with counterfactual and confusing instances.
8E.00
Close-PI
BMZ2-F1
PoPtisoottiss-11
80.00
75,00
20.00
62.00
60.00
00.00
Figure 2: Model performances with different moving- average factor \ on TriviaQA dataset with CDS-PA
should be close to the negative samples in the rep- resentation space. Viewing them as positives will mislead the model learning. This shows the impor- tance of estimating and leveraging label uncertainty in distant supervision, and our confidence-aware contrastive loss is effective for taking the uncer- tainty into consideration.
Conclusion
This paper proposes Contrastive Distant Supervi- sion CDS, which can debias and denoise DS- MRC models by learning to distinguish confus- ing and noisy instances via confidence-awaad trastive learning. Specifically, CDS samples coun- terfactual negative instances eliminate answer bias, and samples confusing negative instances to denoise noisy contexts. Then, confidence- aware contrastive loss is proposed to estimate and
Effects of hyper-parameter X. We analyse the sensitivity of moving- avverge factor X by using dif- ferent values of X. We test the model performance with À (0.0, 0.2, 0.4, 0.6, 0.8, 1.0) on TriviaQA dataset. From Figure 2, we can see that the model performances with Close, BM25 and DPR-Positive settings show similar trends on X, and our method achieves the best results at À = 0.8.

---

leverage the uncertainty of all DS instances dur- ing learning. Experimental results verify the ef- fectiveness of CDS: On close-domain MRC, CDS significantly outperforms previous DS-MRC meth- ods and achieves competitive performances with supervised MRC methods. On open open-domain MRC settings, CDS even outperforms supervised MRC models by a large margin. Furthermore, contrastive distant supervision is model-free and thus effec- tively complement other MRC techniques. We will release all codes, models and datasets on GitHub.
Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1870-1879. Vancouver, Canada. Association for Computational Linguisties.
Nuo Chen, Linjun Shou, Ming Gong, and Jian Pei. 2022. From good to best: Two-stage training for cross-lingual machine reading comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10501-10508.
Tao Chen, Haizhou Shi, Siliang Tang, Zhigang Chen, Fei Wu, and Yueting Zhuang. 2021. CIL: Contrastive instance learning framework for distantly supervised relation extraction. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume I: Long Papers), pages 6191-6200, Online, Association for Computational Linguistics.
Limitations
Current DS-MRC systems still need external passage retrieval component, and we leave the fully end-to-end MRC systems for future work, i.e., the retrieval, the reading, and the answering are jointly modeled and learned. Besides, we use BM25 for passage retrieval and do not use recent dense re- trieval models which may further increase the per- formance of our DS-MRC models.
Ting Chen, Simon Kornblihh, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A simple framework for contrastive learning of visual representations. In Pro- ceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Vir- tual Event, volume 119 of Proceedings of Machine Learning Research, pages 1597-1607. PMLR.
The introduction of additional counterfactual and confusing instances will increase the time of training", because DS-MRC models need learn to distinguish more contrastive and confusing in- stance pairs.
Hao Cheng, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2020. Probabilistic assumptions matter: Improved models for distantly level question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5657-5667, Online. Association for Computational Linguistics.
document-
Acknowledgements
We sincerely thank the reviewers for their insightful comments and valuable suggestions. This work is supported by the Natural Science Foundation of China NN..2212077 and 62106251). Hongyu Lin is sponsored by CCF-Baidu OpenFund.
Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehension. Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 845-855, Melbourne, Australia. Association for Computational Linguistics.
Xiang Deng, Yu Su, Alyssa Lees, You Wu, Cong Yu, and Huan Sun. 2021. ReasonBERT: Pre-trained to reason with distant supervision. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 6112-6127, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
References
Yu Cao, Meng Fang, Baosheng Yu, and Joey Tianyi Zhou. 2020. Unsupervised domain adaptation on reading comprehension. In Proceedings the AAAI Conference on Artificial Intelligence, volume 34, pages 7480-7487.
Danqi Chen, Jason Bolton, and Christopher D. Manning. 2016. thorough examination of the CNN/Daily Mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2358-2367, Berlin, Germany. Association for Computational Linguistics.
Jacob Devlin, Ming-Was Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre- tre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4181-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open- domain questions. In Proceedings of the 55th Annual
Ye Du, Zehua Fu, Qingjie Liu, and Yunhong Wang. 2022. Weakly supervised semantic segmentation by pixel-to-prototype contrast. In Proceedings of
2~221 hours for a training epach on single A100 GPU

---

the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1520-4329.
Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Wei Chang. Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452-466.
Junyu Gao, Mengyuan Chen, and Changsheng Xu. 2022. Fine-grained temporal contrastive learning for weakly-superviied temporal action localization. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Pattern Recognition, pages 19999- 20009.
Kenton Lee, Ming- Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Compula- tional Linguistics, pages 1086-6096, Florence. Italy, Association for Computational Linguisties.
Karl Moritz Hermann. Tomás Kociský, Edward Grefen- stette, Lasse Espeholt. Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28: Anmual Conference on Neu- ral Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 1693- 1701.
Jingjing Li, Tianyu Yang, Wei Ji, Jue Wang, and Li Cheng. 2022a. Exploring denoised cross- cross-video contrast for weakly-supervised temporal action local- ization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pages 19914-19924
Wu Hong. Zhuosheng Zhang, Jinyuan Wang, and Hai Zhao. 2022. Sentence-aware contrastive learning for open-domain passage retrieval. In Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics (Volume I: Long Papers), pages 1062-1074, Dublin, Ireland. Association for Computational Linguistics.
Shikun Li, Xiaobo Xia, Shiming Ge, and Tongliang Liu. 2022b. Selective-supervseed contrastive learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 316-325
Youngkyu Hong and Eunho Yang. 2021. Unbiased clas- sification through costrasttaet and balancee learning. In Advances in Neural Information Process- ing Systems, volume 34, pages 36449-26461 Curran Associates, Inc.
Shuang Li, Yilun Du, Antonio Torralba, Josef Sivic, and Bryan Russell, 2021. Weakly supervised human- object interaction detection video via contrastive spatiotemporal regions. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1845-1855.
Gautier Izacard and Edouard Grave. 2021. Distilling knowledge from reader to retriever for question an- swering. In 9th International Conference on Learn- ing Representations, ICLR 2021, Virtual Event, Aus- Iria, May 3-7, 2021. OpenReview.net.
Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. 2018. Denoising distantly supervised open-domain question answering. In Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 1736- 1745, Melbourne, Australia. Association for Compu- tational Linguistics.
Yunjie Ji, Liangyu Chen, Chenxiao Dou, Baochang Ma, and Xiangang Li. 2022. To answer or not to swer? improving machine reading comprehension model with span-based contrastive learning. In Find- ings of the Association for Computational Linguis- tics: NAACL 2022. pages 290--300, Seattle, United States. Association for Computational Linguistics.
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019, A discrete hard EM ap- proach for weakly supervised question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 2851- 2864, Hong Kong, China. Association for Computa- tional Linguistics.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume I: Long Papers), pages 1601-1611. Vancouver, Canada. Association for Computational Linguistics.
Peng Qi. Haejun Lee, Tg Sido, and Christopher Man- ning. 2021. Answering open-domain questions of varying reasoning steps from text. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 3599-3614, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online. Association for Computational Linguistics.
Haozhe Qin, Jiangang Zhu, and Beijun Shen. 2021. Weakly-supervised question answering with effective rank and weighted loss over candidates. Proceed- ings of the Web Conference 2027, pages 1959-1967.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti,

---

Pranav Rajpurkar. Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016, SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2383-2392, Austin, Texas. Association for Computational Linguisties.
Contrastive learning of class -aass-agnsticc activation map for weakly supervised object localization and seman- tic segmentation. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recog- nition, pages 989-998.
Yuqing Xie, Wei Yang, Luchen Tan, Kun Xiong. Nicholas Jing Yuan, Baoxing Huai, Ming Li, and Jimmy Lin. 2020. Distant supervision for multi-stage fine tine-tuning in retrieval- based question answering. In www '20: The Web Conference 2020, Taipei, Tai- wan, April 20-24, 2020, pages 2934-2940. ACM IW3C2.
Priyanka Sen and Amir Saffari. 2020. What do models learn from question answering datasets? In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2429-2438, Online. Association for Computational Linguistics.
Min Joon Sco, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In 5th Inter- national Conference Learning Representations. ICLR 2017, Toulon, France, April 24-26, 2017, Con- ference Track Proceedings. OpenReview.net.
Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Amold Overwijk. 2021. Approximate nearest neigh- bor negative contrastive learning for dense text trieval. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Zhihong Shao, Lifeng Shang. Qun Liu, and Minlie Huang. 2021. A mutual information maximization approach for the spurious solution problem in weakly supervised question answering. In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Vol- ume Long Papers), pages 4111-4124, Online. As- sociation for Computational Linguistics.
Jicxi Yan, Lei Luo, Chenghao Xu, Cheng Deng, and Heng Huang. 2022. Noise is also useful: Negative correlation-steers latent contrastive learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 31-40.
Fan Yang. Kai Wu, Shuyi Zhang, Guannan Jiang. Yong Liu, Feng Zheng, Wei Zhang, Chengjie Wang, and Long Zeng. 2022. Class-aware contrastive semi-supervised learning. In Proceedings of the IEFF/CVF Conference on Computer Vision and Put- tern Recognition, pages 14421-14430.
Swabha Swayamdipta, Ankur P. Parikh, and Tom Kwiatkowski. 2018. Multi-mention learning for read- ing comprehension with neural cascades. 6th In- lernational Conference on Learning Representations. ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018 Conference Track Proceedings. OpenRe- view.net.
Li Yi, Sheng Liu, Qi She, A Ian McLeod, and Boyu Wang. 2022. On learning contrastive representations for learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16682-16691,
Yi Tay, Anh Tuan Luu, Siu Cheung Hui, and Jian Su. 2018. Densely connected attention propagation for reading comprehension. In Advances in Neural Information Processing Systems 31: Annual Con- ference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 4911 4922.
Xi Zhang, Feifei Zhang, and Changsheng Xu. 2021. Multi-level counterfactual contrast for visual com- monsense reasoning. In Proceedings of the 29th ACM International Conference on Multimedia, pages 1793-1802.
Zhen Wan. Fei Cheng, Qianying Liu, Zhuoyuan Mao, Haiyue Song, and Sadao Kurohashi. 2022. Relation extraction with weighted contrastive pre-training on distant supervision. ArXiv preprint, 205/2200508770.
Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, and Hal Daumé III. 2021. Distantly-supervised dense retrieval enables open-domain question answering without evidence annotation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9612-9622, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Haobo Wang, Ruixuan Xiao, Yixuan Li, Lei Feng. Gang Niu, Gang Chen, and Junbo Zhao. 2022a. Pico: Con- trastive label disambiguation for partial lahel learning. ArXiv preprint, 08.98444..
Minghang Zheng, Yanjie Huang, Qingchao Chen, Yuxin Peng, and Yang Liu. 2022. Weakly supervised tem- poral sentence grounding with gaussian- based con- trastive proposal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- lern Recognition, pages 15565-15564.
Jianing Wang. Chengyu Wang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Jun Huang, and Ming Gao. 2022b. Keep: Knowledge enhanced contrastive prompting for few-shot extractive question answer- ing. ArXiv preprimi, abs/2205.03071.
Jinheng Xie, Jianfeng Xiang, Junliang Chen, Xianxu Hou, Xiaodong Zhao, and Linlin Shen. 2022. C2am:
Peide Zhu, Zhen Wang. Claudia Hauff, Jie Yang, and Avishek Anand. 2022. Answer quality aware ag-

---

gregation for extractive qa crowdsourcing. In Find- ings of the Association for Computational Linguisties: EMNLP 2022, pages 6147-6159.


Reviews:
[{"review_id": "VBfLgpfuOh", "reviewer": "Reviewer_6c51", "summary": "", "questions": "1. Why do you choose a modified NQ dataset (Line 452-456) which is less frequently used to test models instead of the official version which is used by the majority?\n2. The result of Supervised Model under DPR-Positive on TriviaQA in Table 1 seems strange compared to other results. Can you explain or analyze the reason of that extremely low result?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "(1)\n> Lack of explanation and analysis of the model's deep mechanism. Why the dynamic update of confidence score works and why the model can outperformance supervised models so significantly need further detailed explanation. Only description but no explanation makes it short of interpretability.\n\nResponse: \n\nThank you for highlighting the importance of model interpretability. In Section 5.5, we provide a case study that shows an example to illustrate how the dynamic update of confidence scores influences the model's learning process. As shown in the example, along with the learning process, the confidence score of the mislabeled instance is decreased, and the correct instance gains the highest confidence. This demonstrates that CDS can successfully distinguish correct instances by dynamically updating the confidence scores.\n\nAs to the significant outperformance of our models compared to supervised models when given retrieved passages, we believe this is because our models are trained within a distant supervision setting. Our models leverage retrieved passages which can learn more robust reading comprehension abilities, in contrast to supervised models which are trained solely on golden, high-quality passages. This further shows that our CDS method, after addressing the bias and noise issues of DS-MRC, can amplify the capabilities inherent in distant supervision. \n\nWe acknowledge the need for more explicit explanations in these aspects and will ensure they are included in our revised paper.\n\n---\n(2)\n> Inconsistant symbol usage. The w_x in Eq. 5-7 and the w_i^t in Eq. 9 have different formats, which might lead to confusion.\n\nResponse:\n\nThank you for pointing out the inconsistency in symbol usage. We apologize for any confusion this may cause. In our revised version, we will ensure consistent symbol formatting across equations to prevent any ambiguity.\n\n---\n(3)\n> An overview of the workflow and the model, which can make it easier to get the whole picture of the work, is needed.\n\nResponse:\n\nWe acknowledge the importance of providing an overview of the workflow and the model and have illustrated the basic concept of our contrastive learning in Figure 1. Furthermore, we will incorporate an overview of the workflow and the model structure to enhance the clarity of our paper.\n\n---\n(4)\n> Missing Ethics Statement (e.g., the reproductivity of the work).\n\nResponse:\n\nWe will include a comprehensive Ethics Statement in our paper, addressing concerns related to reproducibility and ethical considerations.\n\n---\n(5)\n> Typo on Line 496 (wrong serial number '(2)').\n\nResponse:\n\nThank you for identifying the typo. We will carefully review and correct this mistake in the revision.\n\n---\nQuestions:\n> 1. Why do you choose a modified NQ dataset (Line 452-456) which is less frequently used to test models instead of the official version which is used by the majority?\n\nAnswer: \nWe selected the modified NQ dataset provided by Sen and Saffari(2020) due to its alignment with the extractive MRC setup of our study. This dataset is a subset of NQ and the MRC task is defined as finding the short answer in the long answer. The modified NQ dataset offers a more standard context for MRC tasks, whereas the official NQ dataset involves reading long documents or multi-document retrieval, which does not align with our focus. Therefore, the modified NQ dataset provides a suitable setting for evaluating our model's performance. Using this modified NQ dataset is a common practice for extractive MRC experiments.\n\n> 2. The result of Supervised Model under DPR-Positive on TriviaQA in Table 1 seems strange compared to other results. Can you explain or analyze the reason of that extremely low result?\n\nAnswer: \nThe unusual result of the Supervised Model under DPR-Positive on TriviaQA in Table 1 can be attributed to differences in document sources between our study and the TriviaQA dataset. TriviaQA's document collection involves web search results and Wikipedia articles (please refer to the original TriviaQA paper for details), while we solely relied on retrieving Wikipedia articles for the MRC passages. This difference in passage sources can explain the discrepancy in results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Clearly describes the mechanism of debiasing and denoising.\n2. The description of the confidence-aware contrastive loss is clear and easy to understand.\n3. An unsupervised method that can beat supervised methods is what everyone likes to see.", "reasons_to_reject": "1. Lack of explanation and analysis of the model's deep mechanism. Why the dynamic update of confidence score works and why the model can outperformance supervised models so significantly need further detailed explanation. Only description but no explanation makes it short of interpretability.\n2. Inconsistant symbol usage. The w_x in Eq. 5-7 and the w_i^t in Eq. 9 have different formats, which might lead to confusion.\n3. An overview of the workflow and the model, which can make it easier to get the whole picture of the work, is needed.\n4. Missing Ethics Statement (e.g., the reproductivity of the work).\n5. Typo on Line 496 (wrong serial number '(2)').", "questions_to_authors": "1. Why do you choose a modified NQ dataset (Line 452-456) which is less frequently used to test models instead of the official version which is used by the majority?\n2. The result of Supervised Model under DPR-Positive on TriviaQA in Table 1 seems strange compared to other results. Can you explain or analyze the reason of that extremely low result?", "paper_topic_and_main_contributions": "This paper introduces a new method of distant supervision in MRC based on the idea of contrastive learning, which includes both debiasing and denoising and needs no manual annotations. This method outperforms existing DS-MRC methods and even (under some settings) beats supervised methods.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GRvfMQjYkd", "reviewer": "Reviewer_GuVT", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The term 'counterfactual' in this paper is confusing; I am not sure if this is an appropriate usage of the concept. In this paper, 'counterfactual instance' means the 'uncertain' instances charactarized by lower model confidences. However, as a logical concept, a conterfactual event is an event that is unactualized but possible to happen. I think there is a misalignment with this concept and the usage in this paper.\n\nResponse:\n\nIn our paper, we introduce the term “counterfactual instance” building on well-established concepts within the Visual Question Answering (VQA) domain. This term is particularly drawn from works like \"Counterfactual samples synthesizing for robust visual question answering\" , “Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing”, and \"Question conditioned counterfactual image generation for VQA\", etc. Constructing counterfactual images as training instances is a common practice in VQA to reduce context bias and spurious correlations.\n\nSpecifically, in our paper, a \"factual\" instance means the answer string from the passage is the accurate answer to the question, and a “counterfactual” instance refers to a scenario where, even if the answer is correct, it is presented in an irrelevant or incorrect context (the counterfactual context) given the question. Therefore, such answers should not be marked as the correct answer to the posed question. This term is consistent with the “counterfactual instance” in the VQA field. We will clarify this context in our revision to bridge any potential understanding gaps, ensuring that the meaning of “counterfactual instance” is clear to the readers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The addressed problems (bias and noise) are well-known issues that applies to various types of QA; the proposed method (Confidence-aware Contrastive Learning) seems to have a potential impact on these area. Also, the effectiveness of proposed method is solidly supported by experimental results.", "reasons_to_reject": "The term 'counterfactual' in this paper is confusing; I am not sure if this is an appropriate usage of the concept. In this paper, 'counterfactual instance' means the 'uncertain' instances charactarized by lower model confidences.\n\nHowever, as a logical concept, a conterfactual event is an event that is unactualized but possible to happen. I think there is a misalignment with this concept and the usage in this paper.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper tackles two problems of Distantly Supervised Machine Reading Comprehension(DS-MRC): Answer Bias Problem and Context Noise Problem. The authors introduce Confidence-aware Contrastive Learning to address these issues, which is reported to be effective.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QmpsxPN6KM", "reviewer": "Reviewer_dmb8", "summary": "", "questions": "Hi, thanks for this great submission, I have the following questions for you:\n\n1: Can you provide more analysis examples showing the benefit of the proposed method? on what kind of examples, CDS is better, and on what examples, CDS is worse etc.\n\n2: how sensitive is this method to hyper-parameters? in the paper you showed the performance w.r.t to different lambda, do you also have results for other parameters such as alpha, beta and K etc?\n\n3: In this paper, you mention it outperforms the supervised methods, do you outperform the STOA supervised methods?\n\n4: what's the future work in this direction?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Questions:\n\n> 1: Can you provide more analysis examples showing the benefit of the proposed method? on what kind of examples, CDS is better, and on what examples, CDS is worse etc.\n\nAnswer: \nIn Section 5.5 of our paper, we included a case study that highlights the benefits of the proposed method. This example demonstrates that the dynamic updating of confidence scores can benefit the model training process. We will provide additional analysis examples in the revised paper to better illustrate the benefits of our proposed method. We will include examples that showcase when CDS performs better and when it faces challenges compared to other approaches. \n\n---\n> 2: how sensitive is this method to hyper-parameters? in the paper you showed the performance w.r.t to different lambda, do you also have results for other parameters such as alpha, beta and K etc?\n\nAnswer:\nDue to space constraints of the paper, we were unable to include the results for other hyper-parameters such as alpha, beta, and K in the original paper. We recognize the importance of understanding the sensitivity of our method to these parameters. We have conducted thorough experiments exploring their effects and will ensure that these results are included in the final version of the paper. \n\n---\n> 3: In this paper, you mention it outperforms the supervised methods, do you outperform the STOA supervised methods?\n\nAnswer: \nOur primary focus is not a direct comparison with SOTA supervised methods. Comparing our method with supervised methods that use the same base pre-trained model is more fair. In our experiments, we used BERT as the base model for both the supervised and our proposed method. Furthermore, our proposed method achieves state-of-the-art results under distantly supervised reading comprehension tasks.\n\n---\n> 4: what's the future work in this direction?\n\nAnswer: \nIn the era of large language models, a crucial direction is leveraging external documents and knowledge to enhance LLMs' background information during question answering. This process, similar to our approach, also faces challenges related to bias and noise. Our contrastive learning approach might inspire methods for fine-tuning and aligning LLMs to better incorporate external knowledge and provide more factual answers. This area requires continued exploration. In the revised paper, we will expand the \"Future Work\" section to discuss potential research directions that stem from our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Novel ideas and looks like strong experimental results.", "reasons_to_reject": "Not sure whether this method is sensitive to hyper-parameter tuning or not.", "questions_to_authors": "Hi, thanks for this great submission, I have the following questions for you:\n\n1: Can you provide more analysis examples showing the benefit of the proposed method? on what kind of examples, CDS is better, and on what examples, CDS is worse etc.\n\n2: how sensitive is this method to hyper-parameters? in the paper you showed the performance w.r.t to different lambda, do you also have results for other parameters such as alpha, beta and K etc?\n\n3: In this paper, you mention it outperforms the supervised methods, do you outperform the STOA supervised methods?\n\n4: what's the future work in this direction?", "paper_topic_and_main_contributions": "This paper present a distant supervision based machine reading comprehension method by combing the contrastive learning and distant supervision method.\n\nSpecifically, in their method, they sample counterfactual negative instances and confusing negative instances, and use those sampled instances to do contrastive learning. Also, they modified the loss function, design some confidence estimation formulas.\n\nExperimental results on naturalQA and TrivialQA show that their proposed method is effective and can even outperform supervised machine reading comprehension without any manual annotations.\n\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a distant supervision method for training MRC models using a contrastive loss, with the primary contribution being a model confidence-aware sampling strategy for obtaining negative samples to make training more effective. The reviewers all agreed that the proposed method is sound, that the results are strong, and that the paper is well-written. They recommended that the authors include further analysis of the trained model's errors and sensitivity to hyperparameter choices.
Decision: Accept-Findings