Noisy Self-Training with Synthetic Queries for Dense Retrieval
Authors: Fan Jiang
Tom Drummond
Trevor Cohn
Venue: EMNLP 2023 Findings (2023)
Keywords: dense retrieval
self training
synthetic queries
TL;DR: A noisy self-training method to use synthetic queries to enhance dense retrievers on both general-domain and out-of-domain.

Abstract: Although existing neural retrieval models reveal promising results when training data is abundant and the performance keeps improving as training data increases, collecting high-quality annotated data is prohibitively costly. To this end, we introduce a novel noisy self-training framework combined with synthetic queries, showing that neural retrievers can be improved in a self-evolution manner with no reliance on any external models. Experimental results show that our method improves consistently over existing methods on both general-domain (e.g., MS-MARCO) and out-of-domain (i.e., BEIR) retrieval benchmarks. Extra analysis on low-resource settings reveals that our method is data efficient and outperforms competitive baselines, with as little as 30\% of labelled training data. Further extending the framework for reranker training demonstrates that the proposed method is general and yields additional gains on tasks of diverse domains.\footnote{Source code is available at \url{https://github.com/Fantabulous-J/Self-Training-DPR}}

Noisy Self-Training with Synthetic Queries for Dense Retrieval
Fan Jiang and Tom Drummond and Trevor Cohn
School of Computing and Information Systems The University of Melbourne, Victoria, Australia fan. aig 1@student. wwwmeln.mm....... edu. { {tom. drummond, trevor. cohn]@unimelb.. edu.
Abstract
2008) or BM25 (Robertson and Zaragoza, 2009), which can be applied without any training, neu- ral retrieval methods normally require training on a sufficient number of human-labelled queryy-aasage pairs to work well. Nevertheless, due to the high cost of human annotations, the number of available query- -assaaessee pairs is way smaller compared to the size of passage collections (500k at most (Bajaj et al., 2016) v.s. 21M passages (Kwiatkowski al., 2019)). Moreover, the situation becomes increas- ingly worse out-of-domain applications. where only a few or no training examples are available. Directly applying neural retrieval models trained on high-resource datasets typically achieves low out-of-domail performance and lags behind their lexical counterparts (Thakur et al., 2021).
Although existing neural retrieval models re- veal promising results when training data is abundant and the performance keeps improv- ing as training data increases, collecting high- quality annotated data is prohibitively costly, To this end, we introduce novel noisy self- training framework combined with synthetic queries, showing that neural retrievers can be improved in a self-evolution manner with reliance on any external models. Experimental results show that our method improves consis- tently over existing methods on both general- domain (e.g., MS-MARCO) and out-of-domain (i.e., BEIR retrieval benchmarks. Extra anal- ysis on low- low-resource settings reveals that our method data efficient and outperforms com- petitive baselines, with as little as 30% of la- belled training data. Further extending the framework for reranker training demonstrates that the proposed method is general and yields additional gains on tasks of diverse domains.'
In this work, we aim to improve the performance of state-of-the-art neural retrieval models both general domain and -nt-of-domain datasets, by ing automatically synthesised queries. For this pur- pose, we use query generator to automatically synthesise queries for each passage the target datasct, which we then use for pre-training, based on a self-training objective (Scudder, 1965). More specifically, instead of directly training the model on synthetic query-passage pairs (Ma et al., 2021), use a neural retrieval model that was trained on the labelled dataset the teacher generate soft labels for each synthetic query, providing supervi- sion signal which is more robust to noise in the gen- erated data. This ameliorates issues with the syn- thetic queries which are often generic or ambigu- ous, meaning that the query is only weakly related to its originating passage, and accordingly, other passages may match the synthetic query equally well or better. Table 1 shows motivating exam- ple, where the synthetic query spuriously-related to its originating passage with only the keyword achievement matched. However, the top-2 sampled "negatives" are highly related to the query in topics and semantics, according to both human judgment
1 Introduction
As important task, information retrieval (IR) refers to the task of finding relevant texts from large collection of passages or documents to satisfy the specific information needs users. The infor- mation need is usually expressed as a short textual query, and the task is formulated as retrieving texts that are most relevant to the given query.
Recently, impressive achievements have been made in neural retrieval models through adopting large-scale pre-trained models (Devlin et al., 2019). Dual-encoders typically serve as the backbone ar- chitecture, which enables retrieving relevant knowl- edge from collections with millions or billions of passages in a fraction of time (Karpukhin et al., 2020). Unlike traditional term-maten cerm-matching-bert based lex- ical retrievers, such as TF-IDF (Manning et al.,
"Now at Google DeepMind
'Source code is availahle at https://githhen com/ Sat.uuuuuusst/Seee---aiiinnnne

---

Synthetic Query: what was the major achievement of the manhattan project?
Originating Passage
Logits
Hard
...The only cloud hanging over the impressive achievement of the atomic researchers and engineers is what 0.14 their success truly meant; hundreds of thousands of innocent lives obliterated.
1
Sampled Negatives
...uccesss the 20th century was Manhattan Project. The Manhattan Project assimilated concepts and leaders from scientific fields and engineering disciplines construct the first two atomic bombs.
1.0
The Manhattan Project an epic, secret, wartime effort design and build the world's first nuclear weapon. the $20 billion project resulted in production the first uranium and plutonium bombs.
0.92
0
The Manhattan Project is an American tilm, released 1986. The plot revolves around gifted high school 0.4 student who decides construct atomic bomb for national science fair.
0
A motivating example for the issue of reak--elation between synthetic query and originating passages where top-2 sampled 'negatives" are way more relevant to the query than its originating passage. Negatives and associated Logits are from dense model fine-tuned on MS-MARCO and are normalised for clarity.
Table I:
and model predictions. By contrast, simply tak- ing the originating passage as positive and equally treating all negatives with hard labels regardless of their relevance to the query will completely mis- lead the model, leading to poor semantic matching patterns between query and passages.
202 I), and both neural retrievers and rerankers surpass a series of strong models ($4.6).
Preliminaries
2.1 Task Definition
We focus on the task of short-passage retricval (IR) in this work. Given a query in the form of short text, the task requires retrieving a small set of pas- sages that can satisfy the information needs, from collection of passages in million or billion scale. Formally, suppose have a passage collection P (p1.P2.-- pn.. the retriever is required fetch top-k passages P, {P1, PP,.P..- ... ,Pk} from P that are relevant to a specific query q.
Furthermore, to prevent the student from blindly imitating the teacher's behaviour, we pollute its inputs either in the query passage by injecting noise. This ensures that the student has differ- ent view on inputs compared to the teacher, en- couraging learn generalised signals from the teacher. Moreover, by polluting the inputs (e.g.. shuffling). the student is encouraged to capture salient phrases in addition to semantic matching, being more robust to possible perturbed inputs (Fig- ure 4). After completing pre-training on synthetic queries, further finetune the model on labelled data. Through iterating the pre-training and fine- tuning steps by using the latest model to relabel synthetic data, the resulting model can significantly outperform the one trained only using labelled data. summary, the contributions are as follows:
2.2 Dense Passage Retrieval
In contrast traditional IR methods, such as BM25 (Robertson and Zaragoza, 2009), which rep- resent texts high dimensional and sparse vec- tors with inverted index, dense retrieval methods alternatively adopt neural models to encode texts (queries or passages) in dense latent vectors with much smaller dimensions. A dense passage trieval model (Karpukhin et al., 2020) typically adopts the dual-encoder architecture, where neural models are used encode the query and passage into dense vectors separately. The relevance is mea- sured by the dot product between their embeddings:
1. We proposed novel self-training framework to make use of automatically generated data more wisely for neural retrieval models ($3).
2. Our experimental results on general-domain benchmarks show that the proposed frame- work can not only significantly boost the per- formance of state-of-ther of-thh--rtt-tt-- neural retrievers, but also yield superior results in low-resource settings. We further show our framework is general and can be extended to improve more rerankers ($4). 3. Further experiments are conducted on the out- of domain BEIR benchmark (Thakur et al.,
s(g. p; 0) E,(g: TT (p;
(1)
where E. is an encoder parameterised by 0. The adoption of this form of 'dual-encoder' chitecture decouples the encoding of query and passage. At inference, all passages in P can be en- coded offline. When a query comes in, efficient
powerful cross-encoder-bang

---

nearest neighbour search (Johnson et al., 2021) can be performed to fetch the top-k passages. Contrastive learning is applied to train the dual- encoder. Given a query 2. we have a positive pas- sage pt and a set of n negative passages Pg {P, N==-- The model is being optimised by min-
the learning process is conducted via KL diver- gence by closing the distribution distance between the student and teacher:
LSt -LL-KL P. S(), P.g.llddPPp....... 0s)) (3)
where PV, = which is the union of the pseudo positive passage of synthetic query o' and the sampled negatives as in $2.2. T(-ld', , 0r) and 5(-1g' , are the distributions from the teacher and student, respectively.
imising the negative log likelihood of the positive passage:
fa - -log 1(p P.0)
s(q.pt:0)
=llog E pUE s(g. p; 0)
(2)
We further inject noises into student's inputs, hoping that it can generate more robust embeddings. Through injecting noises, the student needs im- itate the behaviour of the teacher with perturbed inputs (i.e. different views), being encouraged learn generalised signals from the teacher (He et al., 2020). Inspired by Wu et al. (2019), we use follow- ing strategies for noise injection:
P. is the set of irrelevant passages constructed from in-batch negatives (Chen et al., 2020) (i.e.
positive passages of other queries in the same mini- batch) and mined hard negatives from existing re- trievers (Karpukhin al., 2020; Xiong et al., 2021).
3 Method
1. Shuffle: randomly choose some words in the query or passage candidates for shuffling, then randomly shuffle these candidates.
3.1 Self-Training with Synthetic Queries
Self-training (Yarowsky, 1995) has been shown ef- fective in improving model performance through using unlabelled or synthetic data. It first use la- belled data to train a good teacher, then the teacher is used to generate pseudo labels on unlabelled data. Finally, an identical student model is first pre-trained on unlabelled data with soft labels and then finetuned on labelled data. Recently, it has been shown to work well for variety of tasks, including image classification (Xie et al., 2020) and neural machine translation (He et al., 2020). However, its effectiveness has not yet been eval- uated for dense passage retrieval, especially with automatically synthesised queries.
2. Delete: randomly delete some words in the
query or passage.
3. Mask: randomly mask some words in the query or passage with a [MASK] token.
Empirically, we found that applying them sequen- tially to both queries and passages with a proba- bility 0.1 I achieves the best results. Thus, the self- training loss of noised version becomes:
LSt = KL(T(-1d Pig Or), P'r, 0s)) (4)
where 7 and Py are noised query and passages.
3.2 Training Pipeline
Suppose we have a well-trained query genera- tor, which is used to generate set of synthetic query-passane pairs T {(q': p'')), Moreover, we assume that teacher dense retrieval model that is fully trained on labelled data is available. The student retriever has the same architecture as the teacher, but only accesses the soft labels produced by the teacher during training. This ensures that all involved negatives are not treated equally but with different soft labels, and more relevant passages are more likely to be penalised less. Moreover, assume the use of teacher supervision will make the supervision signal for problematic queries (e.g.. general or ambiguous queries) much more diverse, as the teacher will have little clue and a correspond- ing high entropy predictive distribution. Formally,
Algorithm summarises the training pipeline, with an overview shown in Figure 1. More specifically:
Teacher Preparation (line 3 in Alg. 1) We first train teacher model in labelled data using two- stage training similar to (Gao and Callan, 2022). In the first stage, the retriever is trained with hard negatives sampled from a BM25 retriever. Then, the retriever trained in the first stage used to discover hard negatives, which are later used to train a second-stage retriever. The resulted retriever serves as the teacher 0, in our algorithm.
Index Building & Hard Negative Mining (lines 4-5 in Alg. 1) The teacher model 0y encodes all passages on P into dense vectors, which are later used to build the ANN search index with FAISS.

---

Query Gencrotion
Teacher Proparation
4.5
Hard hagelis Mining
Porudo iebiling
7:4
Student Trairing
Preseees
BM22 Negazhee
Derse Rettiever (BM2S)
Fand Negethoee
Teacher Rettisver
Teacher Retzieer
Teain
Fand Negalives
Quay Ganemaor
Sydlbbtic g
Synthatic w/ Fauddo Labelo
Stadknt Retteever
Retrievar
(Phetrain)
dg
Figure 1: The overview of noisy self-training algorithm. Line numbers in Algorithm are above each arrow type.
Algorithm Noisy Self-Training with Synthetic Queries
passage for each query; and Recall@k is the pro- portion of relevant passages that appear the top-k retrievals. Recall@k (k=5, 20, 100) is reported on Natural Questions, which the proportion of top-k retrieved passages that contain the answer string to each query. The evaluation script provided by Py- serini (Lin et al., 2021) is used for all experiments.
Require: Gold
pairs T ((g.2*))
Require:
Passage collection
Train a query generator G gold pairs T.
2: Generute queries from P and construct synthetic query-
passage pairs T = ,''''''' |
3: Train a dual-encoder retriever 0y on gold pairs T.
4: Use Or to build ANN index on PP
5: Retrieve negatives P. for each (g.p*) E T and Par for
each (4' +) E T'
4.2 Experimental Settings
6: Use By as the teacher to generate soft-labels on synthetic queries T' {(4', y ,P,)}
We replicate the coCondenser model (Gao and Callan, 2022) using PyTorch (Paszke et al., 2019) and treat it as our bascline.
Pre-train student retriever 05 soft-labelled synthetic queries T", with noises injected to 0g inputs
Finetune Os on gold pairs [(g.p" Pg )}
Take Ag as new teacher: fg + and go back to line
For self-training, the coCondenser model that was fully trained on labelled data serves as the teacher, which we use to train the student retriever on synthetic queries. After self-training, the student is further finetuned on labelled data, with hyperpa- rameters following Gao and Callan (2022). The last checkpoint is selected for evaluation on test set for all experiments. More details are in Appendix A.
Hard negatives for both labelled data ((q,p*) and synthetic data {/)} are retrieved using the built index, by taking k-nearest neighbours while excluding the gold passage.
Noisy Self Training & Fine-tuning (lines 6-8 in Alg. 1) A student model 05 is first pre-trained on the synthetic queries with the noised self-training objective (Eq. 4), after which it is fine-tuned on labelled data according to Eq. 2.
4.3 Main Results
Table shows the results of our model compared with a range lexical and neural retrievers. Wc ob- serve that the performance of our replicated coCon- denser is competitive with that of Gao and Callan (2022), achieving slightly better results Natural Questions but slightly worse on MS-MARCO, By applying proposed noisy self-training frame- work, the student coCondenser outperforms the state-of-the-a results on both datasets, resulting significant improvements over coCondenser (1.2% MRR 10 and R@5 on MS-MARCO and Natural Questions, respectively). This also shows that al- though coCondenser has already been pre-trained on the target corpus, continuing pre-traine pre-trinnin it on synthetic queries with noisy self-training can still lead to further performance boost.
Iterative Training (line 9 in Alg. 1) The teacher model can be replaced by the student to generate new pseudo labels and do noisy self- self-training and fine-tuning all over again.
4
General Domain Experiments
4.1
Datasets
We evaluate our method on two general- domain datasets: MS-MARCO passage rank- ing (Bajaj et al., 2016) and Natural Questions (NQ) (Kwiatkowski et al., 2019). MRR @ 10, Re- call @500 50. @ 1000 on the MS-MARCO dev set are reported, where MRR represents the Mean Reciprocal Rank, which is calculated as the sum the reciprocal rank of the first retrieved relevant
To test if the proposed method is general, we also combine with various pre-trained models, includ-

---

MS-MARCO
Natural Questions
Method
MRR @ 10
R@50
R@ Ik
R@5
R@ 20
R 100
RM25 (Robertson and 7aragora, 2009))
18.7
59.2
85.7
59.1
DeepCT (Dai and Callan, 2020)*
24.3
69.0
91.0
doc douury (Nogueira and Lin, 2019)*
27.7
75.6
94.7
GAR (Mao et al., 2021>'
60.9
74.4
85,3
DPR KKrrpuhhin et al., 2020)"
74.4
85,3
ANCE (Xiong ct al., 2021)"
33.0
95.9
81.9
87.5
ColBERT (Khattab and Zaharia, 2020)
36.0
82.9
96.8
DPR-PAQ (Oguz ct al., 2022)*
BERTR.
31.4
74,5
83.7
88.6
RoBFiRTagee
32.3
74,2
84.0
89.2
Condenser (Gao and Callan, 2021)
36,6
97.4
83.2
88.4
coCondenser (Gao and Callan, 2022)
38,2
98.4
75.8
84.3
89.0
Our coCondenser (Teacher) (Gao and Callan, 2022)!
37.8
85.5
98.2
75.9
85.0
89.4
Student coCondenser
39.4
86.7
98.5
77.0
85.5
89.5
Table Results MS-MARCO dev set and Natural Questions set. indicates results directly copied from Gao and Callan (2022) and Khattab and Zaharia (2020). f indicates our implementation. The best results are marked bold unavailable results are left blank.
MS-MARCO (Synthatic Gold Datal
MS-MARCO
NQ
Method
.0.00
MRR @ 10
R@
R@S
R@20
311.7
36.4
Student coCondenser
39.4
86.7
77.0
85.5
w/o noises
38.8
86.6
76.7
85.5
BERT
Contriever DOCS (Synthetic Datal
Conderser
RetroMAE
w/o pseudo labels
38.8
84.5
75.9
84.2
consistency filtering
38.8
86.2
76.5
85.3
joint training
38.4
86.1
76.8
85.3
N3112200
18
17.0
16.9
15.8
14.2
14.1
14.9
14.7
Table 4: Results of different variants compared to the student model on MS-MARCO and Natural Questions.
12.4
12
BERT
Condenser
Contriever
RetroMAE
We also adapt the proposed framework to see if it can be applied to improve expensive reranker. Similarly, a teacher reranker that was fully trained labelled data² is used to generate soft labels synthetic data, and an identical student reranker is pre-trained, cross-encoder model is used the backbone to jointly encode both query and passage. As shown in Table 3, we see that by reranking the student coCondenser's top-k predictions, the stu- dent reranker can outperform the teacher, boosting the performance with another 0.7% and 0.8% gains on MRR @ 10 and R@ respectively.
Figure 2: Results on MS-MARCO dev and SCIDOCS test sets when using different pre- greitrained models.
MS-MARCO
Natural Questions
Method
MRR
R@1
RØ5
R@20
Teacher Reranker
43.0
61.9
80.0
86.4
Student Reranker
43.7
62.7
80.3
86.8
Table 3: Compared results between the teacher and stu- dent reranker. The top-50 and top-100 predictions from student coCondenser are re-ranked on MS-MARCO and Natural Questions, respectively.
ing BERT (Devlin et al., 2019), Contriever (Izacard et al., 2021), Condenser (Gao and Callan, 2021). and RetroMAE (Xiao et al., 2022). Note that all models share the same architecture and only differ in their initial weights. Figure 2 shows the results MS- MS-MARCO. It is observed that the student model achieves consistent improvements all pre- trained models. Moreover, for worse teachers, the benefits gained by adopting the proposed method are generally more significant (c.g., 2.3 MRR I0 on BERT vs 1.3 MRR @ 10 on RetroMAE).
4.4 Ablation Studies
We conduct ablation studies to further understand our methods, with results reported in Table 4.
Noise Injection We remove noise injection dur- ing pre-training, and the pre-training objective changes from Eq. 4 to Eq. 3. It is observed that the noisy injection strategies show positive impacts in helping the model rank correct answers higher.
"The teacher retriever hard negatives in Figure are used as negative examples for training. See Figure for full details.

---

Pseudo Labels We remove line in Algorithm 1 and directly use synthetic labels {(q', p to- gether with hard negatives for pre-training in line 5. The results show that directly pre- pre-training on synthetic query-passage pairs leads to inferior per- formance on both datasets, resulting in significant performance degradation all metrics, especially on Natural Questions where the R@20 score is worse than the coCondenser baseline. This indicates that synthetic pussa-e pairs contain a large number of noises, and adopting our pro- posed method can effectively alleviate the negative impacts of noises, which validates our hypothesis.
M85.219
14
1098
30%
50%
70%
100%
Data Siec
Figure 3: Impacts of labelled training data size on MS-
MARCO dev set. Teacher refers to coCondenser.
28
Consistency Filtering We take the teacher consistency filter (Alberti et al., 2019) to remove noises contained in synthetic data. More specif- ically, for a given synthetic query-passage pair .'t),, if p'+ can be retrieved by the teacher in the top-1 position, this pair is kept; otherwise, will be discarded. Although this strategy can effec- tively improve the quality of synthetic data and also yields competitive performance, taking the teacher pseudo-label generator leads to better results.
vengaa
32
Stad::twi
10% 20%
30%
40%
50%
40%
60%
05%
100%
Figure 4: Impacts of randomly shuffling different pro- portions of tokens on MS-MARCO dev queries at test time. Teacher refers to coCondenser.
Joint Training We train the student model on both synthetic and labelled data, where the loss becomes L - Lst + LCL. Different batch sizes are also used for synthetic and labelled data to ensure the overall update steps are roughly the same in pre-train finetune. We observe that joint training leads significantly worse results MS-MARCO but comparable performance on Natural Questions.
labelled data samples decreases. Moreover, the stu- dent is superior in terms of data efficiency, achiev- ing performance comparable to the teacher model using full data samples (i.e., our coCondenser in Table 2) with little 30% of labelled data, and the performance continues to improve when more data is available.
Robustness to Shuffled Query Figure com- pares the performance when different proportions of tokens in each test query randomly shuffled. We observe that the student model is significantly more robust to queries with tokens in random or- ders. By removing noise injection during training, the performance drops become increasingly larger with more tokens shuffled, indicating that the noise injection strategy not only leads to performance gains in normal settings but also improves the ro- bustness towards perturbed inputs. This also signi- fies our model learss lexical matching in capturing salient phrases to some extent, a property typically held in sparse retrievers (e.g., BM25).
4.5 Analysis
Data Efficiency We further analyse how the size of labelled query-passage pairs used for training affects the retrieval performance of our method, Smaller datasets are randomly sampled from the full MS-MARCO training data with different sizes, ranging from % to 70%. Note that in each data size setting, all involved models are restricted to that specific amount of labelled data samples. Fig- ure 3 compares the performance of the teacher and student model trained on labelled data with dif- ferent sizes. The results reveal that the student surpasses the teacher under all data sizes, and the performance gap becomes larger as the number of
4.6 Out-of-Domain Experiments
'The teacher retriever, query generator and student re- triever are all trained on the same amount of labelled data in each setting. The size of synthetic data remains unchanged across all settings.
We further test the domain adaptation ability of our proposed method by reporting the performance on the BEIR benchmark (Thakur et al., 2021), 1), which

---

Model (-)
Lexical & Sparse
Dense Retriever
Retriever Reranker
BM25
DeepCT
doc
query "8
Teacher
Gen
GPL T0
GPL-S
PTR
Student
Retriever
PTR+48
Teacher
Reranker
Student
Reranked
Model Size
110M
110Mx18 66Mx18
110M
110Mx11
110M
110Mx11
110M
110M
QGen. Size
220M
220M
220M
220M
137B
220M
137B
220M
TREC-COVID
65.6
40.6
71.3
74.1
68.4
70.0
73.6
72.7
76.7
76.0
77.6
76.5
BioASQ
46.5
40.7
43.1
29.5
31.0
44.2
44.7
35.7
46.8
46.8
NFCorpus
32.5 28.3
32.8
33.1
34.6
34.5
34,9
33.4
34.9
36.0
31.0
32.6
NQ
32,9
18.8
39.9
48.4
44.4
48.3
49.0
50.3
58.5
59.1
HotpotQA
60.3
50.3
58.0
56.9
56.5
58.2
56.3
60.4
58.4
71.2
66.6
TiQA-2018
23.6
19.1
29.1
30.1
33.0
34.4
34.6
40.4
33.3
45.9
39.1
39.6
Signal-IM (RT)
33.0
26.9
30.7
27.8
27.0
27.6
27.0
29.5
26.5
28.1
TREC-NEWS
39.8
22.0
42.0
40.1
40.7
42.1
42.8
43.8
43.9
44.2
Robust04
28.7 40.8
43.7
43.6
40.6
43.7
46.9
46.8
46.2
48.6
ArguAna
31.5 30.9
34.9
37.3
46.3
55.7
53.4
53.8
50.1
52.1
55.1
57.8
Touché-2020
15.6 36.7
34.7
30.5
17.9
25.5
24.4
26.6
30.4
27.8
33.3
35.0
CQADupStack
29.9 26.8
32,5
32.6
35.5
35.7
36.2
35.8
35.0
35.5
Quora
78.9 69.1
80.2
85.6
86.1
83.6
83.6
86.8
80.0
82.5
DBPedia
331.3
17.1
33.1
37.5
36.3
38.4
38.1
36.4
37.2
41.3
41.5
43.3
SCIDOCS
15.88
12.4
16.2
14.3
15.2
16.9
16.7
16.3
16.1
19.1
15.6
16.4
FEVER
75.3
35.3
71.4
71.8
69.2
75.9
75.9
76,2
72.2
83.8
82.1
82.1
Clivate-FEVER
21.3
6.6
20.1
17.6
21.8
23.5
23.4
21.4
22.2
22.6
23.3
23.3
SciFact
63.0 66.5
67.5
61.0
66.2
67.4
67.6
62.3
65.1
73.2
70.8
72.1
Avg. Performance
PTR-11 Subsets
41.8
29.0
42.6
42.2
42.3
45.5
45.4
45.5
45.2
49.9
48.7
49.7
All
42.3
30.7
43.4
42.9
42.8
45.9
46.1
45.9
48.5
49.6
Table 5: Results on BEIR benchmark (nDCG@ 10). Best results are marked bold and second best results are underlined. indicates results copied from Thakur et al. (2021). f indicates our implementation. 3 means methods using synthetic queries. means methods learning from cross-encoder rerankers, thus not directly comparable to ours. Xn means that these methods train specialised models for each datasets.
contains 18 datasets from retrieval tasks of diverse formats and domains. The average nDCG@ 10 score over all datasets is used for evaluation.
employ the Teacher retriever for initialisation and train a single model, denoted as GPL-S.
We also investigate the proposed method can improve the reranker's out-of-domain perfor- mance. Similarly, the reranker trained on MS- MARCO serves as the teacher: Teacher-Rerankee guided by which an indentical Student Reranker trained. At inference, both rerankers are applied to rerank the top-100 predictions of the Student Retriever. The official evaluation script used for experiments." 6 More details are in Appendix A.2.
Implementation Details We use the coCon- denser model that was fine-tuned on MS-MARCO as the teacher model in our algorithm, which we call Teacher henceforth. For synthetic queries, we directly use the publicly available ones released by Wang et al. (2022).* Since labelled data is not available, the fine-tuning step is eliminated. Unlike previous methods that train specialised retrievers for each task, we train single universal retriever on the union of synthetic queries from tasks. We directly use the last checkpoint for evaluation on the test set of each task. To ensure fair comparison, we also re- re-implement GPL by following the same configurations as in Wang et al. (2022) except that
Main Results We compare our proposed model with variety of models, including BM25 (Robert- and Zaragoza, 2009), DocTSquery (Nogueira and Lin, 2019), DeepCT (Dai and Callan, 2020), GenQ (Thakur et al., 2021), GPL (Wang et al.,
'It achieves better performance compared to TAS-B used their original implementations. htttps: thub. com/beir-cell hta/bbirr-eellrrbeiii
'https: http::://pblicckkg informatik, tu- da-darsstad.. de/ htt/kkanggg///ggeeeeeeeee data/beir

---

2022) and PTR (Dai et al., 2023).? Table shows the experimental results. We first notice that the Teacher model already achieves strong results compared to other baseline models. GenQ which further finetunes Teacher on synthetic data does not show positive effects, indicating that synthetic queries are extremely noisy and directly using them for training is not beneficial. By contrast, the stu- dent retriever trained using our noisy self-training framework significantly improves over Teacher, increasing the averaged nDCG@ 10 by 3.0%. Even when being compared to GPL and GPL-S, the models that also use query generation for train- ing data augmentation but take a prohibitively ex- pensive reranker PTR that prompts a 137B instruction-tuned large language model for query generation, model can still achieve competitive averaged performance and beats these task-specialised retrievers on out of 18 tasks while being comparable on the rest. Note that our model does not rely on any exter- nal models, and is improved in a self-evolution manner, managing to exhibit a high degree of effi- ciency. More specifically, when being compared to GPL, our method significantly diminishes training time by roughly a factor of three (204h -> 74h). and it achieves 25x speed-up in relevance la- belling (20ms/q - 0.8ms/q). Figure shows that our method achieves consistent improvements and beats GenQ across different pre- pre-traine pre-trained models on SCIDOCS, a dataset that is significantly different from MS-MARCO in retrieval needs and domain. See Table 10 for full details.
pus in BEIR datasets). Meanwhile, the retriever trained on in-domain data (i.e. Teacher) gener- ates soft labels for these synthetic queries. As a result, the query generator fabricates distributions of potential queries that could be possibly asked in the target domain; while the teacher retriever cap- tures prevalent patterns of correspondence between queries and passages within the target domain, by leveraging the knowledge it has acquired from the labelled in in-domain data. When exposing to such silver target-domain data during training, new student retriever is enforced to acquire relevant knowledge that is required to complete retrieval tasks in target domains. Consequently, the expo- sure to pseudo target-domain data mandates the improvement of domain-specific aptitude within the student retriever, allowing it to effectively trieve information within target domains.
pseudo-label generator, and
4.7 Self-Training Iterations
Table compares the models trained with varying numbers of iterations. We observe that employing a single iteration yields optimal results, while perfor- mance diminishes with more iterations. We conjec- ture that errors produced in relevance labelling may accumulate over successive iterations, potentially reinforcing the model's bias towards particular types the training process continues.
4.8 Quality of Pseudo Labels
To assess the quality of the generated soft labels, randomly sampled 100 synthetic queries and verified the relatedness of the originating passage to each synthetic query through manual examina- tion. We observed that 49 of these queries were indeed related to their source passages. Among this subset of 49 queries, we noted that, 93% of the time, the soft labels effectively identified alter- native passages capable of accurately responding to the query. This indicated by the assign- ment of a high probability mass to these alternative passages. In contrast, for the remaining 51 query- passage pairs, where the association was found to be incorrect, soft labels can identify matched passages for 42 of them. This observation pro- vides strong evidence for the effectiveness of our proposed self-training methodology.
When adopting the same framework to rerankers, further boost the perfor- mance with another 1.1 average points over the teacher reranker. The results again confirm that our method is general and can be extended to improve a powerful reranker in out-of-domain settings.
the student rereanker
Discussion The adaptation of our proposed framework in out-of-domain tasks can be inter- preted as specific type of unsupervised domain adaptation (UDA) algorithm (Wang and Deng, 2018), where a model aims maximise its per- formance on target domains when only labelled data from in-domain sources and unlabelled data from target domains are available. In our case, the query generator trained on in-domain data (i.e., MS- MS-MARCO) is employed to generate synthetic queries on unlabelled target-domain data (i.e., cor-
Related Work
Neural Dense Retriever Neural retrievers adopt neural networks to encode texts into low-
"See Appendix B for more buseline details.

---

Iteration MM TC
BA
NF
NQ
HP
FQ
SG
TN
RB
AA
T2
CQ
QU
DB
FE
CF
Avg
37.3
74.1
29.5
33.1
48.4
56.9
30.1
27.8
40.1
43.6
37.3
30.5
32.6
85.6
37.5
14.3
71.8
17.6
61.0
42.9
39.4
76.7
35.7
34,9
50.3
58.4
33.3
29.5
43.8
46.8
50.1
30.4
35.8
86.8
37.2
16.1
72.2
22.2
65.1
45.9
39.1
75.7
36.9
34,9
49.2
57.5
33.7
28.7
42.8
46.4
50.6
29.2
36.5
86.2
35.8
16.5
70.9
22.2
66.7
45.6
Table 6: Ablations on the number self-training iterations. Tasks are ordered as in Table MM indicates MS-MARCO and is excluded when computing the average.
dimensional embeddings and show superiority over lexical retrievers when being trained on sufficient data. Karpukhin et al. (2020) adopts a dual-encoder structure to use two independent neural encoders to encode queries and passages into fix-sized vectors separately with their dot product the relevance score. The model is trained to discriminate pos- itive passages from randomly -sampee irrelevant ones or more informative negatives (Xiong al., 2021). Other works adopt the poly-encoder archi- tecture (Humeau et al., 2020), where query and documents are represented as multiple vectors to allow token-level interactions (Khattab and Zaharia, 2020). Although effective, the benefits come the cost of increased index size and more complex scoring function. In this work, we adopt the dual- encoder structure for its simplicity. However, our proposed method is orthogonal to model architec- tures, and we believe its combination with poly- encoder retrievers is worth further exploration.
be extended to boost reranking performance.
Seli-nnai Self-training (Scudder, 1965) refers to class of approaches that learn from unlabelled data with pseudo labels. A good teacher model is first trained on labelled data, which is later used to label unlabelled data. Another student model is then pre-trained unlabelled data first and further finetuned on labelled data. More advanced methods train multiple teachers using features of disjoint partitions on labelled data and student is learned from their ensembles (Blum and Mitchell, 1998). Recently, the effectiveness of self-training has been verified in a wide range of tasks, including machine translation with back translation (Wu et al., 2019) and language generation (He et al., 2020). These approaches termed noisy self-training, as the inputs of the student are perturbed. In this work, we follow this direction to show that, for the first time, noisy self-training can be adopted to make better use synthetic queries to improve neural retrievers both general-domain and out-of-domain settings.
Synthetic Queries for Information Retrieval Synthetic queries have been widely used in infor- mation retrieval. Early work expands passages with synthetic queries for lexical retrievers (Nogueira and Lin, 2019). Recent neural models take syn- thetic queries and their originating passages as positive pairs for model pre-training, resulting in boosted performance (Lu et al., 2021), bet- ter domain adaptation ability (Ma et al., 2021; Gangi Reddy et al., 2022) and promising zero-shot results (Dai al., 2023). In order to reduce the noise in synthetic data, Wang al. (2022) exploits a cross-encoder reranker to generate pseudo labels. It incurs expensive costs in generating soft labels and training rask-customised retrievers, resulting in slow adaptation. Our work follows this direction but significantly differs in that we do not rely on any external model for synthetic data labelling and cre- ate dataset-speciallised retrievers. By contrast, empirically show that by using the retriever itself more efficient pscudo-label generator, it can be improved in self-evolution manner with our introduced noisy self- trainini framework. More- over, we show this framework is general and can
Conclusion
In this paper, we present a novel noisy self-training framework for neural retrieval models. shows that when combined with automatically generated queries, neural retrievers can be improved in a self- evolution manner without relying on any external models. Empirical results on both genernl and out-of-domain benchmarks confirm the supe- riority of our proposed method, significantly out- performing wide range of competitive existing models. We further adapt our method to show it can be applied to improve an expensive reranker.
Limitations
Although proposed method does not change model architectures and the resulting models can perform as efficiently as previous models, the intro- duced training framework does incur the additional training cost, including query generation and the associated hard negative mining. Despite these ex-

---

tra costs, the training of our methods has fairly modest footprint by modern standards, taking about 3 days of a server with 4x A100 GPUs and 450G CPU RAM.
ity hosted at the University of Melbourne, which was established with the assistance of LIEF Grant LE1 701002000
We mainly experiment with dual-encoder- based neural dense retrieval models and addi- tionally adapt the method reranker training in this work. Alternative retriever methods such as ColBERT (Khattab and Zaharia, 2020) and SPLADE (Formal et al., 2021) are compatible with our approach, and their incorporation should lead to further gains. However, they introduce extra complexity compared to the dual-encoder, e.g., re- quiring an index over tokens rather than complete passages. We leave their efficient integration as future work.
References
Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, and Michael Collins. 2019. Synthetic QA corpora generation with roundtrip consistency. In Proceed- ings of the 57th Amnual Meeting of the Association for Computational Linguistics, pages 6168-6173, Flo- rence, Italy, Association for Computational Linguis- tics.
Payal Bajaj. Daniel Campos, Nick Craswell, Li Deng. Jianfeng Gao, Xiaodong Liu, Rangan Majumder. An- drew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2016. Ms marco: A human gener- ated machine reading comprehension dataset.
For out-of-domain experiments, the current method still relies on using general-domain datasets to obtain the teacher model and the query generator. How to achieve this in unsupervised setting needs further exploration. Morcover, on tasks that have different retrieval needs from gen- eral passage retrieval, synthetic queries are nor- mally quite different from the gold standard. For instance, some gold queries in DBPedia are gen- eral and may have multiple matching passages (e.g., Give me all people that were born in Vienna and died in Berlin.), but synthetic queries are usually only related to their originating passages (e.g.. Who was Johannes Mayer). How to generate synthetic queries with high quality and similar properties to gold standard queries is the key to better retrievers.
Amin Bigdeli, Negar Arabzadeh, Morteza Zihayat, and Ebrahim Bagheri. 2021. Exploring gender biases in information retrieval relevance judgement datasets. In Advances Information Retrieval: 43rd Euro- pean Conference on IR Research, ECIR 2021, Virtual Event, March 28 April 1, 2021, Proceedings, Part II, page 216 224, Berlin, Heidelberg. Peringer-Verlage
Avrim Blum and Tom Mitchell. 1998. Combining la- beled and unl unlabeled data with co-training. In Pro- ceedings of the Eleventh Annual Conference Com- putational Learning Theory, COLT 98. page 92-100, New York, NY, USA. Association for Computing Ma- chinery.
Alexander Bondarenko, Maik Fröbe, Meriem Be- loucif, Lukas Gienapp, Yamen Ajjour, Alexander Panchenko, Chris Biemann, Benno Stein, Henning Wachsmuth, Martin Potthast, and Matthias Hagen. 2020. Overview of touché 2020: Argument retrieval. In Experimental IR Meets Mulsilinguality, Multi- modality, and Interaction, pages 384-393 184, Cham. Springer International Publishing.
As this work uses pre-trained language models for generating synthetic queries, it is possible that undesirable biases (c.g.. gender and cultural) from the language models (Wei et al., 2022) is prop- agated downstream models. This is in addi- tional to existing biases in training and evaluation datasets (Bigdeli et al., 2021). Evaluating the ex- tent to which biases affect the synthetic data and the resulting model is an inherently complex problem, and remains an open question for future work.
Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. A full-text learning to rank dataset for medical information retrieval. In Ad- vances in Information Retrieval, pages 716-722, Cham. Springer International Publishing.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597-1607. PMLR.
Acknowledgements
We thank the anonymous reviewers for their help- ful feedback and suggestions. The first author is supported by the Graduate Research Scholar- ships funded by the University of Melbourne. This work was funded by the Australian Research Coun- cil. Discovery grant 0P230102273 This research was undertaken using the LIEF HPCGPGPU Facil-
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th Annual Meeting of the Association

---

for Computational Linguistics, pages 2270-2282,
SIGIR '17. page 1265-1268. New York, NY, USA. Association for Computing Machinery.
Online. Association for Computational Linguistics.
Zhuyun Dai and Jamie Callan. 2020. Context-aware
Junxian He, Jiatao Gu, Jiajun Shen, and Marc' Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations.
term weighting for first stage passage retrieval. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Infor-
mation Retrieval, SIGIR '20, page 1533-1536, New York, NY, USA. Association for Computing Machin- ery.
Doris Hoogeveen, Karin M. Verspoor, and Timothy Baldwin. 2015. Cqadupstack: A benchmark data set for community question-answering research. In Proceedings of the 20th Australasian Document Com- puting Symposium, ADCS '15, New York, NY, USA. Association for Computing Machinery.
Zhuyun Dai, Vincent Y Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith Hall, and Ming-Wei Wei Chang. 2023. Promptagator: Few- shot dense retrieval from 8 examples. In The Eleventh International Conference on Learning Representa- tions.
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux. and Jason Weston. 2020. Poly-encoders: Architec- tures and pre-training strategies for fast and accurate multi-sentence scoring. In International Conference on Learning Representations.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Se- bastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense infor- mation retrieval with contrastive learning.
Jeff Johnson, Matthijs Douze, and Herve Jegou. 2021. Billion-scale similarity search with gpus. IEFE Transactions Big Data, 733):535-547.
Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. Splade: Sparse lexical and expan- sion model for first stage ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '21, page 2288-229 2292, New York, NY, USA. Association for Computing Machinery.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu. Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781 Online. Association for Computational Linguistics.
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, and Avirup Sil. 2022. Towards robust neural retrieval with source domain synthetic freefiintuning. In Proceedings of the 29th International Conference Computational Linguistics. pages 1065-1070, Gyeongju, Republic of Korea. International Commiltee on Computational Linguistics.
Omar Khattab and Matei Zaharia. 2020. Colbert: Effii- cient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 39 49-48. New York, NY, USA. Association for Computing Machinery.
Luyu Gao and Jamie Callan, 2021. Condenser: pre- training architecture for dense retrieval. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 981-993, Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Daniclle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones. Matthew Kelcey, Ming Wei Chang. Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:652-466.
Luyu Gao and Jamie Callan. 2022. Unsupervised cor- pus aware language model pre-training for dense pas- sage retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1843-2853, Dublin, Ireland. Association for Computational Lin- guistics.
Markus Leippold and Thomas Diggelmann. 2020. Climate-fever: A dataset for verification of real- world climate claims. In NeurlPS 2020 Workshop on Tackling Climate Change with Machine Learning.
Faegheh Hasibi, Fedor Nikolaev, Chenyan Xiong, Krisz- tian Balog, Svein Erik Bratsberg. Alexander Kotov, and Jamie Callan. 2017. Dbpedia-entity v2: A test collection for entity search. In Proceedings of the 40th International ACM SIGIR Conference on Re- search and Development in Information Retrieval,
Jimmy Lin, Xueguang Ma. Sheng- Sheng-Chieh Lin, Jheng- Hong Yang. Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th Annual

---

International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2.21), pages 2356-2362.
Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and be- yond. Found. Trends Inf. Retr, 344:3333-39,
Jing Lu, Gustavo Hernandez Abrego, Ji Ma, Jianmo Ni, and Yinfei Yang. 2021. Multi-stage training with im- proved negative contrast for neural passage retrieval. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 6091-6103, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
H. Scudder. 1965. Probability of emor of some adaptive pattern-recogiitoon machines, IEEE Transactions on Information Theory, 1133):63-371.
Ian Soboroff, Shudong Huang, and Donna Harman. 2019. Trec 2019 news track overview. In TREC.
Axel Suarez, Dyaa Alhakour, David Corney, Miguel Martinez, and José Esquivel. 2018. A data collection for evaluating the retrieval of related tweets to news articles. In Advances in Information Retrieval, pages 780-786, Cham. Springer International Publishing.
Ji Ma, Ivan Korotkov, Yinfei Yang. Keith Hall, and Ryan McDonald. 2021. Zero-shot neural passage retrieval via domain targeted synthetic question generation. In Proceedings of the 16th Conference of the Euro- pean Chapter of the Association for Computational Linguistics: Main Volume, pages 1075-1088, Online. Association for Computational Linguisties.
Nandan Thakur, Nils Reimers, Andreas Rücklé, Ab- hishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Con- ference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. Www' 18 open challenge: Financial opinion mining and question answering. In Companion Proceedings of the The Web Conference 2018, WWW '18, page 1941-1942, Republic and Canton of Geneva, CHE. International World Wide Web Conferences Steering Committee.
James Thorne. Andreas Vlachos. Christos Christodoulopoulis, and Arpit Mittal. 2018.
FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Popers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Christopher D. Manning. Prabhakar Raghavan, and Hin- rich Schütze. 2008. Introduction to Information Re- trieval. Cambridge University Press, Cambridge, UK.
Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Ham, and Weizhu Chen. 2021. Generation-augne augmented retrieval for open- domain question answering. In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Vol- ume I: Long Papers), pages 4089-4100, Online. As- sociation for Computational Linguistics.
George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, loannis Partalas. Matthias Zschunke, Michael Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopous los, Yannis Almirantis. John Paviopoulos. Nico- las Baskiotis, Patrick Gallinari, Thierry Artieres, Axel Ngonga, Norman Heino, Eric Gaussier, Lil- iana Blrrio-Alvers, Michael Schroeder, Ion An- droutsopoulos. and Georgios Paliouras. 2015. An overview of the bioasq large-scale hiomedical se- mantic indexing and question answering competition. BMC Bioinformatics, 16:138.
Rodrigo Nogueira and Jimmy Lin. 2019. From
doc2query to doctttttquery.
Barlas Oguz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus. Xilun Chen, Sebastian Riedel, Scott Yih, Sonal Gupta, and Yashar Mehdad. 2022. Domain-matched pre-training tasks for dense retrieval. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1524-1534, Seattle, United States. Association for Computational Linguistics.
Ellen Voorhees. 2005. Overview of the trec 2004 robust retrieval track.
Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R. Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. 2021. Trec-covid: Constructing pandemic information retrieval test collection. SIGIR Forum, 54(1).
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf. Edward Yang. Zach Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Im- perative Style, -errorrrffoeeece Deep Learning Li- brary. Curran Associates Inc., Red Hook, NY, USA.
Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 241 1-51, Melbourne, Australia, Association for Computational Linguistics.

---

David Wadden, Shanchuan Lin, Kyle Lo. Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannanch Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLIP), pages 7534-7550, Online. As- sociation for Computational Linguistics.
189-196, USA. Association for Computational Lin- guistics.
Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2022. GPL: Generative pseudo labeling for unsupervised domain adaptation dense retrieval. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Compula- tional Linguistics: Human Language Technologies, pages 2345-2360, Seattle, United States. Association for Computational Linguistics.
Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputingg 312: 135-133.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe Le. 2022. Finetuned language mod- cls are zero-shot learners. In International Confer- ence Learning Representations.
Lijun Wu, Yiren Wang, Yingce Xia, Tao Qin, Jianhuang Lai, and Tie- Yan Liu. 2019. Exploiting monolin- gual data scale for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing
pages 4207- 4216, Hong Kong, China. Association for Computa- tional Linguisties.
Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. Retromae: Pre- Pre-training retrieval-oriented lan- guage models via masked auto-encoder.
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoe V. Le. 2020. Self-trining with noisy student improves imagenet classification. 2020 IEEE/CVF Conference Computer Vision and Pattern Recog- nition (CVPR).
Lee Xiong, Chenyan Xiong, Ye Li, Kwokk-Fung Tang. Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate nearest neigh- bor negative contrastive learning for dense text re- trieval. In International Conference Learning Representations.
Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages Brussels. Belgium. Association for Com- putational Linguistics.
David Yarowsky. 1995. Unsupervised word sense dis- ambiguation rivaling supervised methods. In Pro- ceedings of the 33rd Annual Meeting on Associa- tion for Computatinnal Linguistics, ACL '95, page

---

A Implementation Details
the teacher reranker are adopted. Completing the whole training pipeline requires approximately 80 hours on MS-MARCO and 67 hours on Natural Questions with up to 4 A100 GPUs.
Statistics about the datasets used in our experiments are reported in Table 7. Tables 8 and show the hyperparameters used in our experiments.
A.1 General Domain Datasets
A.2 Out-of-Domain Datasets
co-condenser-marcoo is used for model initiali- sation on MS-MARCO and co-condenser-wil on Natural Questions. For retrievers, the weights of dual encoders are tied. The teacher is trained using the two-stage training following the hyperparame- ters used in (Gao and Callan, 2022) and (Karpukhin et al., 2020). The student is pre-trainnn pre-trained for epoch for all datasets, with a learning rate 1 x 10-5 and batch size 32. The top-100 passages retrieved by the teacher are used as hard negatives, from which 7 negatives are sampled for each query in a mini- batch. We empirically find conducting the training with one iteration is enough for achieving the best result. The training takes about 48 hours on both datasets, with up 4 A100 GPUs.
BEIR benchmark, the Teacher retriever is used to mine hard negatives for synthetic queries on each dataset. The top-100 passages retrieved by the Teacher retriever are regarded as the hard neg- ative pool. Student Retriever is initialised from the co-condenser--maro checkpoint. On each dataset, we train Student Retriever for 10 epochs with learning rate 10-5, The batch size is set to 32, and each query is paired with 7 sam- pled negatives. Training a single model on each dataset takes about hours using 2 A100 GPUs and two iterations are used to achieve the best average performance. For GenQ, we follow Thakur et al. (2021) to further finetune the Teacher retriever on synthetic queries of each dataset for epoch with batch size 64 and only in-batch negatives.
For generating synthetic queries, we use the publicly available model to generate queries on MS-MARCO with top-k sampling. For Natural Questions, we first finetune a T5 model on labelled query-passage pairs for 200 epochs with learning rate 2 x 10-5, which takes about 10 hours on 1 A100 GPU. One query per passage is generated for all datasets, which requires approximately 7 hours on MS- AS-MARCO and 16 hours on Natural Questions using 2 A100 GPUs.
The reranker is initialised from ERNIB-base and finetuned for 5 epochs with learning rate 10-5 and batch size 12. Each query is paired with 15 negatives sampled from the same hard negative pool as above. Training a single model takes about 18 hours using A100 GPUs.
B Baselines on BEIR
As for the reranker described $3, ERNIE- base is used for model initialisation. The teacher reranker is trained with the teacher retriever hard negatives (Figure 1). It is trained for 2 epochs on MS-MARCO, with learning rate set to 1 x 10 batch size 12, and weight decay 0.1. Each query is paired with 40 sampled negatives; while for Nat- ural Questions, we train for 10 epochs with 15 sampled negatives. For pre-training the student rereanker, the batch size is set to 48 and 128 on MS AASMARCOO and Natural Questions, respectively, and other hyperparameters remain the same. For the finetuning stage, the settings used for training
BM25 (Robertson and Zaragoza, 2009) is a lexical retriever based on token matching. DocTsquery (Nogueira and Lin, 2019) uses a query generator to synthesise queries and append them to passages as expansion. DeepCT (Dai and Callan, 2020) uses BERT model trained MS- MARCO compute the weight of each term in the vocabulary and each passage is represented with keywords multiplied by the term weights. GenQ (Thakur et al., 2021) first trains a dense retrieval model on MS-MARCO and continues to finetune it on synthetic queries with in-batch neg- atives through contrastive learning. GPL (Wang et al., 2022) uses knowledge distillation to train a dense retriever by learning from reranker trained on MS-MARCO. PTR (Dai al., 2023) generates large number of synthetic queries by prompt- ing an instruction-tun large language model and trains taec-specd retrievers.
"https://hugging co////uuggng/aaecccc/uuu// co- marde marco
"nttps://huggiingface wtw.///gg...c../ou/////ooo
co- co-condenser-wiki
10 "httpss//huuggingffee.cc///
/https://huginngface
8 b-base-en

---

Task
Domain
Dataset
Train
dev
test
#aassgee
Passage Retrieval
Misc.
MS-MARCO
502,939 58,880
6,980
8,841.823
Open Domain QA
Wikipedia
Natural Questions
8,757
3,610
21,015.324
BEIR Benchmark
Biomedical
Biomedical
TREC-COVID
171,332
Information
Biomedical
BioASQ
14,914,002
Retricval
Biomedical
NFCorpus
323
3,633
Wikipedia
NQ
3,452
2,681,468
Question Answering
Wikipedia
HotpotQA
7,405
5,233,329
Financial
FiQA-2018
648
57,638
Twitter Retrieval
Twitter
Signal-IM (RT)
97
1,866,316
News
TREC-NEWS
57
594,977
News Retrieval
News
Robust04
249
528,155
Misc.
ArguAna
1,406
8,674
Argument Retrieval
Misc.
Touché-2020
49
382,545
Duplicate Question
StackEchange
CQADupStack
13,145
457,199
Retricval
Quora
Quora
10,000
522,931
Entity Retrieval
Wikipedia
DBPedia
400
4,635,922
Citation Prediction
Scientific
SCIDOCS
1,000
25,657
Wikipedia
FEVER
6,666
5,416.568
Fact Checking
Wikipedia
Hiimat--EEERR
1,535
5,416,593
Scientific
SciFact
300
5,183
Table 7: Number of query examples in Train/Dev/Test sets and passage number in each target corpus. means unavailable data that was not used in our experiments, Datasets in BEIR benchmark are grouped according to Thakur et al. (2021), please refer their paper for full details. TREC-COVID (Voorhees et al., 2021), BioASQ (Tsatsaronis et al., 2015), NFCorpus (Boteva et al., 2016), NQ (Kwiatkowski et al., 2019), HotpotQA (Yang et al., 2018), FiQA- 2018 (Maia et al., 8), Signal- IM (Suarez et al., 2018), TREC-NEWS (Soboroff et al., 2019), Robust04 (Voorhees, 2005), ArguAna (Wachsmuth al., 2018), Touché (Bondarenko al., 2020), CQADupStack (Hoogeveen al., 2015), DBPedia (Hasibi et al., 2017), SCIDOCS (Cohan et al., 2020), FEVER (Thorne et al., 2018), Climate- FEVER (Leippold and Diggelmann, 2020), SciFact (Wadden et al., 2020).
C Adopting Different Pre-trained Models on BEIR
D License
MS-MARCO licensed under the Creative
Commons Attribution 4.0 International.
We test our method on the BEIR benchmark with different pre-trained models. As Table 10 shows, the student model trained using our method achieves consistent improvements all kinds of pre-trained models, confirming the proposed algo- rithm general enough to be adapted diverse models on out-oi-domain settings. Moreover, the patterns for improvements are similar to those in MS-MARCO, with more obvious gains on less- performing teachers (e.g., +4.0% on Contriever vs +2.6% on RetroMAE). GenQ shows positive effects on pessoperfonnng pre-trained models (i.e., BERT, Condenser, and Contriever). However, it struggles with beating the teacher retriever more advanced pre-trained models and even degrades average per- formance. Besides, our method can consistently surpass GenQ across all settings, again confirming its superiority.
Natural Questions licensed under the Apache License 2.0.
3. The BEIR benchmark licensed under the Apache License 2.0.

---

Hyperparameters
MS-MARCO
Natural Questions
BEIR
Max query length
32
32
32
Max Passage length
128
128
128
Shared
Optimizer
Adam Linear warmup
Adam Linear warmup
Adam W Linear Warmup
Scheduler
Warmup proportion
0.1
0.1
0.1
Weight Tying
Yes
Yes
Yes
#Params
110M
110M
110M
Teacher Preparation
Learning rate Batch size
5 x 10-6
x 10
128
Teacher Stage 1
#hard negatives
Hard negative source #Epochs
BM25
BM25
40
Learning rate
5 x 10
x 10 128
Batch size
Teacher Stage 2
#hard negatives
Hard negative source #Epochs
stage teacher
stage teacher 40
Self Training
Leamning rate
10-5
x 10-5
x 10 32
Batch size
Pre-train
#hard negatives
7 stage teacher
7
Hard negative source #Epochs
stage teacher
stage teacher 10
Leamingg rate
5 10-6
x 10 128
Batch size
Finetune
#hard negatives
Hard negative source "Epochs
stage teacher
stage 2 teacher
40
Table 8: Hyperparameter settings for retriever training.
Hyperparameters
MS-MARCO
Natural Questions
BEIR 351
Max length
156 AdamW Linear warmup
156 AdamW Linear warmup 0.1
Optimizer
AdamW Linear Warmup
Shared
Scheduler
Warmup proportion
0.1
0.1
Weight decay
0.1
0.1
0.1
#Params
110M
110M
110M
Learning rate
x 10
x 10-3
Batch size
12
12
Teacher Reranker
#hard negatives
40
15
Hard negative source #Epochs
stage 2 teacher retriever
stage 2 teacher retriever
2
10
Self Training
Learning rate
x 10-5
x 10-5 128
1 x 10 32
Batch size #hard negatives
48
Pre-train
40
15
7
Hard negative source #Eppcch
stage 2 teacher retriever
stage 2 teacher retriever
stage 2 teacher retriever
5
Learning rate
x 10-5
x 10-5
Batch size
12
12
Finetune
#hard negatives
40
15
Hard negative source stage 2 teacher retriever
stage 2 teacher retriever
#Fipochs
2
10
Table 9: Hyperparameter settings for reranker training.

---

Model (-)
BERT
Condenser
Contriever
coCondenser
RetroMAE
Tea-
Stu- dent
Tea-
Stu- cher
Stu- cher
Tea-
Tea-
Stu- dent
Tea-
Stu-
Dataset
GenQ
GenQ
GenQ
GenQ
GenQ
cher
dent
dent
dent
TREC-COVID
66.3
60.4
67.9
64.2
65.3
67.6
58.0
46.2
62.1
68.4
74.1
76.7
70.0
70.0
74.1
BioASQ
33.3
25.7
30.1
29.7
24.6
32.0
34,9
30.6
35.2
31.0
29.5
35.7
40.6
39.0
43.0
NFCorpus
28.8
26.0
30.2
31.3
26.0
30.9
33.8
32.5
34.4
34.6
33.1
34.9
34.8
34.3
35.4
NQ
40.8
45.1
45.9
43.2
44.6
46.9
38.6
47.1
48.4
44.4
48.4
50.3
44.2
50.3
50.5
HotporQA
52.5
53.0
55.1
53.3
52.5
56.2
58.7
62.2
62.8
56.5
56.9
58.4
59.0
62.3
62.1
FiQA-2018
28.1
24.4
28.2
27.7
23.1
29.1
32.9
29.4
33.7
33.0
30.1
33.3
34.9
32.1
34.5
Signal-IM IM (RT)
26.2
24.8
27.0
28.3
27.3
27.3
27.3
26.7
28.4
27.0
27.8
29.5
27.8
29.0
30.5
TREC-NEWS
35.8
33.7
38.3
37.3
32.3
40.5
40.0
39.4
42.9
40.7
40.1
43.8
39.4
37.3
40.7
Robust(4
35.4
35.1
38.6
37.9
35.7
40.9
36.8
41.5
44.6
40.6
43.6
46.8
40.0
42.9
ArguAna
43.1
42.8
44.1
31.7
45.2
42.3
31.2
49.4 20.4
46.3
Touché-2020
16.8
28.7
28.5
17.4
29.1
30.1
16.9
19.4
17.9
30.5
30.4
16.0
26.9
26.1
CQADupStack
31.9
26.0
32.3
32.7
27.7
32.3
33.2
32.4
36.1
35.5
32.6
35.8
35.5
33.8
37.2
Quora
83.8
82.5
84.6
85.6
84.9
85.8
84.5
84.0
86.1
86.1
85.6
86.8
85.3
85.2
86.2
DBPedia
32.8
32.5
32.0
34.7
34.0
33.9
37.4
40.4
38.7
36.3
37.5
37.2
35.4
37.7
35.7
SCIDOCS
14.1
12.4
14.6
14.3
12.6
15.5
15.8
14.8
17.0
15.2
14.3
16.1
16.1
14.7
16.9
FEVER
66.9
66.9
72.0
69.4
68.5
72.2
74.8
73.2
75.6
69.2
72.2
73.0
Climate-FEVER
22.2
18.4
22.3
20.2
14.7
22.3
23.4
17.9
23.6
21.8
17.6
22.2
19.7
SciFact
65.4
57.0
63.4
64.4
57.1
63.2
68.6
66.3
69.8
66.2
61.0
65.1
67.8
64.7
68.7
Avg. Performance
40.2
38.1
41.9
40.9
38.4
42.9
42.1
40.9
44.9
42.8
42.9
45.9
43.3
43.5
46.1
Table Results on BEIR benchmark (nDCG@ 10) when adopting different pre-trained models. Teacher refers to the model finetuned MS MS-MARCO.
Quary Genaration
Toucher Betrissas
Proparation
Hard Napatine Vining
Teasher Reranker
Preporation
Poeudo Labelling
Shusent Training
Prsseges
BA125 Negutines
Deuse Relriever (BM225
Hand Nogutivss
Texchr Retrisser
Teacar Reiziever
dy
Hard Negatives
Quary Genertoe
Tuccher
Syntktic
w Paundo Labals
Tean
Stukeri Rexaalke (Pretraic)
Initsite
Stucknt Raruiuss
Figure 5: The overview of adapting the self training framework to reranker.

---

Motivation
Practical
Cognitive
Intrinsic
Fairness
Generalisation type
Compositional
Structural
Cross Task
Cross Language Cross Domain
Robustness
Shift type
Covariate
Label
Full
Assumed
Shift source
Naturally occuring
Partitioned natural
Generated shift
Fully generated
Shift locus
Teain-test
Finetune train-test
Pretrain-train
Pretrain-test


Reviews:
[{"review_id": "ArsNsJrCn7", "reviewer": "Reviewer_E4NL", "summary": "", "questions": "Please check the \"Reason to Reject\" section, my questions are pretty much there.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments\n\n---\n> W1: The claim that using the soft labels generated by teacher retrievers would be more robust to noise is not well-supported.\n\nWe gave a motivating example in Table 1 to show why soft labels are more robust to potential noise compared to using 0/1 hard negatives. Since the query generator is not perfect, the synthetic queries may contain hallucinations or only reflect one aspect of the originating passages. Moreover, sometimes the synthetic query is only superficially related to the originating passages with only keywords overlapped. As shown in the example of Table 1, we find there exists more than one passage that is more semantically related to the synthetic query than its originating one, and this is not uncommon in the whole dataset. In such a case, simply taking the originating passage as positive with label 1 and treating others equally with label 0 will mislead the model during training. By contrast, soft labels provide the retriever with a larger number of training signals by learning to match a distribution. Thus, soft labels are generally more favorable to hard labels by providing the relevances to a query that is comparable among passages. We also empirically found that using hard labels can hurt the performance, as shown in the 3rd row of Table 4.\n\n---\n> W2: Not clear how the model achieves “out-of-domain” ability\n\nWe show that when given only the unlabelled text corpus from tasks in out-of-domain settings, a teacher model trained in in-domain data (e.g., wikipedia) can generate rich training signals for out-of-domain data (e.g., biomedical and scientific). Moreover, a student model trained on such silver training data can surpass the teacher in out-of-domain settings. That is to say, a model trained in in-domain data can produce a model with a significantly better performance in unseen domains without using any human-annotated supervision signals.\n\n---\n> W3: The out-of-domain results of student retriever in Table 5 seems weak compared to other baselines\n\nThe major baselines we compared are GPL and PTR. As shown in Table 5, PTR uses a massive language model for synthetic queries, while our method only employs a T5-base model with 220M parameters. Considering the comparable results, our method is more practical compared to theirs. Moreover, since the quality of their synthetic queries is expected to be much better than ours, it is reasonable to expect that by integrating with their queries, our method could gain further benefits. For GPL, we use similar settings for query generation. However, they used a more expensive reranker model to generate soft labels, which is 20 times longer than ours based on our rough comparison. Moreover, their training time is also roughly 3 times longer than ours. Therefore, considering the efficiency of our method in query generation, soft labelling, and training, our method is useful and its merits can not be overlooked by simply comparing the raw numbers presented in the table. \n\n---\n> W4: Missing convergence analysis\n\nThis was mentioned in lines 944-945 in Appendix A1 and sorry for not presenting this explicitly. We show the detailed convergence analysis below. As we can see, employing one iteration of our algorithm leads to the best results and the performance starts to decline with more iterations.\n| Iteration| MSMARCO |\n| :--: | :----: |\n| 0 | 37.4 |\n| 1 | **39.4** |\n| 2 | 39.1 |\n\n\n| Iteration | FQ | SF | AA | CF | DB | CQ | QU | SD | FE | NF | TC | T2 | HP | NQ | RB | TN | SG | BA | Avg |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| 0 | 30.1 | 61.0 | 37.3 | 17.6 | **37.5** | 32.6 | 85.6 | 14.3 | 71.8 | 33.1 | 74.1 | **30.5** | 56.9 | 48.4 | 43.6 | 40.1 | 27.8 | 29.5 | 42.9 |\n| 1 | 33.3 | 65.1 | 50.1 | **22.2** | 37.2 | 35.8 | **86.8** | 16.1 | **72.2** | **34.9** | **76.7** | 30.4 | **58.4** | **50.3** | **46.8** | **43.8** | **29.5** | 35.7 | **45.9** |\n| 2 | **33.7** | **66.7** | **50.6** | 22.2 | 35.8 | **36.5** | 86.2 | **16.5** | 70.9 | 34.9 | 75.7 | 29.2 | 57.5 | 49.2 | 46.4 | 42.8 | 28.7 | **36.9** | 45.6 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed are evaluated from several perspectives in the experiment.\n\n- The motivation to solve \"out-of-domain\" problems in retrieving is interesting.\n\n- The paper is overall easy to follow.", "reasons_to_reject": "- The main contribution of this paper is to use self-training framework to learn a student retriever that takes training data query-paraphrase pairs and the negatives given by a teacher retriever in an interactive manner. The authors claim that using the soft labels generated by teacher retrievers would be more robust to noise. However, this claim is not well supported or justified in the paper. More specially, since the teacher model is trained with the labelled data (in the beginning), why using the soft label can be more robust?\n\n- Another main claimed strength of the proposed model is the generalizability to \"out-of-domain\". However, It is not clear how the model achieves this ability. There is not specific design in the training algorithm. Also, the out-of-domain results of student retriever in table 5 seem to be very weak compared to baselines, it is not convincing that the model can really solve \"out-of-domain\" scenarios. \n\n- Since the proposed self-training is an iterative training framework, a convergence analysis (e.g., convergence experiments) is required to understand the training process.", "questions_to_authors": "Please check the \"Reason to Reject\" section, my questions are pretty much there.", "paper_topic_and_main_contributions": "This paper proposes a self-training framework that iteratively enhances the retriever. More specifically, the student retriever is improved continuously trained by the soft-label and negatives given by the teacher retriever. This final student retriever is evaluated on both in-domain and out-of-domain scenarios. Several ablation studies reveal how the model works.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A", "Justification_for_Ethical_Concerns": "N/A"}, "extra_text": {}}, {"review_id": "b2EmB68BbL", "reviewer": "Reviewer_uzg5", "summary": "", "questions": "1. What is the noise rate in $T’$? This information will be a strong evidence to motivate your method. Either a small scale human evaluation or automatic model evaluation is ok.\n\n2. How did you evaluate the quality of the pseudo labels for generated queries? I suppose its quality should be higher than the original $T’$, otherwise the student model can hardly be improved. \n\n3. What is the performance if we use hard labels in Algorithm 1 line 6?\n\n4. Have you considered also updating the query generator via the self-training mechanism?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments.\n\n---\n> Q1: What is the noise rate in $\\mathcal{T}'$?\n\nWe did an automatic evaluation of the noise rate in $\\mathcal{T}'$ in MSMARCO. We consider one query-passage pair as noisy if the originating passage cannot be ranked as the top-1 prediction by a well-trained retriever (the teacher retriever in our case). We find that only 22\\% of query-passage pairs survive after this procedure, which means there exist other passages that are more semantically related than the originating passages for 78\\% of synthetic queries. Moreover, considering the characteristics of MSMARCO where passages are split from documents, there still exist other passages that are semantically related to those surviving queries. In other words, simply using 0/1 hard labels by treating all other passages equally as negatives is not an ideal solution.\n\n---\n> Q2: How did you evaluate the quality of the pseudo labels for generated queries?\n\nWe first randomly sampled 100 synthetic queries and manually checked if the originating passage for each query was correct. We found 49 of them are correct. Out of these 49 queries, we found that 93% of the time the soft labels can find other passages that can also correctly answer the query, that is assigning a high probability mass to these passages. For the other 51 incorrect query-passage pairs, a different passage that makes a better match than the originating one can be found for 42 of them. This shows that soft labels are beneficial to both types of query-passage pairs. \n\n---\n> Q3: What is the performance if we use hard labels in Algorithm 1 line 6?\n\nWe show the experiment of using hard labels in the 3rd row of Table 4 (i.e.w/o pseudo labels). In both datasets, we see directly using the hard labels by taking the originating passage as the positive does not benefit the retriever, and it even hurts the results in most metrics. The corresponding experiment in the out-of-domain BEIR benchmark is the GenQ variant, we see that further training the teacher retriever on hard labels doesn’t provide any benefits in overall performance.\n\n---\n> Q4: Have you considered also updating the query generator via the self-training mechanism?\n\nThe effectiveness of our method mainly depends on two aspects: the quality of generated soft labels and how close the synthetic queries are to the gold ones. In this work, we mainly focus on the first one. However, it is interesting to see how to improve the quality of query generation. One way could be applying large language models but at the cost of massive inference time. Using the self-training framework is a good option by iteratively mixing gold queries with selected synthetic queries with high quality to train an improved generator. Another way could be taking the performance on the target task (i.e. retrieval) as a reward to improve the generator. We would like to explore this as future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is well written and easy to follow.\n\n2. The idea of improving dense retrieval using self-training sounds reasonable to me.\n\n3. The experiments are extensive and support the main claim. \n", "reasons_to_reject": "1. The technical novelty of this paper is limited. But it is a minor issue for an empirical study paper. \n\n2. The motivation of generating pseudo labels for synthetic queries is not strong enough. Even though the authors provide a motivation example in Table 1, a comprehensive noise analysis of the generated queries is favorable.\n", "questions_to_authors": "1. What is the noise rate in $T’$? This information will be a strong evidence to motivate your method. Either a small scale human evaluation or automatic model evaluation is ok.\n\n2. How did you evaluate the quality of the pseudo labels for generated queries? I suppose its quality should be higher than the original $T’$, otherwise the student model can hardly be improved. \n\n3. What is the performance if we use hard labels in Algorithm 1 line 6?\n\n4. Have you considered also updating the query generator via the self-training mechanism?\n", "paper_topic_and_main_contributions": "This paper works on dense retrieval. The authors propose a self-training framework to improve dense retrieval with limited data. Specifically, they propose to generate queries for unlabelled passages. However, such generated dataset may carry noise, the authors propose also to generate pseudo labels by a teacher model to improve the quality of the generated dataset. The authors evaluate their method on both general domain and out-of-domain benchmarks. The results show the proposed method can not only improve the performance, but also reduce the need for labeled data. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CwJdX9Drkw", "reviewer": "Reviewer_ugd9", "summary": "", "questions": "1. Have you tried to train a single bi-encoder model with the synthetic data and cross-encoder scores provided by GPL? If self training is comparable or even better; then, we can further claim that the self training approach can produce equal outcome but saving a lot of time. Maybe give a rough data generation time estimation comparison is better.\n2. Have you tried self training with sentence cropping as COCO-DR[1] or the variant of DRAGON[2]? In this way, the data generation process can be more efficient.\n\n[1] Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk. 2022. Coco-dr: Combating distribution shifts in zero-shot dense retrieval with contrastive and distributionally robust learning. In Proc. EMNLP.\n[2] Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023a. How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval. ArXiv:2302.07452", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments.\n\n---\n> W1: Differences from GPL and PTR\n\nThe main motivation of GPL and PTR is to improve the retrieval accuracy in out-of-domain in a zero- or few-shot manner. By contrast, our method can improve retrievers in both in-domain (rich resources) and out-of-domain (zero resources) settings. Moreover, GPL and PTR all rely on external models to gain improvements, either through soft label generation using an expensive reranker or a massive query generator. Instead, our method proves that a retriever can be self-improved. The merits of our method can not be ignored given the fact that it can achieve comparable results to GPL and PTR while being efficient in both training and model deployment.\n\n---\n> Q1: Single GPL model\n\n| Model | FQ | SF | AA | CF | DB | CQ | QU | SD | FE | NF | TC | T2 | HP | NQ | RB | TN | SG | BA | Avg |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| GPL (66M * 18) | 34.4 | 67.4 | **55.7** | **23.5** | **38.4** | 35.7 | 83.6 | **16.9** | **75.9** | 34.5 | 70.0 | 25.5 | 58.2 | 48.3 | 43.7 | 42.1 | 27.6 | 44.2 | 45.9 |\n| GPL (110M) | **34.6** | **67.6** | 53.4 | 23.4 | 38.1 | **36.2** | 83.6 | 16.7 | 75.9 | 34.9 | 73.6 | 24.4 | 56.3 | 49.0 | **46.9** | 42.8 | 27.0 | **44.7** | **46.1** |\n| Student Retriever | 33.3 | 65.1 | 50.1 | 22.2 | 37.2 | 35.8 | **86.8** | 16.1 | 72.2 | **34.9** | **76.7** | **30.4** | **58.4** | **50.3** | 46.8 | **43.8** | **29.5** | 35.7 | 45.9 |\n\nWe show the extra experiments when training a single GPL model. We follow the same training setting as described in GPL but only change the initialised checkpoint to our teacher retriever presented in the paper to ensure a fair comparison. As shown in the table,  the single GPL model achieves marginal improvements over ours. However, this comes at a substantial extra cost. The time required for soft labelling increases by roughly 20 times (~0.8 ms per query for ours while ~20 ms for reranker labelling) and the training time also triples. Thus, considering such factors, our method is still competitive.  \n\n---\n> Q2: Using crop sentences as query\n\nYes, we tried using crop sentences in our preliminary experiments. In the in-domain settings (i.e., MSMARCO and NQ), we found the results of using crop sentences are inferior to those of using synthetic queries. We reason that since synthetic queries are sampled from a query generator trained on gold query-passage pairs, the distribution of synthetic queries and their formats are much closer to gold ones compared to sentence croppings. In the BEIR benchmark, we found sentence croppings lead to better performance in tasks where the query is in the format of short sentences (such as Climate-Fever for fact verification), while in other tasks where the query is natural questions, using synthetic queries is a better choice. To make the settings consistent and our method comparable to others, we used synthetic queries for all our experiments. Future work would consider how to mix both synthetic queries and sentence croppings to achieve better overall results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Overall, this paper is well motivated and experiments are comprehensive enough.\n1. The motivation of the paper is clear and reasonable. Overall, the paper is easy to read.\n2. The self training approach can improve both bi-encoder and cross-encoder ranking models and the authors demonstrate the effectiveness of the approach across various datasets.\n", "reasons_to_reject": "The paper seems OK. However the noise induced by synthetic queries and possible solutions have been discussed by GPL and PTR. Thus, I feel the main contribution is to save training time compared to GPL; thus, I feel the contribution is not enough for a long paper.\n1. I think the proposed method is very similar to Promptagator (PTR) by generating synthetic queries using seq2seq models and cleaning data with self consistency. And the proposed method is also similar to GPL but replacing the cross-encoder teacher with a bi-encoder teacher. The main difference is that PTR and GPL train one model for each dataset while the authors train one model for all datasets (but still have two models for MS MARCO and BEIR). Although the main claim that synthetic queries are noisy and the corresponding solution are reasonable, this claim and solution are well known according to the PTR and GPL. That is, I do not see any significant impact of the paper rather than just trying the different training settings of GPL.\n2. As for comparison, I think that the authors should also follow GPL’s training strategies and train a single retrievers for all BEIR dataset. In this way, we can see the importance or the gain of self training. However, the authors only compare with the single model trained on GenQ. ", "questions_to_authors": "1. Have you tried to train a single bi-encoder model with the synthetic data and cross-encoder scores provided by GPL? If self training is comparable or even better; then, we can further claim that the self training approach can produce equal outcome but saving a lot of time. Maybe give a rough data generation time estimation comparison is better.\n2. Have you tried self training with sentence cropping as COCO-DR[1] or the variant of DRAGON[2]? In this way, the data generation process can be more efficient.\n\n[1] Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk. 2022. Coco-dr: Combating distribution shifts in zero-shot dense retrieval with contrastive and distributionally robust learning. In Proc. EMNLP.\n[2] Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023a. How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval. ArXiv:2302.07452", "paper_topic_and_main_contributions": "The authors propose a self training strategy to iteratively improve the augmented training data. The experiments on MS MARCO, NQ and BEIR shows that the self training strategy improves over the standard training strategy for both bi-encoder and cross-encoder ranking models. Generally, I think this paper is a variant of GPL by replacing cross-encoder with bi-encoder teacher to improve the synthetic data generation throughput. As for the comparison of PTR, this work requires human label data while PTR does not. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a self-training strategy to iteratively improve the augmented training data. Compared with baseline models, the proposed method can not only improve the performance, but also reduce the need for labeled data. The reviewers generally hold a positive view of this paper. The author has also addressed some of the concerns raised by the reviewers. However, further careful revision and supplementation may still be required. The rebuttal has addressed reviewers' concern on the motivation and novelty concerns, and the authors agree to revise this paper accordingly.
Decision: Accept-Findings