Effective Contrastive Weighting for Dense Query Expansion
Year: 2023

Abstract:
Verbatim queries submitted to search engines often do not sufficiently describe the user’s search intent. Pseudo-relevance feedback (PRF) techniques, which modify a query’s representation using the top-ranked documents, have been shown to overcome such inadequacies and improve retrieval effectiveness for both lexical methods (e.g., BM25) and dense methods (e.g., ANCE, ColBERT). For instance, the recent ColBERT-PRF approach heuristically chooses new embeddings to add to the query representation using the inverse document frequency (IDF) of the underlying tokens. However, this heuristic potentially ignores the valuable context encoded by the embeddings. In this work, we present a contrastive solution that learns to select the most useful embeddings for expansion. More specifically, a deep language model-based contrastive weighting model, called CWPRF, is trained to learn to discriminate between relevant and non-relevant documents for semantic search. Our experimental results show that our contrastive weighting model can aid to select useful expansion embeddings and outperform various baselines. In particular, CWPRF can improve nDCG@10 by upto to 4.1% compared to an existing PRF approach for ColBERT while maintaining its efficiency.

heading: 1 Introduction
text: When using search engines, users frequently enter queries that insufficiently express their desired intent. For instance, a user who issues the query georgia run off elections may indeed be looking for details about a specific electoral procedure in the US state of Georgia. For search algorithms that rely on lexical matching, such as BM25, this can result in a lexical gap, since relevant documents may just as easily use different terms (e.g., GA and 2nd-round). Pseudo-Relevance Feedback (PRF) techniques are often employed to overcome such a lexical gap. Indeed, classical PRF techniques,
such as RM3 (Abdul-Jaleel et al., 2004), have been widely used to enrich the user’s query with terms selected from the initial retrieval top-ranked documents, i.e. the pseudo-relevance feedback set (Amati and Van Rijsbergen, 2002; Roy et al., 2016; Cao et al., 2008). This expanded query is capable of overcoming the lexical gap if the pseudo-relevance feedback documents are relevant and additional related terms can be identified (for instance adding GA to the query). However, there is a risk that the added terms drift the intent of the query (for instance, adding terms such as Tbilisi that relate to the country of Georgia rather than the US state).
An alternative approach for overcoming the lexical gap is to perform semantic search over learned embedded documents (single representation, e.g., ANCE (Karpukhin et al., 2020)) or tokens (multiple representations, e.g., ColBERT (Khattab and Zaharia, 2020)). Such dense retrieval approaches enable queries to retrieve documents that do not necessarily contain the query terms. However, the encoded query vectors might still not adequately express the user’s desired intent. Indeed, several recent works have shown that implementing PRF techniques within the dense retrieval paradigm – such as ANCE-PRF (Yu et al., 2021), VectorPRF (Li et al., 2023) and ColBERT-PRF (Wang et al., 2021, 2022b) – can further improve retrieval effectiveness. ColBERT-PRF has been shown to be
12688
more effective than Vector-PRF and ANCE-PRF variants applied on various dense retrieval models.
A key limitation of ColBERT-PRF is that it relies on clustering and inverse document frequency (IDF) statistics for identifying the expansion embeddings — both of which are heuristics. This approach ignores valuable context present in the embeddings, e.g., for the georgia run off elections query, effectiveness might be improved by adding an embedding for ‘US’, however, this would not likely be selected due to its low IDF (indeed, ‘us’ is also a pronoun, and is often included in stopword lists). Moreover, there is no direct connection between the expansion embeddings selected by the heuristic and the semantic search algorithm itself.
To overcome these problems, we propose a contrastive weighting method, called CWPRF, to select and weight the usefulness of the feedback embeddings for dense expansion. More specifically, for each feedback token, we construct a contrastive objective, where, given positive and negative documents, CWPRF is trained to assign high weights to the tokens that are semantically closer to tokens occurring the positive document than to those in the negative document. Introducing the PRF passages into the training procedure of CWPRF enables the model to take the surrounding context into account when identifying the useful tokens from the PRF passages. Meanwhile, training CWPRF with the contrastive objective allows it to learn the effective weights for expansion embeddings that are tailored for the semantic ranking task.
Figure 1 presents the trade-off between the retrieval effectiveness and the mean PRF stage execution time for a variety of existing dense PRF techniques on the TREC 2019 Deep Learning track queries, including Vector-PRF (Li et al., 2023), ANCE-PRF (Yu et al., 2021), ColBERTPRF variants (Wang et al., 2021, 2022b) and our proposed CWPRF method. As the figure shows, the default ColBERT-PRF implementation outperforms ANCE-PRF and Vector-PRF in terms of retrieval effectiveness but requires a longer execution time. Meanwhile, our proposed CWPRF achieves the highest nDCG@10 score without requiring high computational cost.
Overall, our contributions are summarised as follows: (1) We propose CWPRF, a contrastive weighting method for dense query expansion; (2) We construct the contrastive targets and train our CWPRF model to assign high expansion weights
for tokens that can discriminate the relevant documents from the non-relevant documents. Based on the predicted weights, CWPRF helps to identify useful expansion embeddings for generating refined query representations; (3) We perform an extensive empirical evaluation and demonstrate how to effectively train our CWPRF in a supervised way; (4) Experiments show that our CWPRF can achieve significantly higher retrieval effectiveness but with less execution time than the default ColBERT-PRF.

heading: 2 Preliminaries
text: Given a query q and a document1 d, we employ the pre-trained ColBERT (Khattab and Zaharia, 2020) query and document encoders to encode the query and document, respectively. The ColBERT query and document encoders share weights but are distinguished by the different prepended special tokens. The ColBERT model is defined as a linear layer upon the raw token embeddings obtained from a BERT model: ColBERT = Linear(BERT(t1, ..tn),m)) ∈ Rm, where m is typically set to 128 (Khattab and Zaharia, 2020). In particular, the input query tokens are encoded as a list of query embeddings (each of dimension m), as follows: ϕq = ColBERT([CLS], [Q], q1, ..., q|q|) ∈ R32×m, where m = 128 and the ‘[MASK]’ embeddings are used to pad the input query embeddings to 32. Similarly, for a document d, we encode it into a list of document embeddings, as follows: ϕd = ColBERT([CLS], [D], d1, ..., d|d|) ∈ R|d|×m.
Based on the obtained query and document embeddings, the final similarity score between a query and a document, s(q, d), is given by the summation of the highest cosine similarity among the document embeddings for each query embedding:
s(q, d) = ∑|q| i=1MaxSim(ϕqi , ϕd) = ∑|q| i=1max |d| j=1 ϕ T qiϕdj .
(1)

heading: 3 Contrastive Weighting for Dense PRF
text: This section first provides an implementation overview of CWPRF for dense query expansion in Section 3.1. It then details the contrastive weighting method and the training procedure of CWPRF in Sections 3.2 & 3.3, respectively.

heading: 3.1 CWPRF Implementation Overview
text: An overview of CWPRF in a multiplerepresentation dense expansion framework is illustrated in Figure 2, where three stages are presented: (1) initial retrieval, (2) predicting the PRF tokens weights and (3) retrieval with the refined query representation. We note that the first and the third stages of this framework are shared with ColBERT-PRF (Wang et al., 2021, 2022b).
In the initial retrieval stage, we obtain a result list in response to the original user’s query q. The top fp documents are employed as the pseudorelevance feedback documents. Then, as input for our trained CWPRF model, we append the PRF passages to the query. The model outputs weights for each query token as well as for the feedback tokens. Finally, according to these produced weights, we identify fe feedback tokens with high weights as our expansion tokens and append their corresponding expansion embeddings obtained from ColBERT’s document encoder to the original query representation. Following conventional PRF models going back to Rocchio (Croft et al., 2010), the overall contribution of the expansion embeddings is further controlled by a hyper-parameter denoted by β. Finally, the refined query representation is reissued to the underlying dense retrieval model, i.e. ColBERT, so as to return the final document list.
The core challenge, which lies in the second PRF stage, is how to accurately predict the expansion weights for the refined query representation that can more effectively perform semantic search. We propose a novel contrastive weighting model that learns to weight each feedback token individually based on the extent it will increase the score of the relevant document w.r.t. the non-relevant one(s).

heading: 3.2 CWPRF Feedback Embedding Weighting
text: Building on ColBERT, and taking an initially retrieved set of pseudo-relevant feedback pas1 We use ‘document’ and ‘passage’ interchangeably.
sages as input, the CWPRF model aims to predict the importance of each (token-level) feedback embedding in the feedback passages. This is achieved using a separate BERT model instance, which takes a list of input tokens and returns a scalar weight for each token: CWPRF(t1...tn) = Linear(BERT(t1, ..tn), 1)) ∈ Rn.
More specifically, given a document p in the pseudo-relevant set, which is tokenised into a sequence of PRF tokens p1, p2, ..., p|p|, we employ the ColBERT encoder to obtain its embeddings: ϕp = ColBERT([CLS], [D], p1, ..., p|p|) ∈ R|p|×m. Then we obtain the feedback weight for each PRF token using CWPRF which takes the query representations as well as the PRF representations as input:
ws = CWPRF( query tokens︷ ︸︸ ︷ [CLS], [Q], q1, q|q| PRF tokens︷ ︸︸ ︷ [D], p1, ..., p|p|) .
(2) According to the returned importance score for each of the feedback embeddings in ϕp, we identify the highly important ranked embeddings as our expansion embeddings. The expansion embeddings are appended to the original query embeddings to refine the query representation. Note that the original query is included in the invocation of CWPRF(·) – this is by design, to ensure that the CWPRF model considers the relation of the PRF tokens to the original query. However, we ignore the predicted weights of the original query; following ColBERT-PRF, the weights of the original embeddings are assumed to be unchanged. Furthermore, we apply a ReLU upon ws, to ensure that feedback weights non-negative. Finally, the score for a document can be calculated as the summation of the weighted MaxSims using the refined query representation:
s′(q, fe, d) = s(q, d) + β ∑|fe|
i=1wsi ·MaxSim(fei , ϕd). (3)

heading: 3.3 Training CWPRF
text: To train CWPRF(·), we construct a contrastive target for each feedback token. In particular, we use a conventional training file containing triples of ⟨q, d+, d−⟩, and supplement it with PRF passages, i.e. the passages highly ranked for the original query q, which we assume to be relevant. The aim of our training objective, therefore, is to identify which tokens of a feedback passage p result in the positive passage being scored much higher than the negative passage, when the feedback passage is itself treated as the query. Therefore, for each feedback token, and given the positive and negative documents, CWPRF is trained to assign high weights to the tokens that are semantically closer to the tokens occurring in positive document than those in the negative document. Hence, the target for the i-th PRF token, pi, is obtained as:
t(pi) = MaxSim(pi, d +)−MaxSim(pi, d−),
(4) where MaxSim(., .) measures the semantic similarity between representations, as per Equation (1).
The target generation process for CWPRF is illustrated in Figure 3. This figure presents the interaction matrices between a PRF document (“cause baby heart rate increase”) obtained from the returned documents list in response to the query: “is a little caffeine ok during pregnancy” compared to the positive and negative document. The shading is indicative of the magnitude of dot product similarity between a PRF embedding and a document embedding, while the highest document embedding for each PRF embedding is indicated with a •. For each PRF
embedding, we subtract the negative similarity from the positive similarity, resulting in an importance score for each PRF embedding. In this example, ‘cause’ and ‘heart’ are the most important tokens. These differences are used as targets for learning the CWPRF model. pAAAT = p11, p 1 2, .., p 1 |p1|, [SEP], ..., p k 1, p k 2, .., p k |pk|, [SEP]. However, in common with all BERT models, |pAAAT| is limited to 512 tokens, so some tokens may be cut off for large feedback sets. Hence, in the OAAT training mode, each PRF document is regarded as an individual PRF sequence. The CWPRF training is then conducted for each feedback passage individually.
In-Batch Negative Sampling: In-Batch Negative (IBN) sampling is a technique that has been widely used for training effective dense retrieval models such as DPR (Karpukhin et al., 2020; Lin et al., 2020). However, it has not previously been applied for query expansion weighting. To promote the discriminative expansion embeddings and suppress the unimportant ones during our target generation, we adapt the idea of in-batch negative (IBN) sampling during the training of CWPRF. Thus, each training sample is equipped with one positive sample and B−1 negative samples, where B is the batch size used during training. As a consequence, the target for the i-th PRF token is obtained as:
t(pi) = MaxSim(pi, d +)−max|B−1|j=1 MaxSim(pi, d−j ).
(5) This ensures that the importance of each feedback embedding for ranking a positive passage is discounted by its presence in all negative passages of the batch. While IBN is commonly used for
training ranking models on entire passages, our adaptation focuses instead on the token-level embedding importance.
Loss Functions: CWPRF is trained to assign weights from the target signal using the following objectives. For AAAT training, the loss is computed as follows:
LAAAT = 1
N
N∑
i=1
(tpi − wspi)2 , (6)
where N is the total number of tokens in the PRF sequence. For the OAAT training mode, we compute the loss for each PRF sequence and add them to obtain the total loss:
LOAAT = k∑
j=1
( 1
N
N∑
i=1
(t pji − ws pji )2
) . (7)
At the inference time, we apply CWPRF consistently with its training mode, i.e. AAAT or OAAT.

heading: 3.4 Discussion
text: Connection to ColBERT-PRF: Similar to ColBERT-PRF, CWPRF is implemented in the multiple representation late interaction dense retrieval paradigm. However, in contrast to ColBERT-PRF, CWPRF is a supervised approach, which is tailored for semantic search by selecting and learning the contrastive weights for the discriminate expansion embeddings. The Kendall’s τ correlation between the contrastive weights learned by CWPRF and the IDF weights assigned by ColBERT-PRF is only 0.1, which indicates that CWPRF prioritises differently the feedback embeddings. Moreover, compared to ColBERT-PRF, CWPRF has advantages over ColBERT-PRF in that it can identify expansion embeddings that may have low IDF values. It can also avoid the expensive clustering and nearest neighbour lookups used by ColBERT-PRF. Connection to Learned Sparse Models: In practice, the CWPRF model structure is similar to unexpanded learned sparse retrieval approaches (Dai and Callan, 2020; Mallia et al., 2021; Lin and Ma, 2021). Importantly, however, the learning objectives are different; learned sparse retrieval optimises for relevance directly, while CWPRF is optimised to identify and weight the most helpful query expansion embeddings.

heading: 4 Experimental Setup
text: Datasets: We conduct our experiments using the MS MARCO (Nguyen et al., 2016) passage rank-
ing dataset. The corpus consists of 8.8M passages from web pages, along which are provided 0.5M training queries with sparse document relevance judgements. We employ the TREC Deep Learning track 2019 query set (43 queries with an average of 215 relevant documents per query) as our validation set and use TREC 2020 (54 queries with 211 relevance assessments per query) query set as our test set due to their dense judgements, which can provide more reliable evaluations (Carterette et al., 2006; Craswell et al., 2021). As pseudo-relevance feedback approaches are known not to show a benefit on sparsely judged documents (Amati et al., 2004), we omit the MS MARCO Dev queries. In addition, we also report the performance of CWPRF on four BEIR (Thakur et al., 2021) datasets in Appendix A.2.
We evaluate our method using the official metrics of TREC, such as nDCG@10, MAP@1000 and Recall@1000. We follow the standard practice of TREC (non-relevant = 0 or 1 and relevant = 2 or 3) for the binary-relevance based metrics (MAP and Recall). To investigate the extent that semantic matching, rather than exact token matches occurs when retrieving documents, we also report the semantic match proportion (SMP) (Wang et al., 2022a) for the ColBERT-based system. The calculation of SMP is detailed in Appendix B. For significance testing, we use the paired t-test (p < 0.05) and apply the Holm-Bonferroni multiple testing correction.
Experimental Implementation: Both the AAAT and OAAT training modes are trained using the MS MARCO "small" triples training set, i.e. the triplets of ⟨q, d+d−⟩. Following the settings of ColBERT (Khattab and Zaharia, 2020), we use a ColBERT checkpoint trained using the MS MARCO passage ranking training triplets for 44k batches. We employ the query encoder from the trained ColBERT model to encode the query (the maximum query length is set to 32) and the document encoder to encode the pseudo-relevance feedback documents (the maximum document length is set to 512 for the AAAT training mode and 180 for the OAAT training mode). We set the maximum length to 180 when encoding the positive and negative passages. For ease of notation, we use » to denote a retrieval pipeline, for instance BM25 » ColBERT indicates applying the ColBERT reranker on the results obtained from BM25. For setting the hyper-parameters of CWPRF, we use
the TREC 2019 queries as our validation set; the resulting settings of fp = 3, fe = 10 and β = 5 are obtained, as reported later in Appendix A.1. However, we note that fp = 3, fe = 10 is also the recommended setting for ColBERT-PRF (Wang et al., 2021). The high β value indicates the high contribution of the CWPRF identified expansion embeddings for semantic ranking. We further provide the ablations of performing only the expansion embeddings in Appendix A.1. For both CWPRF and ColBERT-PRF, we perform 5 sets of experiments with varied random seeds for each variant and report the median results.
Compared Systems: To test the effect of CWPRF, we compare the retrieval effectiveness of a CWPRF-based retrieval system with the following 4 families of retrieval approaches: (1) Sparse Retrieval Systems (denoted as Sparse in Table 1): We compare with the traditional lexical retrieval models, namely BM25 and BM25+RM3 (AbdulJaleel et al., 2004), and both with and without the ColBERT reranker, namely BM25 » ColBERT and BM25+RM3 » ColBERT models; (2) Dense Retrieval Systems (denoted as Dense): We compare with both single-representation and multiple-representation dense retrieval models, namely ANCE (Xiong et al., 2021) and ColBERT (Khattab and Zaharia, 2020); (3) Learned Sparse Retrieval Systems (denoted as L-Sparse): We compare with SPLADE-v2 (Formal et al., 2022), DeepImpact (Mallia et al., 2021) and DocT5Query (Nogueira et al., 2020), which are reranked using ColBERT; (4) Dense PRF models (denoted as D-PRF): we compare with the ANCE-
PRF (Yu et al., 2021), Vector-PRF (Li et al., 2023) and ColBERT-PRF (Wang et al., 2021) models. We compare our proposed CWPRF model with the more effective ColBERT-PRF Ranker model using the default KMeans clustering (Wang et al., 2021), rather than comparing with the Reranker. Moreover, when measuring the efficiency of CWPRF, we also compare with the recently proposed variants of ColBERT-PRF, which avoid costly ANN lookups when calculating IDF values for embeddings: KMedoids and KMeans-Closest (Wang et al., 2022b).

heading: 5 Results
text: This section studies the effectiveness as well as the efficiency performance of CWPRF in Section 5.1. The effects of the various training strategies are investigated in Section 5.2. We also provide qualitative analysis of CWPRF in Appendix A.3 and a breakdown performance of CWPRF according to various query types in Appendix A.4.

heading: 5.1 Main Results
text: Effectiveness: To evaluate the effectiveness of implementing the CWPRF model in a dense pseudorelevance feedback framework, we compare CWPRF with various families of baselines in Table 1.
Among the variants of CWPRF, we observe that when comparing the CWPRF-AAAT and CWPRFOAAT models (the bottom block), CWPRF-AAAT, which is trained with all PRF passages processed as a single sequence, consistently obtains a higher performance than CWPRF-OAAT, where the PRF sequences are considered individually. This sug-
gests that AAAT provides more relevant context than OAAT for the CWPRF model.
Next, we compare our CWPRF model with other baseline models. Firstly, we observe that the CWPRF models significantly outperform the sparse retrieval models and exhibit marked improvements over sparse-retrieval reranked with the ColBERT reranker. When compared with dense retrieval models, the CWPRF models significantly outperform both types of dense retrieval models. In particular CWPRF exhibits 7.4% (TREC 2019 queries) and 5.5% (TREC 2020 queries) improvements in terms of nDCG@10 than the ColBERT E2E model where no expansion embeddings are appended to the original query. This indicates the usefulness of our CWPRF model for selecting expansion embeddings to augment the query representation. We also compare the CWPRF models with the learned sparse systems, where the document tokens are enriched and reweighted, then applied with a more advanced reranker. We find that the CWPRF models significantly outperform the learned sparse models, indicating the effectiveness of learning the feedback weights and refining the query representation compared with document enrichment.
Finally, when comparing with existing dense PRF models, namely the ANCE-PRF, Vector-PRF and ColBERT-PRF models, we find that the CWPRF models exhibit significant improvements over ANCE-PRF on both query sets and significantly improves over ColBERT-PRF on the TREC 2020 query set. This indicates that our proposed CWPRF approach can select more appropriate expansion embeddings that can help to retrieve more relevant documents, and minimise topic drift.
Overall these results show that the retrieval effectiveness can be markedly improved with the CWPRF feedback weighting technique. Training CWPRF with all PRF passages as one context gives more precise retrieval at top ranks. In particular, the CWPRF approaches achieve the highest nDCG@10 and MAP performances on both query sets and exhibit upto 4.7% improvements on MAP and a 4.1% improvement on nDCG@10 for the TREC 2020 queries compared to ColBERT-PRF.
Semantic Match Proportion: To further explain the effect of implementing CWPRF for dense query expansion, following (Wang et al., 2022b), we also report the mean semantic match proportion (SMP) values for the models under the ColBERT dense retrieval paradigm in Table 1. In particular, SMP
quantifies the extent to which a query token exhibits an exact match (matching with the same document token) and a semantic match (matching with different document tokens) in the top-ranked documents. On analysing Table 1, we find that, for both query sets, the CWPRF models show lower Mean-SMP values than ColBERT-PRF, implying a more ‘focused’ retrieval. This is because CWPRF’s expansion embeddings correspond to the actual tokens while ColBERT-PRF’s expansion embeddings can be the centroid embeddings from clustering. By using more focused embeddings, nDCG@10 is improved compared to ColBERT-PRF. Efficiency: Following the three stages described in Figure 2, we also report the mean execution time of each stage for various dense PRF systems, including Vector-PRF, ANCE-PRF, variants of ColBERT-PRF with differing efficiency and our CWPRF methods. As Table 2 shows, our CWPRF method performs as efficiently as the most efficient ColBERT-PRF variant (KMedoids variant) and brings upto 3.06x speedup than the default ColBERT-PRF method (KMeans variant). Although CWPRF needs a longer execution time than Vector-PRF and ANCE-PRF, according to the effectiveness and efficiency tradeoff in Figure 1, CWPRF can significantly outperform them without adding much computational cost.
In summary, our CWPRF model achieves the highest nDCG@10 on the test set among all the compared baselines, while reducing the computational overhead costs compared with previous ColBERT-PRF approaches.

heading: 5.2 Ablation Study
text: Next, we inspect the effect of each of the training techniques, namely in-batch negative training, initialisation of the model, different learning objectives and training with PRF passages obtained
from different retrieval approaches. Experiments for each training strategy are grouped in Table 3. Effect of In-Batch Negative Sampling: In Table 3, we see that training CWPRF with further in-batch negative samples achieves higher retrieval effectiveness on both the TREC 2019 and TREC 2020 query sets, for both the AAAT and OAAT training modes. In practice, more negative training samples for the pseudo-relevance feedback tokens give more opportunity for the model to learn to properly weight unimportant terms in the feedback. For instance, the stopword “it” might occur in the feedback and positive passages, and not in the negative passage, resulting in a high weight. By applying IBNs, there is more chance for “it” to occur in any of the negative passages, reducing its learned target weight, and resulting in a more effective CWPRF model. Effect of Model Initialisation: Here, we investigate the training from scratch and training with the parameters initialised from an existing learned sparse model, namely uniCOIL (Lin and Ma, 2021). In the second group of Table 3, we find that this initialisation for CWPRF can lead to higher performance compared with training from scratch. Effect of Initial Retrieval: Now, we further investigate the training of CWPRF using the PRF passages obtained by sparse retrieval, using BM25, as well as by dense retrieval, using the ColBERT E2E retrieval model. From the final experiment group in Table 3, we observe that there is no obvious effectiveness difference between training CWPRF using different initial retrieval systems. Thus, considering the training efficiency, our default CWPRF is trained using the PRF passages obtained from a sparse BM25 initial retrieval.

heading: 6 Related Work
text: Dense Retrieval Models: Different from the popular “cross-encoder” based BERT-rerankers (MacAvaney et al., 2019; Nogueira and Cho, 2019), dense retrieval models usually build upon a BERT-based "bi-encoder" structure. The query and document are encoded separately into dense representations. There are two families of dense retrieval models: single representation dense retrieval and multiple representation dense retrieval models (Macdonald et al., 2021). In particular, in the single representation dense retrieval paradigm, exemplified by DPR (Karpukhin et al., 2020) or ANCE (Xiong et al., 2021), each query or document is represented into a single dense representation. Thus,
with the pre-computed document representations, retrieval can be conducted using the Nearest Neighbour search. In contrast, a multiple representation dense retrieval model encodes each token of the query and document into a dense representation, for instance, ColBERT model introduced by Khattab and Zaharia (2020). During retrieval, ColBERT performs an approximate nearest neighbour search (using FAISS (Johnson et al., 2019)) for each query embedding, followed by an exact scoring.
Pseudo-Relevance Feedback: Traditional lexical pseudo-relevance feedback (PRF) approaches, such as RM3 (Abdul-Jaleel et al., 2004) and Bo1 (Amati and Van Rijsbergen, 2002), as well as some recent proposed neural PRF models (Naseri et al., 2021; Li et al., 2018; Zheng et al., 2020) are applied upon sparse retrieval. Some initial efforts of implementing PRF mechanism for dense retrieval have been proposed recently: for instance, ColBERT-PRF (Wang et al., 2021), which is the most similar work to ours, selects cluster centroids as expansion embeddings. Different from ColBERT-PRF, where the expansion embeddings are prioritised by the closest token’s IDF, our work focuses on learning the contextualised weights of the PRF tokens and identifies the prominent ones as the expansion tokens that can better differentiate between the positive and negative documents. On the other hand, ANCE-PRF (Yu et al., 2021) is a supervised PRF approach, which trains an additional query encoder. Similar to
CWPRF-AAAT, the query and passages are passed to this new encoder. However, unlike CWPRF, ANCE-PRF is trained to produce a new single embedding for the query. Due to the nature of its single embedding output, it is infeasible to analyse how the query representation has been refined in ANCE-PRF, while CWPRF provides explicit weights for each selected expansion embedding. Contrastive Learning in IR: The contrastive learning technique has been used to optimise the query and document representations produced by the BERT-based dense retrieval models in IR. More specifically, some works focus on employing various negative sampling methods, such as the inbatch (Yih et al., 2011) and cross-batch negative sampling (Qu et al., 2021), while some works mine hard negative samples for more effective dense retrieval model (Xiong et al., 2021; Zhan et al., 2021). To the best of our knowledge, our work is the first to leverage contrastive learning for optimising the expansion weights for dense query expansion. Feedback Weighting for PRF: Various sparse PRF models have been proposed for weighting the importance of terms occurring in the feedback documents. For instance, Clinchant and Gaussier (2011) emphasised the importance of term rarity (cf. IDF) in selecting expansion terms, a finding echoed by Roy et al. (2019) – indeed, the importance of IDF is a key insight brought into ColBERT-PRF. Going further, while there have been several approaches that have proposed supervised models for selecting high-quality expansion terms for sparse retrieval, e.g., (Cao et al., 2008; Imani et al., 2019), none of these have tackled the problem from a dense retrieval perspective, as proposed in CWPRF.

heading: 7 Conclusions
text: Pseudo-relevance feedback has recently been shown to be effective for dense retrieval. In this work, we propose a deep language model-based contrastive weighting approach (CWPRF) for selecting useful query expansion embeddings and calibrating their expansion weights for semantic search. In particular, CWPRF is trained with a contrastive objective to learn to assign a high weight for feedback embeddings that can distinguish relevant documents from non-relevant documents. During retrieval, the embeddings of tokens appearing in the feedback documents that CWPRF predicts to be important are appended to the query embeddings. Extensive experiments performed on two query sets
demonstrate that our proposed CWPRF approach can significantly outperform the ColBERT dense retrieval model. In particular, CWPRF significantly improves over ColBERT-PRF by 4.1% in terms of nDCG@10 on the TREC 2020 query set without requiring high computational cost.
Limitations and Future Work
Our approach makes it feasible to learn the discriminative ability of an expansion embedding for dense retrieval. However, it is unclear how it may be adapted for the single-representation dense retrieval PRF model. In addition, in this work, we did not test the effect of the hard negative sampling and the number of negative samples for CWPRF. Finally, while we have focused on passage retrieval, longer document retrieval can be addressed through splitting documents into passages during indexing, retrieval and PRF, and applying a max-passage aggregation (Dai and Callan, 2019) to obtain a document ranking.
For future work, we will consider a hybrid approach to incorporate both the learned weights produced by CWPRF and the statistical information in the expansion embedding identification process. While PRF approaches typically increase query response time, they can also be used as teacher approaches to realise more effective and efficient student models (e.g., ColBERT-PRF is applied as teacher by Kim et al. (2022)). This means that improved PRF approaches, such as CWPRF, can also have downstream benefits to other retrieval approaches.

heading: Acknowledgements
text: Xiao Wang acknowledges support by the China Scholarship Council (CSC) from the Ministry of Education of P.R. China.

heading: A CWPRF Model Description
text: For reproducibility purposes, the source code for the training and inference of our CWPRF model is provided in our virtual appendix.2
A.1 Hyper-parameter Study The hyper-parameters for CWPRF are: the number of expansion embeddings fe and β which controls the overall contribution of the expansion embeddings. In addition, fp defines the number of feedback documents used during training and retrieval of CWPRF.
We first vary the fe and β hyper-parameters during retrieval. Figure 4 and Figure 5 presents the effectiveness of applying the CWPRF models while varying fe and β, respectively. Note that fe = 0 or β = 0 represents the vanilla ColBERT model without any expansion embeddings appended. From Figure 4, we find that for both CWPRF-AAAT and CWPRF-OAAT models, 10 expansion terms give the highest MAP performance. Thus, we set 2 github.com/Xiao0728/CWPRF_VirtualAppendix
fe = 10 as the default. This echoes the default expansion setting identified for ColBERT-PRF (Wang et al., 2021). For the β parameter (Figure 5), we find that for both CWPRF-AAAT and OAAT models, MAP performance shows a rising trend as higher β → 5 and becomes stable for β > 5. Indeed for β > 5, it appears that the feedback embeddings are dominating over the original query embeddings. This indicates the high contribution of the selected expansion embeddings during retrieval. Based on this, we set β = 5 as default in this work.
Indeed, we further quantify the contribution of the expansion embeddings of CWPRF technique and the original query embeddings in respectively in Table 5. We find that for CWPRF-AAAT, using only the 10 selected expansion embeddings for reranking, markedly outperforms using the query embeddings alone, which verifies the high contribution of CWPRF selected expansion embeddings.
Furthermore, we study how many PRF passages are needed for CWPRF. We conduct experiments to train both the CWPRF-AAAT and CWPRF-OAAT
models with a different number of PRF passages. We note that similar to the setting of the ANCEPRF model, due to the input length of BERT-based encoders, for the CWPRF-AAAT training, the maximum number of PRF passages is set to 3. On the other hand, for the OAAT training mode, as each PRF document is treated independently, there is no such requirement. The nDCG@10 results are presented in Figure 6. We observe that for CWPRF-OAAT, three feedback documents employed for training alone or evaluation alone give higher performance than other fp values. Overall, the combination of fp = 3 for both training and retrieval gives the highest performance. In addition, for CWPRF-AAAT, we find that a high MAP performance is achieved by training with only the top two PRF passages. However, this is not stable, as during retrieval, more PRF passages are needed under this setting. This indicates the model might not be trained enough. Moreover, we observe a similar trend for fp = 3 used for both training and retrieval. Thus, based on this observation, we suggest to set fp = 3 as the default for the training and evaluation of CWPRF.
A.2 Performance of CWPRF on BEIR
We examine the performance of the ColBERT and CWPRF (both trained on MSMARCO) in a zero-shot setting, using the BEIR datasets. We choose four datasets from BEIR that have dense judgements (Amati et al., 2004). Table 6 reports the performance of CWPRF as well as that of existing dense PRF models on four BEIR (Thakur et al., 2021) benchmarks. From Table 6, we find that CWPRF shows comparable performance with ColBERT-PRF but with much lower query latency. In addition, CWPRF outperforms ANCE-PRF by a large margin, indicating the superiority of our contrastive weighting method in such zero-shot settings.
A.3 Qualitative Analysis
Table 7 presents an example of the expansion tokens identified by CWPRF and the ColBERT-PRF technique as well as their retrieved top-ranked document. We observe that the two comparing methods can generate some expansion tokens in common but not necessarily received the same weights. In particular, compared to the ColBERT-PRF model, CWPRF can bring a highly relevant document (Label=2) to the top rank, by expanding with tokens: “revision” and “allows”, which are helpful in retrieving the more relevant document (indicated by their darker shading). Indeed, this superior ability to retrieve highly relevant documents at high ranks is more useful in a real-life retrieval scenario. Unexpectedly, “allow” and “allows” are identified by CWPRF as important expansion tokens. This indicates that CWPRF can take the context into account – more so than IDF.
The second example in Table 7 is selected from a case when CWPRF underperforms ColBERT-PRF.
Indeed, while CWPRF experiences a performance drop compared to ColBERT-PRF, it can still retrieve a document with label 3 at the top rank. This indicates the benefits of our contrastive weighting technique for bringing more relevant documents to the top positions.
Overall, we see that CWPRF can select more useful expansion embeddings to help bring more relevant documents on top, which would be more useful when implementing in a retrieval system in a real-life scenario.
A.4 Performance of CWPRF across Different Query Types
We further investigate the performance of the CWPRF models compared to ColBERT on different query types using the query taxonomy of Bolotova et al. (2022). Specifically, we combine the TREC 2019 and TREC 2020 queries to create a single query pool, consisting of 97 queries. Then, the merged queries are classified using a trained query category classifier according to the query taxonomy introduced by Bolotova et al. (2022). Figure 7a and Figure 7b illustrate the absolute difference in performance between the CWPRFAAAT model and the ColBERT-PRF model in terms of MAP and nDCG@10, respectively. Similarly, Figure 7c and Figure 7d provide comparisons for the CWPRF-OAAT model against ColBERTPRF. From Figure 7, it is evident that CWPRFAAAT demonstrates improvement across all query types in terms of MAP and nDCG@10, except
for the NOT-A-QUESTION type. However, it is worth noting that the number of queries belonging to the NOT-A-QUESTION type is quite low, comprising only approximately 1% (a single query) of the total. Similarly, we observe that CWPRF-OAAT also enhances performance across various query types, except for the single NOT-AQUESTION type in terms of MAP, and the REASON type (with a ratio of approximately 4.1%) in terms of nDCG@10. These observations further highlight the effectiveness and robustness of our proposed CWPRF models compared to ColBERTPRF across diverse query types.

heading: B Semantic Match Proportion
text: In ColBERT and other multi-representation models using MaxSim, semantic matching of token-level embeddings occurs when the surface token form of a query embedding is matched with a document embedding that has a different token. To quantify the proportion of the query embeddings performing semantic or exact matching, following Wang et al. (2022a), we report the proportion of average semantic matching occurring for all the ColBERT related models in Table 1. More formally, given a query q and the list Rk of the top-ranked k passages, the Semantic Match Proportion (SMP) at
rank cutoff k w.r.t. q and Rk is calculated as:
SMP(q,Rk) = ∑
d∈Rk
∑ i∈toks(q) 1[ti ̸=tj ]·maxj=1,...,|d| ϕTqiϕdj∑
i∈toks(q) maxj=1,...,|d| ϕ T qi ϕdj
,
(8) where ti and tj denote the token ids of the i-th query embedding and j-th passage embedding, respectively. In this work, we report the Mean-SMP values calculated at rank cutoff k = 10 in Table 1.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations and Future Work
3 A2. Did you discuss any potential risks of your work? Limitations and Future Work
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3 Contrastive Weighting for Dense PRF
B1. Did you cite the creators of artifacts you used? No response.
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Section 4
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Sections 4, 5, 6
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Appendix
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. We conduct our experiments using the MS MARCO (?) passage ranking dataset. The corpus consists of 8.8M passages from web pages, along which are provided 0.5M training queries with sparse document relevance judgements. We employ the TREC-DL 2019 (43 queries with an average of 215 relevant documents per query) query set as our validation set and use TREC 2020 (54 queries with 211 relevance assessments per query) query set as our test set due to their dense judgements, which can provide more reliable evaluations (??).
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? Sections 4, 5, 6
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Section 4 and Appendix
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4 and Appendix
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Sections 5, 6, 7 and Appendix
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4 and Appendix
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Did you describe the limitations of your work?
Limitations and Future Work
3 A2. Did you discuss any potential risks of your work? Limitations and Future Work
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank. B 3 Did you use or create scientific artifacts? Section 3 Contrastive Weighting for Dense PRF
B1. Did you cite the creators of artifacts you used? No response. 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Section 4
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Sections 4, 5, 6
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank. 3 B5. Did you provide documentation of the artifacts, e. g. , coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc. ? Appendix
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. We conduct our experiments using the MS MARCO (?) passage ranking dataset. The corpus consists of 8. 8M passages from web pages, along which are provided 0. 5M training queries with sparse document relevance judgements. We employ the TREC-DL 2019 (43 queries with an average of 215 relevant documents per query) query set as our validation set and use TREC 2020 (54 queries with 211 relevance assessments per query) query set as our test set due to their dense judgements, which can provide more reliable evaluations (??). The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance. C 3 Did you run computational experiments? Sections 4, 5, 6
3 C1. Did you report the number of parameters in the models used, the total computational budget (e. g. , GPU hours), and computing infrastructure used? Section 4 and Appendix
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4 and Appendix
3 C3. Did you report descriptive statistics about your results (e. g. , error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Sections 5, 6, 7 and Appendix
3 C4. If you used existing packages (e. g. , for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e. g. , NLTK, Spacy, ROUGE, etc. )? Section 4 and Appendix
D 7 Did you use human annotators (e. g. , crowdworkers) or research with human participants? Left blank. D1. Did you report the full text of instructions given to participants, including e. g. , screenshots, disclaimers of any risks to participants or annotators, etc. ? No response. D2. Did you report information about how you recruited (e. g. , crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e. g. , country of residence)? No response. D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response. D4