# Contrastive Refinement for Dense Retrieval Inference in the Open-Domain Question Answering Task
## 2 Materials and Methods
### Constructing Pseudo-Labels
By building an index, the FAISS library greatly reduces the search time. Specifically, given a query text q, we use its dense representation \(q\) to retrieve the top-k passage dense representations closest to it in the FAISS index. Finally, we rank the retrieval results obtained through FAISS in descending order of similarity scores to obtain the top-k passages most relevant to each query. \[top-k_{p\in P}(query)=sort(s(query,context))[1:k] \tag{4}\]  
Here, we introduce a PRF algorithm, which evolves from the relevance feedback (RF) algorithm. The RF algorithm is a classic and common algorithm in information retrieval task. It interacts with users to obtain feedback information on search results and uses this feedback information as auxiliary data to enhance the performance of the retrieval algorithm. The general step of the RF algorithm is to improve the query representation through some updating functions:  
\[q_{t+1}=f(q_{t}) \tag{5}\]  
\(q_{t}\) denotes the query representation after \(t\) updates; \(f\) denotes the update function. [PERSON]'s algorithm is a classic RF algorithm, aiming to find the optimal query representation which maximizes the similarity of relevant query-passage pairs and minimizes the similarity of the irrelevant ones:  
\[q_{opt}=\operatorname*{argmax}_{q}\big{[}sim\big{(}q,C^{+}\big{)}-sim\big{(} q,C^{-}\big{)}\big{]} \tag{6}\]  
Figure 2: Embedding Generation Flow of BERT. \(q\) denotes the previous query representation; \(C^{+}\) denotes the relevant context of \(q\); \(C^{-}\) denotes the irrelevant context of \(q\), and \(q_{opt}\) denotes the optimal query representation after algorithm's updating. However, the RF algorithm requires manual interaction and feedback, making it costly. In order to automate user interaction feedback, the PRF algorithm was proposed. A common method of PRF is to select the top \(k^{\prime}\) passages from the top \(k\) retrieval results as the relevant passage set and consider the rest as the irrelevant set. The final query representation update is calculated in a weight calculation as Equation (7):  
\[f(q_{t})=\alpha*q_{t}+\beta*\frac{1}{|C^{+}|}*\sum_{c^{+}\in C^{+}}c^{+}-\gamma* \frac{1}{|C^{-}|}*\sum_{c^{-}\in C^{-}}c^{-} \tag{7}\]  
\(C^{+}\) and \(C^{-}\) respectively denote the relevant and irrelevant context of \(q_{t}\), \(\alpha\), \(\beta\), and \(\gamma\) are hyperparameters that determine different components. Overall, the PRF algorithm is suitable for training based on large-scale unsupervised corpora, as it can easily achieve the automatic generation of feedback labels through certain settings. Therefore, we design an improved PRF algorithm with knowledge filtering based on answers, which divides the retrieval results into two contrastive labels, as shown in Equation (8). \[\left\{ \begin{aligned} & relevant,\ if\ a\ in\ passage_{i}\\ & irrelevant,\ if\ a\ not\ in\ passage_{i}\end{aligned} \right. \tag{8}\]  
\(a\) denotes the answer of the present query; \(passage_{i}\) denotes the \(i-th\) passage text in the Wikipedia dataset. Equation (8) is the feedback generation step in our improved PRF algorithm, which refers to the findings of [PERSON] et al. [7] and employs a more practical method for determining relevance relations to generate pseudo-labels. In addition, considering that the AOpt model needs to iteratively optimize text representations with more reliable and strongly correlated sample pairs, we did not introduce other non-strongly correlated negative sample strategies, such as random sampling. The element in the passage-centric pseudo-labeled data we constructed can be referred to in Table 2, including the passage ID, passage text, passage title, positive query samples of the passage (positive section), and negative query samples of the passage (negative section).