# Towards Robust Neural Rankers with Large Language Model: A Contrastive Training Approach
## 3 Methodology
### Main Framework
#### 3.2.2 Ranking Training  
In this section, we briefly introduce two primary structures of PLM-based ranking models: Dense Retriever and Cross Encoder (Reranker), as well as exploring how to design loss functions for the ranking task. Cross Encoder has demonstrated superior retrieval performance compared to the Dense Retriever, although it falls short in terms of computational efficiency. Therefore, we conducted experiments on both architectures to demonstrate the versatility of our approach. **Dense Retrievers**[17; 48; 49] would employ a uniform BERT encoder to obtain the representations of both queries and documents:  
\[H(q_{i})=Encoder([CLS]\circ q_{i}\circ[SEP]), \tag{2}\]  
\[H(d_{i})=Encoder([CLS]\circ d_{i}\circ[SEP]). \tag{3}\]  
In Equations (2) and (3), the data preprocessing adheres to the input rules of BERT, where \([CLS]\) and \([SEP]\) are special tokens, and \(\circ\) represents the concatenation operation. Generally, we utilize the same encoder (in this study, we use BERT) to obtain the representations of the query and its candidate documents. This dense vector representation will be used to calculate the similarity between the query and the documents in the candidate set, denoted by \(Score(q,d)\):  
\[Score_{<q_{i}d_{i}>}=sim(H(q_{i}),H(d_{i})), \tag{4}\] \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Character Level**} & \multicolumn{2}{c}{**Word-Level**} & \multicolumn{2}{c}{**Sentence-Level**} & \multicolumn{2}{c}{**LLM**} \\  & **WikiQA** & **ANTIQUE** & **WikiQA** & **ANTIQUE** & **WikiQA** & **ANTIQUE** & **WikiQA** & **ANTIQUE** \\ \hline bge-large-en [44] & 96.04\% & 94.88\% & 93.22\% & 91.44\% & 95.99\% & 96.73\% & 92.02\% & 95.08\% \\ bge-base-en [54] & 93.75\% & 94.36\% & 92.28\% & 97.50\% & 96.88\% & 95.29\% & 94.64\% \\ instructor\_2 [46] & 91.94\% & 93.51\% & 90.57\% & 87.49\% & 95.84\% & 96.03\% & 92.26\% & 92.25\% \\ instructor\_large [46] & 96.19\% & 96.82\% & 95.75\% & 94.17\% & 98.04\% & 97.58\% & 96.81\% & 96.93\% \\ gtf-65-all [47] & 89.09\% & 90.78\% & 86.60\% & 81.45\% & 93.82\% & 92.21\% & 89.20\% & 88.57\% \\ gtf-65-all [47] & 89.89\% & 91.14\% & 87.14\% & 81.96\% & 93.97\% & 92.36\% & 88.89\% & 88.43\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Semantic similarity between original query and query variations. Cosine similarity is adopted to compute the semantic similarity between queries. The higher the values, the more similar the two sentences are. The previous methods are categorized based on three different levels of perturbations: character, word, and sentence. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Sentence Length**} & \multicolumn{2}{c}{**Jaccard Similarity**} & \multicolumn{2}{c}{**Levenshtein Distance**} \\  & **WikiQA** & **ANTIQUE** & **WikiQA** & **ANTIQUE** & **WikiQA** & **ANTIQUE** \\ \hline Original query & 36.89 & 47.59 & - & - & - & - \\ Character-level query variation & 36.44 & 44.76 & 71.80\% & 72.68\% & 1.85 & 4.42 \\ Word-level query variation & 21.00 & 23.45 & 43.45\% & 34.05\% & 18.31 & 26.21 \\ Sentence-level query variation & 36.27 & 43.12 & 56.71\% & 63.54\% & 9.07 & 11.97 \\ LLM query variation & 58.48 & 68.66 & **17.72**\% & **21.08**\% & **34.56** & **40.07** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation of the diversity of query variations in comparison to the original query. In terms of Jaccard Similarity, smaller values suggest a greater degree of deviation from the original query. Conversely, for Levenshtein Distance, larger values indicate greater diversity. The previous methods are categorized based on three different levels of perturbations: character, word, and sentence. The best result is **bold-faced**. \(Score(q,d)\) measures the similarity between the query and all documents within its candidate set. In Dense Retriever, the similarity function typically employs the dot product [48,49]. **Cross Encoder (Reranker)** processes both the query and document jointly in order to capture the interactive information within the text.