# Towards Robust Neural Rankers with Large Language Model: A Contrastive Training Approach
## 5 Evaluation Result
### Case Study  
To verify the actual improvement in robustness of the models, we used real data from the ANTIQUE dataset as the subject for analysis. Following the idea shown in Table 1, we selected existing adversarial texts and compared the performance differences of PLM-based IR models trained using different training methods. The results presented in Table 8 showed that users probably make spelling errors during the actual retrieval process, including omissions of some letters and swapping letter positions. Based on the original retrieval metrics, it could be observed that these potential adversarial attacks did pose challenges to the retrieval performance of IR models, especially when the target of the attack was the keyword in the query. Traditional models based on word-matching would be highly likely to fail in matching, while PLM-based IR models may also experience a decline in retrieval performance due to the reduced representation performance after being attacked. For example, in the first case in the table with the misspelling \"_rabbit_\", the MAP and MRR values of the original model were very low, and the relevant document was ranked seventh in terms of MRR. However, after being trained with our proposed robustness framework, the relevant document was ranked third. Of course, it was also possible for common spelling errors to occur simultaneously in both documents and queries, such as in the second case, where \"potatoes\" was often misspelled as \"_potatos_\". Although the MRR of the PLM-based IR model was equal to 1 under both training methods, our proposed approach could also improve the other MAP indicator. This suggested that our proposed method could learn the semantic representation of queries during training and thus improved the ranking positions of other marginally relevant documents in the retrieval process. Overall, the robustness training method proposed in this study enhanced the robustness of pre-trained models. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model Type** & **Method** & **MAP** & **MRR** & **nDCG@10** & **P@10** \\ \hline \multirow{4}{*}{DR} & Original & 0.781(4.1\%/12.3\%) & 0.793(4.1\%/11.8\%) & 0.833(2.9\%/8.6\%) & 0.115(**0.0\%/0.0\%**) \\  & Typo-aware & 0.779(**2.8\%/4.6\%**) & 0.791(**2.7\%/4.1\%**) & 0.832(**1.9\%/3.0\%**) & 0.116(0.0\%/0.4\%) \\  & Dual contrastive & **0.789**(3.9\%/7.9\%) & **0.802**(3.9\%/6.7\%) & **0.843**(2.8\%/5.5\%) & **0.117**(0.3\%/0.7\%) \\  & Our & 0.785(3.6\%/5.9\%) & 0.801(3.7\%/5.4\%) & 0.841(2.7\%/4.0\%) & **0.117**(0.1\%/0.3\%) \\ \hline \multirow{4}{*}{Reranker} & Original & 0.830(7.8\%/12.2\%) & 0.848(8.3\%/12.3\%) & 0.873(6.2\%/9.7\%) & 0.117(1.7\%/3.6\%) \\  & Typo-aware & 0.819(4.9\%/11.5\%) & 0.832(4.9\%/10.6\%) & 0.866(3.7\%/8.3\%) & 0.117(0.4\%/1.1\%) \\ \cline{1-1}  & Our & **0.845**(**3.7\%/7.2\%**) & **0.862**(**3.6\%/6.7\%**) & **0.888**(**2.7\%/5.0\%**) & **0.119**(**0.1\%/0.7\%**) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of different robustness training approaches on WikiQA. The best accuracy is **bold-faced**, and the second-best accuracy is underlined.