# Data Imputation Based on Retrieval-Augmented Generation
## 4 Experimental Verification
### Evaluation of Re-Ranker Effectiveness
#### 4.3.4 Experimental Results and Analysis  
To evaluate the performance of the re-ranker, this study compares it against four baseline methods, two of which innovatively use LLMs as re-rankers. Due to cost constraints, for the methods utilizing LLMs, only a subset of tuples is sampled from the test set of each dataset. Specifically, 180 tuples are selected from the Cricket Players dataset, while 200 tuples are sampled from the remaining datasets. The results for the complete datasets are reported in Table 3, while the results for the sampled subsets are shown in Table 4.  
First, the results in Table 3 indicate that our re-ranker achieves the highest _Success@5_ across various datasets compared to other re-rankers that require fine-tuning. Additionally, on all datasets, the re-ranker shows an average improvement of 24.3% in _Success@1_ and 6.2% in _Success@5_ compared to the initial results from Initial Retrieval. Only six tuples with missing values were used for training, and we observe that all re-rankers need fine-tuning to outperform the retrive  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Re-Ranker**} & \multicolumn{2}{c}{**WikiTuples**} & \multicolumn{2}{c}{**Education**} & \multicolumn{2}{c}{**Cricket Players**} \\ \cline{2-7}  & **S@1** & **S@5** & **S@1** & **S@5** & **S@1** & **S@5** \\ \hline Initial Retrieval & 0.531 \(\pm\) 0.006 & 0.803 \(\pm\) 0.004 & 0.577 \(\pm\) 0.005 & 0.923 \(\pm\) 0.003 & 0.830 \(\pm\) 0.003 & 0.989 \(\pm\) 0.001 \\ monoBERT & 0.501 \(\pm\) 0.007 & 0.799 \(\pm\) 0.005 & 0.973 \(\pm\) 0.003 & 0.984 \(\pm\) 0.002 & 0.622 \(\pm\) 0.004 & 0.931 \(\pm\) 0.002 \\ RoBERTa-LCE & 0.615 \(\pm\) 0.006 & 0.837 \(\pm\) 0.004 & 0.970 \(\pm\) 0.002 & 0.984 \(\pm\) 0.002 & 0.902 \(\pm\) 0.003 & 0.974 \(\pm\) 0.002 \\
**Re-ranker (ours)** & **0.713 \(\pm\) 0.005** & **0.927 \(\pm\) 0.003** & **0.976 \(\pm\) 0.002** & **0.986 \(\pm\) 0.001** & **0.941 \(\pm\) 0.002** & **1.000 \(\pm\) 0.000** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Evaluation of re-ranker performance (average \(\pm\) standard deviation). The best results are bolded. Each experiment was repeated three times.  
Second, Table 4 demonstrates that the re-ranker has a significant advantage over existing re-ranking methods that utilize LLMs across various datasets. The GPT-3.5 model using the Listwise method is clearly the most effective, as its _Success@5_ falls below the re-ranker's initial retrieval results across all datasets. Moreover, its _Success@1_ also lags behind the initial retrieval results on three datasets. This poor performance is primarily attributed to two factors: first, GPT-3.5's limited ability to understand tabular data and, second, the inherent challenge GPT-3.5 faces in reordering a list of tuples based on the relevance of incomplete tuples, especially when dealing with large lists. In contrast, when GPT-3.5 is combined with a pairwise method, the model selects one of two candidate tuples, resulting in significant improvements. This is because making a choice between two options is simpler than ranking an entire list. However, this method performs poorly compared to the re-ranker on most datasets. The reason is that if GPT-3.5 incorrectly ranks a target tuple low in the pairwise comparison, that tuple has almost no chance to improve its subsequent ranking.  
When sufficient training data is available, the re-ranker can outperform traditional re-ranking methods and those utilizing LLMs, demonstrating excellent performance across various datasets, particularly in enhancing success rates.