# Data Imputation Based on Retrieval-Augmented Generation
## 4 Experimental Verification
### Evaluation of Retriever Effectiveness
#### 4.2.4 Experimental Results and Analysis  
The experimental results, as shown in Table 2, demonstrate the effectiveness of the retriever, which achieves the highest recall and success rates across all datasets, except for Education. Additionally, it is noteworthy that the retriever is trained on 40,000 tables from Wikipedia and is directly applied to the three datasets mentioned above without any additional training, showcasing its robustness and generalization ability. Furthermore, the results highlight that contrastive learning is key to retrieving effective tuple encoders. The retriever and BERT share the same underlying model structure and are trained on the same table corpus. The primary distinction lies in the training tasks. However, BERT with the language modeling task lacks generalization ability and performed the worst across all datasets, indicating its unsuitability for our scenario. This contrast underscores the unique advantages and effectiveness of training the retriever using contrastive learning. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Retriever**} & \multicolumn{2}{c}{**WikiTuples**} & \multicolumn{2}{c}{**Education**} & \multicolumn{2}{c}{**Cricket Players**} \\ \cline{2-7}  & **S@1** & **S@5** & **S@1** & **S@5** & **S@1** & **S@5** \\ \hline Initial Retrieval & 0.327 \(\pm\) 0.006 & 0.279 \(\pm\) 0.005 & 0.743 \(\pm\) 0.004 & 0.901 \(\pm\) 0.003 & 0.902 \(\pm\) 0.002 & 0.043 \(\pm\) 0.001 \\ Contriever & 0.485 \(\pm\) 0.007 & 0.456 \(\pm\) 0.006 & 0.758 \(\pm\) 0.005 & 0.825 \(\pm\) 0.004 & 0.074 \(\pm\) 0.002 & 0.043 \(\pm\) 0.001 \\ BERT w/LM Task & 0.275 \(\pm\) 0.005 & 0.201 \(\pm\) 0.004 & 0 \(\pm\) 0 & 0 \(\pm\) 0 & 0 \(\pm\) 0 & 0 \(\pm\) 0 \\
**Re-ranker (ours)** & **0.951 \(\pm\) 0.003** & **0.791 \(\pm\) 0.003** & **0.992 \(\pm\) 0.002** & **0.923 \(\pm\) 0.002** & **1.000 \(\pm\) 0.000** & **0.989 \(\pm\) 0.001** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation of retriever performance (average \(\pm\) standard deviation). The best results are bolded. Each experiment was repeated three times. In this study of the retriever, leveraging contrastive learning, outperformed baselines in recall and success rates across all datasets, proving its superior effectiveness and generalization ability. The construction of the pre-training dataset is crucial for the performance of the retriever. As previously mentioned, we provide an in-depth explanation of the construction methods. To assess the validity of our training data and identify key factors in dataset construction, we detail the anchor tuples, positive tuples, and negative tuples. (1) Impact of Anchor Tuple Construction: The training data has two settings for anchor tuples: with or without missing values. Here, the construction process considers only one of these settings. We find that combining complete anchor tuples with anchor tuples that have missing values can enhance the performance of the retriever. This improvement may be due to the inclusion of missing values in the anchors, aligning with the intention of data imputation, while complete anchors help the retriever learn to identify tuples with the same subject entity as the positive examples, thereby ensuring higher recall. (2) Impact of Positive Tuple Construction: Unlike traditional methods that only consider enhanced anchors as positive tuples, we regard tuples that satisfy certain conditions from other tables as positive tuples. To demonstrate the effectiveness of these added positive tuples, we construct a new dataset by removing these positive tuples and observe a significant drop in the retriever's performance across all datasets. This is because the target tuples in real-world scenarios may come from entirely different tables than the anchors, highlighting the importance of diverse positive samples. (3) Impact of Negative Tuple Construction: We consider other tuples from the same table that the anchor is related to in the negative context.