# Data Imputation Based on Retrieval-Augmented Generation
## 4 Experimental Verification
### Evaluation of the Overall Experimental Effect of Data Imputation
#### 4.1.4. Experimental Results and Analysis  
For the settings involving retrieval methods, the top five retrieved results are fed into the LLMs, along with the incomplete tuple that needs imputation. Table 1 shows the experimental results. The results indicate that our method significantly improves the accuracy of data imputation compared to using the LLMs alone across all datasets. GPT-3.5 shows an average improvement of 41%, while GPT-4.0 shows an average improvement of 33.8%. Directly applying the LLMs resulted in suboptimal outcomes because the knowledge stored in the LLMs' parameters is not sufficiently accurate, introducing a degree of uncertainty. Thus, even for datasets in the Wikipedia domain included in the LLMs' training data, the imputation accuracy is low. In more specific and proprietary fields, such as business and education, both GPT-3.5 and GPT-4 struggle to estimate missing values accurately. The accuracy of GPT-3.5 is similarly very low, while GPT-4's accuracy is slightly higher but still limited. In contrast, our method demonstrates exceptional performance due to the effective integration of the LLMs' advanced reasoning capabilities with the rich knowledge from accurately retrieved tuples from the data lake. Furthermore, the results emphasize the importance of effective retrieval methods within the retrieval-augmented imputation framework. BM25 exhibits poorer results than our method on most datasets, except for the business dataset, where there is a large lexical overlap between the incomplete tuples and the target tuples, allowing BM25 to perform well. Therefore, the effectiveness of the LLMs combined with BM25 is almost always inferior to that of our method. It is also evident that using suboptimal retrievers like BM25 adversely affects the data imputation accuracy of less advanced models like GPT-3.5. This is primarily because inaccurately retrieved tuples can mislead the LLMs, especially when they lacks strong reasoning capabilities, impairing predictive performance. Additionally, in most cases, GPT-4 shows better performance than GPT-3.5 across all datasets, except for the Cricket Players dataset without retrieved tuples. This anomaly arises because both models lack sufficient domain-specific knowledge to accurately estimate missing values in these cases. In such instances, GPT-3.5 tends to guess answers based on the content of the complete tuples. However, without additional contextual information, GPT-4 often does not provide an answer, resulting in empty responses. Nevertheless, GPT-4's advanced reasoning capabilities generally lead to better performance, particularly in identifying target tuples and inferring missing values. Notably, when implementing \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Reasoner** & **Retrieval Modules** & **WikiTuples** & **Education** & **Cricket Players** \\ \hline \multirow{3}{*}{GPT-3.5} & w/o & 0.715 \(\pm\) 0.008 & 0.017 \(\pm\) 0.002 & 0.896 \(\pm\) 0.005 \\  & w/tuples (BM25) & 0.577 \(\pm\) 0.007 & 0.892 \(\pm\) 0.004 & 0.889 \(\pm\) 0.006 \\  & w/tuples (ours) & 0.886 \(\pm\) 0.005 & 0.976 \(\pm\) 0.003 & 0.964 \(\pm\) 0.004 \\ \hline \multirow{3}{*}{GPT-4.0} & w/o & 0.752 \(\pm\) 0.009 & 0.597 \(\pm\) 0.006 & 0.863 \(\pm\) 0.005 \\  & w/tuples (BM25) & 0.800 \(\pm\) 0.006 & 0.925 \(\pm\) 0.004 & 0.909 \(\pm\) 0.004 \\ \cline{1-1}  & w/tuples (ours) & **0.902 \(\pm\) 0.004** & **0.979 \(\pm\) 0.002** & **0.972 \(\pm\) 0.003** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Evaluation of data imputation performance. The best results are bolded. We ran each experiment three times and report the average \(\pm\) standard deviation. retrieval-augmented imputation, the performance gap between GPT-3.5 and GPT-4 narrows. This suggests that retrieval-augmented methods can effectively mitigate the limitations of less advanced language models in data imputation.