## 4 Experiments
### Main Results  
The experimental results are presented in Table 5. The table contains two main contents: base results for comparison (Kaggle TOEIC, Random and Brute) and results of the proposed methods (POS-tagging, Lemmatizing, Mixed). The former includes training results on Kaggle TOEIC data and randomly and brutely augmented data, wherein wrong answer options were randomly constructed for all word segments of the entire data. The latter are the results for POS-tagging based, lemmatizing based, and mixed (merged POS-tagging based and lemmatizing based data in a 5:5 ratio) data.  
As indicated by the results, the proposed POS-tagging and lemmatizing based data positively affected the performance improvement. It can be speculated that the model trained on the POS-tagging based data concentrated on the meaning and compared the options to infer the correct answer based on the intention of the TOEIC question because the POS-tagging based data induced the model to better focus on the meaning corresponding to  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Data**} & \multicolumn{2}{c}{**BERT**} & \multicolumn{2}{c}{**RoBERTa**} & \multicolumn{2}{c}{**ELECTRA**} \\ \cline{2-7}  & **Only Aug** & **With FT** & **Only Aug** & **With FT** & **Only Aug** & **With FT** \\ \hline Kaggle TOEIC & & 86.22\% & & 96.14\% & & 90.63\% \\ \hline Random \& Brute & 84.02\% & 92.01\% & 93.11\% & 96.41\% & 94.21\% & 97.24\% \\ \hline POS-tagging & 90.08\% & 94.21\% & 95.59\% & 96.96\% & 96.14\% & 97.52\% \\ \hline Lemmatizing & 90.63\% & 94.21\% & 95.04\% & 97.24\% & 96.96\% & 98.07\% \\ \hline Mixed & 91.46\% & 95.04\% & 96.41\% & 97.52\% & 97.24\% & 98.07\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: Experimental results on Kaggle TOEIC data, Randomly and Brutely Augmented Data, POS-tagging Based Data, Lemmatizing Based Data, and Mixed Data.  
the blank in the sentence. In case of the lemmatizing based data, the model can focus better on the grammatical relationship within the sentence, and it appropriately concentrated on the grammar in the blank when solving the TOEIC problem. In addition, the Mixed data incorporating the POS-tagging based and lemmatizing based data enabled targeting both question types. This allowed the model to better understand the intent of the problem, which raised the performance. The results indicate that the performance of TOEIC problem-solving model can be improved through POS-tagging based and lemmatizing based data augmentation methods in the data sparsity setting.  
Furthermore, additional fine-tuning on Kaggle TOEIC data was performed in succession to the model trained with augmented data to examine the performance variation. The results are presented in the \"With FT\" column. It can be observed that the accuracy of first fine-tuning the augmented data and then continuing to learn with Kaggle TOEIC data outperformed that of training with Kaggle TOEIC data, thereby mitigating the data sparsity issue. In particular, the best performance was achieved when learning with mixed data composed with POS-tagging based and lemmatizing based data. It was inferred that applying both of the methods proposed in this paper contributed significantly to the model's understanding of semantic/grammatical relationships in sentences.