## 5 An Efficient English Embedding Model  
In this section, we conduct experiments to find the best model for conversational sentence embeddings in English. Our approach follows the same experimental setup as described in Section 4, but with English datasets instead of Korean datasets. Additionally, to compare our approach with existing methods, we use all of the selected English models  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Korean SBERT Models**} & \multicolumn{5}{c}{**Accuracy (\%)**} \\ \cline{2-6}  & **GPT-ko** & **Paraph-ko** & **GPT-en** & **Paraph-en** & **Average** \\ \hline snunlp/KR-SBERT-V40K-klueNLI-augSTS & 65.65 & 79.40 & 30.50 & 45.90 & 55.36 \\ jhgan/ko-sroberta-multitask & 80.95 & 86.40 & 53.40 & 68.00 & 72.19 \\ Hufton/sentence-klue-roberta-base & 85.50 & 89.00 & 55.25 & 70.95 & 75.18 \\ \hline (KLUE-RoBERTa-base) KLUE & 87.00 & 89.05 & 54.10 & 69.90 & 75.01 \\ (KLUE-RoBERTa-base) KorSTS & 80.55 & 84.85 & 57.25 & 69.55 & 73.05 \\ (KLUE-RoBERTa-base) KSTS & 84.05 & 85.95 & 60.00 & 69.55 & 74.89 \\ (KLUE-RoBERTa-base) KSICKR & 63.60 & 76.05 & 46.90 & 66.10 & 63.16 \\ (KLUE-RoBERTa-base) KLUE+KorSTS & 88.10 & 88.80 & 55.95 & 71.05 & 75.98 \\ (KLUE-RoBERTa-base) KLUE+KSTS & 88.30 & **89.85** & 61.30 & 72.05 & 77.88 \\ (KLUE-RoBERTa-base) KLUE+KSICKR & 87.60 & 89.65 & 58.20 & 72.35 & 76.95 \\ (KLUE-RoBERTa-base) KLUE+KSTS+KorSTS & **88.35** & 89.40 & **62.40** & **73.50** & **78.41** \\ (KLUE-RoBERTa-base) KLUE+KSTS+KorSTS+KSICKR & 87.40 & 89.15 & 60.75 & 69.40 & 76.68 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of existing Korean sentence embedding models (the first three models) and our embedding models trained with different datasets.  
available on sbert.net ([[https://sbert.net/docs/pretrained_models.html](https://sbert.net/docs/pretrained_models.html)]([https://sbert.net/docs/pretrained_models.html](https://sbert.net/docs/pretrained_models.html)), accessed on 2 May 2023). The experimental results are presented in Table 5.  
In these results, one of our approaches (fine-tuning on STSb and STS English datasets) achieves the best average accuracy. An interesting thing is that the Korean pre-trained model \"KLUE-RoBERTa-base\" performed well for English sentence embedding. Our hypothesis is that, during its pre-training phase, the Korean datasets used to train the model may have contained some English expressions. This could have enabled the pre-trained model to also learn how to process English expressions effectively. However, although it is helpful for our approaches to train on English training datasets, state-of-the-art SBERT models still significantly outperform our approaches in the English test datasets. Surprisingly, the best performing English models are very small models: all-MiniLM-L6-v2 performs the best in the GPT-en dataset, and all-MiniLM-L12-v2 performs the best in the Paraph-en dataset and on average in the English test datasets.  
Our experiments showed that, although the English models achieved exceptional performance on the English datasets, they performed poorly on the Korean datasets. This suggests that using two separate monolingual models, one for English and one for Korean, in collaboration may lead to better results. In the next section, we will explore this possibility further.