## 6 A Hybrid Approach  
In the previous sections, we conducted an analysis to identify the best-performing embedding models for each language in our conversational datasets. Based on our findings, we aim to answer the following research questions:  
* Can we achieve better accuracy in our test datasets by using our Korean and English sentence embedding models together, compared with using state-of-the-art multilingual models? * If so, can we achieve both higher accuracy and faster processing times in our test datasets by using even smaller versions of the two monolingual models in combination, compared with using state-of-the-art multilingual models? In order to answer these questions, we have configured our hybrid approach as follows:  
* We utilized the KLUE-RoBERTa-small pre-trained model for developing a Korean monolingual model. It is worth noting that this model is smaller than the KLUE \begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**English SBERT Models**} & \multicolumn{5}{c}{**Accuracy (\%)**} \\ \cline{2-6}  & **GPT-ko** & **Paraph-ko** & **GPT-en** & **Paraph-en** & **Average** \\ \hline paraphrase-albert-small-v2 & 1.30 & 0.25 & 78.60 & 72.55 & 38.18 \\
**all-MiniLM-L6-v2** & 12.60 & 17.20 & **81.25** & 77.95 & 47.25 \\ paraphrase-MiniLM-L3-v2 & 14.35 & 21.85 & 77.65 & 76.70 & 47.64 \\ multi-qa-mpnet-base-dot-v1 & 16.10 & 23.30 & 80.00 & 72.95 & 48.09 \\ all-mpnet-base-v2 & 16.50 & 21.25 & 80.45 & 74.40 & 48.15 \\
**all-MiniLM-L12-v2** & 15.05 & 25.05 & 80.55 & **79.30** & 49.99 \\ multi-qa-MiniLM-L6-cos-v1 & 18.60 & 25.40 & 80.70 & 76.10 & 50.20 \\ multi-qa-distilbert-cos-v1 & 17.45 & 26.80 & 80.80 & 76.35 & 50.35 \\ all-distilroberta-v1 & 18.60 & 30.50 & 76.60 & 75.85 & 50.39 \\ \hline (KLUE-RoBERTa-base) STSb & 74.45 & 83.65 & 65.45 & 69.95 & 73.38 \\ (KLUE-RoBERTa-base) STS & 82.60 & 85.30 & 66.65 & 69.65 & 76.05 \\ (KLUE-RoBERTa-base) SICKR & 52.15 & 74.25 & 47.60 & 58.90 & 58.23 \\ (**KLUE-RoBERTa-base) STSb+STS** & **83.30** & **85.90** & 69.15 & 71.40 & **77.44** \\ (KLUE-RoBERTa-base) STSb+STS+SICKR & 81.50 & 85.15 & 66.05 & 70.50 & 75.80 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of existing English sentence embedding models (the first nine models) and our embedding models trained with different datasets. RoBERTa-base utilized in Section 4. Hence, it may achieve lower accuracy but perform faster. To fine-tune our model, we employed the KLUE+KSTS datasets, which demonstrated the best performance on Korean test datasets, as introduced in Section 4. * To speed up the inference time, we applied PCA (Principal Component Analysis) to reduce the dimensionality of the resulting vectors. We followed a similar approach used in sbert.net: we randomly extracted 20,000 sentences from the KLUE-NLI dataset (which is not relevant to our test datasets) to generate sample embeddings for PCA. The original dimensionality of the Korean model was 768, but our resulting vectors, applied after PCA, had a dimensionality of 384. We added the \"PCA layer\" to our model, and the PCA process ran very quickly. * For the English monolingual model, we used the all-MiniLM-L12-v2 model, which was the best performing model on average in the English test sets. Because this model was already a very small model, we did not need to make the model smaller. Additionally, the output of the model is also small (384-dimensional vectors), so we did not apply PCA to reduce the dimensionality. * For combining these two monolingual models, we used a very simple hybrid approach: if a query sentence has at least one Korean letter, then the Korean model makes the corresponding 384-dimensional vector. Otherwise, the English model makes the corresponding 384-dimensional vector. Table 6 displays the experimental results of existing multilingual models and our approaches. The comparators include two types of models.