## 3 Dataset Preparation
### Test Data  
We propose two types of test datasets to evaluate the effectiveness of Korean and English SBERT models: one made by AI and another made by humans. Our conversational dataset, called \"GPT-ko\", was generated by ChatGPT (using GPT-3.5 and GPT-4). The other dataset, called \"Paraph-ko\", consists of 10,000 conversational sentences extracted from ParaKQC, which was generated by humans. Although ParaKQC is also used for KLUE-STS, \"Paraph-en\" and KLUE-STS do not have common sentence pairs. \"GPT-en\" and \"Paraph-en\" are the English versions of \"GPT-ko\" and \"Paraph-ko\", respectively, translated by the gpt-3.5-turbo model. Table 2 summarizes the test data used in our experiments. To construct the GPT-ko dataset, we followed the algorithm below:  
1. We asked ChatGPT to recommend appropriate topics for generating datasets. Based on its response, we selected 10 topics (Culture, Travel, Science, Sports, Education, Food, Health, Technology, History, Humanities) and 38 subtopics. 2. We manually selected 34 subtopics from the 38, eliminating the very similar ones. 3. For each of the remaining 34 subtopics, we asked ChatGPT to randomly generate conversational sentences in Korean. 4. For each conversational sentence, we asked ChatGPT to generate one paraphrase sentence in Korean that differed in syntax as much as possible. 5. Although the generated sentences were mostly of high quality, for some subtopics, ChatGPT did not generate enough diverse sentences. In these cases, we tried switching from GPT version 3.5 to 4.0. However, generating diverse sentences was still difficult for some subtopics. For those subtopics, we stopped generating and moved on to the next ones. We have created the GPT-ko dataset based on this algorithm, which includes 2000 conversational sentences generated in Korean across 34 subtopics. Furthermore, we have used GPT-3.5-turbo to translate the GPT-ko dataset into English, resulting in the GPT-en dataset. Table 3 displays the statistics and examples of the generated sentences. Due to limitations in table space, we present only one example sentence per topic in this paper. However, the complete dataset is available on our GitHub page ([[https://github.com/too-eeworld/multiple-monolingual-model](https://github.com/too-eeworld/multiple-monolingual-model)]([https://github.com/too-eeworld/multiple-monolingual-model](https://github.com/too-eeworld/multiple-monolingual-model)), accessed on 2 May 2023). Unlike the GPT-ko and GPT-en datasets, the Paraph-ko and Paraph-en datasets are generated by humans. To create these datasets, we extracted 2000 sentences from the existing ParaKQC paraphrase dataset, which contains 10,000 sentences organized into 1000 groups, with each group consisting of 10 semantically identical sentences. We randomly selected two sentences from each group to obtain the 2000 sentences in the Paraph-ko dataset. The Paraph-en dataset is the English-translated version of the Paraph-ko dataset, which was translated using gpt-3.5-turbo. \begin{table}
\begin{tabular}{c c c c} \hline
**Language** & **Dataset Name** & **\# of Sentences** & **Description** \\ \hline \multirow{2}{*}{Korean} & GPT-ko & 2000 & Conversational sentences (across 34 topics) generated by ChatGPT. \\  & Paraph-ko & 2000 & Sentences extracted from an existing paraphrase dataset. \\ \hline \multirow{2}{*}{English} & GPT-en & 2000 & English translations of the GPT-ko dataset, generated using the gpt-3.5-turbo. \\  & Paraph-en & 2000 & English translations of the Paraph-ko, generated using the gpt-3.5-turbo. \\ \hline \end{tabular}
\end{table} 
Table 2: Summary of the test data used in our experiments. Note that in all of the generated test sets (GPT-ko, GPT-en, Paraph-ko, and Paraph-en), there are 2000 conversational sentences. Each sentence has exactly one semantically identical sentence (paraphrased sentence) within the dataset. When evaluating the models, each model found the most semantically similar sentence for each query sentence within the dataset, and we counted the number of times it accurately finds it.