## 4 Description of the Experiment
### Datasets
#### 4.3.2 Training Sample Size Exploration  
For the experiments in this paper, the training set parts from the five datasets ATEC, BQ, LCQMC, PAWS-X, and Chinese STS-B were used for fairness in comparison, and only sentences were used as the training corpus; no labels were involved. The data in the five datasets were disrupted and integrated, and the duplicate sentences were removed to obtain a corpus of size 438,650. This corpus will be used as the training set for this paper. The trained model is evaluated by calculating the Spearman correlation coefficient on the open test set in each dataset. The experiments are trained using the whole corpus as the training set, and the SimCSE approach is used as the base method for tuning the reference to \"bert-base-chinese\", \"chinese-roberta-wwm-ext [20]\", \"chinese-roberta-wwm-ext-large [21]\" as the base models. For convenience, the subsequent content in this paper will refer to the three models mentioned above as \"bert-base\", \"roberta-base\", and \"roberta-large\", respectively. The same goes for tables and images. In order not to damage the performance of the model itself, the learning rate range was set to 1 \(\times\) 10\({}^{-6}\)-5 \(\times\) 10\({}^{-5}\), the batch size range was set to 8\(\sim\)128, and the Dropout range was 0.1-0.8. Hyperparameter search was performed using grid search, and the best hyperparameter settings were obtained based on performance on the test sets of the five datasets, as evaluated in Table 4.  
The base in the table corresponds to the \"bert-base\" and \"roberta-base\" models, and the large corresponds to the \"roberta-large\" model. It was found that the loss value of the model had become zero after a certain number of samples were trained in one epoch, so we continued to explore the appropriate number of training samples. Due to the large randomness of sample extraction, the samples in each dataset may not be drawn evenly, so during the experiments, the best performance was taken as the result of each group of experiments after ten trials, and this criterion was followed in subsequent experiments. Table 5 shows the results of the training sample part of the search for bert-base. It can be seen that as the number of training samples decreases from the full data set to 10,000 data points, the performance on the test set generally shows an increase, and below 10,000 it shows a decrease. Table 6 shows the results of the roberta-base training sample exploration experiments. It can be seen that the model performance of the training sample generally decreases from using the full amount of data to using 10% of the full amount of data, and the model performance is more volatile when the number of samples is taken from 30,000 to 10,000, and the best performance is achieved at 10,000; it still shows volatility when 8000 to 3000, and the model still has good performance when 3000 samples are taken for training. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Learning** & \multirow{2}{*}{**Batch Size**} & **Drop Out** & **Weight** & **Learning Rate** \\  & **Rate** & & **Rate** & **Decay** & **Decay Strategy** \\ \hline base & \(3\times 10^{-5}\) & 32 & 0.1 & 0.01 & Cosine decay \\ large & \(5\times 10^{-6}\) & 16 & 0.1 & 0.01 & Cosine decay \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Training hyperparameter settings. As can be seen from Table 7, the performance of the roberta-large model varies with sample size in a similar way to the first two models, with the difference that the model performs worse than the first two models if the sample size is less than 10,000. The model still performs best at a training sample size of 10,000. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Sample Size** & **ATEC** & **BQ** & **LCQMC** & **PAWS-X** & **STS-B** & **AVG** \\ \hline  
\hline \hline \end{tabular}
\end{table} 
Table 6: Roberta-base training sample size exploration.