## 5 Experimental Procedure
### Fine-Tuning of Discriminators For Priori Datasets  
In this paper, we employed BERTlarge, BERTbase, RoBERT_large_, or RoBERT_base_ as the discriminator model. Since these models are pretrained, they need to be fine-tuned to achieve optimal results in downstream tasks. During the fine-tuning process, it is recommended to use a lower learning rate and a smaller number of epochs to update the model. This is because using large learning rates and epochs may cause the model to fail to converge or overfit, which can negatively impact the model's performance in this task.  
**Results** As illustrated in Figure 3, we observed that the large model performed better on the CoLA task, with RoBERTa exhibiting the lowest loss rate. During the pretraining process, we set the maximum length of the tokenizer to 45, enabled padding, and used a minibatch size of 512. We fine-tuned the model using several of the most commonly used parameter settings. As summarized in Table 2, we achieved the best results with a learning rate of \(3\times 10^{-5}\) and 10 epochs. Following the fine-tuning process, the RoBERTa\({}_{large}\) model demonstrated the ability to make judgments about grammatical plausibility.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Operation** & **Text** \\ \hline \multirow{7}{*}{Original Text} & our friends won’t buy this analysis, let alone the next one we propose. \\  & one more pseudo generalization and i’m giving up. \\  & i ’ll fix you a drink. \\  & we’re dancing the night away. \\  & My little \#ObessedWithMyDog@CafeSolstice Capitol Hill \\  & More \#tinyepic things \#tinyepicwestern, this one is crazy @user. I may be one of your... \\  & Last night @ Omnia. Night Club At Caesars Palace \\  & friendship at its finest....\#pixar \#toystry \#buzz \#woody \#friends \#friendship \#bff... \\  & I L VE working for a cause! Yesterday’s balloon decor for SNN 11 th Annual Back 2 School Health. \\ \hline \multirow{7}{*}{After preprocessing} & our friends will not buy this analysis let alone the next one we propose \\  & one more pseudo generalization and iam giving up \\  & i will fix you a drink \\  & we are dancing the night away \\  & My little ObsessedWithMyDog\_CafeSolstice Capitol Hill \\  & More tinyepic things \#tinyepicwestern, this one is crazy \#friends \#friendship \#bff... \\  & Last night \# Omnia. Night Club At Caesars Palace \\  & friendship at its finest \#pixar toystory buzz woody friends friendship bff \\  & I L VE working for a cause Yesterdays balloon decor for SNN 11 th Annual Back 2 School Health. \\ \hline \hline \end{tabular}
\end{table}
Table 1: Example of data preprocessing for the training set in downstream tasks.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & \multicolumn{3}{c}{**Learning Rate**} \\ \cline{3-5}  & & \(\mathbf{2\times 10^{-5}}\) & \(\mathbf{3\times 10^{-5}}\) & \(\mathbf{4\times 10^{-5}}\) \\ \hline \multirow{3}{*}{epoch} & 5 & 0.378204 & 0.392302 & 0.425639 \\  & 10 & 0.360829 & **0.359063** & 0.370926 \\ \cline{1-1}  & 15 & 0.363742 & 0.362149 & 0.360642 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Setting different parameters allows the RoBERTa\({}_{large}\) model to fine-tune the loss rate of the validation set in the CoLA task.