## 6 Conclusions and Future Work  
In this article, we introduced EvoText, a training process for two pretrained models aimed at addressing the challenges of insufficient sample data and computational resources, allowing models to continue learning after deployment. Through fine-tuning discriminators and prewarm-up training generators, we achieved better model performance with just 156 training steps, significantly improving performance without requiring additional training data. This approach steadily improves the performance of natural language understanding and generation tasks without changing the model structure, with the potential for even greater performance improvements over time. EvoText is an effective and scalable training model that holds great promise for low-resource NLP tasks. Our extensive \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**LAMBADA**} & \multicolumn{2}{c}{**CBT**} & \multicolumn{2}{c}{**WikiText**} & \multicolumn{2}{c}{**PTB**} & \multicolumn{1}{c}{**enwiki8**} & \multicolumn{1}{c}{**texts**} & \multicolumn{1}{c}{**1 BW**} \\ \cline{2-11}  & **PPL \(\downarrow\)** & **ACC \(\uparrow\)** & **ACC \(\uparrow\)** & **(CN)** & **ACC \(\uparrow\)** & **(NE)** & **PPL \(\downarrow\)** & **(WikiText103)** & **PPL \(\downarrow\)** & **BPC \(\downarrow\)** & **BPC \(\downarrow\)** & **PPL \(\downarrow\)** \\ \hline
**Init** & & & & & & & & & & \\ Full module EvoText & **14.21** & **57.20** & **92.85** & **87.0** & **21.94** & **25.12** & **45.93** & **1.00** & **1.06** & **55.01** \\ \hline
**Ablation** & & & & & & & & & \\
**Experiments** & 14.80 & 55.94 & 92.95 & 86.53 & 22.46 & 27.54 & 46.99 & 1.01 & 1.06 & 55.72 \\ Remove D pretraining & 14.80 & 55.94 & 92.95 & 86.93 & 22.12 & 25.54 & 45.99 & 1.00 & 1.06 & 55.11 \\ Remove G & 14.34 & 57.03 & 92.90 & 86.93 & 22.12 & 25.54 & 45.99 & 1.00 & 1.06 & 55.11 \\ Remove supervised fine-tuning & 15.12 & 55.62 & 92.89 & 87.23 & 22.70 & 27.64 & 47.90 & 1.01 & 1.06 & 56.11 \\ Remove & & & & & & & & & \\ semi-supervised fine-tuning & & & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 10: We evaluated the impact of removing a particular module of EvoText GPT-2\({}_{media}\) on ZERO-SHOT performance in seven tasks. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **PPL\(\downarrow\)** (**arXiv**)** & **PPL\(\downarrow\)** (**LAMBADA**) (**zero-SHOT**)** \\ \hline
**Baseline** & & & \\ GPT-2\({}_{unif}\) & 37.14 (ZERO-SHOT) & 35.13 \\ OPT-1\({}_{251\text{M}}\) & 36.80 (ZERO-SHOT) & 32.93 \\ \hline
**Traditional** & & & \\ GPT-2\({}_{unif}\) & 20.42 & **28.10** \\ OPT-1\({}_{251\text{M}}\) & 20.13 & 28.01 \\ \hline
**Ours** & & & \\ EvoText GPT-2\({}_{unif}\) & **19.81** & 28.12 \\ EvoText GPT-1\({}_{251\text{M}}\) & **18.94** & **28.00** \\ \hline \hline \end{tabular}
\end{table} 
Table 9: We evaluated the performance of both models on the arXiv dataset by retraining the discriminator at a learning rate of \(1\times 10^{-4}\) and the generator at a learning rate of \(5\times 10^{-5}\). experiments demonstrate the potential for improving pretrained model performance and highlight the importance of supervised learning. Future research directions may include exploring the potential for EvoText in other NLP tasks and applications, investigating the impact of different discriminator and generator architectures on model performance, and further exploring the potential for continued learning after deployment in other settings. Additionally, our study highlights the importance of supervised learning in NLP and suggests that future research should continue to focus on developing effective training processes for pretrained models in low-resource settings. Conceptualization, methodology, formal analysis, and software, Z.Y. ; verification, Z.Y., C.Z. and H.X. ; writing--original draft preparation, Z.Y., C.Z. and H.X. ; writing--review and editing, Y.L. ; visualization, C.Z. and H.X. ; supervision, Y.L. ; funding acquisition, Y.L.