Self-Evolving GPT : A Lifelong Autonomous Experiential Learner (2024)

Abstract: To improve the performance of large language models (LLMs), researchers have explored providing LLMs with textual task-solving experience via prompts. However, they rely on manual efforts to acquire and apply such experience for each task, which is not feasible for the growing demand for LLMs and the variety of user questions.To address this issue, we design a lifelong autonomous experiential learning framework based on LLMs to explore whether LLMs can imitate human ability for learning and utilizing experience. It autonomously learns and accumulates experience through experience transfer and induction, categorizing the types of input questions to select which accumulated experience to employ for them.Experimental results on six widely used NLP datasets show that our framework performs reliably in each intermediate step and effectively improves the performance of GPT-3.5 and GPT-4. This validates the feasibility of using LLMs to mimic human experiential learning and application capabilities, offering a new path worth further exploration for the evolution of machine intelligence. Additionally, we provide a detailed analysis of the behavior of our framework at each step.We will open source codes after the acceptance, fostering open research in the NLP community and beyond.

## introduction recently, large language models (llms) like chat-gpt have achieved excellent performance in various nlp tasks @xcite . however, numerous nlp tasks still cannot be effectively addressed by them @xcite . this is mainly because they have not accumulated enough experience to handle these tasks during their training. to address these issues, previous studies have explored injecting task-solving experience into llms during the inference stage via prompts (as shown in figure 1 ). their experience is textual descriptions of the task-solving processes, guidelines, and other ## methodology figure 2 shows the framework of our proposed self-evolving gpt, which consists of one taskspecific experience memory and five experiencecentric modules based on chatgpt. our framework continuously receives various user questions. it automatically categorizes the task type of the question, and adds it to memory if it is a new task not yet stored. for tasks that are not proficiently mastered, it performs experience transfer, autonomous practice, and experience induction to update their experience in memory. finally, it refers to experience stored in memory to respond the user question. in practice, we provide a basic implementation of our framework, which may be further optimized. we primarily focus on the overall framework, and aim to analyze its effectiveness and behavior. the prompts and execution examples of our implementation are presented in appendix d and e. ## conclusion & future work in this paper, we propose a lifelong autonomous experiential learning framework based on llms. it continuously and autonomously accumulates experience in solving tasks through experience transfer and induction, recognizing the nature of input questions to align them with relevant experience. considering the increasing demand for llms and the emergence of new types of user questions, our framework effectively reduces the human labor associated with previous methods. experiments show that the implementation of our framework can reliably execute each intermediate module and effectively enhance overall performance for responding to the input question. the following content may be subject to our research in future work: 1) enhanced engineering designs. we only offer a basic implementation for our framework, and there is still room for improvement, e.g., supporting more complex functions; 2) cold start. at present, we run our framework completely from empty memory. however, the existing manually annotated datasets can be used to replace the autonomous practice module. our framework can first learn from the manually annotated datasets, complete the cold start, and then run independently; 3) employing a combination of different-scaled llms to implement the framework. it is evident that not all tasks necessitate using chatgpt; integrating llms of various scales can achieve a balance between cost and performance; 4) experience distillation. distilling the rich experience summarized by gpt-4 onto smaller-scale llms to enhance their performance on tasks that have been adequately learned by gpt-4.