E xp N ote: Black-box Large Language Models are better Task Solvers with Experience Notebook (2023)

Abstract: Black-box Large Language Models (LLMs) have shown great power in solving various tasks and are considered general problem solvers. However, LLMs still fail in many specific tasks although understand the task instruction. In this paper, we focus on the problem of boosting the ability of black-box LLMs to solve downstream tasks. We propose ExpNote, an automated framework to help LLMs better adapt to unfamiliar tasks through reflecting and noting experiences from training data and retrieving them from external memory during testing. We evaluate ExpNote on multiple tasks and the experimental results demonstrate that the proposed method significantly improves the performance of black-box LLMs. The data and code are available at https://github.com/forangel2014/ExpNote.

## introduction large language models (llms) have demonstrated astonishing capabilities in natural language understanding and generation @xcite @xcite . however, due to the limited parameters and context processing length, llms are not able to master all task-specific knowledge in real-world applications. as a result, llms may perform mediocre on some specific tasks, such as inductive reasoning @xcite and entity recognition @xcite . therefore, how to make llms adapt to the downstream tasks has tracked more and more attention. recent techniques such as prefix-tuning @xcite , p-tuning @xcite and @xcite proposed low-cost solutions for fine-tuning llms. however, these methods are not capable of black-box powerful llms, such as @xcite . a: jack is maria's son. q: @xcite and his son @xcite are at a bar waiting for their drinks. @xcite showed up and sat with her husband @xcite . who is jack to maria? to empower black-box llms on specific tasks, several works @xcite have focused on equipping the llms with external dynamic memory to store useful taskspecific knowledge and facts. however, these taskspecific knowledge and facts in the memory usually come from expert annotation or human feedback, which is very costly to obtain. on the other hand, several researchers @xcite @xcite try to exploit the reflection ability of llms to automatically generate such knowledge for specific tasks. however, most reflection-based methods are only able to empower the llms in the same case, without the ability to generalize to other instances. thus, this paper proposes a framework exp-note (experience notebook), to empower blackbox llms on downstream tasks by learning and using task-specific experience automatically. we equip the llms with a dynamic memory and design several commands to help llms interact with it. in specific, in the training stage, expnote guides llms to generate task-specific experiences and store them in an external memory. in the testing stage, expnote uses a retriever to retrieve relevant experiences from the memory, the learned experiences will help the llms to solve the cases which they failed to answer directly (figure 1 ). we evaluate expnote on multiple tasks. the results show that expnote can empower the llms effectively, and significantly outperform other prompting methods like ordinary in-context learning (cot, @xcite , memory-based method (teachme, @xcite , and case-by-case reflection-based method @xcite . moreover, we empirically compared different types of experiences to examine their effectiveness in helping llms adapt to unfamiliar tasks. specifically, we compared learned task-specific experiences with original cases and experiences learned from positive cases (succeeded) and negative cases (failed). we find that prompting with experiences is more helpful than original cases for llms to generalize to new cases, and experiences from both positive and negative cases are beneficial. the major contributions of this paper are twofold: ## experiments in this section, we want to answer the following research questions: • rq1. is expnote able to help llms to adapt to new tasks effectively? • rq2. which kinds of experiences help llms solve tasks better? ## conclusion in this paper, we propose expnote, an automated framework to help black-box llms adapt to specific downstream tasks by interacting with dynamic memory. we carried out experiments on multiple datasets from different tasks and showed that ex-pnote can effectively improve the performance of llms better than other prompting methods. we also found that the learned task-specific experiences help llms to better generalize than the original cases in the task, and experiences learned from both positive cases and negative cases are valuable.