Should We Fine-Tune or RAG ? Evaluating Different Techniques to Adapt LLM s for Dialogue (2024)

Abstract: We study the limitations of Large Language Models (LLMs) for the task of response generation in human-machine dialogue. Several techniques have been proposed in the literature for different dialogue types (e.g., Open-Domain). However, the evaluations of these techniques have been limited in terms of base LLMs, dialogue types and evaluation metrics. In this work, we extensively analyze different LLM adaptation techniques when applied to different dialogue types. We have selected two base LLMs, Llama-2 and Mistral, and four dialogue types Open-Domain, Knowledge-Grounded, Task-Oriented, and Question Answering. We evaluate the performance of in-context learning and fine-tuning techniques across datasets selected for each dialogue type. We assess the impact of incorporating external knowledge to ground the generation in both scenarios of Retrieval-Augmented Generation (RAG) and gold knowledge. We adopt consistent evaluation and explainability criteria for automatic metrics and human evaluation protocols. Our analysis shows that there is no universal best-technique for adapting large language models as the efficacy of each technique depends on both the base LLM and the specific type of dialogue. Last but not least, the assessment of the best adaptation technique should include human evaluation to avoid false expectations and outcomes derived from automatic metrics.

## introduction in recent years, large language models (llms) have been employed for the task of response generation in human-machine dialogues @xcite @xcite . such models have been applied to several dialogue types, including open-domain dialogues (i.e. informal conversations about trivial matters), knowledge-grounded dialogues (i.e. ## experiments we study and compare in-context learning and fine-tuning as techniques to adapt llms for human-machine dialogues. we select llama-2 chat (llama2 c ) @xcite and mistral instruct (mistral i ) @xcite as base llms, and experiment in the context of four dialogue types: open-domain dialogue (odd), knowledge-grounded dialogue (kgd), task-oriented dialogue (tod), and question answering (qa). for each technique and dialogue type, we assess the impact of grounding the generation on documents in the scenarios of retrieved knowledge (rag) and gold knowledge. ## evaluation we conduct a comparative study on the impact of in-context learning and fine-tuning to adapt llms 2 https://github.com/langchain-ai/langchain for dialogues. we select llama2 c and mistral i as base llms and experiment in four dialogue types: odds, kgds, tods, and qa. for each dialogue type, we study the impact of external knowledge, both retrieved and gold. further details about the implementation and the resources used are available in the appendix ( §a.2). ## conclusion we have conducted an extensive analysis on the efficacy of fine-tuning and in-context learning to adapt llms for different dialogue types. we have experimented with retrieval-augmented generation (rag) and gold knowledge to assess the impact of grounding the response generation on external knowledge. we have studied the models' performance using consistent criteria in both automatic (perplexity, explainability studies) and human evaluations. our study highlights the limitation of currently available automatic metrics and the necessity of conducting human evaluations to advance humanmachine dialogue research, as the evaluations by human judges correlate poorly with automatic metrics. furthermore, conducted human evaluations indicate that there is no universal best-technique for adapting llms to a dialogue type and the performance of each technique depends on the base llm as well as the dialogue type. in addition, the correct incorporation of external knowledge depends on various factors such as the retriever accuracy, the representation of the knowledge, and the presence of noise (non-gold) documents, as it can be the least contributing element in the input vector according to explainability studies.