Towards Optimizing and Evaluating a Retrieval Augmented QA Chatbot using LLM s with Human-in-the-Loop (2024)

Abstract: Large Language Models have found application in various mundane and repetitive tasks including Human Resource (HR) support. We worked with the domain experts of a large multinational company to develop an HR support chatbot as an efficient and effective tool for addressing employee inquiries. We inserted a human-in-the-loop in various parts of the development cycles such as dataset collection, prompt optimization, and evaluation of generated output. By enhancing the LLM-driven chatbot’s response quality and exploring alternative retrieval methods, we have created an efficient, scalable, and flexible tool for HR professionals to address employee inquiries effectively. Our experiments and evaluation conclude that GPT-4 outperforms other models and can overcome inconsistencies in data through internal reasoning capabilities. Additionally, through expert analysis, we infer that reference-free evaluation metrics such as G-Eval and Prometheus demonstrate reliability closely aligned with that of human evaluation.

## introduction in recent years, incorporating artificial intelligence (ai) into various sectors has led to significant improvements in automated systems, particularly in customer service and support. since the offset of large language models (llms), more companies are now incorporating natural language processing (nlp) techniques to minimize the need for human support personnel, especially domain experts @xcite . with a chatbot providing accurate and comprehensive responses promptly, domain experts can redirect their focus towards higher-value tasks, leading to potential cost savings and improved productivity within the hr department. moreover, an effective chatbot can play a pivotal role in enhancing overall employee satisfaction and engagement by delivering timely and relevant assistance. to this end, we worked with a sap se on developing an hr chatbot to evaluate the potential of llms on industrial data. we used domain experts as a human-in-the-loop through various iterations of llm-centric development such as dataset collection, prompt optimization, and most importantly the evaluation of model outputs. the well-known retrieval augmented generation (rag) @xcite approach is ideal for this use case as it allows the model to produce more grounded answers, hence reducing hallucinations. we optimized different modules of the standard rag pipeline such as the retriever and model prompts, while constantly incorporating feedback from the domain experts. while the retrieval accuracy of an llm could still be assessed to a degree, the generative nature of llms makes evaluation of the generated output quite challenging. to overcome this, we explored the effectiveness of both traditional reference-based and reference-free (llm-based) automatic evaluation metrics while using human evaluation as a baseline. we benchmark openai's models in our experiments while using the open-source longt5 @xcite and bert @xcite as a baseline. in essence, both the industry and the research community could benefit from our findings related to the retriever and the reliability of automatic evaluation metrics. ## corpus the dataset used in the development of the hr chatbot was compiled using sap's internal hr policies with the help of domain experts. while each sample forms a triplet consisting of a question, answer, and context, additional metadata such as the user's region, company, employment status, and applicable company policies were also included. a snippet of such a sample is shown in appendix a.4. the dataset was compiled using two separate sources to have a mix of a gold dataset (faq dataset) and a user-utterance dataset (ut dataset). both datasets follow the same structure and differences exist in the distribution of the questions. we extracted all unique hr articles to form a knowledge base for answering new user questions. additionally, an evaluation set of 6k samples was used to evaluate both the retriever and the chatbot as a whole. ## methodology in general, the hr chatbot follows the standard rag pipeline with optimizations done on individual modules with the help of domain experts as shown in figure 2 . the methodology illustrates various parts of the chatbot pipeline that are influenced by a human-in-the-loop and is further discussed in appendix b. ## related work previously, domain-specific chatbots meant for a specific task were designed using conversational ai frameworks like rasa @xcite . latest advancements in nlp have shifted focus towards employing and optimizing llm-based rag @xcite . @xcite experiment with chatgpt and several other open-source models like vicuna to benchmark their capabilities in rag, and @xcite use a smaller secondary domain-specific model to assist a bigger llm on a domain-specific question answering task on industrial data. recent studies have explored various retrieval methods, including dense vector retrieval @xcite , sparse retrieval @xcite , and hybrid approaches @xcite , to improve the relevance and diversity of retrieved documents. @xcite uses various rag techniques to ensure that chatbot responses are based on relevant hr policies, leading to accurate and helpful user support. given the diverse distribution of the text generated by llms, conventional metrics are not suitable for its evaluation @xcite @xcite . consequently, a lot of follow-up research has come up in the area of nlg evaluation @xcite . specifically focusing on rag, @xcite released a framework for the automatic evaluation of generated output using llm-based metrics with a focus on faithfulness. a similar approach is followed by @xcite in their framework ares which also evaluates the performance of rag systems over relevance and faithfulness by fine-tuning a lightweight lm judge. ## conclusion by optimizing retrieval techniques and benchmarking state-of-the-art llms with the help of domain experts, we show how llm-based applications could benefit from a domain expert as humanin-the-loop within various iterations of the development. even though our optimizations on the openai-based retriever show minor improvements, the accuracy remains quite low due to the poor quality of the evaluation dataset. nonetheless, both chatgpt and gpt-4 show competence when addressing the user query. this hints that the internal reasoning capabilities and domain knowledge of these llms are strong enough to overcome the knowledge in the supposed incorrect article. this also suggests that, given the nature of the dataset used, the accuracy metric used for the evaluation of the retriever is not a good measure of its performance. we employed and studied a range of evaluation metrics and concluded that in contrast to traditional evaluation approaches such rouge & bertscore, llm-based metrics such as prometheus and g-eval come very close to human evaluation on average. nonetheless, our findings reiterate the importance of human judgment, particularly in use cases that require an understanding of a specific domain.