Empowering Conversational Agents using Semantic In-Context Learning (2023)

Abstract: Language models are one of the biggest game changers in downstream NLP applications, especially in conversational agents. In spite of their awesome capabilities to generated responses to solve the inquireis, there are still some big challenges to using them. One challenge is how to enable the LLMs to use the private internal data to solve inquires. And secondly, how to keep the LLMs updated with newly incoming data without the burden of fine-tuning as it is not only expensive but also not an available option for some commercial LLMs, such as ChatGPT. In this work, we propose Semantic In-Context Learning (S-ICL) to address the aforementioned challenges. Our approach was participated in the BEA 2023 shared task and ended up having the fourth place in both development and evaluation phases.

## introduction conversational agents are one of the most important applications of nlp. if implemented successfully, they can bring tremendous benefits for both organizations and clients, such as improving the efficiency of customer service in terms of support and availability of the services. with the emergence of powerful large language models (llms) such as chatgpt, there is a lot of interest in leveraging llms to develop ai agents. even though llms are capable of answering a broad spectrum of questions, there are still two major bottlenecks for using them as an ai assistant. first, each organization has some valuable internal knowledge such as faqs, policies, regulations, etc. that can or should be used to resolve incoming inquiries. however, the llms are trained based on public datasets and may not be aware of private knowledge sources that could help them to resolve incoming inquiries more accurately. secondly, fine-tuning these llms on the organization's internal data is not an easy task due to factors such as the size of the llms, cost of training, frequent updates in the internal data, and data privacy. for example, in news media, news articles are published every day that llms are not aware of them. if the news media decides to use an llm as an agent, the agent would be unable to provide users with information about current events or answer their questions about what is happening now. on top of that, the fine-tuning option is not available for certain llms (e.g., chatgpt with the gpt-3.5-turbo engine). one possible solution to the mentioned problems is in-context learning (icl), as it can enable the llms to perform well on the tasks or data that they have never seen before @xcite . in icl, a prompt containing an instruction, few labeled samples, and an unlabeled sample is given to the llm. then, the llm would be able to label the unlabeled sample without the need for any gradientbased training @xcite . however, it is infeasible to show all the available samples to the llm due to the high cost of computation. also, previous research shows that the format of the prompt, the selection of samples, the number, order, and structure of samples could have not only significant but also unforeseeable effects on llms' performance @xcite @xcite . to solve the aforementioned problems, we propose semantic in-context learning (s-icl) which utilizes a semantic search engine (i.e., an sbert model @xcite ) and an llm (i.e., chatgpt with the gpt-3.5-turbo engine) to build a conversational agent. this agent not only benefits from the knowledge of an llm but also utilizes available private knowledge sources to provide the correct answer to the inquiries. we also propose a flexible architecture that allows experts to apply and compare different approaches for prompt engineering. the proposed model is developed and participated in the bea 2023 shared task @xcite . however, the proposed model is flexible, and the agent can be used in other domains such as news media, customer service, and more. the rest of the paper is as follows. in section 2, we describe the proposed architecture along with its components. in section 3, we compare different configurations of the proposed model on the created test set, and we also evaluate the model on the competiton's data. finally, this paper is wrapped up with the conclusion in section 4. ## proposed model in this section, we present our proposed approach for generating a response to the inquiry. our proposed approach uses semantic search @xcite to enable the agent to utilize private domain data. it also uses a large language model not only to provide higher quality answers but also to enable the agent to answer questions that are significantly different from past questions and answers in the private domain data. ## experiment this section has three subsections. in the first subsection, we introduce the dataset used, split the train portion of the data into our created train and test sets, and show how the pre-processing has been done. in the second subsection, we conduct experiments on the proposed architecture using the created test set (i.e., selected from the original training set) and compare the accuracy of the model using different prompt building approaches. in the third subsection, we will demonstrate the model's performance on the development and testing sets of the competition data. ## conclusion we proposed a semantic in-context learning (s-icm) approach for conversational agents using the combination of a semantic search and a large language model (i.e., chatgpt). we also implemented an architecture enabling users to apply and compare different approaches for prompt engineering. we applied our proposed method on the bea 2023 shared task and our approach ended up ranking fourth in both the development and evaluation phases.