Exploring the Factual Consistency in Dialogue Comprehension of Large Language Models (2024)

Abstract: LLMs (Large Language Models) usually interact with users in the form of dialogue and generate responses following their instructions, which naturally require dialogue comprehension abilities. However, dialogue comprehension is a general language ability which is hard to be evaluated directly. In this work, we propose to perform the evaluation focusing on the factual consistency issue with the help of the dialogue summarization task. Besides evaluating and analyzing the dialogue summarization performance (DIAC-Sum) of different LLMs, we also derive factual questions from the generated summaries and use them as a more flexible measurement of dialogue comprehension (DIAC-FactQA). Our evaluation shows that, on average, 26.8% of the summaries generated by LLMs contain factual inconsistency. Even ChatGPT, the strongest model evaluated, has such errors in 16% of its summaries. For answering the factual questions, which is more challenging, the average error rate of all evaluated LLMs is 36.1%. Both results indicate serious deficiencies. Detailed analysis shows that the understanding of subject/object of the conversation is still challenging for LLMs. Furthermore, to stimulate and enhance the dialogue comprehension ability of LLMs, we propose a fine-tuning paradigm with auto-constructed multi-task data, which achieved a relative error rate reduction of 11% on DIAC-FactQA.

## introduction with the development of large language models (llms), such as @xcite , opt @xcite , llama @xcite , etc., it has becoming a promising way to interact with users through conversations, where llms generate responses following users' instructions in the dialogue. this form of conversational communication naturally requires high dialogue comprehension ability to capture the factual information, which has becoming a prerequisite for successfully completing tasks in the conversation. previous studies have evaluated the dialogue comprehension ability of relatively small language models using dialogue question answering (qa) data, e.g., dream @xcite and friend-sqa @xcite . however, the questions in these data come from either human exams or random sampling of the dialogue content, neither of which targets on the dialogue comprehension (diac) ability of llms. to fill this gap, we propose to perform the evaluation with the help of the dialogue summarization task (diac-sum), since summarization extracts important information from the dialogue which naturally requires a correct understanding of the dialogue. the summaries generated by 5 popular llms are collected and manually evaluated for factual consistency. the results are reported as an empirical study of the current situation. more importantly, we derive a set of factual questions from the factual inconsistencies in the generated summaries (diac-factqa), which corresponds to the difficult parts of understanding these dialogues. by answering these questions, the dialogue comprehension ability of llms could be evaluated in a more flexible and precise way. please note that the resulting dataset could be used for evaluating the dialogue comprehension ability of other llms in the future, without any more human annotation. our evaluation results show that existing llms still have serious deficiencies in conversational understanding. on average, 26.8% of the summaries generated by llms contain factual inconsistencies. even the summaries generated by chatgpt, the strongest llms in our evaluation, has 16% identified to be incorrect. further factual question probing results shows that the average error rate of we also present attempts to stimulate the dialogue comprehension ability by fine-tuning the model with auto-constructed multi-task data. experimental results demonstrate that after finetuning, the model exhibits a better dialogue comprehension ability, providing a potential direction for future work and improvements. ## conclusion in this paper, we conducted a detailed evaluation and analysis of the factual consistency in dialogue comprehension of existing llms, and attempted to improve upon them. specifically, we annotated the consistency of summaries generated by the llms, and constructed factual questions from generated errors to evaluate the existing models. the evaluation results showed that there are still serious dialogue comprehension defects in current llms. therefore, we made some initial attempt by using automatically constructed multi-task pseudo data to fine-tune the model. the experimental results showed that after fine-tuning, the model's dialogue comprehension capabilities were indeed enhanced, providing a possible solution for further research.