Detecting Response Generation Not Requiring Factual Judgment (2024)

Abstract: With the remarkable development of large language models (LLMs), ensuring the factuality of output has become a challenge.However, having all the contents of the response with given knowledge or facts is not necessarily a good thing in dialogues.This study aimed to achieve both attractiveness and factuality in a dialogue response for which a task was set to predict sentences that do not require factual correctness judgment such as agreeing, or personal opinions/feelings.We created a dataset, dialogue dataset annotated with fact-check-needed label (DDFC), for this task via crowdsourcing, and classification tasks were performed on several models using this dataset.The model with the highest classification accuracy could yield about 88% accurate classification results.

## introduction large language models (llms) have undergone considerable development and can solve various natural language processing tasks. however, they output content that is different from the fact, i.e., hallucination, making it difficult to ensure the factuality of the output @xcite @xcite . although hallucination in dialogue systems using llms has been extensively studied, they focused on methods for detecting/suppressing hallucinations and investigated the causes of their occurrence @xcite @xcite . wizard of wikipedia (wow), a knowledgebased dialogue dataset created by @xcite , contains many subjective opinions and feelings of the speaker. @xcite labeled utterances in wow datasets that contained subjective opinions and feelings as hallucinations and showed that models fine-tuned on wow datasets produce more hallucinations. however, for open-domain dialogue systems such as chatbots, unlike systems in other fields such as summarization or machine translation, not all output in a response are based on a given input or knowledge. to promote smooth dialogue and increase engagement, expressing personal feelings and opinions is important @xcite . moreover, the tolerance of factual correctness regarding the response of these contents is high @xcite . to address these issues, we propose that sentences not requiring factual correctness judgment should be detected and removed before judgment (hallucination detection) during response generation in dialogue systems. by detecting such sentences first and judging the factual correctness of remaining sentences, responses that maintain the attractiveness of the dialogue can be generated while ensuring the factuality of the dialogue. first, we set the task of detecting sentences that do not require factual correctness judgment, and created a new dataset. then, the dataset was is the sentence only indicating agreement/disagreement or a feedback? q1 ・agreement ・disagreement ・interjections (i) is the sentence providing new information? q2 ・suggestions ・advice (ii) is everything in the sentence based on the speaker's subjective opinion, personal experience, thoughts, or feelings? q3 ・subjective opinions ・personal experiences /thoughts/feelings (iii) ・objective information validated using classification models. figure 1 overviews the created dataset, dialogue dataset annotated with fact-check-needed label (ddfc). the construction method and contents of ddfc are described in section 3. ## ddfc dataset the ddfc dataset created herein contained external knowledge, responses based on external knowledge, responses split by sentences, sentence labels based on discourse acts, and labels to determine whether factual correctness judgments are required. we used four types of labels, and crowdworkers assigned them through annotation based on the flowchart (figure 2 ). ## experiment 1: classification we prepared some classification models and experimentally evaluated the results of the classification (binary classification) of sentences that do not require factual correctness judgment. ## experiment 2: relation between train data amount and accuracy the relation between the training data amount for fine-tuning and classification accuracy was investigated by conducting an experiment. ## future directions improving the performance of classification models. herein, fine-tuning was performed on 1,000 data, which is a small amount compared to the training data size (about 18,400 responses) of the base dataset, faithdial. thus, the dataset can be expanded. as further improvement in classification accuracy can be expected by expanding the dataset, future studies will involve large-scale data collection. it may also clarify the reason for the sudden increase in accuracy when the number of training data exceeds 800 in figure 3(a) , and whether the trend of gradual increase in accuracy in figure 3( @xcite continues when training data is increased. moreover, because our dataset was small, the sentence labels (i), (ii), and (iii) had to be treated as a single label, "not required factual correctness judgments," for the binary classification task. after collecting sufficient data, we would like to investigate whether the four labels can be used for classification. ## conclusion in this study, a task to detect sentences that do not need to be judged as factually correct or incorrect was proposed against hallucinations in a dialogue system using llms. we created a dataset containing 1,317 sentences labeled with sentence types using the amazon mechanical turk. several classification models were developed as a baseline for this task. results revealed that the best model could classify with an accuracy of approximately 88%. in the future, we would like to collect data on a larger scale and apply the several models trained in this study to the dialogue system.