Models See Hallucinations: Evaluating the Factuality in Video Captioning (2023)

Abstract: Video captioning aims to describe events in a video with natural language. In recent years, many works have focused on improving captioning models’ performance. However, like other text generation tasks, it risks introducing factual errors not supported by the input video. Factual errors can seriously affect the quality of the generated text, sometimes making it completely unusable. Although factual consistency has received much research attention in text-to-text tasks (e.g., summarization), it is less studied in vision-based text generation. In this work, we conduct the first human evaluation of the factuality in video captioning and annotate two factuality datasets. We find that 56% of the model-generated sentences have factual errors, indicating it is a severe problem in this field, but existing evaluation metrics show little correlation with human factuality annotation. We further propose a weakly-supervised, model-based factuality metric FactVC, which outperforms previous metrics on factuality evaluation of video captioning.

## introduction video captioning is a challenging cross-modal task that aims to describe videos with natural language sentences. in recent years, video captioning has received much attention in computer vision and natural language processing communities. substantial progress has been made to generate descriptions for videos that contain a single event @xcite or multiple events @xcite . however, like other text generation tasks, video captioning models risk introducing factual errors not supported by the input video. examples are shown in caption 3: a man in a white shirt is standing at a dart board. he throws a dart at the end. the video. factual errors can cause misunderstandings of the video content, sometimes making the generated captions completely unusable. factual consistency evaluation has received much research attention in text-to-text tasks, including summarization @xcite , knowledge-grounded dialogue @xcite ), text simplification @xcite , and large language models @xcite . nevertheless, it is less studied in vision-totext tasks, especially video captioning. therefore, this work focuses on the research gap in the factuality evaluation of video captioning. recently, more works have focused on videos with multiple events @xcite , and it may bring more factual errors. so we choose multiple events video captioning for our evaluation. we use activitynet captions @xcite and youcook2 @xcite as our video datasets, for they are the most common datasets for this task. then we carefully select six recent models on each dataset to generate video captions. the models differ in model framework, pretrained features, and input signals. after collecting the videos and captions, we design a factuality annotation protocol and conduct human annotation. in the end, we obtain two humanannotated factuality datasets activitynet-fact (200 videos, 3,834 sentences) and youcook2-fact (100 videos, 4,080 sentences). by analyzing the human annotation, we find that factual error (hallucination) is a severe problem in video captioning. to sum up, there are 87% of the paragraphs, 56% of the sentences, and 15% of the words have factual errors. there are different types of factual errors, including person-related errors, action-related errors, object-related errors and so on. since hallucination is a severe problem in video captioning, we test to what extent existing automatic evaluation metrics can measure the factuality of video captions. we find that most existing metrics correlate poorly with human judgment. therefore we propose a new metric factvc (factual consistency for video captioning). we use the clip model @xcite to encode video frames and captions. considering the clip model is trained on image-text pairs, it may have a gap transferring to video factuality evaluation. so we automatically construct a training set using text augmentation skills and finetune clip on it. our factvc metric achieves a higher correlation with human factuality annotation. the main contributions of this work are as follows: ## related work factuality evaluation is first proposed in the field of document summarization. @xcite conducted a human annotation on the xsum dataset @xcite and found that more than 70% of summaries generated by summarization models have factual errors. other human annotations @xcite reach similar conclusions. to measure the factual consistency, researchers proposed different metrics, which can be roughly divided into entailmentbased metrics @xcite and qa-based metrics @xcite @xcite . inspired by the works in summarization, factuality evaluation is studied for other tasks, including knowledgegrounded dialogue @xcite ), text simplification @xcite and large language models @xcite . for vision-based text generation tasks, the most widely used metrics are based on n-gram matching between references and generated captions, including bleu @xcite , rouge @xcite , meteor @xcite , cider @xcite . they cannot match deeper semantics between captions and vision inputs. recently, there are model-based metrics such as bertscore @xcite , clip-score @xcite , emscore @xcite , pac-s @xcite . they leverage large-scale pretrained models to compute a matching score, even do not requiring reference captions. a little work pays attention to the hallucination problem in vision-based text generation. chair @xcite proposes an image relevance metric to evaluate the object hallucination in image captioning. they restrict their evaluation to 80 mscoco objects. coaha(ullah and mohanta, 2022) uses object and action words matching to asses hallucination in video captioning. emscore @xcite and pac-s @xcite , designed for the overall evaluation of video captioning, also shows the potential to identify hallucinating captions. however, there is still no related human-annotated dataset and complete factuality evaluation work. ## human annotation considering there does not exist factuality annotation of video captioning, we decide to construct our own datasets. we use activitynet captions @xcite and youcook2 @xcite as our source video datasets and select six recent models for each dataset to generate captions. then we design a factuality annotation protocol and conduct our human annotation. ## metric analysis now that factual errors broadly exist in video captions, we want to know to what extent existing metrics can measure the factuality of video captions. we test the correlation between automatic metrics and human annotation (for sentence/word-level annotation, we use the ratio of factual sentences and words as annotation score). we test model-free metrics bleu @xcite , rouge @xcite , meteor @xcite , cider( vedantam ## factvc metric although emscore achieves a good correlation with human factuality annotation, it has two drawbacks: 1) emscore uses the pretrained model clip, which is trained on image-text pairs from the internet, and it may not transfer well to the video captioning data; 2) emscore is designed for evaluating the overall quality of the video caption, not specifically designed for factuality evaluation. as a result, we propose a new metric factvc (factual consistency for video captioning). we first automatically construct a factuality training set using text augmentation skills and then finetune the clip model. we also improve the calculation of the similarity score so that it is more suitable for factuality evaluation. ## conclusion in this work, we focus on the factuality evaluation in video captioning. we collect two humanannotated factuality datasets for video captioning and find that hallucination is a severe problem, with 56% of the model-generated sentences having different kinds of factual errors. however, most existing metrics show little correlation with human annotation. so we propose a new factuality metric factvc. it is trained on an automaticallyconstructed training set and correlates much better with the factuality annotation. experiments also show the potential of our method in evaluating image captions. although factual consistency is a hot research topic in nlp tasks, it is less studied in video captioning. we hope our work can fill this research gap and promote further research in video captioning.