Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models

Authors: Eunseop Yoon, Hee Suk Yoon, Mark A. Hasegawa-Johnson, Chang D. Yoo
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Multimodal Large Language Model, Alignment

Abstract:
In the broader context of deep learning, Multimodal Large Language Models have achieved significant breakthroughs by leveraging powerful Large Language Models as a backbone to align different modalities into the language space. A prime exemplification is the development of Video Large Language Models (Video-LLMs). While numerous advancements have been proposed to enhance the video understanding capabilities of these models, they are predominantly trained on questions generated directly from video content. However, in real-world scenarios, users often pose questions that extend beyond the informational scope of the video, highlighting the need for Video-LLMs to assess the relevance of the question. We demonstrate that even the best-performing Video-LLMs fail to reject unfit questions-not necessarily due to a lack of video understanding, but because they have not been trained to identify and refuse such questions. To address this limitation, we propose alignment for answerability, a framework that equips Video-LLMs with the ability to evaluate the relevance of a question based on the input video and appropriately decline to answer when the question exceeds the scope of the video, as well as an evaluation framework with a comprehensive set of metrics designed to measure model behavior before and after alignment. Furthermore, we present a pipeline for creating a dataset specifically tailored for alignment for answerability, leveraging existing video-description paired datasets.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730115623389

thread_type:
official_review

<thread_object>
creator: reviewer_jkxc

<content>
summary:
This paper addresses the issue of "answerability" in Video Large Language Models (Video-LLMs). Current Video-LLMs excel at answering questions directly related to the content of a video, but struggle with questions that go beyond the video's scope, often hallucinating plausible but incorrect answers. The authors propose a framework called "alignment for answerability" to train Video-LLMs to recognize and refuse to answer such unanswerable questions.

This framework involves:
- Alignment:  Training Video-LLMs to assess the relevance of a question to the video content and respond with "unanswerable" when appropriate.  They formally define this alignment process and associated scoring functions.
- Evaluation Metrics:  Beyond simple accuracy, the authors propose a set of metrics to comprehensively evaluate the alignment process. These include measuring how often the model refuses to answer valid questions (Excessive Refusal), how often it correctly answers previously refused questions (Permissiveness), and how well it identifies and declines truly unanswerable questions (Discretion).
- Dataset Creation (UVQA): The authors created a new dataset, UVQA, specifically for training and evaluating answerability.  They leverage existing video-description datasets, altering the descriptions to generate questions that are unanswerable based on the accompanying video.  They further categorize these questions based on the type of mismatch (object, attribute, or relationship).

Experiments demonstrate that models trained with this framework and the UVQA dataset significantly improve their ability to handle unanswerable questions. The authors also explore an alternative approach based on decomposing questions into multiple existence-based sub-questions (inspired by the POPE approach for image-based LLMs), but find it less effective and computationally more expensive than their proposed framework.

strengths:
- The paper tackles an important yet often overlooked issue of answerability in Video-LLMs.
- The paper defines sound metrics and data generation pipeline.
- The results on the in-domain evaluation set are convincing, notably with a positive comparison against the POPE baseline.

weaknesses:
- The evaluation set is curated from the same distribution as the training set, which may overestimate the benefits and results of the alignment procedure. It would be great to see the results on a fully human-annotated set, ideally on videos from a different distribution (e.g. using videos from MSR-VTT).
- Having results on academic VideoQA benchmarks (e.g. the full Activitynet-QA evaluation set) would enable to double check the absence of accuracy degradation compared to the state of the art. Has any trade-offs between answerability performance and standard VideoQA performance been observed?

questions:
- The approach could be extended to images in future work. Is there any challenge the authors anticipate in doing so?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730170819703

thread_type:
official_review

<thread_object>
creator: reviewer_yile

<content>
summary:
The paper aims to improve Video-LLMs’ capabilities of differentiating visually-unaligned questions, while maximally retain the performance on answering common answerable questions.
Specifically, the paper first defines a suite of metrics to evaluate alignment for answerability. It then collects data for both training and evaluation. The experiments show that existing Video-LLMs almost cannot refuse to answer unaligned questions. While SFT and DPO can effectively improve the overall performance, DPO significantly outperforms SFT in maintaining the models’ original performance while handling unaligned questions.

strengths:
1.	The work is overall well-developed and shows the authors’ insights in the problem.
2.	The constructed benchmark could be valuable, with dedicated evaluation metrics and annotations (reason for unanswerable).
3.	The paper implements both SFT and DPO for improving existing Video-LLMs with the training data and shares helpful insights.

weaknesses:
1.	The paper neglects to discuss many existing works that study the unanwerability of (V)QA models (see my attached references). 

2.	It would be better to conduct more in-depth analysis to help understand what kinds of unanswerable QA are more challenging to resolve in videos: static objects/attributes/relations vs. dynamic actions. 


[1] Whitehead S, Petryk S, Shakib V, et al. Reliable visual question answering: Abstain rather than answer incorrectly[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 148-166.

[2] Guo Y, Jiao F, Shen Z, et al. Unanswerable visual question answering[J]. arXiv preprint arXiv:2310.10942, 2023.

[3] Zhao Y, Zhang R, Xiao J, et al. Towards Analyzing and Mitigating Sycophancy in Large Vision-Language Models[J]. arXiv preprint arXiv:2408.11261, 2024.

[4] Brahman F, Kumar S, Balachandran V, et al. The art of saying no: Contextual noncompliance in language models[J]. arXiv preprint arXiv:2407.12043, 2024.

[5] Wen B, Yao J, Feng S, et al. The art of refusal: A survey of abstention in large language models[J]. arXiv preprint arXiv:2407.18418

questions:
see weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730670305160

thread_type:
official_review

<thread_object>
creator: reviewer_y4aq

<content>
summary:
This paper studies the phenomenon that some video language models can't refuse to answer a question when the question is irrelevant to the video. The authors first showcase such failure cases in existing VLLMs, then provide a method and metric to identify such case, and finally propose a dataset to finetune existing VLLMs to improve this issue.

strengths:
- This paper executed a valid pipeline of improving VLLM on a task: identifying the problem, curating a dataset to fix the problem, and finetuning the model to show improvements. The paper shows a good practice on the task of introducing refusing to answer for VLLMs.

- The alignment score defined in section 4 makes sense to me.

- The authors conducted experiments on a variety number of VLLMs in Table 1.

weaknesses:
- My main feeling about reading this paper is I feel the problem of refusing to answer is a bit small and artificial. While showing the failure case in Figure 1, I feel it is also important to show if the problem can be migrated by explicit prompting, e.g., add to the prompt "Say 'can't answer' if you are not sure". Even though I believe the authors proposed method with finetuning on the curated dataset may still be better, I feel the problem is not as big as the author claimed.

- Adding to the above point, the authors only evaluated relatively small VLLMs in Table 1 (Video-LLaVA/ VideoChat2/ LLaMA-VID), while skipping the more recent SOTA VLLMs like Qwen2-VL, LLaVA-OneVision, and GPT-4o or Gemini 1.5. I am expecting these more well instruction-tuned models may have less answerability issues.

- If I understand correctly, the authors propose to solve the problem by finetuning the VLLM on a dataset. A clear shortcoming is that the models might drop performance on other tasks, which is not desirable. Please provide some discussion on this.

questions:
- Overall, I feel this paper propose a valid solution to a small problem. The solution might have issues on overfitting the model on the particular task, and the problem may not exist on more modern VLLMs. My current rating is a week reject. I am happy to raise my rating if the authors address my concerns in the rebuttal.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730706552874

thread_type:
official_review

<thread_object>
creator: reviewer_gebz

<content>
summary:
This paper introduces the concept of alignment for answerability in the context of video LLMs. Current models and data do not consider out-of-scope questions that a user might ask, and hence they don't recognize them as unanswerable. A definition and a metric to assess this phenomenon are presented. Furthermore, a new dataset is proposed containing unanswerable questions, and fine-tuning models on this dataset shows improved alignment towards recognizing unanswerable questions.

strengths:
The paper is well-written and clearly introduces the concepts, definitions and metrics. The contribution is original as it formally recognizes a new problem, i.e. multimodal LLMs answering clearly unanswerable questions, and proposes solutions to it.
- The proposed problem framing and metric make intuitive sense. The definition takes into account the reasoning of why a question is unanswerable. The metric takes into account excessive refusal to answer which is a big problem with LLMs.
- A new dataset for alignment for answerability is presented. This is a valuable addition to the community as it creates out-of-scope QA pairs which we rarely see.
- The dataset creation process is simplistic, modifying a constrained set of objects and their relations, or changing object attributes. Furthermore, the evaluation dataset is verified manually. So, the dataset is likely dependable.
- Experimental results after fine-tuning on this dataset show clear improvements in alignment (Table 1), and the ablation study in Figure 5 outlines the drawback of an alternative.

weaknesses:
The main weakness is that the proposed QAs in the dataset seem to be mostly geared towards detection capabilities without requiring much reasoning. This in turn makes the corresponding dataset construction, and improving model's capabilities on this axis rather easy. Some qualitative examples presented are "What breed is the cat in the video?", "What color laptop the presenter is holding?", "How many times does a person in gray shirt appear in the video?", etc. Recently, many reasoning based video-language benchmarks are proposed [1, 2] while this paper is more similar to older benchmarks [3].

Other notable weaknesses:
- The metric is rather complex. If we just want to detect unanswerable questions, and balance precision and recall, why not use F1-Score?
- The metric is defined between aligned model and unaligned model. However, if one wants to test the ability of a given model regarding unanswerable questions, the proposed metric cannot be used.
- Using LLM-in-the-loop to create the dataset may create some bias, hallucinations and incorrect QA pairs. 
- Human performance on the evaluation set is not presented.

[1] Perception Test: A Diagnostic Benchmark for Multimodal Video Models \
[2] MVBench: A Comprehensive Multi-modal Video Understanding Benchmark \
[3] ActivityNet: A large-scale video benchmark for human activity understanding

questions:
Can the authors provide more qualitative examples of unanswerable questions from the proposed dataset? Similarly, can we also get more qualitative examples of existing & aligned model's reactions to such questions?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732487467984

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to Reviewers

comment:
We would like to thank all the reviewers for the detailed review and feedback. We have addressed each noted question separately. We have uploaded a revised version of the manuscript with the following additions (**revised parts are indicated in blue in the manuscript**):

* **Figure 1-(b)** has been added, which shows that scaling the model  (7B $\rightarrow$ 72B) does not improve the answerability performance.

* *Unanswerability of Question Answering* section has been added to **Section 2: Related Work**.

* In **Table 1**, we have added binary correctness for answerable vs. unanswerable predictions using the **F1-score**. 

* In **Appendix A.4.3**, we added more qualitative examples of the generated UVQA dataset.

* **Appendix A.7** has been added to show more qualitative examples of model predictions.

* **Appendix A.8** has been added to analyze the performance difference with respect to the unanswerability type. 

* **Appendix A.9** has been added to show if prompt addition alone (such as adding the prompt `If the question cannot be answered using the video content, state that it is unanswerable and provide a reason.') can achieve alignment for answerability.

* **Appendix A.10** has been added to visualize the trade-off between the answerable QA accuracy and the unanswerable QA accuracy using the Pareto front visualization.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The authors address a timely and relevant task to the community: Enabling VLMs to refuse an answer when there is not sufficient information in the video. Reviewers had some initial concerns about the paper (ie degradations on existing tasks, human evaluations, how much is this really a problem in practice), but the authors addressed these comprehensievely in the rebuttal. As a result, all the reviewers were in favour of accepting the paper. Thank you for updating and improving the paper during the rebuttal phase.