Coling- U ni A at S ci VQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models (2025)

Abstract: This paper describes our system for the SciVQA 2025 Shared Task on Scientific Visual Question Answering. Our system employs an ensemble of two Multimodal Large Language Models and various few-shot example retrieval strategies. The model and few-shot setting are selected based on the figure and question type. We also select answers based on the models’ confidence levels. On the blind test data, our system ranks third out of seven with an average F1 score of 85.12 across ROUGE-1, ROUGE-L, and BERTS. Our code is publicly available.

## introduction visual question answering (vqa) requires systems to answer natural language questions about visual content. the complexity of these questions can range from binary questions to free-form and open-ended questions. existing vqa datasets address various types of images, e.g., vqa v2 focuses on real-world photos @xcite , docvqa focuses on scanned documents @xcite , while chartqa @xcite and plotqa @xcite focus on charts. in this paper, we describe our system submission for the 2025 shared task on scientific visual question answering (scivqa) @xcite . the dataset 2 comprises 3000 real-world scientific figure images, which were collected from the acl-fig @xcite and scigraphqa @xcite . most existing vqa approaches that focus on charts rely on models explicitly tuned for this domain @xcite @xcite . in contrast, our approach uses multimodal large language models (mllms) in a zero/few-shot setting without any fine-tuning. we test several strategies for retrieving 1 https://github.com/coling-unia/few-shot-scivqa2025 2 https://huggingface.co/datasets/katebor/scivqa few-shot examples from the training set based on question or question-and-image similarity. we find that performance varies widely by question/figure type and by mllm. our best-performing approach first selects highly confident answers from a configuration of an mllm and a few-shot setting. for all remaining instances, the system configuration is varied by the instance's question type. in the official evaluation, our system ranks third. ## method our system is configurable to use different mllms in either a zero-shot or a few-shot setting. these settings are combined using an ensemble approach (see figure 1 ) that first selects all high-confidence answers from a configuration that we find to be well-calibrated, i.e., the predicted confidence scores align well with the actual empirical accuracy on the development set. we approximate answer confidence by exponentiating the mean logprobability of all generated answer tokens. for the remaining instances, the model configuration is selected based on question type as identified on the development set. the mllm is prompted with each image and the associated question (see ap- the task description depends on whether there are pre-defined answer options for the questions. the model is instructed to answer and to determine whether it is possible to answer based solely on the provided information. to enhance the reproducibility, our selection of mllms is constrained to open-weights models. we use internvl3-78b @xcite and pixtral-large-instruct-2411. 3 we run all models using 16-bit quantization and a temperature of 0. ## development results and ablations since our approach does not require any finetuning, we combine the training and validation sets into one development set. this section describes our careful experimentation and ablation studies on the development set. for more information on the dataset, see appendix a.3. we rely on the metrics of the shared task, f1, precision, and recall of rouge-1, rouge-l @xcite , and bertscore (berts, @xcite , respectively, to evaluate our approach. however, we focus on rouge-1 f1, as the berts scores are similar for all approaches, and the rouge-l scores are comparable to rouge-1. we run our experiments on nvidia a100 (80 gb) gpus, using up to 4 gpus in parallel. the total amount of gpu hours was about 3600h. ## results on test set ## discussion and conclusion this paper described our submission to the scivqa 2025 shared task. our results show that mllms are highly effective at answering questions about scientific figures. however, performance varies greatly by question type. results on finite answer sets are considerably better than on infinite ones. in particular, answering infinite answer set questions about visual features of images remains challenging, highlighting the need for a more sophisticated approach. the use of few-shot examples improves performance. however, there are no major performance differences between retrieving the examples by question or question-image similarity.