MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based
  Scientific Research
Authors: name: James Burgess, name: Jeffrey J Nirschl, name: Laura Bravo-Sánchez, name: Alejandro Lozano, name: Sanket Rajan Gupte, name: Jesus G. Galaz-Montoya, name: Yuhui Zhang, name: Yuchang Su, name: Disha Bhowmik, name: Zachary Coman, name: Sarina M. Hasan, name: Alexandra Johannesson, name: William D. Leineweber, name: Malvika G Nair, name: Ridhi Yarlagadda, name: Connor Zuraski, name: Wah Chiu, name: Sarah Cohen, name: Jan N. Hansen, name: Manuel D Leonetti, name: Chad Liu, name: Emma Lundberg, name: Serena Yeung-Levy

Scientific research demands sophisticated reasoning over multimodal data, a
challenge especially prevalent in biology. Despite recent advances in
multimodal large language models (MLLMs) for AI-assisted research, existing
multimodal reasoning benchmarks only target up to college-level difficulty,
while research-level benchmarks emphasize lower-level perception, falling short
of the complex multimodal reasoning needed for scientific discovery. To bridge
this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark
designed to assess three reasoning capabilities vital in research workflows:
expert image understanding, hypothesis generation, and experiment proposal.
MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology
experts across diverse microscopy modalities, ensuring VQA samples represent
real scientific practice. In constructing the benchmark, we find that standard
MCQ generation methods induce language shortcuts, motivating a new two-stage
pipeline: an optimized LLM prompt structures question-answer pairs into MCQs;
then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking
on state-of-the-art MLLMs reveal a peak performance of 53\%; models with
smaller LLMs only slightly underperform top models, suggesting that
language-based reasoning is less challenging than multimodal reasoning; and
tuning with scientific articles enhances performance. Expert analysis of
chain-of-thought responses shows that perception errors are the most frequent,
followed by knowledge errors and then overgeneralization errors. These insights
highlight the challenges in multimodal scientific reasoning, showing MicroVQA
is a valuable resource advancing AI-driven biomedical research. MicroVQA is
available at https://huggingface.co/datasets/jmhb/microvqa, and project page at
https://jmhb0.github.io/microvqa.