S-Chain: Structured Visual Chain-of-Thought For Medicine
Authors: name: Khai Le-Duc; name: Duy M. H. Nguyen; name: Phuong T. H. Trinh; name: Tien-Phat Nguyen; name: Nghiem T. Diep; name: An Ngo; name: Tung Vu; name: Trinh Vuong; name: Anh-Tien Nguyen; name: Mau Nguyen; name: Van Trung Hoang; name: Khai-Nguyen Nguyen; name: Hy Nguyen; name: Chris Ngo; name: Anji Liu; name: Nhat Ho; name: Anne-Christin Hauschild; name: Khanh Xuan Nguyen; name: Thanh Nguyen-Tang; name: Pengtao Xie; name: Daniel Sonntag; name: James Zou; name: Mathias Niepert; name: Anh Totti Nguyen
Published: 2025-10-26 15:57:14

Abstract: Faithful reasoning in medical vision-language models (VLMs) requires not only
accurate predictions but also transparent alignment between textual rationales
and visual evidence. While Chain-of-Thought (CoT) prompting has shown promise
in medical visual question answering (VQA), no large-scale expert-level dataset
has captured stepwise reasoning with precise visual grounding. We introduce
S-Chain, the first large-scale dataset of 12,000 expert-annotated medical
images with bounding boxes and structured visual CoT (SV-CoT), explicitly
linking visual regions to reasoning steps. The dataset further supports 16
languages, totaling over 700k VQA pairs for broad multilingual applicability.
Using S-Chain, we benchmark state-of-the-art medical VLMs (ExGra-Med,
LLaVA-Med) and general-purpose VLMs (Qwen2.5-VL, InternVL2.5), showing that
SV-CoT supervision significantly improves interpretability, grounding fidelity,
and robustness. Beyond benchmarking, we study its synergy with
retrieval-augmented generation, revealing how domain knowledge and visual
grounding interact during autoregressive reasoning. Finally, we propose a new
mechanism that strengthens the alignment between visual evidence and reasoning,
improving both reliability and efficiency. S-Chain establishes a new benchmark
for grounded medical reasoning and paves the way toward more trustworthy and
explainable medical VLMs.