S-Chain: Structured Visual Chain-of-Thought For Medicine
Authors: name: Khai Le-Duc, name: Duy M. H. Nguyen, name: Phuong T. H. Trinh, name: Tien-Phat Nguyen, name: Nghiem T. Diep, name: An Ngo, name: Tung Vu, name: Trinh Vuong, name: Anh-Tien Nguyen, name: Mau Nguyen, name: Van Trung Hoang, name: Khai-Nguyen Nguyen, name: Hy Nguyen, name: Chris Ngo, name: Anji Liu, name: Nhat Ho, name: Anne-Christin Hauschild, name: Khanh Xuan Nguyen, name: Thanh Nguyen-Tang, name: Pengtao Xie, name: Daniel Sonntag, name: James Zou, name: Mathias Niepert, name: Anh Totti Nguyen

Faithful reasoning in medical vision-language models (VLMs) requires not only
accurate predictions but also transparent alignment between textual rationales
and visual evidence. While Chain-of-Thought (CoT) prompting has shown promise
in medical visual question answering (VQA), no large-scale expert-level dataset
has captured stepwise reasoning with precise visual grounding. We introduce
S-Chain, the first large-scale dataset of 12,000 expert-annotated medical
images with bounding boxes and structured visual CoT (SV-CoT), explicitly
linking visual regions to reasoning steps. The dataset further supports 16
languages, totaling over 700k VQA pairs for broad multilingual applicability.
Using S-Chain, we benchmark state-of-the-art medical VLMs (ExGra-Med,
LLaVA-Med) and general-purpose VLMs (Qwen2.5-VL, InternVL2.5), showing that
SV-CoT supervision significantly improves interpretability, grounding fidelity,
and robustness. Beyond benchmarking, we study its synergy with
retrieval-augmented generation, revealing how domain knowledge and visual
grounding interact during autoregressive reasoning. Finally, we propose a new
mechanism that strengthens the alignment between visual evidence and reasoning,
improving both reliability and efficiency. S-Chain establishes a new benchmark
for grounded medical reasoning and paves the way toward more trustworthy and
explainable medical VLMs.