TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis
Authors: Sijing Li
Zhongwei Qiu
Jiang Liu
Wenqiao Zhang
Tianwei Lin
Yihan Xie
Jianxiang An
Boxiang Yun
Chenglin Yang
Jun Xiao
Guangyu Guo
Jiawen Yao
Wei Liu
Yuan gao
Ke Yan
Weiwei Cao
Zhilin Zheng
Tony C. W. MOK
Kai Cao
Yu Shi
Jiuyu Zhang
Jian Zhou
Beng Chin Ooi
Yingda Xia
Ling Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: medical multimodal large language model; multimodal interleaved Chain-of-Thought (CoT); tumor analysis
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Accurate tumor analysis is central to clinical radiology and precision oncology, where early detection, reliable lesion characterization, and pathology-level risk assessment directly guide diagnosis, staging, and treatment planning. Chain-of-Thought (CoT) reasoning is particularly critical in this setting, as it enables stepwise interpretation from imaging findings to clinical impressions and pathology-level conclusions, ensuring traceability and reducing diagnostic errors. Here, we target the clinical tumor analysis task and build a large-scale benchmark that operationalizes a multimodal reasoning pipeline, spanning findings, impressions, and pathology predictions. 
We curate TumorCoT, a large-scale dataset of 1.5M CoT-labeled VQA instructions paired with 3D CT scans, with step-aligned rationales and cross-modal alignments along the “findings → impression → pathology” trajectory, enabling standardized evaluation of both final accuracy and reasoning consistency.
We further propose TumorChain, a multimodal interleaved reasoning framework that tightly couples 3D imaging encoders, clinical text understanding, and organ-level vision-language alignment. 
Through cross-modal alignment and iterative interleaved causal reasoning, TumorChain grounds visual evidence, aggregates conclusions, and issues pathology predictions after multiple rounds of self-refinement, improving traceability and reducing hallucination risk.
TumorChain demonstrates consistent gains over strong unimodal and pipeline baselines in lesion detection, impression quality, and pathology classification, and successfully generalizes to the public DeepTumorVQA benchmark. Ablations validate the key contributions of interleaved reasoning and clinical CoT. Clinically, these advances lay the groundwork for reliable, interpretable tumor assessment to support real-world decision-making. To advance safe, explainable, and reproducible multimodal reasoning for high-stakes tumor analysis, detailed information about our project can be found on our project homepage at https://github.com/ZJU4HealthCare/TumorChain.

Published as a conference paper at ICLR 2026
TUMORCHAIN: INTERLEAVED MULTIMODAL CHAIN- OF- OF-THOUGHT REASONING FOR TRACEABLE CLINI- CAL TUMOR ANALYSIS
Sijing Lit.2* Zhongwei Qiu Jiang Liu', Wenqiao Zhang-1, Tianwei Lin Yihan Xie Jianxiang An Boxiang Yun Chenglin Yang'. Jun Xiao Guangyu Guo 2.1 Jiawen Yao?, Wei Liu?, Yuan gao Ke Yan?, Weiwei Cao² Zhilin Zheng". Tony C. W. MOK². Kai Cao", Yu Shi", Jiuyu Zhang Jian Zhou', Beng Chin Ooi', Yingda Xia Ling Zhang²
"Zeejiang University, DAMO Academy, Alibaba Group, *Hupan Lab, SSanghai Institution of Pancreatic Disease "Shengjing Hospital of China Medical University, "Sun Yat-sen University Cancer Center
{suzylee, wengiaozhang]er ju edu.cn cn, (qiuzhongwei. qzw, yingda. x a @al bbaba inc. com
ABSTRACT
Accurate tumor analysis is central to clinical radiology and precision oncology. where early detection, reliable lesion characterization, and pathology-level risk assessment directly guide diagnosis, staging, and treatment planning. Chain-of- Thought (CoT) reasoning is particularly critical in this setting, as it enables step- wise interpretation from imaging findings clinical impressions and pathology- level conclusions, ensuring traceability and reducing diagnostic errors. Here, we target the clinical tumor amalysis task and build large-scale benchmark that op- erationalizes a multimodal reasoning pipeline, spanning findings, impressions, and pathology predictions. We curate TumorCoff CoT-labeled VQA instructions paired with 3D CT scans, with step-aligned ra- tionales and cross-modal alignments along the "findings impression + pathol- ogy" trajectory, enabling standardized evaluation of both final accuracy and rea- soning consistency. We further propose TumorChain, a multimodal interleaved reasoning framework that tightly couples 3D imaging encoders, clinical text un- derstanding, and organ-level vision-language alignment. Through cross-modal alignment and iterative interleaved causal reasoning, TumorChain grounds visual evidence, aggregates conclusions, and issues pathology predictions after mul- tiple rounds of self-refinement, improving traceability and reducing hallucina- tion risk. TumorChain demonstrates consistent gains over strong unimodal and pipeline baselines in lesion detection, impression quality, and pathology classi- benchmark.
large-scale dataset of 1.5M
fication, and successfully generalizes to the public DeepTTumorVQe Ablations validate the key contributions of interleaved reasoning and clinical CoT. Clinically, these advances lay the groundwork for reliable, interpretable tumor To advance safe. explain-
assessment to support real-world decision-making able, and reproducible multimodal reasoning for high-stakes tumor analysis, de- tailed information about our project can be found on our project homepage https://githu..oo///U4teee CeteaaatiCaeeeuuuohhhaaall lumorChain
INTRODUCTION
Although large vasion-language models (LVLMss) have achieved considerable success in general healthcare domains (Yang et al. 2024 Lin et al. 2025) Xu et al. [0255] Lai et al.) 2025). their application to multi-modal clinical understanding and reasoning remains challenging, especially in
"Equal contributions. The work was done during Sijing's 's internship at DAMO Academy. "ooresponniig author.

---

Published as a conference paper at ICLR 2026
a,
Task1 Localization
Task2 Lesion Attributes
Boundary
Esophagus
Organ stomach located
Seguenttttion Rerognition
Stomach
<Thnio_... <iThink 10.08, 0.27. 6.74, 0.87)
Shape
Offers (ed strenglian)
Liver
Deasity
Please confrm the distibution hepatocellular carcinoma.
Count
Colon
Pancreas
Tumor
5 Organ
11+ Atributes
Task3 TNM Pathology Prediction
egg. classification (Regional Lymph Nodes)
b.
Bossed CT imagn, please aradyce palient
lymph metastasis and provice clear conclusion.
anndings
LSM COTVA VOA
<Thinlo-Supickka lymph rodes are otsenved the
TUNORCHAIN
4 Tasks
retropan prnpperreddioduudeee group, around the common hepalic arlery. Ther marphtooy and location suggest high comelation with lumkr... <TTinkk> Condusion Lymph node motastas's is presont.
Shape
m Impression
Density
Task4 COT Report Generation
The question requires greater attentoon pancreas on the CT scan sorgan feature?
Boundary
Please analyze spectfc manttsiations varlou anatomical image. Do you think this polieet hes panceellic Iesio?? Please provide your analysk procasa and predicted condusion
pancreas
Cyst
Tumor
...
Inflammation
"Thinkc First, looking al the whole pancreas, an Isodense mass meesurin appraximately 4.3 4.3-3.0 om is visibie.. Second, examining the pancrestic sections: the pancreatic head shows... ; the pancraatic body... the pancreatic tail... Third. observing the perpancreatic structures... Based on the abovo findings, the imaging festures srongyy suggeoi a pancreatic malignancy.... </Think> Conclusion: There is a pancreatic lesion.
Paahooogy
Benign
Identification
cancer
Figure I: (a) TumorChain focuses on tumor analysis of the five major organs in the digestive
system. (b) We establish a corresponding medical reasoning logic chain from radiology findings to radiology impressions to pathological conclusions. (c) The system tackles four diverse tasks across 1.5 million multimodal VQA instructions, enabling advanced clinical reasoning.
cancer discovery and risk assessment QQu et al. cologists need to identify suspected tumors on radiological images, distinguish benign from malig- nant lesions, and assess cancer risk and TNM staging (Amin et al. 2017)) combined with pathology report and other multidimensional patient features to perform treatment planning. Recent medical LVLMs (Med-LVLMs) primarily focus on study-level report generation. Nevertheless, these models exhibit notable limitations when applied to clinical oncologic scenarios: (i) Limited Tumor-centric Specialization. Current Med-LVL.Ms are optimized for broad clinical tasks (report generation, clas- sification, coarse detection) (Bai et al. 2024; Xin et al., 2025 Chen et al., 2025b), but fall short in oncology-specifie workflows: they do not reliably connect radiologic findings to pathology- endpoints (TNM stage, nodal metastasis, risk stratification), and thus remain insufficient in clinical oncologic decision-making. (ii) Scarcity of Diverse and Tumor-specifie Datasets. Tumor-related data are scarce and difficult to collect in existing medical datasets such as CT-RATE (Hamamci et al., 2024b) and 3D-RAD (Gai et al.) (025). which are designed as knowledge-constraieg choice and short-text QA tasks and rarely support multi-granular analysis of individual cases. To this end, fine-grained visual representations of organ substructures and lesions are often misaligned with textual reasoning (Pan al. 2025), increasing the risk of erroneous or fabricated medical con- clusions. (iii) Insufficient Reasoning Depth. Most Med-LVLMs (Qiu et al.) 20222] Wu et al. 20255 Xu al. 20256) are restricted processing 2D medical images and rely on single-step reasoning for downstream tasks. In 3D radiologic scenarios, the greater structural and informational com- plexity makes single-step reasoning inadequate for multi-stage clinical inference. Additionly, the construction and assessment of reasoning chains remain insufficient.
2025) based on medical images. For example, on-
multiple-
To address these challenges, we build large-scale, multi-institutionall dataset targeting five ma- jor digestive organs (Fig. [p) [nd and present a tumor-centric multimodal Chain-of-Thouge (CoT) framework that operationalizes the full clinical pipeline from radiology findings and impressions to pathology-level diagnosis (Fig. [b).
(i) Dataset. We collect multi-institutional 3D CT images from patients with pathology-confimee tumors, each paired with the corresponding clinical radiology and pathology report . We also de- signed a multi-agent, knowledge-grase graph -ugih- engine that converts original reports into four CoT VQA tasks localizatii,,,, lesion attributes, TNM pathology prediction, and CoT report genera- tion-yielding the largest known multimodal tumor-related dataset TumorCot-1 15M with approx- imately 1.5M CoT VQA instruction (Fig. (c).
A staging system describe the amount and spread of cancer in patient's body, using TNM from AICC.
2

---

Published as a conference paper at ICLR 2026
(ii) Benchmark. We introduce standard evaluation protocol that extracts "subject Srubject-relation- relation-object" triplets (e.g. "pancreatic scoring with an LLM-basee evaluator. It measures the accuracy of abnormality detection and the log- ical correctness of downstream conclusions from findings to impressions to aatholoy-proviinee comprehensive assessment of clinical relevance and overall performance of Med-LVL.Ms.
maaleoeerdd-mllggaattt tumor") ) from CoT chains and performs stepwise
(iii) Model. Technically, we propose TumorChain, a topology- hybrid-model interleaved reasoning framework that integrates clinical priors and organ topology. As tumors frequently affect sub-structures and surrounding organs or distantly metastasize, TumorChain performs multi-round interleaved reasoning: the LLM ingests global tokens and the task prompt to surface intermediate "thoughts" and candidate organs; a segmentation expert returns masks and ROI-level tokens for the indicated regions; the augmented tokens are fed back the L.LM and the loop continues until no new ROIs emerge, producing progressively refined CoT chain that fuses global context, local evidence, and task intent. To better optimize this framework, we adopt a collaborative hybrid-model joint training strategy: segmentation expert produces spatial organ masks, lightweight abnormality classifier built on local organ features enhances the visual encoder's organ-level abnormality dis- crimination, and the LLM performs multimodal integration and high-level clinical reasoning. This design enables topology-awas global local fusion, reduces hallucinations, and improves end-to-end tumor assessment from findings impressions to pathology.
The main contributions of this paper can be summarized as follows:
Clinical Tumor Reasoning Formulation: We formulate the clinical tumor analysis task into a complete reasoning pipeline from radiology findings - study-level impressions + pathology predictions. This pipeline not only covers the core oncologic workflow, but also ensures traceability and interpretability, thereby reducing diagnostic errors.
We design construct a large CoT-annotated VQA corpus (1.5M instances) for tumor understanding across five digestive organs, together with CoT evaluation protocol that measures stepwise clinical reasoning.
Tumor CoT Data Engine and Evaluation:
iiteractive-valddate data engine and
Interleaved Multimodal Reasoning: A medical multimodal interleaved-reasoning framework ChamirChain) with hybrid-mode collaborative optimization (i.e., segmentator, an abnormality classifier and an LLM) that achieves organ-level, global- multimodal aligmment for fine- fine-grained 3D tumor analysis.
Extensive experiments across downstream tumor tasks show consistent gains on the TumorCoT-1. 5M test set and strong generalization to the public DeepTummrr VQA (Chen et al.) (025a) benchmark. Our work provides actionable insights to develop multimodal foundation models in high-stakes clinical tumor analysis.
Substantial Performances and Insights:
2
TUMORCOT-1.5M DATASET
Segmentation Recogritiin
2.1 DATA COLLATION AND ORGANIZATION
Density
Shape
Tumor
Lesion Attributes
Most public medical datasets focus on general health- care scenarios, featuring simple multiple-choice or short- answer formats lacking stepwise reasoning chains or region-specific causal annotations. To advance reliable tu- mor analysis with LVLMs, we curate TumorCoT-1, 5M, a large-scale scale dataset centered on comprehensive tumor analysis in 3D CT, encompassing the entire clinical work- flow from radiological findings and impressions to patho- logical prediction.
Meta-
Node
ThIM Stage
10.15h
60.01%
stasis
prediction
Liver
CoT Report
Boundary
Pancreas
Colon
18,712
zation 11.05%
Others
Esophagus
Organ
Tumer
Figure 2: TumorCoT-1- 5M statisties.
For data collection, we collaborated with multiple medical institutions to collect 3D CT scans from patients with tumors originated from five major digestive organs (liver, pancreas, stomach, colon, includes approximately 41,059 CT images with corresponding captions, 10,708 radiology reports, and partial pathology reports. During the data partitioning stage, we strictly followed a petient-level protocol to divide the dataset into training and test sets at a ratio of 9:1, ensuring that there no overlap between the sets and preventing any data leakage.
and esophagus). TumorCoT-1-
3

---

Published as a conference paper at ICLR 2026
Row Data Processing
Standardized Report:
Diagnostic Knowledge Graph
Findings- EmpressionnCoxxluiion
R Attributes:
Leston Positton, Density, Shape, Count,
Boundary,
3D CT Image
Segmentation Organ Mask
Pathology Report AUributes:
Export
TN M Staging, Pathologg Conclusion
Imuge Attributes:
Organ Volume, Organ Location Bounding
CT and Pathology Report
Box, Tumor Location Bounding Box
Extractor
Features
CoT Data Building
Finding Pathology Chains
Par hiza
Structured Features
CT
Simple Col' Task:
Fy Imp.
Localigttion Lesion Attribute Asalysis
Pathotogy
CoT Reasener
CT Reasoning Chains
Logic Calibrator
Summ mammmmrizer
Clinical Task:
TNM Pathology Prediction, CoT Report Generation
Figure 3: The overview of the interactive-validate CoT Data Engine, which is system of multi- agent collaboration, including the procedure of raw data processing, diagnostic knowledge graph importing, and CoT data building and validation.
To fully leverage tumor CT data from diverse sources and formats, developed an interactive- validated CoT data engine to construct TumorCoT-11 5M. In addition, panels of radiology and pathology experts conducted rigorous reviews of the reasoning chains(details see Appendix E.5 supplementing missing links, and ensuring logical consistency and traceability at every step.
Ultimately, as shown in Figure 2 TumorCoT-1. 5M comprises 1,497,8 7,888 CoT-VQA pairs, cover- ing four key tasks in tumor analysis: (i) Localization, (ii) Lesion Attribute Analysis, (iii) TNM Stage Prediction, and (iv) CoT Report Generation. Each type is further divided into subtasks based on organ substructures or tumor grades, with tasks ranging from simple to complex both multiple-choice and open-ended formats. Every sample includes traceable reasoning process and summary. This organizational framework markedly improves the model's reasoning interpretability and cross- cross-modal consistency.
2.2 COT DATA ENGINE
We designed an interleaved-validatin CoT data engine construct TumorCoT-1. 5M.. Guided by an expert-level knowledge graph, we generated diverse fine-grained CoT-VQA samples across three major steps in tumor analysis: ROI localization, CT interpretation, and pathology prediction. As shown in Figure 3 the data engine comprises the following 3 procedures with 6 agentic experts (detailed in Appendix E and details of the 6 experts in Appendix E.
(i) Raw Data Processing:
This process includes two expert agents: the Segmentation Expert Model (TotalSegmentator Wasserthal et al. (2023)) and the Structured Feature Extractor (Qwen3- 235B-A22B (Yang et al. (0055)), aiming to clean data, standardize terminology, obtain organ masks(Appendis E.4), and extract structured features from CT and pathology reports.
(ii) Diagnostic Knowledge Graph-Driven Prompt Engineering: In collaboration with radiologists and pathologists, we construct diagnostic knowledge graphs (KGs) covering five major organs (The KG for 5 digestive organs is shown in Appendix E.2). During reasoning chain construction, relevant nodes and relationships are retrieved and then authoritative clinical guidelines integrated into the LLM prompts to structurally constrain the model to follow professional medical standards, thereby ensuring traceability and high reliability of the logical chain.
This process includes agents: CoT Reasoner, Logic Calibrator, and Summarizer. To fully utilize the strengths of different LLMs, three agents are assigned distinct models. Additionally, we use only the textual content from the reports as their sole source of information, thereby preventing these models from making subjec- tive assumptions about complex modalities such CT imaging. CoT Reasoner uses GPT-4o-mini for its strong language and medical capabilities, processing structured CT report features and in- tegrating external KGs to link radiological findings with impressions. Logic Calibrator, based on Claude3.5-Hain automatically validates reasoning accuracy. If potential issues are detected, the system randomly applies two prompting exxpandin the organ region providing sus-
(iii) CoT Data Building and Cross-Review Mechanism:

---

Published as a conference paper at ICLR 2026
pected cause- to guide the reasoning model in re-evaluating its chain. Summarizer employs GPT- 5-mini to process structured pathology reports, aggregate and verify reasoning chains, and generate QA pairs and CoT-formatted reports when chains align with pathological conclusions; otherwise, it triggers upstream re-reasoning. The agents' prompts are shown in Appendix E.3
2.3 CLINICAL COT EVALUATION: TUMORCHAIN EVAL
Existing evaluation metries like accuracy, semantic similarity metric (Rouge-L (Li 2004),
Veddntam et al. (2015) Ra RaTEScore Zhhoo et al. (2024), etc.) ignore the correctness of clinical logic and step-wise evaluation of the CoT reasoning process. To address this issue, we propose -uloorChiinEvvl, novel evaluation framework with metric CoT, specifically for oncology- related CoT reasoning, which provides a fine-grained and interpretable scoring system based on reasoning logic chains. In this approach, the CoT reasoning process of the ground truth and pre- diction is extracted into three reasoning chains, all of which can be represented as subject-relation- object triplets: (i) Finding Chain: Describes the primary radiology findings, which are independent facts and do not involve higher-order reasoning. (ii) Impression Chain: Summarizes intermediate- level clinical impressions based the Finding Chain, involving simple reasoning conclusions from multiple findings. (iii) Long Reasoning Chain: Combines multiple findings and impressions to generate higher-order reasoning, such conclusive diagnostic hypotheses with strong logical in- ference. Each type of chain is evaluated using a detailed set of scoring criteria, and the scores are calculated by using GPT-4 with scoring guidelines. Finally, the Col, score is computed by weighted sum of Syc, Sic, SLRC, which are the FC score, LC score, and LRC score, respectively. The detailed scoring process and criteria for each reasoning chain can be found in the Appendix
METHODOLOGY
3.1 OVERVIEW OF TUMORCHAIN
Before presenting our method, we first introduce some basic notions and terminologies. The frame- work F(.) of TumorChain is shown in Figure 4 which consists of five key modules: 3D vi- sion encoder (-).), an organ segmentation expert Seg(-), an auxiliary classification model Cls(-), a multi-layer perceptron (MIP) projector P(.), and a LLM L.C((.))) Given the CT volumes Vet E R xD with task prompt text Trask as input, the TumorChain outputs the response Rest of the interlcaved mulli-modaa chain of thought (CoT), which simplifies as below:
F(Vot: Teask)
(.) - Seg(.) - Cls(.) + P() - LLM(.) - Reot
TumorChain
Global&Local
Visual Alignment
Interleaved Multi-modal Inference
Global & Local Visual Alignment. For global visual tokens, the 3D encoder & takes the Ves E Hxx as input that encode 3D CT volumes to a series of vision tokens as T, = (Ve.. Then, all vision tokens are aligned into LLM space by the projector Tg P(Tr), where Tg denotes the global tokens and has a size of Lg x K (Lgg tokens and embedding dimension K).
For local visual tokens, the organ segmentation expert Seg(.) first segments the organ mask as Morgan = Seg(V) E H to provide fine-grained location for interlaced reasoning. Accord- ing to the task prompt, TumorChain identifies the target task organ mask Mtask from Morgan by matching the organ in the task text and extracts local vision tokens 7 as
=
Menck) s.l. Mrask A(Morgan, Trask),
(2)
where A(-) denotes the organ matching operation and I(-) represents the operation of extracting local token Ti of size L x K according to mask.
Besides, to enhance the 7, learning, we introduce the auxiliary classification model Cls(-) that pro- jecting 7. into two classification logits, which can be formulated as = Cls(n), where the Cls(.) distinguish normal or abnormal for each local organ, thereby significantly enhancing the encoder's discriminative capability. When the visual features and task prompts are jointly input into the LLM, the model's 'ss attention key information and anomalous regions be further enhanced.
Interleaved Multi-modal Inference. After obtaining all vision and text tokens, they are combined into interleaved tokens serve as the input for I.M,, to further generate the inference response Rooo
5

---

Published as a conference paper at ICLR 2026
Segmentation Expert Model
Organ Mask List
colon
Liver
Esophagus
spleen
CT volumes
Va
<ihelettver Chain
Auxiliary Classification Model
Spleen Chain
Vision
</Thinks
Encoder
The answer mentioned spienomegally You need to pay attention the spleen.
<knswer-hherafree
The question equires attention liver.
diagnesis
Iver dirhoss..
Rest
T
Re
r
R
7
MLP
Global Tokens
Large Language Model
Qustion:
Does CT scan
show any lesions the liver?
task
Text Input
Step
Step
II
Step III
LastStep
The overview of TumorChain. Given CT volumes and task prompts, TumorChain ana- lyzes the task requirements and performs iterative interleaved reasoning by integrating a segmenta- tion expert model for ROI localization, auxiliary classification for local feature enhancement, and organ-guided iterative reasoning for clinical tumor analysis.
Figure 4:
of CoT as
Root CCM((T)) s.t. Tin [Tg Trask: T
where Ten indicates all input tokens of LLM, and is the combination of global vision tokens Tg. task text prompt tokens Trask: and task target organ tokens of with its text pair T Here {l..... .} indexes the O potential related or ROI organs/suborgans reasoned by L.L..
Through the aforementioned process, TumorChain achieves interleaved alignment of global visual features, task text, and task-specific local visual features during single inference by leveraging the collaborative interaction between large and small models.
3.2 ORGAN-GUIDED ITERATIVE INTERLEAVED REASONING
Clinical 3D tumor analysis demands organ identification, fine-grained tumor localization, and reli- able ROI evidence for LLM reasoning, making single-pass inference insufficient and complicating multimodal self-validation. To achieve a reliable clinical reasoning chain of "finding + impres- sion pathology prediction", TumorChain adopts organ-guided Iterative Interleaved Rea- soning (ITR). IIR uses each L.M output to refine ROI selection, feeds the resulting ROI features back for the round, and repeats, which enables multi-round interleaved reasoning with built-in self-validation that maintains visual-text alignment across iterations. For clarity, we abstract the IIR process into three steps:
Step I: Initial Round of Reasoning. The initial reasoning step takes the encoded global CT vol- ume tokens Tg. and the medical question text Taakk as the inputs to the LLM:
R R LLM((Tg: Trask);
where RI represents the initial diagnostic result output by the LLM.
Step H: Interlcaved Self-reflection and Organ Localization. (i) Keyword Extraction and Organ Identification: According to the initial output R and Equation determining the local ROI mask Mt obtain local vision tokens 7/. (ii) Augmented Prompt Construction: After generating organ- specific tokens based on the extracted organ features, then construct augmented prompts as:
T ("The question requires greater attention to", orran))
(5)
where organ} represents the identified ROI from R! (iii) Combining Inputs for the Next Itera- tion: Then, the original question Teask, the initial answer R and identified organ tokens TI are combined to form new input for the LLM:
[Tg, Trask; Roo Tit 7t
Tin

---

Published as a conference paper at ICLR 2026
Step III: Iterative Causal Reasoning After completing the previous self-reflection, the frame- work continues to perform iterative reasoning based on the newly generated answers. If additional relevant organs are introduced, the ROI extraction process is repeated sequentially obtain the fea- tures of all organs involved in the reasoning chain, supporting further verification. According to Equation 3 6 this iterative process be formulated
R LLM(TTg Trask: Rist T TT)
For example, in cases where cirrhosis of the liver is suspected and the imaging findings indicate splenomegaly and altered hepatic lobe proportions, IIR automatically extracts the organ tokens for the spleen and appends pertinent prompt before it feeds both the original question and the previous answers into the LLM. The framework undertakes multi-round causal verification of each organ in the answer until all relevant organs have undergone feature extraction and causal reflection.
3.3 HYBRID-MODEL COLLABORATIVE OPTIMIZATION STRATEGY
Typically, a 3D CT scan contains over 100 slices, each with resolution of 512 512 pixels. In comparison to 2D image-caption pairs (e.g.. resolutions of 224 or 384). this makes challenging achieve fine-grained alignment between the visual representations and textual descriptions. To address the above challenges and further optimize TumorChain, we employ a Hybrid-model Collab- orative Optimization (HCO) strategy. The core concept of HCO lies in the collaboration of models at different scales: the segmentation model localizes ROIs, continuously providing precise local features for reasoning. The classification model ensures discriminative local feature learning during optimization, enabling the visual encoder to effectively distinguish abnormal from normal patterns, preventing subtle anomalies from being overshadowed during LLM training. The LLM integrates reasoning outcomes and leverages the segmentation model for iterative decision-making.
During training, the classification model and L.M are jointly optimized via loss functions. The training loss Litoaa is:
1 N Σ log Porn (n-1).
1 M
Liotal
Σ (P(y)n),
(8)
where N represents the output text length and M represents the sample number. T; E RN a is a loss weight. y and ý are the prediction and ground-truth of classification (normal/abnormal) for local organs. During inference, the segmentation model utilizes the ROI organ locations provided by the LLM to extract ROI visual features, which are then fed back into the LLM for self veff-verificttion.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Model Overview. Both TumorChain-BB and TumorChain-27 7B employ the pretrained LLM back- bone of Qwen2.5-VL- VL-33BBBBBBBB the large language model component, integrating M3D as the vision encoder (Bai et al. 2024) to enable 3D) CT feature extraction. Full architectural details and training configuration are provided Appendix [.I.]
Baselines. To comprehensively evaluate the performance of TumorChain, we compare it against a diverse set of baseline models, including seven open-world LVLMs (e.g., Claude (Anthropic)
2024), Gemini 2.0 (DeepMind) 2025). GPT-5 (OpenAl) 2025). Qwen2.5-VL. (Bai et al. 2025
VL- (Chen et al. 2024). MiniCPM-V4.5 Yu et al. (2025b), LLaVA-CoT (Xu et al. 2025a )
three 2D-based Med-LVLMs (c.g., HealthGPT (Lin et al. 2025). Lingshu (Xu al. 2025b
MedVLM-RI (Pan et al. 2025 ), and two 3D-based Med-LVLMM ( (c.g., RadFM (Wu et al.
2025
M3D-Phi-3 (Bai al. 2024)).
Benchmarks, All models are evaluated on the test subset of TumorCoT-1.55 partitioned via a 9:1 split. Evaluation is conducted based our definitions of four main tasks and twelve subtask types.
Evaluation Metrics. We developed diverse protocols focusing on the analytical process and con- clusions within the reasoning chain. All models are guided generate CoT reasoning chain, which
7

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of TumorChain with other LVLMs on TumorCoT benchmark. Bold and
underlined text indicates the best and second-best performance, respectively.
Position
Lesion Attributes
TNM Prediction
CoT- Report
Model
Avg.
Organ Pos.
Tumor
Seg. Loc.
Shape
Boundary
Deusity
Count
Others
Tumor
Node
Met.
Commercial Models
Claudeš-Haile
32.61
33.05
52.38
58.51
50.00
50.74
37.51
50.95
59.19
56.59
43.40
33.21
46.51
Gemini220--Ri O-Fllshh
43.27
17.69
46.81
35.97
38.22
39.1
40.22
52. 86
34.03
42.21
49.06
56.08
41.29
GPT-5-Mini
44.18
37.73
44,96
47.72
62,58
57.13
64.85
66.67
34.25
47.40
49.60
62.02
51.59
Generalist Models
Dwen2.5-VL-778
29.57
32,52
27.41
43.96
57.51
49.
46.73
58.02
34.17
57.49
51.15
60.54
44,04
IstemMVL-225588
29.10
27.67
33.50
49:96
51.45
49.48
42.68
51.70
36.16
35.07
48.09
52.18
42.25
MiniCPM-V4.5-9R
29.56
44.11
52.61
53.67
50
46.92
44.27
45.98
LLaVA-CoT-IIB
31,63
27.64
37.70
39:44
49.34
49.60
37.90
55:48
48.20
47.91
49.23
44.81
44,91
20 Medical Models
HealthGPT-3.8B
30.58
22,80
45.87
51.35
45.77
47.27
31.24
40.13
45.68
49.02
39.69
39.20
40.72
Lingshu-7H 7H
25. 28
32.58
24.90
51.70
60.61
53.40
45.21
63.63
30.35
42.79
56.49
54.50
45.10
McdVLM-R1-28
19.37
30.38
3D Medical Models
RadFM
13.86
15.97
15.23
21.25 .5
49. 84
21.07
19:34
24.15
49.58
45.00 02
28.47
21.34
26.32
M3D-Pth-33 Phi-3-4B
32.39
28.28
22.69
23.93
42.74
30.49
27.19
52.35
44.82
30.80
35.86
34.79
32.84
Our Models
Tumor
99.93
73.57
81.04
76.34
76.78
TumorChaln-77
99.97
97.57
86.88
82.28
84.52
85.05
86.20
86.57
88.83
61.63
71.07
82.36
84.41
w/o CoT. IIR
99.9
97.45
78.95
75.78
74.60
80.18
82.82
78.75
84.37
59.29
66.
79.83
79.90
sofo CoT
99.91
97.10
82.75
78.43
82.57
68.1
w/b IIR
99.87
97,55
79.36
79.52
74.62
81.46
80.60
79.20
84.65
60.32
65.87
81.06
80.34
is scored using our proposed TumorChain-Ever For conclusion, accuracy is calculated for multiple- choice tasks (e.g., segment localization in Task 2 and Task 1). For open-ended QA tasks, semantic consistency of the conclusions is assessed using GPT-5 and semantic accuracy is computed. Our evaluation prompt design and implementation details are provided in Appendix
4.2
MAIN RESULTS
CoT-Conclusion Analysis. We evaluated both the reasoning process and the correctness of con- clusions for our four proposed tumor analysis tasks. The experimental results for conclusion ac- curacy are presented in Table lines to 13, and organ-level accuracy is displayed in Figure 5p. The main observations are as follows: (i) Superior Performance. TumorChain-77 7B achieves state- of-the-art results across all subtasks, with an average aceuracy of 84.41%, substantially outper- forming all baseline models and. TumorChain-3B also demonstrates strong performance, high- lighting the effectiveness of our dataset and model design in complex tumor analysis scenarios. (ii) Limitations of Existing Med-LVLMs. Current mainstream 2D or 3D medical models do not surpass, and in some metrics even underperform, compared to general-purpose or commer- cial models in tumor analysis. This indicates the limited generalization and adaptability of ex- isting Med-LVLMs, especially in complex, disease-specific settings, and in unseen datasets. No- tably, some 3D medical models perform significantly worse than 2D models CT analysis, vealing critical bottlenecks in spatial understanding and multi-attribute analysis of large-scale CT scans. (iii) Effectiveness of Multi-granularity Benchmark Design. The performance of our mod- els across various subtasks demonstrates the multi-level and ine-grained nature of our dataset. Near-perfeet accuracy basic localization tasks and relatively lower scores in more challenging patholog- ical prediction and report generation emphasize the comprehensive and progressive challenges posed by our benchmark, providing valuable insights for fine- grained AI development in the domain of tumor anal- ysis and diagnosis.
Table 2: TumorChain-Eval with other rea- soning LVLMs on Tumo CoT.
Reasoning Models
Sec
Sic
St,0
Culi
Commerciaal Models
Claude3-Haiee
59.31
43.66
30.80
43.21
Gemini2.00 F.0shlash
63.10
57 60
45.12
54,28
GPT-5-Mini
55.09
Co CeT-Rneag Analysis. We compare CoT reason- ing ability with other reasoning models Table 2 via the proposed CoT, metric (Section 2.3) TumorCCainn 7B demonstrates high accuracy in generating trace- able reasoning (FC, and complex reasoning chains (LRC), achieving CoT, scores of 58.33 (see (i: for example). Its clinical reasoning ability outperforms
Open- OpeneSowrce Medlels
Qwen2.5.V1.n9
49.15
22.89
12.09
26.45
1nlem/V--2.5.33
66
MinCCPM-VLLL V1,5-9B
60.51
58.52
45.14
53.89
HcakhGPPT-3.B
49.64
49.68
37.15
44.65
Lingshu-7B
52.51
31.93
16.46
31.92
MedVLMMRR1228
40.25
36.05
21.23
31.38
TumorChain-31
61.75
49.99
35.68
47.98
MmmorChain--B
62.37
62.60
52.57
58.33
8

---

Published as a conference paper at ICLR 2026
a.
Boundary
Shape
Let
C.
Question:
Plaso carklyl oadie to queili canderrens checif aaos micilamraat pxomaii ruccE mage Di thi hat patount cialonl floe provite pirouate reprr induding your nuivica expman and prodiced oor chatin
Pinding Chains
Goncrated Answor
Entre panciaa 80 Eirpsaa the
ppncratic head fress meat
barter pasrical setear aniiiee
goocett bocyy shows no desomnitty
- 0.5
Poe.
stwered.
harmatt ipgressttion Beiii callloni
ey
di diverem bu ner wegeet
oulire
EsT
hii sagari uga sold sumon ludor with isagia
Scomenn that, shows turastoo
aprionsity tdee deedd of thc cneteae
3. (Gaddaruryy fids instomentt
abeooval concaed Meppartllonnn
hypiddenee
paredhyruil otfice
b.
Nole
Pancreas
sur man, tigiigh the lette purearryy bookove
Impresston
1-9 repestt malggangg
These fiud dudttt of turor with locil
dital panoraa
pinoreate bioh bot inceut mi that bokee is
ferpecre atrestanes
A U. mrars, Orgin reshorant
ciated the corene te observe including stlery Incheeeed the decce rling part of Baccsss retes.
he trant contuct with uperie returees arties callic nether tomnctionn nar satten As seppiouu
Ferlww sorelt
Long Reasoning
Nomal
Katodie skowe foding semerding chandtrruto of sold laier thew connitane with paorese ate dacid sociion
thire
Cylo Baw Kaa of nu ligoont of drandterooo of cancnome of the paromatic head
Buopphagus
Figure 5: (a) Ablation on classification loss weight a. (b) Model performance on five organs of TumorCoT. (c) Workflow of reasoning chains extraction and evaluation for task 4.
the general LVLMs (e.g., Qwen2.5-VL-77B and Med-LVLMS (e.g.. Lingshu-7B), which are trained on more medical data with diverse modalities. Furthermore, TumorChain shows better pxunrfor- mance than commercial models (Claude3-Haikee Gemini2.0 2.-iiii... Nevertheless, the CoT, score is slightly lower than the performance of the latest GPT5-mini, demonstrating its strong capabilities in the medical domain.
4.3 ABLATION AND GENERALIZATION STUDIES
Effect of CoT and HR. To comprehensively evaluate the impact of our proposed CoT reasoning and Interleaved Iterative Reasoning (IIR) approaches, we conducted systematic ablation studies. As shown in Table I lines 13 to 16, the combined use of CoT and IIR yields substantial performance gains across all subtasks, with an overall accuracy improvement of 5.64% compared to the baseline. This highlights that focusing on ROI region features markedly enhances model performance in 3D image analysis. Models employing CoT alone consistently improved results on most tasks, notably advancing clinical interpretability without sacrificing analytical accuracy. We further assessed infer- ence latency after incorporating the IIR mechanism. The segmentation model incurs only a minimal increase in inference time (2.51 seconds per sample), while delivering an approximately 4% accu- racy boost. The significant benefit of emphasizing ROI features far outweighs the slight time cost, strongly validating the reliability and effectiveness of our method for intelligent tumor analysis.
Impact of Classification Loss Weight. We systematically investigated the model's 's performance un- der different classification loss weight settings, as illustrated in F (a). The results indicate that appropriately increasing the classification loss weight (e.g., o=1.0) leads to significant performance improvements across multiple tasks, achieving the highest average accuracy of 84.41%. %. However, setting the weight too low or too high (such as a=0.0 or a=155) results in diminished performance, further demonstrating the essential role of loss weight adjustment the collaborative optimization of hybrid models. Detailed results are available in Appendix D.3
Effect of finetuning by T01.550T train set. To more directly demonstrate the inddpen- dent contribution of our dataset to model fine-
Hodel Fine-Tuning Performance (Beforw/After)
Averape Score of Four taokss
40.4
75.8
34.
tuning, selected three representative baseline models for finetuning on our TumorCoT- 1.5M training set: the 3D medical LVLM M3D-Phi-3
4B, the 2D medical LVLM Lingshu-7B, and the 2D general-purpose purpose model Qwen2.5-VL-7B.. Fig-
ure 66 shows the performance changes of these baseline models before and after finetuning (see Appendix 3 for detailed results). The results indicate that all bascline models achieve signifi- cant improvements after finetuning, demonstrat- ing the unique and important contribution of our dataset in the tumor domain. Meanwhile, our proposed TumorChain-7B still substantially surpasses the finetuned baselines, further highlighting the effectiveness of our model design.
Tonttion
Ledd' Adt butee
TMM Reed
CoT Ragort
Avg
Ungahu- performance (balancideee)
Figure 6: Baseline model improvements by finetuning with TumorCoT-L.5M.
9

---

Published as a conference paper at ICLR 2026
Generalization DeepIImorVVQA Benchmark.
Table 3: Results on DeepTumor VQA.
To evaluate the generalization capability of our model and methods in tumor analysis, conducted comprehensive experiments on the latest public tu- mor benchmark, Deep'lumorVQA. As shown in Ta- ble 3 without seeing any data from this dataset, TumorChain-7B achieved an accuracy of 73.30% in lesion recognition and demonstrated substantially better performance both visual and medical reason- ing compared to leading commercial and open-source models. In addition, average accuracy exceeds that of the second-best reasoning model, MedVI.M-RI, by 14.84%. These results strongly demonstrate the exceptional generalization ability and robustness of our approach in open-domain tumor analysis tasks.
Model
Recog,
V.R.
M.R.
Avg.
Commercial Models
Claude3-Hail Gemini2.0-Flass GPT-S-mini
38.70
31.53
34.81
35.01
50.88
33.81
35.06
39.92
49.29
35.57
29.87
38.24
Open-Source Models
Qwen2.5-VD --L-2B
45.31
36.94
37.04
39.76
Lntem
50.06
31.35
44.20
41.87
MiniCPM V4.5 9B
50.14
33.69
42.86
42.23
HeadhGGPT-3888
43.32
Lingshu-7B 7B
50.24
36.19
37.78
41.40
MedVLM-RI-28
56.41
38.27
33.33
42.67
RadPM
50.19
39.82
30.87
40.29
M3D-Phi-3-4B
29.53
23.22
23.70
25.48
TumorChain-778
73.30
RELATED WORK
Medical Large Vision-Language Models. Med-LVLMs such as LLaVA-Med (Li et al. 2023),
Lingshu (Xu al. 20256) and HealthGPT (Lin et al. 2025 have rapidly advanced multimodal clinical decision support by aligning ViT-based 2D image features with language models. Meth- ods like M3D (Bai et al. 2024), CT-Chat (Hamamci al. 2024a) and Merlin (Blankemeier al.) 2024) incorporate 3D CNNS and ViTs to better capture volumetric data (Qiu et al. 2023), enabling more precise analysis of clinical 3D images. However, these Med-LVLMs lack causal reasoning for complex cases, prompting shift toward CoT and knowledge-guiie
reasoning paradigms. Med-
VLThinker (Huang et al., 2025) and MedVLM-R1 (Pan et al., 2025) combine CoT reasoning with reinforcement learning to improve interpretability. while ReasonMed (Sun et al.) (025) optimizes reasoning paths via multi-agent verification. MedResearcher--R (Yu et al.) 2025a) enhances deep reasoning by leveraging knowledge graphs for multi-hop QA sample construction. By emphasizing logical chains, medical vision reasoning models are overcoming traditional 2D and 3D VLM limita- tions and opening new directions for explainable and reliable clinical decision-making We further summarize related work with our TumorChain in three aspects: Medical Benchmark, Medical Large Vision-Langune Models, and Online Interleaved Reasoning. The detailed discussions are shown Appendix
CONCLUSION
We introduce Chain, multimodal framework designed for comprehensive clinical tumor reasoning, spanning radiological findings, study-level impressions, and pathology predictions. We also construct TumorCoT-1 1. 5M, the largest multimodal dataset for tumor reasoning, and propose a benchmark protocol evaluate stepwise reasoning fidelity. Extensive experiments demonstrate TumorChain's 's superior performance across diverse tumor-related tasks, benefiting the advancement of clinical Al in oncology. Furthermore, Appendix G presents case studies and qualitative error analyses useful references for future model improvement and clinical practice.
ACKNOWLEDGMENTS
This work has been supported in part by the NSFC (No. 62436007), the China Postdoctoral Science Foundation under Grant Number 2024M752794, the ZINSF (No. LZ225F22004),, the Key Research and Development Projects in Zhejiang Province (No. 2025C01 128, 2025C01030, 2025C02156), Ningbo Yongjiang Talent Introduction Programme (2023A400GG).
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This study used hospital-held clinical imaging data and was reviewed and approved by the hospital Ethies Committee. Data collection and use complied with applicable laws, regulations, and ethical standards. All data were de-identified and anonymized prior to use, with personally identifiable information removed. Access to the data was restricted authorized personnel. and appropriate safeguards (e.g., eneryption and access controls) were implemented to prevent disclosure of personal information. The data were used solely for the purposes of this research.
REPRODUCIBILITY STATEMENT
To advance safe, explainable, and reproducible multimodal reasoning for high-stakes tumor analysis, detailed information about our project be found project homepage at C Tuaior/al Detailed descriptions of hyperparameters and ex- perimental settings can be found in Appendix D.I
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 2rXiv:2303.088 087744 2023.
Mahul B Amin, Stephen B Edge, Frederick L Greene, David R Byrd, Robert K Brookland, Mary Kay Washington, Jeffrey Gershenwald, Carolyn C Compton, Kenneth R Hess, Daniel C Sullivan, et al. AJCC cancer staging manual, volume 1024. Springer, 2017.
Anthropic. Introducing the next generation of claude, 2024. URL com/ http/new//claase 3- 3- Family]
Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, et al. Health- bench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775, 2025.
Fan Bai, Yuxin Du, Tiejun Huang, Max Q-H Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models. arXiv preprint arXiv:2404.0003 2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 13923,,50 2025.
Asma Ben Abacha, Sadid A Hasan, Vivek V Datla, Dina Demner-Fushman, and Henning Müller. Vqa-med: Overview of the medical visual question answering task at imageclef 2019. In Pro- ceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes. 9-12 September 2019, 2019.
Louis Blankemeier, Joseph Paul Cohen, Ashwin Kumar, Dave Van Veen Syed Jamal Safdar Gardezi, Magdalini Paschali, Zhihong Chen, Bean-Benoit Delbrouck, Eduardo Reis, Cesar Truyts, et al. Merlin: A vision language foundation model for 3d computed tomography. Research Square, pp. 38-3, 2024.
Yixiong Chen, Wenjie Xiao, Pedro RAS Bassi, Xinze Zhou, Sezgin Er, Ibrahim Ethem Hamamei, Zongwei Zhou, and Alan Yuille. Are vision language models ready for clinical diagno- sis? a 3d medical benchmark for tumor-centric visual question answering. arXiv preprint arXiv:2505. 18915, 2025a.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of sounes multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271,, 2024.
11

---

Published as a conference paper at ICLR 2026
Zhixuan Chen, Yequan Bie, Haibo Jin, and Hao Chen. Large language model with region region-guided guided referring and grounding for ct report generation. IEEE Transactions on Medical Imaging, 2025b.
Google DeepMind. Gemini 2.x model family report: Gemini 2.5 pro/flash and gemini 2.0 flash/flash-itt Technical report, Google DeepMind. June 2025. URL
googgleapis com/ wwweepiind-meedee gemigi/mi///// mem.iiiigmmiii. v2 report. Sec-
tion covering Gemini 2.0 Flash and Flash-Lite; Accessed: 2025-09-25.
Xiaotang Gai, Jiaxiang Liu, Yichen Li, Zijie Meng, Jian Wu, and Zuozhu Liu. 3d-rad: compre- hensive 3d radiology med-vqa dataset with multi-temporal analysis and diverse diagnostic tasks. arXiv preprint arXiv:2506. / 147, 2025.
Ibrahim Ethem Hamamci, Sezgin Er, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Muhammed Furkan Dasdelen, Bastian Wittmann, Enis Simsar, Mehmet Simsar, et al. A foundation model utilizing chest et volumes and radiology reports for supervised-level zero-shot detection of abnormalities. CoRR, 2024a.
Ibrahim Ethem Hamamci, Sezgin Er, Chenyu Wang. Furkan Almas, Ayse Gulnihan Simsek, Sev- val Nil Esirgun, Irem Doga, Omer Faruk Durugol, Weicheng Dai, Murong Xu, et al. Developing generalist foundation models from a multimodal dataset for 3d computed tomography. arXiv preprint arXiv:2403. 17834, 2024b.
Xuehai Hc, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint WwXiv22033.102860 2020.
Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjum He, Yu Qiao, and Ping Luo. Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical Ivlm. In CVPR, pp. 22170- 22183, 2024.
Xiaoke Huang, Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou Medvlthinker: Simple baselines for multimodal medical reasoning. arXiv preprint arXiv:2508. 02669, 2025.
Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-rl: Reinforce- ment learning for generalizable medical reasoning in vision-language models. arXiv preprint arXiv:2503.13939, 2025.
Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman, A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1-10, 2018.
Chunyuan Li, Cliff Wong, Sheng Zhang. Naoto Usuyama, Haotian Liu, Jianwei Yang. Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training large language-and-vising assis- tant for biomedicine in one day. NeurIPS, 36:28541-28566 2023.
Wenjie Li, Yujie Zhang, Haoran Sun, Yueqi Li, Fanrui Zhang, Mengzhe Xu, Victoria Borja Clausich, Sade Mellin, Renhao Yang. Chenrun Wang, et al. Cx-mind: A pioneering multimodal large language model for interleaved reasoning in chest x-ray via curiculum-guided reinforcement learning, arXiv preprint arXiv:2508.0373 037333 2025.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Tianwei Lin, Wenqiao Zhang, Sijing Li, Yuqian Yuan, Binhe Yu, Haoyuan Li, Wanggui He, Hao Jiang, Mengze Li, Xiaohui Song, et al. Healthgpt: A medical large vision-language model for unifying comprehension and generation via heterogeneous knowledge adaptation. ICML, 2025.
Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically- labeled knowledge-enhure 1654. IEEF, 2021.
dataset for medical visual question answering. ISBI, pp. 1650-
OpenAl. Gpt-5 system card. attps://cdh.ooenatt wtt. Seatt---ssstee--cr....o August 2025. Accessed: 2025-09-25.
12

---

Published as a conference paper at ICLR 2026
Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-rl: Incentivizing medical reasoning capability of vision- language models (vlms) via reinforcement learning. arXiv preprint arXiv:2502.19634. 2025.
Zhongwei Qiu, Kai Qiu, Jianlong Fu, and Dongmei Fu. Dgen: Dynamic graph convolutional net- work for efficient multi-person pose estimation. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 34, pp. 11924-11931, 2020.
Zhongwei Qiu, Huan Yang, Jianlong Fu, and Dongmei Fu. Learning spatiotemporal frequency- transformer for compressed video super-resolution. In European conference computer vision, pp. 257-273. Springer, 2022.
Zhongwei Qiu, Qiansheng Yang. Jian Wang, Haocheng Feng, Junyu Han, Errui Ding, Chang Xu, Dongmei Fu, and Jingdong Wang. Psvt: End-to-end multi-person 3d pose and shape estimation with progressive video transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 21254-21200 2023.
Zhongwei Qiu, Hanqing Chao, Tiancheng Lin, Wanxing Chang, Zijiang Yang. Wenpei Jiao, Yixuan Shen, Yunshuo Zhang, Yelin Yang. Wenbin Liu, et al. Bridging local inductive bias and long- range dependencies with pixel-mamba for end-to-end whole slide image analysis. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 22738 22737722277---222 2025.
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943-950, 2025.
Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Yu Rong, Wenbing Huang, Qifeng Bai, and Tingyang Xu. Reasonmed: A 370k multi-agent generated dataset for advancing medical reasoning. arXiv preprint arXiv:2506. 09513, 2025.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving trieval with thain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212. 2022.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-bris based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566-4575, 2015.
Jakob Wasserthal, Hanns-Christian Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter, Tobias Heye, Daniel T Boll, Joshy Cyriac. Shan Yang, al. Totalseg- mentator: robust segmentation of 104 anatomic structures in et images. Radiology: Artificial Intelligence, 53522233 2023.
Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Towards generalist foun- dation model for radiology by leveraging web-scale 2d3d medical data, 2023.
Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Hui Hui, Yanfeng Wang, and Weidi Xie. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data. Nature Communi- cations, 16(1):7866, 2025.
Yu Xin, Gorkem Can Ates, Kuang Gong, and Wei Shao. Med3dvlm: An efficient vision-language model for 3d medical image analysis. arXiv preprint arXiv:2503.20047, 2025.
Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2087-2098. 2025a.
Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang. Cheng- hao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, et al. Lingshu: A generalist foun- dation model for unified multimodal medical understanding and reasoning. arXiv preprint arXiv:2506. 07044, 2025b.
13

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint (0Xiv:2500509388, 2025.
Lin Yang, Shawn Xu, Andrew Sellergren, Timo Kohlberger, Yuchen Zhou, Ira Ktena, Atilla Ki- raly, Faruk Ahmed, Farhad Hormozdiari, Tiam Jaroensri, et al. Advancing multimodal medical capabilities of gemini. arXiv preprint arXiv:2405.03162, 2024.
Ailing Yu, Lan Yao, Jingnan Liu, Zhe Chen, Jiajun Yin, Yuan Wang, Xinhao Liao, Zhiling Ye, Ji Li, Yun Yue, et al. Medreseacher-rl: Expert-level medical deep researcher via a knowledge-informed trajectory synthesis framework. arXiv preprint arXiv:2508. 14880, 2025a.
Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang. Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, Bokai Xu, Junbo Cui, Yingjing Xu, Liqing Ruan, Luoyuan Zhang, Hanyu Liu, Jingkun Tang. Hongyuan Liu, Qining Guo, Wenhao Hu, Bingxiang He, Jie Zhou, Jie Cai, Ji Qi, Zonghao Guo, Chi Chen, Guoyang Zeng, Yuxuan Li, Ganqu Cui, Ning Ding, Xu Han, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Minicpm-v 4.5: Cooking efficient milms via architecture, data, and training recipe, 2025b.
Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu et al. Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl. arXiv preprint arXiv:2505.15436. 2025.
Weike Zhao, Chaoyi Wu, Xiaoman Zhang, Ya Zhang. Yanfeng Wang, and Weidi Xie. Ratescore: A metric for radiology report generation. arXiv preprint arXiv:2406. 16845, 2024.
Ke Zhou, Zhongwei Qiu, and Dongmei Fu. Multi-scale contrastive adaptor learning for segmenting anything in underperformed scenes. Neurocomputing, 606395, 2024.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
Appendix
LLM Usage Statement.
Appendix B
Notation Table.
Appendix
Related Work.
Appendix
D
Experiments and Implementation Details.
Appendix
D.
Training Details.
Appendix
D
Semantic Accuracy Metric by GPT5,
Appendix
D.3
Experiment Result Details.
Appendix
P
Validattee-Viidateed COT Data Engine.
Appendix
Data Engine Components Details.
Appendix E.2
Diagnostic Knowledge Graph.
Appendix
E.3
Prompt Design of Data Engine.
Appendix
Multi-Organ Masks and Merged Organ Masks.
Appendix
Formal, quantitative evaluation of the CoT data engine by expert elinicians.
Appendix
CoT
Evaluation Details: TumorChain-Eve Eval..
Appendix
Stage 1: Extraction of Medical Entity-Relation-Enie Triples.
Appendix
2
Stage 2: CoT Evaluation Protocol.
Appendix
G Cases Study.
15

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
In this work, we employ LLMs in three strictly controlled ways. First, LLMs are utilized to rephrase the source data during dataset construction in the CoT data engine D thereby constructing high-quality training dataset based on the original information while preserving the clinical context. Second, leverage LLMs as experts to evaluate model against baselines (see Appendix F)) following clearly defined criteria for consistency and fairness. Third, we employ LLMs to identify and correct grammatical errors in manuscript, ensuring clarity and linguistic accuracy throughout the paper.
B NOTATION TABLE
To facilitate understanding and ensure consistency symbols used throughout the paper of Tumor- Chain Table 4 provides concise summary of all key notations, offering readers a quick reference for the variables and operators involved.
Table 4: Notation in the TumorChain Pipeline.
Notation
Description
F()
Represents the TumorChain framework.
E.()
vision encoder
Seg(-)
Organ segmentation expert
Cla(-)
Auxiliary classification model
P()
Multi-layer perceptron (MLP) projector
(CM
Backbone LLM
Va "XX
Represents a CT volume height 1/, width W, depth
Tinsh
Input of task prompt text The output of TumorChain A series of vision tokens after 3D encoder E
Rest
To
To
Global CT volume tokens (projector (projector-alinned aligned L.M space).
Mergan
Denotes the organ mask generated by segmentation model Seg(-)
Local vision tokens
A(-)
Denotes the organ matching operation
T(-)
Represents operation extracting local r size K according to mask. Result after Classification layer Cla(n)
Ten ( Taak. T T. T vt...]
Indicates all input tokens of LLM
Tras
Task text prompt tokens Task target organ tokens
T iE [L
Represents the potential related ROI organs/suborganss reasoned by LLM.
Ris
The initial diugnostic result output by the LLM
Mc
Local ROI mask
Leotat
Training loss
Represents the output text length represents the sample number
M
Loss weight
C RELATED WORK
Medical Benchmark. Early public medical benchmarks such as VQA-RAD Laau et al. 2018) VQA-Med (Ben Abacha et al. 2019). SLAKE (Liu al.) (02T), and Vat (He al.) 2020) have greatly advanced LVLM development in healthcare. These datasets are primarily 2D and fea- ture simple, template-basee QA pairs. Recent works employ data synthesis build larger and more multimodal benchmarks: RadFM (Wu et al. 2023) generates expert-verified questions from literature and cases. OmniMedVQA (Hu et al.] 2024) leverages category-driven templates with GPT-4 (Achiam al. 2023) for paraphrasing and distractor creation. HealthBench (Arora et al.) 2025) combines synthetic generation and adversarial testing for multi-turn, multilingual dialogues. In the field of tumor analysis, DeepTumorVOO. (Chen et al. 2025a) focuses fine- mne-grained 3D CT tumor detection. However, most benchmarks remain limited to multiple-chote formats and basic reasoning, insufficient to meet clinical demands for interpretability and in-depth analysis.
16

---

Published as a conference paper at ICLR 2026
Medical Large Vision-Language Models. Mcd-LVLMs such as LLd (Li et al. 2023
Lingshu (Xu et al. 2025b), Med-PaLM (Singhal et al. 2025) and HealthGPT (Lin et al. (025) ) have
rapidly advanced multimoaal clinical decision support by aligning ViT-based (Zhou al. 2024)
21) image features with language models. Methods like M3D (Bai et al. 2024), CT-Chat (Hamamei
eet al (024a) and Merlin (Blankemeier et al. (024) incorporate 3D CNNS and ViTss to better capture volumetric data, enabling more precise analysis of clinical 3D images. However, these Med-LVLMs lack causal reasoning for complex cases, prompting a shift toward CoT and knowledge-guider rea- soning paradigms. MedVLThinker (Huang et al. 2023) and MedVLM-RI (Pan et al., 2025) com- bine CoT reasoning with reinforcement learning to improve interpretabilityy while ReasonMed (Sun) et al. 2025) optimizes reasoning paths via multi-agent verification. MedResearcher-RI Yuu et al., 2025a) enhances deep reasoning by leveraging knowledge graphs for multi-hop QA sample con- struction. By emphasizing logical chains, medical vision reasoning models are overcoming tradi- tional 2D and 3D VLM limitations and opening new directions for explainable and reliable clinical decision-making.
Online Interleaved Reasoning, Online interleaved reasoning (IR) dynamically alternates thinking and answering, enabling more efficient multi-hop reasoning than traditional methods (Qiu et al., 2020 [rivedi et al.] 2022). IRCoT (Trivedi et al.) (022) shows that interleaving retrieval, rea- soning, and response generation significantly improves performance on complex tasks. Chain-of- Focus (Zhang et al. (025) further introduces adaptive visual search and zooming with reinforcement learning to enhance multimodal reasoning. CX-Mind (Li al. 2025) adopts interleaved reasoning strategies for chest X-ray diagnosis, generating verifiable reasoning chains. Online IR aligns Med- LVLM decision-makee with clinical workflows, motivating us explore IR for 3D tumor analysis.
D EXPERIMENTS AND IMPLEMENTATION DETAILS
D.1 TRAINING DETAILS
Data Preparation and Preprocessing, TumorChain takes as input complete three-dimensional CT scans in mii. gz format. It is important to note that these volumetric images exhibit considerable variability in spatial dimensions along the x. y, and z axes. To ensure consistency and model stability, we employ a multi-step preprocessing pipeline.
Initially, we extract the soft tissue window level and window width for each CT scan according to standard radiological protocols, followed by normalization of voxel intensities. This step reduces inter-scan variability and facilitates robust feature extraction. Given the heterogeneity in image shapes, we address dimensional inconsistency by applying precise cropping and zero-padding strate- gies. These operations are carefully performed to prevent geometric distortion and loss of anatomical information Finamll,io..Fnally... all processed CT volumes are resized to a fixed shape of (256, 256. 32)(hight width x depth), in order to meet the input requirements of the 3D vision encoder. This standardized preprocessing framework ensures that all input volumes possess the same spatial resolution, thereby improving model compatibility and computational efficiency.
Model Architecture. We adopt Qwen2.5-VI--BB and Qwen2.5-VLL-7B as the base multimodal backbones for TumorChain-3B and TumorChain-7B,, respectively. Since the original Qwen-V VL. vi- sion encoder only supports 2D images, we replace with M3D encoder (Bai al. 2024) to support volumetric 3D CT inputs. To bridge visual and language modalities, we employ a two-layer mul- tilayer perceptron (MLP) projection module, consisting of two Linear layers with an intermediate ReLU activation, to map the extracted organ-level visual embeddings into the LLM token space and facilitate multimodal fusion. For the large language model component, the pretrained LLM backbone of Diven2.5-VL-3BB77 tion. Additionally, an auxiliary classification head is implemented as a single fully connected layer to perform binary classification (normal vs. abnormal) on each segmented organ region, providing supervisory calibration. Both TumorChain models are trained under identical hyperparameter con- figurations, including learning rate and precision settings, following the hybrid model fine-tuning strategy described in the main text.
is used for high-level multimodal reasoning and report genera-
Hardware and Distributed Training. Model training is conducted 32 NVIDIA A800 GPUs. For the 3B model, training is performed for 12 hours, whereas the 7B model is trained for 16 hours. The training set comprises 1. 1.5M CT samples, utilizing 90% of the available data. All models are
17

---

Published as a conference paper at ICLR 2026
You professional zatiolopiat ET ViA eraiaator. Your taak is judge whather the nodel's gaxen Vol qusation serentically couzattar: niss the seference (poond-ttedc) arowss in the contect of laves selated of inegirg fistirgs. The qusssion tyses include Shape, Seardaay, Derasty, coant and Otheza.
For each (oasstioe, Reference Aromer, Redel Answect tziple:
1. Clamsiyy Quration Type:
Analyee the qustion and assign it ta are the folloming (apes 1) shipe concenning lesim shape, siee. thickness, nerroingg dilatient Waastaty 10.g. vteiler the lmvorgge augin is smothh Clear ot discinct) 3) beriity (6.g.. heppitescilpptteeeeeeeee dessity atserwterss 4) cousting le.g.. rembet lesion, sirgle vs, reltiple)
2. Cometttr Concistenry Fosliar on:
conwey idertica) same to pasalize for eiror vartations aaruuge, struciure, explanatary detail of the eais point and Ciniccl nearing eatch.
regarding
differs
the (lintcal fineing intespretation
arrput
Trople:
Sementie Corsiatency sudgeant: ("Corract" if consistent, "Insccrnct" othervite)
Fecus only on ciinteel aaaning relevart to the question type. Igroce snall aiffacerces in asplanatory process or vereing.
Figure 7: Semantic Consistency Evaluation Prompt.
trained using the DeepSpeed distributed framework (configuration: ZeRO-2), effectively enabling large-scale parallel training and efficient memory utilization.
Hyperparameter Configuration. All training and evaluation pipelines are conducted with strict reproducibility control. Detailed hyperparameter settings are as shown in table 5
D.2 SEMANTIC ACCURACY METRIC BY GPT5
To assess the quality of open-ended answers in the VQA task for CT imaging, introduce semantic consistency metric specifically designed for shape, boundary, density, count, TNM stage prediction, and other clinical questions. For each question-answer pair, the evaluator first classifies the question type, identifies its key clinical focus, and then compares the model's answer to the ground truth (reference answer). The metric judges the response as correct if the essential medical meaning and main clinical finding are preserved, regardless of minor differences in expression or reasoning process. This approach enables reliable evaluation of the model's clinical interpretability and factual accuracy, focusing whether the core diagnostic content is communicated consistently. The semantic consistency metric ensures that the assessment robust against linguistic variation and prioritizes agreement in clinical understanding over superficial textual similarity.
illustrative example of the evaluation prompt used guide semantic consistency assessment shown in Figure 7
D.3 EXPERIMENT RESULT DETAILS
D.3.1 EXPERIMENTAL RESULTS BY ORGAN CATEGORY
In this section, we present more detailed ablation studies and comprehensive Shain-of-Though (CoT) evaluation results than those included in the main text. Table and Table 7 summarize the organ-level accuracy of our model and various baselines the TumorCoT and DeepTumorVQA benchmarks, respectively. The scope of our evaluation focuses on the five major digestive organs: liver, stomach, pancreas, colon, and esophagus. For generalization tests, we specifically analyze the overlapping three organs-liver, pancreas, and colon-within within the DeepTumorVQA dataset, as these closely match our benchmark task settings.
Table 5: Hyperparameter Configuration
Hyperpammeter
Value
Optimizer
AdamW x 10-3
Learning rate
Learning scheduler
decay
Weight decay
0.0
Warmup
0.03
Mixed precision
bf16 (fp16 disubled)
Gradient nccumulation steps
devicevice batch
Number of dataloader workers
16
Number epachs
1.0
18

---

Published as a conference paper at ICLR 2026
Table 6: CoT-Report comparison of TumorChain with other Reasoning VLMs on TumorCoT.
Reasoning Models
Liver
Pancreas
Colon
Stomach
Esophagus
Average
Claude3-Haile
Gemini2.0-Flash
42.35
29.03
56.24
71.15
38.64
47
GPT-5 Mini
44.48
35.71
59.76
76.85
65.12
$6.38
Qwen2.5 VL--BB
58.15
54.60
69.79
52.41
IntemV1L-2.5--8
48.89
49.20
48.89
$5.84
52.54
$1.07
MiniCPM-V-4..
46.07
43.29
45.69
50.26
47.59
46.58
HealthGPT-3356
63.74
36.44
31.74
45.81
45.28
Lingshu-78 7B
53.03
33.55
53,23
60.71
44.42
48.99
MedVLM-RI-2B
38.58
42.78
37.18
54.79
34.90
41.65
RadFM
11.34
15.42
2.50
35.20
13.71
15.63
M3D-Phi3-4B
21.51
28.76
9.51
52.57
17.72
26.01
"JumrCChinn338
69.89
72.20
67.39
91.63
64.34
73.09
Tumor Chain-C
83.45
78.43
68.65
93.45
88.71
82.54
Table 7: Comparison of TumorChain with other VLMs on the public DeepTumorVO2 benchmark.
Model
Liver
Pancreas
Colon
Average
Generalist Models
Qwen2.5-VE
36.32
32.39
46.33
38.35
VntermVL-2.5.55
34.52
35,86
$2.75
41 11.04
MiniCPM-V4.5-9B
40.02
45.20
52.93
46.05
Commercial Models
Claude3-Haike
32.67
32.62
18.40
27.90
Gemini2.00 Flassh
35.33
35.02
44,80
38.38
GPT:5 mini
41.20
42.48
73,60
52.43
2D Medical Models
HealthGPT-3.8B
30.32
30.26
53.56
38.05
Lingshu-7B
38.39
38.01
50.68
42.36
MedVLM-R111 ---22
35.98
36.73
69.85
47.52
3D Medical Models
RadFM
31.97
34.11
40.60
M3D-PPhi-4B
24.82
24.88
19.68
23.13
Tumor Chain-7B
55.41
62.45
84.33
67.40
D.3.2 ABLATION EXPERIMENTS UNDER DIFFERENT CLASSTFICATION LOSS WEIGHT
Table 8 reports the results of ablation experiments under different classification loss weight (a) set- tings. We systematically investigate the impact of varying a, and the findings show that moderately increasing the classification loss weight (e.g., a=1.0) yields significant performance improvements across multiple tasks, with a highest average accuracy of 84.41% %. In contrast, setting a either too low or high leads reduced performance. These results highlight the critical role of appropriate loss weight adjustment in optimizing the collaborative training of hybrid models.
D.3.3 ABLATION EXPERIMENTS ON BASELINES BEFORE AND AFTER FINE TUNING
Our contributions include both large-scale, high-quality dataset and novel model architecture. Accordingly, our experiments cover three aspects: main evaluations demonstrating overall perfor- mance, ablation studies assessing the impact of individual model components, and generalization tests exploring robustness across different data distributions. The main results the paper demon- strate significant improvements across multiple metrics. In addition, we conducted rigorous abla- tion studies on three key components of the architecture- laaeee--mmressio--nniiiiei Informaaiionn Retrieval (IIR), classification-loss weighting, and CoT-formatted data. The results indicate that each module delivers clinically meaningful performance gains.
To further isolate and validate the contribution of the dataset itself, we fine-tuned three representative baseline models on our training set to facilitate direct performance comparisons and thoroughly assess the gains attributable to the dataset alone. The supplemental results, shown Table 9 reveal
19

---

Published as a conference paper at ICLR 2026
Table 8: Ablation study of the effect of classfication loss weight in TumorChain.
Position
Lesion Attributes
TNM Prediction
CoT-
Organ Pos.
Tumor
Seg. Loc.
Report
Avg.
Shape
Boundary
Density
Count
Others
Tumor
Node
Met.
Pos.
0.0
99.92
97.01
82.46
81.45
75.82
83.01
80.89
82.35
86.11
$7.88
65.22
77.41
80.79
0.1
99.90
95.58
84.01
79.78
75.09
82.10
83.85
81.85
88.69
58.16
69.54
78.29
81.40
0.5
99.90
97.14
83.14
81.60
81.15
83.51
84.01
82.45
86.43
60.85
70.65
79.54
82,53
1.0
99.97
97.57
86.88
82.28
84.52
85.05
86,20
86.57
88.83
61.63
71.07
82.36
84.41
1.5
99.94
83.21
79.40
74.46
75.37
81.83
77.41
86,23
57.51
57.46
79.21
79.16
Table 9: Ablation study of baselines before and after finetuning on TumorCoT.
Avg. Score (Before finetuning/ After finetuning)
Model
Position
Lesion Attributes
TNM Prediction
CoT Report
Avg.
M3D-Phi-3-4B
30.36/86.99
33.23/ 61.78
37.16/59.48
34.79/ /1.19/ 61.18
32.84/ 66.98
Lingshu-7B
28.93/85.46
49.91 /7..66
43,21 / 67.65
54.30/65.98
45.10/74.11
Qwen2.5-VI.-2 VL.-BB
31.05 (005//1.00 /9.002
47.17/76.45
40.94/62.37
60.54/74.21
44.04/76.01
TumorChain-7B
-/98.77
-/85.25
-/73.84
-/82.36
-/84.41
that all baseline models achieve notable improvements after fine-tuning, underscoring the unique and substantial value of our dataset within the tumor domain.
Meanwhile, our proposed TumorChain-7B continues to significantly outperform these fine-tuned baselines, reflecting the effectiveness of our architectural innovations.
E
COT DATA ENGINE
E. DATA ENGINE COMPONENTS DETAILS
(i) Diagnostic Knowledge Graph. In collaboration with five organ-specialist radiologists, we con- struct triplet-based ("entityrrelaton--enii"" knowledge graph from diagnostic guidelines, text- books, and representative cases, covering anatomy, findings, impressions, histopathology. and risk factors (see Appendix Figure 8). All organ segmentation standards follow international conventions, enabling hierarchical substructure and tumor-grade analysis. During reasoning chain construction, the data agent retrieves relevant nodes and relations from the knowledge graph ensure traceable and reliable logic and minimize factual and logical errors.
(ii) Structured Feature Extractor: We employ Qwen3-235B-A22 as the structured feature extrac- tor to perform rigorous cleaning and terminology standardization across multi-source, multi-format data, with all imaging descriptors aligned to the RadLex international standard. Leveraging organ segmentation from the knowledge graph, the module automatically extracts substructure- de- scriptions and structured cecturrs--inlludin
clinical information (age. gender). lesion attributes (location, shape, margin, density, count), and six additional categories- ferries-frro CT and pathology re- ports. TNM staging information and final diagnostic conclusions are also extracted from pathology reports, ensuring the completeness and accuracy of subsequent reasoning chains.
(iii) Segmentation Expert Model. We utilize TotalSegmentator to segment 117 organs and merge them to 56 organs (see Appendix Table 10). The organ and tumor masks of five digestive organs are refined by radiologists, enabling structured ROI localization for subsequent VQA tasks.
(iv) Traceable CoT Reasoner. GPT-4o-mini, selected for strong language and medical expertise, ceives multidimensional structured reports and knowledge graph links, generating organ- and lesion- level VQA that conforms to medical guidelines.
(v) CoT Logic Calibrator. During reasoning chain generation, Claude3.5 Milee3.5-Habh 5-Haiku automatically verifies logical integrity. Upon detecting inconsistencies, the system uses prompt strategies: expanding organ regions via segmentation for re-reasoning, or requesting further clarification and self-reflection from the reasoning module.
20

---

Published as a conference paper at ICLR 2026
T
Perpoatt
ligh
Ponoreas
Stomach
Tar your Higel
Liver
Founday
Colon
Esophogut
Figure 8: The overview of the diagnostic knowledge graph of 5 organs by doctors.
(vi) Summarizer for SonddingImppessiin Pathoolog Chains. We employ GPT-5-mint to aggre- gate all reasoning chains. This module receives structured information from pathology reports to construct a higher-level pathology analysis VQA. If inconsistencies arise between reasoning chains and pathological conclusions, the system automatically reinitiates inference. Upon successful vali- dation, it extracts findings and impressions related to staging, lymph node involvement, and metas- tasis, generating TNM prediction VQA pairs. Finally, traceable CoT-format report is produced for each organ, facilitating standardized and interpretable reasoning outputs.
E.2 DIAGNOSTIC KNOWLEDGE GRAPH
As figure shows, the diagnostic knowledge graph constructed in this study serves as the core clinical prior for the TumorCoT-1.5M dataset and TumorChain model, providing structured, trace- able medical knowledge support for multimodal CoT reasoning in tumor analysis. Unlike general medical knowledge graphs that cover broad healthcare domains, this KG is tumor-centric and organ- specific, focusing exclusively on the five major digestive organs (esophagus, stomach, colon, pan- creas, and liver) closely associated with clinical tumor diagnosis and treatment. Its value lies in bridging the gap between unstructured medical text (c.g., radiology reports) and structured reason- ing logic, ensuring that every step of the CoT reasoning process (from imaging findings pathology predictions) is grounded in evidence-based medical knowledge rather than arbitrary inference.
The construction of the KG adheres a multi-source, hierarchical verification principle, integrating three levels of authoritative data sources to balance comprehensiveness, accuracy, and clinical rel- evance. A-level primarily includes international and domestic clinical diagnostic guidelines of five organs, which define the core knowledge boundaries of the KG, including TNM staging criteria and standard imaging features of malignant tumors. B-level covers classic textbooks and high-impact academic papers. These sources supplement critical anatomical details and advanced imaging fea- tures. C-level consists of real-world expert-annotatee cases, including typical tumor cases labeled by 5 experienced organ-specialist radiologists and teaching cases from the Department of Radiology at top-tier tertiary hospitals.
This design not only ensures that the knowledge conforms to international clinical guidelines but also incorporates real- world diagnostic experience from expert physicians, making it more suitable for high-stakes tumor analysis scenarios.
21

---

Published as a conference paper at ICLR 2026
You professional radiolopiat ET dector. bullding traçirg disgrestic motel and nead v0s detasst foz texinarg.
Irput: imaging linlopp celotol the lives fioe climisal une anpsat: sPat0IMO
Your tack; Rised on these firdingss cosermet the following 6 cotegaries of Intepeobent viA poirs nee witt mallogut, ro inforsation Tinkages:
Segnnetat
patre regentss) including glvwn
lofe, right necttane space-ocssyyfng
locatiors,
nemtonung
Fxapple:
decrriptton:
eligtely hypateese
queettin:: "Ie sagrent the Hoer teclos tocated
Baseeiia was russtning this IT tmage, I losk for any atremaal darettt withis the Tiver panwortyes In the left labe, specifiially to segnarr sth, 1 chserse atodd citghtly typerense leston awaartin sbout 7.7 2:3 on Thersfore. Inter that the Textoo is the left labe of the Tiver, tuwaar::
2. Shape: costeact: 19A paies froe sencences if they inclute descriptiors of size, shapt, trickness (thickenel), dilatios, stenosis, or the stape of lesson low lovidesstt shallw.
Scapple:
original decrripptin: "The dists: part of the stedie hegarte wain te porrly desnrcated free the exee, sith tocaltzed luntnal narrowing.
queecton: "Ie there auttence Suntral nxrreating in the stodle hapartc wete?*
"eescoring:
right
tnear
necsuming distelly.
sameay: lhus, the arovet tuat the willle tepolis. show lncelieed nazcoming."
deceriprtons acundory (Cleamanclasry margie (swott//mmaet) aarglin sesoth aauntay related parre hauntariee stctioct.
asundary:
argan
whether
Ccample:
dectriprion:
boundary
surecurding
Queettin:: "Are
smooth and cleasty deraceated tren surmundding cianure?"
hiswer: Heasering wher evaluating the panccees ci leginng cleck whtther its ofjes are intac: and witther the trarsition to adjacest fac tisset distinct. this case, the sargaa are descrited as soeoth and well definel, with Clear separation froe the sunousting fat tissue, his iedicates thet the poncraatie teurdary intact distiigsstalle Sumwary this,
the panteratic eargirs smeeth tleerly dessrcaced.
6. deseity: Coverrurt H9A gatre of hanesty of there are decertpttons related to deeetry, eith as argnn deestry, en hypdeneeenggeeeteee lestors in unstructumes
Seapple:
original descripttio: *a hypodence Teston is observed is the pancreatic heat, acouriing repretiateely 2.5 on.
questtin: "waat is the densiry cirrcteristicc of the lestion in the poereatte head"
Beacoring: teglng, ceciiyy dasges ponrrestie parettlyn,, indirating reluced attesiatin
doarettrizing
hest, sboomsa Suarary:
apperss
darker than typereese.
decerised
counting-
repsrt cleatly
segnent (n.g.. essy? relitary?
mittsple7).
Fxmpple:
decrriptton: hypodeace
cegnent
right 'iver,
queetton: Sieu
lactioe pracent the Huvrr
kccuer: Bescoring Whet rectesing the IT ctan of the Sturr, I note ten di dierieer hypodeoce lectore tne is Iseaned in cegaent 6t and the other in ongeese of the right labe. Theee reprecene two caparate lactene, not a single corfiuent ausralltty. fonmary: This, the aroued is
6. Others: Construct rolevant questiors if these other bey imaging findings covered the atove categeztes.
Ouiput Rales:
Tach antver include chatr-of- thoghh (massnning)
Estract key inforaation froe the provides fandings and contrruct qusstions and ressondrg chates, atthout sentioningg so referencing the criginal repect. lse diverss opressions and sentence stracturen.
zeasesing, rppcopriats medical sneging Avold seduncency. geiecitice sepectant faesings.
Figure 9: Prompt example: Traceable CoT Reasoner for extracting lesion attributes.
E.3 PROMPT DESIGN OF DATA ENGINE
When designing prompt templates for each agent in our data engine, we adhere to the following principles to ensure both data quality and clinical validity.
Standardization of Medical Terminology. All descriptions are required to follow internationally recognized medical terminology standards (e.g., RadLex Radiological Lexicon, AJCC TNM staging system). This prevents ambiguity and ensures consistency across tasks (for example, using "hypo- dense lesion" instead of the vague expression "Iow-density shadow")).
Mandatory Reasoning Chain. Each response must consist of two components: a "Reasoning Pro- cess" and a "Summary." The reasoning process should explicitly reflect clinical diagnostic logic. typically progressing from organ localization to feature observation and finally to pathological cor- relation. The summary should present the diagnostic conclusion in a standardized form, e.g.. "Thus, the answer ...".
Task Boundary Specification. To avoid cross-task redundancy, prompts are designed to clearly delineate the input scope for different subtasks (e.g., lesion localization tasks focus exclusively on anatomical positioning without involving density assessment or malignancy determination).
Multi-Source Information Integration. Prompts are required to guide the agent to incorporate both structured medical knowledge (e.g., anatomical ontologies, tumor metastasis pathways) and quantitative image-derived features (c.g., lesion boundary, density values). This ensures that the reasoning process is grounded in objective evidence rather than subjective speculation.
Following these principles, the CoT inference framework aligns with clinical diagnostic standards and effectively supports the four core tasks of tumor analysis: anatomical localization, lesion at- tribute characterization, TNM staging prediction. and structured report generation. An illustrative example of prompt designed for the Traceable CoT Reasoner to extract lesion attributes is shown in Figure 9
22

---

Published as a conference paper at ICLR 2026
E.4 MULTI-ORGAN MASKS AND MERGED ORGAN MASKS
Table TO provides the detailed organ mask IDs from TotalSegmentor the mapping with our merged organ mask IDs.
Wasserthal et al.
2023 and
Table 10: Mapping Table of ToralSegmentator Organ Indexes After Aggregation.
New Label
Segmentator Name
Included Organs
TotalSeg. Label
D
background
background
1
splocn
splocn
2
Kidney right
Kidney right
3
Kidney
kidney Jeft
4
gallbladder
gallbladder
5
liver
liver
6
stamach
stomach
7
pancreas
pancreas
8
adrenal adrenal-glande rignn
adrenal glladd
9
adienaa. dddrenal.glandJJod
adrenal.gland.Iet
10
lung Loingupper__lbees Ject
sung.upper_lobc.ae
lung_Jower_loke Joh
lung_lower__ook
12
lung upper right
lung jupper Jobe right
13
lung wwwddlee..obbre right
lung midddle...eee right
14
lung lobe right
lung lower lobe right
15
esophagus
esophagus
16
trachca
trachea
17
thyroid.gland
thyroid.
18
amall_bowe
small_bowel
19
ducrdenum
duodenum
20
colorectum
colon
21
urinury. bladder
urinary bluilder
22
prostate
prostate
kidnecyccsstJef
kidney eyst
kidney syystright
24
24
vertebrae_St
vertebrae.St St
vertebrac 1.5
27
vertebrac
lumbar vembebrveriebrre
vertebrae
vertebrac.1.
vertebrac L1
vertebrae_1 T12
vertebrue, T11
33
vertehrae T10
vertebrae_TV
vertebrae T7
26
thoracic vertebrac
T6
38
vestebrue T5
39
vertebrac. T4
40
vertebrae_T3
vertebrae.1 TI
vertehrue_CT
vertehrme C6
45
vertebrue C
46
27
servical.verterrae
vertebrue C4
vertehrae
vertebrue.Co
vertehrue CC
28
humertislefft
29
humerus
humerus right
70
scapula
71
30
scapula
scapula_right
clavicula left
31
clavicula
clavicula_right
femur
32
femur
femur-right
hip_left
33
hip
hip.right
rib Ieft.
rib left
93
23
34
rib Jell

---

Published as a conference paper at ICLR 2026
3eb
rib.Jefi.4
J
96
rib lel..
97
rib.left.7
rib__efi88
Jib_e
rib,lefi_10
ribllefi11
102
rib_left_12
103
riburight_1
104
right.
right
riburight_4
107
riburight.5
108
rib right 6
109
35
riburight
riburight_7
110
riburight88
riburight_9
riburight_10
113
riburight11
114
rib rih_right_12
115
36
stemam
sternum
116
37
cartalccrr
cartilaaes
117
38
heart
heart
39
sorta
aorta
52
40
pulmonary.vein yein
pulmonary _een
41
brachiozephalic trunk
brichiceehaliic trunk
subclavian artery right
42
subclavian
subelavian .artery
common parteiddrttery.righh
43
common.carotdd.uoo
commoon_arotidd_artrryy.. Jeft
brachiocephalic. left
44
bruchiocephalic.ver vein
brachiocophalic. vein right right
45
strial.apennagee... Jeft
apraalappe Jeft
46
superior, yena vena_cava
superior cetrriorveena.. .cava
62
47
inferior, vena, cava
inferior vena cava
63
48
purtal wwwe_eeeindeeeeeeeeeeeee splenicseecccceeee vein
portal, yein_eeii___pp..aaa.mm selesiccenic.eiin
64
iliac artery. left
65
49
iliec_amtry
ilaac artery right
iliac vens. Jeft
50
iliac vena
iliac_vena_right
gluteus, mantemusmaximuuu... Jeft
gluteus gluteus_maximusriige
51
gluteus
gluteus_medius right
83
muteus..inimusslee
84
gluteus. www.imiinm.m... right
autochthon
52
autochtbon
right
iliopsous
53
iliopsons
iliopsons_right
54
spinal
spinal_cord
55
hrain
brain
56
skull
skull
E.5 QUANTITATIVE EVALUATION OF THE COT DATA ENGINE BY EXPERT CLINICIANS.
To systematically assess the factual correctness and clinical plausibility of the generated reasoning chains, we randomly sampled 5,000 VQA instances from the TumorChain- 1.5M dataset across task types and submitted them to a team led by a board-certified radiologist(8 years oncologic imaging experience) for evaluation from two perspectives:
The clinician first assessed whether the VQA result matches the corresponding infor- mation in the original medical report. If the VQA result is correct and can provide reliable support in clinical practice, it marked as usable; otherwise, it is marked as unusable.
Usability:
Clinical Reasonableness: For all usable data, we further categorized the samples:
24

---

Published as a conference paper at ICLR 2026
High quality: For usable cases, the reasoning chain must exhibit completeness, scientific sound- ness, and logical coherence; only when all criteria are satisfied is marked as high quality.
b. Acceptable: For usable cases where the overall logic is sound but some steps slightly brief or marginally insufficient, it is marked as acceptable.
Evaluation results show that the usability rate reaches 95.88%, and among all usable samples, 97.85% are high-quality. These results validate the effectiveness of our data-generation pipeline and demonstrate the reliability and practical clinical value of the dataset.
F COT EVALUATION DETAILS: TUMORCHAIN-EVAL
This appendix provides comprehensive description of the evaluation framework for Chain-of- is organized into two stages. In
Thought (CoT) reasoning. The framework TumorChain-Be the first stage, structured (subject, relation, object) triples extracted from medical texts serve as factual building blocks for reasoning. In the second stage, the generated CoT reasoning chains assessed for quality based on these triples.
F.1 STAGE EXTRACTION OF MEDICAL SEIITY-RELATION-TT
TRIPLES
This stage serves as prerequisite for CoT evaluation. Its goal is to convert unstructured medical text into structured knowledge representations, thereby supplying accurate and unambiguous factual units for subsequent reasoning-chain assessment.
Task Definition. Given medical radiology report, the objective is to extract all (subject, relation, object) triples that capture medical facts. Subject: Typically organ (c.g., liver), anatomical structure (e.g., pericolonic vessels), or a lesion (e.g., enlarged lymph node). Relation: A verb or state term that describes the medical semantic relationship between the subject and the object (e.g., not observed, suggests, absent, supports). Object: Generally a medical finding, lesion (e.g., wall thickening, abnormal density), condition (e.g., mass-like change), or another related organ.
Table 11: Prompts for medical triples extraction in stage of CoT evaluation
Prompt for Medical Fact Triple Extraction
You are medical fact-extraction assistant specialized in parsing knowledge from the given clinical text. Your task is to analyze the input and extract factual triples (subject, relation, object), where each triple expresses single, clear medical fact. Ensure that all information is presented strictly in triple form, focusing on organs/structures, relations, and lesions.
Input Example: First, examine the right hemicolon, cecum, ascending colon, and hepatic flexure; wall thickening or abnormal density is observed, indicating definite or in- flammatory change in the right hemicolon. Second, check the left hemicolon, transverse colon, and splenic flexure, descending colon, sigmoid colon, and rectum; again no abnormal thick- ening or abnormal density is found, supporting the absence of significant lesions in the left hemicolon. Third, evaluate colon-related internal structures and surrounding tissues; no peri- colonic vascular abnormality is detected, and the liver, gallbladder, pancreas, spleen, kidneys, and pelvic structures show no colon-related significant lesion, enlarged lymph nodes, or free fluid. These findings collectively do not support tumor active inflammatory changes. Over- all, the imaging features do not suggest colorectal cancer or other qualifying pathology. Output Requirements: In each triple, the first and third elements must be organ or lesion. and the second element must describe their relationship. If the second and third positions are reversed, correct them. The output format should be:
(Right hemicolon, not observed, wall thickening);
Prompt Design and Specification To guide the model in performing triple extraction, we design standardized prompts tailored to the characteristics of medical tasks. The prompt clearly specifies
25

---

Published as a conference paper at ICLR 2026
Table 12: Finding Chain (FC) scoring criteria.
Chain Level
Scoring Dimension
Description
Scoring Criteria
10: Prediction contains all key GT facts with no omis- sion or redundancy.
8-9: Prediction covers the vast majority of GT facts, with only minor omissions redundanciess (110%).
Degree to which predicted facts
Existence Match
match the ground-truth (GT) facts.
6-7: Prediction covers some GT facts but exhibits mod-
erate omissions or redundancies (10-30%).
4-5: Prediction covers only a small portion of GT facts; substantial omissions redundancies (30- 30-50)).
Finding Chain
1-3: Low match rate; only a few are correct and the error rate very high.
Prediction includes none of the GT facts.
10: Prediction covers 100% of GT facts with no omnis
Extent which key facts fully expressed without missing spurious content.
8-9: Only minor omissions (<100%), high coverage.
Completeness
6-7: Noticeable omissions (10-30%), coverage is mod-
crately compromised.
4-5: Majority of key facts are missing (30-50%), ), poor coverage.
1-3: Extensive omissions (> (2%50)) very low coverage.
0: Prediction fails to cover any key facts.
10: All predicted entries are accurate with no invalid or incorrect statements.
8-9: Vast mujority accurate (<,00% prohlematic), only minor issues.
Correctness of factual statements in the prediction.
Accuracy
6-7: Moderate errors or invalid entries (10-30%) that
negatively affect overall quality.
4-5: Numerous erroneous entries (> (x30%)) only a small
fraction correct.
1-3: Very few accurate entries (>-50% emor rate).
0: All predictions are invalid; no correct entries.
the model's role, the extraction objective, and the required output format, while providing illustrative examples to facilitate structured responses. The prompt for LLM to extract medical entity-relation- entity triples is shown in Table II
F.2 STAGE 2: CoT EVALUATION PROTOCOL
Following the triple-extraction stage, this phase aims to comprehensively assess the quality of the model-generated Chain-of-Thought (CoT) reasoning chains. Evaluation is performed by comparing model predictions (Pred) against the reference ground truth (GT).
Reasoning-Chaii Categorization. Both the predicted (Pred) and reference (GT) reasoning chains are segmented into three hierarchical levels to enable fine-grained evaluation: (i) Finding Chain (FC): Consists of basic facts directly extracted from radiological descriptions (e.g., no wall thick- ening observed, soft-tissue mass detected). These represent independent, objective observations without inferential steps. (ii) Impression Chain (IC): Comprises intermediate medical impressions or preliminary suggestions derived from multiple $1 findings (e.g.. suggests localized inflammatory change). This level reflects simple, initial reasoning. (iii) Long Reasoning Chain (LRC): Repre- sents high-level medical reasoning and conclusions that integrate all findings (SI) and impressions (S2) (e.g., consistent with imaging features malignant tumor). This chain must exhibit complete logical derivation and clinical diagnostic value.
Scoring Criteria. We employ GPT-4o to score the FC, IC, and LRC sub-chains along multiple dimensions. All metrics are rated on 10-point scale.
The scoring criteria of FC are shown in Table 12
The scoring criteria of IC are shown in Table 13
26

---

Published as a conference paper at ICLR 2026
Table 13: Impression Chain (IC) scoring criteria.
Chain Level
Scoring Dimension
Description
Scoring Criteria
10: Impression is completely clear, logically coberent, unambiguous.
8-9: Description is clear with only minor incomplete
Clarity
Medical clarity of the impression statement.
expressions or slight ambiguity (<10%).
6-7: Basically clear but contains some ambiguities that
moderately hinder understanding.
4-5: Blurry description with numerous ambiguities sig-
Impression Chain
uificnnly affecting medical interpretation.
1-3: Very difficult to understand; almost unusable due severe ambiguity.
Impression chain empty entirely invalid and un-
clear.
10: Fully consistent with the factual chain; all impres- sions are derived from facts with no unrcasonalle con-
Logical consistency of the impression with the underlying FINDING chain.
8-9: Overall consistent with only minor (< (c10%) devi- ations from the factual chain.
Consistency
6-7: Partially inconsistent with the factual chain, show- ing moderate deviation (10-30%)
4-5: Large propotton content inconsistent with or weakly related to the factual chain (30-50%).
1-3: Impression content is mostly illogical and lated to the factual chain.
0: Impression is entirely invalid completely contra- diets the factual chain.
10: Impression chain is highly useful. directly support-
ing diagnosis and clinical decision-making with no ad- ditional input needed.
Clinical usefulness of the impression for diagnosis and decision making.
8-9: High clinical utility: most content is medically meaningful with only minor adjustments required.
Medical Utility
6-7: Partial diagnostic value considerable portions lack utility or have vague meaning (10-30%).
4-5: Very limited clinical utility (>50% of content lacks diagnostic value).
1-3: Impression provides almost diagnostic signifi- cance or is largely incorrect.
0: Impression chain is invalid contains medically meaningful statements.
The scoring criteria of LRC are shown in Table 14
Scoring Procedure. Input: The ground-truth (GT) reasoning chain and the model prediction (Pred) are provided to a scoring large language model (LLM). Processing: Guided by a set of predefined scoring rules embedded in the evaluation prompt, the scoring LLM compares the two chains and performs detailed analysis. Output: The LLM produces structured JSON object containing the classified chains and the numerical scores for each metric, for example: "scor- ing": "iindinn"""" ""existence
"8/10", "completeness": "7/10", "accuracy"; "9/10" }, "s2 im.prreesso""" {...). "i.."ssrras" reasoning": {...). "overall.score": "xx/100" }
Overall Score Computation. The final chain-of -hahouof- evaluation score (cot e) is calculated as a weighted average of the sub-chain scores, balancing the relative importance of different reasoning levels:
N Σ (tc) + Wio
1 N N
Σ (Sic) + WERC 1 N Σ N (St.c))
CoT. = WFO
(9)
N
27

---

Published as a conference paper at ICLR 2026
Table 14: Long Reasoning Chain (LRC) scoring criteria.
Chain Level
Scoring Dimension
Description
Scoring Criteria
10: reasoning chain perfectly covers all key points,
with no logical gaps omissions.
8-9: Reasoning largely complete, with only minor
Logical Completeness
Logical closure and completeness
(<10%) logical gaps or omnilted details.
of higher-order reasoning.
6-7: Some logical interruptions omissions exist, but
most reasoning paths remain valid (10-30%).
4-5: Significant logical gaps, missing many key points, and notable interruttins in reasoning (30-503) (0%).
1-3: Musi of the reasoning chain is invalid; significant
logical flaws, key derivations incomplete.
Long Reasening Chuin
0: No reasoning precess or the reasoning chain com- pletely fails.
10: Reasoning Jemonstrates highly complex hierarchi- cal relationships and deep cross-cntity connections.
Whether the reasoning depth reflects cross-entity and hierarchical associations.
8-9; Reasoning shows moderate depth; most steps are reasonable with minor (< 10%% missing complexity.
Reasoning Depth
6-7: Reasoning depth is insuflicient, logical chains are relatively shallow, capturing only surface-level infer- ence (10-30% missing depth).
4-5: Reasoning lacks depth, limited to single-layer derivations or simple restatements facts.
1-3: Reasoning is very superficial; most content invalid or lacks analytical depth.
0: Reasoning chain has no depth; no higher-order infer- ence.
10: Reasoning fully aligns with medical context and is highly relevant and practical.
Whether the reasoning contributes diagnosis and aligns with medical context.
8-9: Most reasoning medically meaningful; only mi- content is irrelevant (< 10%)
Clinical Relevance
6-7: Some reasoning entries are meaningful, but overall relevance is limited (10-30% invalid content).
4-5: Majority of content lacks medical significance; only a few entries provide support (>-00% clinically ir- relevant).
1-3: Reasoning is almost clinically useless; content shows deviation from medical background.
Reasoning chain entirely meaningless or invalid.
10: Reasoning scamlessly integrates all evidence from findingfippessioon chains, supporting conclusions.
8-9: Most evidence is integrated, with minor (< 10%)
Fvidence Integration
Whether multiple findings and cues are integrated reasonably.
gaps weak concentration.
6-7: Partial integration: some information not adopted weakly related (10-30%)
4-5: Integration is poor; reasoning limited single
evidence items (>300 weak inlegration).
1-3: Integration largely insufficient; evidence shows no clear relation.
0: Reasoning completely detached from evidence; integration or lugical coherence.
28

---

Published as a conference paper at ICLR 2026
where Sic is the score the sample in FC scoring, Sic is the score the sample i in FC scoring, LR is the score the sample in FC scoring, and N represents sample number.
The weighting coefficients satisfy:
WFC + Wie WLRC =1
where W can be used to adjust the importance of reasoning chains during evaluation. In this study, we set WFC = 0.3, Wi 0.3, and WLRC = 0.4, emphasizing the importance of high-level logical reasoning (LRC) while assigning equal foundational weight to factual findings (FC) and medical impressions (IC).
G
CASES STUDY
G.1
TUMORCHAIN HALLUCINATION CORRECTION
In this section, present a comparative case study to evaluate the clinical utility of TumorChain. focusing on its performance in CoT report generation. We compare the generated reports of our proposed TumorChain with those of 2D medical model (HealthGPT) and generalist model (Qwen2.5-VL).
Figure I0 showcase the report outputs of the three models across clinical scenario of a pancre- atic malignant tumor, and explicitly contrast each output with the ground truth. In this example, TumorChain-T7 7B accurately identified the location of the pancreatic tumor, abnormal density within the pancreatic parenchyma, and key features such as the capsule boundary. Although HealthGPT was extensively finetuned on general medical datasets and possesses strong image analysis capabili- ties, it misdiagnosed the tumor's location, incorrectly identifying it as originating from the posterior instead of the head the pancreas. Qwen2.5- VL, meanwhile, provided an erroneous diagnosis of no abnormality in the pancreatic head.
This case demonstrates that our dataset construction by organ substructure significantly improves the model's ability to detect abnormalities within organ subregions.
original report information
G.2 QUALITATIVE ERROR TAXONOMY
In table 1 in our paper, Tasks 1-3 correspond to fine mne-grained VQA pairs targeting lesion attributes. The results demonstrate that our model achieves strong performance on single-step, Single-atrribute tumor questions. To further analyze the sources of errors open-ended report generation, we ran- domly selected 100 reports with incorrect conclusions and invited clinical experts to qualitatively review each failed case. This analysis aims to identify major error patterns and determine which lesion attributes types and representative examples are summarized as follows:
prone to mistakes that could affect the final diagnosis. The main error
False positives due to similar CT appearances: For example, in pancreatic cases, abnormalities such as pancreatitis or pancreatic pseudocysts are often misclassified as malignant tumors because some malignancies can also present inflammatory changes.
Missed small lesions at organ boundaries due to anatomical overlap: On 3D CT images, rare lesion morphologies at organ edges corners are difficult for the model to detect. In case, small (approximately 2.0x1 O cm) mixed-density lesion at the posterior edge of the pancreatic tail was missed because the overall pancreas size and surrounding anatomy appeared normal, causing the model to overlook this subtle abnormality.
Incorrect identification of the primary organ due tumor compression: In some cases, mors compress adjacent organs, leading the model to misattribute the origin of the malignancy. For example, tumor in the pancreatic tail compressing the stomach resulted in the model erroneously diagnosing a malignant lesion in the stomach.
29

---

Published as a conference paper at ICLR 2026
Question
HealthGP
Please carefully analyze specific monifestations each anotomical part of the pencreas this CT image. be think this patient has pancreatic lesions? Please provide poncreatic report, including your andlytical approach and predicted conclusion.
Besed on this CT image, con observe the specific monifestations verious onstomical ports of the pancreas. In the anterior part of the pancreas with distinct obnormal area, which moy manifestction of poncrestic lesions. Thore also small Binornal area the postemer port poncreas, These abnormal may indicate the
circulor
of poncreatic
lesions. Considening the presence of these abnormal areas, speculate that the patient has poncreatic lesions. This andysis based on the abvious manifestatiors abnormal
images, combined with knowledge of pathological diegnasis.
TumorChain
focal solid of opproximately 17 16 mm can be seen the uncinate process of the poncreatic head, with slightly lower donsity the enhonced parenchymal phose, indiauting aisplid moss with local invosve features, There no clear the pancreatic copsule invasion pancrestic structures, indicating ho abvisus capsule rupture or extensive infiltration Secondly, by observing the segmentation be seen the head of the pancreas (uncinate process), and no abnormaitiie found the panorectic body and tail, indicating that the lesion limited to the uncinate grocess the head of the parcreas
Analysis appreach: Firstly, observe the
Qwen2. 5-VL
the abeve mentioned
CT image shows cross- sectional reveoling partial structures of the pancreas. The following anolysis various ports the pancreas::
abdomen,
Thirdly, observe
significant contact betweer the arterial side and the superior mesenteric artery, abdominal trank,
hepotic
There significant
Pancreatic head: located in the upper right corner of
contact between the venoust side and the portal vein superior mesenteric imoging suggests
the circulor elliptical in shope, with uniform
surface with superior mesenteric vein is 160 No truncation or dilction of pancreatic duct suspicious enkorgement found nodes, and no signs of metostosis were observed the liver and poritoneum.
density and clear boundonies. The head of the poncrsos
part of the absious abnormalities
and should not have
surnounding lymph
Poncreatic body: located below the head of the ponareas, in the shape of long strip, with uniform denaity and clear boundonies.
above findings, highly suspacted poncreatic malignent tunat, which is consistent with the imaging findings pancrestic cancer 16 mm), with local vasculor contact (superion mesenteric vein; 180 '). and distant metastasis.
pancreas
about
Pancreatic tall: located below the poncreatic body. it elangated, uniformly dense, and has clean boundories.
Summory: lesion poncreas
Figure 10: A case of CoT report generation.
30


Reviews:
[{"review_id": "7dYxNjIonC", "reviewer": "Reviewer_RmsC", "summary": "The paper presents TumorChain, an interleaved multimodal chain-of-thought (CoT) framework for clinical tumor analysis on 3D CT. The authors also release TumorCoT-1.5M, a large CoT-labeled dataset spanning findings → impressions → pathology, and propose TumorChain-Eval for stepwise reasoning assessment. TumorChain integrates a 3D vision encoder, an organ segmentation expert, an auxiliary abnormality classifier, and an LLM that iteratively reasons over global and organ-level tokens. On TumorCoT and the public DeepTumorVQA, TumorChain outperforms strong generalist and medical LVLMs; ablations suggest CoT and the iterative organ-guided loop contribute meaningfully.", "questions": "(1) Can you please clarify the data-splitting strategy for TumorCoT-1.5M? Are the training, validation, and test sets split at the patient level to prevent data leakage?\n\n(2) The CoT data engine is impressive. Beyond the expert reviews mentioned, was there a formal, quantitative evaluation by clinicians to measure the factual accuracy and clinical plausibility of the LLM-generated reasoning chains?\n\n(3) For the open-ended report generation task, what mechanisms are in place to prevent the model from hallucinating pathological findings that are not grounded in the visual evidence?\n\n(4) Would you consider providing results for a baseline (e.g., Qwen2.5-VL-7B) fine-tuned on TumorCoT to create a more direct comparison?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "+ The TumorCoT-1.5M dataset is a highly valuable contribution to the field. A 3D, instruction-tuned dataset of this scale, specifically designed for traceable, multi-step reasoning in a high-stakes domain like oncology, is a significant achievement and will likely enable new research directions.\n+ The TumorChain model's core mechanism, Iterative Interleaved Reasoning (IIR), is an effective way to handle the high dimensionality of 3D medical images.\n+ TumorChain achieves top performance across subtasks and generalizes to DeepTumorVQA; ablations show the value of CoT/IIR and of the auxiliary classifier.", "weaknesses": "- The dataset relies heavily on an MLLM-based data-generation pipeline. This introduces a risk of factual errors, bias, or hallucinations. The paper does not sufficiently discuss the quality control process for this generated data, beyond mentioning expert reviews. Furthermore, critical details on data splitting (e.g., ensuring splits are at the patient level to prevent leakage) are not detailed in the main text.\n- The paper uses terms like \"causality-guided inference\" and \"iterative causal reasoning\". However, the method appears to be an exemplary case of step-by-step reasoning that follows a clinical logic path (Findings $\\rightarrow$ Impression $\\rightarrow$ Pathology), rather than a formal causal inference model (e.g., using structural causal models, do-calculus, etc.).\n- The main results in Table 1 compare the TumorChain model (trained on the new 1.5M TumorCoT dataset) against baselines (like RadFM, Lingshu, GPT-5) that were not trained on this data. This makes it impossible to disentangle the performance gains from the novel architecture versus the massive, high-quality dataset. A fairer comparison would require fine-tuning the baseline models on TumorCoT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fE7lbNXqJh", "reviewer": "Reviewer_XLFD", "summary": "This paper builds a tumor‑centric, end‑to‑end multimodal reasoning pipeline for 3D CT that goes from radiology findings,impressions,pathology‑level conclusions. It first constructs TumorCoT‑1.5M (~1.5M CoT‑labeled VQA instructions paired with CT scans across five digestive organs) via a multi‑agent, knowledge‑graph–guided data engine, enabling standardized, step‑aligned supervision. It then introduces TumorChain—an interleaved reasoning framework that tightly couples a 3D vision encoder, an organ segmentation expert, a lightweight abnormality classifier, and an LLM to iteratively align global context with organ‑level ROI evidence, produce traceable chains of thought, and calibrate uncertainty. Finally, it proposes TumorChain‑Eval/CoTe, which parses CoTs into triplet‑based chains (Finding, Impression, Long Reasoning) for stepwise scoring. Experiments show consistent gains over strong generalist and medical LVLM baselines on TumorCoT and strong generalization to DeepTumorVQA, with ablations validating the value of interleaved reasoning and clinical CoT.", "questions": "see Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper introduces a reasoning paradigm aligned with clinical workflows, featuring a traceable closed-loop process of 'Findings,Impressions,Pathology'.\n2. This paper presents TumorCoT, a large-scale dataset\nof 1.5M CoT-labeled VQA instructions paired with 3D CT scans, with stepaligned rationales and cross-modal alignments along the “findings→impression→pathology” trajectory.\n3.  This paper proposes TumorChain, a multimodal interleaved reasoning framework that tightly couples 3D imaging encoders, clinical\ntext understanding, and organ-level vision-language alignment.", "weaknesses": "1. The authors use a proposed multi-agent, knowledge-graph-guided engine to generate chain-of-thought reasoning, which may introduce bias. Could the authors conduct a small-scale CoT evaluation with medical experts to validate the reasoning quality?\n2. Does the use of the segmentation model increase inference or training latency, potentially reducing efficiency? How do the authors address this issue?\n3. The comparison with commercial models only includes GPT-5-Mini and Gemini2.0-Flash. How do the latest models, such as Gemini 2.5 Pro and GPT-4o, perform on this benchmark?\n4. Table 1 shows that all baselines are non-CoT models. How would the proposed method perform when compared against CoT-enabled models? like LLava-COT?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LAmiIcxmbN", "reviewer": "Reviewer_Xddo", "summary": "This paper addresses the limited tumor-specific reasoning and lack of reasoning depth in current medical large vision-language models (Med-LVLMs), by introducing a standardized formulation of tumor reasoning that mirrors the oncology workflow—from radiology findings to impressions to pathology-level predictions. It constructs TumorCoT-1.5M, a large-scale dataset of 1.5M multimodal chain-of-thought (CoT) instructions aligned with 3D CT scans, and proposes TumorChain, an interleaved multimodal CoT reasoning framework that integrates 3D imaging encoders, segmentation experts, and large language models for organ-guided, iterative reasoning. The resulting performance on the TumorCoT and DeepTumorVQA benchmarks demonstrates substantial improvements in classification, lesion analysis, and reasoning traceability compared to existing Med-LVLM baselines", "questions": "1. Are baseline models trained on the training set of TUMORCOT-1.5M? If not, and your model is instead trained on it, this is not a fair comparison.\n2. Is the CoT data validated by human experts?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The creation of TumorCoT-1.5M, a comprehensive CoT dataset, provides a valuable resource for the community.\n2. The proposed TumorChain framework, enables deep reasoning steps and achieves significant performance gains on the TumorCoT and DeepTumorVQA benchmarks", "weaknesses": "1. The modality of the dataset and the empirical experiment is limited to CT modality only, It remains unclear whether the approach generalizes to other medical reasoning tasks or modalities.\n2. The CoT data generation are relying on LLMs, lacking Human Clinical Validation. This raises concerns about potential biases, and hallucination.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fBz8DLuF7k", "reviewer": "Reviewer_JGa5", "summary": "This paper introduces TumorChain, a comprehensive framework for traceable clinical tumor analysis. The authors address limitations in current Medical Large Vision-Language Models (Med-LVLMs), which often lack specialized oncology reasoning, fine-grained 3D analysis, and interpretable decision-making. The authors propose: (i) TumorCoT-1.5M, a large-scale, multimodal dataset of 1.5 million Chain-of-Thought (CoT)-annotated Visual Question Answering (VQA) samples, built from 3D CT scans and corresponding radiology/pathology reports, focusing on five major digestive organs. (ii) TumorChain Model, a multimodal interleaved reasoning framework for stepwise reasoning from findings to impressions to pathology predictions. (iii) TumorChain-Eval, a specialized evaluation protocol that moves beyond conclusion accuracy to assess the stepwise logical consistency and clinical relevance of the generated reasoning chains. The paper demonstrates state-of-the-art performance on their TumorCoT benchmark and shows generalization on the public DeepTumorVQA dataset.", "questions": "1. For the benchmark setting, are the 9:1 splits patient-level and cross-institutional? How are radiology/pathology reports prevented from leaking textual supervision across splits?\n2. How is the “calibrated uncertainty” computed and validated?\n3. What are the exact backbones (3D encoder, LLMs) for 3B/7B? What is $P(\\cdot)$’s architecture and $Cl(\\cdot)$’s head? Any per-organ heads or class imbalance handling?\n4. Any blinded reader study or qualitative error taxonomy? Examples where LLM-only baselines hallucinate but TumorChain corrects?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The idea of Chain-of-Thought (CoT) reasoning is timely for current Med-LVLMs.\n2. The curated TumorCoT-1.5M dataset is a major contribution.\n3. The proposed model outperforms state-of-the-art LVLMs.\n3. The paper is well organized and easy to follow.", "weaknesses": "1. The iterative interleaved reasoning process, which involves multiple calls to the segmentation model and the LLM with progressively growing context, could be computationally expensive. A discussion of the inference time/latency compared to single-pass baselines would be valuable, especially for potential real-time clinical applications.\n2. The data engine uses GPT-4o-mini/Claude/GPT-5 variants, CoTe also relies on an LLM scorer. Even with logic calibration, it is unsafe to rely on these models, as the authors show in their experiments that general-purpose LVLMs perform bad on these medical tasks. Providing human adjudication rates, inter-rater agreement, and sensitivity to the scorer model would be more convincing for people to use the proposed data, especially in this safety-critical domain.\n3. Again, the experimental results are primarily quantitative, lacking human reader studies or case-level blinds to confirm clinical utility and error modes (e.g., missed small lesions, false positives in complex anatomies).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)