CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
Authors: Yuxuan Sun
Yixuan Si
Chenglu Zhu
Kai Zhang
Zhongyi Shui
Bowen Ding
Tao Lin
Lin Yang
Venue: NeurIPS 2025 poster ()
Keywords: Agent
Computational pathology
Large multimodal model
Primary area: applications

Abstract: Recent advances in computational pathology have led to the emergence of numerous foundation models. These models typically rely on general-purpose encoders with multi-instance learning for whole slide image (WSI) classification or apply multimodal approaches to generate reports directly from images. However, these models cannot emulate the diagnostic approach of pathologists, who systematically examine slides at low magnification to obtain an overview before progressively zooming in on suspicious regions to formulate comprehensive diagnoses. Instead, existing models directly output final diagnoses without revealing the underlying reasoning process.
To address this gap, we introduce CPathAgent, an innovative agent-based approach that mimics pathologists' diagnostic workflow by autonomously navigating across WSI through zoom-in/out and move operations based on observed visual features, thereby generating substantially more transparent and interpretable diagnostic summaries. To achieve this, we develop a multi-stage training strategy that unifies patch-level, region-level, and WSI-level capabilities within a single model, which is essential for replicating how pathologists understand and reason across diverse image scales.
Additionally, we construct PathMMU-HR², the first expert-validated benchmark for large region analysis. This represents a critical intermediate scale between patches and whole slides, reflecting a key clinical reality where pathologists typically examine several key large regions rather than entire slides at once. Extensive experiments demonstrate that CPathAgent consistently outperforms existing approaches across benchmarks at three different image scales, validating the effectiveness of our agent-based diagnostic approach and highlighting a promising direction for computational pathology.

CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolutions Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
Yuxuan Sun 1.2.1, Yixuan Si Chenglu Zhu?, Kai Zhang
Zhongyi Shuil Bowen Ding Tao Lin Lin Yang" College of Computer Science and Technology. Zhejiang University, China Reeeaarch Center for Industries of the Future and School of Engincering, Westlake University, China DDeparmentt of Computer Science and Engineering, The Ohio State University, USA
Abstract
Recent advances in computational pathology have led to the emergence of numer- ous foundation models. These models typically rely on general-purpose encoders with multi-instance learning for whole slide image (WSI) classification apply multimodal approaches generate reports directly from images. However, these models cannot emulate the diagnostic approach of pathologists, who systematically examine slides at low magnification to obtain an overview before progressively zooming in on suspicious regions to formulate comprehensive diagnoses. Instead, existing models directly output final diagnoses without revealing the underlying reasoning process. To address this gap, introduce CPathAgent, innova- tive agent-based approach that mimics pathologists diagnostic workflow by au- tonomously navigating across WSI through zoom-in/l innuut and move operations based on observed visual features, thereby generating substantially more transparent and interpretable diagnostic summaries. To achieve this, we develop multi-stage train- ing strategy that unifies patch-level, region-level, and WSI-level capabilities within single model, which is essential for replicating how pathologists understand and reason across diverse image scales, Additionally, we construct PathMMU-HR? the first expert-validated benchmark for large region analysis. This represents a critical intermediate scale between patches and whole slides, reflecting key clini- cal reality where pathologists typically examine several key large regions rather than entire slides at once. Extensive experiments demonstrate that CPathAgen consistently outperforms existing approaches across benchmarks at three different image scales, validating the effectiveness of our agent-based diagnostic approach and highlighting a promising direction for computational pathology.
Introduction
Pathology serves as the gold standard for diagnosing numerous diseases, particularly cancer. The advent of digital pathology has enabled the digitization and computational analysis of histopatho- logical slides, opening transformative opportunities for Al-assisted diagnostics. Recently, numerous pathology foundation models have emerged to automate and enhance diagnostic workflows.
These foundation models (Figure left) adopt two primary architectural paradigms. The first employs recoder-based models such as DI or CLIP to extract letch-level representations from
I Equual contribution. Cooresponniig author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

a) MIL Approach
b) LMM Approach
CPathAgent (ours)
Region selection
S1 Navigation planning
Question/ Instruction
S3
LMMS (e GPT-6a, Gemini- 2.5-Pro, PathGen-ULVVA
Duerview
Zoom in
Move
Move
Zoom
Gemini
Encode & MIL Apgregation (e.R., ABMIL, DSMIL) Black Box!
Black Box!
Multi-scale (view) Pathologist-lkke Reasoning
Interpretable!
Classification / VQJ
Ien's start with the overview
[51],
Description Generation
the upper right quadrant (52) at 3.5x examine- ..... w........ are
Classification Subtyping (e.g. Ductal Corciame!
(ea. image shows flroudipaze
Bikeey Let's slide left along (53) can Now, moving inferiorly
dssue consittnnt with bucost ...
intt
(54).
Zooming
Comparison of the "black box" traditional MIL approach, LMM approach, and our proposed CPathA- gent for analyzing pathology images, CPathAgent interpretably mimics pathologists reasoning by performing operational actions (c.g., zooming in, moving the view) while describing analytical logic.
Figure I:
WSIs. which are then aggregated via tions. The second utilizes large multimodal models (LMMs) that jointly process visual data and text instructions to generate pathology reports or structured outputs 77 & 9 TO]. While these approaches achieve impressive benchmark performance, they share a fundamental mismatch with clinical practice. In real-world diagnostics, pathologists employ a systematic multi-scale examination strategy. They begin by scanning slides low magnification (2-4x) to assess overall tissue architecture and identify regions of interest, then strategically navigating to atypical regions while progressively increasing magnification (10x, 20x, 40x) to examine fine-grained details. Throughout this process, pathologists continuously integrate observations across scales while conducting step-by-step diagnostic reasoning.
Learning (MIL) 4 for diagnostic predic-
However, current models bypass this systematic sequential process by directly aggregating infor- mation across the WSI or processing visual inputs in a single forward pass, leading to two critical limitations. First, weak supervision: relying solely on slide-level labels without explicit guidance on diagnostically important regions or when to examine specific magnifications makes them vulnerable to shortcut learning (6. exploiting spurious correlations (staining artifacts, background patterns) rather than genuine pathological features. Second, lack of interpretability and verifiability: pro- ducing only final predictions without providing natural language descriptions of the intermediate reasoning prevents pathologists from understanding what the Al actually discovered and validating whether its findings are based on clinically relevant features.
To address these limitations, we introduce CPathAgent, a novel agent-based framework inspired by recent advances in Al agent systems that have achieved remarkable success from strategic gameplay [ and web navigation [12 to task automation [3] and multimodal content generation 16 I7]. The key to their success lies in decomposing complex tasks into sequential Following this paradigm, CPathAgent bridges
14
15
reasoning steps with dynamic decision-makinn the gap between clinical practice and Al-assisted diagnosis by rethinking pathology diagnosis as adaptive visual reasoning. Unlike existing black-box models that treat diagnosis single-shot classification, CPathAgent emulates pathologists' examination strategy through WSI navigation with dynamic magnification adjustment and multi-view synthesis, conducting explicit step-by-step reasoning through natural language to enable interpretable and verifiable diagnostic assessments.
Specifically, CPathAgent performs a three-stage diagnostic workflow (Figure D. In the global sereening stage, CPathAgent analyzes a low-resolution WSI overview to identify suspicious regions that warrant detailed examination. In the navigation planning stage, CPathAgent plans an exploration strategy for each selected region, determining a sequence of view coordinates and magnification levels to investigate, mimicking how pathologists mentally map their examination before diving into details. Finally, in the multi-scale reasoning stage, the planned navigation path executed through systematic analysis of fields of view at different magnifications (e.g., zooming in for cellular details, moving laterally across areas, zooming out for context), and synthesizes observations via step-by-step diagnostic reasoning to reach conclusion. Overall, this work makes the following contributions:

---

1) CPathAgent framework: We propose an agent- based diagnostic system that performs diagnosis through strategic WSI navigation and multi-scale visual reasoning, mirroring pathologists' systematic examination process while providing interpretable step-by-step rationales.
2) Mat-MMU-HRR henchmark: We develop the first visual question answering benchmark specifi- cally designed for high-resolution pathology image regions (16000x166) 16000 pixels), rigorously vali- dated by three professional pathologists ensure clinical relevance and diagnostic accuracy.
3) Multi-stage training strategy: We design a progressive training approach that equips CPathAgent with perceptual and reasoning capabilities across different image scales within unified framework.
4) Comprehensive evaluation: We conduct experiments multiple scales, from patches and large regions to WSIs, highlighting CPathAgent's strong performance and practical clinical utility.
2 Related Work
2.1 Pathology Foundation Models
Recent years have seen significant progress in developing foundation models for computational pathology, broadly categorized into ancoder-based models and large multimodal models (LMMs).
Encoder-based models focus on extracting rich patch-level representations from WSIs. Early efforts in this area relied on ImageNet-pretrainee CNNs [8] 19] as feature extractors, combined with multi-instance learning (MIL. 4
5 6 to aggregate these features for slide-level prediction. More
recently, the field has shifted toward pathology- speciifcc foundation models that adopt self-supervised learning paradigms. Architectures such as DINO [ 2] and CLIP 33, when trained on large-scale pathology datasets, have demonstrated superior feature extraction capabilities (20 g 21 22 23)
These improved representations, integrated via MIL or incorporated into pretrained WSI transformers 20,9, have significantly advanced the WSI-level classification performance. LMMs for pathology have emerged in parallel, enabling joint processing of pathology images and text instructions to generate textual diagnostic outputs. These models integrate visual and linguistic information through
various designs. Patch-level LMMs 7 24 g 25 26 adapt general vision-language frameworks like LL 27 for patch-level pathology image interpretation, while WSI-level models 9 28 29]
aim to process entire WSIs for generating reports or answering diagnostic questions. Recent unified frameworks 10 attempt handle both patches and WSIs within a single model architecture.
Despite these advances, current models typically process pathology inputs in a static manner, con- trasting with expert pathologists who dynamically navigate regions of interest, examine tissues at multiple magnifications, and integrate contextual information to formulate diagnoses.
2.2 Agent-based Models
Agent-basce models represent an emerging paradigm in AI research, characterized by systems that can perceive their environment, make decisions, and take actions to achieve specific goals [30] Recent advances in large language models (LLMs) have catalyzed significant progress in this domain, enabling more sophisticated reasoning and planning capabilities. Notable examples include WebGPT [12] and WebShop [31], which navigate web environments to complete information-seeking
and shopping tasks; ReAct [32] which interleaves reasoning and action steps for improved task
performance; AutoGPT [33 and AutoGen [34,, which decompose complex tasks into manageable
subtasks with minimal human supervision; and task-specific agents for code development [4] creative content generation 35 scientific research (36), and clinical decision-making and discovery that simulate physician workflows in hospital environments [37] 38 39 40). These agent models typically incorporate several key components [5): perception modules to process environmental inputs, reasoning modules to interpret observations and plan actions, and execution modules to implement planned actions. This architecture enables agents to adapt dynamic environments and accomplish complex, multi-step tasks through iterative
cycles.
Despite success in various domains, agent- agent-based approaches remain largely unexplored in computa- tional pathology. PathFinder [41] represents a successful early attempt at introducing agent- systems for WSI analysis. The system segments WSIs into fixed 512x512 patches and employs multi-agent framework that iteratively generates importance maps to select diagnostically relevant regions, produces natural language descriptions of these patches, and then regenerates subsequent
3

---

importance maps conditioned on previous descriptions. However, PathFinder primarily operates through region selection at a fixed magnification level within predefined 512x512 patches, lacking the dynamic multi-scale navigation capability that characterizes pathologist workflows.
To address this gap, CPathAgent takes step further by incorporating multi-scale navigation capabili- ties that mimic pathologist behavior. It autonomously decides where to examine, when to zoom in or out, and how to navigate across WSIs, closely mimicking the iterative diagnostic workflow of expert pathologists. Notably, CPathAgent provides verbalization of its diagnostic reasoning, effectively simulating the internal thought processes that pathologists experience during slide examination (specific examples can be found in Appendix Figures A12 to Ao1). This approach enables enhanced interpretability through traceable reasoning paths and improved diagnostic performance.
2.3 Pathology Benchmarks and Datasets
Benchmark datasets provide standardized evaluation frameworks crucial for advancing computational pathology. Traditional WSI benchmarks like Camelyon16 1 [42] and TCGA [3 focused primarily on slide-level classification tasks. More recently, datasets like WSICaption 44 and WSI-VQA 45 have
expanded the scope to more complex tasks, including WSI description generation and VQA.
In parallel, several patch-level benchmarks have emerged for evaluating multimodal models in pathology. Path VaA [46] introduced the first VQA tasks for pathology images, drawing on textbook image-caption pairs. QuittVQA 477 expanded this approach by extracting image-caption pairs from pathology lectures on YouTuue, resulting in figher--uality VQA pairs due to the educational nature of these materials. PathMMU [48] marks more recent advance, offering a comprehensive benchmark across diverse pathology sources validated by expert pathologists to ensure clinical relevance.
Despite this progress, existing benchmarks predominantly focus either localized patch-level tasks or broad WSI analysis, overlooking a critical intermediate scale: the huge region. This scale is particularly important because it mirrors real-world diagnostic workflows. In routine practice, pathologists typically begin by scanning the WSIs to identify suspicious areas, then focus their diagnostic attention on these large, yet manageable regions. It is often within these huge regions that key diagnostic decisions are made, as they provide sufficient context for interpretation.
To address this limitation, we introduce PathMMU-H2 benchmark specifically designed for huge region analysis in pathology. By targeting this intermediate scale between isolated patches and WSIs, PathMMU-HRA better aligns with how pathologists actually examine and interpret tissue. It provides a more controlled, context-rich, and diagnostically meaningful setting for evaluating multimodal models, especially those incorporating agent-based reasoning across different scales. To ensure clinical relevance, we also engaged three board-certified pathologists to validate the dataset, making PathMMU-HR* an expert benchmark for intermediate-scale pathology evaluation.
3 Methods
In this section, we detail the construction of CPathAgent-Instie Instruct training data, the PathMMU-HE benchmark, and the CPathAgent model architecture with its multi-stage training process.
3.1 Reasoning Workflow of CPathAgent
To simulate expert-level diagnostic reasoning, we design CPathAgent as an agent-based system capable of dynamic region selection, strategic navigation planning, and multi-scale reasoning. As illustrated in Figure 2 the entire diagnostic process is decomposed into three key stages:
1) Global Sereening: To address the computational challenge of gigapixel WSIs, CPathAgent employs a toasse-to-fine region selection that mirrors pathologists systematic screening protocol.
Given WSIZ e #x333, we first generate thumbnail Tummb via 32x downsampling and partition it into a grid of N regions with 5% overlap to ensure boundary continuity: G = {91. 92. (21.92..... ... ........ . }.
where each y corresponds to a 16000 x 16000 pixel region at 40x magnification. CPathAgent So takes the annotated thumbnail input and generates structured text outputs: (1) Region grouping: CPathAgent clusters regions with similar pathological characteristics into K diagnostic regions
C (Ci...., Ck }, where each cluster Ci 1......... Ric, } is assigned a descriptive semantic

---

Global Screening
Navigation Planning
Multi-scale Multi-view Sequence Reasoning
Huge ragion
Overview
Zoom
Move
Focus
Crop Sequence
Navigation peth
vicw
Trace Agent
Navigate Agent
Observe & Reasoning Agent
Group
Pizz nov novipation path
performs multt scole obsen rensoning ecross sequence Of WIEW
Meg
Pamton
Reasoning
me start with overall view 1.0 [2RAA]]: Left shows derse lirous
Skart
Mra
Eivi Loc
Reaton
anima grewn entier: nute ciplee co moitee ports in
Let me zoom on the transition zone 5.0w (00.0(00...0 (1.44.15.77 75,04. infirrathe
Mila
Paxion
contin incicee
upper right
Locate
focus deeper on the lower right 4.Dc (D.C55 BEA coam
Finally, examining the interface at 4.0 (00 0,0.5A7771 DCS to-iaaeee
Discard
Preserve
(moderate pleom orphimm iim missus
Chater
Summary Description. The breast tisue invasive ductal differentaztion, muchn pook, evidence of ductal carcinama with
Chnter
Figure 2: Illustration CPathAgent framework, which mimics the diagnostic workflow of pathologists via global screening, navigation planning, and multi-scalc reasoning across sequential views.
label 4. (c.g., "Core Tumor Regions," "Lymph Node Assessment," "Background Adipose and Stromal
Tissue"); (2) Region prioritization: For each cluster Ck. CPathAgent predicts a severity level 8 E {0. 1, .5) where higher values indicate greater clinical importance and inspection priority, along with binary decision dę E {0. 1} indicating whether C2 requires high-magnificationn review (dg = 1) or can be skipped (ds- 0). This stage produces structured output R = {(l,, Ck, 8k, dx H that filters uninformative regions (#k- = 0), reducing computational burden for subsequent analysis.
For each preserved huge region r, the model generates dynamic nav- igation plan sequence of viewing steps. Each step is specified by tuple (z, y, m, m, o). where (z, y) denotes spatial coordinates in normalized space [0, 1], m represents the magnification level relative the original huge region (where 1.0x corresponds to the full view), and describes the diagnostic focus for that position (i.e., what features need to be observed). The navigation plan ,r, my, 0r)} generated autoregressivell. where each step conditioned on the visual content of the region and WSI source information (e.g., lung, colon, etc.).
2) Navigation Planning:
P. { y1, m1, 01),..................... ... (IT, yr,
Following the planned navigation path P., the model receives the complete sequence of cropped images corresponding to all viewing steps {(311 yı, mi, 01),,........ (wT, yr, MIT, oT)) as input once. Given this multi-view image sequence along with their associated diagnostic focus descriptions and spatial coordinates, the model performs holistic reasoning across the entire viewing trajectory in a first-person perspective similar pathol- ogist's examination process. The reasoning process systematically observes features across multiple scales and positions, establishes and refines diagnostic hypotheses by cross-eefernnning evidence, and maintains logical continuity throughout the navigation sequence. Finally, the model synthesizes all observations into a coherent pathological report that summarizes the diagnostic findings.
3) Multi-scale Multi-view Sequence Reasoning:
Illustrative examples, including global screening results, navigation path planning, and multi-scale reasoning outputs, shown in Appendix Figure A12 Figure A15 The prompts used for CPathAgent through these stages are provided in Appendix Figure A26 to Figure A28
3.2 CPathAAgent-Isstuuct Dataset Construction
In contrast to methods relying on inference-tine nimeencce-tmmeeeee prompting with closed-source models (e.g., OpenAl 03, Gemini-2.5-Pro we focus on curating high-quality data to train open- source models for expert- level agent- besent pathology analysis. To this end, we construct the CPathAgent-Insseee Instruct dataset, comprising corresponding subsets that target the three critical stages detailed in Section 3. While we use Gemini-2.5- P-o following the approaches in Section 3.1 to synthesize training data, prompting alone proves insufficient for expert-level performance. We therefore incorporate reference information such WSI reports guidance to ensure data quality. The construction process is follows:
5

---

Retrieval-guided Region Description
scate Patch Captioning
Multi-scale patch crop
40X
Fixed Region Sequence (21 patches)
Numn
Suph
Numris
1X
1X
2X
4X
4X
Gemini
Reglon description
Face Miiee
Gemini
+
the left breant alat-spur ry mantecionny demoretrates invasive dustal caronoma characteriss and by mucinous fastures and is graded as 5BR grade There associated ductal corcinom in stu (DCS) of incormeiiate nudear grade, with no necrosi observed Microcllittaaiin are present in conjunction with the inasive card noma
1X
Multi-scale Navigation Diagnostic Reasoning
Gemini
Reasoning Summory
([ULLA))::
dense tirous shroma
art
Variable Region Sequence (N patches)
Eetematuus/muutee stroms, suggecting reoplasm, Let me zoom is on the transiion zone 0x (00...) 1.0.55.00 463) inlibrative
Gemini
Now. moving upper right (j005,555,00000) Expanded
Figure 3: Overview of the generation process for CPathAgent's navigation planning subsct and multi-scale multi-view sequence reasoning instruction-tuni data.
Source Data: We use WSI reports from HistGen 49 and corresponding WSIs from TCGA, splitting 80% (5,254 WSIs) for training and 20% for testing and PathMMU-HRR generation. We apply patient-level splitting to ensure no data leakage across sets. Details are provided in Appendix A
This dataset follows the same WSI processing as CPathAgent's global screening stage, with one key difference: we leverage Gemini-2.5-Pro, using WSI overview as primary inputs and corresponding WSI reports additional guidance, to generate high-quality structured outputs (region groupings, priority scores, and examination flags) as training data.
Global Screening Subset:
Navigation Planning Subset: We implement refereece-guided three-step generation pipeline to synthesize high-quality navigation plans P. = {(Z1, V1. m1, 01)..... (ØT, yr, my, or)} that decide when and where to examine tissue at different magnifications, leveraging expert knowledge from WSI reports as shown in Figure 3 1) Retrieval-guided Region Description: For each important region identified during global screening, we extract region-specific descriptions from expert WSI reports by prompting Gemini-2.5-Prrs with the WSI report and region image, extracting a region description Dregion that captures the features of this region. 2) Multi-scale Patch Captioning: To provide comprehensive visual references for navigation path generation, we segment patches at three scales: x (overview), 2x (four views), and 4x (sixteen views), forming multi-scale patch set {IIx ,I I }. We then prompt Gemini-2.5- 2.5-Pro with these patches and D Dregion to produce descriptions {dl, d2. d2.-... d21) for these patches that guide subsequent navigation path generation. 3) with towddiintt-annnttaee (marked with 0.1 relative position intervals), patch descriptions {di...., d21 }. and Dregion to generate navigation plans where each step (Iz, Vt. m, ot) can navigate to any position and magnification, enabling flexible navigation strategies that fully mimic pathologist rather than fixed sequential routes.
Navigation Path Generation: We prompt Gemini-2.5-
region images
The synthesis of this subset follows the same process as CPathAgent's 's multi-scale reasoning stage (Section B.II). In addition to the cropped image sequences extracted along the planned navigation paths, we provide the region description D Dregion as reference guidance to Gemini-2.5-Pa for generating high-quality step-by-step reasoning chains.
Multi-scale Multi-view Sequence Reasoning Subset:
VQA-oriented Subset: To enable CPathAgent to handle both general diagnosis and VQA scenarios, this is additional subset that adapts the navigation planning process to question-driven contexts. We first prompt Gemini-2.5- P.o-Pr to generate pathology questions from multi-scale patch descriptions {d1, d2..... ... d21 J. then generate question-oriented navigation paths by prompting with both the question and region description Dregion- Finally, we prompt Gemini-2.5-Pai with the cropped image sequences along with the question and region description Diegion to produce question-oriented reasoning chains. This trains CPathAgent to dynamically adjust its examination strategy based on question-oriented targeted areas and magnifications to efficiently answer diagnostic questions.
Overall, CPathAgent-Instruuee Instruct comprises 278K instruction-tuning samples for training CPathAgent's pathologist-Ie -ike agent capabilities. Detailed data statistics provided in Appendix A

---

3.3 PathMMU-HR'' Dataset Construction
To evaluate CPathAgent's multi-scale reasoning capabilittie on huge regions, we construct PathMMU- HR², a Huge Region Huge Resolution benchmark specifically designed to assess the model's capa- bility for huge region analysis. We sample huge regions from the held-out TCGA test set, ensuring broad coverage across tissue types (e.g.. TCGA-BRCA, TCGA-LUAD), Following the construc- tion pipeline of Franthhgen-nnstruuce VQA-oriented subset, we generate VQA pairs that require synthesizing observations across different scales, necessitating multi-scale reasoning for accurate diagnosis. Three board-certifidd pathologists independently review and filter generated VQA pairs based on clinical relevance and diagnostic accuracy, necessity of multi-scale integration, and align- ment with standard pathology practice. The final PathMMU-HR comprises 1,668 expert-validated VQA samples, providing a robust benchmark for evaluating huge region analysis capabilities.
3.4 CPathAgent Model Architecture and Training
CPathAgent builds upon the LLaVA-OneVision (50), integrating Qwen3-14B (51 the LLM backbone and CPath-CLIP [00] as the vision encoder, connected via a two-layer MLP. We adopt three-stage progressive training strategy that systematically develops CPathAgent's 's capabilities, advancing from foundational multimodal understanding to agent-based diagnostic reasoning.
To be specific, stages 1 and 2 follow the CPath-Omni [10] training protocol: stage 1 aligns vision and language components using CPath-Patche PatthhaPcchaatioon with only the MLP trainable, while stage fine-tunes all parameters CPath-PathInstuu PathInstruct to develop comprehensive patch-level pathology understanding. including VQA, classification, and image description capabilities.
Notably, stage 3 is the critical phase that transforms CPathAgent into pathologist like agent. During this stage, we train on our specialized EPaathggnt-Instrrue dataset, supplemented with 20% of CPath- Instruct data, with all parameters unfrozen. This phase equips CPathAgent with advanced agent capabilities that emulate pathologists diagnostic workflows while preserving the strong patch-level analysis skills from earlier stages. This multi-stage approach yields versatile model that seamlessly integrates fine-grat grained patch-level analysis with pathologist-lic agent-based WSI navigation.
Experiments
We evaluate CPathAgent across multiple tasks: patch understanding, huge region analysis, and WSI classification to assess its capabilities and benchmark performance against existing approaches.
4.1 Patch Understanding Evaluation
Since patch-level understanding serves as the foundation for reasoning over larger-region reasoning, first evaluate CPathAgent's 's ability interpret caandad--esolutien pathology patches Path- MMU [48], currently the largest and most diverse expert- verified pathology dataset. We compare CPathAgent with the most advanced general-purpose LMMs, including InstructBLIP-LLAN-
XXL LLa 1.5- 3B LLaVA- OneeVssion (50 series, Swven2.5-VL.5.. series, GPT-V (55)
GPT-4. [56] series and Gemini-2.. 2.5 series, along with domain-specinf pathology LMMs such as
LLaVA-Med (5), Quilt-LLaVA 8 PathGen-LLaaA (6 and CPath- meath-Omii[ii Omni[10).
Results: CPathAgent significantly outperforms both general-purpose and pathology-specific LMMs. As shown Table [ it achieves the highest performance across all subsets, with 80.5% on the tiny test set and 78.6% on the full test set, surpassing the strongest general-purposs model (Gemini-2.5-PP) 68.7% Tiny/ / 37.5% All) and surpasses the SOTA comain-specific model (CPath-Omni 72.4% Tiny / 72.2% All). While CPathAgent shares patch-level training data with CPath-Omni, its superior performance stems from the third-stage training on high-quality, multi-scale data, which strengthens its analytical capabilities and lays the foundation for agent-based diagnostic workflow.
CaathAgent even outperforms the expert-annotated baselines in several subsets, suggesting that its general patch-level understanding is already quite promising. This may also reflect limitations in human annotations, as individual experts often specialize in narrow domains and may struggle with unfamiliar cases in a diverse dataset like PathMMU. While CPathAgent benefits from broad training and shows potential as seneral-purpose pathology model, pateh-level tasks remain relatively simple compared to more complex reasoning across larger spatial contexts.
7

---

Table 1: Overall results of models the PathMMU test set. The best-performing LMM in each subset for general and pathology LMMs in-hold, and the second-hest performing LMM underlined.
Test Overall
PubMed
SocialPath
EduContent
Atlas
PathCLS
Tiny (1156)
Tiny (235)
Tiny (255)
All (1938)
Tiny (208)
(9677)
(281)
(3068)
(1855)
(1007)
(177) (1809)
Expext performance
71.8
69.0
68.3
General Large Multimodal Models
34.3
33.9
39.1
37.2
33.6
34.3
34.5
36.0
38.5
39.3
22.6
MaVA-I15--3R
38.8
37.6
44.5
41.0
40.4
40.4
34.1
39.4
47.1
44.3
24.9
23.5
LLaVA-OneVision
36.9
34.4
37.7
36.4
35.7
38.4
47.1
38.3
38.9
38.4
20.3
20.4
MaVA--neVisioo-
59.1
52.9
54.9
39.7
37.5
40.2
40.2
41.7
39.0
47.1
42.0
40.4
38.9
24.9
25.6
Owen2.5-V1.-22--mee
51.2
63.3
S6.1
55.5
65.5
58.3
54.3
$5.2
26.0
GPT-4V-1106
53.9
49.8
5944
53.5
58.7
53.9
60.4
53.6
48.1
52.8
36.2
33.8
GPT-4. 1-mini-2025-04-14
60.7
59.9
66.9
63.4
62.1
61.8
65.1
61.1
60.1
62.8
43,5
48.8
1PF---1-2025-0--11
Gomini-2.5- 2.5iiii-.5--iiss--Peee Fissh--tcoviiw000-17
68.0
65.2
74.9
69.2
71.1
64.9
68.1
65.3
67.5
69.3
53.7
56.3
Gemini-2.5 1eminn-2.55-Poo- P-o--PP--PP- P1----eee-eis
68.7
67.5
73.8
21.5
72.1
68.1
20.2
68.6
68.3
20.6
$7.5
Patbology-specin Large Multimodal Models
LLaVA-Med
25.3
26.2
28.5
27.7
28.9
27.3
22.7
27.2
22.6
30.7
22.6
20.3
Quilt-LLaVA
45.6
41.5
47.3
42.6
46.4
46.6
51.8
45.3
46.2
42.7
32.2
29.2
LLaVA
60.1
58.4
60.1
60.11
60.9
58.8
60.8
60.7
63.5
64.9
54.2
48.9
CPatb-Omai
72.4
70.6
CPathAgent (Ours)
80.5
78.6
80.8
78.4
83.4
77.9
83.5
79.6
75.4
77.6
78.0
79.2
4.2 Huge Region Understanding Evaluation
We evaluate CPathAgent's ability to autonomously navigate and reason over huge, high-resolution re- gions using PathMMU-HR?, which includes 1,668 expert-validated VQA pairs requiring multi-scale analysis. We benchmark the same models as in Section 4.1 Additionally, we apply CPathA- gent's prompt (from global screening multi-scale reasoning, detailed Section 3.1 stronger agent-capable models like Gemini-2.5-PP and GPT-4. 1144.1-mini. enabling them to mimic pathologists' workflows as CPathAgent does and assess whether this approach yields performance gains.
Results: CPathAgent demonstrates superior performance on large, high-resolution regions across diverse cancer types. As shown in Table CPathAgent achieves 88.6% the PathMMU- HR2, substantially outperforming both general-purpose models (Gemini-2.5-PPre by 15,4%) and pathology-specific models (CPath-Oni Omiii by 16.9%). Despite operating at lower single-view resolu- tion (1008x100) 1008) compared to closed-source models like Gemini-2.5-PP5 (3072x3072), CPathAgent compensates through emulating pathologists' diagnostic workflow through strategic navigation and multi-view reasoning over huge regions. Notably, while CPathAgent and CPathOmni share patch- level training data from stages 1-2 (Section 3.4). they differ significantly in their stage 3 training, where CPathAgent incorporates agent-based based navigation and reasoning for WSI and large region analysis. The substantial performance gain validates the effectiveness of our CPathAgent-Insse dataset and agent training approach. Representative examples are provided in Appendix E
Agent-based approaches consistently enhance model performance for pathology analysis. As shown in Table incorporating agent-based approach improves performance across models. Specifically, with CPathAgent's prompting strategy, Gemini-2.5-PPr improves from 73.2% 76.4% and GPT-4.1 mini from 60.1 1 % to 62.3% %. The enhancement particularly pronounced in complex cancer types like KICH, where Gemini-2.5-Prr shows dramatic improvement from 56.5% 69.3% (+12.8%). This confirms that dynamically navigating and reasoning over huge regions that are similar to expert pathologists is crucial for accurate diagnosis, especially for cases requiring integration of findings across multiple scales and regions. These results suggest that empowering LMMs with expert-like navigation and reasoning strategies a promising direction for advancing AI pathology tasks.
4.3 WSI Classification
We evaluate CPathAgent's performance on WSI classification tasks across six diverse datasets spanning multiple cancer types and diagnostic tasks: TCGA-BRCA (breast cancer subtyping), TCGA-NSCLCC (lung cancer subtyping). TCGA-RCC (renal cell carcinoma subtyping). TCGA- ESCA (esophageal cancer subtyping), TCGA-BLCA (bladder cancer subtyping). and TCGA-THCA (thyroid cancer subtyping). We benchmark CPathAgent against traditional MII. approaches, including ABMIL 4 and DSMIL (5), as well as both pcrproe-purpose and pathology LMMs.

---

Table 2: Performance comparison of huge region classification across multiple cancer types using approaches: General L.Mss, and Agent-based Methods. The best-perfrrming model for each cancer type is in-bold, and the second-bes performing model underlined.
BRCA (368)
LUAD (192)
LUSC (230)
KIRP (153)
KIRC (151)
KICH (62)
ESCA (70)
THCA (249)
BLCC (141)
TGCT (52)
Overall (1668)
General LMM Approach
42.9
43.2
33,5
28.1
57.1
43,5
40.0
41.8
39.7
30.8
38.8
MaVA-Onn Vision-Qwen2-273
Qwen2-72R-OV
52.1
40.0
39.2
33.9
45.8
57.4
36.5
43.0
Owen2.5 Instruct
47.3
57.3
35.2
46.4
47.7
25.8
51.4
49.8
56.0
38.5
46.9
Dwen2.5-V1..22R--0 Instmmtt
48.4
61.5
44.3
52.9
56.3
27.4
52.9
48.6
62.4
51.9
$1.2
GPT-..-mmni- 202-044422--
56.0
64.6
60.4
62.1
61.6
35.5
70.0
59.4
68.1
59.6
60.1
GPT-4. 1-2025-04-14
57.9
63.5
62.2
67.3
69.5
54.8
65.7
66.3
67.4
71.2
63.7
Gemini-2.5-Flann 2..5.aaahh.Preevee...
76.1
40.3
72.7
76.6
63.5
70.4
Gemini-2.5.. 2.5-i-2.PPPrroo Pro-ProPPeeee Preview-03-25
64.4
70.8
78.3
74.5
80.8
$6.5
77.1
76.7
80.1
75.0
73.2
Quilt-LLaVA
29.3
45.8
44.8
38.6
38.4
30.6
35.7
39.8
47.5
40.4
38.8
PathGen-LLaVA
62.0
27.6
67.4
58.2
74.2
48.4
67.1
84.4
48.1
67.2
67.2
CPath-Oma
72.6
27.6
71.3
64.1
67.5
59.7
74.3
71.5
76.6
78.8
71.7
Agent- based Approach
GPT-4 1-mini-2025-04-14
56.0
69.3
63.9
62.7
64.9
41,9
71.4
59.8
68.1
73.1
62.3
Gemini Preview 03-25
80.0
77.8
82.8
78.7
85.1
76.4
CPathAgent (Ours)
87.0
88.5
87.8
87.9
92.9
78.9
90.7
89.0
90.7
93.0
88.6
Since CPanhAgent generates detailed diagnostic descriptions rather than dataset-specific classification labels like MIL methods, we implement two-stage evaluation process for fair comparison with WSI benchmarks. In the first stage, CPathAgent employs our agent-based approach to identify suspicious regions and generate comprehensive diagnostic summaries of significant findings within each WSI. While these detailed descriptions effectively capture complex pathological characteristics, they must be mapped to the predefined classification schemes used in standard benchmarks. Therefore, in the second stage, we provide these descriptions to Gemini-2.5-PP for conversion into the specific labels required by each dataset. Consequently, more accurate and detailed diagnostic descriptions from CPathAgent lead to more precise WSI-level classifications. For other LMM- approaches, following CPath-Omni [10], we directly prompt the models to generate descriptions for all WSI regions. These descriptions are then similarly provided to Gemini-2.5-PPu to map them to the corresponding classification labels, ensuring fair evaluation across all methods.
CPathAgent achieves competitive performance on WSI classification using fundamen- tally different agent-based approach. As shown in Table CPathAgent achieves an average accuracy of 82.8% across six different cancer classification tasks, outperforming traditional MIL approaches (ABMIL: 79.9%, DSMIL: 76.8%) and substantially surpassing both general- purpose LMMs like Gemini-2.55Pro (72.1 1)) and previous aathology-specifee SOTA CPath- (77.4% (7.4%%). CPathAgent shows particularly strong performance on TCGA-BRCA (88.5% and TCGA-ESCA (97.1%) 1%), where it matches exceeds specialized MIL methods. Although the overall margin over MIL approaches is modest (+2.9% vs. ABMIL), this represents a significant paradigm shift: CPathAgen achieves these results through interpretable, expert-like diagnostic reasoning rather than opaque feature aggregation.
Results:
Upper bound results validate synthetic data quality and reveal substantial room for improvement. To assess data quality, we establish a theoretical upper bound by directly using our generated agent training data for WSI classification (Table 3. The near-perfect upper bound accuracy across 97.9%, TCGA-RCC: 96.5%, TCGA-BRCA.
most datasets (TCGA-ESCA: 100%, TCGA-NSCLA 97.0%) confirms that our synthetic navigation trajectories and diagnostic reasoning are clinically accurate and diagnostically complete. While CPathAgent achieves significant advances with 82.8% overall accuracy, the 9% gap to the upper bound (91.7%) indicates considerable room for further improvement, especially on challenging datasets like TCGA-BLCA and TCGA-THCA. As an agent- based exploration for pathology, CPathAgent demonstrates substantial potential for advancement.
4.4 Out of Distribution Evaluation
Although our model achieves strong diagnostic performance across multiple TCGA cancer types, we further evaluate its generalization capability on -iitr---iitiittiion (OOD) datasets. Specifically, we
assess CPathAgent on two tasks: (1) binary WSI classification for distinguishing LUAD and LSCC in
the CPTAC-Lung dataset (59 60), and (2) three-class classification (benign tumors, atypical tumors,
malignant tumors) on huge image regions (up to 10K 10K pixels) in the BRACS dataset [61 We
benchmark our method against representative WSI-level (PRISM [9. CPath-Omni 10] TITAN 28
and region-level models (GPT-4.1, Gemini-2.. 2.5-Poo CPath-Omni (10).

---

Table 3: Overall results of WSI classification tasks. The best-performing model in each subset is in-bold, and the second-hest performing model is underlined. CPath-Omni indicates retrained version using CPathAgent train/test split to ensure comparison, as original training data differed. Balanced accuracy (%) reported.
TCGA-BRCA TCGA-NSCLC TCGA-RCC TOGA-ESCA
TCGA-BLCA
TCGA-THCA
Avg.
Multi-instance Learning Approach (Truditiona)
ABMIL
80.5
92.8
96.4
91.2
50.1
68.4
79.9
Large Multimodal Models
$2.8
68.2
$2.8
85.3
$7.0
54.3
59.6
177-4-1-2025-04
61.0
53.8
61.3
58.9
64.3
Gemini-22 2.5 Flaas--rreviiw 14-1i
56.7
84.5
57.9
93.8
61.8
55.4
68.4
Gemini-2.55 P.i..-2--P--PPPP- Pro-Preview-03325
72.4
69.2
59.8
44.9
72.1
Quil--LaaA
55.3
54.0
53.3
58.6
61.9
45.8
54.8
MethGee--L.a.V
66.5
53.9
$9.0
78.6
54.7
49.2
60.2
CPaill-runi
Agent-based Approach
CeathAgent (Oun)
88.5
90.8
94.6
97.1
62.7
63.2
82.8
Upper Bound
97.0
97.9
96.5
100.0
77.8
81.0
91.7
CPTAC-Lung SW
BRACS (region).
100
88.1
Balanced accuracy
83.2
22.3
57.9
80
60
-9.0
44.2
60
40
50
40
PRISM
CPath-Dmni
CPathAgent
TITAN
GPT-4.1
Gemini-2.5PPr CPatn-Omel
CPathAgent
Figure 4: 0OD results for WSI (left) and region-level (right) classification tasks. Representative models from WSI and region-level approaches are selected for comparison.
Results: CPathAgent achieves competitive 0OD performance with exceptional data efficiency. As shown in Figure + despite being trained on significantly fewer slides than baseline models, CPathAgent demonstrates reasonable generalization on both WSI-level and region-level OOD tasks. CPTAC-Lung. CPathAgent achieves 88. I % balanced accuracy with only 5.254 training WSIs, outperforming CPath-Omni (82.3%. 11,728 WSIs) and PRISM (83.2%, 587,196 WSIs), while slightly below TITAN (91.9%, 335,645 WSIs). Notably, CPathAgent uses 64x fewer WSIs than TITAN, 2.2x fewer than "Path-Ommi and 112x fewer than PRISM, yet achieves competitive or superior performance. For BRACS region classification, CPathAgent achieves 64.3% accuracy, substantially surpassing general LMMs GPT-4.1 (44.2%) and Gemini-2.5-Pro (49.0%), well the CPath-Omni (57.9%), demonstrating its capability in huge regions analysis.
While current OOD performance may not yet meet clinical deployment standards, these results demonstrate meaningful generalization beyond TCGA with remarkable data efficiency, suggesting that scaling to larger datasets represents promising direction for further improvement.
Conclusion
In this study, we present CPathAgent an agent-based framework for computational pathology that emulates pathologists' diagnostic reasoning through strategic examination of high-resolution pathol- ogy images. Through systematic three-stage process of global screening, navigation planning, and multi-scale sequence reasoning, CPathAgent faithfully replicates the critical diagnostic workflow by dynamically zooming in and out while systematically shifting focus across regions of interest, transparently documenting not only what features are observed but also why specific regions warrant closer examination and how evidence accumulates across viewing scales, thereby achieving both superior interpretability and ffute-of-the-eee performance across diverse pathology tasks. Our compre- hensive evaluation spanning three scales of image analysis (patch-level understanding. large regional assessment, and whole slide image analysis) demonstrates that this agent-based paradigm, which closely mirrors how experienced pathologists navigate and scrutinize tissue specimens, effectively bridges the crucial gap between human diagnostic expertise and Al-assisted analysis.
10

---

deep multiple instance 6 Acknowledgements
This study was partially supported by "Pioneer" and "Leading Goose" R&D Program of Zhejiang (Grant 2025SDXHDX0003, the National Natural Science Foundation of China (Grant No.625063000 foundation of Muyuan Laboratory (Program ID: 14106022401,141066000 and the Westlake Education Foundation.
References
[1] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. Featured Certification.
[2] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pages 9650-9660, October 2021.
[3] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763.873..662 PmLR, 2021.
[4] Maximilian Ilse, Jakub Tomezak, and Max Welling. learning. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2127-2136. PMLR, 10-15 Jul 2018.
[5] Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with supe-supe contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11318-14328, 2021.
[6] Yunlong Zhang, Honglin Li, Yunxuan Sun, Sunyi Zheng. Chenglu Zhu, and Lin Yang. Attention- challenging multiple instance learning for whole slide image classification. In European Conference on Computer Vision, pages 125- 143. Springer, 2024.
[7] Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou. A visual-Language foundation model for pathology image analysis using medical twitter. Nature medicine, 29(9):2377-2366 2023.
[8] Wisdom Ikezogwo, Saygin Seyfioglu. Fatemeh Ghezloo, Dylan Geva, Fatwir Sheikh Mo- hammed, Pavan Kumar Anand, Ranjay Krishna, and Linda Shapiro. Q One million image-text pairs for histopathology. Advances in neural information processing systems, 36:37995-38017, 2023.
[9] George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D Kunz, Juan A Retamero, Gerard Oakley, David Klimstra, Christopher Kanan, et al. Prism: A multi-modal generative foundation model for slide-level histopathology. arXiv preprint arXiv:2405.100 10254, 2024.
[10] Yuxuan Sun, Yixuan Si, Chenglu Zhu, Xuan Gong, Kai Zhang, Pingyi Chen, Ye Zhang, Zhongyi Shui, Tao Lin, and Lin Yang. Cpath-omni: A unified multimodal foundation model for patch and whole slide image analysis computational pathology. arXiv preprint arXiv:2412. 12077, 2024.
[11] Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma. and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560, 2023.

---

[12] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, al. Webept: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:21 2021.
[13] Timo Schick, Jane Dwivedi-Yi Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539- 68551, 2023.
[14] Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303. 2023.
[15] Joon Sung Park, Joseph O' O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceed- ings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST '23, New York, NY, USA, 2023. Association for Computing Machinery.
[16] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang. Ceyao Zhang, Zili Wang. Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, 2024.
[17] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Chatdev: Communicative agents for software development. arXiv preprint wxXi:2307.00994. 2023.
[18] Gabriele Campanella, Matthew G. Hanna, Luke Geneslaw, Allen Miraflor. Vitor Werneck Krauss Silva, Klaus J. Busam, Edi Brogi, Victor E. Reuter, David S. Klimstra, and Thomas J. Fuchs. Clinical- Elinical-grade computational pathology using weakly supervised deep learning on whole slide images. 25(8):3301-333
[19] Osamu lizuka, Fahdi Kanavati, Kei Kato, Michael Rambeau, Koji Arihiro, and Masayuki Tsuneki. Deep learning models for histopathological classification of gastric and colonic epithelial tumours. 10(1):1504.
[20] Hanwen Xu, Naoto Usuyama, Jaspreet Bagga, Sheng Zhang, Rajesh Rao, Tristan Naumann, Cliff Wong, Zelalem Gero, Javier González, Yu Gu, Yanbo Xu, Mu Wei, Wenhui Wang, Shuming Ma, Furu Wei, Jianwei Yang. Chunyuan Li, Jianfeng Gao, Jaylen Rosemon, Tucker Bower, Soohce Lee, Roshanthi Weerasinghe, Bill J. Wright, Ari Robicsek, Brian Piening, Carlo Bifulco, Sheng Wang, and Hoifung Poon. A whole-slide foundation model for digital pathology from real-world data. Nature, 2024.
[21] Eugene Vorontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Kristen Severson, Eric Zimmermann, James Hall, Neil Tenenholtz, Nicolo Fusi, al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nature medicine, 30(10):2924-2935 2024.
[22] Erie Zimmermann, Eugene Vorontsov, Julian Viret, Adam Casson, Michal Zelechowski, George Shaikovski, Neil Tenenholtz, James Hall, David Klimstra, Razik Yousfi, et al. Virchow2: Scaling self-supervised mixed magnification models in pathology. arXiv preprint arXiv:2408.00733 2024.
[23] Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Andrew H Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, et al. Towards a general- purpose foundation model for computational pathology. Nature Medicine, 30(3):850-862, 2024.
[24] Yuxuan Sun, Chenglu Zhu, Sunyi Zheng, Kai Zhang, Lin Sun, Zhongyi Shui, Yunlong Zhang, Honglin Li, and Lin Yang. Pathasst: A generative foundation ai assistant towards artificial gen- eral intelligence of pathology. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5034-5042, 2024.
12

---

[25] Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Melissa Zhao, Aaron K Chow, Kenji Ikemura, Ahrong Kim, Dimitra Pouli, Ankush Patel, et al. A multimodal generative ai copilot for human pathology. Nature, 634(8033):466 473, 2024.
[26] Yuxuan Sun, Yunlong Zhang, Yixuan Si, Chenglu Zhu, Zhongyi Shui, Kai Zhang. Jingxiong Li, Xingheng Lyu, Tao Lin, and Lin Yang. Pathgen- 1,6 m: 1.6 million pathology image-text pairs generation through multi-agent collaboration. arXiv preprint arXiv:2407.00222 2024.
[27] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023.
[28] Tong Ding. Sophia J Wagner, Andrew H Song, Richard Chen, Ming Y Lu, Andrew Zhang, Anurag J Vaidya, Guillaume Jaume, Muhammad Shaban, Ahrong Kim, et al. Multimodal whole slide foundation model for pathology. arXiv preprint arXiv:241 1.111....666 2024.
[29] Ying Chen, Guoan Wang, Yuanfeng Ji, Yanjun Li, Jin Ye, Tianbin Li, Ming Hu, Rongshan Yu, Yu Qiao, and Junjun He. Slidechat: A large vision- lsngun-anguage assistant for whole-slide pathology image understanding. arXiv preprint arXiv:2410. 221244111166.... 11761, 2024.
[30]
Stuart Russell and Peter Norvig. Artificial intelligence: a modern approach. pearson, 2016.
[31] Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744-2000 20757, 2022.
[32] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[33] Significant Gravitas. AutoGPT.
[34] Microsoft. autogen.
[35] Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. Re3: Generating longer stories with recursive reprompting and revision. In Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing (EMNLP), 2022.
[36] Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes Autonomous chemical research with large language models. Nature, 624(7992):570-577 2023.
[37] Zhihao Fan, Jialong Tang, Wei Chen, Siyuan Wang, Zhongyu Wei, Jun Xi, Fei Huang, and Jingren Zhou. Ai hospital: Interactive evaluation and collaboration of lims as intern doctors for clinical diagnosis. arXiv e-prints, pages arXiv-2402, 2024.
[38] Junkai Li, Yunghwei Lai. Weitao Li, Jingyi Ren, Meng Zhang, Xinhui Kang, Siyu Wang, Peng Li, Ya-Qin Zhang, Weizhi Ma, et al. Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint arXiv:2405.02957 2024.
[39] Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, and Marinka Zitnik. Empow- ering biomedical discovery with ai agents. Cell, 187(22)66125-6151, 2024.
[40] Samuel Schmidgall, Rojin Ziaci, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. Agentelinic: a multimodal agent benchmark to evaluate in simulated clinical environments. arXiv preprint (:X:::2445.009900
2024.
[41] Fatemeh Ghezloo, Mehmet Saygin Seyfioglu, Rustin Soraki, Wisdom O Ikezogwo, Beibin Li, Tejoram Vivekanandan, Joann G Elmore, Ranjay Krishna, and Linda Shapiro. Pathfinder; A mudti-modal multi-agen system for medical diagnostic decision making applied to histopathol- ogy. arXiv preprint arXiv:2502.00 08916, 2025.
[42] Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes Van Diest, Bram Van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen AWM Van Der Laak, Meyke Hermsen, Quirine F Manson, Maschenka Balkenhol, et Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama, 318((2)22199-22) 2017.
13

---

[43] John N Weinstein, Eric A Collisson, Gordon B Mills, Kenna R Shaw, Brad A Ozenberger, Kyle Ellrott, Ilya Shmulevich, Chris Sander, and Joshua M Stuart. The cancer genome atlas pan-cancer analysis project. Nature genetics, 45(10):1133-112) 2013.
[44] Pingyi Chen, Honglin Li, Chenglu Zhu, Sunyi Zheng, Zhongyi Shui, and Lin Yang. Wsicap- tion: Multiple instance generation of pathology reports for gigapixel whole-slide images. In International Conference on Medical Image Computing and Computer-ANee pages 546-556 Springer, 2024.
Intervention,
[45] Pingyi Chen, Chenglu Zhu, Sunyi Zheng, Honglin Li, and Lin Yang. Wsi-vqa: Interpreting whole slide images by generative visual question answering. European Conference on Computer Vision, pages 401-417. Springer, 2024.
[46] Xuchai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003. 2020.
[47] Mehmet Saygin Seyfioglu, Wisdom O Ikezogwo, Fatemeh Ghezloo, Ranjay Krishna, and Linda Shapiro. Quilt-llava: Visual instruction tuning by extracting localized narratives from open- source histopathology videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13183-13192, 2024.
[48] Yuxuan Sun, Hao Wu, Chenglu Zhu, Sunyi Zheng, Qizi Chen, Kai Zhang, Yunlong Zhang, Dan Wan, Xiaoxiao Lan, Mengyue Zheng, al. Pathmmu: A massive multimodal expert-level benchmark for understanding and reasoning in pathology. In European Conference Computer Vision, pages 56-73. Springer, 2024.
[49] Zhengrui Guo, Jiabo Ma, Yingxue Xu, Yihui Wang, Liansheng Wang, and Hao Chen. Histgen: Histopathology report generation via local-global feature encoding and cross-modal context interaction. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 189-199. Springer, 2024.
[50] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03:: 033326, 2024.
[51] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arX1v:2505.09388, 2025.
[52] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36, 2024.
[53] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 26296 26300, 2024.
[54] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang. Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie. Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprini a50iv: 13923, 2025.
[55] OpenAL. -vision) system card. https://cch.opon www.di...pp.i....mmmm http:/chh.ppaai..cccappaapppeeee compppapprrrr///T/..yeeeeen Card. pdf 2023.
[56] OpenAl. Introducing g in the api. http:://oppnaiico///iee//////// hptp://pt-44444 2025.
[57] Koray Kavukcuoglu. Gemini 2.5: Our most intelligent ai model, March 2025. Accessed: 2025-05-16.
14

---

[58] Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training large sunguage-and-vise assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36:28541-28564, 2023.
[59] Michael A Gillette, Shankha Satpathy, Song Cao, Saravana M Dhanasekaran, Suhas V Vasaikar, Karsten Krug, Francesca Petralia, Yize Li, Wen- Liang, Boris Reva, al. Proteogenomic characterization reveals therapeutic vulnerabilities in lung adenocarcinoma. Cell, 182(1):200- 225, 2020.
[60] Shankha Satpathy, Karsten Krug, Pierre M Jean Beltran, Sara R Savage, Francesca Petralia, Chandan Kumar-Sinha, Yongchao Dou, Boris Reva, M Harry Kane, Shayan C Avanessian, et al. proteogenomic portrait of lung squamous cell carcinoma. Cell, 184(16):4348444 -371,, 2021.
[61] Nadia Brancati, Anna Maria Anniciello, Pushpak Pati, Daniel Riccio, Giosuè Scognamiglio, Guillaume Jaume, Giuseppe De Pietro, Maurizio Di Bonito, Antonio Foncubierta, Gerardo Botti, et al. Bracs: dataset for breast carcinoma subtyping h&e histology images. Database, 2022:baac093, 2022.
[62] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXfv:2412.19437, 2024.
[63] Yuxuan Sun, Hao Wu, Chenglu Zhu, Yixuan Si, Qizi Chen, Yunlong Zhang, Kai Zhang, Jingxiong Li, Jiatong Cai, Yuhan Wang, et al. Pathbench: Advancing the benchmark of large multimodal models for pathology image understanding patch and whole slide level. IEEE Transactions on Medical Imaging, 2025.
[64] Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6):555 5700, 2021.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims in the abstract and introduction an agent-based framework (CPathAgent), a multi-scale reasoning model, and a new benchmark (PathMMU- HR?) are well aligned with the actual contributions and evaluations detailed in our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss limitations, particularly in Appendix D It acknowledges that our diagnostic paths are synthetically generated rather than collected from real pathologists, the training dataset scale is smaller compared to recent foundation models, and the current scope focuses on diagnostic classification without extending to prognostic or biomarker detection tasks.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattio, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
16

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include any theoretical results formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed descriptions of the model architecture, training setup, datasets used, and evaluation protocols. All necessary information reproduce the main results included, and code and model checkpoints will be released in later stage.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility.
17

---

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The code and model checkpoints will be released in later stage. The supple- mental material provides detailed instructions for reproducing the main experimental results, including environment setup and data preprocessing steps.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.co... cttilss.cc/pubbcc/uuuuu Cttpae//CoeSSSeiiiiieeeeee SttSsuiiisiioooooiooy..oooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The code and model checkpoints will be released in later stage. The supple- mental material provides detailed instructions for reproducing the main experimental results, including environment setup and data preprocessing steps.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Although we do not report error bars or statistical tests, we believe the scale and consistency of our evaluations across large models and datasets are sufficient to demonstrate the significance of the results this level of evaluation.
18

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details about compute resources are provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research fully adheres to the NeurIPS Code of Ethics, including responsible data usage, transparency, respect for human rights, and care minimizing potential societal harms.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
19

---

Answer: [Yes]
Justification: The paper discusses the positive societal impact of improving diagnostic accuracy and efficiency in pathology through AL.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The study focuses on pathology-specific models and data, which do not pose significant risks of misuse or dual-use.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All existing datasets and code used in the study are properly cited, and their respective licenses and terms use have been fully respected.
Guidelines:
The answer NA means that the paper does not existing assets.
20

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license ge.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, pappessitthheeeocc thoode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pane re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:[Yes]
New assets, including dataset and model checkpoints, will be released in later stage, Documentation covering usage, license, and limitations will be provided at the time of release.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
21

---

Justification: Although human annotators were involved, their work did not constitute human subjects research as defined by our institution, and no IRB approval was required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: only use LLMs for refining, and formatting purposes.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolutionn Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic Supplementary Material
Table of Contents
A Additional Details of Proposed Dataset
24
A. Details of InashAgent-mmstruct
24
Details PathMMU-HR
27
B Additional Experiments and Discussion
B.1
Comparison of Description Generation Performance Among sptholo
LMMs
29
B.2
Pathologist evaluation results
29
Additional Results on Path -ro
30
Ablation Study on Agent Components
31
Ablation Study on Applying Gemini-2.5 as an Agent for WSI Tasks
31
Experimental Details
33
Details for based WSI classification
33
Detaiss for Patch Level Benchmark
34
Hardware Training Cost of CPahhAgel
C44 Training Hyperparameters
34
D
Limitations
35
E Examples and Prompts
36
Examples Region Descriptions Extracted from WSI Report Pairs
36
Examples PathMMU-HR Samples
E.3 Examples of Region Selection Process Based on WSI Overview
36
Examples of Navigation Paths and Multi-Scale Reasoning Generated by CPathAgent
36
Prompts
45
23

---

1X Back
2X
Trat cortite
Valid questions
1X
Gemini
Gemini
Which statement dharacterizes tissue's epithelia morphology and stromel context ?
Question:
Stept
Meg
Bioe los
Roason
Expanstve epthelal structures wet seve-ikk fenestrations and infilrate
Step2
Mag
Bbca Loc
Reason
Diffuse
open lumens bilayered spithelum... D. Large cytte spaces lined by attenuated opthellum adjoin complex ...
Predeminandly
Magg
Crap to Sequence
Gernini
LLMs Vote on and Filter Out Guessable Questions
Gertini
(V04 oniented Keasoning
barme start emt overall View 1.0x (10,0.1.1],, 1se
overall distro distribution lanee pro lererii foo and oeporate me
Invalid questions
Valid questions
Eduntify
glands
Therefore,
Figure Al: Overview of the VQA-oriented dataset generation process: from multi-scale description to question generation, question filtering, question-oriente navigation path generation, and VQA-related visual reasoning.
A Additional Details of Proposed Dataset
In this section, we present detailed information about our datasets: (PathAgent-Instrres and PathMMU-HR3. We provide comprchensive statistical analysis of both datasets, along with further details regarding their construction process.
A.1 Details of CPathAgent-Instruu
Data Sources: CPathAgent derives its WSI report data from HistGen 49], while the corresponding WSI data is sourced from The Cancer Genome Atlas (TCGA). TCGA provides a comprehensive collection of pathology WSIs contributed by various participating institutions, encompassing diverse range of distinct tissue and cancer types. This extensive repository ensures diverse coverage of pathological conditions and features essential for dataset and model development.
Additional Details CPathAgent-Instruus Our dataset utilizes 5,254 WSIs, which 80% the total HisGen data for constructing CPathhgget-Insstuuee The remaining WSIs are reserved for model testing and for constructing the PathMMU-HR' dataset.
We provide the following additional dataset construction details that could not be included in the main text due to space constraints:
I) Global Screening Subset Enhancement: For the region selection subset within CPathAgent- Instruct, face challenge this capability was not developed during earlier training stages (stage I and stage 2 as detailed in Section 3.4), and 5,254 WSIs are insufficient for acquiring this skill, Considering that region selection is manageable for advanced large models like Gemini-2.5-Pro and doesn't heavily depend on WSI reports, we expand our data by utilizing all 24,429 TCGA overview images (excluding test data). The WSI overview images are 32x downsampled versions of the original WSIs as mentioned in the main paper, providing a overview the entire slide at lower resolution. To be specific:
For WSIs with associated pathology reports, prompt Gemini-2.5-PPt with both the WSI overview image and the corresponding report. The prompt specifically guides the model in identifying and selecting suspicious regions, particularly those exhibiting pathological features explicitly mentioned in the report.
For WSIs without accompanying reports, we provide only the WSI overview image, prompting the Gemini-2.5-Pro identify regions showing abnormal tissue characteristics potential pathological features by leveraging Gemini-2.5-Pro' powerful visual recognition capabilities.
To increase the diversity of our data, we use a generation temperature 0.8 and create two separate region selection results for each WSI. This strategy produces slightly different region selections from the same WSI overview, effectively doubling our dataset size. At the same time, it introduces natural variations in how regions selected, which reflects the inherent subjectivity in how different
24

---

17.6%
32.8%
32 8%
8.4 %
8.4%
25%
50%
7995
100%
Region Selection (17.
Viewing Path Planning (32.8%)
VQi-ooieented (Planning) (8.4%)
Mutt--mage Understending Ressoning 85% VGA-orfented (Reesoning) (8.4%)
Figure A2: Illustration of the proportional distribution of each subset within the CPathAgent-Insstee Instruct dataset.
Breest
10224
Lang
Kilney
Bladder
61:52
Liver
5148
Thyusis
Statistics
Colos
1064
Cevvx
2015
Braie
2357
Total::
78658
Least: Lymphoid
2345
Most:
Breast
Breast %: 13.4%
Bolt Issues
2102
Types:
Thymus
1927
Stemach
Exophagus
Top Tissues
Skin
1054
10521
Kidney:
Duary
1043
Rectum
Lung: 8756
Head & ne ok: 6408
Bladder: 5152
Mosotheleam
Tesis
853
Sle duct
es3
Lymphoid Essue
Figure A3: Illustration of the distribution regions across tissue types.
pathologists might evaluate the same slide. The specific prompt templates we be found in Section E.5
2) Navigation Path Diversity Augmentation: Similarly, for the navigation path planning subset, recognize that pathologistsmay follow multiple valid trajectories when examining a large region to reach accurate diagnoses. To capture this inherent diversity, to 0.8 and prompt Gemini-2.55Pro randomly 1-3 times per WSI using the approach described in Section 3.2 This stochastie prompting process generates diverse navigation paths for identical regions. Correspondingly. the multi-scale multi-image reasoning also changes based on the different navigation paths. creating additional variations our dataset while improving the model's ability to generate diverse, valid trajectories.
the generation temperature
dataset. As result, we substantially expand
3) VQA-oriented Subset Construction and Quality Control:
We systematically construct the VQA subset as shown in Figure Al First, we use 21 multi-scale descriptions from sub-patches at different magnifications (1x, 2x, 4x), which are generated through WSI-report guided prompting (detailed in Section 3.2 We then input these descriptions along with their corresponding region images to Gemini-2.5-PPo to create the initial VQA pairs
However, recognize that models often exploit textual shortcuts when answering VQA questions rather than performing genuine image analysis. Inspired by PathMMU [48), we present text-only questions to both Gr [57] and DeepSeek (62. then eliminate any questions both models answer correctly without visual input. This ensures our dataset exclusively targets vision- dependent reasoning tasks that cannot be solved through textual cues alone.
For questions that pass validation, we proceed to generate navigation plans through a two-stage process. First, we provide Gemini-2.5-Pro with the validated question, multi-scale image descriptions, region descriptions, and full region images to generate a navigation path. Second, following the approach established in our region description reasoning generation, we combine the validated question with (1) cropped patches extracted along the planned navigation path and (2) corresponding region descriptions. This combined input prompts Gemini-2.5-Pro to generate comprehensive reasoning that systematically addresses the crestion-proggessiing logically from overall tissue architecture to specific diagnostic features, comparing and eliminating answer options through continuous reasoning to arrive at the correct conclusion.
25

---

Lung
3150
Breast
3110
Kidnay
2872
Brain
1484
Colon
1425
Word Frequency in Multi-scale Multi-image Reasoning Subsets: We analyze word frequency distributions in our General description-bas (left, Figure (55 and VQA-oriented (right, Figure Head nack
Over
Statistics
Thyrald
931
Total: 24429
Bladder
$10
Endom
859
Most: Lung
Liver
856
Least: Lymphoid
tieuua
B46
Cervix
560
Lung %: 12.9%
Rectum
525
Pancreas
459
Types: 26
Tests
Exophagua
Top Tissues
Adrenal gland
Thymus
Lung: 3150
Kidney; 2672
Mesoteeleumm
Breast: Brain: 1484
Colon: 1425
144
duci
109
Lymphold tieaue
100
6
600
1000
100
2000
200e
2000
Figure A4: Illustration of the distribution regions across tissue types.
eus Significant cellular
with thin see
pattern
Forus
looki
pattern
inf
nuclear
stroma
MALNERLIIN SU look
also
Clear
estroma
there
jest aapper
eos
Bosinophi
Lie
prom
Dremthent
nest
Teaturr
appean
cheer
Perreall
buc
within etsee
carcinoma
itrate
at stypia option
graggn a) General Description Reasoning
mmm sniriiatioo
So consistent
b) VQA oriented Reasoning
Figure A5: Visualization of a word cloud derived from general description-hase and VQA-oriented reasoning.
This question-guided navigation and reasoning framework yields dataset that enables CPathAgen to dynamically adapt to diverse diagnostic queries by focusing on task-relevant regions and features based on the specific question at hand.
Proportion of CPathAgeett-nsste Inntructt subsets Figure A2 illustrates the distribution of different subsets within the CPathAgent-Insitee Understanding and Reasoning" each constitute 32.8% of the total dataset. These equal proportions reasoning instance, creating one-to-
dataset. Among them, "Viewing Path Planning" and "Multi-Image
exist because each planning instance directly corresponds one relationship between these components. The "QA-oriented subset, though smaller, is sufficient for model adaptation shares structural similarities with the main components.
Statistics of Source Regions: As shown in Figure A3 CPathAgent dataset encompasses 26 distinct tissue types with total of 78,658 individual regions. Breast tissue represents the highest proportion at 13.4% (10,521 samples), while lymphoid tissue appears with 270 samples.
Statisties of Overview WSIs within region selection subset: As shown in Figure A4 the CPathA- gent dataset consists 26 distinct tissue types with total of 24,429 WSI overviews. Lung tissue represents the highest proportion at 12.9% (3,150 WSIs), followed by Breast with 3, 1,0 WSIs and Kidney with 2.672 WSIs.
A5 reasoning approaches. The visualization reveals distinctive reasoning patterns in pathology diagnosis. General description reasoning emphasizes procedural terms like "zoom." " "move," " and simuulatis-mumiiin pathologist's 's physical navigation through tissue samples as they methodi- cally adjust magnification, reposition focus areas, and explore specimens. Meanwhile, VQA-oriented reasoning prioritizes diagnostic decision terms like "look," "see," and "feature," reflecting hypothesis testing and evidence evaluation clinical reasoning. Both approaches incorporate critical patholog- ical terminology ("stroma," "nuclear," "infiltrate") essential for accurate diagnosis, demonstrating
26

---

ol (neop reccurate
feature
Lind
dla
key
dese
magniiication
ghns
on
be
RIRDY
opt
oPetall
S
deserve YOUE
acCu
Pa
arriratele
aanos
CACaT TOn
d
t
OLTE FITILLM Find
eelatoonshp
Tumo
norpholog
S
adanizare
delace
atatement
Like saly Polll
Statement
enteentt
ooddr varaaiii
cel
Fellular
integratt
Fate
integra
feature
re
feld
rate
Lesion
Lung
php
ORASH
Eumor Suppe
to Lake
Amor pho ogy
tytelogical
pees
C
FOOS
Loh
stronall
pest
Hikell
entin
a
OSS
accurat
pattern
relattoonsbip seneiiun wer character nor pho Lgg ca ty
character
al VQA-oriented GpathAgent-instruct subset
b) PathMMU-HR
Figure A6: Visualization a word cloud derived from questions in VQA-oriented subsets CPathAgent-insstuucs and PathMMU-HRA.
that these reasoning strategies center key diagnostic elements fundamental to clinical pathology assessment and disease classification.
Word Frequency of Questions in VQA-oriented Subset: We analyze the question terminology
in our VQA-oriented subset, as shown in the left part of Figure A6 The visualization reveals "diagnosis" as the central focus, surrounded by practical terms like "describe," "integrate," " and "feature." " Magnification terms ("40x,""10x" indicate how pathologists navigate between different viewing scales, while specific organs ("thyroid," "breast," "lung") appear frequently in questions. Action words like "evaluate" and "observe," along with uncertainty terms like "likely" and "consider," demonstrate the methodical reasoning process. This word pattern demonstrates that constructed diagnostic questions demand multiscale examination with systematic feature assessment.
A.2 Details of PathMMU-HR*
The construction of PathMMU-HR is similar the construction of the VQA-oriented subset in CPathAgen-Instrre Instruct, as shown in Figure Al However, they differ in that after filtering VQA pairs that can be solved through text-only shorteuts, experience for human validation. The evaluation criteria include: clinical relevance of questions, accuracy of provided answers, necessity of multi-scale integration for answering, and alignment with standard pathology practice. Questions failing any criterion are deemed invalid. Under pathologists' evaluation, identify 1,668 valid questions from the initially annotated 2,200 questions, with the annotation platform interface shown in Figure A7
engage three pathologists with over 10 years of
We also visualize the question part of PathMMU-HR? with word cloud in Figure A6 As shown, as "diagnose," " "integrate, "accurately." and "across" "across""highlipp" highlight that many queries require multi-scale analysis and complex reasoning. The presence of micro- level descriptors like "nuclear," " "cellular," and "morphology," alongside macro-level terms such as "tumor," "lesion," " and "pattern" underscores the necessity of integrating observations across different magnification levels (indicated by terms like "40x," "10x," and "2x"). This multi-scale integration is essential for correctly answering questions that demand the model to synthesize information from subcellular details to tissue architecture, evaluating both cytological features and architectural patterns simultaneously to arrive accurate answers.
the most prominent
27

---

Wat Meveriine carirone oxtemsivve bubale formatioo." 24
Fooryy affereiittd cardrome with high iibustcc ac
Moderataly affientassd canona
tnuatiee cxcinon dence satainn bynphorsaaaale
Redonine cancinoma
Connec Answer
Viliity
+ Inval
Resson for Inveldly??
with ongylt
relvard innurscichee
Update
Figure A7: Example PathMMU-HR2 annotation interface.
28

---

B Additional Experiments and Discussion
B.1 Comparison of Description Generation Performance Among Pathology-speciili LMMs
We randomly sample 100 region samples from TCGA and compare them with two advanced pathology-specifie trained LMMs, PathGen-LLaVA [26] and Quilt-LLaVA [47], to generate de- scriptions for these regions. To assess model performance, we utilize Gemini-2.5- and GPT-4.1 as evaluators, with outputs categorized as wins, losses, or ties when performance comparable. To ensure clinical relevance of the evaluation, we also engage professional pathologist with over 10 years of clinical experience provide expert assessment.
Results: CeathAgent significantly outperforms both PathGen-LLaVA and Quilt-LLaVA across all evaluation metrics. As shown in Figure A8 most notably, CPathAgent achieves perfect 100% win rate against Quilt-LL.aVA across all three evaluators. Against PathGen-I.La.. CgethAgent maintains consistently strong performance with over 96% win rates across all evaluators.
The evaluation consistency between human and LMM evaluators remarkably high. However, we observe subtle differences in evaluation patterns: compared to the general-purpose LMMs, the pathologist evaluator awards PathGen-LLar hypothesize this difference stems from the distinct evaluation priorities between expert pathologists and LMMs. perrpoo- LMMs, may favor CPathAgent's comprehensive reasoning, while the pathologist evaluator places equal weight both reasoning quality and diagnostic accuracy, resulting in more reliable assessment that recognizes PathGen-LLaa LA'ss clinical precision some cases.
slightly more ties and one additional win. We
Win
Tie
Lose
Evaluator: Pro.
CPathAgent vs PothGen
98%
CPsthAgent vs Qutt-LLaVA
100%
Evaluator: GPT-4.1-2025 --245-1-2025-04
OPathAgent PalVGen-LLaVA
OPathAgent vs Quit-LLaVA
100%
Evaluator: Pathologist
CPathAgent vs PothGen-LLaVA
96%
CPethAgent va Quil-LLaVA
100%
Figure A8: Comparison CPathAgent with different pathology-specific LMMs description//tiannoiis generation on 100 randomly sampled cases, evaluated by Gemini-2.5-Pro GPT-4.1, and expert pathologists.
B,2 Pathologist evaluation results
We invited pathology expert to evaluate both diagnostic description generation and VQA tasks across 40 regions, assessing whether the navigation path planning and multi-scale multi-view reasoning align with clinical logic. The model was run 8 times to calculate pass @k metrics.
Results:
Pathologist evaluation confirms that both navigation paths and diagnostic reasoning meet climical expectations. As shown in Figure A9 the model demonstrates strong performance with both tasks. For navigation path evaluation, CPathAgent's initial satisfaction rates are acceptable but modest, starting 70% for general description generation and 77.5% for VQA. However, the pass @k metric demonstrates promising improvement when increasing model runs, with both tasks ultimately exceeding 85% satisfaction. For reasoning evaluation, satisfaction rates start higher at 82.5% for
29

---

General Description Generation
Visual Question Answering
60%
65%
89%
10%
60%
50%
35%
85%
55%
60%
80%
80%
75%
135
142. Hd ond knd 1-5 1nd but Ind
14] bnd 1xd Ind
145 146 Ind
knd
IMC. MM IHS
end
1nd
1nd Auf ked
AHC HMC 1nd Ind lnd kof bid
Succcsss Rate
O Suacess Ratee
Success Rate
Successs Rale
NovigationPal Path
Descripiinn Rewsoning
Navigation Path
Question Reisoning
Figure A9: Success rates (pass @) across 8 generations, evaluated by pathologists across 40 regions. Results show the percentage of pathologist satisfaction with navigation paths and reasoning for General Description Generation (left) and VQA (right) as the number of generations increases.
Table A4; Overall results of models on the PathMMU-Pro. The hest-performing model in each subset is in-bold.
Test Overall
PubMed
SocialPath
EduContent
Tiny (430)
ALI.
Tiny
ALL.
Tiny
All
Tiny (147)
All
(4755)
(147)
(1865)
(136)
(1081)
(1809)
Expert performance
69.4
71.2
70.1
66.9
CPath-Omni
62.1
61.8
60.5
60.8
58.8
62.1
66.7
63.0
CPathAgent (Ours)
66.3
65.3
64.6
65.8
65.4
63.6
68.7
66.2
general description generation and 87.5% for VQA, and show continued improvement, reaching 90% for general description generation and 92.5% for VQA at higher k values.
Across both tasks, the consistent upward trend in pass@k k metrics suggests that the model can reliably generate clinically sound navigation strategies and reasoning when given multiple attempts. Additionally, VQA-based approaches consistently outperform general description tasks across both navigation planning and reasoning. We attribute this to VQA's focused attention target features that enables better identification of key elements, whereas general descriptions require additional judgment about content prioritization.
B.3 Additional Results on PathMMU-Pro
While CPathAgent demonstrates strong performance on PathMMU, achieving results that exceed the reported expert baselines, we acknowledge that the observed gap between our model and expert performance can be attributed to two primary factors inherent to the benchmark design. First, the expert annotations in PathMMU were conducted by pathologists with specialized expertise in one or two specific domains, whereas the dataset encompasses a broad spectrum of tissue types and pathological conditions across multiple subspecialties. Consequently, no individual expert possessed comprehensive knowledge across all domains represented in the evaluation set. Second, as discovered by the PathMMU, large multimodal models may exploit rext-based reasoning shorteuts, leveraging strong language priors and contextual cues from question formulations that are not readily accessible to human experts performing traditional diagnostic tasks.
To address the concern regarding text-based shorteuts and provide a more rigorous assessment of our model's genuine pathological reasoning capabilities, we conducted additional evaluation on PathMMU-Pro 633, a refined version of the benchmark specifically designed with more carefully constructed questions that minimize textual biases.
Results: On PathMMU-Pro, CPathAgent no longer surpasses expert performance but maintains substantial lead over the previous SOTA CPath-Omni As shown in Table expert performance now leads with an average accuracy of 69.4%, surpassing CPathAgent' 's results. This shift substan- tiates that the more rigorous question design in PathMMU-Pro effectively mitigates the text- based guessing capabilities that LMMs could previously exploit. Nevertheless, CPath demonstrates consistent advantages over CPath-Omni, achieving 66.3% versus 62.1% on the Test- Tiny subset and 65.3% versus 61. 61.8% on the Test-All subset, demonstrating that CPathAgent possesses stronger multimodal understanding capabilities for pathology.
30

---

Table A5: Results of ablation study agent components for WSI classification. Balanced accuracy (%)
reported.
Model
TCGA-BRCA
TOGA-NSCLC
TCGA-RCC
TCGA-ESCA
TCGA-BLCA
TCGA-THCA
Avg.
CPathAgent
62.7
63.2
w/o Gloabal Screcning w/o Navigation & Reasoning
84.7
88.7
94.6
94.1
60.8
58.9
80.3
80.0
83.6
88.7
94.1
52.2
51.3
75.0
Table Results of ablation study on applying Gemini-24 Pro as an agent for WSI tasks. Balanced accuracy (%) is reported
TCGA-BRCA
TCGANNSCLC TCGA-RCC TCGALESCA TCGA-BLCA
TOGA-THICA
Avg.
General Large Multimodal Models
Gemini P.5.Pro-Preview 003-25
69.2
Agent-based Appreach
Gemini-2.5-Pro-PPon Pro-Prexiew-03-22 CPalhAgent (Ours)
52.0
73.4
58.8
97.1
46.3
88.5
90.8
94.6
97.1
62.7
63.2
82.8
B.4 Ablation Study on Agent Components
To assess the contribution of each agent component, we conduct systematic ablation studies by isolating individual module impacts. We evaluate two critical components. For global screening, replace the learned region selection module with random selection of 70% of WSI regions. For navigation planning and multi-scate reasoning, we ablate the dynamic navigation and multi-view analysis by directly generating descriptions from entire regions at once, followed by WSI classification based on these single-view descriptions.
Results: Both agent components contribute meaningfully to performance, with navigation and multi-scale reasoning being particularly critical for diagnostic accuracy. As shown in Table [A5] removing strategic global screening leads 2.5% performance drop (from 82.8% to 80.3%) demonstrating that learned region selection effectively identifies diagnostically informative areas and mimics pathologists' selective attention during screening. The impact of removing navigation and multi-scale scale reasoning is substantially more pronounced, with performance degrading by 7.8% (from 82.8% to 75.0%). This significant drop reveals two key limitations of direct WSI region processing: (1) directly processing high-resolution regions (up to 16000 16000 pixels) necessitates aggressive downsampling to fit model input constraints, causing critical diagnostic details to become indiscernible; (2) the absence of pathologist-like reasoning that strategically examines multiple fields of view varying magnifications fundamentally limits the model's 's diagnostic capability. These results confirm that CPathhAgent' effectiveness stems from the integration of all components, with navigation planning and multi-scale reasoning being particularly essential for achieving high diagnostic accuracy on WSI tasks.
B.5 Ablation Study Applying Gemini-2.5-Pro as an Agent for WSI Tasks
We also apply CPathAgent's inference prompts to directly prompt Gemini-2.5-Pro, enabling it to perform agent-basee based reasoning and diagnosis similar to CPathAgent.
Results: Interestingly, while this approach yields modest improvements on TCGA-BLCA and TCGA-THCA datasets, the overall performance decreases by 6.7% (from 72.1% to 65.4%). as shown in the "Agent-basee Approach" part of Table|A6 This contrasts with the findings in Section 4.2 of the main paper, where applying agent-based approaches to advanced LMMs on large regions VQA demonstrated performance gains.
We attribute this discrepancy to the fundamental differences between task types. Since our WSI clas- sification task involves first generating descriptions for all regions and then performing classification based on these descriptions, unlike VQA tasks that provide clear directional guidance for navigation planning, general description tasks lack explicit objectives to guide the agent's 's exploration strategy. Consequently, when directly applying the agent approach to Gemini-2.5-PP with effective navigation path planning within large regions, as it lacks clear guidance which critical areas examine. Since subsequent reasoning heavily depends on the planned navigation path, performance suffers significantly, particularly datasets like TCGA-BRE BRCA and TCGA-NSCLC.
the model struggles
31

---

For example, we observe that on TCGA-NSCLC (Non-Small Cell Lung Cancer) tasks, where all WSIs belong Non-Small Cell Lung Cancer and the task to classify subtypes into lung adenocarcinoma and lung squamous cell carcinoma, after applying the agent- approach, Gemini- 2r sometimes even misclassifies samples as Small Cell Lung Cancer, demonstrating fundamental diagnostic emrors or hallucinations.
This finding validates our progressive training data generation framework, which leverages compre- hensive WSI reports to guide region-specific descriptions. These targeted descriptions enable us to prompt Gemini-2..5-Pro to capture multi-scale morphological features across different viewing perspectives. By understanding which locations contain specific pathological features, we can further prompt Gemini-2.5
generate nigher-qualit, navigation paths with enhanced spatial awareness, which in turn enables more precise and accurate scale. multi-view reasoning generation. This creates hierarchical sequence where each stage builds upon the quality of previous stages, resulting in cascading improvements throughout the entire data generation pipeline.
32

---

C Experimental Details
C.1 Details for MII--based WSI classification
C.I.1 Details for Training and Testing Dataset
To ensure a fair comparison, MIL-based methods are evaluated using identical training and testing datasets such as CPathAgent and other comparative LMMs. All datasets were derived from WSI samples corresponding to the HistGen [49] WSI reports. We select representative subsets suitable for whole slide image classification tasks, comprising the following datasets:
TCGA-RCC (Renal Cell Careinoma, train: 422, test: 238): three-class classification task including kidney chromophobe, kidney renal clear cell carcinoma, and kidney renal papillary cell carcinoma.
TCGA-NSCLC (Non-Small Cell Lung Cancer) (train: 642, test: 195): binary classification task distinguishing between lung adenocarcinoma and lung squamous cell carcinoma.
TCGA-ESCA (Esophageal Carcinoma, train: 83, test: 31): A binary classification task differentiating adenomas and adenocarcinomas from squamous cell neoplasms.
TCGA-BRCA (Breast Invasive Carcinoma, train: 637, test: 226): A binary classification task between invasive ductal carcinoma and invasive lobular carcinoma.
TCGA-THCA (Thyroid Carcinoma, train: 323, test: 119): A three-class classification task en- compassing papillary adenocarcinoma, papillary carcinoma columnar cell, and papillary carcinoma follicular variant.
TCGA-BICCA (Bladder Urothelial Carcinoma, train: 227, test: 79): A binary classification task distinguishing papillary transitional cell carcinoma from transitional cell carcinoma.
Additionally, we extract 10% of WSIs from the complete TCGA dataset that are not included in the HistGen dataset, as training the MII approach requires separate validation set. Models with the best performance on the validation set are selected for evaluation on the test set.
C.1.2 Details for MIL preprocessing
Our WSI preprocessing pipeline follows the approach established by CPath-Omm [10 to implement multi-scale MIL-based methodology. The preprocessing workflow consists of several key steps: First, employ CLAM (64 automatically identify and segment tissue regions by applying appropriate segmentation threshold. From these identified regions, we extract non-overlapping patches 2048 x 2048 pixels 40x magnification. To ensure tissue quality, patches only retained if they contain more than 10% valid tissue area. To capture multi-scale information, each 2048 x 2048 patch undergoes hierarchical subdivision: we maintain the original 2048 2048 patch while simultaneously extracting four 1024 x 1024 patches and sixteen 512 x 512 patches from the same region. Features extracted from all these multi-scale patches using CPath-CLIP [10] and subsequently averaged to generate a unified representation for each 2048-resolution region.
C.1.3 Details for WSI Task-Specific Fine- Tine-Tuning
All models are trained for 20 epochs using a fixed learning rate of 1 x 10-5 without any learning rate scheduling. We employ the Adam optimizer without weight decay, maintaining batch size of throughout training. To ensure reliable evaluation, conduct experiments across different random seeds and report the averaged results.
Model Architecture. The MIL framework commonly used for WSI classification includes three learnable components: (1) a fully-connected layer reduce the dimensionality features 256, (2) an attention network to aggregate and transform the instance features, and (3) final fully-connected layer for making predictions. We experiment with ABMIL [4] and DSMIL (5), both of which use the same fully-connected layers for reducing feature dimensionality and making predictions. For the attention network, ABMIL. uses a gated attention mechanism, while DSMIL introduces a dual-stream architecture.
33

---

C.2 Details for Patch Level Benchmark
PathMMU [48]: An expert-validated pathology benchmark comprising 33,428 multimodal multiple- choice questions and 24,067 images from diverse sources including PubMed, pathology atlases, expert social media posts, and educational content. Constructed using GPT-4V with rigorous validation by seven pathologists, it covers multiple organ systems and pathology subjects for expert-level evaluation of L.Mss pathology understanding and reasoning capabilities.
PathMMU-PPro [63] Building upon PathMMU, PathMMU-Pro introduces more strict assessment address a critical limitation in multimodal evaluation: the potential for models solve questions using text-only reasoning without truly leveraging visual information. PathMMU-Pro employs systematic two-step refinement process: (I) training specialized text-only gquestion-guessingg models that receive only textual inputs to identify and filter out questions answerable without examining pathology images, and (2) enhancing remaining questions by generating more confusing options that are textually similar but require careful visual examination to distinguish, thereby preventing models from guessing answers through superficial text patterns. This enhanced benchmark provides more strict assessment of models' intrinsic visual reasoning capabilities in pathology.
C.3 Hardware and Training Cost of CPathAgent
We employ NVIDIA H800-80G GPUs for CPathAgent training, with computational requirements of 9 hours for Stage (multimodal alignment), 25 hours for Stage 2 (pathology task training), and 39 hours for Stage (agent capability development).
C.4 Training Hyperparameters
For Stages 1 and 2 of CPathAgent training, we follow the hyperparameter configuration of CPath- Omni. For Stage training, employ 8 NVIDIA H800-80G GPUs and train the model for single epoch using a per-device batch size of 1 with gradient accumulation of 8 steps. We implement cosine learning rate scheduler with base learning rate of le-5 and warmup ratio of 0.03, while fine-tuning the vision tower with a reduced learning rate of 2e-6.
34

---

D Limitations
While CPathAgent successfully demonstrates the feasibility of mimicking pathologists' diagnostic reasoning pathways, several limitations merit future improvement:
1. Synthetic diagnostic pathways. Currently, the diagnostic viewing paths employed by CPathA- gent are synthetically generated rather than derived from real pathologist workflows. Although our synthetic paths are designed based established diagnostic principles and demonstrate effectiveness in our experiments, they may not fully capture the nuanced decision-making processes and adaptive strategies employed by experienced pathologists in clinical practice. Future work should incorpo- rate authentic diagnostic paths collected from pathologists through eye-tracking studies or explicit annotation of their reasoning sequences.
Limited task scope. CPathAgent currently focuses exclusively on diagnostic classification tasks. While this represents critical clinical application, computational pathology encompasses broader spectrum of tasks including progmostic prediction and biomarker detection. Extending CPathAgent to these domains is theoretically feasible and would significantly expand its clinical utility.
Training data scale. Our model was trained on 5,254 WSIs, which, while sufficient to demonstrate proof-of-concept, remains considerably smaller than datasets used by state-of-the-ard foundation models such as PRISM (587,19 .96 WSIs) and TITAN (335,645 WSIs). This disparity in scale may limit the model's generalization capabilities and its ability to capture rare morphological patterns. Scaling up the training data represents a clear path toward improved performance and robustness.
35

---

E Examples and Prompts
E.1 Examples of Region Descriptions Extracted from WSI Report Pairs
As shown in Figure A10 we extract relevant region descriptions from WSI reports by identifying specific diagnostic findings in targeted areas. This process systematically parses pathological reports to highlight key diagnostic information related to the given region, addressing the limitation in TCGA datasets where WSI reports exist but region-specific descriptions missing.
E.2 Examples of PathMMU-HR& Samples
Figure A11 shows the examples of PathMMU-HR& dataset. These VQA samples require multi-scake multi-view reasoning to arrive at accurate diagnoses. As illustrated in the examples, the correct identification of conditions such as fibromatosis (desmoid tumor) and well-demarcated clear cell tumors necessitates careful examination of architectural patterns and cellular features across different magnification levels. The diagnostic process requires the model to analyze tissue structure at lower magnifications while progressively zooming in evaluate the cellular morphology and stromal characteristies at higher magnifications. This multi-scale analytical approach reflects real-world pathological practice, where definitive diagnoses depend on the integration of aacro-architecturll features with microscopic cellular details.
E,3 Examples of Region Selection Process Based on WSI Overview
Figure A12 and Figurd A13 demonstrate the generated region selection results. As illustrated, the model segments the WSI overview into distinct regions and identifies which areas require high- magnification examination. For example, in Figure A1 the model correctly prioritizes core tumor areas for detailed analysis while also identifying critical tumor interface/periphery regions and margin assessment areas to evaluate potential residual tumor presence. addition, the model appropriately excludes the benign/backgroung breast tissue areas from maggnaaniiicttion requirements, these regions primarily consist of adipose tissue and fibrous stroma that appear benign and uninvolved at low magnification, thus optimizing the diagnostic workflow by focusing computational resources on clinically significant regions.
E.4
Examples of Navigation Paths and Multi-Scale Reasoning Generated by CPathAgent
E.4.1
General Navigation Path Planning and Reasoning
Figures A14 and A15 show how CPathAgent navigates and analyzes pathology images like expert pathologist. For example, as illustrated in Figure A14 CPathAgent begins with a comprehensive overview at low magnification (1.0x), scanning the entire tissue to identify distinct regions and architectural patterns. In this case, it recognizes adipose tissue on the left and dense cellular areas on the right, establishing the overall tissue landscape.
Next, CPathAgent focuses on the main area of concern, examining the primary tumor mass at moderate magnification (2.5x). Here it identifies irregular cell clusters and abnormal tissue architecture that warrant closer investigation. The agent then increases magnification (4.0x) to evaluate cellular details crucial for grading, such as nuclear pleomorphism, mitotic activity, and tubule formation patterns. CPathAgent systematically examines the tumor's 's relationship with surrounding tissues, checking the tumor-stroma interface (3.0x) for invasive patterns and examining infiltration into adjacent adipose tissue. also assesses the immune response by evaluating lymphocytic infiltrates near tumor edges (3.5x) and investigates suspicious ductal structures in other regions (3.0x).
To ensure diagnostic accuracy, CPathAgent performs validation checks by examining additional areas at various magnifications (2.5x, 4.0x), confirming that findings are consistent across different regions of the tissue. This systematic, multi-scale approach allows CPathAgent to integrate architectural observations from low magnification with detailed cellular features from high magnification, ulti- mately reaching a comprehensive diagnosis of invasive ductal carcinoma with appropriate histological deadinn-mirroring the thorough diagnostic process used by expert pathologists in clinical practice.
36

---

E.4.2 VQA-oriented Navigation Path Planning and Reasoning
Figure A16 demonstrates CPathAgent's -egided reasoning approach for VQA task. When presented with specific question about "morphological patterns describing the relationship between neoplastic cell populations and stromal elements at their interface," CPathAgent tailors its entire analysis strategy to address this inquiry.
CPathAgent begins with a systematic overview at low magnification (1.0x), scanning the tissue locate regions relevant the question. It identifies striking heterogeneity across the tissue and recognizes that the lower right quadrant contains the key neoplastic-stroman interface mentioned in the question. With the target area identified, CPathAgent zooms in progressively (2.5x, 4.0x) character- ize the neoplastic component, confirming the presence of nested clear cells with high-grade nuclei. At higher magnification (5.0x), performs detailed cytological assessment, noting the striking nuclear features that support high- grade classification. CPathAgent then specifically examines the critical interface zone multiple magnifications (3.5x, 4.0x). systematically documenting the relationship between neoplastic cells and surrounding stromal elements. It identifies key morphological features including the transitional pattern between clear cell neoplasm and eosinophilic tissue, along with dense lymphocytic infiltrate at the junction. Finally, CPathAgent moves to eosinophilic areas (3.0x) to contrast these regions with the ncoplastic component, ensuring a comprehensive understanding of the interface characteristics.
This question-driven navigation allows CPathAgent methodically evaluate multiple-choice options, ultimately selecting the answer that best describes the observed "nested clear cells focally transitioning to solid cosinophilic cells with high- grade nuclei and dense lymphocytes. This demonstrates how CPathAgent's reasoning process is specifically optimized to answer targeted pathological questions through systematic, multi-scale analysis.
37

---

Complete Report
incureesuinn then
inttrrniing inbile cantiooma chercleenned moderstely offeectinned Tiaoo Recerdioon soom Eslentiv kdbuar Inmunsbrttsheence anabys shows involies
regetvily HER, luree brewd raskestrry stonee looa
urpt the broand cocsenn kobuter cancioom situ alangiid usual duckal apiindd pepepiooia and solummer change.
Region Description
nassecooyy spedimen, infiirating
margin by
Completo WSl Report
miiooacpecc coaminstio porkel nech recamry specinen (A)recsah popplar any
sodd taae Fulemant medear fostanca The glonentooceooee margin, Exaninaion porton
lalley reaudden
conical
Region Description
papiliey desigreisc type giade
Figure A10: Examples of region description extraction from WSI reports.
38

---

Evaluate architectural and celluar features across magnifications. What is the most likely diagnosis?
Fibromatosis (Desmoid Tumor)
Pscuddoangiomttous Stromal Hyperpiasia (PASH)
Benign Fibrosis/ / Scar Tiesue
Nodular Fascitts
E
Well-iiffereniittedd Liposarcoms
Analysis magnilications of this primarlly reveals which snoor-stroma relationship and stromal character?
Indirative: permeating highly colider, desmoplastic
Well-demarate caled clear cell abutting broad, paucicellular, hyalinized fibrous stroma.
Sarcomataid change with atypical spindle colls replacing clear cel nests.
Extersive cyslic change and necrosés, with inflamedd
Figure Al1: Examples of VQAs from the PathMMU-HR dataset.
39

---

Core Tumor Areas
Tumor Interface/Parfphary 12 Litt:
Lat:
Requirss High Megniccatio
Rarquina Mageeloaton and kdenaljeng broprovvessee (J4 kataara
lighgggd
Mudeer wubble allow losion
Margin Assessment
Areas Areas
BenignlBackground Breast Tissue
Pres maaningartt prodiit
Paimurly compoyyd adlaoou baausrand fireso vboree apersdeedfllke glbsnig
margin
Severity Levels:
High Magnificatio:
fynnet Phenety
Arean)
Rwyyled
Locil 4 High Prloityy (Tumor PeiehhoeePPesheyy
Looi 3 Mediun Pririty Margi Asoessment Arnos)
(hweggillicggrnn Tiveet Thoxsw))
Figure A12: An example of region selection based WSI overview.
40

---

7
8
10
11
18
12
13
14
15
16
17
Core Tumor Regions
Tumor Periphory and Infiltrative Zones
ID Usi:
Requircs High Magi Roaddo
Towgu mes High Mtggiitaaioon
Denee, heperoel udte smon with dicorgaime arontectue, atrong yeugpentiv of high prade carcinoma Phortee Contirs malignancy, dedermine tmmo type, aasass histo ogicall grade
Celider Wiratoo
Prority Ascess Investor exeer, search for LVFPM evilaat yrrph rode in paai 10 Severty Lewel44
Lymph Node Assessment
Perivesical FatiStroma Subtle Changes
Ranqures High Magr cistion
High MagriPcation
poor frauue ledered tumor oele. intemmettio Sevvriy level
kiaertoor vtgle
Pronity Scremn for insiolall calaa andiiledare Sanorelledd
Background Stromal Tissue List
Scant Tissue Background
Usst
High Magriffation
High Mogeiocatio
Predomsnantyy add paus sus and locse connect tuasse rite ice suapicion Sewerity
Jouza with minimaal no discemble ble tasus, Blayy alice bacoground dlogrooti
anoess surgior gine
Level
Sevarity
Magnification:
Lovel 5 Hignest Pioorty (Coee Tumor Regions)
Rosuired
Leal 4- Pridty (Tuner Pwrshwry intiinetee Thnws)
Not Requine
flymphh Assesoment)
Level Mediun Pitrrty (Perkekkcl FariSmmaa Subtle Changes)
Lovdl Prioity (Backgrond Auloose end Stomnl Toaus)
(Bcart
Figure A13: An example of region selection based on a WSI overview.
41

---

Okay, the acanination, At law power (1.0x). 08 1, 1j. overall arch tecture distinct demarcation between adiposs tissue on laft and densely cellular, Infiltrative process occupying the right side, within a fibrcus stroma. Ihere's also prominent, somewhat dilated duct like-llke structure towards lower central area. The celluiar area appears quite extensive. My Initial Impression Invasive carcinoma.
Let's zoom into the main tumor in the upper right quadrant (255). co comossed of irregular nests and cords of epithelial cells infiltrating a desmoplastic stroma. There's some attempt at gland formation, but it seems limited.. The cellularity high.
Here, appreciate the tumor architecture better, It's
Now, I need a closer look at the cytology for grading (4.t) [0.75, 8.15, 8.85, 0.35). The nuclei show moderate pleomorphism - variation in size and shape apperent, maybe fold variation. Chromatin appears
nuclelli visible but generally prominent large, scanning for mitotic figures... ..I see tow, but they don't seam partooulrty numerous in this field. Tubule formation is definitely present but constitutes area, probably somewhere
then 75%
10-75% This paints towords intermediate grade, likely grade
Let's examine the interface between the tumor and the stroma more centrally (8.00), io-4,, 02, 8.75, 0.75, o.j.. The infiltrative pattern clear, with nosts and pushing into surrounding fibrous tissue, this stromal interface.. don't see definite LVI this specific field, but Iil keep eye out.
shows desmoplastic reaction. looking carefully lymphovascular invasion (LVD) within
Moving down towards the adipose tissue (3.0x), 18.5, 8.5, 0.i can confirm invasion into fat. Tumor nests are clearly infiltrating between edipocytes, accomparied the resctive stroma
New lat's assess that lymphocytic infiltrate noted near the tumor edge (3.5x)), [0.55, 0.65, 6,85, n.ss There's moderately dense lymphocytic infiltrate, predominantly
clustering around turnor dossn't appear significanlly fumoral. This represents host response.
Let's invectigate that large ductal stracture in the lower contral area (3.0s). [84, 1.0). The spithollal lining looks relatively bland, perhaps some usual ductal hyperplasia apocrine metaplasa,, doeen't show the cytological atypia the invasive component features high grade DCIS. It seems distinct from the inwasive carcinoma, passialy benign entrapped duct showing low grade DCIS, though the latter is less Ikely given the bland appearance.
Now, I'll scan the stromal and adipose tissue the left side, from the main This area consists mainty of fibrous stroms and mature adipose tissue with some small vessels. don't see any definite isolated tumor nests or clear here.
(2.5x), 1.1.04.00
check for satellite foci LVI
Finall, do a high-power power check in different area of the tumor, the lower right (6.D)) 187, 07, 0.95, 6.95). to ensure cor corsistency. Yes, the teatures are simiar here: moderate nuclear pleomorphism, visible nuclooli, limited tubule formation, occasional mitotic figures. The overall Impression remains consistent with an invasive ductal carcinoma, Nottingham grade 2.
The differential diagnosis would include other types invasive carcinoma, but morphology strongly favors type, NOS (No Special Typa). There's no evidence lobular features (ack of single-filngg presence of cohesion), metaplastic elements, other specific subtypes. The presence of some tubule formation moderate pleomorphism fits grade
"Pathological Report:**
carcinoma characterized by infiltrating nests epithelial stroma, extending into adjacent adipose tissue. The tumor cells exhibit moderate nuclear pleomorphism, visible nucleoli, and limited tubule formation (estimated 10-75%). Mitotic figures present
examined
desmoplastic
numerous, moderate stromel lymphocytic infiltrate lymphovascular invesion is identified in the sections examined. large ductel structure sppears benign or possibly shows low-grade DCIS, distinct from the invasive component.
No definite
Overall findings are consistent with invesive ductal carcinoma, No Special Type (NO5), Nottingham histological grade (Tubules: 2.
Nuclei: 2, Mitoses: Total Score: 5).
Figure A14: An example of CPathAgent's navigation path planning and multi-scale, multi-view reasoning process.
42

---

Starting with an overal view of the entire slide at 1.0k magrification 1..001.11), I can immediatlly see two distinct regions. On the left, there's a more organized, somewhat lobulated tissue that appears be renal parenchyma, characterized by the presence of glomerull and tubules. On the rigns much larger, densely cellular, and tumorr adjacent
that looks like neoplasan The interface neoplactic Kidney
these regions somewhat
Imeegsarr My initial Impression
Now, I'm moving to the upper left quadrant at 2.5k maginficationn (Q.35, Ba5, 845, 8.55ji to examine the non neoplastic renal parenchyma more dosely. II can identfyy several glomenull, some of which appear globally sclerotic, meaning they are solidified and acellular. The tubules are variably with thickened basement membranes while others dilated, The Interstittum clearty expanded fibrodie and contains significant inflammatory infiltrate, predoninantly lymophocytic. There's prominent, thick-walled artery the upper features of arterioscterosss These fnndngs are indicative of dhronic kidney disease.
stredt
this field, showing
Let's increase the magnification to 4.6. within this non neoplastic kidney area II54 8.1.0.0...00 to get a better look the cellular details. The walls. segmental and others global. The arterioles exhibit byaline arteriolesierross, with signs atrophy, simplifled spithellum and thickened basement
glomeruli show varying degrees of scierosis
hamagenous eosinophilic thicteniig membranes, alongside some dilated tubules. The interstitial fibrosis extensive, and the chronic inflammatory infiltrate composed mainly of hymphoyyte and some plasma cells. These festures confirm significant chronic nephropathy.
Now, shirting attention the interface between the non-neoplactic kidnay and the neoplamn
nagnification [1B-5, a.50. The
border appears somewhat irregslar, suggesting an infiltrative growth pattern rather than well-defined pushing margin or thick capsule. There's desmoplastic stromal reaction at the interface. with increaced collagen deposition and some inflammatory cells. car see some entrapped, atrophic tubules within periphery.
tumor, right, composed densely packed with
morphology
the parenchyma
sumar-kidney infitrating between the residual renal tubules, These tumor cells are predominantly spindle-saaped, with elongated, hyperchromatic nuclei and eosinophilic cytoplasm. They are arranged in fascicles. The adjacent renal tubules show reactive changes, including strophy and some epithelial evident here
Zooming
0.450. can appreciate the interaction.
clearly
simplification The infiltrative nature
Next, moving
main body
neoplasm
magnitication 09.6 0.5, 6.5)) to
architecture.
highly celluler and predominantly composed spindle cells arranged intersecting fascicles, creating a storiform herringbone pattern some areae. The stroma relatively scant in some parts, more collagenoux in others, Vaccolarity is present, with small caliber veccals courding through the tumor. The overal impressionss is of sarcomataid spindle cell neoplaon.
the neoplasm 39.75 066 spincle calls achibit significant nuclear pleamorphium, with variation visille, though not strikingly prominent, nucleoli The cytoplasm is eosinophilic and fibnilar. Cell borders are indistinct. I am actively looking for mitatic figures, and cwn idantily fwn, suggenting proliferative activity Thrre fraturm comestent with high-grade malignancy
Let's go higher magnification
representative
evaluate cytological fentures. The
shape. The nuclei are byparchoomatic. same are vasicular with
II8.7 with what I've seen before- dansely packed spindle fascicles, However in thi feld, can see some entrupped, atrophic renal tubules within the tumor, forther supporting its infiltratie maturn. The strame appears somewhet more collugenous in this particuler view. The nigh-grade cytological fentures
I1l examine
within neoplasm
0.860) assess any heterogeneity. morpholegy
largely concistent
location, more central/left. at 3.5x mognitication [[045, confirm the consistency of the chronic changes. indeed. the findings are similar what observed earlier: widesprend glomeruloseeross, tubular strophy. interstitial fibrasis, and chronic inflammatory infitrate Some glomeruli are globally aclerosed, others show segmental sclerosis The tubules variably atrophic and dilated The interstitium fibratic and inflemed. There chrenic changen uppeir diffuse throughout the neoplastic kidnay tissue presented
Finally, revisiting non non-neoplastic kidney parenchyma
to
Rased on these nhserwiien nan-nsoplactic kidnay shown dggifiicatt chronic nephropathy. inclding glamandlecderask tubular strophy. intensitial fibrosis and chronic inflummation, along with vascular changus. The nuoplasm is high-gnuda spindlu cull sarcama, churacterized by marked nuclear pleomorphism, hyperchremsii, mitatic activity, and an infiltraiive growth pattem into the adjacent renal parenchyma. The tumor arranged fascielus. the lidrey location, sarcomatsid renal sarcoman could be the differential. chereeeenne to "sarcomatnid features' wign with my findings high-grade spindle malignancy.
would primary considenation, although spindle
"PPathological Report
The kidney tissue examined shows distinct componints. The non nooplastic renal paronehyma exhibits features severe chronic nephropathy, characterized by widcsproad gloneralescerosis (both global and seqmental, tubular atrophy, interstitaal fibresis, and a significant chronic Inflummatory infiitrate. Vascular changes, induding arteriesderesis and byaline arteriolosclerosk, are alse present. Adjacott to this, there high- grade malignant neopham composed predominuntl spindle
Those mesplasiit cells amanged in intersecting fascicles mitedic activity. The tumor demonstrates an Infittaaive growth patters, Invading adjacent
display
marked mudiea pleomorphism. hyperchronasiaa, renal parenchyma and ontrapping atrophic tubules. The overall features are consistent with a high grade spindle cell sarcoma, such as sarcematold renal cell carcinoma, artsing a baciground of severe chronic kidney disease.
Figure A15: An example of CPathAgent's navigation path planning and multi-scale, multi-view reasoning process.
43

---

Across magnifiaaioos what murphological patiern describes the reletionship butwwwn neoplestic populatiios and strumal dements, particularty Interface?
Nested calss focally tranitionn solid aosinophili high-graid nuciei, deras Jymphacytes.
8. Uniforityy nested dear tolka delicane vaccukot.re. rc. low grode nuciet throughour.
Distinz ahroslar scsinophilic supilary
spars
U'ffure tymohooytes separat ng strophic tubules whth clear sytoplaem
6. Widesproed eosinophilic adher cells, desmoplactic strma minimel tymohocytts.
Bay--Sneeggnnngg my
magsiffaation
striking heterageneity. majority tissee, particularly upper portions. intensely cssinophilic appeors somewhat disorganized,
region
pallor might suggest mecrosis degeneration. sontiess, lower right quadrant prasets us more densely colluler, basophilic area. This intrad idaayy saggria neaplestic prooms the lower right, contratting with the costnophilic tisue storwtere. hhree's dlear Interface between these two zones.
conferms denuely colluter nature. The colls oppear to be arranged in neets or perhaps an aherolar pattern. Many cells have dear cytoptasm, and the nuciel are quite prominent The stroma supporting these nests seems delicate and voscular. T highly suggestive of dear cell neoplasm. The interfacee with the more cssinophlic tissue to the upper left of this field is viable, and appears somewhat imeguler.
New: I'm zoomiin 2.56 focusing
right quadrant (1S5, 85, 14B)
Let's incr case magnifiiation to A.s. looking deeper into this ncoplastic area 1187. B., oas, easg Yes the features are dearer now. The cells are indeed arranged neets and aiveoler structurer. The sytoplaem abundant and optically clear. The muclal are large, show significant vartation in size and shope (ateomorphisnn, and many have prominent nudieali The conseentitt high grade dear cell neoplasm. The strema delicate, with thin walled blood vessels.
Pushing 5 eogellication withinn this some neoplactic region 0875 67% 688, 0858 slowe for detailed cynological assessment The aucler fextures strikingg marked pleomorphiam, imregular mudear contours, chrematin, and very prominent, often large. cosinophilic nucleoli. These unequivocally wigh-grade ruclesr teatures. The cytoplam also evident.
Now: I'm moving magnificttion examine the interface between this dlaar call neopleam and the adjccet cosinophiic tasus [J8.5, R.. Ed8i): On (lower right view um the diaracteretic nested cells high nudel On othw eide left', the degenerated necroti,
cosinophilic, cells appear intermingling
prominent fibrotic, There's particularly
this interface showing degenerative changes. vay appaeet signtficatt infarmeatory anflammatory prrdombnently small,
The
cosinophilic particularly junction.
colls consistent Jymphocytes, they are quite
sceinophilic away showing loss distnct borders, soudged appearance. vacuolation, condetent widespread necrosis
neoplastic
region
extencive costnophilla, degenenation. stroma fibrotic. confino that the cosinophilic component distint from viable clear neoplamm.
high- grade nudel, neoplesm interfaces disinct, assinophilic mecrotic/degeeerativ Basue interface thane's notable, denss lymphocytic infiltrate. The torn "focally tranaition" seems appropriate for the Interface, as It's not uniform boundary but rather an area of interaction and change-
Integr sting these obser
neoplastic component characteriee by nested
Now: uvatuate options:
Nested clear cells focally transition to colld anstnophilic calls, high- grade mucell, dense lymphorytes This aliges very well cow nedtdd clear cells
high guide nudls, and cense tymphocytes the interisce, whtch can be describe is a focal trasstion.
ii Unfoomlly nester clerr colls, deizete verridature, invegrade mudle thrughout. Thi incoment because the nuciel are high grade, not low-grade, and the trsue a not untormly nested deor cels due the extensive eos nophiiic necratic component
Distict dear aveolar and cosinopniii papillar populations sparse strona, While there dear alveslar colls. the cosinophillc component is uniicmlly
necratic nncroti/diegeerrani not dearly papilary. and the
it's demae and fibottc ccsinuphicc arse and delicats doar
Cifl.se tymphooctess separating atrophi tobules with cytopkamm, This incorrect tymphoccyes separating atesphic
interface,
The decroplacie
reop atrophic ymphorytec
dequoedd wooinophllle there ard demee lymphorrytos tos, minimal.
cosinophilic component necrotic. not acharr,
Option describes
merphological pattern
neoplactic poputations elements.
euidenza stmagly suppors prwcense recrodio/oogennettive area, accompaniod by donse tymphocyyk infiiaae interfaee Theroforc. is (A).
Nigh-grade focally asxinophille
Figure A16: An example of CPathAgent's VQA-orienter navigation path planning and multi-scale, multi-view reasoning process.
44

---

E,5 Prompts
This section presents the prompts used in our dataset construction and experiments.
We design series of prompts to guide Gemini-2.5-Pro in constructing our pathologist-like planning and reasoning dataset generation pipeline. Figure A17 presents the prompt used to extract region-relevant descriptions from WSI reports, while Figure A18 shows the prompt for region selection using WSI overview images and paired reports. For multi-scale image description generation, Figure displays the prompt that generates detailed descriptions for image patches different magnifications. Figure contains the prompt for creating navigation plans using coordinate-annotate region overviews and multi- scale descriptions. Finally, Figure A2 shows the prompt that generates pathologist-lik diagnostic reasoning based on extracted patches from navigation paths and paired region descriptions.
Gemini-2.5-Pro Prompts for Dataset Construction:
For VQA dataset construction, that instructs the model to generate VQA samples, Figure A23 presents the prompt for predicting question answers from text-only inputs (applied to both Gemini-2.5-PP and DeepSeck-V3), Figure diddisss the prompt that guides the model to create dneestionguuddes navigation plans, and Figure A25 contains the prompt for generating reasoning- baased answers.
several specialized prompts: Figure A22 shows the prompt
The prompts for CPathAgent follow similar design, while removing the additional information utilized by Gemini-2.5-Prr (such as pathology reports), focusing on vision-only pathological analysis that replicates clinical diagnostic processes. A27 and A28 present the core prompts used for CPathAgent's 's training and inference. Figures A26
CPathAgent Training and Inference Prompts:
These prompts guide CPathAgent to perform navigation planning and reasoning using only visual input. The VQA-specific prompts shown in Figures A29 and A30 display the prompts that used for CPathAgent's question-guided navigation and reasoning.
45

---

Surgical Pathology Ruport
Region Overview Image
WSI Report
Prompt: This is the report for (Tissue Type) whole slide image. What you're seeing now is one specific region from this whole slide image. Please determine if there are any descriptions in the report that match this region.
First, carefully examine visual
present in this specific region of whole slide image.
Next, go through each description the report
3. For each description,
whether could match the visual features you observe this region.
4. Consider both positive findings (features that are present) and negative findings (features that are explicitly absent).
5. a description seems to match, evaluate how complete the match is does it fully match or only partially?
6. For partial matches, consider whether the description might modification.
7. Identify the most detailed and gra granular matching descriptions from the report.
8. Evaluate whether this specific region representative the overall whole slide image diagnosis and, so, modify or represent
final diagnosis descriptions match prepare output 'None'.
observed
region.
Please final matched description (start with * matthhd description:"")
Gemini
Report: (WSI Report)
Figure A17: Prompt for Gemini-2.5-P to extract description related to a given region from the corresponding WSI paired pathology report.
46

---

Surgical Pathology Report
WSI Overview Image
WSI Report
RE
Prompt: thumbnail pathological slide image (Tissue Type).
expert pathologist, need you to analyze the given pathology image which has been divided into multiple patches pre-drawn IDs.
will also provide with the physician's pathology report for this whole slide image. While may reference please conduct your analysis without mentioning have the report your reasoning and response. Original report: (WSI Repot)
Please:
1. Design systematic reading path by grouping the patch based on their visual patterns (should include all patches), patch
only belong group
Order groups according standard pathological diagnostic workflow
Note: Be aware that whole slide images contain:
Multiple discrete tissue fragments
Serial sections from the same tissue block
Different tissue types/organs on the same slide
These variations should inform your grouping strategy and reading path design.
Output analysis the following JSON format: "json
"groups":
"name": "[group name]", ## brief description of this group
region]", "id_list": [patch IDs in recommended viewing order],
"description": "(description
"require_high_nee" gigg_mmanification" [true/false), true the require detailed high-power examination confirm
diagnosis
"severity reasoning" "(why this severity level was assigned]".
baltapoostic_rrorrity [1-5), // Lower number indicates higher examination priority (1 highest)
"observation "ittervatioo_poittstt.
"key feature 1 need to examine". "key feature need to examine",
Gemini
Figure A18: Prompt for Gemini-2.5-Pro to generate region selection results given WSI overview and its paired pathology report.
47

---

Num:1
Num:4
poorly differentiated invasive
ductal carcinoma
Multi-Scale Image Patches
Characterized by areas of n Region Description ane mid muciear
pleomorphism with bizarre nuclei. The tumor shows...
4X
4X
Prompt:
will provide you images
single large pathology image three different
1 full-size image (the complete pathology image)
half-size images (dividing each edge into halves, creating grid)
16 quarter-size images (dividing each edge into quarters, creating 4x4 grid)
Image info: (IMG_INFO INFO Iscale, position, bbox))
also provide reference description the entire region (which may not be completely accurate or detailed).
Please provide descriptions for each image all three scales professional pathologist would, covering:
For the full-size image (1/1): Provide comprehensive overview of the entire pathology sample, noting major features, patterns, and potential diagnoses.
2. For the half-size images (1/2): Describe key regions with intermediate detail, focusing an distinct areas and their pathological significance.
the quarter-size images (1/4]: Provide detailed microscopic descriptions focusing cellular features, architectural patterns, and diagnostic indicators visible in each specific portion.
Since all images come from the same large region but at different scales, ensure consistency between your descriptions while noting the unique features or variations in each image. Please do not reference other images or their IDs in your destriptions.
Output your description each image ""sson
like
images": [
paach_id":: "*
"scale":
"position":
"description": Provide detailed description image, not mention magnification"
...and for all 21 images
Gemini
Reference description: (Region Description)
Figure A19: Prompt for Gemini-2.5-P S-Pro to generate descriptions for image at 1X scale, 4 images at 2X scale, and 16 images at 4X scale, given these images and their paired region descriptions.
48

---

Region Overview Image with Coordinates
charac erized S areas of necrosis, an Image, nuclear
poodl connntntttee invasive Multi-Scale
pl Descriptions nucler. The turnor ShoWS....
Prompt: Please carefully examine this pathology image simulate pathologist's viewing path. showing potential pathological changes that warrant careful observation, please analyze using these guidelines:
You can zoom and out to observe the image. In the default state (1.0x), you see the complete image at 1/5 of its original resolution.
For any zoom level Field of view: 1/2" the total Image area (1/Z of each side length)
Downscale ratio: 5/Z
Examples:
1/4 (downscaled 2.5x)
Shows 1/6.25 (downscaled 2x)
3.0x: Shows
(downscaled 1.67x)
3.5x: Shows 1/12.25 area (downscaled 1.43x)
4.5xx Shows 1/20.25 area (downscaled 1.11x) 5.00: Shows 1/25 area (downscaled 1x]
You zoom level up to 5.0k, including decimal values (such as 2. or 4.5x), Image coordinates represented in relative values ranging from D to 1.
"eeeerence Image Captions**
provide reference images captions at different magnifications:
11.0x (full image)
(4 sections)
(16 sections)
-These are reference only- your viewing should:
Not exactly match these reference images' coordinate
Consider other magnification levels
Create regions that may span across provided regions
Your viewing path steps should:
Begin with low-power scanning to identify key histopathological patterns and tissue architecture
Check
heppy//bcckrrund/unimp.a diagnostic hotspots unimorant
diagnostic
showing pathological changes, abnormal cell populations, or architectural distortion)
Pay special attention transition zones between normal and abnormal tissue
Selectively analyze areas with representative features using diagnostic hierarchy (primary lesions before secondary changes)
Apply systematic differential diagnostic reasoning when examining cellular and tissue features
Ensure minimal overlap between sampled patches reduce redundancy, except when:
Examining the same region different magnifications clinically necessary
Comparing similar features across multiple sites relevant diagnosis
Demonstrate logical continuity between each step that reflects pathological reasoning
Movements between steps should follow systematic pattern
Prioritize fields that demonstrate key diagnostic criteria rather than simply sliding the window Consider the blue coordinate grid for position reference Only Zoom /ooooo out when necessary
Please provide the viewing path as JSON list: "jsson
"step": : [step number).
"magnification": [zoom level from 2.0x-5.0x),
aingion_coordiaane [x1, yl, x2, y2], # Coordinates represent [top_left_x, top_left_y, nattom_right_x, bottom_rightt pighttt_rggt_
"reasoning": jreasoning for moving to this position),
"need te_s [description of what should observed at this position),
Gemini
Reference image captions: (Mulli-Scale Image Descriptions!"
Figure A20: Prompt for Gemini-2.5-PP to generate navigation path planning given the region overview with coordinates and 21 multi-scale image descriptions generated in the previous step.
49

---

Overstev
Form
poorly differentiated invasive ductal cheracterized Region and are
Description
Cropped Images from Navigation Path
bizarre nuclei. The turnor shows....
Prompt: This represents a pothologist's logical viewing path for examining a large pathological region, alonng with corresponding field veld-of-veew images. each this systematic examination, please analyze associated images based on the specified observations needed. The viewing parameters are defined as:
"magnification"
6x, where shaws entire image
"region coordinate": : Coordinates of the viewing position
seeeee" Specific features structures be for each area, pay special attention to the structures and features according to its [o to_see" description.
!will also provide reference description for this region (which may not be entirely accurate and detailed), You should only this description for reference pretend have prior knowledge of during your analysis and do not include index. Reference caption: (Reglon Description)
including reasoning, thinking as you perspeclive and follow this framework for analyzing pathological images:
describe image
person
1. Initial Positioning:
Establish nermal histological standards reference
Describe key features overall field view
Systematic Observation Process:
"four layer progressive' obsarvation mode:
Basic layer: tissue structure and basic construction
Cellular layer:
morphological Tissue layer: stromal reaction, inflammatorr manilestation, vascular changes
(size, shape, arrangement)
Intogration laye:
pathological changes (such
fibrosis, etc.)
3. Field View Switching Strategy:
A. Systematic scanning:
Start with low magnification (overall view)
Gradually increase magnification High power observation in key areas
B. Dynamic movement description: Clearly
observation position (e.g.. "moving upper comer")
Describe mevement path (e.p.. 'scanning from left to right")
Record
changes "noticed
area,
in")
Multi scale Observation (1.0x
magnification magnification):
Available magnifications 1.0x 6.0x
Norralizedd coordinates (0-1) for each view (X2, v1, x2, y2)
Your siewing window 1/6 of original resolution where:
1.0x shows
image 2.0% shows 1/2 image length
3.0x shows 1/3 image length
Reasoning Process Requirements:
A. Use "thought flow" expression:
Think observing
Record process real-time (give coordinate)
Allow returning for re aa-aarrevaaion
Record thinking turning points
Diagnestic reasoning strategy:
Establish initial impression
Propose diagnestic hypotheses
Seek supporting evidence
Consider contrary evidence
Modify diagnostic hypotheses
Deepen analysis and argumentation
Think about whether the diagnosis of subciasses can be furthor determined
6. Differential Diagnosis Method: systematic approach
diagnostic possibilities (include
Basic losions (e.g., inflammation, degeneration) Neoplastic lesions (benign vs malignant) changes)
Special lesions
Possibility concitions
7. Evidence Usage Method: Positive supporting evidence
Negative exclusion evidence, provide reasons non-confarmity
Diagnostic Requirerments:
A. Clear statement:
pathological findings
Secondary supporting
Special/atypical presentations
Special Considerations:
Refiect real time abservation and thinking process
Allow for thought repetition and correction
Maintain professions diagnostic prudence
Gemini
Please more natural approach when describing pathology observations as pathology speciarst who's naturally examining thinking through what they see, rather than teaching students. description should flow narrative **in one paragraph** rather than a bulleted list of observations. region
Finally, please output an
summary
**Pthhological Report:"* paragraph.
Prompt for Gemini-2.5-Pe to generate multi-scale multi-image reasoning description and diagnosis based on the planned navigation path, given the cropped images along the navigation path and the overall region description extracted from the previous step.
Figure A21:
50

---

poodl ate invasive Multi-Scale
Region Overview Image
charac erized areas of Image nuclear
necrosis, dnd
Descriptions
nuciel. The turnor shows....
Prompt: Please carefully examine this large region pathology image. will provide descriptions from different magnification viewpoints.
challenging, expert levvl multiple-choice question reasoning skills across multiple magnifications and requires constant zooming in and across differrnn fields of view answer correctly. The question should be suitable for board examinations or specialist assessments.
advanced
For this challenge, the magnification notation refers to the level of zoom relative to the base image: Original image
2x: magnified
image
3x: times magnified from base image
4xc times magnified from base image
Sx: times magnified from bese image
Follow these guidelines:
Construct question stem that:
Presents neutral framing
revealing diagnostic Requires the test-taker to analyse the image independently
Uses language that forces visual examination rather than recall facts
Demands integration findings
multiple magnification levels and viewpoints
Requires step-by-step visual reasoning to solve correctly.
multiple answer
options]
The correct answer accurately describes what can be observed acress multiple magnifications
Each distractor option must be visually plausible based on partial examination of the image
options be inditinguishable in correctness WITHOUT seeing the image
Place the correct answer in any position (A-E)
Ensure consistent terminology and level detail
options
Include plausible pisinterpretations that would occur if only examining single magnification level
Document your analytical process canalysic<//analyst tags, including:
Critical visual features across multiple magnifications that determine the correct answer
Potential visual misinterpretations that make distractors compelling
-Why
necessary
-How isolated examination any single magnification would lead to incorrect conclusions
image descriptions different magnification levels: Coordinates given as four values (x1.y1x22y2) representing the top-left and bottom-right points, ranging from 0-1
provide
These include:
1.0x (full image overview)
2.0x (4 quadrant sections)
(36 detailed sections)
Your question should:
Not limited these exact reference coordinates
Integrate findings
multiple magnification levels
Potentially require examination of reglons that span across my provided sections
the
repeatedly
problem
5. Coordinate System:
Use consistent coordinate system for regions yxx,21x2,y2 with values from 0-1] Include reference points multiple magnifications (minimum of different zoom levels)
Ensure critical diagnostic features span across multiple reference regions
Present final question ISDN format:
json
"question_sterf" stem": "Concise questions", ## within 20 words
"aptions":
('id": "text":
poorect_ansser"" "ID of correct option", "reasoning pathway": "Step-by-step visual reasoning process required across magnifications", "visual integration, challenges': "Explanation of why scale analysis necessary"
Gemini
Reference image coptions: (Multi-Scak Scale Image Descriptions)
Figure A22: Prompt for Gemini-2.5-Pra generate VQA samples given the region overview and the multi-
scale image descriptions generated from previous step.
51

---

This question was originally meant to be paired with an image. No image is provided, so please answer based on the text alone.
Gemini
(Question)
Think step step.
You must **ensure** your answer ends with "The answer X", where your final answer index.
Figure A23: Prompt for Semini-2.5-PPa to conduct educated guesses given only the text portion of the generated VQA samples.
52

---

Question: Considering architectural patterns, cellular atypia, and Taa
poorly differentiated Multi-Scale
Region Overview
Image with Coordinates
farcinoma image characterse by
acr Question
diagnosis?
Descriptions marked nuclear
A. Well-differentiated
HCC, prominent fibrosis
Prompt: Please analyze this pathology image by creating step-by-step viewing path that simulates how professional pathologist would examine the specimen to answer the given question. Focus specifically on areas relevant to the question and potential pathological changes.
Image Viewing Mechanics
default (L.0x) the complete image
original resolution 5.0x (including decimal values like 2.3x)
any
Image coordinates relative values from to
At any zoom level Z:
Field of view: 1/2² of total image area (1/Z of each side length)
Downscale ratio: 5/Z
Zoom Level Examples
1.0x: image (downscaled 5x]
area (downscaled 2.5x)
-3.0x: Shows (downscaled 1.67x)
-4,0x: 1/16 (downscaled 1.25x)
5.0x: Shows 1/25 (original resolution)
*Reference Image Captions**
will provide reference images captions at different magnifications:
-1.0x (full image) 2.0x (4 sections) 4.0x (16 sections)
-These are reference only your viewing path should:
Not exactly match these reference images' coordinate
-Consider other magnification levels
CCreate regions that may span my provided regions
## Viewing Path Guidelines
Your viewing path should follow these principles:
1. Begin with low-power scanning to assess tissue architecture and identify key patterns
Focus exclusively on areas relevant to given question
empty or background areas lacking diagnostic value
Prioritize regions showing key diagnostic criteria
specific question
appropriate magnification each observation (only changing zoom when necessary)
Follow logical diagnostic hierarchy (primary lesions before secondary changes)
Ensure minimal redundant overlap between examined areas
8. Demonstrate systematic movement and reasoning between viewing steps
provide the viewing path as JSON list, where each step includes:
"json
"step": [step number),
"magnification": [zoom level from 2.0x-5.0x].
"regian_coonee coordinate": [x1, y1, x2, y2], # Coordinates represent [top_left_x, top_left_y, cottom_right_x bottom_ (ictoom_iight)))
"reasoning": [reasoning for moving to this position).
"need to see": [description what should be observed this position),
Reference image captions: Question: (Question)
MMulti-Sclle Image Descriptions)
Gemini
Figure A24: Prompt for Gemini-2 2.5- generate navigation path planning given the region overview with coordinates, multi-scale image descriptions and the question generated in the previous step.
53

---

Overvicw
Move
pod pod Region ated duccall Description
Question: Considering arch Question. cellular atypia, and
Cropped Images from Navigation Path
Prompt: You are pathology specialist, you are examining images from pathologist's viewing path through pathological region. Based on these images, you need to provide detailed reasoning to answer specific question or select the most appropriate option from multiple choices.
each image viewing path, you will receive:
Magnification: Zoom level from 1.0k to 5.0x (where 1.0x shows the entire image)
Region coordinates: Position rou're viewing within the image (normalized from 0-1)
Need to see: Specific features ar structures to observe at this pasition
will provide reference descriptions for overall region caption. Use these descriptions only as reference pretend you don't have prior knowledge them when conducting your analysis.
Region captions: (Region description)
When analyzing the images, follow this structured approach:
1. Observational Evidence
Document key findings visible in each image relevant to the question
Be specific about which immage/region shows each finding
Describe morphology, tissue architecture, special features
Evidence Integration
Connect and compare findings different images in the viewing
3. Option Analysis (for multiple-choice questions)
Evaluate how the observed evidence supports or contradicts each option
Explain why incorrect options don't align with the visual evidence
Justify why the best option supported by your findings
Question-Speciffi Reasoning
Apply pathological principles directly question Explain how specific visual clues lead to your conclusion
Address any ambiguities or limitations in the visible evidence
5. Multi-scale Observation (1.0x-5.00 Ox, Low magnification-Hig! magnification): Consider how features appear across different magnifications (1.0x-5.0x) your viewing window represents portions original image at different zoom levels
viewing window
original resolution where:
1.0x shows full
2.0x shows 1/2 image length
3.0x shows 1/3 image length
6. Reasoning Process Requirements:
Use "thought flow" expression:
Think while observing
Record thinking process in real-time (give coordinate)
Allow returning re-observation
Record thinking turning points
Present your analysis as natural, flowing narrative from first-person perspective. Think and reason as observe, like an experienced pathologist examining the images in real-time. Record your thought process, including any moments when you need revisit certain images or when your thinking changes based on new observations, The description should flow as cohesive narrative *eveeral paragraphs** rather than bulleted list observations.
Conclude with brief
summary of your answer, followed "Therefore,
(X). Gemini
Question: Question
Prompt for Gemini-2.5 2.5-Pro to generate VQA-oriented reasoning based the planned navigation path, given cropped images along the navigation path, overall region description extracted from previous step, and the target question.
Figure A25;
54

---

WSI Overview Image
Prompt: thumbnail pathological slide image (Tissue Type). an expert pathologist, need you to analyze the given pathology image been divided into multiple patches with pre-drawn IDs."" "
Please:
1. Design systematic reading path by grouping the patch IDs based on their visual patterns (should include all patches), each patch ID should only belong to one group
2. Order the groups according to standard pathological diagnostic workflow
Note: aware that slide images may contain:
Multiple
fragments
Serial sections from the tissue block
Different tissue types/organs the same slide
These variations should inform your grouping strategy and reading path design.
Output the analysis in the following JSON format: "json
"groups":
"name": "[group name]", brief description of this group
"description": "[description of this region]".
"id_ iid_iist: [patch IDs in recommended viewing order).
"require_hice high caft__mimaaattin. [true/false), // = true the area require detailed high-power examination to confimm
diagnosis or unsure area
"severity_ree" reasoning": "[why this severity level was assigned]".
"diagnostic, "diagnosticpprioritt" [1-S), // Lower number "observation, !
higher examination priority is highest)
"key feature need to examine", "key feature need to examine",
b
Figure A26: Prompt for CPathAgent generate region selection results given a WSI overview.
55

---

Region Overview Image with Coordinates
Prompt: carefully examine pathology image simulate pathologist's viewing path. You and observe the image. In the default state (1.0x), the complete image 1/5 of its original resolution.
For zoom level
Field of view: 1/2" of the total image area (1/Z of each side length)
Downscale ratio: 5/7
Your viewing path steps should:
Check Avoid ponot//lcckgroun/uunnmotttt
Pay special attention transition
Selectively analysis areas with representative features (avoid redundant areas)
Ensure minimal overlap between sampled patches (except when examining the same region different magnifications) to reduce redundancy
Demonstrate logical continuity between each step movements between steps, including zooming in/out, should follow a systematic and logical pattern
Consider the blue coordinate grid for position reference (you can specify positions between grid points)
Please provide the viewing JSON list where each step includes: "jjson
"step": [step number),
"magnification". [zoom level from 2.0x-5.0x),
"region_cee coordinate": (xl, y1, x2, y2], # Coordinates represent (top_left_ top_left_y. ceatom_rrghh___e bottom_right, ritgt_rggttt Avoid empty
"reasoning": [reasoning for moving this position],
"need to__ee"_ [description of what should observed this position).
Figure A27: Prompt for CPathAgent to generate navigation path planning given the region overview with coordinates.
56

---

Overview
Zoom
Move
Focus
Cropped Images from Navigation Path
Prompt: This represents pathologist's logical viewing path for examining large pathological region, along with corresponding field-of-viev images. For each step in this systematic examination, please analyze the associated images based on the specified observations needed. The viewing parameters are defined as:
"magnification": Zoom level from 1-5x, where 1x shows the entire image "region_coom coordinate": Coordinates viewing position "need to rooseee" Specific
structures
position
For each pay special attention to the structures and features according its "need _o__tt____________ seeeeee description.
Then describe this image in detail, including reasoning, thinking you observe, like doctar. You should use first person perspective and follow this framework for analyzing pathological images:
Multi-scale Observation (1.0x-6.0 Ox, Low magnification-HHgg magnification):
Available magnifications
Normalized coordinates (0-1) each view (x1, y1, y2)
Your viewing window 1/6 original resolution where:
1.0x shows full Image
2.0x shows 1/2 image length
3.0x shows 1/3 image length
Reasoning Process Requirements:
A. "thought expression:
Think while observing
Record thinking process in real-time (give the box coordinate)
Allow returning for re-observation
Record thinking turning points
B. Diagnostic reasoning strategy:
Establish initial impression
Propose diagnostic hypotheses
Seek supporting evidence
Consider contrary evidence
Modify diagnostic hypotheses
Deepen analysis and argumentation
Think about whether the diagnosis of subclasses can be further determined
Evidence Usage Method:
Positive supporting evidence
Negative exclusion evidence, provide
non-conformity
4. Special Considerations:
Reflect real-time observation and thinking process
Allow for thought repetition and correction Maintain professional diagnostic prudence
Finally, please output additiional breif summary this region start with **Pathological Report:** one paragraph.
Figure A28: Prompt for CPathAgent to generate multi-scale
reasoning for description and diagnosis based the planned navigation path, given the cropped images along the navigation path.
57

---

Question: Considering architectural patterns, cellular atypia, and stre intera e'een aci Question diagnosis? Well-differentiated HCC, prominent fibrosis
Region Overview Image with Coordinates
Prompt: Please analyze this pathology image by creating step-by-ste viewing path that simulates how professional pathologist would examine the specimen to answer the given question. Facus specifically on areas relevant to the question and potential pathological changes.
Image Viewing Mechanics
default view (1.0x) shows the complete image
original resolution any up (including decimal values like 2.3x)
Image coordinates relative values from
At zoom level
Field of view: 1/2? of total image area (1/Z of each side length)
Downscale ratio: 5/Z
## Zoom Level Examples
Full image (downscaled 5x)
area (downscaled 2.5x) (downscaled 1.25x)
3,0x:
(downscaled 1.67k)
5.0x: Shows 1/25 (original resolution)
## Viewing Path Guidelines
Your viewing path should follow these principles:
1. Begin with low-power scanning to assess tissue architecture and identify key patterns
Focus exclusively on areas relevant the given question
Avaid empty or background areas lacking diagnostic value
Prioritize regions showing key diagnostic criteria the specific question
Use appropriate magnification each observation (only changing zoom when necessary)
Follow logical diagnostic hierarchy (primary lesions before secondary changes)
Ensure minimal redundant overlap between examined areas
8. Demonstrate systematic movement and reasoning between viewing steps
Please provide the viewing path as JSON list:
"jjson
"step": [step number),
"magnification": [z00m level from 2.0x-5.0x),
"region "reglon_coordinatee" coordinate": [x1, y1, x2, y2), # Coordinates represent [top_left_x top_left_y bowtooottttieeeeeeeeeeee right_xthhthpp bottom _appt_rrggttt
## Avoid empty area
"reasoning": Treasoning for moving this position],
"need_ to [description what should
position),
]
Question: (Question)
Figure A29: Prompt for CPathAgent to generate VQA-oriented navigation path planning given the region overview with coordinates and question.
58

---

Overview
Zoom In
Move
Focus
Question: Considering arc Question celluian atyple,
Cropped Images from Navigation Path
and
Prompt: pathology specialist, you examining images pathologist's viewing path through pathological region. these images, you provide detailed reasoning specific question select the appropriate option from multiple choices.
For each image in the viewing path, you will recelve:
Magnification: Zoom level from 1.0x to 0x (where 1.0x shows the entire image)
Region coordinates:
Position you're viewing within image (normalized from 0-1)
to see: Specific features structures observe position
When analyzing images, follow this structured approach:
1. Observational Evidence
Document key findings visible in each image relevant to the question Be specific about which image/region shows each finding
Describe cellular morphology, tissue architecture, and special features
Evidence Integration
Connect compare findings different images the viewing path
3. Option Analysis (for multiple-choice questions)
Evaluate how the observed evidence supports contradicts each option Explain why incorrect options don't align with the visual evidence
Justify why the best option is supported by your findings
Question- Specifccc Reasoning
Apply pathological principles directly question Explain specific visual clues lead your conclusion Address any ambiguities limitations the visible evidence
5. Multi-scale Observation (1.0x-5.0k, Low magnification--iil) magnification):
Consider how features appear across different magnifications (1.0x-5.0x)
Note how your viewing window represents portions original image at different zoom levels
Your viewing window 1/5 original resolution where:
image
shows image length
3.0x shows 1/3 image length
6. Reasoning Process Requirements:
Use "thought flow' expression:
Think while observing
Record thinking process in real-time (give the box coordinate)
Allow returning re-observation
Record thinking turning points
Present analysis as natural, flowing narrative from first-person perspective. Think and reason as observe. like experienced pathologist examining the images in real-time, Record your thought process, including moments when need to revisit certain images or when your thinking changes based on new observations. The description should flow as cohesive narrative *seeeral paragraphs** rather than a bulleted list of observations.
Conclude with a one-sentence summary of your answer, followed by "Therefore, answer is (X).
Question: Question)
Figure A300: Prompt for CPathAgent to generate VQA-oriented reasoning based the planned navigation path, given the cropped images along the navigation path and the target question.
59


Reviews:
[{"review_id": "ZobbsvsqRA", "reviewer": "Reviewer_EKpp", "summary": "This paper presents CPathAgent, an extension of CPath-Omni that adds agent-based sequential processing of whole slide pathology images, requiring the model to move around the image, zooming in and out at different resolutions, to produce region/slide descriptions and answer questions.", "questions": "1. For acceptance, a more complete comparison with CPathOmni would be necessary, along with further evaluation on whole-slide-level pathology tasks outside of the training distribution. Additionally (but less importantly), comparisons with recent foundation models capable of whole-slide representations, like TITAN or PRISM, would be helpful.\n\n2. More discussion/evaluation is needed to motivate agentic sequential processing of pathology images.\n\n2a. It makes sense that different magnifications are sufficient for different information that can be integrated into a question answer or a diagnosis. However, is it necessary to scan around the image sequentially and zoom in and out? Does this have any advantage over a model that takes the full region or whole slide image at multiple scales as input? Also, is there any additional training signal in the sequential setup that cannot be used in the everything-at-once input approach?\n\n2b. You mention that non-agentic models \"fail to replicate the diagnostic process of pathologists\". Could you explain why this is a problem?\nYou mention that non-agentic models have \"poor alignment with clinical workflows, undermining their credibility in diagnostic settings\". Could you explain what you mean by \"poor alignment with clinical workflows\" and why this undermines the credibility of validated model performance?\n\n3. Minor questions.\n3a. When all parameters are unfrozen, does this include CPath-Clip? If so, then all of it (including Virchow2) or part of it?\n3b. How are images passed to Gemini when evaluating it without \"CPathAgent's prompting strategy\"? Do they still cover a mix of magnifications? Does mixing magnifications improve performance or does the prompting strategy improve performance?\n3c. How do differences between models look when computing statistical significance in the tables?", "limitations": "See the evaluation concerns discussed above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "STRENGTHS\n\n1. Mimicking pathologists' navigation through whole slide images is an interesting way to extend region-only models to whole slide images. The navigation examples provided are compelling, suggesting that this navigation makes sense.\n\n2. A pathologist-validated visual QA benchmark dataset is proposed (I imagine the data will be released to the community). The dataset excludes questions that get answered (trivially) correctly without an image.\n\n3. CPathAgent appears to outperform CPathOmni on the PathMMU dataset (however, it is suspicious that CPathAgent also substantially outperforms expert pathologists).\n\n\n\n\nWEAKNESSES\n\n1. CPathAgent appears to be pretrained on the \"training sets\" of most of the evaluation tasks, to solve these tasks. It is unclear whether the model generalizes to new data, so there is a need for evaluations that are distinct from the training data.\n\n1a. CPath-Omni was pretrained to solve the TCGA classification tasks for NSCLC, BRCA, UCEC, THCA, ESCA, BLCA, and TGCT and it appears CPathAgent may be as well: \"training data inherited from CPath-Omni\". This makes TCGA WSI classification not actually zero-shot and it performs similary to MIL aggregators trained on each of these tasks.\n\n1b. PathMMU-HR2 is developed using the same method / tools as the training data for CPathAgent, so I expect CPathAgent to have an advantage on it.\n\n2. While I imagine that the proposed agentic approach may improve model performance, I am not convinced this is the case by the presented results.\n\n2a. To support the claim that the agentic component of CPathAgent improves over CPathOmni, it would be useful to evaluate CPathOmni on all tasks but it's only evaluated on PathMMU, where CPathOmni is close to expert human performance and CPathAgent somehow greatly improves over pathologists (which is weird). Can CPathOmni be evaluated on PathMMU-HR2 and the TCGA WSI classification tasks as well? Does CPathAgent outperform it there?\n\n2b. Greatly exceeding expert pathologist performance on PathMMU makes the quality of this dataset suspicious. Is the QA frequently wrong? The PathCLS subset of PathMMU was derived from labeled datasets and may be the cleanest and most correctly annotated subset of PathMMU. On this subset, there is no difference in performance between CPathOmni (79.0) and CPathAgent (79.2).\n\n2c. Why are CPathOmni results not shown in Table 3 (TCGA WSI classification)? Looking at the CPathOmni paper, it appears that there may not be much difference between CPathAgent and CPathOmni on these tasks, after all: TCGA-BRCA 88.5 vs 89.2, TCGA-NSCLC 90.8 vs 88.8, TCGA-RCC 94.6 vs 94.0, TCGA-ESCA 97.1 vs 92.1, TCGA-BLCA 62.7 vs 70.7, TCGA-THCA 63.2 vs 58.5; an average balanced accuracy of 82.8 for CPathAgent and 82.2 for CPathOmni. Did the addition of agentic processing of slides really improve over CPathOmni?", "was_revised": true, "final_justification": "With the additional results and clarifications, I am raising my score. The reasons to accept now outweigh the reasons to reject.", "rebuttal": "Thank you for your thorough review and  constructive feedback.  We are grateful for your recognition of CPathAgent as an interesting approach by mimicking pathologists' navigation. Below, we provide detailed responses to each of your concerns:\n\n> ### W1a & Q1. CPathAgent inherits training data from CPath-Omni (pretrained on TCGA), making TCGA evaluation not truly zero-shot and similar to traditional MIL approach. \n\nThank you for this important question. Although CPathAgent inherits patch-level pretraining from CPath-Omni, its **WSI-level reasoning is fundamentally different**.\n\n**Key Distinction:** While CPath-Omni and MIL methods operate as **black-box models outputting only final predictions**, CPathAgent introduces a **paradigm shift** through agent-based reasoning that mimics pathologists' diagnostic workflow, dynamically navigating, zooming, and generating interpretable step-by-step explanations of observed morphological features and diagnostic rationale across WSIs.\n\n**Zero-shot Validation:** To demonstrate true generalization, we evaluated on the out-of-domain **CPTAC-Lung cohort** using balanced accuracy:\n\n|Model|Training WSIs|BACC|\n|-|-|-|\n|PRISM|587,196|83.2|\n|CPath-Omni|11,728|82.3|\n|CPathAgent|5,254|**88.1**|\n\nDespite training on only 5,254 WSIs, CPathAgent outperforms PRISM (trained on 100× more data) and CPath-Omni (trained on 2× more data) by learning **intermediate reasoning processes** rather than memorizing dataset-specific patterns. \n\n> ###  W1b. PathMMU-HR2 is developed using the same method as the training data for CPathAgent, so I expect CPathAgent to have an advantage on it.\n\nYou raise an important point. While we used similar methods for both training data and benchmark construction, this  was **unavoidable as no comprehensive ultra-high resolution pathology benchmarks existed**.\n\n**To further validate our model's generalization**, we try our best to find a **completely held-out**, highest resolution pathology dataset, BRACS (breast carcinoma classification), the only available dataset with regions up to **10K pixels**, and evaluate CPathAgent on it:\n\n|Model|BRACS|\n|-|-|\n|GPT-4.1|44.2%|\n|Gemini-2.5-Pro|49.0%|\n|CPath-Omni|57.9%|\n|**CPathAgent**|**64.3%**|\n\nThe 6.4% improvement over CPath-Omni on this **independent dataset** demonstrates the robustness of CPathAgent's agent-based reasoning capabilities.\n\n> ###  W2a&2c&Q1. Can CPathOmni be evaluated on PathMMU-HR2 and the TCGA WSI classification tasks as to properly validate CPathAgent's agentic improvements?\n\nThank you for this valuable suggestion. We have conducted the requested evaluations:\n\n#### PathMMU-HR² Result\n\n|Model|BRCA|LUAD|LUSC|KIRP|KIRC|KICH|ESCA|THCA|BLCA|TGCT|**Overall**|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|CPath-Omni|72.6|77.6|71.3|64.1|67.5|59.7|74.3|71.5|76.6|78.8|71.7|\n|**CPathAgent**|**87.0**|**88.5**|**87.8**|**87.9**|**92.9**|**78.9**|**90.7**|**89.0**|**90.7**|**93.0**|**88.6**|\n\nThe 16.9% performance gap occurs because other models directly resize ultra-high resolution images (16000×16000) to their fixed input size (e.g., 336*336). In contrast, CPathAgent uses strategic navigation planning and dynamic zoom-in to examine critical diagnostic details without losing important information.\n\n#### WSI Classification Result\n\nDirect comparison on the TCGA WSI tasks is unfair as **CPath-Omni was trained on the larger TCGA dataset (11,728 cases)**, while CPathAgent used only cases with WSI reports (5,254 cases) with **different train/test splits.**\n\nFor fair evaluation, we **retrained** CPath-Omni's stage 3 using our identical training split:\n\n|Model|TCGA-BRCA|TCGA-NSCLC|TCGA-RCC|TCGA-ESCA|TCGA-BLCA|TCGA-THCA|**Avg**|\n|-|-|-|-|-|-|-|-|\n|CPath-Omni (retrained)|78.6|86.7|91.3|89.9|**65.5**|52.6|**77.4**|\n|**CPathAgent**|**88.5**|**90.8**|**94.6**|**97.1**|62.7|**63.2**|**82.8**|\n\nCPathAgent achieves a significant 5.4% improvement, demonstrating that learning pathologist-like diagnostic reasoning and intermediate processes leads to better generalization and learning compared to weakly-supervised approaches.\n\n> ###  W2b. Greatly exceeding expert pathologist performance on PathMMU makes the quality of this dataset suspicious, while the PathCLS subset shows nearly identical results.\n\nThank you for raising this important concern. We provide clarification on several key points:\n\nThe PathCLS subset evaluates **simple perceptual classification tasks** with low-resolution images (e.g., PatchCamelyon: 96×96 patches where **simple vision encoders can achieve 99%+ accuracy**), fundamentally different from other complex reasoning questions. Since CPathAgent inherits patch-level classification data from CPath-Omni, similar PathCLS performance is expected. \n\n**Regarding Expert Performance**, CPathAgent exceeds reported expert baselines due to: \n\n- Limited Expert Coverage: PathMMU experts specialize in 1-2 areas while the dataset spans diverse tissues. No single expert masters all domains.\n- Text-based Shortcuts: PathMMU authors acknowledged LMMs can exploit strong text-based guessing abilities that humans cannot easily leverage.\n\nTo address the text-based shortcut, PathMMU authors created PathMMU-Pro with more rigorous question design. We  conduct further evaluation on PathMMU-Pro:\n\n|Model|Overall Tiny|Overall All|PubMed Tiny|PubMed All|SocialPath Tiny|SocialPath All|EduContent Tiny|EduContent All|\n|-|-|-|-|-|-|-|-|-|\n|Human|**69.4**|-|**71.2**|-|**70.1**|-|66.9|-|\n|CPath-Omni|62.1|61.8|60.5|60.8|58.8|62.1|66.7|63.0|\n|CPathAgent|66.3|65.3|64.6|65.8|65.4|63.6|**68.7**|66.2|\n\nCPathAgent **no longer exceeds expert performance on this cleaner benchmark** while **still outperforming CPath-Omni**.\n\n> ### Q2a : Is it necessary to scan around the image sequentially and zoom in and out? Does this have any advantage over a model that takes the full region or WSI at multiple scales as input? Is there any additional training signal in the sequential setup that cannot be used in the everything-at-once input approach?\n\nThis is an excellent question that highlights a fundamental design choice in our approach. Sequential navigation offers critical advantages:\n\n**1. Interpretable Clinical Reasoning**\n\nOur method generates **step-by-step natural language explanations** of the diagnostic process, unlike MIL/CPath-Omni that produce only final predictions. For **life-critical cancer diagnosis**, AI must provide transparent reasoning that pathologists can verify.\n\n**2. Rich Sequential Supervision Signal**\n\n- Traditional Multi-scale approaches use **weakly supervised WSI-level labels** with MIL, which: (1) Cannot explicit  learn which regions are diagnostically important. (2) Have no guidance on optimal magnification levels (3) Fail to capture the diagnostic reasoning process. Therefore, MIL models can **easily learn spurious correlations** from limited training data, especially when the training signal is very weak (only slide-level labels).\n- Our approach provides **explicit supervision at each step**: where to navigate, when to zoom, and what to examine, this creates a fundamentally stronger learning signal that enables robust reasoning.\n\n**3. Superior Generalization**\n\n- CPathAgent learns the diagnostic **process** rather than dataset-specific patterns, enabling stronger generalization to new datasets and achieving better performance with less data (experiments in W1a&1b)\n- Gemini-2.5-Pro improves from 73.2% to 76.4% when equipped with our sequential navigation approach (Table 2)\n\n\n> ### Q2b: Could you explain why  \"fail to replicate the diagnostic process of pathologists\" is a problem?   Could you explain what you mean by \"poor alignment with clinical workflows\" and why this undermines the credibility of validated model performance?\n\nAs mentioned in Q2a's response, **cancer diagnosis is life-critical and requires human verification**. Models that only output final predictions force pathologists to manually re-examine entire slides to ensure diagnostic accuracy, as they cannot trace the AI's reasoning to verify critical findings weren't missed.\n\n**\"Poor alignment with clinical workflows\"** refers to models that don't mirror how pathologists systematically examine tissue: starting with low-magnification overview, then strategically zooming into regions of interest. This misalignment creates two problems:\n\n1. **Verification inefficiency**: Without interpretable reasoning paths, pathologists cannot quickly validate AI findings\n2. **Limited generalization**: As our experiments demonstrate (W1a&1b), models aligned with clinical workflows learn the general diagnostic process rather than dataset-specific patterns, enabling better performance with less data. \n\n\n\n> ###  Q 3a. When all parameters are unfrozen, does this include CPath-Clip? If so, then all of it (including Virchow2) or part of it?\n\nYes, all parameters are unfrozen during Stage 3 training, including the entire CPath-CLIP module (with Virchow2). This allows end-to-end optimization for agent-based reasoning.\n\n> ### Q 3b.  How are images passed to Gemini when evaluating it without \"CPathAgent's prompting strategy\"? Do they still cover a mix of magnifications?\n\nFor non-agent evaluation, models receive the full large region image (16000×16000) directly as a single input, without sequential navigation and do not cover a mix of magnifications. As shown in Table 2, Gemini with \"CPathAgent's prompting strategy\" compared to without it can improve performance by 3.2%.\n\n> ### Q3c. How do differences between models look when computing statistical significance in the tables?\n\nWhile we acknowledge the importance of statistical significance testing, LMMs during evaluation typically use a single final result, as the computational cost of running multiple trials with large pathology models is time-consuming and impractical within the rebuttal timeframe; we will address this in future work. However, our method demonstrates consistent improvements across four diverse evaluation settings, providing empirical evidence of effectiveness."}, {"review_id": "7abiwGkGOm", "reviewer": "Reviewer_e2cS", "summary": "This paper introduces CPathAgent, an agent-based framework for pathology image analysis that mimics pathologists' diagnostic workflow. The paper defines an agent as: a system capable of autonomously executing zoom-in/out and navigation operations across pathology images, achieving specific goals through perceiving the environment, making decisions, and taking actions. Specifically, CPathAgent emulates pathologists' diagnostic process through three stages: (1) Global Screening stage to identify suspicious regions; (2) Navigation Planning stage to formulate examination paths; (3) Multi-scale Multi-view Sequence Reasoning stage to integrate observations across different magnification levels. The system can mimic pathologists by starting with low-magnification scanning, progressively zooming in on regions of interest, and formulating diagnostic conclusions through dynamic visual reasoning processes. This paper also proposed the 'PathMMU-HR' dataset.", "questions": "1. How do your proposed AI agent adapt the newly incomed training data? \n2. Decision-making mechanism entirely opaque: During training, Gemini generates navigation paths based on WSI reports, but during inference without such reports, how does the model determine where to examine next?\n3. Bridging mechanism unexplained: Does the model memorize navigation patterns or genuinely learn visual reasoning? Generalization capability questionable: If the system merely memorizes training navigation patterns, how does it handle novel pathological presentations?\n4. Generalization capability questionable: If the system merely memorizes training navigation patterns, how does it handle novel pathological presentations?\n5. Base model unspecified: The paper merely mentions \"open-source VLMs\" without detailing the specific architecture.", "limitations": "No assessment of the error accommulation. This paper proposed the method contains three agents. Each agent can make mistake. The author did not provide the detailed information about the performance when error may accommulates. For instance, the first stage 'Global screening and region selection' made a mistake(such as filtered a important tile), what will happend at the end? Backtracking mechanism: Can the system return to previously examined locations for re-evaluation? If can not, if would be interest for future research. Lack of interactivity: Real pathologists adjust staining, section angles based on preliminary findings - this system cannot.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Clinical Relevance\nSuccessfully addresses the gap between AI systems and actual pathologist workflows by mimicking their multi-scale examination process, potentially improving interpretability and clinical adoption.\n2. Novel Benchmark (PathMMU-HR²)\nIntroduces the first benchmark for huge region analysis (16000×16000 pixels) - a critical but previously overlooked scale in pathology. Validated by three board-certified pathologists.\n3. Comprehensive Evaluation\nEvaluates across three scales (patch, huge region, WSI) on multiple datasets, showing consistent improvements and versatility of the approach.\n4. Pathologist Validation\nIncludes real pathologist evaluation of navigation paths and reasoning quality, adding clinical credibility beyond just benchmark metrics.\n\nWeakness:\n1.Novelty Issues with Agent Definition\nThe paper claims to be \"the first agent-based framework for pathology,\" but it essentially repackages traditional multi-scale image analysis as an agent concept. The authors should compare their work with existing popular agent frameworks and emphasize the rationality of their proposed pathology-specific agent definition. The so-called \"agent behaviors\" (zooming, navigation) are fundamentally predefined image cropping and scale selection, lacking genuine autonomous decision-making capabilities. Compared to existing attention mechanisms or multi-scale feature fusion methods, the conceptual innovation is limited.\n\n2.Reliability of Data Generation\n\nThe work heavily relies on Gemini-2.5-Pro to generate training data (278K instances) - how is the quality of such synthetic data ensured? Do the generated navigation paths and diagnostic reasoning genuinely reflect general pathologists' actual workflows? There is a lack of comparative validation against real pathologist behavioral data.\n\n3.Fairness of Evaluation\nIn the PathMMU-HR² benchmark, only CPathAgent uses the agent-based approach, while other models do not employ the same prompting strategy. Part of the performance improvement may stem from the inherited CPath-Omni training data rather than the agent mechanism itself. The pass@k evaluation uses only 40 samples, which lacks statistical significance.\n\n4.Ambiguity in Technical Implementation\nHow is \"navigation planning\" actually implemented? Is it merely selecting image patches according to predefined rules? The paper does not clearly explain how the agent \"decides\" when to stop exploration.There is a lack of analysis on computational complexity and inference time.", "was_revised": true, "final_justification": "Thank you for your rebuttal. I think you have answered my concern. I have changed the rating. I think it is worth discussing this work in the conference.", "rebuttal": "We greatly appreciate your thoughtful feedback and are grateful for your recognition of CPathAgent's clinical relevance  and for addressing the gap between AI systems and actual pathologist workflows by mimicking their examination process. However, some concerns may stem from misunderstandings, and we address these one by one below:\n\n> ### It essentially repackages traditional multi-scale image analysis as an agent concept. The so-called \"agent behaviors\"  are fundamentally predefined image cropping and scale selection, lacking genuine autonomous decision-making. Compared to existing attention mechanisms or multi-scale feature fusion, the conceptual innovation is limited.\n\nWe appreciate the reviewer's concern but respectfully disagree that CPathAgent merely repackages traditional multi-scale analysis.\n\n**1. Most agent frameworks operate within constrained action spaces**, classical agent systems define constrained action sets: WebGPT [1] (browser commands), ReAct [2] (search/lookup/finish), MM-ReAct [3] (crop/enhance). **Autonomy in agents is characterized by learning policies for action selection, not by discovering new actions**. This is commonly recognized in agent works.\n\n**2. CPathAgent exhibits genuine autonomous decision-making** Unlike static multi-scale fusion, CPathAgent:\n- **Dynamically plans navigation paths** based on visual observations.\n- **Learns when and where to zoom and move**.\n- **Generates action rationales** explaining why each decision was made\n\nThis differs fundamentally from predetermined attention mechanisms lacking intermediate reasoning and interpretability.\n\n**3. Quantitative evidence** The significant performance gaps between agent-based and non-agent approaches in Table 2 (Gemini-2.5-Pro: 73.2%→76.4%, GPT-4.1-mini: 60.1%→62.3%) demonstrate that dynamic navigation policies  benefits model performance.\n\nNotably, several pathology agent works have emerged after our submission [4,5], further supporting our agent paradigm.\n\n[1] Nakano et al. WebGPT. arXiv:2112.09332, 2021. \n\n[2] Yao et al. ReAct. ICLR 2023. \n\n[3] Yang et al. MM-ReAct. arXiv:2303.11381, 2023. \n\n[4] Li et al. OmicsNavigator. bioRxiv, 2025. \n\n[5] Chen et al. Evidence-based diagnostic reasoning with multi-agent copilot for human pathology. arXiv:2506.20964, 2025.\n\n> ###  How is the quality of such synthetic data ensured? Do the generated navigation paths and diagnostic reasoning genuinely reflect pathologists' actual workflows?\n\nThis is a good question. We ensure synthetic data quality through:\n\n**Prompt Refine**: During prompt engineering, board-certified pathologists provided iterative feedback to refine our prompts, resulting in comprehensive prompts (Appendix Figures D.8-D.16).\n\n**Empirical Validation**: The quality is demonstrated by **upper bound performance in Table 3**: achieving **100%** acc on TCGA-ESCA, **97.9%** on TCGA-NSCLC, and **97.0%** on TCGA-BRCA when directly using synthetic 'ground truth' for WSI classification.\n\nTo further validate the synthetic data, we invite pathologists evaluate 150 randomly sampled navigation paths and diagnostic reasoning examples based on: (1) diagnostic reasoning validity, (2) coverage of critical diagnostic regions, and (3) clinical soundness. **92.7% (139/150)** met clinical standards, which confirms the data quality.\n\n> ### Fairness of Evaluation In the PathMMU-HR² benchmark, only CPathAgent uses the agent  approach, while other models not. Part of the performance improvement may stem from the inherited CPath-Omni training data rather than the agent mechanism. The pass@k evaluation uses only 40 samples.\n\nThank you for this important concern. \n\n1. Regarding  **evaluation fairness,** as shown in Table 2, we provided the **same agent-based prompting strategy** to GPT-4.1-mini and Gemini-2.5-Pro. Under identical prompts, CPathAgent significantly outperformed both (88.6% vs. 62.3% and 76.4%).  These models also improved with our agent approach (GPT-4.1-mini: 60.1%→62.3%, Gemini-2.5-Pro: 73.2%→76.4%).\n\n**2. Performance attribution:** To isolate our agent mechanism from inherited data, we evaluated **CPath-Omni**  on PathMMU-HR²:\n|Model|BRCA|LUAD|LUSC|KIRP|KIRC|KICH|ESCA|THCA|BLCA|TGCT|**Overall**|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|CPath-Omni|72.6|77.6|71.3|64.1|67.5|59.7|74.3|71.5|76.6|78.8|71.7|\n|**CPathAgent**|**87.0**|**88.5**|**87.8**|**87.9**|**92.9**|**78.9**|**90.7**|**89.0**|**90.7**|**93.0**|**88.6**|\n\nThe 16.9% improvement shows that performance gains come from our pathologist-like reasoning (Stage 3's data), not inherited training data.\n\n**3. Statistical significance:** We expanded pathologist evaluation to 100 samples. Results show navigation path satisfaction reached 85.0% and reasoning satisfaction reached 89.0% at pass@8. Due to rebuttal format limitations, we will present the figures in the revised paper.\n\n> ### How is \"navigation planning\" actually implemented? Is it merely selecting image patches according to predefined rules?  There is a lack of analysis on computational complexity and inference time.\n\nThank you for this important technical question. Our navigation planning is **not rule-based** but fully autonomous. The model:\n1. Observes each huge region at low magnification\n2. Analyzes visual features to decide next actions (zoom locations and scales)\n3. Outputs navigation decisions in JSON format for execution\nThis autonomous navigation is learned during training, not predefined. \n\nRegarding computational complexity, we leverage **vLLM** parallel processing to analyze multiple regions simultaneously. On H800 GPU, CPathAgent processes each 16000×16000 region in **12.7 seconds on average** (two VLM calls, averaged over 1,500 huge regions), with **237.4 seconds for each WSI**. While slower than MIL, this trade-off is worthwhile as the intermediate reasoning process is crucial for interpretability and diagnostic accuracy. In this work, we focus on propose a novel paradigm for pathology analysis, with efficiency optimization left for future work.\n\n\n> ### Q1 How do your proposed AI agent adapt the newly incomed training data?\n\nThank you for this important question. CPathAgent handles new data through two approaches:\n1. **Zero-shot Generalization:** CPathAgent demonstrates strong generalization by learning pathologists' inherent reasoning logic rather than memorizing dataset-specific patterns. We evaluated CPathAgent on the CPTAC-lung (non-TCGA, holdout data) without  fine-tuning:\n\n|Model|Training WSIs|BACC|\n|-|-|-|\n|PRISM|587,196|83.2|\n|CPath-Omni|11,728|82.3|\n|CPathAgent|5,254|**88.1**|\n\nDespite pretrained on significantly fewer WSIs, CPathAgent achieves superior generalization to unseen datasets.\n\n**2. Domain Adaptation:** When needed, our framework can automatically generate high-quality agent-based training data for fine-tuning using WSI reports (easily available in practice), without costly manual annotations.\n\n> ### Q2 During training, Gemini generates navigation paths based on WSI reports, but during inference without such reports, how does the model determine where to examine next?\n\nWe need to clarify that the WSI reports are used **only during training data generation**, not during model training or inference.\n\nIn the **Data Generation Phase,** we use WSI reports to guide Gemini-2.5-Pro in generating high-quality navigation paths and reasoning sequences. Without this guidance, Gemini-2.5-Pro often produces low-quality data with incorrect diagnoses and navigation patterns.\n\nDuring the **Training/Inference Phase,** the model **never** uses WSI reports. It learns to navigate and reason based purely on visual features and autonomously decides where to examine like pathologists do without prior reports.\n\n>  ### Q3: Does the model memorize navigation patterns or genuinely learn visual reasoning? If the system merely memorizes training navigation patterns, how does it handle novel pathological presentations?\n\nThank you for this insightful question. We provide several  **empirical evidence against pure memorization:**\n\n1. **Low Navigation Path Similarity:** We analyzed navigation paths across 1,000 test samples using step-wise Region of Interest (ROI) Intersection over Union (IoU) as the similarity metric. The average correlation between navigation paths from different images was only **0.12**, demonstrating that the model generates content-adaptive navigation strategies rather than memorizing fixed patterns.\n2. **Performance on Diverse Test Cases:** TCGA WSIs exhibit significant heterogeneity across different cases. If model only memorized patterns, performance would deteriorate dramatically. Instead, CPathAgent maintains high performance compared to MIL methods and CPath-Omni.\n3. **Generalization capability:** As shown in Q1's response, CPathAgent achieves 88.1% on the unseen CPTAC-lung dataset, outperforming models trained on 100× more data. This strong generalization proves the model has learned genuine visual reasoning  rather than memorized patterns.\n\n> ### Q4 Base model unspecified: The paper merely mentions \"open-source VLMs\" without detailing the specific architecture. \n\nThank you for highlighting this omission. We apologize for the lack of this information. Due to space constrains, we placed the detailed model architecture and training in Appendix Section C.1.  We will move it to the main paper.\n\n> ###  Each agent can make mistake. The author did not provide the detailed information about the performance when errors may accommulates. Can the system return to previously examined locations for re-evaluation? .\n\nThat is correct,  error accumulation in multi-step agent systems is indeed a common challenge in agent systems. Currently, CPathAgent follows a forward-only navigation strategy without backtracking. We agree that  backtracking  would be valuable future work to mitigate error accumulation.\n\nWe will provide detailed error cases in the revised appendix (as figures cannot be included in the rebuttal) to better show when and how errors occur in our system."}, {"review_id": "jNVWJEoYWV", "reviewer": "Reviewer_m8m7", "summary": "Traditional computational pathology models lack the multi-scale reasoning process employed by pathologists during diagnosis, resulting in limited interpretability and misalignment with real-world clinical workflows. This paper proposes CPathAgent, an innovative agent-based model designed for high-resolution pathology image analysis that mimics the diagnostic logic of expert pathologists. CPathAgent dynamically performs zooming and navigation operations, and incorporates a multi-stage training strategy to achieve interpretable analysis. In addition, the study introduces PathMMU-HR², the first benchmark dataset for huge region reasoning, covering diverse tissue types and pathological features. The method proposed in this paper achieves better results than the SOTA methods.", "questions": "Please refer to the “Weaknesses” section.", "limitations": "None", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. This paper enhances the performance of current Pathology Language-Vision Models by simulating the diagnostic process of human pathologists reading pathology slides, demonstrating that mimicking this missing process can improve diagnostic accuracy.\n2. It constructs a dataset specifically designed for training pathology model agents.\n3. It constructs a dataset for analyzing huge region pathology images.\n\nWeaknesses:\n1. I think this paper places too much emphasis on dataset construction during the writing process, which leads to an oversimplified and weakened description of the agent’s architecture, training process, and reasoning mechanism. This may give other researchers the impression that the paper is primarily about dataset track rather than method innovation.\n2. Many of the training steps in this paper overlap with CPath-Omni. However, the differences from CPath are not sufficiently highlighted, and the paper should better emphasize its innovations.\n3. The dataset used for training the navigation module is mainly generated by large models. It remains unclear whether the generated navigation paths align with the actual behaviors of human pathologists when reading pathology slides. This needs further validation through experiments or expert analysis.\n4. The paper lacks an analysis of the functional contributions of different agent components; specifically, it does not include ablation studies on individual modules, making it difficult to assess their respective impacts on overall performance.\n5. Compared with CPath-Omni, the proposed method requires multiple invocations of VLMs, yet the paper does not provide a comparative analysis in terms of time efficiency and computational cost.\n6. Will the dataset constructed in this paper be open-sourced?", "was_revised": true, "final_justification": "The authors’ responses have addressed most of my concerns. I will maintain my score.", "rebuttal": "We greatly appreciate your thoughtful feedback and are grateful for your recognition of CPathAgent's innovative approach in mimicking pathologists' multi-scale diagnostic reasoning and our contributions to dataset construction for pathology model training,  and we address all the concerns you have point by point below:\n\n>  ### I think this paper places too much emphasis on dataset construction during the writing process.\n\nWe appreciate the reviewer's valuable feedback.  Due to space limitations, technical details are currently placed in Appendix C.1 (model architecture and training strategy).  We will move this part to Section 3, and restructure the paper to prioritize the **agent-based architecture** and **multi-scale reasoning framework** in the main paper.\n\n\n>  ### Many of the training steps in this paper overlap with CPath-Omni. However, the differences from CPath are not sufficiently highlighted, and the paper should better emphasize its innovations.\n\nWe appreciate the reviewer's important observation. While CPathAgent inherits some patch-level training components from CPath-Omni, it represents a **fundamentally different paradigm** for WSI and huge region analysis:\n\nThe critical innovation lies in **Stage 3 of our training pipeline**, the agent-based reasoning framework for WSI/huge region analysis:\n\n**1. Architectural Innovation:**\n\n- **CPath-Omni**: Static MIL-based approach that embeds all patches and produces a single output\n- **CPathAgent**: Dynamic agent framework that performs hierarchical examination: first planning at low magnification, then strategically navigating to regions of interest and reasoning across multiple scales and multiple images.\n\n**2. Interpretability & Clinical Utility:**\n\n- **CPath-Omni**: Black-box predictions without reasoning traces\n- **CPathAgent**: Full interpretability with:\n  - Real-time natural language explanations for each navigation decision\n  - Step-by-step diagnostic reasoning (see Appendix Figures D.5-D.7)\n  - Verifiable examination paths that clinicians can validate\n\nIn cancer diagnosis, AI can't replace doctors since no model is 100% accurate. With black-box models like CPath-Omni that only give final answers, doctors still need to manually check the entire slide, as they can't tell whether the AI actually \"saw\" the cancer or just made a lucky guess. In contrast, CPathAgent shows its WSI navigation and reasoning step-by-step, so doctors can quickly verify the AI's reasoning, transforming AI from a black box into a **collaborative diagnostic tool**.\n\n\n\n>  ### The dataset used for training the navigation module is mainly generated by large models. It remains unclear whether the generated navigation paths align with the actual behaviors of human pathologists when reading pathology slides. This needs further validation through experiments or expert analysis.\n\nThis is a good question. We ensure synthetic data quality through:\n\n**1. Expert-Guided Prompt Development:**  During prompt engineering, board-certified pathologists iteratively refined our prompts to ensure generated paths reflect clinical examination patterns (detailed prompts in Appendix D.8-D.16).\n\n**2. Empirical Performance Validation:** The quality of synthetic data is demonstrated by  directly using synthetic 'ground truth' data for WSI classification, indicated by **upper bound results** in Table 3:\n\n- TCGA-ESCA: 100% accuracy\n- TCGA-NSCLC: 97.9% accuracy\n- TCGA-BRCA: 97.0% accuracy\n\nThese results confirm that our synthetic diagnostic descriptions contain accurate and complete clinical and diagnostic information.\n\n**3. Expert Validation:** Following your suggestion, we conducted further validation where pathologists evaluated 150 randomly sampled navigation paths and reasoning traces on three criteria:\n\n- Diagnostic reasoning validity\n- Coverage of critical diagnostic regions\n- Clinical soundness\n\n**Result: 92.7% (139/150)** met clinical standards, confirming strong alignment with actual pathologist workflows.\n\nWe will add these validation results to the main paper to strengthen our claims about clinical relevance.\n\n\n\n> ### The paper lacks an analysis of the functional contributions of different agent components; specifically, it does not include ablation studies on individual modules, making it difficult to assess their respective impacts on overall performance.\n\nThank you for this valuable suggestion. Following your suggestion, we conducted systematic ablation studies to evaluate the functional contribution of each agent component, we isolated the impact of individual modules by introducing controlled perturbations:\n\n1. **Global Screening Ablation**: We replaced the strategic region selection with random selection of 70% of WSI regions. This 70% threshold aligns with the global screening module's design objective of identifying diagnostically relevant regions from WSI thumbnails while filtering out ~30% uninformative areas.\n2. **Navigation Planning & Multi-scale Reasoning Ablation**: Replaced dynamic navigation paths and multi-scale reasoning with direct whole-region description.\n\n**Results**\n\n| Model                                  | TCGA-BRCA | TCGA-NSCLC | TCGA-RCC | TCGA-ESCA | TCGA-BLCA | TCGA-THCA | **Avg**  |\n| -------------------------------------- | --------- | ---------- | -------- | --------- | --------- | --------- | -------- |\n| Full CPathAgent                        | **88.5**  | **90.8**   | **94.6** | **97.1**  | 62.7      | **63.2**  | **82.8** |\n| **w/o Strategic Global Screening**     | 84.7      | 88.7       | **94.6** | 94.1      | 60.8      | 58.9      | 80.3     |\n| w/o Navigation & Multi-scale Reasoning | 80.0      | 83.6       | 88.7     | 94.1      | 52.2      | 51.3      | 75.0     |\n\n**Random Global Screening Impact (-2.5%)**: The modest performance drop validates that strategic region selection effectively balances efficiency and accuracy, successfully mimicking pathologists' selective attention.\n\n**Navigation & Multi-scale Reasoning Impact (-7.8%)**: The substantial performance degradation reveals:\n\n- Direct processing of high-resolution regions (16000×16000) causes critical information loss through forced resizing\n- Absence of pathologist-like reasoning that strategically integrates multiple fields of view at varying magnifications impact model performance\n\nThese results demonstrate that our agent-based approach derives its effectiveness from the synergistic interaction of all components, with navigation and multi-scale reasoning being particularly crucial for diagnostic accuracy.\n\n\n\n>  ### Compared with CPath-Omni, the proposed method requires multiple invocations of VLMs, yet the paper does not provide a comparative analysis in terms of time efficiency and computational cost.\n\nThank you for this important question about computational efficiency. We conducted a comparative analysis using 100 randomly sampled WSIs to evaluate the time and computational costs between CPath-Omni and CPathAgent on H800 GPU.\n\n| Model      | Vision Preprocessing Part | LMM Part                                           | Total Time |\n| ---------- | ------------------------- | -------------------------------------------------- | ---------- |\n| CPath-Omni | 153.6s                    | 0.7s （only generate single classification result) | 154.3s     |\n| CPathAgent | 0s                        | 237.4s                                             | 237.4s     |\n\nThe two models exhibit fundamentally different computational profiles:\n\n**CPath-Omni**: Requires extensive preprocessing, which extracts and encodes patches at multiple scales across the entire WSI. For a single region, it processes three different scales, resulting in substantial preprocessing overhead (153.6s).\n\n**CPathAgent**: Eliminates vision preprocessing by directly operating on regions of interest. To be specific,  we leverage vLLM for parallel region processing. On H800 GPU, CPathAgent processes each 16,000×16,000 region in **12.7 seconds** (averaged over 1,500 regions), with a total of **237.4 seconds per WSI**.\n\nWhile CPathAgent shows some increase in total processing time (83.1s), this represents a worthwhile trade-off. The agent-based approach provides Interpretable diagnostic reasoning and superior accuracy through explicit navigation paths. In this work, we focus on proposing a novel paradigm for pathology analysis, with efficiency optimization left for future work.\n\n\n\n>  ### Will the dataset constructed in this paper be open-sourced? \n\nYes, we are committed to fully open-sourcing our complete dataset, including all synthetic navigation paths, diagnostic reasoning data, and associated annotations to support reproducibility and future research in interpretable pathology AI."}, {"review_id": "X718KhscqW", "reviewer": "Reviewer_cWNz", "summary": "This paper proposes a multimodal large language model (MLLM) for pathological image huge regions, with a design approach that shows certain innovation. The authors identify the limitations of existing methods in pathological image analysis: patch-based methods are not clinically applicable, while WSI-based methods exceed the processing capabilities of MLLMs. Therefore, designing huge regions that fall between these two approaches is a meaningful attempt.", "questions": "## 1. Instruction Dataset Construction\nFigure 2 shows the instruction dataset construction process, where the key step is generating patch descriptions for each specific patch based on huge region descriptions. This process is reasonable under zoom-in operations, as each patch remains clear for the Gemini model. \n\nHowever, the paper does not clearly specify the specific input format during training: **is it only using huge region + final summary, or huge region + patch + patch description + summary, or other combinations?**\n\n## 2. Model Design\nThe authors adopted the LLaVA-OV model, which can only accept 1008×1008 resolution input. Therefore, the 16000×16000 huge region is downsampled, which raises the \"pseudo zoom-in\" problem: the model performs zoom-in operations on 1008×1008 low-resolution images, which is actually not true zoom-in. Not operating on huge regions is inconsistent with the operators used when instructions construction. The reasonable approach would be to obtain zoom-in coordinates, extract corresponding regions from the huge region, and then input them to the model, rather than operating on downsampled images.\n\n## 3. Experimental Setup and Result Analysis\n### i) PathMMU Experimental Result: \n- **Lack of detailed experimental setup descriptions**, and corresponding PathMMU experimental details are not found in the supplementary materials. CPathAgent has already exceeded Expert performance, and it needs to be clearly stated whether it was trained on the PathMMU training set. If it was indeed trained on the training set, the results are acceptable, but this is unfair to other comparison models. \n-  The authors briefly state that performance gains come from \"multi-scale and reasoning-focused data\" (line 258), but lack corresponding ablation experiments to support this claim. **Are the performance gains from patch and patch descriptions (as shown in Figure 2) or from reasoning-focused instruction data?**  Please provide detailed analysis and experiments, as this question has important guiding significance for subsequent research.\n### ii) WSI Classification Result:\n- Lack of training method descriptions for WSI classification tasks, and relevant experimental details are not found in supplementary materials C.2. The MIL method results are conventional, but the evaluation method for MLLMs is unclear. **Need to clearly specify key information such as input image, resolution and prompt design**.\n### iii) PathMMU-HR² Benchmark:\n- This is a super-resolution evaluation dataset, but CPathAgent cannot directly process super-resolution images, and input images will be resized to 1008×1008. Under these circumstances, **do the evaluation dataset images only need to be 1008×1008? **Since CPathAgent achieves 88.6 performance at 1008×1008 resolution. **Can I consider this not strictly a huge region benchmark?**\n\n- **The results showing CPathAgent far exceeding Gemini-2.5-Pro need detailed analysis**, which would bring greater value to readers than simply presenting data\n\n## Comment\n(To authors) I greatly appreciate the amount of work involved and am very interested in the entire work. According to the review principles, I have carefully completed the review and submitted key questions, and I look forward to the authors' response. I will further consider the scoring based on the authors' reply. (To AC) If needed, I am also willing to discuss with the AC.", "limitations": "See above questions.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper involves multiple core components including instruction dataset construction, model design, VQA benchmark, and experiments proven, but the introduction of each part is not clear enough. Although considering the limitations of the main text length, this causes difficulties for readers to understand. After repeatedly reading the main text and supplementary materials, I still have many confusions. See the question for details.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your comprehensive review and constructive feedback, and we are truly grateful for your expressed interest in our work. However, we believe there may be some misunderstandings regarding our methodology and experimental setup. We will address each concern below to provide clarification:\n\n> ### The paper does not clearly specify the specific input format during training: \n\nThank you for this clarification request. Our training input format for huge regions follows a two-part data structure:\n\n**Part 1 - Navigation Planning:**\n\n- **Input**: Huge region image (16,000×16,000 downsampled to 1,008×1,008)\n- **Output**: JSON-formatted navigation path with magnification levels and coordinates\n\nExample:\n\n```json\n[{\n  \"step\": 1,\n  \"magnification\": 2.5,\n  \"region_coordinate\": [0.4, 0.4, 0.9, 0.9],\n  \"reasoning\": \"Initial scan at medium power to get...\",\n  \"need_to_see\": \"General tissue architecture...\"\n},\n{\n  \"step\": 2,\n  ....\n},\n.....]\n```\n\n**Part 2 - Multi-scale Sequential Reasoning:**\n\n- **Input**: Sequential images [Image1@2.5x, Image2@3.5x, Image3@5.0x, ...] with their metadata \n- **Output**: Integrated diagnostic reasoning across all views\n\nExample output:\n\n> \"Starting with the overall view at 2.5x ([0.4, 0.4, 0.9, 0.9]): The left border shows normal epithelium transitioning to dysplastic tissue. Moving to 3.5x ([0.6, 0.6, 0.95, 0.95]) at the transition zone, I observe ...\"\n\n**Both parts are trained together.** This enables CPathAgent to learn both **strategic planning** (where to examine) and **diagnostic reasoning** (interpreting sequence observations). The model **never receives pre-generated patch descriptions during training** , it learns to generate all reasoning directly from visual inputs. Examples are provided in Appendix Figures D.5-D.7.\n\n\n> ### The authors adopted the LLaVA-OV model, which can only accept 1008×1008 resolution input. Therefore, the 16000×16000 huge region is downsampled, which raises the \"pseudo zoom-in\" problem: the model performs zoom-in on 1008×1008 low-resolution images, which is actually not true zoom-in. The reasonable approach would be to obtain zoom-in coordinates, extract corresponding regions from the huge region, and then input them to the model, rather than operating on downsampled images.\n\nThank you for raising this concern.  **We do NOT operate on downsampled images for actual analysis.** Instead, we implement exactly as you mentioned '**obtain zoom-in coordinates, extract corresponding regions from the huge region, and then input them to the model**'\n\n##### Our Actual Workflow\n\n**1. Planning Phase:** The 16,000×16,000 huge region is downsampled to 1,008×1,008 solely for navigation planning. \n\n**2. Execution Phase:** Using the planned coordinates, we return to the **ORIGINAL 16,000×16,000 huge region** and crop patches at native resolution. These high-resolution patches are then provided as sequential multi-image inputs to the model.\n\n**Example:**\n\n- Planning output: \"Examine region [0.7, 0.7, 0.95, 0.95] at 3.5x magnification\"\n- Execution: Extract pixels [11,200, 11,200, 15,200, 15,200] from the **original 16,000×16,000 image**\n\nThis  ensures that while navigation planning uses a downsampled overview for efficiency, all diagnostic analysis operates on **native-resolution patches cropped directly from the original resolution regions**. \n\n\n\n> ### Lack of detailed experimental setup descriptions, and corresponding PathMMU experimental details are not found in the supplementary materials. CPathAgent has already exceeded Expert performance, and it needs to be clearly stated whether it was trained on the PathMMU training set. \n\nThank you for raising this important point. We clarify that **CPathAgent was only evaluated zero-shot on PathMMU**.\n\n**Why CPathAgent exceeds expert performance:**\n\n1. **Domain specialization gap**: Individual pathologists typically specialize in 1-2 disease areas, while PathMMU spans the entire pathology spectrum. This creates an inherent disadvantage for human experts on out-of-specialty cases, whereas LMMs can learn across all disease types.\n2. **Known LMM  shortcuts**: The PathMMU authors found  their benchmark contains text-based shortcuts that LMMs can exploit. To address this, they created PathMMU-Pro with more rigorous data cleaning.\n\nTo verify this, we conduct **additional experiments on PathMMU-Pro:**\n\n|Model|Overall Tiny|Overall All|PubMed Tiny|PubMed All|SocialPath Tiny|SocialPath All|EduContent Tiny|EduContent All|\n|-|-|-|-|-|-|-|-|-|\n|Human|69.4|-|71.2|-|70.1|-|66.9|-|\n|CPath-Omni|62.1|61.8|60.5|60.8|58.8|62.1|66.7|63.0|\n|CPathAgent|66.3|65.3|64.6|65.8|65.4|63.6|68.7|66.2|\n\nOn this cleaner benchmark, CPathAgent **no longer exceeds human performance** but  still significantly outperforms CPath-Omni.\n\n\n> ### Are the performance gains from patch and patch descriptions (as shown in Figure 2) or from reasoning-focused instruction data? Please provide detailed analysis and experiments.\n\nThank you for this insightful question. The performance gains come from reasoning-focused instruction data, not patch descriptions. \n\n**In fact, the comparison between CPathAgent and CPath-Omni inherently serves as the ablation experiments**. As both models share identical Stage 1 & Stage 2 (patch-level perception) training data. They differ **only** in Stage 3:\n\n- **CPath-Omni**: Outputs only final diagnostic results\n- **CPathAgent**: Trained with **reasoning-focused** instruction data including:\n  - Rationale for zoom/navigation actions\n  - Step-by-step diagnostic feature identification  \n  - Multi-scale multi-image reasoning chains\n\n**Clarification on Figure 2:**  Figure 2 shows our data generation pipeline, not inference (shown in Figure 1). The patch descriptions serve **solely** as auxiliary information to help Gemini-2.5-Pro produce more accurate training data for CPathAgent.\n\nTo further validate CPathAgent's advantages, we  conducted additional experiments to compare CPathAgent and CPath-Omni on two datasets:\n\n**PathMMU-HR²:**\n\n|Model|BRCA|LUAD|LUSC|KIRP|KIRC|KICH|ESCA|THCA|BLCA|TGCT|**Overall**|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|CPath-Omni|72.6|77.6|71.3|64.1|67.5|59.7|74.3|71.5|76.6|78.8|71.7|\n|**CPathAgent**|**87.0**|**88.5**|**87.8**|**87.9**|**92.9**|**78.9**|**90.7**|**89.0**|**90.7**|**93.0**|**88.6**|\n\n**BRACS (Held-out High-Resolution Dataset for breast carcinoma classification):**\n\n|Model|balanced acc|\n|-|-|\n|GPT-4.1|44.2%|\n|Gemini-2.5-Pro|49.0%|\n|CPath-Omni|57.9%|\n|**CPathAgent**|**64.3%**|\n\nThe **16.9%** improvement on PathMMU-HR² and **6.4%** improvement on the independent BRACS dataset demonstrate that performance gains stem from our pathologist-like reasoning approach in Stage 3.\n\n\n> ### Lack of training method descriptions for WSI classification tasks, and relevant experimental details are not found in supplementary materials C.2. The MIL method results are conventional, but the evaluation method for MLLMs is unclear. Need to clearly specify key information such as input image, resolution and prompt design.\n\nThank you for highlighting this important omission. We adopt the same training methodology and MLLM evaluation implementation as CPath-Omni to ensure fair comparison. We will add these details in the revised paper.\n\n\n> ### This is a super-resolution evaluation dataset, but CPathAgent cannot directly process super-resolution images, and input images will be resized to 1008×1008. Under these circumstances, **do the evaluation dataset images only need to be 1008×1008? **Since CPathAgent achieves 88.6 performance at 1008×1008 resolution. **Can I consider this not strictly a huge region benchmark?**\n\nThank you for raising this important question. I understand your concern, but there's a key misunderstanding about how CPathAgent processes huge regions. **CPathAgent DOES process super-resolution images, it analyzes multiple original-resolution sub-cropped regions, not a single downsampled image.**\n\nHere's how it works (as shown in Figure 1 and detailed in Section 3.1):\n\n**1. Navigation Planning Phase:**\n\n- Input: 16000×16000 huge region → temporarily downsampled to 1008×1008\n- Purpose: Only to plan WHERE to look (like a pathologist viewing at low magnification)\n- Output: Navigation path with coordinates and magnifications\n\n**2. Multi-scale Diagnostic Reasoning Phase:**\n\n- Returns to the **ORIGINAL 16000×16000 image**\n- Crops multiple regions based on planned path at **native resolution** (e.g., 1000×1000 pixels from the **original** image)\n- Reasoning across ~10 different views sequentially\n\n**The 88.6% accuracy comes from analyzing multiple native-resolution patches across the huge region, NOT from a downsampled 1008×1008 image. **In addition, as shown in Table 2, when other models adopt our agent-based approach, they gain ~3% on average (Gemini-2.5-Pro: 73.2%→76.4%, GPT-4.1-mini: 60.1%→62.3%). This confirms PathMMU-HR² is indeed a huge region benchmark requiring multi-view analysis.\n\n\n>  ### The results showing CPathAgent far exceeding Gemini-2.5-Pro need detailed analysis**, which would bring greater value to readers than simply presenting data.\n\nThis is a very good question. The reason lies in the training data generation, Gemini-2.5-Pro had access to **ground truth WSI reports as guidance**, enabling it to produce significantly higher quality navigation paths and diagnostic reasoning **than its zero-shot capabilities**. CPathAgent, trained on this high-quality data, can therefore surpass Gemini-2.5-Pro's zero-shot performance.\n\n**Quantitative Evidence** for WSI classification (Table 3):\n\n- **Upper Bound** (synthetic training data generated by Gemini-2.5-pro): 91.7%\n- **Zero-shot Gemini-2.5-Pro**: 72.1%\n- **CPathAgent**: 82.8%\n\nThe 19.6% gap between **upper bound and zero-shot Gemini-2.5-Pro** quantifies how WSI report guidance dramatically improves data quality. CPathAgent successfully distills this guided knowledge, outperforming zero-shot Gemini by 10.7%."}]

Author rebuttal:

Meta-review: This paper proposes CPathAgent, an agentic pipeline that plans where to zoom and explains its reasoning, similar to how a pathologist reviews slides. They also created a new huge-region benchmark. The paper reports gains over recent LMMs and competitive WSI results, with pathologist-readable traces.

Strengths: Their clinically aligned agent workflow with transparent navigation/reasoning is novel and interesting. The huge-region benchmark could be very useful for others to build upon, which the authors say they will release in the NeurIPS checklist (hopefully without guardrails and made easily downloadable).

Weaknesses: Training includes TCGA while evaluation also uses TCGA, so several results are in-distribution; this must be disclosed and quantified (patient-level non-overlap, cohort counts), with stronger out-of-distribution emphasis. Statistical rigor is thin. Human comparisons may be structurally biased because AI can exhaustively scan every high-mag tile, unlike time-limited pathologists. In other words, AI can compensate for the inherent weaknesses of the human visual system, which is unable to analyze every region at the highest possible resolution (it just would be too tedious and time consuming). I'm concerned that this method may be prone to missing smaller tumors, especially for out-of-distribution datasets. See Table 4 of this paper, for an example analysis looking at performance across different region sizes: https://arxiv.org/pdf/2412.02012v3

Discussion and Rebuttal: Reviewers converged to borderline-accept after authors added some out-of-distribution results. However, concerns about TCGA train/eval coupling, missing statistics, and clarity of the navigation policy persist. I weigh the new evidence positively but still have some concerns as described in weaknesses.

Decision: The novel agentic approach and a useful dataset justify a poster. Camera-ready must: (1) clearly indicate in the paper the issues with the train/eval set, and (2) move key out-of-distribution results and results provided in the discussion period into the main paper.
Decision: Accept (poster)