Title: CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
Venue: NeurIPS 2025
Accepted: yes
Keywords: Agent

Computational pathology

Large multimodal model
Abstract: Recent advances in computational pathology have led to the emergence of numerous foundation models. These models typically rely on general-purpose encoders with multi-instance learning for whole slide image (WSI) classification or apply multimodal approaches to generate reports directly from images. However, these models cannot emulate the diagnostic approach of pathologists, who systematically examine slides at low magnification to obtain an overview before progressively zooming in on suspicious regions to formulate comprehensive diagnoses. Instead, existing models directly output final diagnoses without revealing the underlying reasoning process.
To address this gap, we introduce CPathAgent, an innovative agent-based approach that mimics pathologists' diagnostic workflow by autonomously navigating across WSI through zoom-in/out and move operations based on observed visual features, thereby generating substantially more transparent and interpretable diagnostic summaries. To achieve this, we develop a multi-stage training strategy that unifies patch-level, region-level, and WSI-level capabilities within a single model, which is essential for replicating how pathologists understand and reason across diverse image scales.
Additionally, we construct PathMMU-HR², the first expert-validated benchmark for large region analysis. This represents a critical intermediate scale between patches and whole slides, reflecting a key clinical reality where pathologists typically examine several key large regions rather than entire slides at once. Extensive experiments demonstrate that CPathAgent consistently outperforms existing approaches across benchmarks at three different image scales, validating the effectiveness of our agent-based diagnostic approach and highlighting a promising direction for computational pathology.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multimodal large language model (MLLM) for pathological image huge regions, with a design approach that shows certain innovation. The authors identify the limitations of existing methods in pathological image analysis: patch-based methods are not clinically applicable, while WSI-based methods exceed the processing capabilities of MLLMs. Therefore, designing huge regions that fall between these two approaches is a meaningful attempt.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper involves multiple core components including instruction dataset construction, model design, VQA benchmark, and experiments proven, but the introduction of each part is not clear enough. Although considering the limitations of the main text length, this causes difficulties for readers to understand. After repeatedly reading the main text and supplementary materials, I still have many confusions. See the question for details.\n\nQUESTIONS:\n## 1. Instruction Dataset Construction\nFigure 2 shows the instruction dataset construction process, where the key step is generating patch descriptions for each specific patch based on huge region descriptions. This process is reasonable under zoom-in operations, as each patch remains clear for the Gemini model. \n\nHowever, the paper does not clearly specify the specific input format during training: **is it only using huge region + final summary, or huge region + patch + patch description + summary, or other combinations?**\n\n## 2. Model Design\nThe authors adopted the LLaVA-OV model, which can only accept 1008×1008 resolution input. Therefore, the 16000×16000 huge region is downsampled, which raises the \"pseudo zoom-in\" problem: the model performs zoom-in operations on 1008×1008 low-resolution images, which is actually not true zoom-in. Not operating on huge regions is inconsistent with the operators used when instructions construction. The reasonable approach would be to obtain zoom-in coordinates, extract corresponding regions from the huge region, and then input them to the model, rather than operating on downsampled images.\n\n## 3. Experimental Setup and Result Analysis\n### i) PathMMU Experimental Result: \n- **Lack of detailed experimental setup descriptions**, and corresponding PathMMU experimental details are not found in the supplementary materials. CPathAgent has already exceeded Expert performance, and it needs to be clearly stated whether it was trained on the PathMMU training set. If it was indeed trained on the training set, the results are acceptable, but this is unfair to other comparison models. \n-  The authors briefly state that performance gains come from \"multi-scale and reasoning-focused data\" (line 258), but lack corresponding ablation experiments to support this claim. **Are the performance gains from patch and patch descriptions (as shown in Figure 2) or from reasoning-focused instruction data?**  Please provide detailed analysis and experiments, as this question has important guiding significance for subsequent research.\n### ii) WSI Classification Result:\n- Lack of training method descriptions for WSI classification tasks, and relevant experimental details are not found in supplementary materials C.2. The MIL method results are conventional, but the evaluation method for MLLMs is unclear. **Need to clearly specify key information such as input image, resolution and prompt design**.\n### iii) PathMMU-HR² Benchmark:\n- This is a super-resolution evaluation dataset, but CPathAgent cannot directly process super-resolution images, and input images will be resized to 1008×1008. Under these circumstances, **do the evaluation dataset images only need to be 1008×1008? **Since CPathAgent achieves 88.6 performance at 1008×1008 resolution. **Can I consider this not strictly a huge region benchmark?**\n\n- **The results showing CPathAgent far exceeding Gemini-2.5-Pro need detailed analysis**, which would bring greater value to readers than simply presenting data\n\n## Comment\n(To authors) I greatly appreciate the amount of work involved and am very interested in the entire work. According to the review principles, I have carefully completed the review and submitted key questions, and I look forward to the authors' response. I will further consider the scoring based on the authors' reply. (To AC) If needed, I am also willing to discuss with the AC.\n\nLIMITATIONS:\nSee above questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTraditional computational pathology models lack the multi-scale reasoning process employed by pathologists during diagnosis, resulting in limited interpretability and misalignment with real-world clinical workflows. This paper proposes CPathAgent, an innovative agent-based model designed for high-resolution pathology image analysis that mimics the diagnostic logic of expert pathologists. CPathAgent dynamically performs zooming and navigation operations, and incorporates a multi-stage training strategy to achieve interpretable analysis. In addition, the study introduces PathMMU-HR², the first benchmark dataset for huge region reasoning, covering diverse tissue types and pathological features. The method proposed in this paper achieves better results than the SOTA methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper enhances the performance of current Pathology Language-Vision Models by simulating the diagnostic process of human pathologists reading pathology slides, demonstrating that mimicking this missing process can improve diagnostic accuracy.\n2. It constructs a dataset specifically designed for training pathology model agents.\n3. It constructs a dataset for analyzing huge region pathology images.\n\nWeaknesses:\n1. I think this paper places too much emphasis on dataset construction during the writing process, which leads to an oversimplified and weakened description of the agent’s architecture, training process, and reasoning mechanism. This may give other researchers the impression that the paper is primarily about dataset track rather than method innovation.\n2. Many of the training steps in this paper overlap with CPath-Omni. However, the differences from CPath are not sufficiently highlighted, and the paper should better emphasize its innovations.\n3. The dataset used for training the navigation module is mainly generated by large models. It remains unclear whether the generated navigation paths align with the actual behaviors of human pathologists when reading pathology slides. This needs further validation through experiments or expert analysis.\n4. The paper lacks an analysis of the functional contributions of different agent components; specifically, it does not include ablation studies on individual modules, making it difficult to assess their respective impacts on overall performance.\n5. Compared with CPath-Omni, the proposed method requires multiple invocations of VLMs, yet the paper does not provide a comparative analysis in terms of time efficiency and computational cost.\n6. Will the dataset constructed in this paper be open-sourced?\n\nQUESTIONS:\nPlease refer to the “Weaknesses” section.\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CPathAgent, an agent-based framework for pathology image analysis that mimics pathologists' diagnostic workflow. The paper defines an agent as: a system capable of autonomously executing zoom-in/out and navigation operations across pathology images, achieving specific goals through perceiving the environment, making decisions, and taking actions. Specifically, CPathAgent emulates pathologists' diagnostic process through three stages: (1) Global Screening stage to identify suspicious regions; (2) Navigation Planning stage to formulate examination paths; (3) Multi-scale Multi-view Sequence Reasoning stage to integrate observations across different magnification levels. The system can mimic pathologists by starting with low-magnification scanning, progressively zooming in on regions of interest, and formulating diagnostic conclusions through dynamic visual reasoning processes. This paper also proposed the 'PathMMU-HR' dataset.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Clinical Relevance\nSuccessfully addresses the gap between AI systems and actual pathologist workflows by mimicking their multi-scale examination process, potentially improving interpretability and clinical adoption.\n2. Novel Benchmark (PathMMU-HR²)\nIntroduces the first benchmark for huge region analysis (16000×16000 pixels) - a critical but previously overlooked scale in pathology. Validated by three board-certified pathologists.\n3. Comprehensive Evaluation\nEvaluates across three scales (patch, huge region, WSI) on multiple datasets, showing consistent improvements and versatility of the approach.\n4. Pathologist Validation\nIncludes real pathologist evaluation of navigation paths and reasoning quality, adding clinical credibility beyond just benchmark metrics.\n\nWeakness:\n1.Novelty Issues with Agent Definition\nThe paper claims to be \"the first agent-based framework for pathology,\" but it essentially repackages traditional multi-scale image analysis as an agent concept. The authors should compare their work with existing popular agent frameworks and emphasize the rationality of their proposed pathology-specific agent definition. The so-called \"agent behaviors\" (zooming, navigation) are fundamentally predefined image cropping and scale selection, lacking genuine autonomous decision-making capabilities. Compared to existing attention mechanisms or multi-scale feature fusion methods, the conceptual innovation is limited.\n\n2.Reliability of Data Generation\n\nThe work heavily relies on Gemini-2.5-Pro to generate training data (278K instances) - how is the quality of such synthetic data ensured? Do the generated navigation paths and diagnostic reasoning genuinely reflect general pathologists' actual workflows? There is a lack of comparative validation against real pathologist behavioral data.\n\n3.Fairness of Evaluation\nIn the PathMMU-HR² benchmark, only CPathAgent uses the agent-based approach, while other models do not employ the same prompting strategy. Part of the performance improvement may stem from the inherited CPath-Omni training data rather than the agent mechanism itself. The pass@k evaluation uses only 40 samples, which lacks statistical significance.\n\n4.Ambiguity in Technical Implementation\nHow is \"navigation planning\" actually implemented? Is it merely selecting image patches according to predefined rules? The paper does not clearly explain how the agent \"decides\" when to stop exploration.There is a lack of analysis on computational complexity and inference time.\n\nQUESTIONS:\n1. How do your proposed AI agent adapt the newly incomed training data? \n2. Decision-making mechanism entirely opaque: During training, Gemini generates navigation paths based on WSI reports, but during inference without such reports, how does the model determine where to examine next?\n3. Bridging mechanism unexplained: Does the model memorize navigation patterns or genuinely learn visual reasoning? Generalization capability questionable: If the system merely memorizes training navigation patterns, how does it handle novel pathological presentations?\n4. Generalization capability questionable: If the system merely memorizes training navigation patterns, how does it handle novel pathological presentations?\n5. Base model unspecified: The paper merely mentions \"open-source VLMs\" without detailing the specific architecture.\n\nLIMITATIONS:\nNo assessment of the error accommulation. This paper proposed the method contains three agents. Each agent can make mistake. The author did not provide the detailed information about the performance when error may accommulates. For instance, the first stage 'Global screening and region selection' made a mistake(such as filtered a important tile), what will happend at the end? Backtracking mechanism: Can the system return to previously examined locations for re-evaluation? If can not, if would be interest for future research. Lack of interactivity: Real pathologists adjust staining, section angles based on preliminary findings - this system cannot.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents CPathAgent, an extension of CPath-Omni that adds agent-based sequential processing of whole slide pathology images, requiring the model to move around the image, zooming in and out at different resolutions, to produce region/slide descriptions and answer questions.\n\nSTRENGTHS AND WEAKNESSES:\nSTRENGTHS\n\n1. Mimicking pathologists' navigation through whole slide images is an interesting way to extend region-only models to whole slide images. The navigation examples provided are compelling, suggesting that this navigation makes sense.\n\n2. A pathologist-validated visual QA benchmark dataset is proposed (I imagine the data will be released to the community). The dataset excludes questions that get answered (trivially) correctly without an image.\n\n3. CPathAgent appears to outperform CPathOmni on the PathMMU dataset (however, it is suspicious that CPathAgent also substantially outperforms expert pathologists).\n\n\n\n\nWEAKNESSES\n\n1. CPathAgent appears to be pretrained on the \"training sets\" of most of the evaluation tasks, to solve these tasks. It is unclear whether the model generalizes to new data, so there is a need for evaluations that are distinct from the training data.\n\n1a. CPath-Omni was pretrained to solve the TCGA classification tasks for NSCLC, BRCA, UCEC, THCA, ESCA, BLCA, and TGCT and it appears CPathAgent may be as well: \"training data inherited from CPath-Omni\". This makes TCGA WSI classification not actually zero-shot and it performs similary to MIL aggregators trained on each of these tasks.\n\n1b. PathMMU-HR2 is developed using the same method / tools as the training data for CPathAgent, so I expect CPathAgent to have an advantage on it.\n\n2. While I imagine that the proposed agentic approach may improve model performance, I am not convinced this is the case by the presented results.\n\n2a. To support the claim that the agentic component of CPathAgent improves over CPathOmni, it would be useful to evaluate CPathOmni on all tasks but it's only evaluated on PathMMU, where CPathOmni is close to expert human performance and CPathAgent somehow greatly improves over pathologists (which is weird). Can CPathOmni be evaluated on PathMMU-HR2 and the TCGA WSI classification tasks as well? Does CPathAgent outperform it there?\n\n2b. Greatly exceeding expert pathologist performance on PathMMU makes the quality of this dataset suspicious. Is the QA frequently wrong? The PathCLS subset of PathMMU was derived from labeled datasets and may be the cleanest and most correctly annotated subset of PathMMU. On this subset, there is no difference in performance between CPathOmni (79.0) and CPathAgent (79.2).\n\n2c. Why are CPathOmni results not shown in Table 3 (TCGA WSI classification)? Looking at the CPathOmni paper, it appears that there may not be much difference between CPathAgent and CPathOmni on these tasks, after all: TCGA-BRCA 88.5 vs 89.2, TCGA-NSCLC 90.8 vs 88.8, TCGA-RCC 94.6 vs 94.0, TCGA-ESCA 97.1 vs 92.1, TCGA-BLCA 62.7 vs 70.7, TCGA-THCA 63.2 vs 58.5; an average balanced accuracy of 82.8 for CPathAgent and 82.2 for CPathOmni. Did the addition of agentic processing of slides really improve over CPathOmni?\n\nQUESTIONS:\n1. For acceptance, a more complete comparison with CPathOmni would be necessary, along with further evaluation on whole-slide-level pathology tasks outside of the training distribution. Additionally (but less importantly), comparisons with recent foundation models capable of whole-slide representations, like TITAN or PRISM, would be helpful.\n\n2. More discussion/evaluation is needed to motivate agentic sequential processing of pathology images.\n\n2a. It makes sense that different magnifications are sufficient for different information that can be integrated into a question answer or a diagnosis. However, is it necessary to scan around the image sequentially and zoom in and out? Does this have any advantage over a model that takes the full region or whole slide image at multiple scales as input? Also, is there any additional training signal in the sequential setup that cannot be used in the everything-at-once input approach?\n\n2b. You mention that non-agentic models \"fail to replicate the diagnostic process of pathologists\". Could you explain why this is a problem?\nYou mention that non-agentic models have \"poor alignment with clinical workflows, undermining their credibility in diagnostic settings\". Could you explain what you mean by \"poor alignment with clinical workflows\" and why this undermines the credibility of validated model performance?\n\n3. Minor questions.\n3a. When all parameters are unfrozen, does this include CPath-Clip? If so, then all of it (including Virchow2) or part of it?\n3b. How are images passed to Gemini when evaluating it without \"CPathAgent's prompting strategy\"? Do they still cover a mix of magnifications? Does mixing magnifications improve performance or does the prompting strategy improve performance?\n3c. How do differences between models look when computing statistical significance in the tables?\n\nLIMITATIONS:\nSee the evaluation concerns discussed above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-21"}