Title: Sequential Attention-based Sampling for Histopathological Analysis
Venue: NeurIPS 2025
Accepted: yes
Keywords: Deep Learning

Deep Reinforcement Learning

Multiple Instance Learning

Histopathological Analysis

Whole Slide Images
Abstract: Deep neural networks are increasingly applied in automated histopathology. Yet, whole-slide images (WSIs) are often acquired at gigapixel sizes, rendering them computationally infeasible to analyze entirely at high resolution. Diagnostic labels are largely available only at the slide-level, because expert annotation of images at a finer (patch) level is both laborious and expensive. Moreover, regions with diagnostic information typically occupy only a small fraction of the WSI, making it inefficient to examine the entire slide at full resolution.
Here, we propose SASHA -- Sequential Attention-based Sampling for Histopathological Analysis -- a deep reinforcement learning approach for efficient analysis of histopathological images. 
First, SASHA learns informative features with a lightweight hierarchical, attention-based multiple instance learning (MIL) model. 
Second, SASHA samples intelligently and zooms selectively into a small fraction (10-20\%) of high-resolution patches to achieve reliable diagnoses.
We show that SASHA matches state-of-the-art methods that analyze the WSI fully at high resolution, albeit at a fraction of their computational and memory costs. In addition, it significantly outperforms competing, sparse sampling methods. 
We propose SASHA as an intelligent sampling model for medical imaging challenges that involve automated diagnosis with exceptionally large images containing sparsely informative features. Model implementation is available at: https://github.com/coglabiisc/SASHA.


Peer reviews:
comments: SUMMARY:
This paper proposes a new RL technique for a whole-slide classification of histopathology images. The idea is to use RL to select higher magnification patches to save on computation while maintaining classification performance. The novelty of this work is around the feature aggregator that allows the author to combine high magnification feature with low magnification, and to use targeted state updater to select only patches with high similarity for updating the state of RL machine. Experiments show some improvement on two histopathology tasks, and ablation studies demonstrate the importance for each component of the method.

STRENGTHS AND WEAKNESSES:
Strength
- The proposed method works relatively well for the selected task. Achieving 4x faster inference time and comparable accuracy.
- The design is well motivated, and ablation study shows the necessity for each component of the technique.

Weakness
- The tasks selected here are limited to tumor detection-style binary classification. With the complexity of RL, it would be helpful to demonstrate its ability in more complex, multi task classes. One potentially interesting case is breast cancer mitotic activity scoring, which requires the model to detect dividing cells and classify it into three density classes.

QUESTIONS:
N/A

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents SASHA, a reinforcement learning-based framework for the efficient classification of gigapixel whole-slide histopathology images (WSI). Unlike traditional methods, that process the entire high-resolution slide, SASHA learns to adaptively sample a small subset (only 10/20%) of high-resolution patches while still achieving state-of-the-art diagnostic classification accuracy.\n\nThe framework consists of three main components:\n1)Hierarchical Attention-based Feature Distiller (HAFED): a two-stage multi-head attention MIL module that extracts label-aware feature representations from WSI high-resolution patches and aggregates them for slide-level classification.\n2)Targeted State Updater (TSU): a network that maintains a global state of the WSI with a d-dimensional feature vector for each low-resolution patch. When a patch is zoomed in to high resolution, the state is updated selectively only for patches with features correlated to the sampled one.\n3)Deep RL agent: a policy network that, given the current WSI state (low-res features and any updated high-res embeddings), sequentially selects the next patch to zoom, aiming to maximize classification accuracy.\n\nOn CAMELYON16 and TCGA-NSCLC benchmarks, SASHA matches or exceeds full-slide MIL baselines (e.g., ACMIL, DTFD) using only 20 % of patches, reduces inference time by 4–8×, and cuts memory use by 16×. Extensive ablation studies confirm the importance of each component of the framework, and explainability analyses show that the agent focuses on tumor-rich, highly informative regions of the WSI.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1) Real-world motivation: the challenge of computationally efficient yet accurate WSI analysis is central to digital pathology. The paper tackles the inefficiencies of full-resolution WSI analysis and prior sparse sampling methods.\n\n2)Novelty of the architecture: combining label-aware MIL (multi-head HAFED), targeted state propagation (TSU), and on-policy RL into a coherent framework is innovative and well justified.\n\n3)Strong empirical results: Under the same observation budget, SASHA consistently outperforms RLogist by a notable margin (e.g., +9% AUC at 10% sampling). Moreover, SASHA-0.2 achieves near-SOTA performance using only ~20% of high-res patches, and HAFED (100%) is competitive with top full-slide MIL models like DTFD and ACMIL. In addition to its strong performance, the framework offers significant gains in efficiency, with an inference time reduced by up to 8× and memory usage by over 16×.\n\n4)Each component of the framework is well justified: removing any key component such as multi-head attention of HAFED, targeted update mechanism of TSU, or replacing the learned policy with a random one—leads to substantial drops in performance, clearly justifying each design decision.\n\n\nWeaknesses\n1) Training time cost: although the framework is efficient at inference, training remains costly and less scalable. It requires access to all high-resolution patches and involves computing high-res embeddings V(i) for all patches within each correlated set C, as part of TSU’s training. This contradicts the efficiency objective of avoiding exhaustive high-resolution patch processing (l.195), a trade-off that is acknowledged but not extensively examined.\n\n2)Generalization beyond binary classification: the evaluation is limited to binary classification tasks. While this is common in the literature, it remains unclear how SASHA would perform in multi-class or multi-label settings common in real-world pathology.\n\n3)Minor presentation inconsistency: there’s a discrepancy between Section 3.3 and Figure 2b regarding the TSU input. The figure and appendix suggest that S(i), S(a_t), and V(a_t) are concatenated, whereas the text says only S(i) and V(a_t) are used.\n\nQUESTIONS:\n1) TSU input clarification: does the MLP of TSU receive [S(i), S(a_t), V(a_t)] (as per Fig. 2b & Appendix) or only [S(i), V(a_t)] (as per Sec. 3.3)?\n2) Definition of “time fraction”: in Fig. 3a and 3b, the patch boxes are colored according to the “time fraction in episode.” However, the precise meaning of this term is not defined in the text. Is it meant to indicate the normalized timestep at which each patch was visited within an RL episode? Clarifying this would aid interpretability.\n3) Multi-class extension: have you attempted (or can you comment on) applying SASHA to tasks with > 2 labels, such as tumor grading or multi-label subtype classification?\n4) Pretraining sensitivity: In your ablations, replacing the WSI-pretrained ViT with an ImageNet-pretrained ResNet50 caused a substantial drop. Could you discuss how robust is performance if the ViT encoder is pretrained on smaller or out-of-domain datasets?\n\nLIMITATIONS:\nnone relevant", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduces SASHA, a novel deep RL framework designed for the efficient diagnosis of gigapixel-sized whole-slide images (WSIs) in histopathology. SASHA uses an RL agent that intelligently samples a small fraction (10-20%) of informative patches from a low-resolution view of the WSI to \"zoom in\" on for high-resolution analysis. It consists of several key modules: \n\nHierarchical Attention-based Feature Distiller (HAFED): A 2-stage attention model that learns to extract diagnostically relevant features from high-resolution patches for aggregation into both a set-based representation (for updating state via TSU) as well as slide-level representation (for classification).\n\nTargeted State Updater (TSU): An efficient method to update the representation of the WSI. After a patch is selected and analyzed at high resolution, the TSU updates the state of not only the selected patch but also other, un-sampled patches that are highly correlated with it, supervised by their ground truth features during training time.\n\nVia 2 WSI-classification benchmarks, the paper demonstrates that SASHA achieves classification performance comparable to state-of-the-art methods that analyze the entire WSI at high resolution, but does so at a fraction of the computational cost, with up to 8x faster inference times and requiring significantly less memory.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThe method is well thought out and motivated, maximizing the learning signal from multiple resolution while allowing computationally efficient inference after training. \n\nWeakness\nExperiments is lacking with only 2 datasets that are focused on binary diagnosis (e.g. no prognosis tasks or more challenging large-scale classification tasks) and are already near saturation in performance, making it difficult to assess potential performance impact across diverse tasks in computational pathology. The norm in the field for MIL-based methods is usually 5+ benchmarks, in addition to potential few-shot / ablation experiments. \n\nThere's also no comparison with non-RL-based sampling methods such as ZoomMIL.\n\nQUESTIONS:\nI recommend expanding the scope of evaluation to at least 2 - 3 other benchmarks, including more challenging classification tasks (e.g. BRACs or EBRAINS), as well as prognostic tasks (e.g. TCGA or SurGen). Comparison with non-RL based sampling methods such as ZoomMIL should also be added.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Sequential Attention-based Sampling for Histopathological Analysis (SASHA), a deep reinforcement learning (RL) framework that selectively analyzes diagnostically relevant high-resolution regions in gigapixel whole-slide images (WSIs). SASHA achieves performance comparable to state-of-the-art full-resolution models while utilizing only 10–20% of the high-resolution patches.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\n(1) The proposed approach closely mimics practical diagnostic workflows—initial low-resolution scanning followed by selective high-resolution inspection.\n\n (2) The paper is well-written, clearly structured, and easy to follow.\n\n (3) Experimental results on two benchmark datasets demonstrate that the proposed method outperforms existing approaches while using only 10–20% of the high-resolution patches.\n\n (4) Qualitative visualizations further support the effectiveness of the proposed sampling strategy.\n\n**Weaknesses**:\n\n(1) Training Cost: The method requires access to all high-resolution patches during training, which contradicts its low-resource inference objective and significantly increases training time and memory usage.\n\n (2) Inference Time Reporting: While Section 4.3 provides a general analysis of inference efficiency, the paper would benefit from more detailed, side-by-side comparisons of actual inference times across methods.\n\n (3) Clinical Deployment Considerations: Although the approach is promising, the paper lacks discussion on practical deployment challenges in clinical settings, such as interpretability, robustness to failure, and integration into existing diagnostic workflows.\n\n (4) TSU Similarity Threshold (τ): The threshold parameter τ plays a key role in the targeted state update mechanism, yet no sensitivity analysis or justification is provided for its choice.\n\nQUESTIONS:\nSee the Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}