ASMIL: Attention-Stabilized Multiple Instance Learning for Whole-Slide Imaging
Authors: Linfeng Ye
Shayan Mohajer Hamidi
Zhixiang Chi
Guang Li
Mert Pilanci
Takahiro Ogawa
Miki Haseyama
Konstantinos N. Plataniotis
Venue: ICLR 2026 Poster (2026)
Keywords: Whole slide image
Multiple instance learning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Attention-based multiple instance learning (MIL) has emerged as a powerful framework for whole slide image (WSI) diagnosis,  leveraging attention to aggregate instance-level features into bag-level predictions. Despite this success, we find that such methods exhibit a new failure mode: unstable attention dynamics.  Across four representative attention-based MIL methods and two public WSI datasets, we observe that attention distributions oscillate across epochs rather than converging to a consistent pattern, degrading performance. This instability adds to two previously reported challenges: overfitting and over-concentrated attention distribution. To simultaneously overcome these three limitations, we introduce attention-stabilized multiple instance learning (ASMIL), a novel unified framework. ASMIL uses an anchor model to stabilize attention, replaces softmax with a normalized sigmoid function in the anchor to prevent over-concentration, and applies token random dropping to mitigate overfitting. Extensive experiments demonstrate that ASMIL achieves up to a 6.49% F1 score improvement over state-of-the-art methods. Moreover, integrating the anchor model and normalized sigmoid into existing attention-based MIL methods consistently boosts their performance, with F1 score gains up to 10.73%. All code and data are publicly available at https://anonymous.4open.science/r/ASMIL-5018/.

Published as a conference paper at ICLR 2026
ASMIL: AFTENTION-STABILIZED MULTIPLE IN- STANCE LEARNING FOR WHOLE SLIDE IMAGING
Linfeng Ye', Shayan Mohajer Hamidi", Zhixiang Chi', Guang Li', Mert Pilanci', Takahiro Ogawa', Miki Haseyama', Konstantinos N. Plataniotis Uniiersityy of Toronto, 2Stanford University. 'Hokkaido University
'[linfeng. ye, zhixiang shi) utoronto.ca "kostas@ece. utt.oeto.roootoo.oommmmm
2{ (smoha jenhaaer,, pil pilanci pimolaaerrpillaanceeetteenoo festtaforddd..ooo edu
(guang, ogawa, mmaseyama lmd. ist www.hoo. ac. jp.
ABSTRACT
Attention-based multiple instance learning (MIL.) has emerged powerful framework for whole slide image (WSI) diagnosis, leveraging attention to ag- gregate instance-level features into bag-level predictions. Despite this success, we find that such methods exhibit a new failure mode: unstable attention dynam- ics. Across four representative attention-based MILL methods and two public WSI datasets, we observe that attention distributions oscillate across epochs rather than converging consistent pattern, degrading performance. This instability adds to two previously reported challenges: overfitting and over-concentrated attention distribution. To simultaneously overcome these three limitations, introduce attention-stabilizee multiple instance learning (ASMIL), a novel unified frame- work. ASMIL uses an anchor model stabilize attention, replaces softmax with a normalized sigmoid function in the anchor to prevent over-concentration, and applies token random dropping to mitigate overfitting. Extensive experiments demonstrate that ASMIL achieves up to a 6.49% Fl score improvement over state-of-the-art methods. Moreover, integrating the anchor model and normal- ized sigmoid into existing attention-basse MIL. methods consistently boosts their performance, with score gains up to 10.73%. All code and data are publicly available at littt::: /tttt::://gttubb www.huu.oooo..aoooooooooo....eee Linfeng- Ye/ASMIL
INTRODUCTION
Computational pathology, at the intersection of digital imaging, machine learning, and clinical di- agnostics, has transformed modern workflows (Verghese al. 2023). Advances in whole slide imaging (WSI) now allow glass slides to be digitized into gigapixel images (Bacus) (001), which are central to cancer diagnosis and treatment planning. WSIs preserve rich spatial context and en- able large-scale sharing, but their extreme size and sparsity create major challenges: diagnostically relevant regions often occupy only tiny fraction of the slide, and exhaustive pixel- or tile-level an- notations are infeasible in practice. As result, most datasets provide only weak slide-level labels, making it critical to design methods that learn effectively under weak supervision.
This weakly supervised setting naturally motivates multiple instance learning (MIL) Keelerr et al. 1990 Dietterich et al. 1997 Maron & Pozano-Peérez 1998). MIL. a bag of instances is mapped to single bag-level label. For WSIs, the image is divided into tiles, each treated instance, while only the slide-level label is required. This dramatically reduces annotation costs and makes large-scale WSI datasets more practical for research and clinical use.
Early approaches to MIL-based WSI analysis focused on simple aggregation strategies, such as clus- tering instance features (Xu al. 2014) applying global pooling layers (Kraus et al. (016). A major breakthrough came with the introduction of attention-based MIL (ABMIL) (Ilse et al. 2018). which provided theoretical guidance for neural network- based MIL algorithms and introduced
"Correspondence author.

---

Published as a conference paper at ICLR 2026
permutation invariian attention mechanism aggregate instance information into bag-level repre- sentations. ABMIL established a strong baseline for WSI analysis (Shao et al. (025) and, impor- tantly, enhanced interpretability through visualized attention scores, which is an essential property for clinical adoption. Building on this foundation, subsequent works have refined A ABMIL to fur- ther improve performance, scalability, and robustness (Xiong al., [202]] [Shao et al.) 2021 Zhang iet al., [2022] Tang et al. 22023b] Zhang et al., (2024). In particular. TransMIL replaces independent instance weighting with a transformer encoder that explicitly models inter-instance relations within a bag (Shao et al. 2021). As a result, attention- ditention-based MIL has become the de facto choice for WSI subtyping not only because it aggregates instance features but also because its attention maps are used as clinical evidence of model interpretability.
Despite its success, attention- based MIL. still suffers from three major problems, which we denote as (PI), (PIT), and (PIII,, and elaborate on in the sequel.
critical yet underexplored aspect of MIL-based WSI analysis is the convergence behavior of at- tention mechanisms during training. The gigapixel scale of WSIs, coupled with weak supervision, high variability, and sparsity, makes it difficult for models to consistently identify informative tiles among thousands of candidates. Our investigation reveals that existing MIL algorithms often fail to converge stably on WSI datasets. To the best of our knowledge, we are the first to identify and systematically analyze (PI) unstable attention dynamics, where attention distributions for individual WSIs oscillate substantially across epochs instead of converging into consistent patterns. To quantify
this phenomenon, measure the Jensen-Shannon divergence (Cover 1999) between consecutive
attention distributions of the same WSI, as illustrated for TransMLL (Shao et al. (021) in Figure
Additional experiments across methods and datasets are provided in Appendix This persistent oscillation results in unstable training and degraded performance, reflected in higher cross-entropy values compared to our proposed method.
Beyond this new limitation identified in our study, prior work has highlighted two additional chal- lenges. One is (PII) over-concentrated attention distribution (Zhang et al., [024] Lu et al. [021). where models allocate excessive importance to only a few tiles, thereby harming generalization and interpretabiliity. The other (PIII) overfitting (Zhang et al., 2022 et al., 2023). common issue in histopathology WSI classification caused by the limited number of available training samples.
In this paper, we aim to simultaneously address the challenges (PI)(PIII), To stabilize attention distribution and the training process, we introduce an anchor model, which has the same archi- tecture as the online model's attention module and receives the same input, but is updated via an exponential moving average (EMA) instead of by backpropagation. Acting as stable reference, the anchor provides smoother and more consistent attention distributions. To transfer this stability, encourage the online model to mimic the anchor by minimizing the Kullback-Lebler (KL) di- vergence between their attention distributions. To mitigate over-concentration, which we attribute the exponential sensitivity of the softmax function, we replace softmax in the anchor branch with a normalized sigmoid function (NSF), as defined in Equation (5). Finally, we propose a simple yet effective token dropout strategy that regularizes the model and reduces overfitting. Together with the anchor model, these components form a unified framework called attention-stabile multiple instance learning (ASMIL), which improves both the stability and generalization of bil--based WSI analysis.
In summary, this paper's 's contributions are as follows:
We are the first to identify and systematically analyze the problem of unstable attention dynamics in attention-based MIL for WSI analysis. This overlooked issue not only limits predictive perfor- mance but also undermines interpretability, since fluctuating attention distributions prevent consis- tent identification of the tissue regions that drive the model's decisions.
To overcome this instability, we introduce an anchor model that stabilizes attention distribution throughout training. The anchor model is updated using an exponential moving average of the online model, which ensures stable training dynamics and improves both performance and interpretability.
We show mathematically that replacing softmax with an NSF alleviates attention over- concentration. Since applying the NSF to the online model causes vanishing gradients, apply it to the anchor model instead, ensuring stable and well-distributed attention.
2

---

Published as a conference paper at ICLR 2026
To mitigate overfitting, introduce token dropout, which randomly discards portion of feature tokens during training while retaining all tokens during inference.
By integrating these innovations, present iriention-stabilizee MIL (ASMIL), novel MIL- based WSI analysis algorithm. Through comprehensive experiments on multiple public WSI datasets, demonstrate that ASMIL achieves state-of-the-arr performance in subtyping and lo- calization tasks.
Paper Organization. The remainder of this paper structured follows: Section reviews related work on MIL and attention mechanisms in WSI analysis; Section presents the preliminaries and motivation of approach; Section 3 details the ASMIL framework; Section Section|] the experi- mental setup and results; and finally Section Iontion&@cnnci concludes the paper with future research directions.
TransMill.
10
TransMIL
ASMIL (ours)
CE
0.6
JSD JSD (o (
0.4
0.
0.2
-05
0.0
20
epochs
30
50
ASMIL (Ouns)
Figure 1: Visualization of attention dynamics on a tumor WSI for TransMIL. (Shao et al. 2021 vs.
ASMIL (our method). The green contours in the figures indicate the annotated tumor regions. Top:
TransMIL attention distribution at selected training iterations. Middle: Jensen-Shanner divergence (ISD) between attention distributions at successive steps and the cross entropy loss (CE), comparing TransMIL (blue) and ASMIL (red). Bottom: Attention distribution from ASMIL over different training iterations. Due the weakly supervised nature of WSI subtyping datasets, TransMIL's attention patterns never converge during training. further, it focuses on only a subset of cancerous regions. contrast, our method (i) produces stable attention distributions throughout training and (ii) consistently highlights cancerous regions.
2 RELATED WORK
Early weakly supervised approaches in computational pathology leveraged multi-view convolutional neural network ensembles and basic MIL pooling to transition from patch-level labels to slide-level predictions (Das al. 2017 2018). As datasets scaled and slide-level supervision became the norm, methods shifted from fixed pooling to attention mechanisms that make aggregation learnable. Building on this trend, attention based MII. (Ilse et al., (018)) introduced learnable instance weights and generated heatmaps from slide-level labels, achieving breast and colon cancer classification on par with fully supervised methods at scale. Complementary to weighting instances, subsequent work reduced morphological redundancy in tile representations, (Sont et al. (2024) used a Gaussian mixture model, and sped up inference by skipping irrelevant patches (Dong et al. 2025). Li et al. (2021b) propose DSMIL, dual-stream MIL framework that selects critical instance via max- pooling and then applies a trainable localocal, distance-based attention from this instance to all others to form bag embeddings for WSI classification. Subsequent works extend this line of research

---

Published as a conference paper at ICLR 2026
by leveraging multi-scale fusion to aggregate information across resolutions Zhang al.
2021
Guo
ct al. 2023 Tran et al. 2025: Buzzard et al. 2024 Li et al. 2019).
Several works further refine training strategies for attention-basee MIL. To prevent the attention distribution from collapsing onto a few input patches and to obtain more faithful attention maps, Zhang et al. (2024) stochastically masks the top-K instances, while ZZhag et al. (2025b) adds an entropy regularization term that explicitly flattens the attention distribution. In a complementary direction, Fourkioti al. (2024) introduces neighbor-constrained attention to suppress noise in the feature maps. Because WSI datasets usually contain only few hundred training samples, many methods focus on mitigating overfitting, for example, by introducing bag splitting create pseudo- bags (Zhang et al.] 2022). designing efficient mnstance-based classifiers (Qu et al. (024). and per- forming hard-negative mining with EMA teachers (Tang et al.) (023b). Lu al. (2021) introduce cesstring-constrainngg attention multiple-instance learning (CLAM), which replaces max-pooling with class-specific attention pooling and adds instance-level clustering supervision so that weakly supervised slide-level MIL can be both data-efficient and interpretable WSIs, using contrastive
critical-instance branches (Li et al. 2021a). Recently, Zhu et al. (2025) 2023) systematically studied the effect of random dropping MIL and proposed to randomly remove the top-K instances with the highest attention weights together with G x & similar tokens during training, which mitigates overfitting and encourages convergence to flatter regions of the loss landscape, thereby improving generalization. Since our anchor leverages an EMA update, we relate it to EMA/teacher models and provide additional details in Appendix A
3 PRELIMINARIES AND MOTIVATION
3.1 NOTATION
Scalars are denoted by non-bold letters (e.g., a, (9), vectors by bold lowercase letters (e.g., a), and matrices by bold uppercase letters (e.g., A). The i-th entry of vector is written as A
C-dimensional probability simplex is denoted by AC. For two distributions PI, P2 E A the
Rd Kullback-Leibler divergence (KL divergence) is defined as KL(P:IIP2) = - Pic log P3
3.2 MULTIPLE INSTANCE LEARNING WITH ATTENTION
In MIL, supervision is provided only at the bag level. A slide is represented as bag X = {x; J.v with unknown instance labels. After pretrained encoder, we obtain instance embeddings {h, }
Attention-based MIL assigns scalar attention score each embedding via learnable scorer fo: 24 fo (h:),
z (21, zN) R.
Scores are normalized into an attention distribution on the probability simplex N using a softmax:
exp(z;)
Σ a ===1
a = (a) (a....,, aN) E A
a;
Σド= exp(2))
The seide-level representation, hbag weighted by the attention distribution and is passed to a classifier to produce the bag-level prediction..
ΣN
== Og h, is convex combination of instance features
N
3.3 MOTIVATION
MIL. is effective for WSI analysis, but its weak supervision and small WSI dataset sizes introduce three failure modes: unstable attention dynamics, over-concentrated attention, and overfitting.
(PI) Unstable attention dynamics. Under bag-level supervision, empirically observe that attention distribution oscillates across epochs rather than converging to a consistent pattern. To the best of our knowledge, this phenomenon has not been previously identified explicitly addressed in the literature. To quantify stability, we measure the Jensen- -hasnnn- divergence (JSD) between consecutive attention distributions for the same WSI. Let a A denote the attention over N tiles at epoch t. With KL(-II)) denoting the KL divergence and ā [a, + at+1), we define
JSD(a KKL(,, l() (B0))+/KKL(alll (L()++))aa.
(3)
4

---

Published as a conference paper at ICLR 2026
As shown in Figure TransMIL (Shao et. 2021) exhibits large JSD fluctuations, indicating lack of stable convergence. Similar behavior appears in other attention-baun MIL models; additional results provided in Appendix[[
(PII) Over-coonentrattinn of attention. Complementary to instability, prior works report that ABMIL often assigns most mass to few tiles, which harms generalization and interpretability Zhang et al. 2024 2025b). Distinct from previous approaches, we attribute these over-concentrated attention distributions the exponential nature of the softmax function.
(PIII) Overfitting. WSI datasets typically contain only few slides per class and highly redundant tiles (Zhang et al. 2022). High-capacity neural-network netlll--twok-bbaeee -based MII. models can memorize spurious tile-level patterns, leading to poor out- -itto--distrbutiion performance. To alleviate this, we introduce a random token drop mechanism specialized for our method.
In the next section, we present our proposed methodology, which simultaneously addresses the three problems (PI), (PII), and (PIII).
4 METHODOLOGY
To address the limitations of attention-based MIL, propose a framework illustrated in Figure Our methodology addresses (PI) by stabilizing attention through an anchor model, tackles (PII) by replacing softmax with an NSF in the anchor, and mitigates (PIII) by token random dropping to regularize training. The next subsections detail each component and the overall objective.
4.1 STABILIZING ATTENTION DISTRIBUTIONS VIA AN ANCHOR MODEL
As discussed in Section 3.3 weak supervision in MII. often leads to unstable attention distributions that fluctuate across epochs, prevent- ing convergence. To mitigate this, introduce anchor model that mirrors the attention block of the online model. The anchor serves as a stable reference by being up- dated through an EMA of the on- line model's parameters. Specifi- cally, training step f, the anchor parameters 0r are updated as
FEAT
rd Transformer Encoder
Clossifier
<CLS>
Transformen Encoder
Attention
Transformer Encoder
Score
Output takens
Online Model
Input tiles
EMA
Seftmax
MLP
a emue
Layer Norm
Frozen
Learnable
sg /
Attention Score
MultisHeed Attention +
Feature extractor
Transformer Encoder
NSF
0' + 1-mm0_1+ (1 m)...
(4)
Feature tokens
Layer Norm
Learnable tokens
where O2 are the online model's pa- rameters and m E 1) is the EMA factor. Both the anchor and online models receive the same in- puts, but only the online model updated by backpropagation, the anchor is updated via EMA. The goal is align the online atten- tion distribution to the anchor dis- tribution, which yields a stabiliza- tion loss.
Attention
Irput takens
Model
Score
Figure 2: Overview of ASMIL. Each WSI is divided into tiles and embedded into vision tokens using a pretrained encoder. These tokens, along with trainable FEAT tokens, feed into both online and anchor encoders. The anchor encoder's atten- tion scores the FEAT tokens transformed into a prob- ability vector using an NSF, while the online encoder applies softmax. To stabilize training and prevent the online model's attention from becoming overly concentrated, we compute the KL divergence between the two distributions. Gradients are blocked to the anchor encoder using stop-gradient (sg) oper- ator, and its parameters are updated via EMA from the online encoder. During training, randomly drop (rd) N FEAT to- kens, feed the remaining tokens into a second transformer with trainable [CLS] token, and train a classifier its output. & and * indicate learnable and frozen components, respectively.
In Appendix C we show that stan- dard attention-based MIL yields poorly separated leg-level feature clusters during training because at- tention distributions do not con- verge reliably. Introducing the an-
5

---

Published as a conference paper at ICLR 2026
chor model stabilizes attention, improves convergence, and produces clearly separated bag-level clusters.
Remark 1. Why anchor model instead of single regularizer. Scalar penalties on attention, such as entropy, ly, or temperature, are content-agnoste and act only on the current batch. They cannot encode relational structure among instances. An EMA anchor model yields a data-dependent attention distribution conditioned on the bug. Encouraging the online attention to stay close to this target performs functional regularization that captures inter-instance relations and stabilizes training, which scalar regularizer cannot do.
The anchor discarded at inference, adding no extra FLOPs or latency. In the next subsection, describe how we further improve the anchor's attention using NSF, which alleviates over- concentration before applying this stabilization loss.
4.2 PREVENTING ATTENTION CONCENTRATION WITH NSF IN THE ANCHOR MODEL
In conventional transformer architectures, the softmax function maps self-attention scores R to a probability vector. However, softmax often produces over-concentrated attention, in which few tokens dominate while the weights of the remaining tokens vanish. Temperature scaling is an incomplete remedy: small temperatures preserve concentration, while large temperatures flatten the distribution so aggressively that weak tokens receive undue weight. We therefore seek a mechanism that equalizes attention among genuinely informative tokens while suppressing weak ones.
We compare softmax with normalized sigmoid function (NSF) For 2 (21, ,... define
e
(=x)) (z) = Σ}=1 (zz)'
1
0% amx (z:T)
(1)) 1+ -
(5)
For thresholds > O and bandwidth 7 2 let S(T, H. L) be the of score vectors with
"high" indices H satisfying 2; E [T, T + 7] for i E H and "low" indices L satisfying 2j S
for j E L. Denote h |7] and é A [C]. The following theorem (proof deferred to Appendix B formalizes the selective flattening property of NSF and shows that softmax cannot match with single temperature.
Theorem (NSF achieves selective flattening; softmax cannot with single T), FixT > 0, 7 0. and index sets H. L with h 2 1. l 2 1. For any z E S,
(A) NSF bounds. For any i, h. E H and any E C,
Of (2) (z) o(zw) I a(T +7)
at naf (z) (25))
o(-r) a(T) = (+e-(r+)) $1e (x) ho(T) h
1te
r
(6)
Hence, NSF equalizes the high tokens up to a fuctor 4 and suppresses lows to at most e e As T xc with fixed 7, ratios among high tokens approach 1 and low-token weights vanish.
//.
(B) Softmax incompatibility with one temperature. Suppose we desire suppression and equalization targets (E, K) on S(T, 7, H, L):
max O smx (2:
(Suppression)
smx 0g
(x:T) TETTE S Vj E L,
(Equalization)
minweN aamx (z: S k.
2t and T log
Then T must satisfy T
<
simultaneously, which is impossible whenever
log
γ
2r l)
Thus, no single temperature achieves both targets for all z E S((T,,
log K
We further illustrate this effect in Figure 3 by comparing attention maps with softmax and NSF using ABMIL (Isse al.) 2018) on cancer slide from the CAMELYON-16 dataset (Ehteshami Bejnordi et al. 2017). Softmax yields a highly concentrated map that obscures broader context, whereas NSF produces less concentrated attention map that highlights most cancerous regions.
We discuss alternatives NSF, including entmax and softmax with temperature scaling in Appendix

---

Published as a conference paper at ICLR 2026
A naive option is apply NSF directly in the online model. In practice, this induces vanish- ing gradients and degrades performance; see Appendix G We therefore place NSF in the an-
chor model as stable prior, guiding the online model without hindering its learning dynamics.
As attention distributions lie on the probability simplex,
225
the KI.
200
divergence to align the online atten- tion distribution with the NSF-based anchor distribution:
75
atteruated after softmax
More dominazed after softmax
Number per bin
so
3
CAS KL(a"ef [))) (7)
Atemion scuees
where a is the online attention (soft- max over z) and a usf the an- chor attention (NSF over the anchor scores). Using dag Te, 09(6,, os)
and treating a nst as fixed, the gradi- ent with respect to the online atten- tion score 23 is
(b)
(c)
Figure 3: (a) Distribution of attention scores in ABMIL,
which exhibits a long-tailed pattern. (b) Attention distri- bution obtained with the softmax function and (c) with the NSF. Unlike softmax, the normalized sigmoid suppresses large values in the long tail, yiclding less sparse and more interpretable attention distribution.
OKL(a (1))
Σ
nsf
(6.
04)
02:
nsf
(8)
Thus, gradient descent moves the online attention toward the anchor distribution, promoting stability and discouraging over-concentration.
The anchor in ASMIL superficially resembles the teacher in MHIM-MIL (Tang el al. 20236): both are EMA- AMA-updated copies of the online model. Their roles, however, differ in two important ways.
Remark 2.
MHIM- uses the teacher to mine hard instances, whereas ASMIL uses the anchor to stabilize attention and prevent over-concentration (ü) MHIM-MII, matches softmax bag-level features, while ASMII. directly matches attention distributions. Appendix discusses why sofimax bag-level matching fails stabilize attention maps.
4.3 MITIGATING OVERFITTING WITH TOKEN RANDOM DROPPING
To reduce overfitting, we designed token-level regularizer, specialized for ASMIL,, that operates on the trainable tokens used by the online model. Let a WSI æ be partitioned into M tiles and embedded by a pretrained encoder into tile tokens T = [tl..... t. We augment these with N trainable FEAT tokens P {PI.... pN } and feed the concatenation [T; P] into the online encoder. After the online encoder, only the FEAT tokens are retained. Since the number of FEAT tokens is much smaller than the tile tokens (i.e., N < M), this design acts information aggregation via token reduction.
During training, sample independent Bernoulli mask over FEAT tokens and drop fraction B 1) of them. Denote the kept set by Pusep with |Pkeepl N Binomial 1 B) and E[ (1 B)N. The remaining tokens, together with trainable [CLS] token, are passed second transformer to produce a bag representation hsag: which is then classified to obtain ÿ. At inference time, no tokens are dropped (B 0). Since ASMIL stabilize the attention via aligning the anchor model, which assumes a one-to-one correspondence, as thus general instance dropout method, such as MIL-Dropout Zhu et al. 2025), could not be integrated easily.
This stochastic removal prevents co-adaptation among FEAT tokens and discourages the model from over-relying on a subset of tokens, while preserving image content by keeping all FEAT tokens at inference. Empirically, this acts effective regularizer that improves generalization. In Ap- pendix K.4 we study the effect of and observe a consistent peak in performance around B zs 0.5.
4.4 OVERALL TRAINING OBJECTIVE
Based on the discussion thus far, train with joint objective that couples standard bag-level classification with attention stabilization:
L LCB + B LAS,
(9)
7

---

Published as a conference paper at ICLR 2026
Table 1: The Fl score and AUC of different MIL approaches across three WSI datasets. Bold and underlined values denote the best and second-best results, respectively.
Dataset
CAMELYON-16
CAMELYON-17
BRACS
Backbone Method
FI score
AUCT
FI score
AUCT
FI score
AUCT
ABMIL JCML
2013
20.020
20.00
20.002
0.779.o.02
20.022
0.723
Clam-SB
0.742 -0.024
0.763 10.01
1..04
0.778
0.521
0.750
ImageNNet Pretrained
TransMIL
a.499
a.794
0.4.4
0.732
DSMIL evv
0. 037 ono
0.773 one
0.473
0.705go.om on
0.51 1 1
0.751 go now
ResNet- 18
DIED-MII
2003
0.758 osi
0.81 Sgn ons
0. 0.546g ono
0. 0.735go o11
0.469 g0 ore
0.717 no3
IBMIL
0.777 2 ooe
7..999asm 2o one
0.533 ors
0.81320.00
0.510 2000s
0.726 (72622..om
MHIM-MIL
ICCY
30236
0.752 +0.004
0.772 20.0..0
0.56
0 2.815
0.511 10.000
0.775
ACMIL
0.798 +0.029
0,841 10.000
0.528 so.0s5
0.789 20.005
0.552 10.00s
o..0s
CAMIL
KLL
3164
0.778
0.812 2oo.or
1.5033..omm
0.800go.000
0.569 200000
0.787 20.011
AFM MIDKA
0 1.804gnmm no0
0.859 ox1
0.525ga oxx
0.828 os4
0.554go.om
0.764go.m
HDMIL
0.790ga ons
1.85622 our
0.557 O0T
(.578 orn
0.761 g0
ASMIL (Ours)
0.81420.00
0 0.870
0.5642..20
0.851
O. 1.601
O 2.800
ABMIL JIMI
0.91 4a0.m
0.945go.om
0.522
0.33220000
0.68020.m
0886b20.00
Clam-SB Nakee
0.925
0.969
0.523
0.846go
0.631 g0
(L863
TramsMIL
Na
5031
0. 1.9222ss 20 o19
0. 0.943.2.om ocp
0.55 0.554 o1s
0.7 A.79222.00 020
0.631 20.000
0.841 20
VIT-S SSL pretrained
DSMIL CVPS
J0c
0,943 20.007
1.9666...oo
0.53 55322...o
0, 20.00
0.577.0.025
0.81620.00
DTFD-MIL
0,948 vo.007
0,980 v0.003
0, 6277.....
0 0.866 00.022
0.612 ro.0ab
0.870
IBMIL
0.912 20.004
0.954=
0.557 20.04
0.850
0.645
0.871 20.014
MHIM-MIL
500th
0.932 28222..00 mo4
0.970 oay
0.541 م0
0.845 noo
0.625
0.865
ACMIL BETS
0.954ga onn
0.974 onn
0.562 osn
0.863 004
0.722g oon
01.888 20 o10
CAMIL
KUS
1004
0,930 20 one
0,963 or
0, 1.633 oz2
0.886 0014
0. 7..709. 20 013
0.836 20.04
AEM
0. 0.947:a0.om
0,974
0. 1.647720.oor
0. 0,887=a..0
0. 0.742 20.000
0.905=0.000
HDMIL ASMIL (Ours)
0.958 r0.023
0.976 F0.017
0.671 10.012
0.796 0.796....22
0.717 +0.00
0.874.0.00
0.965 g. dea
0.985 s:0
0.689 8.0
0.898 100010
0.78 1 $:0.042
0.914:000
where the coefficient 3 > 0 balances the stabilization and classification objectives. In practice, to calculate LAS, a is computed by a softmax over the online scores, nsf is computed by applying the NSF to the anchor scores, and the anchor model is treated as ttop-gradient while its parameters are updated via EMA. The KL divergence is taken over the attention distributions on the FEAT token set used for aggregation. This objective discourages attention concentration through LAs and preserves task performance through LCE. ASMIL can be easily applied to other tasks, including survival pre- diction by replacing the objective function and the classification head accordingly. During training, the online model is updated by gradient descent
oc+1 = 0, VoL,
(10)
where 17 is the learning rate, L is computed as in Equation 9 The anchor model is then updated according to Equation (4). The gradient is only used to update the online model, while the anchor model influences learning through Equation 77. At inference time, ASMIL uses only the online model and discards the anchor model; therefore, the anchor does not increase the computational budget at inference.
EXPERIMENTS
To demonstrate the effectiveness of ASMIL, we evaluate on three well-known public WSI subtyp- ing datasets: (i) CAMELYON-16 (Ehteshami Bejnordi et al. 2017), (ii) CAMELYON-17 (Bándi et al.) 2019). and (ii) BRACS (Brancati et 2022). Details of the data splits, preprocessing, train- ing setup, and baselines are provided in the Appendix B We further evaluate ASMIL survival prediction and non-WSI datasets in Appendix N and Appendix O respectively.
5.1 SUBTYPING PERFORMANCE
We compare ASMIL against eleven attention- MIL baselines that are designed for WSIs:
CLAM-SB (Lu et al., 2021), TransMIL. (Shao et al.) 2021), DSMIL (Li et al. (021b). DTFD-MII. (Zhng et al. 2022). IBMIL (Lin et al. (003), MHIM-MIL. (Tang et al. 2023b). ABMII (Tsee et al., 2018), ACMIL (Zhang al. 2222), CAMIL (Fourkioti et al. (024). AEM (Zhang et al. (025b) and HDMIL (Dong et 2025). Because WSI datasets are ilass-imbalanced, we report the Fl score
and area under the ROC curve (AUC) for each dataset in Table
See Appendix for details metric computation and interpretation.
8

---

Published as a conference paper at ICLR 2026
Table 2: Applying anchor model and NSF other attention-based MII. methods.
Dataset
CAMELYON-16
CAMELYON-17
BRACS
Method
Anchor
NSF
F score
AUC
F score
AUCT
F score
AUC
x
0.91
0.9452o.007
0.522ao.om
0.85320
0.6802o.om 053
0.86620.cm
ABMIL
XIMI bo1s
x
0,951 gn
0.9632o..0
0.573gmm1
0.871
0.751gmmx
0.877
+0.037
+0.018
+0.001
+0.018
+0.071
+0.011
/
/
0.95322
0.967
0.57
0.883
0.753
0.887
+0.039
+0.092
+0-052
030
+0.073
+0.001
x
x
0.925 283
0.969 004
0.5232n.om noe
O. 0.84620
0.631
0.86320.oo 001
CLAM-SB Nammr
/
0.987 20 D0A
0.9792u.0r
0.547
0.887 20.004
0.678gu.urx
0.806i
/
0.948
0.981
0.550
0.8862o
0.679 D1X
0.887
+0.003
+0.022
+00 007
+0.040
+0.018
+0.074
0.322 DE9
0.9432o oo1
0.554 oas
0. 0.7922e
0.631
0.841
TransMIl. Neadts
/
x
0.931
0.947
0.577
0.824
0.647
0.853
+000009
+0.004
+-3 00.223
+0.022
+0.016
010
0.9033 noa
0.954 or
0.580 gn
0.829
0.672
0.883
+0.017
+0.01
+-0 296
03.077
+0.011
0-15
x
X
0.94322
0,9662o
0.5322o
0.804
0.577
0.8
DSMIL CVT aith
/
x
0.943
0.974
0.544
0.819
0.6092o.me
0.837 2377
10.000
+0 00s
+0
+4.015
+0.001
+0 021
0.942
0.985
0.559
0.823
0.612
0.849
Overall, ASMIL demonstrates superior performance, achieving state-of-the-art performance all datasets when paired with an in-domain ViT-SSL backbone, and remains competitive with the best baseline ImageNet-pretrainee R8sNet-18 features. On the BRACS dataset, our method attains an F1 score of 0.781 and an AUC of 0.914, exceeding the previous best results by 3.9 and 0.9 percentage points, respectively. This shows its effectiveness in capturing subtle histopathological features in heterogeneous subtyping tasks.
For CAMELYON-16 and CAMELYON-I7 datasets with sparse tumor regions, where malignant tissue may occupy as little as 5% of slide (Cheng et al. 2021), advantages are even more pronounced. on CAMELYON-16 16, we observe a 3.3% increase in F1 score and a 1.6% uplift in AUC compared to the strongest baseline; similarly, on CAMELYON-IT 17, ASMIL improves the Fl score by 6.49%, which highlights ASMIL's efficacy under an ill- posed, weakly supervised task. We compare the computational cost of ASMIL with that of other benchmarks in Appendix M.
5.2 INTEGRATING THE ANCHOR MODEL AND NSF WITH OTHER MIL METHODS
We regard the anchor model as a general plug-in module for attention-based MIL in WSI analy- sis. Accordingly, for each baseline we evaluate two variants while keeping all other components and byperparameters fixed: (i) +Anchor (EMA-updated anchor with attention matching), and (iz) +Anchor+NSF (anchor updated by EMA and using NSF). The results are summarized in Table [ As shown, adding the anchor model and the NSF consistently improves performance, with F1 score gains up to 10.73% (for ABMIL on BRACS), except when adding the anchor DSMIL on the CAMELYON- 16 dataset, where the Fl score decreases by 0.001 relative to the original model. The additional computational cost introduced by the anchor model is reported in Appendix M.2
Table 3: Component-wise wiss ablation of AS- MIL on BRACS. We evaluate the contri- bution of the anchor model, NSF, and ran- dom drop (rd).
5.3 LOCALIZATION
We evaluate tumor localization on CAMELYON- 16 both qualitatively and quantitatively. Qualitative heatmaps are shown in Figure Compared with base- line methods, ASMIL consistently highlights all can- cerous regions. We attribute these gains reduced over-concentration by the NSF in the anchor model, which yields more faithful attention distributions.
Anchor
NSF
rd
Fl score 1
AUC
0. 0...az
0.914
0.014
X
0.765 ooe
0.903
0.750.o.oz
0.895
X
0.747
0.887=0.0:3
V
0.728
X
X
0.712
0.860
Following the official CAMELYON-16 and Fourkioti et al.] (2024), we report lesion-level Free-
Response ROC (FROC) (Miller 1969 Bunch 1978) the Dice coefficient on cancerous slides, and tile-level specificity normal slides. To obtain the predicted masks, we use scaled attention dis-
tributions for CLAM (Lu et al. 2021). TransMIL (Shao et al., 2021). DSMIL (Li et al. 2021b), and CAMIL (Fourkioti et al. 2024); tile-level logits for DTFD-MI -IIL (Zhang al. 2022 and for ASMIL, the tir- average of FEAT-token attentions. Quantitative results for FROC, Dice, and specificity, well as additional attention-map visualizations, are provided in Appendix
9

---

Published as a conference paper at ICLR 2026
Annotated WSI
ASMIL
DTFDMIL
CAMIL
TransMIL
Figure 4: Visual comparison of attention maps on the CAMELYON- dataset. The left column shows the original WSI with ground-truth tumor annotations outlined in red; the remaining columns present attention maps for ASMIL (ours), DTFD-MIL, CAMIL, and TransMIL (left to right). 5.4 ABLATION STUDY
Lastly, we evaluate the effect of the anchor model, NSF, and random drop (rd) by enabling dis- abling them all combinations. As shown in Table the full model (all three enabled) achieves the best Fl score and AUC. Removing any component degrades performance, with the anchor model having the largest impact. Without all three, the model drops to the lowest scores, confirming that each component contributes the overall effectiveness of ASMIL Additional ablations the loss weight 3, the number of trainable FEAT tokens, the EMA factor m, the anchor update frequency, and the random drop rate are reported in Appendix
CONCLUSION
In this work, we identified a previously overlooked failure mode attention-based MIL for WSI: un- stable attention dynamics that hinder convergence. We proposed ASMIL, which stabilizes training via an anchor model, prevents over-concentration by using normalized sigmoid in the anchor, and mitigates overfitting with token dropout, Across multiple WSI benchmarks, ASMIL improves clas- sification performance and state-of-the-art localization performance. These results underscore the importance of jointly controlling attention stability, concentration, and overfitting in weakly super- vised WSI analysis. We anticipate that the proposed anchor model and normalized sigmoid function will serve building blocks for future MIL-based WSI analysis algorithms, ultimately facilitating more accurate and interpretable analysis of gigapixel pathology images. Due to space constraints, defer the discussion of future work and limitations to Appendix
ETHICS STATEMENT
All WSI datasets used in this work are publicly available and obtained from open-access web- sites. The usage of these datasets strictly follows the terms and conditions set by the dataset providers and adheres to established academic and research community standards. No personally identifiable information or sensitive patient data is involved.
REPRODUCIBILIIY STATEMENT
We have taken steps to ensure our results are reproducible. All model and algorithmic details, train- ing procedures, hyperparameters. evaluation protocols, and metrics are specified in the main text. The appendix provides complete proofs, implementation notes, ablations, and additional qualita- tive results. anonymized GitHub repository contains the source code and configuration files, and pre-trained checkpoints. All datasets used in our experiments are publicly available; download links, data splits, and preprocessing steps are documented in the repository and referenced in the appendix.
REFERENCES
Stuart Andrews, loannis Tsochantaridis, and Thomas Hofmann. Support vector ma- chines for multiple-instance learning. In S. Becker, S. Thrun, and K. Obermayer (eds.), Advances in Neural Information Processing Systems, volume 15. MIT Press, 2002. URL https: /tt.hh/pss//ppooceeeeeeeerrrrrrr
10

---

Published as a conference paper at ICLR 2026
James Bacus. Method and apparatus for acquiring and reconstructing magnified specimen im- ages from a computer-conttooled microscope, December 2001. URL https:://////. ptttpps:/patentss. google. http://lee.ooooeeeee//////00////
US Patent Application.
Nadia Brancati, Anna Maria Anniciello, Pushpak Pati, Daniel Riccio, Giosuè Scognamiglio, Guil- laume Jaume, Giuseppe De Pietro, Maurizio Di Bonito, Antonio Foneubierta, Gerardo Botti, et al. Bracs: A dataset for breast carcinoma subtyping in h&e histology images. Database, 2022: baac093, 2022.
PC Bunch.
Free response approach to measurement and characterization of radiographic observer performance. AJR Am J Roentgenol, 130(2):382, 1978.
7ak Buzzard, Konstantin Hemker, Nikola Simidjievski, and Mateja Jamnik. Paths: A hierarchical transformer for efficient whole slide image analysis. arXiv preprint urXiv:2411.888 18225, 2024.
Péter Bándi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, Quanzheng Li, Farhad Ghazvinian Zanjani, Svitlana Zinger, Keisuke Fukuta, Daisuke Komura, Vlado Ovtcharov, Shenghua Cheng, Shaoqun Zeng, Jeppe Thagaard, Anders B. Dahl, Huangjing Lin, Hao Chen, Ludwig Jacobsson, Martin Hedlund, Melih Çetin, Eren Halıcı, Hunter Jackson, Richard Chen, Fabian Both, Jörg Franke, Heidi Küsters Vandevelde, Willem Vreuls, Peter Bult, Bram van Ginneken, Jeroen van der Laak, and Geert Litjens. From detection of individual metas- tases to classification of lymph node status the patient level: The camelyonl7 challenge. IEEE Transactions on Medical Imaging, 38(2):550-50) 2019. doi: 10. 1111099TMM.20182867753.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
David Chapman and Ajay Jain, Musk (Version 1). UCI Machine Learning Repository, 1994a. DOI: https://doi.or//10. 10.4443////55K3333
David Chapman and Ajay Jain. Musk (Version 2). UCI Machine Learning Repository, 1994b. DOI: http://doi.orr/100 01..:///orr//022443/55555560 24432/C51608.
Richard Chen, Ming Y Lu, Muhammad Shaban, Chengkuan Chen, Tiffany Y Chen, Drew FK Williamson, and Faisal Mahmood. Whole slide images are 2d point clouds: Context-aware sur- vival prediction using patch- based graph convolutional networks. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 339-349 Springer, 2021.
Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Bowen Chen, Andrew Zhang. Daniel Shao, Andrew H Song, Muhammad Shaban, et al. Towards a general- purpose foundation model for computational pathology. Nature Medicine, 2024.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PmLR, 2020.
Jun Cheng, Yuting Liu, Wei Huang, Wenhui Hong, Lingling Wang, Xiaohui Zhan, Zhi Han, Dong Ni, Kun Huang, and Jie Zhang. Computational image analysis identifies histopathological im- age features associated with somatic mutations and patient survival in gastric adenocarcinoma. Frontiers Oncology, 11:623382, 2021.
Zhixiang Chi, Li Gu, Tao Zhong, Huan Liu, YUANHAO YU, Konstantinos N Plataniotis, and Yang Wang. Adapting to distribution shift by visual domain prompt generation. In The Twelfth Inter- national Conference on Learning Representations, 2024.
Thomas M Cover. Flements of information theory. John Wiley & Sons, 1999.
Kausik Das, Sri Phani Krishna Karri, Abhijit Guha Roy, Jyotirmoy Chatterjee, and Debdoot Sheet. Classifying histopathology whole-slides using fusion of decisions from deep convolu- tional network on collection of random multi-views at munttimmaniiiiatiinn. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 1024-1027, 2017. doi: 10. 109//181.201...955666
11

---

Published as a conference paper at ICLR 2026
Kausik Das, Sailesh Conjeti, Abhijit Guha Roy, Jyotirmoy Chatterjee, and Debdoot Sheet. Multiple instance learning of deep convolutional neural networks for breast histopathology whole slide classification. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 578-581,, 2018. doi: 10. 109//BB1.2088.88 8.666..........
Thomas G. Dietterich, Richard H. Lathrop, and Tomás Lozano-Pérez. Solving the multiple in- stance problem with axis axis-parallel rectangles. Artif. Intell., 89112)331-71, January 1997. ISSN 0004-3702. doi: (0000///000043702990000000000000 50004-3702 (96) 00034-3
URL hattps://oois /ttp:/::///61..o//11...0.0.
Jiuyang Dong. Junjun Jiang, Kui Jiang, Jiahan Li, and Yongbing Zhang. Fast and accurate gigapixel pathological image classification with hierarchical distillation multi-instance learning. In Pro- ceedings of the Computer Vision and Pattern Recognition Conference, pp. 30818 10188888,, 2025.
Zhaolong Du, Shasha Mao, Yimeng Zhang, Shuiping Gou, Licheng Jiao, and Lin Xiong. RGMIL:
Guide your multiple-instance learning model with regressor. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
peatts:://oennevieet wwtpeeiwwwww.eeeeeeeeeeeeeeeeeee wwt/
id=eCoE9CVRRC
Zhaolong Du, Shasha Mao, Xuequan Lu, Mengnan Qi, Yimeng Zhang, Jing Gu, and Licheng Jiao. Rethinking multiple-instance learning from feature space probability space. In The Thirteenth International Conference on Learning Representations, 2025. URL https::/p//... htttt::://ppeneeiiee n 2tw.aummmiiiie www.ommooddddtoooossod...u
Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes van Diest, Bram van Ginneken, Nico Karsse- meijer, Geert Litjens, Jeroen A. W. M. van der Laak, and the CAMELYON16 Consortium. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22):199-22110 12 2017. ISSN 0098-7484. doi: /doi 2..001/jama.221..455555. URL httpp:::/db..oog/11...000
2017 14585
Olga Fourkioti, Matt De Vries, and Chris Bakal. CAMIL: Context-aware multiple instance learning for cancer detection and subtyping in whole slide images. In The Tweifth International Confer- ence Learning Representations, 2024. URL /tttps://operreee
net forum?id
r2BakAEmoc
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Ghesh- laghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent: A approach to self-supervised learning, 2020.
Ziyu Guo, Weiqin Zhao, Shujun Wang, and Lequan Yu. Higt: Hierarchical interaction graph- transformer for whole slide image analysis. In International Conference on Medical Image Com- puting and Computer-Assisted Intervention, pp. 755-764. Springer, 2023.
Shayan Mohajer Hamidi and Linfeng Ye. Adversarial training via adaptive knowledge amalgamation org/ abs 13324
of an ensemble of teachers, 2024. httt::ts::
Shayan Mohajer Hamidi and Linfeng Ye. Distributed quasi-newton method for fair and fast fed- erated learning. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URI.
hnttps://on httpsss//ppeeeeieeieee net www.:neeeeeooteeeeeeeeooot www.forrrddiiiioo (=rrummmddd
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
doi: 10.1109/CVPR.20169900
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.0255 2015.
Maximilian Ilse, Jakub Tomezak, and Max Welling. Attention-based deep multiple instance learn- ing. In International conference on machine learning, pp. 2127-2136. PMLR, 2018.
Mingu Kang, Heon Song, Sconwook Park, Donggeun Yoo, and Sérgio Pereira. Benchmarking self- supervised learning on diverse pathology datasets. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp.
June 2023.
12

---

Published as a conference paper at ICLR 2026
James Keeler, David Rumelhart, and Wee Leow. Integrated segmentation and recog- nition of hand-printed numerals. In R.P. Lippmann. Moody, and D. Touretzky (eds.), Advances in Neural Information Processing Systems, volume 3. Bargan-Kaufmani ncs.s//sss.sserrrrc............ ccppppprrrppppppp.pppppppp fi fiwes//eaaaa /ttss//ppp/91999////
1990. URL.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6940 6980, 2014.
Oren Z Kraus, Jimmy Lei Ba, and Brendan J Frey. Classifying and segmenting microscopy images with deep multiple instance learning. Bioinformatics, 32(12):i52-159 2016.
Bin Li, Yin Li, and Kevin W. Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In 2021 IEEE/CVE Con- ference on Computer Vision and Puttern Recognition (CVPR), pp. 14313-14323, 2021a. doi: 10.1 N..1099CVVR4444372200
.01409.
Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 14318-14328, 2021b.
Jiayun Li, Wenyuan Li, Arkadiusz Gertych, Beatrice S Knudsen, William Speier, and Corey W Arnol.. An attention basee multi-resolution model for prostate whole slide imageclassification and localization. arXiv preprint arXiv: / /905. 13208, 2019.
Tiancheng Lin, Zhimiao Yu, Hongyu Hu, Yi Xu, and Chang- Wen Chen. Interventional bag multi- instance learning on whole-slide pathological images. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 19830- 19839, 2023.
Pei Liu, Luping Ji, Jiaxiang Gou, Bo Fu, and Mao Ye. Interpretable llsounl survival analysis with ordinal inductive bias for computational pathology. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL htttps:///ppeeeeee tr (rj2gqrriz]
net/ fert/ffrrm?id
Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology whole-slide images. Nature biomedical engineering, 5(6): 5700 2021.
Ming Y. Lu, Bowen Chen, Andrew Zhang, Drew F. K. Williamson, Richard J. Chen, Tong Ding, Long Phi Le, Yung-Sung Chuang, and Faisal Mahmood. Visual language pretrained multiple in- stance zero-shot transfer for histopathology images. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 19764-19775, June 2023.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9NNov) 2008
Oded Maron and Tomás Lozano-Pérez. A framework for multiple- iinsalleeeetneeeeeeeeeeeee learning. In Proceedings of the 1997 Conference on Advances Neural Information Processing Systems 10, NIPS '97, pp- 570-576, Cambridge, MA, USA, 1998. MIT Press. ISBN 0262100762.
Andre Martins and Ramon Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Ma- chine Learning Research, pp. 1614-1623, New York, New York, USA, 20-22 Jun 2016. PMLR. URL httphhtts/ htctps://proceedinggt mwr. www.spressssee. / hatpaes/////mrrtrree /ttm/aaaiinssioooo htmml
Harold Miller. The froc curve: A representation of the observer's performance for the method of free response. The Journal of the Acoustical Society of America, 46(6B): 1969.
Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Ar- mand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision, 2023.
13

---

Published as a conference paper at ICLR 2026
Linhao Qu, Yingfan Ma, Xiaoyuan Luo, Qinhao Guo, Manning Wang, and Zhijian Song. Rethinking multiple instance learning for whole slide image classification: A good instance classifier is all you need. IEEE Transactions Circuits and Systems for Video Technology. 34(10))9732-9744 2024. doi: 10.1 0..10997CSST122244440033
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJC), (5(3):211-2522 2015. doi: 10. 1007/1 1aeaa11233--10 1263-015-0816 16---
Daniel Shao, Richard J. Chen, Andrew H. Song, Joel Runevic, Ming Y. Lu, Tong Ding, and Faisal Mahmood. Do multiple instance learning models transfer? In Forty-second International Conference on Machine Learning, 2025. URL https:://openeeiieeee net hwtp:/eeffforuuree fwrumorrmii
hfLedquVt3
Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, and Yongbing Zhang. TransMIl.: Transformer based correlated multiple instance learning for whole slide image classification. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, 2021. URL Eorum? LKU f uwxa jHc]
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, al. Dinov3. arXiv preprint arXiv:2508. 10104, 2025.
Andrew H Song. Richard J Chen, Tong Ding, Drew FK Williamson, Guillaume Jaume, and Faisal Mahmood. Morphological prototyping for unsupervised slide representation learning in compu- tational pathology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Yuxuan Sun, Yunlong Zhang, Yixuan Si, Chenglu Zhu, Kai Zhang, Zhongyi Shui, Jingxiong Li, Xuan Gong, XINHENG LYU, Tao Lin, and Lin Yang. Pathgen-1.6m: 1.6 million pathology image-text pairs generation through multi-agent collaboration. In The Thirteenth International forum?
Conference on Learning Representations, 2025.
id=rFp2nn11qi
Wei Tang, Weijia Zhang, and Min-Ling Zhang. Disambiguated attention embedding for multi- instance partial-label learning. In Thirty-sevnnth Conference Neural Information Processing Systems, 2023a. htttts::///ppeeeeeeeeeee http://wwww....eeeeooeeeoooee/tn de:.nd==WWwwrrr..
net. fo Porommmii.
Wenhao Tang. Sheng Huang, Xiaoxian Zhang, Fengtao Zhou, Yi Zhang, and Bo Liu. Multiple instance learning framework with masked hard instance mining for whole slide image classifica- tion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp- 4078-4087 October 2023b.
Wenhao Tang, Fengtao Zhou, Sheng Huang, Xiang Zhu, Yi Zhang, and Bo Liu. Feature re- embedding: Towards foundation model-level performance in computational pathology. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11343- 11352, 2024.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged con- sistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
Paul Tourniaire, Marius Ilie, Paul Hofman, Nicholas Ayache, and Hervé Delingette. Ms-clam: Mixed supervision for the classification and localization of tumors in whole slide images. Med- ical Image Analysis, 85:102763, 2023. ISSN 1361-8415. doi: http://www...ooceceeeeeeeereeeea
2023.102761 URL https: www. www.sscciieeeeceeeeeee
Manuel Tran, Sophia Wagner, Wilko Weichert, Christian Matek, Melanie Boxberg, and Tingy- ing Peng. Navigating through whole slide images with hierarchy, multi-object, and multi-scale data. IEEE Transactions on Medical Imaging, 44(5):2002-00) 2015, 2025. doi: 1099
3532728.
14

---

Published as a conference paper at ICLR 2026
Constantino Tsallis. Possible generalization of boltzmann- -gibbs statisties. Journal of statistical physics, 52(1):479-45 1988.
Gregory Verghese, Jochen K Lennerz, Danny Ruta, Wen Ng. Selvam Thavaraj. Kalliopi P Siziopikou, Threnesan Naidoo, Swapnil Rane, Roberto Salgado, Sarah Pinder, et al. Com- putational pathology in cancer diagnosis, prognosis, and prediction-presse day and prospects. The Journal of pathology, 260(5):551-563 2023.
Yanan Wu, Zhixiang Chi, Yang Wang, and Songhe Feng. Metaged: Learning to continually learn in generalized category discovery. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 1655-1665, 2023.
Yanan Wu, Zhixiang Chi, Yang Wang, Konstantinos N Plataniotis, and Songhe Feng. Test-time domain adaptation by learning domain-aware batch normalization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15961-15969, 2024.
Jinxi Xiang and Jun Zhang. Exploring low-rank property in multiple instance learning for whole slide image classification. The Eleventh International Conference on Learning Representations, 2023. httpt://
ew. net. forum? d=01 Kmh8sEPFO)
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Nyströnformer: A nystrôm-based algorithm for approximating self-attention. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 14138-14148, 2021.
Yan Xu, Jun- Yan Zhu, I Eric, Chao Chang, Maode Lai, and Zhuowen Tu. Weakly supervised histopathology cancer image segmentation and classification. Medical image analysis, 18(3): 591-604, 2014.
En-hui Yang and Linfeng Ye. Markov knowledge distillation: Make nasty teachers trained by self- undermining knowledge distillation fully distillable. In European Conference on Computer Vi- sion, pp. 171- Springer, 2024.
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, and Beverly Yang. Condi- tional mutual information constrained deep learning for classification (2023). URL hutps://arxiv. (ttabbs/309.091233,
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, and Beverly Yang. Condi- tional mutual information constrained deep learning: Framework and preliminary results. In 2024 IEEE International Symposium on Information Theory (ISIT), pp. 569-574, 2024. doi: 10. (1119///STT77864.2224.666666
En-Hui Yang. Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, and Beverly Yang. Conditional mutual information constrained deep learning for classification. IEEE Transactions on Neu- ral Networks and Learning Systems, 36($):15436-15488 2025. doi: 20.. 3540014.
Jiawen Yao, Xinliang Zhu, Jitendra Jonnagaddala, Nicholas Hawkins, and Junzhou Huang. Whole slide images based cancer survival prediction using attention guided deep multiple instance learn- ing networks. Medical image analysis, 65:101789, 2020.
Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan, and En-Hui Yang. Bayes conditional distribu- tion estimation for knowledge distillation based on conditional mutual information, 2024. URL
https://arxxtt wwwparwwww.v..rvvooaaea www.ooggs/ss.. 2401.o///2111188/
Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah E Coupland, and Yalin Zheng. Difd-mil: Double-tier feature distillation multiple instance learning for histopathol- ogy whole slide image classification. In Proceedings of the IEEF/CVF conference on computer vision and patlern recognition, pp. 18802-18812, 2022.
Houston H Zhang, Tao Zhang, Baoze Lin, Yuanqi Xue, Yincheng Zhu, Huan Liu, Li Gu, Linfeng Ye, Ziqiang Wang. Xinxin Zuo, et al. Widget2code: From visual widgets to code via multimodal llms. arXiv preprint arXiv:25/2. 19918, 2025a.
15

---

Published as a conference paper at ICLR 2026
Jingwei Zhang, Ke Ma, John Van Aram, Rajarsi Gupta, Joel Saltz, Maria Vakalopoulou, and Dimitris Samaras. A joint spatial and magnification based attention framework for large scale histopathology classification. In 2021 IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 3771-3779, 2021. 00418.
doi: 10.109/CVPRW533982202
Yunlong Zhang, Honglin Li, Yunxuan Sun, Sunyi Zheng, Chenglu Zhu, and Lin Yang. Attention- challenging multiple instance learning for whole slide image classification. In European Confer- ence on Computer Vision, pp. 125-143, Springer, 2024.
Yunlong Zhang, Zhongyi Shui, Yunxuan Sun, Honglin Li, Jingxiong Li, Chenglu Zhu, and Lin Yang. Aem: Attention entropy maximization for multiple instance learning based whole slide image classification. International Conference on Medical Image Computing and Computer Assisted Intervention, 2025b.
Wenhui Zhu, Peijie Qiu, Xiwen Chen, Oana M. Dumitrascu, and Yalin Wang. Pdl: Regularizing multiple instance learning with progressive dropout layers, 2023.
Wenhui Zhu, Peijie Qiu, Xiwen Chen, Zhangsihao Yang. Aristeidis Sotiras, Abolfazl Razi, and Yalin Wang. How effective can dropout be in multiple instance learning In Forty-second International Conference on Machine Learning, 2025. URL Eorum? ficummid=-ss.. YHoLF CH5
A RELATED WORK ON EMA MODELS AND ANCHORING STRATEGIES
EMA-based target networks are central in self-supervised representation learning. Mean Teacher Tarvainen & Valpola (2017) maintains an EMA of student parameters and enforces prediction con- sistency with this temporal ensemble under limited supervision. BYOL (Grill et al. 2020 Wu et al., 2023) uses an EMA EMA-updated target network to provide representation targets for an online network with an additional predictor, and avoids collapse through architectural asymmetry and the EMA up- date instead of negatives. DtNO-style methods (Caron et al.. [02T] Oquab et al. 2023) adapt EMA self-distillation to Vision Transformers, where an EMA teacher produces soft probability targets on multi-crop views; centering, sharpening, and momentum schedule control the stability- trade-off of these targets.
DINOv3 (Siméoni et al. 2025) revisits EMA teachers for dense prediction and studies how dense features drift or collapse under long training. It introduces Gram anchoring, which aligns Gram matrices of patch-patte patch similarities between a student and its EMA teacher so that dense features remain close to temporally smoothed reference. The EMA momentum and the strength of this anchoring loss jointly determine how strongly dense features are tied to the teacher versus how quickly they adapt.
ASMIL also maintains an EMA-updated copy of the model, but uses it in a different regime and on different target. Training is fully supervised the bag level, and the EMA branch does not supply pseudo-labels or representation targets. Instead, it defines a temporally smoothed attention distribution over tiles, and the anchor enters the loss only through the KL term in Eq. equation 7 while the bag-level cross-entropy in Eq. equation 99 provides all semantic supervision. The shared encoder and classifier parameters are optimized by standard backpropagation; the EMA update acts purely as temporal regularizer attention, in contrast to BYOL/DINO, which anchors global embeddings, and DINOv3, which anchors patch-patch similarity structure.
The same EMA hyperparameters induce an analogous mability-adappationn trade-off but at the level of attention rather than features. The EMA momentum in ASMIL sets how rapidly the anchor fol- lows the online model, and the weight on the KL term controls how strongly attention is pulled toward the temporally smoothed reference. Unlike BYO and DINO/DINOV3, where EMA model is designed to avoid global representation collapse, ASMIL uses EMA anchoring to reduce unsta- ble and concentreedd attention patterns observed under purely online MIL training, while the supervised objective already discourages trivial constant-attention solutions.
16

---

Published as a conference paper at ICLR 2026
B EXPERIMENTAL DETAILS
We train all models for 50 epochs with batch size of I, using Adam (weight decay 10-4) and cosine learning rate schedule with an initial learning rate of 10-4 All reported results are averaged over five random seeds.
B. WSI PRE-PROCESSING
For all datasets, we used the publicly available CLAM WSI preprocessing toolbox (Lu et al. 2021 to segment tissue regions and divide each slide into non-overlapping 256 x 256 patches 2 mag- nification. Tissue segmentation was performed automatically using Otsu's thresholding. To reduce computational overhead and leverage previously learned representations, we adopted a ResNet- 18 model (He et al. (016) pretrained on ImageNet (Russakovsky et al., 2015) and an open-source self- supervised ViT-small model (Kang et al. 2023) as feature extractors" The ViT-small model was pretrained on 36,666 whole slide images from The Cancer Genome Atlas (TCGA) and the inter- nally collected TULIP dataset. For consistency and fairness in the subtyping task, we used the same feature extractors across all baseline methods.
For the localization experiments, following Tourniaire et al. (2023), we used a ResNet-18 backbone pretrained with SimCLR (Chen al. 2020 This feature extractor maps each tile 1024- dimensional feature vector.
B.2 DATASETS
CAMELYON-16 (Ehteshami Bejnordi et al., 2017 is a widely used publicty available WSI dataset designed for lymph node metastasis detection. It contains 270 training and 129 test slides collected from two medical centers, with detailed pixel-level annotations provided by expert pathologists. Notably, some slides include only partial annotations, making the dataset particularly challenging due to the presence of small or sparse metastatic regions. CAMELYON-16 has become a stan- dard benchmark for evaluating weakly supervised and fully supervised algorithms in computational pathology.
CAMELYON-17 (Bándi et al. 2019) extends the scope CAMELYON- 16 by including total of 1,000 WSIs from five medical centers, making it more diverse and clinically representative dataset. Among these, 500 slides are publicly available and come with slide-level labels, while the remaining 500 are held out for challenge-based evaluations. The inclusion of data from multiple institutions introduces significant variability in staining and scanning conditions, making CAMELYON-I7 a suitable benchmark for testing the generalization performance of WSI-based models.
The BRACS dataset (Brancati et al. 2022) is large-scale WSI dataset curated for the task of breast cancer subtype classification. It comprises 547 WSIs collected from several medical institutions and annotated by expert pathologists into clinically relevant categories: benign tumors, atypical tumors, and malignant tumors. These labels reflect the progression of breast lesions and are critical for diagnostic decision-making and treatment planning. BRACS captures a wide range of histological appearances and staining variations, making a valuable resource for developing and benchmarking MIL and weakly supervised classification models in real-world clinical settings.
B.3 DATA SPLITS
Following Zhang et al. 2025b: 2024). we partition the datasets as follows. For CAMELYON-16
the WSIs are divided into training, validation, and test sets. The 270 WSIs from Hospital five times, into training (90%) and validation (10%) subsets; the 130 WSIs from Hospital 2 are used as a test set. The official test set of 129 WSIs used for final evaluation. For CAMELYON-17 we use 500 WSIs in total: 300 WSIs from three hospitals for training/validation (90%, 10%) and 200 WSIs from two other hospitals for testing to assess out-of-disttibuttio (OOD) performance. For BRACS, we follow the official split: 395 slides for training, 65 for validation, and 87 for testing.
split,
'The checkpoint
is
available at
com cot.:/ctooooiiioiioooooi
benchma r- ss pptthology
Thh checkpoint available at ht wtthttt::////tt http::://ggttbb....p com/bi 121/1123ddmmm 1-wsi
17

---

Published as a conference paper at ICLR 2026
The task is chree-class WSI classification benign tumor, atypical tumor, and malignant tumor. All results are averaged over five random seeds, and we report the mean performance on the official competition test set.
Epoch 10
Epoon 20
Epoch 30
Epoch 40
Epoch 50
Epoch
Epoch 20
Epoch 30
Epoch
Epoch 50
Figure 5: T-SNE embeddings of ASMIL bag-level features the BRACS training set across train- ing epochs. Top: with the anchor model; Bottom: without the anchor model.
Epoch
Eooch
Epoch ad
Epoch 40
Epach sd
Epoch
Epoch
Epoch 30
Epoch
Epoch
Figure 6: T-SNE embeddings of TransMIL bag-level features on the BRACS training across training epochs. Top: with the anchor model; Bottom: without the anchor model.
C T-SNE VISUALIZATION OF BAG-LEVEL FEATURES
To assess how the anchor model stabilizes attention during training, we visualize the bag-level rep- resentations learned by ASMIL using I-SNE Maaten Hinton (2008;; see Figure 5 Compared to ASMIL without the anchor, the model with anchor forms more distinct clusters and exhibits clearer inter-class boundaries across training epochs, indicating faster convergence and more dis- criminative features. We observe similar trend for TransMIL in Figure 6
D MACRO AUC AND MACRO F1 SCORE UNDER CLASS IMBALANCE
report macro-averaged vari- ants of the area under the ROC curve (AUC) and the Fl score as our primary summary metrics. Mvero-aver assigns equal weight to each class and therefore prevents majority classes from dominating the overall score.
Since all datasets considered in this work are class-imbalanced,
Setup. Let y [1.....K] denote the set of classes. For sample x with true label E Y, let sx(x) E R be the model score for class k. Define one-vs-rest binary indicators yk HFyy k] for each class k and the corresponding confusion-matii counts (TP FP,. FNg, TN₂) computed by treating class tt as "positive" and all others "negative".
18

---

Published as a conference paper at ICLR 2026
D.1 MACRO-F1
For class k, precision and recall are
TP
TPR
Precisiong
Recall:
(11)
TPx + FP,
TPx + FN+
The per-class the harmonic mean of precision and recall:
2 Precision Recallę
F1, Precisions + Recallk
(12)
The macro-F1 averages the per-class values uniformly:
K
Macro-Fi F1
Σ FLA A=1
(13)
K
As thresholded, decision-level metric, F1, (and thus macro-F1) depends the classification threshold applied to scores (z)... We use a threshold of 0.5 for all experiments. The same definition applies multilabel settings by averaging over labels.
D.2 MACRO-AUC (ROC)
For class k, the ROC curve plots the true positive rate against the false positive rate the threshold on sy (z) varies:
TP' TP +NN,
FP
TPR
FPR,
(14)
FP + TN
The per-class AUC, AUC, E [o, 1]. is the area under this curve; equivalently, is the probability that randomly chosen positive example (for class k) receives higher score than a randomly chosen negative example. The macro-AUC is the uniform average across classes:
K
Macro-AUC =
AUC
(15)
K
Unlike F1, AUC is preshold-agnostic and measures the ranking quality of scores.
E PROOF OF THEOREM 1
Proof. We proceed in two parts.
Part A: NSF bounds. Let s; = (2,) and Ei o(2;), so a. = ss/S.
Equalization among highs. For i, h E H,
5; x
(16)
Since a is strictly increasing and 2;, En E [T, + 7).
( +2) = 1 ((+)) S 1+ee1,
-
(17)
(2N) a(T)
Suppression of lows. For any j E L we have 2j S -T. Using monotonicity and identity
o(-t) = -
for all t R,
(18)
we get <(- Mcanwhile
N S Σ
(21) 2 O(s)) 2 ho(T),
(19)
1=1
iEW
19

---

Published as a conference paper at ICLR 2026
since 2; for i E H. Hence
a(2y)
S S TT(
e
(20)
(T(
h
For completeness, equation 18 follows from a = Itel
Ite = (()).
Part B: Softmax temperature constraints, Fix > 0 and 2 E H(, Equalization among highs. For any h' E H.
a
e = (())-
ea
(21)
Over S(T, H, C), the worst high to high ratio occurs at 2, = + and 2/r =, so
maX ag
7/T
(22)
minw'e ah"
Therefore, the uniform bound
m af miny' EN o
x for all S(T, H. L)) implies
(23)
log Suppression of lows. Fix j E L. For given T, the quantity oz (² T) is maximized over S(T,7, H. L) by taking 2j = -T, J Vi E H. 2 + -co for k & H U {5}. which mini- mizes the denominator subject to the constraints. Thus
--/
sup xES(T
(2; T)
her/T + 2 +1
(24)
Consequently, the uniform suppression requirement a T) S for all S(T, 7, H. L) forces
2r
S
2T/T 2
T log 1) log h
(25)
he27/T
+1
Combining
equation
23
and
equation
yields the simultaneous constraints T
S
2r
T
If
log log
log
7 log
27 1) loog h
(26)
log(3 1
no T can satisfy both.
Instantiating NSF targets. Set Ensf = e /h and K: = Knsf 14e 11 Then
1te
log
-1)
log -log
log e--// -1) log h -ooghhe - 1) -log h logl h-1))
Enst
(27)
so the right side of the incompatibility condition equals
2r
2.
(28)
log(er - h-1) +
Meanwhile,
log Knsf log(1 + t log(1 + E
(29)
E T (1- (1- =log(1 |
(++-(1++)
NeT (7)
(30)
hence
XX.
(31)
log Kinsf
T-+00
Therefore, for any fixed 7 > 0, the incompatibility condition holds for all sufficiently large т, so no single softmax temperature match NSF uniformly on S(T, H, L).
Remark (Middle scores). Allowing additional scores in (-T,T) only strengthens the NSF sup- pression bound because the denominator S increases, and it does not weaken the softmax lower bound equation 23 on the high to high ratio since that ratio is independent of other coordinates. The softmax low suppression supremum equation 24 is still attained by driving all non-high and non-j scores to - 0o, so the temperature constraints remain necessary.
20

---

Published as a conference paper at ICLR 2026
(a)
(b)
(c)
(d)
Figure 7: Ablation study applying the softmax function with temperature scaling to the attention
scores: (a) attention distribution of the proposed ASMIL, (b) softmax with T = applied to the
anchor model, (c) softmax with = 4 applied to the anchor model, (d) softmax with T applied to the anchor model.
F ALTERNATIVE TO NSF IN ANCHOR MODEL
F.1 SOFTMAX WITH TEMPERATURE SCALING
A straightforward approach mitigating over-concentraation is apply softmax with temperature scaling (Hinton et al. 2015; Ye et al. 2024, Yang et al.; (024: 2025). This can indeed yield less con- centrated attention distribution; however, as we observe this section, a large temperature produces an overly smooth distribution, approaching a uniform distribution. This makes all tiles nearly indis- tinguishable, effectively reducing the operation mean pooling and compromising interpretability. To illustrate this, we conduct experiments on the BRACS dataset using the same training protocol as in Section summarize the results in Table and visualize the attention maps Figure
(b)
(C)
670
VOINA
(d)
EE)
Figure 8: Histograms of (a) raw attention scores, (b) attention distribution obtained by the softmax function with temperature T 1, (c) 2, (d) = (e) T = 8, and (f) attention distribution computed using an NSF, The Y-axis is displayed on logarithmic scale for better visualization.
Furthermore, clarify the differences between the NSF and softmax, plot the histograms of the attention scores-4e (a) outputs from the NSF and softmax with various temperature scalings- Figure g As shown, the saturation property of the NSF suppresses excessively large values.
21

---

Published as a conference paper at ICLR 2026
Table 4: Subtyping performance BRACS, when apply softmax with temperature scaling to the anchor model.
BRACS
Normalized
Softmax T-2
Softmax T=4
Softmax
Sigmoid
T-8
Fl score 1
Fl score
AUC
FI score
AUC
Fl score
AUCT
0.781
0.667 2.
ore
0.86020 o27
0.712go.00 029
0.876100012
0.688 007
0.85840.00
Table 5: ASMIL performance when replacing NSF with entmax on CAMELYON-16.
CAMELYON-16
Metric
NSF
entmux,... (sparsemaa))
enrmax.
enimax,. (enimax 13)
enimax 85
enimax. (sotimax)
FI score
0.338
0.927
0,337
0.310
0.912 wore
AUC
0. 9.5
2.93
0.960 8.451s
0.925
a.963
Time per epoch
6,340s
8.451s
8.452s
8.450s
6.336s
F.2 ENTMAX
Entmax is a family of mappings that convert a score vector z E Id into probability vector E A by maximizing linear score plus Tsallis-a entropy Tsallis (1988) H.:
entmax.. (2) argmaz pesd p+ H (p).
(32)
The solution admits closed form
7) with Σa==
(33)
where T is a threshold chosen so that the probabilities sum to one. As limiting cases, a + 1, yields softmax, and while a = yields sparsemax (Martins & Astudillo, 2016).
While entmax offers controllable sparsity, two drawbacks are pertinent to MIL-based WSI analysis:
(i) Lack of selective flattening, entmax is monotone 2 on its active support and does not explicitly equalize top-probability entries. (ii) Higher computational cost. Computing T in Equation (33 requires the bisection method, which adds non-trivial overhead vs. NSF's fully closed-form nor- malization. These differences matter for MIL on WSIs, where multiple correlated tumor foci can be present: we prefer mechanisms that both discourage over-peaky attention and keep computation predictable. We replaced NSF with enlmax, inside ASMIL and swept E {2, 1.75, 1.5, 1.25, 1). For a = 1, we used Py Torch softmax; for a >> 1, we solved for T via bisection. The implementation follows the reference code from DeepSPIN' 1 All other hyperparameters, model, and data pipeline were kept fixed. We report results on CAMELYON-LE 16 in Table 5 As seen, across a, entmax underperforms NSF both F1 and AUC and incurs 33.5% increase in epoch time vs. NSF
G APPLYING NORMALIZED SIGMOID TO THE ONLINE MODEL
One might question the rationale behind applying the NSF to the anchor model while using the soft- max function for the online model during training. To investigate this design choice, we experiment with applying the NSF to both the online and anchor models and evaluate the model's subtyping per- formance on the CAMELYON-1 and BRACS datasets. The results, presented in Table 6 reveal a F1 score drop of over 6% the BRACS dataset. We attribute this degradation to the inherent char- acteristics of the sigmoid function: when it saturates, its gradients diminish, leading to vanishing gradients in the attention mechanism and thereby impairing the learning process.
To further investigate the potential of applying NSF in the online model, we consider the following mixed attention variant:
SMX (z) + (1 - () or .SF (=), d((=) { a SMM
(34)
where < = (E) and ę is a trainable scalar that balances the contributions of the softmax and NSF mappings, initialized with & 0. We evaluate this variant on CAMELYON-16, CAMELYON-17,
https://githhbe com dete//eepppseee sp n/entmax
22

---

Published as a conference paper at ICLR 2026
Table 6: Ablation study the impact of applying the normalized sigmoid (NS) function both the online and anchor models. / indicates that NSF is applied to both models, while X denotes the default setting where NSF is applied only to the anchor model. Subtyping performance is evaluated on the CAMELYON-16 and BRACS datasets using F1 score and AUC. A significant performance drop is observed on CAMELYON-16 when NSF is applied both models.
Dataset
CAMELYON-16
Online NSF
FI score 1
AUCT 1
0.920 20.020 20.200
0.936
20.021
X
0.965
0.985
Dataset
BRACS
Online NSF
Fl score ?
AUC f
0.72620.01
0.805
X
0.781+0.042
0.914+0.014
Table 7: Comparison between ASMIL trained with the standard softmax function in the online model (ASMIL w. Softmax) and with the mixed attention function defined in Equation (34) (ASMIL w. Mixture). The more flexible trainable mapping does not yield improvements over the simpler
softmax baseline.
Dataset
CAMELYON-16
CAMELYON-17
BRACS
Mertie
Fl score
AUC
Fl score
AUC
F1 score
AUC
ASMIL W. SoftMax
0.965+a.02
0.985+c.011
0.689 to.
0.898
1..08-00010 -0.010
0.781-o.c4z
0.914+n.01s
ASMIL W. Mixture
0.953 to.02s
0.972
0.686+a.012 0.889 -0.009
0.774 0.004
0.910
Ç in Equation 34
0.9952
0.9894
0.9963
and BRACS, and report the results in Table 7 The mixed mapping does not outperform the default softmax, and the learned Ç consistently converges values close one, indicating that the online model prefers softmax, which does not suffer from gradient-vanishing issues.
H ALTERNATIVE STABILIZATION METHODS AND WHY THE ANCHOR IS PREFERABLE
Let (x) E A denote the attention distribution for slide x at epoch f, obtained from scores 2, [x) R. We diagnose instability by the -hasnn divergence
(S) = JSD(a (x) ((-1
(35)
which we empirically find remains high when training attention-based MIL with only bag-level labels. We present natural alternative that targets this instability and explain why the anchor model preferred.
H.1 ALTERNATIVE: PER-SLIDE TEMPORAL ENSEMBLING OF ATTENTION
Maintain a per slide exponential moving average (EMA) of past attentions and penalize deviation from it:
(x) = pā=-((x) + (1 -1-08 (1-p)a(()) PE (0, 1); Cas(*) Kl(a((o)ls(((((t))
(36)
The EMA target changes slowly when p is close to one, which directly shrinks spoch-to-epocld drift of a4 and reduces JSD( ()-)). However,
(i) It has to maintain a length-N vector per slide. For S slides and average N tiles, memory is (SSN) floats, which be substantial for gigapixel WSIs and prevent scaling to larger datasets. (ii) The EMA target still uses softmax normalization, which cannot achieve selective flattening across informative tokens; see Theorem
23

---

Published as a conference paper at ICLR 2026
H.2 WHY ASMIL'S ANCHOR IS PREFERABLE
We highlight two main reasons for using an anchor model to stabilize the attention distribution rather than relying on temporal ensembling.
NSF provides selective flattening that softmax cannot match.
Replacing softmax with the normalized sigmoid function (NSF) in the anchor yields a unf (æ)) which equalizes probabilities among truly high-score tiles while suppressing low-score
By The-
orem 1 no single softmax temperature can realize both behaviors across a broad class of score vectors. Consequently, methods that retain softmax-based targets inherit these limitations.
Memory and implementation simplicity.
The anchor-based approach adds only extra forward pass and maintains an exponential mov- ing average (EMA) of the anchor parameters during training. It does not require storing per-slide attention distributions, making the approach scalable to large WSI datasets.
Thus, an anchor model is preferable for scalable training on large MII. datasets and for preventing aftention over-concentration.
I WHY MATCHING THE TEACHER (ANCHOR) MODEL'S SOFTMAX FEATURE VECTOR CANNOT STABILIZE THE ATTENTION DISTRIBUTION
Table 8: Ratio of affinely dependent feature bags in the CAMELYON-16, CAMELYON-17, and BRACS datasets; most bags affinely dependent.
Dataset
CAMELYON-16 CAMELYON-17 BRACS
The ratio affine dependent feature bags
99.24%
99.80%
96.08%
In this section, we show why matching the softmax of the bag-level feature is suboptimal strategy for stabilizing attention distributions. To this end, prove that recovering the attention vector a by matching softmax((" X) is, in general, ill-posed: the map f A K + Ad, defined by f(a) softmax(a" with X E kxdd fails to be injective when the feature matrix X is affinely dependent.
Proof. Assume the rows T1, ..,..., UK E Rd of X are affinely dependent. By definition there exists nonzero vector R such that
K
Σ c 0 and
b5. 0.
Let a E A be any probability vector and choose e > 0 small enough that a' = a + eØ satisfies
af 2 0 for every i. Note Σ, = E, a; + + v/a = so a' E A Since Σ thz; = 0 we have
(a') = Tx + e X TTx
Therefore
f(a') = softmax((a')" = softmax(c axxx==xx-x=== = f(a).
Because ub f 0 and e + 0 we have a' + a, hence f is not injective.
Thus, matching the softmax of the bag feature cannot reliably recover stabilize the attention distributions when the feature bag is affinely dependent. Table & confirms that most feature bags extracted by VIT-S (Kang et al. 2023) from WSI datasets indeed affinely dependent.
24

---

Published as a conference paper at ICLR 2026
Table 9: The Fl score and AUC of different MII. approaches two WSI subtyping datasets.
PathGen Chieen--liiVVIEEE
Dataset
CAMELYON-16
CAMELYON-17
Method
Fl score f
AUC
FI score
AUC 1
Clam-SB
0.94120.014
0.960
0.62220.081
0.890
TransMIL
0.951 024
0.968=0.022
0. 0.656
0.892
DSMIL
0.895
0.049
0.58220.c0
0.887
IBMIL
0.935 014
0.953 009
0. (6292..0s
0.884
MHIM-MIL
0.946
0.984
O. 0.5944+0.000
0.912
ABMIL
0.953+o.08 0188
0.972+n.010
0.6 0.610 +0.025
0.864
AEM
0.967 025
0.988
0.688 1:0 016
0.905 = 000
ASMIL
0.974
0.99022.014
0.699 1.0.0.00 0.929=2.om
UNI-VIT-L.
Method
Fl score
AUC 1
Fl score
AUC 1
ABMIL
0.968 011
0.996=0.0mm 0.003
0.605 047
0.885=0.0r
AEM
0.975 003
0.998 .008
0.633 024
0.863
ASMIL
0.980
0.998
0.672
0.866
J APPLYING ASMIL TO FEATURES EXTRACTED BY A WSI FOUNDATION MODEL.
In recent years, foundation models have enabled strong open-source feature extractors that markedly improve the performance of computational pathology systems. To assess the generalizability of our approach, we apply ASMIL to features produced by two such extractors, UNe et al. (2024) and PATHGEN-clip Sun et al. (2025), for the subtyping task on the CAMELYON-T6 and CAMELYON- 17 datasets. reported in Table 9 ASMIL consistently outperforms all baseline methods when used with features extracted by foundation models, yielding the best FI and AUC.
K ABLATION STUDY
K.1 ABLATION OF THE CORFFICIENT 3
The coefficient B 0 in Equation balances the stabilization and classification objectives. To assess its impact on final performance, we sweep 3 E {0, 0.1, 0.25, 0.5, 0.75, 1.0, 1.5, 2.0, 2.5, 4, 5} on the CAMELYON-16 and BRACS datasets. Except for 3. all experimental settings are identical to those in Section S.I] We report Fl score and AUC in Figures 9 and I0] results are averaged over five random seeds. Overall, model performance is relatively insensitive to the choice of 3:: both
T
0.7
0.25
0.5
1.5
Figure 9: Ablation study on the coefficient B, on CAMELYON-16 16 dataset.
F1 score and AUC plateau for 8 (0.5,1.5) experiments.
Accordingly, we set B the default for all
K.2 ABLATION STUDY ON NUMBER OF TRAINABLE FEAT TOKENS
In this section, investigate how varying the number of trainable tokens influences model perfor- mance. To this end, we sweep a number trainable tokens the range of [2, 4, 8, 16), and report the corresponding accuracy CAMELYON-16, CAMELYON-17, and BRACS in Table 10 In the
25

---

Published as a conference paper at ICLR 2026
1.00
0.95
AUC
0.90
AUC
0.25
1.5
Figure 10: Ablation study on the coefficient 3, on BRACS dataset.
Table Ablation results on the number tokens on different WSI datasets.
CAMELYON-16
FEAT tokens
2
4
8
16
Fl score f AUC
0..00
0.946+0.0om
0.9
1055000012
0.960=a.om 00G
0.932 20.017
0.973
0.985
0.981 000000
CAMELYON-17
Fl score 1 AUC
0.556 20.022
0.610
0.674
0.689=0.00m
0.784
0.833
0.879
0.898
BRACS
FI score 1 AUC f
0.721 +0.009
0.766+0.0:2
0.781 +0.004
0.782=0.004
0.871+0.00
0.903
0.914+a.004
0.912
experiment, we apply 8 trainable tokens for CAMELYON-16 and BRACS, and trainable tokens on the CAMELYON-I7 dataset.
K.3 ABLATION ON ANCHOR MODEL UPDATE
K.3.1 EFFECT OF ANCHOR MODEL UPDATE FREQUENCY
Table 11: Ablation study anchor model update frequency, where batch-wise updates consistently outperform epoch-wise updates in both F1 score and AUC on BRACS and CAMELYON-16.
Dataset
BRACS
Update
Fl score 1
AUC 1
Epoch
0.742+o.or
0.87 +0.00s
Batch
0.781 20.012
0.914
Dataset
CAMELYON-16
Update
Fl score 1
AUC f
Epoch
0.920
0.936 021
Batch
0.965 20.020
0.984
To assess the impact of anchor update frequency, we compare epoch-wise and batch-wise update strategies on BRACS and CAMELYON-16 (Table II The results show that batch-wise updates consistently deliver superior performance. On BRACS, batch-wise updates improve the Fl score by 3.9% and the AUC by 4.9%. On CAMELYON- 16, the improvement is even more substantial, with the Fl score increasing by 4.9% the AUC by 5.1% These gains confirm that frequent updates enable the anchor model to provide stable and closely aligned attention reference for the online model, leading better performance.
K.4 IMPACT OF THE RANDOM DROP RATE
We evaluated the effect of random token dropping model performance using CAMELYON-16 and BRACS, measuring both Fl and AUC across several trainable-token budgets. Results in Fig- ure show consistent trend: performance rises from low B, peaks around B 0.5, then degrades
26

---

Published as a conference paper at ICLR 2026
1.00
100
1.000
100
1.000
1.0
0.75
2.75
1.50
0.75
316
0.33
8.50
5.00
0350
326
8505
0325
6.90
840
0.9
150
3.0
0.0
140
310
8:01
F1
AUC
F11
8.05
AAC F11
AUC
0.75
0.91
5.755
1.00
0315
AUC
3.4
0:25
0.90
0.0 1.700
1.00 8300
3.66
0.8
1.0
346
0.88 2.000
186 660
3.4
0.23
6.2 0.00
8.0 022
F1
AUC
F1
(b)
AUC F1
(C)
AUC
Figure 11: Ablation study of random drop probability (B) vs. model FI score and AUC on CAMELYON-16 (top row) and BRACS (bottom row). Across both datasets and trainable-toker
settings (a) 4 tokens, (b) 8 tokens, and (c) 16 tokens, the test F1 score and AUC consistently peak around 0.5.
for larger values. This pattern holds across datasets and capacities, indicating a stable trade-off be- tween regularization and information loss.
Mechanistically, moderate token dropping (0.4-0.7) provides useful regularization, encouraging ro- bustness missing context and reducing overfitting to redundant spurious tiles, while excessive dropping increases the chance of discarding diagnostically critical patches and thus harms recall and ranking. We therefore recommend tuning B in the range of 0.4 0.6. In Appendíx [..] we plot test F1 score and AUC across training epochs to demonstrate that random token dropping mitigates overfitting.
ACVIL
30
10 epech
Figure 12: Performance comparison between ABMIL (Ilse al. 2018 ASMIL with random drop (ASMIL W. rd), and ASMIL without random drop (ASMIL w/o rd). Both ABMIL and ASMIL w/o rd show signs overfitting, as their Fl score and AUC peak and then decline. In contrast, ASMIL with random drop maintains stable performance across training, demonstrating that random drop effectively mitigates overfitting.
K.S RANDOM DROP MITIGATES OVERFITTING
To verify that random drop is an efficient regularizer for attention-based MIL WSIs, trained three variants on CAMELYON- 16: (i) ABMIL, (it) ASMIL without random drop, and (ii) AS- MIL with random drop (ours) with B 0.5. The figure reports validation F1 and AUC over training epochs.
both ABMIL and ASMIL without random drop exhibit overfitting: Fl score and AUC rise early, peak, and then decline with continued training. In contrast, ASMIL with random drop maintains high and stable F1/AUC throughout later epochs, with noticeably reduced
As shown in the Figure 12
27

---

Published as a conference paper at ICLR 2026
Table 12: Statistical comparison of ASMIL with and without the anchor model. We report the mean performance over 10 random seeds along with p-values from DeLong tests for AUC and permutation tests for Fl.
Dataset
Model
AUC
F1
PAUC
Pr1
w/o anchor
0.942
0.979
CAMELYON-16
0.013
0.024
w/ anchor
0.967
0.983
w/o anchor
0.642
0.879
CAMELYON-17
0.024
0.035
w/ anchor
0.693
0.899
w/o anchor 0.729
0.866
BRACS
0.012
0.009
w/ anchor 0.784
0.916
run-to-run variability (shaded regions). These trajectories empirically validate that random drop curbs the late-epoch degradation that accompanies weak supervision on CAMELYON-16. This observation aligns with our analysis that overfitting is recurring failure mode for attention-based MIL on WSI datasets.
K.6 SIGNIFICANCE TEST THE EFFECT OF ONLINE MODEL
To assess whether the performance gains from the anchor model are statistically meaningful, we perform paired significance tests between ASMIL with and without the anchor over multiple 10 seeds. For AUC, we apply DeLong's 8 test, and for F1, we use a non-parametric permutation test. Across CAMELYON-I6, CAMELYON-17, and BRACS, the anchor-augmeented ASMIL consis- tently achieves higher AUC and FI than its non-anchor counterpart, and these improvements are statistically significant (p < 0.05) for both metrics on each dataset (see Table T2).
L QUANTITATIVE LOCALIZATION RESULTS AND ADDITIONAL VISUALIZATION
Predicted masks generated follows. For attention-based methods (CLAM (Lu et al. 2021 TransMIL (Shao et al. 2021). DTFD-MIL (Zhang et al., (022). DSMIL (Li et al.) 2021b) and CAMIL Fourkioti et (2024)) we use the tile-level attention distribution. For ASMIL, the per- tile attention distribution is computed by averaging the attention distributions from all FEAT tokens that tile. Unless otherwise noted, we rescale all per-tile scores to [0, 1] and threshold 0.5 to produce binary masks across all methods.
For tumor localization on CAMELYON-I6. we follow the official challenge protocol and report the lesion-level Free-Response ROC (FROC) (iiller 1969 Bunch 1978, Zhang et al.) (025a). Concretely, model outputs are converted to point detections; a detection is counted as a true positive if it lies within 75 um of any ground-truth tumor region (implemented in the official script via distance-transform "evaluation mask"), otherwise it is false positive. We then sweep the detection score threshold to trace sensitivity versus the average number of false positives per normal WSI, and compute the standard CAMELYON-IE FROC score as the mean sensitivity at 0.25, 0.5, 1, 2, 4, FP/WSI.
Quantitative results for FROC, Dice, and specificity are reported Table 13 ASMIL achieves the best FROC and Dice cancerous slides and higher specificity normal slides, yielding fewer false positives and more contiguous lesion maps compared to baselines.
Figure preeeents additional visualizations the CAMELYON-LE dataset. It shows ASMIL atten- tion maps for tumor slides containing both small and large cancerous regions; rows 1 and 3 provide the ground-truth annotations, and rows 2 and 4 show the corresponding attention maps.
28

---

Published as a conference paper at ICLR 2026
Small cancerous region
Large cancerous region
Figure 13: Additional qualitative examples of tumor regions and ASMIL attention maps. Rows and 3 show the ground-truth tumor annotations (cancerous regions outlined in red), and rows 2 and show the corresponding ASMIL attention maps.
Table 13: Localization results on CAMELYON-16
Method
Dice f
Specificity f
FROC 1
CLAM-SB
0.459
0.987
0.4257
TransMIL
0.103
0.999
0.0866
DTFD-MIL
0.525
0.999
0.4712
DSMIL
0.259
0.863
0.4506
CAMIL
0.515
0,980
0.4612
ASMIL
0.586
0.999
0.4941
M COMPUTATIONAL COST
This section reports the computational cost of ASMIL, as well as the additional cost incurred when integrating the anchor model into the baseline methods.
M.1 COMPARISON THE COMPUTATIONAL COST BETWEEN ASMIL AND BASELINE METHODS
We conducted detailed evaluation of the computational overhead introduced by proposed AS- MIL framework, focusing on three primary metrics: floating-pon point operations (FLOPs), training time per epoch, and peak memory consumption. All experiments conditions, specifically a single NVIDIA RTX 5000 GPU coupled with an Intel Xeon W-2265 CPU and 64 GB of RAM, ensuring fair comparison across methods.
executed under uniform hardware
29

---

Published as a conference paper at ICLR 2026
Table 14: Computational cost BRACS (lower is better). We report training time and peak memory per epoch, and inference FLOPs, latency, and memory. ASMIL (ours) delivers efficient inference, cutting compute by 30.6%, latency by 29.2%. and memory by 20.3% compared with TransMIL, while requiring 4x less training memory than MHIM-MIL.
BRACS
Training
Method
CLAM-SB
ABMIL
TransMIL
MHIM-MIL
ASMIL
Time
2.26s
0.95s
5.99s
19.4s
7.49s
Memory
94MB
90MB
340MB
2178MB
570MB
BRACS
Inference
FLOPs
162M
164M
781M
345M
542M
Time
0.45s
0.37s
0.74s
0.40s
0.52s
Memory
69MB
39MB
246MB
61MB
196MB
Table 15: Inference FLOPs, training time per epoch (Time), and memory usage (Memory) for four well-known methods, CLAM-SB, TransMIL. DSMIL, and ABMIL, with and without the anchor model. The anchor model incurs only minor computational overhead. FLOPS are measured using fixed bag size of 2000 instances.
BRACS
Training
Inference
Method
Time
Memory
FLOPs
Time
Memory
CLAM-SB w/o anchor
2.26s
94MB
162M
0.45s
69MB
CLAM-SB W. anchor
2.69s
120MB
162M
0.45s
69MB
TransMIL w/o anchor
5.99s
340MB
781M
0.74s
246MB
TransMIL W. anchor
7.27s
443MB
781M
0.74s
246MB
DSMIL w/o anchor
0.57s
60 MB
103M
1.09s
113 MB
DSMIL W. anchor
0.58s
145MB
103M
1.09s
113 MB
ABMIL w/o anchor
0.95s
90MB
164M
0.37s
39MB
ABMIL W. anchor
1.17s
162MB
164M
0.37s
39MB
During training, ASMIL demonstrates competitive balance between efficiency and computational demand. On average, ASMIL requires 542M FLOPs per batch, which is lower than MHIM-MIL. The training time per epoch for ASMIL is 7.49s, substantially faster than MHIM- MIL (19.4s) and comparable to TransMIL (5.99s). while remaining higher than ABMIL and CLAM-SB. In terms of peak memory usage, ASMIL consumes 570 MB, markedly lower than MHIM-MIL (2178 MB). These results indicate that ASMIL maintains favorable computational profile, offering scalable alternative to more resource-intensin methods.
In inference, ASMIL continues to show strong efficiency. It requires 542M FLOPs, substantially fewer than Trans and comparable to MHIM-MIL. Inference time for ASMIL is 0.52s per epoch, slightly slower than CLAM-SB (0.45s) but faster than MIl. Peak memory usage during infer- ence is 196 MB, markedly lower than TransMIL, highlighting ASMIL's efficient memory footprint relative its computational performance. Overall, ASMIL delivers high-performance multiple- instance learning while keeping computational cost affordable.
M.2 ADDITIONAL COMPUTATIONAL COST INTRODUCED BY ANCHOR MODEL
We conducted a detailed evaluation of the computational overhead introduced by integrating the anchor model into four widely used MII. methods, namely CLAM-SB, TransMiL, DSMIL, and ABMIL, all measured on the BRACS dataset. The results summarized in Table 15
Because no gradients are computed through the anchor model, and only the attention layer is up- dated during training, the computational overhead is small. As shown in Table 15] integrating the anchor model into CLAM-SB, TransMIL, DSMIL, and ABMIL introduces only a modest increase in training time and memory usage, while the FLOPs remain unchanged. For example, training time for CLAM-SB increases from 2.26s to 2.69s and memory usage from 94 MB to 120 MB, with larger models like TlansMII. showing slightly higher overhead. Importantly, during inference, the
30

---

Published as a conference paper at ICLR 2026
Table 16: C-index for WSI-based survival prediction using vision-only MII. models.
Method
BLCA
BRCA
GBMLGG
LUAD
UCEC
ABMIL ICML
0.5581 +0.00++..00..
0,5825 +0.0a5
0.7935-p
0.6121+o.om
0.6667+
TransMIL NeurlPs 2021
0.5885+0.053
0,6140+oo. +0.060
0.7956-a.01s
0.5708 +o.00a
0.6380+o.067
ILRA ICLR
0.5549
0.5705
0.7742_o.o14
0.51
0.650331.. 066
T-M- CVPR
0.5775
0.5476+0.002
0.7757=
0.5711++..m
0.6510
DeepAttnMiSS MIA 2020
0.5646 035
0.5346 200000
0.6750=a.oza
0.4678
0.6259 08G
Patch- Mach
0.6124
0.6375 to0.0s
0.7999
0.5922
0.7212
ASMIL L (Ours)
0.6133+t.oo
1.6396+*o.0s
0.8036=0.0xz
0.6001 +0.099
0.7243 o00008
anchor model is discarded, resulting in identical FLOPs, execution time, and memory consumption compared to the baseline methods. These results demonstrate that the anchor model provides per- formance benefits during training with minimal computational cost and does not affect deployment efficiency, making it an effective and practical addition to existing MIL frameworks.
N SURVIVAL PREDICTION
To assess whether ASMIL is also beneficial for prognosis, we extend ASMIL from slide-level clas- sification to discrete-time overall survival prediction on histopathology WSIs. Following (Liu al., 2025), we apply an incidence-based discrete survival formulation, i.e., the survival times are mapped to C' non-overlapping time intervals, and the model outputs discrete distribution over first-event times.
We follow the experimental setup of (Liu al. 2025), and evaluate five TCGA datasets, namely BLCA, BRCA, LUAD, and UCEC. We use the concordance index (C-index) to evaluate the model's performance; specifically, it measures how often the model assigns higher risk score to a patient who experiences the event earlier. Formally, with little abuse of notations, let 4,, 5,, Rg denote the observed time, event indicator, and predicted risk for patient i, the C-index is defined as
CI Eig 144 ( > R2
(37)
Σx, 1lt. < <tyl6;
where [[-] is the indicator function. A value of CI = 0.5 corresponds to random ranking, and larger values indicate better risk discrimination Yang & Yo (2024) Hamidi Ye (2024) 2025). Following Liu et al. (2025), we compare ASMIL against six vision-oni only WSI survival prediction methods, 2018), TransMIL- (hao al. 2021). ILRA Xiang & Zhang 2023
namely ABMIL (Ise
R'T-MIL Tang et al. 2024). DeepAttMMSLL Yao et al. 2020), and Patch-GCN (Chen et al. 2021 all implemented on top the same CoNCH-derived patch features (Lu et al.) 2023).
Table 16 reports the C-index on each TCGA dataset. ASMIL achieves the highest mean C-index among all vision-only baselines. These results indicate that stabilizing slide-level attention not only improves weakly supervised classification but also yields stronger prognostic discrimination in sur- vival analysis.
O EVALUATE ASMIL OVER NON-W- WSI DATASET
Table 17: MII. dataset statistics.
Dataset
Domain
Bags (pos/neg)
Total instances
Dins/inst.
MUSKI
Drug activity
92 (47/45)
476
166
MUSK2
Drug activity
102 (39/63)
6598
166
TIGER
Images (Blobworld segments)
200 (100/100)
1220
230
FOX
Images (Blobworld segments)
200 (100/100)
1320
230
ELEPHANT
Images (Blobworld segments)
200 (100/100)
1391
230
31

---

Published as a conference paper at ICLR 2026
To demonstrate ASMIL's upplicability beyond WSI, evaluate it five classic multiple-instance learning (MIL) benchmarks: MUSKI Chapmman & Jain (1994a) and MUSK2 CCapman Jain (1994b), where each bag is molecule and instances its low-energy 3D conformations described by 166 attributes (a bag is positive at least one conformation is active); and the image MII. datasets TIGER, FOX, and EL ELJEPHANT Andrews al. (2002), where each bag is Corel image segmented into "Blobworld" regions (instances) with 230-D color/lecturc/hhape features (a bag is positive if at least one segment contains the named animal). Standard size statistics are reported in Table 17
Table 18: Results on the small MIL benchmark datasets.
Methods
MUSKI
MUSK2
FOX
TIGER
ELEPHANT
ABMIL ICMI
2016
0.9162o 118
0.928 108
0.952 051
0.95320.00 042
0.96920.om 030
DSMIL CVPR
0.959
0.95220
0.939 10090.0.o
0.951 o5a
0.989
TransMIL
NeuriPS
2021
0.927 +0.003
0.877 +21
0.9444+0.008 +00000
0.963++.oom +0.0442
0.979+o.0sm
DEMIL NeuriPS
0.963
0.961 +o
0.941 +o
0.965
0.969+0.0s
RGMIL Neurip 2023
0.968 o60
0.963 +0 048
0.954 .48
0.949 + 047
0.965
PSMIL
0.968
0.968
0.942
0.947 047
0.985 o3e
ASMIL
(Ours)
0.971 00000
0.968 a. o58
0.961 t0.025
0.969 00.037
0.985
Since these datasets are relatively balanced, following Du et al. (2025), we report accuracy as the primary metric. We train for 40 epochs with the Adam optimizer (Kingma & Ba 2014) and learning rate of 0.0005.
We compare ASMIL against six MIL methods ABMIL (Ilse al. 2018), DSMIL (Li et al. 2021b), TlansMIL. (Shao et al. 2021). DEMIL (Tang et al. (2023)), RGMIL (Du et al. 2023 and PSMIL (Du al.
2025) and report accuracies in Table 18 As shown, ASMIL outperforms
all baselines on 4 of 5 datasets, demonstrating strong performance on non-WSI benchmarks.
P ATTENTION DYNAMICS OF DIFFERENT MIL METHODS ON VARIOUS DATASETS
In this section, illustrate that the issue of attention convergence the WSI dataset is not unique to the ABMIL and CAMELYON-LE 16 datasets. To this end, similar to the method we describe in Figure ! we plot the JSD of two attention distributions between two consecutive epochs.
32

---

Published as a conference paper at ICLR 2026
P.1 CAMELYON- 16 DATASET
ABMIL
ABMIL
+Anchor +NSF
CE
JSD
0.4
JSD 04 (4+11)
0.5
0.3
0.2
0.1
-0.5
0.0
10
epochs
30
40
50
Anchor NSF
ABMIL+
Figure 14: Visualization of attention dynamics on normal WSI for ABMIL vs. ABMIL + anchor + NSF
ABMIL
1.
ABM
Anchor +NSF
CE
JSD
0.5
02
0.0
01
005
-1.0
0.
10
20
epochs
30
40
50
ABMIL+ + Anchor+NSF
Figure 15: Visualization of attention dynamics on a tumor WSI for ABMIL vs. ABMIL + anchor
NSE
33

---

Published as a conference paper at ICLR 2026
TransM
1.0
TransMIL
Anchor +NSF
CE
JSD
0.75
0.5
30.0
0.50
0.25
0.5
-1.0
20
epochs
30
40
50
Trans Anchor
Figure 16:
Visualization of attention dynamics on normal WSI for TransMIL vs. TransMIL
anchor + NSF.
ThansMIL
2
Anchor+
JSD
0.75
0.50
0.250
0,00
epochs
TransMIL+ Anthor+NNS NNSF
Figure 17:
Visualization of attention dynamics on a tumor WSI for TransMIL TransMIL.
anchor + NSF
34

---

Published as a conference paper at ICLR 2026
DSMIL
DSMIL
Anchor + NSF
CE
JSD
JSD (E
0.2
10
20
epochs
40
50
DSMIL+ Anchor+ NSF
Figure 18:
Visualization of attention dynamics on normal WSI for DSMIL vs. DSMIL + anchor
+ NSF
DSMIL
1.5
DSM
Anchor NSF
CE
JSD
0.2
(aD(aa/ (6+11
0.5
0.1
0.0
epochs
Anchor+
DSMIL+
Figure 19: Visualization of attention dynamics on a tumor WSI for DSMIL vs. DSMIL + anchor
NSF.
35

---

Published as a conference paper at ICLR 2026
P.2 BRACS DATASET
ABMIL
ABMIL
Anchor +NSF
JSD
0.100
15D 04 (4H))
1.0
0.075
CR
0.5
0.050
0.0
0.025
-0.5
0.000
20
epochs
30
40
50
ABMIL+ Anchor+NSF
Figure 20:
Visualization of attention dynamics normal WSI for ABMIL vs. ABMIL + anchor
+ NSF
ABMIL
ABMIL
Anchor +NSF
CE
JSD
0.100
(a4 (46-1)
0.075
0.050
0.025
0.000
10
20
epochs
30
40
50
ABMIL.+ Anchor+NSP
Figure 21: Visualization of attention dynamics tumor WSI for ABMIL ABMIL anchor
+ NSF.
Despite these advances, several avenues remain open for future investigation:
36

---

Published as a conference paper at ICLR 2026
TransMILL
0.100
TransMIL
Anchor +SSF
CE
JSD
0.075
00
CE
0.050
(
0.025
0.000
10
20
epochs
30
40
50
TransM Anchor+
Figure 22:
Visualization of attention dynamics on a tumor WSI for TransMIL vs. TransMIL.
anchor + NSF.
TransMILL
TransMIL
+Anchor +NSF
CE
JSD
0.10
(5D(aulou ((()
CR
0.05
epuchs
30
TransMiLL+ AnchortNSE
Figure 23:
Visualization of attention dynamics tumor WSI for TransMIL TransMIL
anchor + NSF
ASMIL employs an EMA-updated anchor model to stabilize attention dynamics, but this introduces additional computational overhead. An important direction is the development of intrinsic train-
37

---

Published as a conference paper at ICLR 2026
DSMIL
1.0
DSMIL
+Anchor +NSF
CE
JSD
0.04
(SD )
0.5
0.02
0.0
0.00
10
epochs
40
50
Anchor+ Eachor+NSF NSF
DSMIL+
Figure 24: Visualization of attention dynamics on a tumor WSI for DSMIL vs. DSMIL + anchor
NSF.
DSMMI
DSMIL
+Anchor +NSF
CE
JSD
0.06
(SD(cul(oss)
CE
0.04
0.02
0.00
20
epochs
40
DSML+ Anchor+NSP
Figure 25: Visualization of attention dynamics tumor WSI for DSMIL DSMIL anchor NSF
ing strategics, such as regularization, that achieve comparable stability without auxiliary modules, thereby improving efficiency in large-scale WSI applications.
38

---

Published as a conference paper at ICLR 2026
Table 19: Rate of cancerous WSIs without missed regions on the CAMELYON-16 dataset.
Method
Clam
TransMIL
DTFD-MIL
DSMIL
CAMIL
ASMIL
Rate 46.93%
3.246%
50.34%
48.22%
49.78%
54.63%
Q LIMITATIONS AND FUTURE WORK
Meanwhile, as more advanced regularization techniques such as MIL-dropout (Zhu al. 2025) continue to emerge, integrating them into the ASMIL framework represents a highly promising direction for future work. Such enhancements could further improve the model's generalization ability while yielding more faithful and stable attention distributions.
Furthermore, a limitation of our approach is that AS- MIL can fail by assigning low attention to tiny foci and small tumor regions (see Figure 26), particu- larly when large and small cancerous regions coexist within a single WSI. We fix the attention threshold at 0.5 and count cancerous WSI successfully lo- calized all regions inside the tumor annotation ex- ceed this threshold; the success rates reported in Table T9 ASMIL achieves the highest success rate, which attribute to the NSF in the anchor model that mitigates over-concentrated attention. This in- dicates room for improvement. Nevertheless, com- pared with published baselines, ASMIL'S attention maps consistently achieve higher Dice and FROC scores. One avenue to further enhance localization performance is to bootstrap training with a mixture of synthetic data and real WSI data. These directions are beyond the scope of this work and will be inves- tigated in future rescarch Wu et al. (202)); Chi al. 2024).
R LLM USAGE STATEMENT
LLM used only for grammar and wording edits; no generation of ideas, methods, analyses, results, citations. The authors reviewed all edits and accept full responsibility.
Figure 26: Left: annotated WSL. Right: at- tention map generated by ASMIL, which fails to assign high attention to all tumor re- gions, highlighted by the green arrow.
39


Reviews:
[{"review_id": "TOCuL9DWq5", "reviewer": "Reviewer_JdSj", "summary": "The dynamic instability of attention mechanisms leads to overfitting and excessive concentration in attention distribution. This paper proposes a framework called Attention-Stabilized Multiple Instance Learning, which employs an anchor model to stabilize attention. It replaces the softmax function in the anchor with a normalized sigmoid function to prevent over-concentration and mitigates overfitting through labeled random dropout. The approach demonstrates innovation, and final experiments show promising results.", "questions": "1、Although the relationship between the anchoring model and the online model has been theoretically elucidated, their interactive mechanisms in practical applications may influence the final outcomes. Are there further details regarding how the anchoring model guides the attention mechanism of the online model? Does the anchoring model consistently demonstrate robust stability across diverse training scenarios?\n\n2、This paper primarily validates the effectiveness of the ASMIL model in subtype classification and tumor localization tasks, but the multimodal learning framework can also demonstrate advantages in other tasks. Can this method be extended to other tasks?\n\n3、The author proposes reducing overfitting risk by randomly discarding some feature labels, but this mechanism may impact model performance differently under various training settings. Could you provide more experimental data on how the label discard rate (e.g., the B value) affects model performance? Is there a recommended optimal discard rate, or should the discard proportion be adjusted based on different datasets?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1、By retaining the same attention module architecture and identical inputs, updates are performed using exponential moving averages instead of back propagation, thereby resolving the dynamic instability issues inherent in previous attention mechanisms.\n\n2、Replacing softmax with a non-symmetric normal distribution in the anchoring model effectively mitigates attention over-concentration, ensuring stable and uniform attention distribution.\n\n3、The paper introduces a feature token dropout mechanism, which randomly drops a portion of feature tokens during training while retaining all tokens during inference to mitigate overfitting.\n\n4、The article provides a comprehensive overview, and the experimental content is rich.", "weaknesses": "1、The motivational section in Subsection 3.3 bears similarities to discussions on issues with attention mechanisms in related work. It requires further clarification of differences or highlighting of innovative aspects, and may also be subject to corresponding revisions or consolidation.\n\n2、The paper compares its results with multiple baseline models, but some of these comparison models (such as CLAM-SB and DSMIL) lack in-depth discussion regarding the rationale for their selection and their applicability. Key differences in design between these models and ASMIL, their respective strengths and weaknesses, and how these factors influence performance across different types of tasks remain unexplored.\n\n3、The font in the model diagram (Figure 2) is partially obscured, and some words appear in a smaller font size that is not sufficiently clear.\n\n4、The paper proposes updating the anchor model via EMA to stabilize the attention distribution of the online model, yet it does not delve deeply into the relationship between these two components. Specifically, the interaction mechanism between EMA updates and backpropagation, as well as the performance of the online model during inference, are not sufficiently elaborated in the method description.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jo2UL9lgjN", "reviewer": "Reviewer_e83y", "summary": "This paper addresses key limitations of attention-based multiple instance learning (MIL) in whole slide image (WSI) analysis, a critical task in computational pathology. The authors first identify three major challenges of existing attention-based MIL methods: (PI) unstable attention dynamics (attention distributions oscillate across epochs instead of converging, quantified via Jensen-Shannon divergence), (PII) over-concentrated attention distribution (excessive focus on a few tiles), and (PIII) overfitting (due to limited WSI training samples). To tackle these issues simultaneously, the authors propose ASMIL (Attention-Stabilized Multiple Instance Learning), a unified framework with three core components: (1) an anchor model updated via exponential moving average (EMA) to stabilize attention dynamics by aligning the online model’s attention with the anchor’s via Kullback-Leibler (KL) divergence; (2) a normalized sigmoid function (NSF) in the anchor (replacing softmax) to prevent attention over-concentration, supported by theoretical analysis (Theorem 1); and (3) token random dropping to mitigate overfitting. Extensive experiments on three public WSI datasets (CAMELYON-16, CAMELYON-17, BRACS) and five non-WSI MIL benchmarks demonstrate that ASMIL achieves state-of-the-art (SOTA) performance (up to 6.49% F1 score improvement over baselines) and consistently boosts the performance of existing attention-based MIL methods (up to 10.73% F1 gain) when integrating its anchor and NSF modules.", "questions": "1. **Anchor Model Hyperparameter Tuning**: The ablation study shows that an EMA factor \\( m = 0.999 \\) yields the best performance (Table 10). However, the authors do not explain why this specific value is optimal across datasets (e.g., CAMELYON-16 vs. BRACS) or whether \\( m \\) should be adaptively adjusted for WSIs with varying tile counts or sparsity (e.g., slides with <5% tumor regions). Could you provide insights into the sensitivity of \\( m \\) to dataset characteristics?\n\n2. **Token Random Dropping Rationale**: The paper finds that a drop rate \\( B \\approx 0.5 \\) is optimal (Figure 11). However, this value is derived from experiments on CAMELYON-16 and BRACS—does this optimal rate hold for other WSI datasets with different tissue types (e.g., prostate or lung WSIs) or non-WSI MIL tasks (e.g., MUSK)? Additionally, how does token dropping interact with the anchor model (e.g., does dropping affect the anchor’s ability to learn stable attention)?\n\n3. **NSF vs. Alternative Normalization Functions**: The paper compares NSF to softmax (with temperature scaling) and entmax (Table 4, Table 5) but does not explore how NSF performs against other attention normalization methods (e.g., sparsemax, Tsallis entropy-based functions) in WSI scenarios with extreme class imbalance or high tile redundancy. Could you provide additional experiments or analysis to confirm NSF’s robustness across diverse WSI data distributions?\n\n4. **Addressing Tiny Tumor Localization**: The authors note ASMIL’s limitation in focusing on tiny tumor foci (Figure 26). Beyond synthetic data, are there immediate modifications to ASMIL—e.g., adjusting the KL divergence weight \\( \\beta \\) for small regions, or adding a small-region attention regularization term—that could mitigate this issue? Have you quantified how often this limitation occurs (e.g., percentage of WSIs with missed tiny foci) across datasets?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **Novel Identification of a Critical Underexplored Problem**: The paper is the first to systematically identify and quantify \"unstable attention dynamics\" (PI) in attention-based MIL for WSI analysis. By measuring JSD between consecutive attention distributions and visualizing convergence failures (e.g., Figure 1), the authors highlight a previously overlooked limitation that harms both performance and interpretability—an important contribution to understanding MIL’s failure modes in WSI tasks.\n\n2. **Unified and Theoretically Grounded Framework**: ASMIL effectively addresses three distinct challenges (PI–PIII) in a single framework, avoiding fragmented solutions. The NSF’s superiority over softmax (Theorem 1) provides rigorous theoretical justification for preventing attention over-concentration, while the EMA-based anchor model’s design (as a stable reference) is well-motivated and distinct from existing EMA teacher models (e.g., MHIM-MIL) that focus on hard-instance mining rather than attention stabilization.", "weaknesses": "1. **Incomplete Analysis of Edge-Case Performance**: While ASMIL outperforms baselines overall, it lags slightly behind SOTA on a specific setting: using an ImageNet-pretrained ResNet-18 backbone on CAMELYON-17, ASMIL’s AUC is 0.002 lower than the best baseline. The authors do not investigate this edge case—e.g., whether it arises from dataset-specific characteristics (e.g., CAMELYON-17’s multi-center staining variability) or interactions between the backbone and ASMIL’s components—weakening the claim of \"SOTA across all datasets.\"\n\n2. **Limited Discussion of NSF’s Application Scope**: The authors explain that NSF is applied only to the anchor (not the online model) to avoid vanishing gradients, but they do not explore potential workarounds (e.g., gradient clipping, modified sigmoid variants) to enable NSF in the online model. This missed opportunity raises questions about whether NSF could further improve performance if integrated more broadly, rather than being restricted to the anchor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DIqdRZGm3F", "reviewer": "Reviewer_Pd6q", "summary": "This paper investigates the instability of the attention scores in different training epochs during MIL training. In addition, the authors try to solve two more problems: over-concentrated attention distribution and overfitting. They propose several corresponding solutions: 1) an anchor model updated by EMA, 2) a normalized sigmoid function that replaces softmax, and 3) an effective token drop strategy. The comprehensive experiments have validated the effectiveness of this proposed method.", "questions": "Please refer to the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThe authors investigate a commonly neglected problem: the unstable attention distribution throughout the training that might be detrimental to MIL model training.\n\n2.\tThe paper is well-written and easy to understand. The presentation of this method is quite clear.\n\n3.\tThe solutions are intuitive and simple yet effective. The proposed solutions are not only a model but several plug-in modules that could be reused in other MIL models, which might broader impact in the community.\n\n4.\tThe experiments are quite comprehensive and provide most of the ablation studies required to demonstrate the effectiveness of this method.", "weaknesses": "1.\tIn the model design, the online model is updated through backpropagation, and the anchor model is updated through EMA. They are both randomly initialized, and why the anchor model would help? The anchor model will start with random guess (which at least will not be near the correct attention distribution), and the EMA is somewhat restraining the anchor model within the starting point. Would it be better to start EMA after several epochs (so that the model has learnt something meaningful) to remove the noises of the first few epochs? \n\n2.\tThe authors are suggested to give a more comprehensive evaluation of the effectiveness of random drop strategy (e.g., whether this strategy is capable with other models, not just the proposed ASMIL). Also, it would be even better if the authors could compare this strategy with this paper published in ICML2025 “How Effective Can Dropout Be in Multiple Instance Learning?” This paper proposed to drop top-k most important instances. Overall, this would increase the solidity of the proposed strategy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fzj0wzWhzu", "reviewer": "Reviewer_YCZ5", "summary": "The paper proposes ASMIL, a modular framework to stabilize attention in attention-based MIL for weakly supervised WSI diagnosis. The method combines: (1) an EMA-updated anchor branch that provides a stable target attention distribution; (2) a normalized sigmoid attention in the anchor to reduce softmax over-concentration; and (3) token dropping for regularization. The anchor is discarded at inference. Experiments on BRACS and CAMELYON16/17 show consistent improvements over strong baselines and gains when plugging the module into existing MIL models. The paper highlights attention instability across training epochs and uses divergence-based measures and visualizations to motivate the approach. Anonymous code is released to support reproducibility.", "questions": "- How does ASMIL’s anchor compare to DINO/DINOv3 EMA teacher and anchoring strategies (objective/targets, momentum/lag tuning, collapse avoidance)? Do similar trade-offs arise?\n- Can the authors provide statistical correlation/significance analyses linking stability metrics to performance across seeds and hyperparameter sweeps? Are there identifiable over-stabilization regimes?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper focused on attention instability in attention-based MIL for WSI and provides quantitative and qualitative evidence (epoch-wise measures, attention maps) to motivate stabilization.\n- The proposed module is simple and practical (EMA anchor + normalized sigmoid + token dropping), integrates with diverse MIL models, and adds no inference-time cost.\n- Empirical results show consistent improvements across datasets and baselines, and the paper includes useful training/resource details and hyperparameter guidance (e.g., $m$ and $\\beta$ ranges).", "weaknesses": "- Related work could more clearly situate ASMIL within teacher/EMA anchoring in self-supervised ViT literature (e.g., DINOv3 [1], BYOL [2]). Since DINOv3 discusses anchoring trade-offs (stability vs. adaptation), collapse risks, and EMA lag, a brief positioning of ASMIL relative to these mechanisms would help readers contextualize the anchor’s role and expected behavior.\n- The stability–performance link is motivated but not statistically established: Including correlation analyses across seeds and hyperparameter sweeps and reporting significance would make the claim that reducing attention instability improves downstream performance more convincing.\n\n[1] Siméoni, Oriane, et al. \"Dinov3.\" arXiv preprint arXiv:2508.10104 (2025).            \n[2] Grill, Jean-Bastien, et al. \"Bootstrap your own latent-a new approach to self-supervised learning.\" Advances in neural information processing systems 33 (2020): 21271-21284.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)