Do Multiple Instance Learning Models Transfer?
Authors: Daniel Shao
Richard J. Chen
Andrew H. Song
Joel Runevic
Ming Y. Lu
Tong Ding
Faisal Mahmood
Venue: ICML 2025 spotlightposter (2025)
Keywords: Multiple Instance Learning
Transfer Learning
Finetuning
Computational Pathology
Primary area: applications->health_medicine
TL;DR: We investigate whether Multiple Instance Learning models can transfer to new tasks

Abstract: Multiple Instance Learning (MIL) is a cornerstone approach in computational pathology for distilling embeddings from gigapixel tissue images into patient-level representations to predict clinical outcomes. However, MIL is frequently challenged by the constraints of working with small, weakly-supervised clinical datasets. Unlike fields such as natural language processing and computer vision, which effectively use transfer learning to improve model quality in data-scarce environments, the transferability of MIL models remains largely unexplored. We conduct the first comprehensive investigation into transfer learning capabilities of pretrained MIL models, evaluating 11 MIL models across 19 pretraining tasks spanning tissue subtyping, cancer grading, and molecular subtype prediction. We observe a substantial performance boost with finetuning pretrained models over training from randomly initialized weights, even with domain differences between pretraining and target tasks. Pretraining on pan-cancer datasets enables consistent generalization across organs and task types compared to single-disease pretraining. Remarkably, this pan-cancer pretraining leads to better transfer than that of a state-of-the-art slide-level foundation model, while using only 6.5\% of the training data. These findings indicate that MIL architectures exhibit robust adaptability, offering insights into the benefits of leveraging pretrained models to enhance performance in computational pathology.

Do Multiple Instance Learning Models Transfer?
Daniel Shao Richard J. Chen Andrew H. Song² Joel Runevic? Ming Y. Lu 12 Tong Ding²
Faisal Mahmood 2
Abstract
0.78
Multiple Instance Learning (MII.) is comeestone approach in computational pathology (CPath) for generating clinically meaningful slide-level em- beddings from gigapixel tissue images. However, MII, often struggles with small, weakly super- vised clinical datasets. In contrast fields such NLP and conventional computer vision, where transfer learning is widely used to address data scarcity, the transferability of MIL models re- mains poorly understood. In this study, we sys- tematically evaluate the transfer learning capabil- ities of pretrained MIL models by assessing 11 models across 21 pretraining tasks for morpho- logical and molecular subtype prediction. Our results show that pretrained MIL models, even when trained on different organs than the tar- get task, consistently outperform models trained from scratch. Moreover, pretraining on pancancer datasets enables strong generalization across or- gans and tasks, outperforming slide foundation models while using substantially less pretrain- ing data. These findings highlight the robust adaptability of MIL models and demonstrate the benefits of leveraging transfer learning to boost performance in CPath. Lastly, we provide a re- source which standardizes the implementation of MIL models and provide model weights for FEATHER, P0- pretrained ABMIL model at http://ghhhbb..ommmmmmmdddbbbbbd
Average performance
0.74
0.72
0.70
PC-108 Random
0.68
ABMIL
DFTO
WING
DSMIL
TransMIL
RRT
mransformer
ILRA
mean
max
CLAM
MIL architectures
Figure I. Average performance with supervised pretraining random initialization. Performance of MIL models trained from random initialization (black) initialized with weights from model pretrained (red) on a 108-class pancancer task (n=3,944 944 WSIs). Performance is averaged across the classification tasks, using AUROC for the binary classification tasks, weighted kappa grading, and balanced accuracy for the multiclass tasks.
whole slide images (WSIs), pose unique challenges due to their gigapixel resolutions and sparse diagnostic regions. MIL framework addresses these challenges by (1) using a pretrained encoder to tokenize WSI into patch-level features, and (2) using trainable aggregator to pool these patch features into final slide-level representation for downstream classification (Song et al., 2023; Campanella et al., 2024).
The
1. Introduction
Multiple Instance Learning (MIL) has been the foundational paradigm in computational pathology (CPath) for over decade. Digitized human tissue sections, referred to as
While substantial progress has been made developing powerful pretrained patch encoders for pathology (Chen et al., 2024a; Vorontsov al., 2024: Xu et al., 2024b; Lu et al., 2024), learning effective aggregation scheme re- mains an open challenge due to the relatively small train- ing cohorts and sparse diagnostic regions characteristic of CPath datasets. As a result, new MIL architectures have been continually emerging since 2015, with each approach introducing new inductive biases improve data efficiency
"asssachusetts Institute of Technology. Cambridge MA, USA Harvard University, Boston MA, USA. Correspondence to: Daniel Shao <dshao@ mit.eduz, Faisal Mahmood <FaisalMah- mood@bwh.harvar..c
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Do MIL Models Transfer?
and model generalizability during training.
whether models can transfer different organs, disease indications, or task types, does transfer learning always outperform training from scratch, and more. We summarize our main findings below:
Despite the overwhelming research interest in MIL architec- ture development and well-recognized benefits of transfer learning in low-data regimes in the biomedical community, surprisingly little is known about how well MIL models transfer in CPath. To date, random weight initialization re- mains a common standard in MIL model development and evaluation. Though random weight initialization is appro- priate for ablating architecture syper-parameter and devel- oping academic benchmarks, given the enormous challenge of bringing pathology Al models to clinical adoption us- ing small patient cohorts and widespread use of patch-level transfer in MIL, it is surprising that there is little empirical understanding of how well slide-level transfer performs, e.g. using a MIL model pretrained on one task and transferring it to another task with frozen feature evaluation lightweight finetuning.
Pretrained MIL models consistently outperform MIL. models trained with randomly initialized weights, even when pretrained on
tasks.
Models pretrained on pancancer tasks are data-efficient and generalize effectively across organs and task types. Transfer performance varies with MIL architecture and model size. Larger models benefit more from pre- training, and pancancer pretraining unlocks favorable scaling trends.
The aggregation scheme learned during pretraining pivotal for observed gains from model transfer.
Finally, share GitHub library to standardize MIL im- plementation and simplify loading pretrained weights across various CPath tasks.
This knowledge gap is in stark contrast to the enormous progress made in general machine learning on answering many fundamental questions, such as: How does model architecture (Zoph et al., 2018; Kornblith et al., 2019; Zhou et al., 2021; Renggli et al., 2022), data scale (Kolesnikov et al., 2020), and pretext task (Tripuraneni al., 2020; Er- iesson et al., 2021; Zhao et al., 2021; Mehra et al., 2024) affect transfer performance? Does better supervised perfor- mance lead to better transfer? (Recht et al., 2019; He et al., 2019; Kumar et al., 2022; Fang et al., 2024) When does transfer not work? (Raghu et al., 2019; Jang et al., 2019; Pruksachatkun et al., 2020; You et al., 2021) And what features are being transferred? (Neyshabur et al., 2020)
2. Related work
2.1. MIL in CPath
MIL is a form of supervised learning where models are trained on labeled collections of instances, known as "bags", without individual instance labels. In CPath, MIL has emerged as the predominant framework for modeling WSIs, where tissue regions from a WSI (bag) are first segmented and then divided into non- ovn-llellap image patches (instances), with variable bag sizes of around 1,000 to 10,000 (Campanella et al., 2019; Lu et al., 2021). A MIL model learns a mapping from the collection of image patches of each slide to the slide-level labels, without su- pervision at the patch level. Numerous variations of MIL methods have been proposed, distinguishing from one an- other by the aggregator function (e.g. non-parametric max- pooling (Campanella et al., 2019) vs. Iearned parametric pooling (Ilse et al., 2018)), assumption permutation in- variance (e.g. modeling instances an unordered set (Ilse et al., 2018) vs. ordered sequence (Shao al., 2021)), the precise model architecture (e.g. feedforward layers only (Lu et al., 2021; Ilse et al., 2018; Li et al., 2021) vs. transformer attention (Shao et al., 2021)) and the use of auxilliary objec- tive functions during training (Lu et al., 2021; Zhang et al., 2022). We cover diverse types of architectures to derive general insights on MIL transfer.
Indeed, the transfer of MIL models is also pertinent in light of recent interest in whole slide foundation models, which aim to extract general-purge purpose slide-level representations that can transfer and generalize to challenging clinical tasks in low-data regimes (Shaikovski et al., 2024; Wang et al., 2024; Ding et al., 2024; Vaidya al., 2025). Though self- supervised learning has demonstrated favorable properties for developing slide foundation models via data scaling, model generalization, and emergent capabilities, we contend that an MIL model trained with supervision on large-scale, diverse hierarchical classification tasks can also function as a slide foundation model. We hypothesize this approach can embed WSIs and generalize to diverse, challenging tasks, while requiring substantially less pretraining data than its self-supervised counterparts.
In this work, we investigate transfer learning capabilities of pretrained MIL models in CPath, evaluating 11 MIL models and using 21 tasks for supervised pretraining and transfer evaluation which spans many- and few-shot morphological classification, cancer grading, and biomarker prediction. This work provides roadmap for how to transfer MIL models, answering: which MIL architectures transfer better,
2.2. Large-scale pretrained slide foundation models
With the increasing availability of WSIs, recent works have also explored pretraining MIL architectures to develop slide foundation models that produce general-purpose slide rep- resentations (Chen et al., 2022; Jaume al., 2024; Wang et al., 2024; Xu et al., 2024a; Ding et al., 2024; Vaidya et al.,
2

---

Do MIL Models Transfer?
2025). Instead of leveraging carefully curated labels for task- specific supervised MIL. training. these approaches often rely on large-scale unimodal and multimodal data and task- agnostic objectives such as self- difddisiilation and contrastive learning to learn representations that can be transferred to diverse downstream tasks. We provide a rigorous compari- son of supervised pretraining against such slide foundation models.
tasks. For example, to assess the tranferrability of ABMIL pretrained on NSCLC subtyping, we would first train an ABMIL model from scratch on the NSCLC subtyping task, followed by end-to-end finetuning on the other 18 target tasks such as BRACS coarse-grained subtyping. For MII. architecture f. pretrain task & and target task t, we perform the following main comparisons which answer:
fa-z vs. fradoo init- For the same MIL architecture f. does pretraining task s outperform training from scratch on task !?
3. Preliminaries
fo-z vs. fer For the same MIL architecture f. do different pretrain tasks s and s' transfer better to t? fe-z vs. fe Using the same pretrain task s, do different MII. architectures f and f' transfer better to task t?
The primary goal of this work is to understand how well su- pervised MIL models transfer to various downstream tasks in computational pathology (CPath). This investigation is motivated by the overwhelming interest in developing slide foundation models, e.g. self-supervised MIL models that extract general-pun purpose slide representations with little-to- zero finetuning. In this context, supervised MII. with model weight transfer is a simple and fundamental alternative over- looked in the research community, and we hypothesize that transferring supervised MIL models may outperform cur- rent techniques and learning paradigms proposed for solving challenging CPath tasks. To this end, we exhaustively eval- uate transfer performance of 11 MII. architectues across 19 publicly available benchmarks. We outline the evaluation protocol, pretraining and target datasets, and MIL architec- tures used for assessing MIL transfer below.
Though this experimental setup follows previous investiga- tions in computer vision, one limitation in assessing model transfer in CPath is the lack of large-scale, diverse classi- cation datasets similar to ImageNet- Ik (IN-1k), which has been the standard pretraining task for assessing transfer of various image recognition backbones over the past decade. In addition to the 19 tasks, we also include two pan-cancer tasks called PC-43 and PC-108 (Chen et al.. 2024a). These represent 43-class and 108-class cancer subtyping tasks en- compassing diverse malignancies from 17 organ types and are curated from the same hierarchical classification dataset for pretraining purposes only. PC-43 and PC-108 consist of the same set of 3,499 WSIs with either coarse labels (43 main cancer types) or Ene-grained labels (108 OneoTree codes), respectively, with at least 5 and 15 WSIs per On- coTrree code for training and testing. Overall, these tasks were developed to emulate IN-1K in label complexity and flexibility for supervised pretraining, which we hypothesize to be similar to IN-1k in developing transferrable models.
3.1. Supervised MIL Transfer
The experimental setup for evaluating supervised MIL trans- fer follows that of previous works in transfer learning, in which a network f is first pretrained using a supervised task from source domain Ds (pretrain task) and then eval- uated on a different task from target domain Dr (targer task). We study the following 1 architectures for assessing MIL transfer: ABMIL (IIse et al., 2018), CLAM (Lu al., 2021), DSMIL (Li et al., 2021), DFTD (Zhang et al., 2022), TransMIL (Shao et al., 2021), Transformer (Wagner et al., 2023; Vaswani et al., 2023), ILRA (Xiang & Zhang, 2023), RRT (Tang et al., 2024), WIKG (Li et al., 2024), Mean- MIL and MaxMIL. We measure transfer performance to the target task using two evaluation settings: (1) end-to-end finetuning, and (2) frozen feature evaluation via K-nearest neighbors (KNN) pre-extracted slide-level embeddings.
3.2. Implementation Details
For WSI preprocessing, we use the following standardized protocol for comparisons across MIL models: 256 x 256
non-overlapping tissue patching at 20x magnification (0.5 pm/pixel) followed by pre-extracting features using UNI. a DINOv2-pretrain V encoder (Oquab et al., 2024; Chen et al., 2024a). Unless specified otherwise, all MIL models are implemented using the author's 's original model definition, trained with UNI features, and with standardized hyperparameters: Adam optimzier with a learning rate of x 10-4, cosine decay scheduler, and a maximum of 20 epochs with early stopping patience of 5 epochs on the validation set. Further details are provided in Section B.I.
For evaluation, we assess MIL transfer performance on 19 publicly available CPath tasks, with training datasets rang- ing size from 314 to 8,492 WSIs and in label complexity from 2 to 30 classes. To ensure that the observed trends in this study are not confined to specific organ, disease, or type of task,
comprehensively evaluate on four organs (breast, lung, prostate, and brain), and diverse task types such as cancer classification, cancer grading, and molecu- lar subtyping, which are used as both pretrain and target
3

---

Do MIL Models Transfer?
4. Results
(ABMIL, TransMIL, Transformer). As shown in the contin- gency matrix in Figure 2, representations from pretrained models consistently outperform random-weight bascline (improvement indicated by red in the matrix). We observe that pretraining on both in-domain and out-of-domain data yields strong results; for example, models pretrained on lung cancer (NSCLC) transferred to breast biopsy tasks (BCNB) as effectively as models pretrained on breast cancer (BRCA, BRACS). Notably, pancancer pretraining delivered the most substantial performance gains. Pretraining on PC- 108 and PC-43 improves performance over the baseline an average of +9.8% and 8.6%, respectively. These findings suggest that pancancer pretraining is an effective strategy for obtaining highly transferable slide-level representations.
4.1. Which pretrain tasks are best for MIL transfer?
We first characterize the pretraining task quality by assess- ing how well a model tully-trained on a task can generalize to new tasks with frozen weights. Though not intuitive from brain tissue
clinical perspective how model trained would lead to better performance on lung tissue, we hypoth- esize that tasks with diverse diagnostic entities may confer unique advantages by allowing the model to learn from a more diverse set of slide representations across pretraining and training. Additionally, many histological entities are consistently of low diagnostic relevance, such as smooth muscle, red blood cells, and tissue processing artifacts, just as the recognition of nuclei and immune cells remain perti- nent across an array of tasks in CPath. These commonalities across tasks suggest that MIL models may learn highly transferable aggregation methods. We evaluate the quality
4.2. How does MIL architecture relate to transfer performance?
Using PC-108 as the representative pretraining task, we next perform more comprehensive assessment of how well MIL architectures transfer. New MIL. architecturss are constantly emerging, each proposing new ways model inductive biases such as cell-cell and cell-tissue relationships in the tissue microenvironment, yet it remains unclear whether these architectural innovations translate to improved gen- eralizability or transferability. To gain further insights into this question, we compare the performance of 11 MII, archi- tectures pretrained on PC-108 against their counterparts ini- tialized with random weights. Aside from the initialization weights, all other training settings are fixed (Section B.2).
Single Organ
Pan-Cancer
SELE SU STA SEO SALT Da aid sace du lac our Se SAu 8
BRACE
SOY 54. St
DONB-EX 322
Ralating
Target Task
RCHB-HIMZ
SCHD-F0-2 256 Y5a
596-339 99
055643 430
SBCLC
ERRADNIIC
Figure 1 and Table 1 illustrate the average performance across all 19 tasks and performance grouped by distinct datasets, respectively. We observe that all MIL approaches benefit from pretraining, with an average improvement of 3.3% when using a pretrained PC-108 model compared to training from scratch. ABMIL is among the lop-performing models across both initialization strategies, with a 3.8% average increase in performance. This aligns with recent findings that, given strong patch foundation models, simpler MIL approaches tend to outperform more complex alterna- tives (Chen et al., 2024a;b; Song et al., 2024). Additionally, DSMIL,, closely related method ABMIL which adds patch-level predictions to ABMIL's global pooling method, attains the highest performance with random initialization. This further substantiates the efficacy of simpler weighted pooling approaches under random initialization.
505
acou
Pretrain Task
Figure 2. Transfer performance across pretrain tasks. The con- tingency table shows the average KNN performance of three MIL models (ABMIL, TransMIL, Transformer transferring from the 21 pretrain tasks (columns, bottom axis) to the 19 target tasks (rows, left axis). The rightmost column (colored gray) shows the baseline of random weights. We use AUROC for binary classifica- tion, Cohen's weighted kappa for prostate grading, and balanced accuracy for multiclass classification tasks. The heatmap indicates the relative performance compared to baseline, with red indicating improvement and blue indicating performance decrease for each task.
While pancancer pretraining improves the performance of all models, the extent of improvement is architecture- dependent. For instance, we find that Transformer basedd models, (TransMIL and Transformer), exhibit substantial improvements from pretraining, showing average improve- ments of 5.82%, and 5.83% respectively across all tasks. We hypothesize that because these architectures are highly
of "'off-thessheff slide representations from pretrained MIL models. To do so, we use a KNN classifier (k=20) to mea- sure transfer performance from 21 pretraining tasks to 19 target tasks, averaging results across three MII. architectures
4

---

Do MIL Models Transfer?
Table Finetuning performance across different MIL and datasets. Performance with random initialization (Base) and PC-108 pretraining for each MIL framework, averaged within each task group. Reported metrics are balanced accuracy for multi-class tasks
(C' 2) and weighted x for PANDA; AUROC otherwise. A denotes the performance difference between PC-108 and Base. Parentheses
indicate the average standard deviation across the grouped tasks, with standard deviation determined by 1,000 bootstrap trials.
Task
Init
ABMIL
CLAM
DFID
DSMIL
ILRA
RRT
TransMiL
Transtormer
WIKG
maxXIL
meanMMLL
Average
BRACS
Base
53.6
43.6,..)
50.0
53.5
55.7a..
53.7a.
50.2:a.;
36
50.3:4.9)
49,4,a.)
49.4
49.7
(2 tasks)
PC-108
57.8um)
+4.2
44.6
+1.0
57.9
+7.9
57.0
+3.5
3.9
52.40m
1.3
50.7
+0.5
53.2
+16.4
62.1
+11.8
48.9
.0.5
41.4cax:
-8.0
+2.8
52.5
(3 tasks)
BCNB
PC-108
Base
84.5
80.9,.
82.4..
80.Sa.
78.6..
86.1a.
79.6
83.0
19:0...:
814
79.1.m
78.3
76.9:4.)
84.1
79.1..))
75.8
834...1)
82.4aa))
83.0aa.)
83.1
83.4
79.8:..
82.6
79.6
+3.6
+1.9
+7.5
+34
+24
+0.8
+7.2
+3.4
1.0
+0.1
+3.6
+3.0
BRCA
Base
69.2
69.4..
689a.
68.7aa
66.7am
64.7
64.2a..)
6600:.4)
66.7ta.)
71.2:a.::
67.6
lasks)
PC-108
73.3
+4.1
65.1.4.)
2.7
75.5
+6.1
69.4..)
+0.5
72.2m..
+3.5
704
+3.7
72.1ma))
+7.4
7144..
+7.2
69.1
+3.1
70.9
+4.2
70.7:az
-0.5
70.9
+3.3
EBRAINS lusks)
Hase
PC-108
78.4.
80.1
18
80.30.0
76.60
77.2a
76.8
78.4.
77.60
79.60.)
72.2
79.20
16.2
78.2
79.6
73.4
788:
80.1
78.9
76.2
+1.8
+1.3
+3.7
+2.0
+0.8
+7.4
43.0
+3.1
+6.2
+1.3
+2.6
GBMLGG
Base
68.91a.a)
67
12.2
13.6
69.712.)
68.9a.
67.01..4
72.212.3)
70.322.:
69.9
PC-108
73.6
a
74
75.4
70.2
74.1
71.912.)
70.2
67.9:2.)
73.0
70.4:2.2:
72.1
tasks)
+4.7
+4.1
+6.7
+3.1
+2.2
+1.3
+0.9
+0.8
-0.1
+2.2
NSCLC Morph
Base
a
m
94.0
93.9
91.3
93.3m
91.3wa)
95.9
92.7
PC-108
92.0m.a)
76.6
95.3mm
94.8
94.7mm.)
95.4mmm
94,4
94.7mu.7
95.4
92.3
94.7
(1 task)
+0.8
+0.9
+4.5
+1.3
44.4
40.8
+4.1
+3.4
0.5
+1.2
+2.0
NSCLC Molec
Base
67
67.5m..)
64.0m..)
a)
62.8..)
66.1m.
624m..)
(r.
67.4%a.:
66.2
(4 tasks)
PC-108 108
A
72.8
+5.4
68.5m.a)
+1.0
74.3
+10.3
68.8maa)
+0.5
71.0
+8.2
73.3
+6.1
75.0ma.)
+8.9
73.2
+7.1
75.0
+12.6
62.4ma.0
6.7
74.1
+6.7
+5.5
71.7
PANDA
Base
21.6
21.2...)
87.9..7)
VI.I
90.5r..9
9044..0
w..
89.70..)
21.2
503
(1 task)
A
PC-
P0-108
93.31a.5)
+1.7
91.8
+0.6
93.2m.
+5.3
93.5io.n
+2.3
91.5
0.0
91.
91.5
+0.4
89.9
-0.6
90.7ie.a)
+0.3
93.0ma.)
+0.9
88. 88.9ra.s
-088
90.9
-0.3
+1.0
91.8
parameterized, they are also more liable to overfit the train- ing set, and may consequently benefit more from effective initializations. We directly investigate this relationship be- tween model size and transfer performance in Section 4.4.
4.3. Pretrained models are few-shot learners
An important consideration for MIL methods is learning with only a limited number of samples, a common scenario when dealing with rare diseases (Huang et al., 2023; Lu et al., 2024; Chen et al., 2024a). We investigate the data ef- ficiency of transferred MIL models by probing performance in few-shot scenarios. Specifically, we randomly sample K E {4, 16, 32} samples from each class of C' diagnostic classes to construct a dataset of K C samples, and train five different MIL methods (ABMIL, Transformer, Trans- MIL, DFTD, CLAM) on the molecular subtyping tasks of NSCLC-TP53/STV PEPPS//STKK//FFRRRRRR 1/EGFR, BCNB ER/PR/HER2, and GBMLGG-C. The experiments are repeated five times to mitigate sampling bias. To delineate the effects of pretrain- ing datasets, we ablate over random initialization, PC-43, and PC-108 pretraining datasets.
Meanwhile, the non-parametric aggregation models (mean- MIL and maxMIL) exhibit comparatively small changes in performance, suggesting that the transfer of knowledge be- tween aggregation components is the key ingredient of MIL transfer. We also found that DSMIL and CLAMM, closely related due to reliance on auxiliary loss for classifying key instances, exhibit lower improvements from transfer com- pared to other methods. We hypothesize that since the instance-level auxiliary loss guides the training dynamics of CLAM and DSMIL on top of the original slide-level loss, the initial weights are less pertinent to the model conver- gence, thereby reducing the benefit of MIL transfer.
Furthermore, we observe that the best performance from ran- dom initialization (72.3 with DSMIL) is lower than 9 of the 1l pretrained models, suggesting that the use of an effective initialization plays a more important role in performance than the quality of the MIL. method. While many techniques have been proposed for developing new MIL architectures, we find that most techniques do not demonstrate any signifi- cant margin of improvement over ABMIL. Altogether, these results indicate that, rather than further architectural innova- tions, high-quality pretraining of simple but effective MIL models leads to the most performant models. To confirm the robustness of these results, we repeated experiments across five random seeds for ABMIL, DFTD, TransMIL, and RRT (Table A4). In all cases, the performance gains from pre- training remained consistent, reinforcing the reliability of these gains from pancancer pretraining.
Across all five methods in Figure 3, we observe a clear trend of pancancer pretraining outperforming random ini- tialization across all shots. The difference gap is especially pronounced for lower number of shots for all MIL models, with DFTD boasting 171 % increase for K = 4 over ran- dom initialization. This underscores the data-efficiency and potential for MIL transfer to assist in data-sparse regimes. Between PC-108 and PC-43, we observe that PC-108 ob- tains higher performance at every configuration, suggesting that the models trained with more challenging fine-grained classification task exhibit better data efficiency.
4.4. Do larger MIL architectures transfer better?
As a subsequent step to understanding transfer trends across different MIL models, we investigate the transfer trends
5

---

Do MIL Models Transfer?
0.77
0.8
0.78
few-shot performance
Average Performance
0.7
0.75
0.6
0.74
PC-108 Random init.
0.73
0.4
0.72
PC-108
K=4
PC-43
K=16
Random init. O K-32
0.2
ABMIL
Transformer
TransMil
DFTD
CLAM
0.70
0
2
Number of Parameters (x10")
MIL architectures
Figure 4. Transfer at different model scales. Average perfor- mance different ABMIL scales across 19 ovaluation tasks with initialization from random weights and PC-108 pretraining.
Figure 3. Few-shot performance. Few-shot performance with different initialization strategies for K = {1, 16, 32) samples over five MIL methods molecular subtyping tasks. Performance averaged over test splits of 5-fold voosdaanii.
GigaPath, a LongNet- model (Ding et al., 2023), was pretrained on 171,189 WSIs via self-supervised masked autoencoder approach, with VIT patch encoder pretrained over 1.3 million patches using DINOv2.
within a MIL. model, by analyzing the relationship between transfer performance and model scale. With ABMIL demon- down-
strating the best MIL transfer performance, up-scale the model size of ABMIL (s= 9 x 10a parameters), by adjusting the depth and width of the pre-attention MLP, with changes described in Table A2.
For fair comparison, we benchmark GigaPath against an ABMIL model pretrained with PC-108 using GigaP- ath's VIT patch features. Similarly, CHIEF benchmarked against an ABMIL model pretrained with PC-108 using CTransPath features. Transfer quality is assessed by com- paring both KNN and finetuning performance.
We observe that PC-108 pretraining consistently and sub- stantially outperforms random initialization at all model scales (Figure 4). Furthermore, while random initializa- tion led variable performance across different model scales, the pretrained model performance remains compar- atively stable across different model scales, with mono- tonic increase from 0.1 million to million parameters, and subsequently decreasing in performance at 9 million parameters. This consistency in performance, even with a model 5-fold larger than vanilla ABMIL (5M parameters 1M parameters), suggests that pancancer pretrained models are less likely overfit. Furthermore, the mono- tonic increase in performance for PC-108 indicates that an effectively -initaaliiii model may even exhibit favorable scaling properties, highlighting the potential and importance for large-scale slide-level foundation models. We also exam- ine different Transformer sizes in Table A3, finding similar trends in transferability and model size.
Our results reported in Table 2 indicate that, while all pretrained models outperform training from scratch, PC- 108-based slide representations yielded superior KNN per- formance in 12/15 tasks against CHIEF (average increase: +5.9%) and 13/15 tasks against GigaPath (average increase: +9.7%). This improvement is significant because PC-108's pretraining set had no sample overlap with these down- stream tasks, whereas CHIEF was pretrained on TCGA and PANDA, which cover 11 of the 15 tasks. This underscores the high generalizability of PC-108's frozen slide-level fea- tures from pancancer pretraining.
These advantages extended to finetuning. PC-108 pretrained representations led to better finetuning performance than CHIEF in 1 1/15 tasks (+2.2% average improvement) and GigaPath in 10/15 tasks (+0.8% average improvement). Cru- cially, PC-108 achieved this using a pretraining dataset that was merely 6.5% and 2.3% the size of CHIEF's and GigaP- ath's ss respectively. As both CHIEF and our PC-108 model employ the same ABMIL architecture, we conclude that PC-108's superior transferability stems from its pretraining methodology rather than architectural differences.
4.5. Can supervised MIL. transfer close the gap with slide foundation models?
We conduct a head-to-head comparison of our PC-108 pre- trained models' transferability against two oftt--of-the-ar slide-level foundation models: CHIEF (Wang et al., 2024) and GigaPath (Xu et al., 2024a). CHIEF, an ABMIL-based model, was pretrained on 60,530 slides using supervised contrastive learning (Khosla et al., 2021) and CLIP (Rad- ford et al., 2021) to detect cancer across 19 organ types.
Although CHIEF and our work both employ supervised pre- training, we hypothesize that this difference in performance may likely be a result of the complexity of the pretraining dataset. Specifically, CHIEF was pretrained on binary task
6

---

Do MIL Models Transfer?
Table Comparison of MIL transfer with slide foundation models. KNN and finetuning performance for MIL transfer with slide foundation models (GigaPath, CHIEF) compared against an ABMIL model initialized with pancancer pretraining (PC- and random weights (Base). Evaluations are grouped by patch encoder (GigaPath VIT. CTransPath). Best results are bold, second best are underlined.
KNN
Finetuning
GigaPath VIT
CransPaah
GigaPath VIT
C'Iransaath
Tisk
PC-108
GigaPath
Basc
PC-108
CHIEF
Base
PC-108
GigaPath
Base
PC-108
CHIEF
Base
BRACS-C
44.7(3.8) 47.8(3.3) 60.3(..6)
8)
(6.5(3.6)
66.5(3.3)
49.9
75.0k4.0) 0)
65.6(4.1)
69.4(4.8) 8.
(1.1
74.6(4.4)
60.6(5.2) 2)
BRACS-F
35.3(4.1)
24.5(4.3)
18.2(4.8)
46.3(4.2)
37.5(4.8)
16.4(3.1)
48.3:4.4) 4)
(3.5(4.5)
44.1 (4.6
(3.1
43.1(4.0) (2.1(..1)
BRCA-ER
884(2..)
11.9(4.1)
70.2(4.7)
84.2(3.8)
77.4(a.5)
84.8(4.4 69.1(3.9)
81.7(4.4)
(6.9.4.5)
(6.7
84.0(4.0)
85.3(3.0)
BRCA HER2
62.7(5.0) 03
15.9(1.1)
59.6(1.6) 6)
60.3(1.8)
54.0(4.1)
63.3(4.0)
68.9(5.1 1)
(7.0().4)
63.5(6.0) 0)
65.3
(3.1
69.6(4.9)
BRCA-PIK3CA
60.8:3.3) 3)
60.2(3.5)
54.7(3.8) 8)
61.03.5)
60.6(3.7)
52.9(3.7)
66.9 3)
64.2
60.9:3.6)
(4.3
62.9(3.9)
55.7(4.1)
BRCA PR
83.0(3.8)
(7.4
68.2(5.43
70.4(3.9)
(3.2(3.6)
67.5 ()
75.3(5.0) 6)
(8.6(3)6)
80.0cs.4)
(8.1
(3.5
(5.5(4.0)
NSCL.C-FGIR
72
37.0(7.0)
58.8(7.4)
(1.)
59.7(7.3)
50. (0.1().1)
73.677 6)
58.1
62.3 7)
58.8(a.0)
65.1
(3.9(a.1)
NSCLC-KRAS
60.8(4.2)
63.2(6.3)
(7.3(4.7)
60.666.5)
51.4(G.9)
50.2(G.3)
(9.8: T)
58.0 (G.8().4)
2)
(G.1
57.5
67. (7.1
NSCLC-STK11
(5.5(
77.416.7)
63.2(5.7)
(5.5(5.9)
62.3(8.9)
50.1(9.2)
88.6(4.5)
(4.3().4)
83,5(3.9)
77.5(7.0)
(1.1(0.3)
76.2(7.6)
NSCLC-
(0.3
19.7
68.3(4.4)
(4.9
72.0(4.4)
60.0((.44
(0)9(4.3)
78.4(4.7)
77.3(4.0)
(1.3
(4.8(4.6)
75.2(4.8)
EBRAINS-C
78.5 ()
72.9(2.0)
34.6(2.3)
67.2(2.1)
51.8(2.2)
(8.1(2.5)
85.5(2.2) 2)
(2.4
85.9 2)
78.2(2..)
(1.9
(9.5(2.2)
EBRAINS-F
60.6 23
56,5(1.9)
54.2
$1.9(1.)
38.6(1.6)
(2.
70.2:a.0)
(2.1(1.9)
(9.5(1.8)
59.2(2.33
60.0:2.6)
58.3
GBML.G--C
(3.9(00.)) 6)
84.4(1.4
79.7(1.2
89.4(0.9)
77.0(0.8)
82.3(1.1)
93.4(2.1
(4.7(1.3)
92.0c1.6)
92.6(1.7)
91.2(118)
91.3(1.8)
GBMLGG-F
55.3
45.
29. 19.1 5)
38.9(2.1) 49.2(2.3)
38.2(3.0)
$6.8 8)
53.0(3.4)
48.0;3.2)
56.5(3.3)
54.4(3.3)
56.4(3.5
PANDA
(7.5
67.4(1.0)
53.0k0.9)
25.0(1.1)
83.2(0.8)
718(1.1)
93.1(1.1)
94.5(0.7)
91.5(0.9)
(0.8(1.0)
84,2(1.4)
00.5(1.0)
Average
70.3
60.8
56.9
66.2
60.3
54.5
75.4
72.6
71.5
72.0
69.8
69.2
of distinguishing cancer from non-cancer, utilizing both supervised contrastive learning for visual pretraining and CLIP for image-text alignment. We believe the strength of our approach lies in the diversity of our pretraining dataset and the challenging nature of differentiating a large number of classes simultaneously: PC-108 is a pancancer, hierarchi- cal classification scheme, requiring the model to implicitly characterize each WSI in terms of its organ, cancer type, and cancer subtype(s). This diverse and challenging pretraining task likely promotes the learning of comparatively more detailed, generalizable slide-level representations. Overall, our results highlight the feasibility of supervised pancancer pretrained models to surpass the ability of self-supervised slide-level foundation models while requiring less than 10% of the pretraining data.
The benefit of PC-108 pretraining is particularly evident for ABMIL, which improves over random initialization on 13/15 tasks with CTransPath, 12/15 tasks with ResNet50, 12/15 tasks with GigaPath, 10/15 tasks with UNIv2, and 11/15 tasks with CONCHv15, We repeat this experiment over CTransPath and ResNet50 with TransMIL (Table A5), finding that TransMIL leads improved performance in 8/15 tasks with CTransPath and 10/15 tasks with ResNet.
Table 3. Combined ABMIL pretraining performance with dif- ferent encoders. Performance across different tasks for ABMIL using ResNet-50, UNIv2, and CONCHv1.5 1.5 as patch feature en- coders with PC-108 pretraining and random initialization (Base). Best performance between Base and PC- 10-108 for each encoder is bold.
ResNet. 50
UNIv2
CONCIEVI.5
4.6. How do patch encoders impact transfer performance?
Task
Base PC-108
Base
PC-108
Base
PC-108 108
BRACS C
(4.9
o)
71.6(44)
74.6(1.)) 78.4(1.6 78.0(4.8)
BRACS-F
19.4(3.6)
27.3 T) 45.6(4.6) 6)
50.0
49.4(a.5) 53.2(4..6)
To investigate whether the observed benefits of pancancer pretraining are inherent to MIL architecture versus choice of pretrained patch encoder, we also investigate transfer perfor- mance across various feature encoders of varying strengths. Specifically, we conduct experiments with a general-purpose image encoder for natural images, ResNet-50 (He et al., 2015) pretrained on IN-IK, CTransPath (Wang et al., 2022), GigaPath ViT (Xu et al., 2024b), UNIv2-h (Chen et al., 2024a) and CONCHv1.5 .5 (Iu et al., 2024). For each encoder, we evaluate the finetuning transfer of randomly initialized ABMIL models to their counterparts initialized from PC- 108-trained models.
BRCA-ER
77.5(4.0)
12.3(G.1)
85.3:3.6)
84.5(4.0)
87.3(a.9) 88.7(4.0)
BRCA-HER2
58.016.3)
65.5(6.6) 71.1553)
(7.1 64.5(5.5) 56.8(5.5)
BRCA-PIK3CA
59.0c4.5)
60.7(4.0)
62.7(4.8)
57.1(8.7)
66.1(4.1) 69.1(3.6)
BRCA-PR
65.6(4.4)
62.2(4.8) 76.4(3.6
77.7(4.2) 73.0(3.9) 77.5(3.9)
EBRAINS
51.3(2.7) 53.6;2.8) 89.3(2.1)
90.5(1.7) 90.1(1.9) 88.0(2.2)
FIRAINS.
28.2(1.9)
34.7(2.) 70.62.11
70.4(1.9) (1.1 71.1
GBMLGG-C
78.5(2.2)
3) 91.0;2.1) 92.7(1.7) 91.5(t17) 92.8(1.6)
GBMLGG-F
37.7(8.7)
42.6(3.8) 26.(3322
59.2(3.2) 59.7(8.0) 60.6(2.4)
NSCL.C-EGRR
56.1(9.7)
52.0ca 8) 70.9(10.1) 72.4.6.) 56.4(8.5) 71.4(7.0)
NSCLC-KRAS
56.8(6.7)
63.5(t.8)
(..9
58.3(6.7)
59.5(6.7) (0.7(6.6)
NSCLC-STKI1
50.7(8.0)
64.1
70.43.3)
87.9(4.2) 70.0r7.0) 76.1
TPS3
69.9(1.9)
76.3k..)
81.0g.8)
83.1 76.2(4.9) 83.1
PANDA
79.6(1.6)
$0.1 7)
93.3 7)
93.8(0.7)
91.8(0.8) 91.4(0.8)
Average
56.2(4.5) 59.6
72
74.6(1.1) 72.3(4.1) 74.6(4.1)
4.7. Can public, single-organ datasets serve as a pretraining task?
The results of these experiments are displayed in Tables 2 and 3. We observe that across all encoders, PC-108 pre- training leads to an average improvement in performance.
Section 4.1 establishes that pancancer pretraining produces transferable frozen slide-level representations, although single-organ pretraining also consistently outperforms ran-
7

---

Do MIL Models Transfer?
dom weight initialization. Here, we investigate if these findings extend to finetuning. To compare the benefits of pancancer and single-organ pretraining, we evaluate the transferability of four models (ABMIL, DFTD, TransMIL, and Transformer). We train each model on a specific task from our 19 public datasets and then use its weights to initialize the model for evaluation on all other tasks.
4.9. What features are being transferred?
Supervised MIL transfer learning offers substantial perfor- mance gains, but it remains unclear which layer's features are most important for transfer. While early layers often prove most transferable in standard CNNs (Raghu et al., 2019), this principle may not extend to MIL models. These models, designed for gigapixel inputs, use a distinct two- stage process, consisting of a patch-level MLP followed by attention- baseed aggregation, raising the questions: How much do pretrained layers change during finetuning, and how does their transfer impact performance? We exam- ine quantitative and qualitative measures to answer these questions in this section.
The results, summarized in Figure A1, align with our previ- ous findings frozen feature evaluation. We observe that any form of pretraining improves average downstream task performance over random initialization, even when transfer- ring between different organ types. Furthermore, PC-108 consistently achieves the highest finetuning performance, surpassing all single-organ pretraining approaches. These findings highlight pancancer pretraining the preferable strategy for developing generalizable models with super- vised pretraining, while validating single-organ pretraining as feasible and accessible alternative.
4.9.1. FEATURE STABILITY
We first study feature stability of pretrained layers, hypoth- esizing that layers possessing the most transferable knowl- edge would change less during fine-tuning. We measure feature stability with Singular Vector Canonical Correlation Analysis (SVCCA) (Raghu et al., 2017), which provides a means of assessing the linear similarity between the ac- tivation spaces of each neural network layer before and after finetuning. We measure layer stability with using the average canonical correlation on a scale from 0-100.
4.8. Why does pancancer pretraining transfer so well?
Encouraged by the performance of pancancer pretrained models, we next seek to understand why initialization from pancancer tasks would lead to higher performance with fine- tuning. We hypothesize that, because large pancancer tasks require models to learn slide-level representations which can distinguish between huge variety of organs and mor- phological appearances, these slide-level representations would likely help differentiate between classes of unseen tasks without any finetuning. In Figure 5, we indeed ob- serve clearer class separation among 12 histologic subtypes (EBRAINS-C) C) for slide features from ABMIL trained on PC-108, when compared to randomly initialized version.
We compare the pre- and post-fine-tunnng activations for every layer in Standard (S, 9 10s params) and Large (L, 5.25 x 10 params) ABMIL models, using 45,232 patches from 30 NSCLC-KRAS slides. ABMIL-S uses a single linear layer (Lin. 1) for patch-specific processing, while ABMIL-L uses a three-layer MLP (Lin. 1, 2, 3) and wider aggregation layer, allowing us investigate whether the same trends hold across different model scales.
The results, presented in Table 4, demonstrate that pre- trained layers exhibit significantly less change during train- ing compared to their randomly-initialin
Random init.
PC-108
counterparts. Notably, the deep attention layer shows a remarkably low correlation (2.5 0.0 and 16.3 + 0.0 for ABMIL-L and -S, respectively) with its initial state when training from random initialization. In contrast, when using pretrained weights, these models retain high similarity (82.7 + 0.0 and 97.7 + 0.0). Given that the attention layer plays the critical role of converting each patch embedding into scalar weight for slide-level aggregation, this finding strongly suggests that the benefits of MII. transfer learning are closely tied to the transfer of learned aggregation strategies.
ISNEE
SINE
1-SNE1
1-SNE1
4.9.2. LAYER-LEVEL PRETRAINING IMPORTANCE
Figure 5. t-SNE of slide-level features. Visualization of the slide- level features from randomly initialized ABMIL compared to AB- MII. pretrained on PC-108 for 12-class brain subtyping for rare brain disease classification.
We next investigate the contribution of different components within the pretrained MIL model towards overall perfor- mance. To do so, we assess finetuning performance while progressively resetting layers of the aggregation module to
8

---

Do MIL Models Transfer?
representative attention heatmap of TP53 positive lung cancer (TCGA) in Figure 6 across three different scenarios: Randomly-initializz weights, PC-108 pretrained weights, and finetuned on NSCLC TP53 mutation prediction from PC-108 108 pretrained weights.
Table SVCCA Similarity. Mean and standard deviation of canonical correlation between layers before and after fine-tuning, on the scale of 0 to 100 for Large (L) and Standard (S) ABMIL.
Layer
Base-L
PC108-L
Base-S
PC108-S
Lin.
92.8 + 18.3
95.5 + 14.7
93.9 + 16.0
97.2 + 12.0
Lin. 2
81.2 + 22.6
89.2 17.9
Lin. 3
47.2 + 25.0
66.5
+ 26.3
WSI
Frozen (random init.)
Attention
2.5 0.0
82.7 = 0.0
16.3 0.0
97.7 0.0
Average
61.6
83.1
75.7
96.2
random weights, starting from the final layer and moving to earlier layers. We compare these results against the base- line where all weights are transferred for PC108-pretrained ABMIL-L (PCI08-L).
2
Frozen (PC-108 init.)
Finetuned (PC-108 init.)
The experiments are setup as follows Reset Attn: Re- initialize only the attention layer weights. Reset Lin3+: Re-initialize the third linear layer and attention layer. Reset Lin2+: Re-initialize the second and third linear layers, and the attention layer. Reset All: Re-initialize all weights in the aggregation module, same randomly initialized weights.
Attention score
High
Figure 6. Heatmaps for visualizing attention transfer. Visu- alization of the three different ABMIL attention heatmaps for lung squamous cell carcinoma: ABMIL with randomly-initialaa weights, PC-108 pretrained weights, and finetured with NSCLC TP53 mutation prediction from PC-108 pretrained weights.
Table Impact of re-initializing aggregation module layers. Values represent the change in performance relative to the PC108- L baseline (full weight transfer).
PC108-I. Reset Attn
Reset Linde
Reset Lin2+
Reset
BRACS-C
71.9
8.5
8.5
-12.8
-1..0
We observe that ABMIL with random weights exhibit dif- fuse attention, without focusing on regions of clinical impor- tance. Meanwhile, the pancancer pretrained model foeuses tumor regions prior to finetuning, with high-attention regions remaining relatively stable after finetuning. We hy- pothesize that by allowing the model to place high attention from the start on regions that are diagnostically-releann across cancer types, pretrained MIL models may be less prone to spurious correlations and learn more nuanced dis- tinctions between classes over the course of training.
BRACS-F
53.3
-12.1
-10.5
-10.4
-10.1
GBMLGG-C
95.4
-1.2
-1.0
0.0
-0.2
GBMLGG-F
51.7
0.0
-0.8
-1.8
NSCLC-EGFR
76.1
-8.6
-10.2
-13.1
-12.8
NSCLC-KRAS
68.4
-2.0
-4.5
-5.8
-8.7
NSCLC-STK11
86.7
0.0
-0.6
-8.3
-15.0
NSCLC TPS3
81.5
-7.3
-5.9
-0.5
-10.0
Average
73.1
-5.0
-5.2
-6.6
-8.3
Performance is reported as the change relative to the full weight transfer (PC108) baseline. The results, presented in Table 5, reveal that re-initializing the attention layer (Re- set Attn) causes the largest single performance drop (-5.0% average decrease). Re-initializing the preceding MLP lay- (Reset Lin3+ and Reset Lin2+) leads to further, albeit smaller, decreases, culminating in an 8.3-point drop when all weights are reset. This underscores the critical role of all MIL layers, particularly the attention component, for suc- cessful transfer lcarning. These findings diverge from prior work (Raghu et al., 2017), which observed minimal impact from transferring later CNN layers, thereby highlighting the unique characteristics and requirements of MIL. transfer.
5. Conclusion
We investigated transfer learning capabilities of MIL mod- cls in CPath, with models pretrained on pancancer tasks ex- hibiting remarkable generalization across organs and tasks, and models pretrained on single-organ tasks transferring effectively out-of-domain tasks. Limitations of our study include the absence of state-space MIL models, survival prediction tasks, and augmented pretraining that could fur- ther improve generalizability. Nonetheless, by providing insights into MII. transfer, this work offers a roadmap for leveraging supervised pretrained MIL models to enhance performance in various clinical tasks. Model weights for our pest-performing ABMIL model, named FEATHER, as well as initialization schemes for all MIL methods, can be accessed at
4.9.3. VISUALIZING TRANSFERABILITY
We examine whether the transferability observed in Sec- tions 4.9.1 and 4.9.2 is also reflected in ABMIL atten- tion heatmaps, which indicate the learned importance of each patch according to the aggregation layer. We show

---

Do MIL Models Transfer?
Impact Statement
This includes transparency in model development and limi- tations, proactive strategies to identify and mitigate algorith- mic bias, and ongoing efforts ensure that the advantages of these technologies are accessible equitably across diverse healthcare settings.
The research presented in this paper aims to advance the application of Machine Learning computational pathol- ogy by introducing a more resource-cfficient paradigm for pretraining multiple instance learning models. Our work demonstrates that leveraging an institution's 's own diverse, pancancereer data can lead to significant improvements in data efficiency, requiring less data than traditional large- scale approaches. This, in turn, contributes to notable compute and storage efficiency for both model develop- ment and subsequent fine-tuning. The primary societal benefit of this approach is the potential to make advanced AI tools for pathology more accessible, particularly for research groups and healthcare institutions that may lack the extensive resources typically required for pretraining with techniques like large-scale self-supervised learning (SSL). To this end, we share the model weights for our best-performing MIL pretrained model, an ABMIL model which we call FEATHER, and initialization methods in the aforementioned GitHub for public use. We envision this fostering wider innovation, accelerating the development of diagnostic aids, and potentially improving the consistency and reach of pathological services by making powerful Al tools easier to develop and adapt.
References
Brancati, N., Anniciello, A. M., Pati, P., Riccio, D., Scog- namiglio, G., Jaume, G., De Pietro, G., Di Bonito, M., Foncubierta, A., Botti, G., Gabrani, M., Feroce, F., and Frucci, M. BRACS; A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images, November 2021, arXiv:2111.0% 1.04440 [cs, eess, q-bio].
Brennan, C. W., Verhaak, R. G. W., McKenna, A., Campos, B., Noushmehr, H., et al. The Somatic Genomic Land- scape of Glioblastoma. Cell, 155(2):462 477, October 2013. ISSN 0092-8674.
Bulten, W., Kartasalo, K., Chen, P.-H. C., Ström, P., Pinck- aers, H., al. Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA chal- lenge. Nature Medicine, 28(1):154-163, January 2022. ISSN 1546-170X. doi: 10. 10000//41111-00--01600000 Publisher: Nature Publishing Group.
While this work offers path toward more accessible and efficient Al development, it is crucial to address potential ethical considerations and societal consequences:
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A.,
Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., and Fuchs. T. J. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25(8): 1301-1309, August 2019, ISSN 1546-170X. doi: 10. 1038//41591-009-0508-- Publisher: Nature Publishing Group.
Algorithmic Bias and Equity: Models pretrained on data from a specific institution, even if pancancercer, may inad- vertently inherit biases present in that data (e.g., related to patient demographics, disease prevalence, specific equip- ment, or institutional recording practices). In the case of PC- 108, all cases were acquired from Brigham and Women's Hospital, which may lead to bias and performance dispari- ties when applied to underrepresented groups or different clinical contexts, potentially exacerbating existing health inequities. In the future, more rigorous bias audits, diverse dataset validation, and ongoing monitoring are therefore critical next steps to ensure the equitable translation of pre- trained models.
Campanella, G., Chen, S., Verma, R., Zeng, J., Stock, A., Croken, M., Veremis, B., Elmas, A., Huang, K.-I., Kwan, R., et al. A clinical benchmark of public self- supervised pathology foundation models. arXiv preprint arXiv:2407.06500 2024.
Campbell, J. D., Alexandrov, A., Kim, J., Wala, J., Berger, A. H., Pedamallu, C. S., Shukla, S. A., Guo, G., Brooks, A. N., Murray, B. A., et al. Distinet patterns of somatic genome alterations in lung adenocarcinomas and squa- mous cell carcinomas. Nature genetics, 48(6):607-616. 2016.
While our approach low- ers the barrier for model development, ensuring equitable access to the benefits of these Al tools across diverse health- care settings, including those in low-resource environments, will require concerted effort beyond the technical aspects of model creation.
Resource Allocation and Access:
Overall, our work contributes to more sustainable, effi- cient, and potentially equitable approach to developing AI tools in computational pathology. The realization of these benefits, however, depends critically on a continued com- mitment to responsible research and development practices.
Chen, R. J., Chen, C., Li, Y., Chen, T. Y., Trister, A. D., Kr- ishnan, R. G., and Mahmood, F. Scaling vision transform- ers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16144 16155, 2022.
10

---

Do MIL Models Transfer?
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., Williams, M., Oldenburg, L., Weishaupt, L. L., Wang. J.J. J., Vaidya, A., Le, L. P.. Gerber, G., Sahai, S., Williams, W., and Mahmood, F. Towards a general- purpose foundation model for computational pathology. Nature Medicine, 30(3):850-862 March 2024a. ISSN 1546-170X. doi: 10. lisher: Nature Publishing Group.
Jang, Y., Lee, H., Hwang, S. J., and Shin, J. Learning what and where transfer. In International conference on machine learning, pp. 3030-3039. PMLR, 2019.
Jaume, G., Oldenburg, L.,, Vaidya, A., Chen, R. J., Williamson, D. F., Pecters, T., Song, A. H., and Mah- mood, F. Transcriptomics soaipppoomm---didedddd guided slide representation learning in computational pathology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9632-9644, 2024.
Pub-
Chen, S., Campanella, G., Elmas, A., Stock, A., Zeng, J., Polydorides, A. D., Schoenfeld, A. J., Huang, K.-l., Houldsworth, J., Vanderbilt, C., and Fuchs, T. J. Bench- marking Embedding Aggregation Methods in Computa- tional Pathology: A Clinical Data Perspective, July 2024b. arXiv:2407.078411
Khosla, P., Teterwak, P., Wang, C.,, Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. Supervised Contrastive Learning, March 2021. arXiv:2004. 11362 [cs].
Kolesnikov, A., Beyer, I.., Zhai, X., Puigcerver, J., Yung. J., Gelly, S., and Houlsby, N. Big transfer (bit): General vi- sual representation learning. In Computer 2020: 16th European Conference, Glasgow, UK, Au- gust 23-28, 2020, Proceedings, Part V 16, pp. 491-507. Springer, 2020.
Ding, J., Ma, S., Dong, L., Zhang, X., Huang, S., Wang, W., Zheng. N., and Wei, F. LongNet: Scaling Transformers to 1,000,000,000 Tokens, July 2023. arXiv:2307.02486 [cs].
Ding, T., Wagner, S. J., Song. A. H., Chen, J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., et al. Multimodal whole slide foundation model for pathology. arXiv preprint arXiv:241 1. 19666, 2024.
Kornblith, S., Shlens, J., and Le, Q. V. Do better imagenet models transfer better? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2661-2671, 2019.
Ericsson, L.,, Gouk, H., and Hospedales, T. M. How well do self-supervised models transfer? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5414-5423, 2021.
Kumar, M., Houlsby, N., Kalchbrenner, N., and Cubuk, E. D. Do better imagenet classifiers assess perceptual similarity better? Transactions on Machine Learning Research, 2022. ISSN 2835-8856.
Fang, A., Kornblith, S., and Schmidt, L. Does progress on imagenet transfer to real-world datasets? Advances in Neural Information Processing Systems, 36, 2024.
Li, B., Li, Y., and Eliceiri, K. W. Dual-stream Multiple Instance Learning Network for Whole Slide Image Classi- fication with Self-supervised Contrastive Learning, April 2021. MrXiv:2011.0892 [cs].
Gillette, M. A., Satpathy, S., Cao, S., Dhanasekaran, S. M., Vasaikar, S. V., et al. Protcogenomic Characterization Reveals Therapeutic Vulnerabilities in Lung Adenocar- cinoma. Cell, (tl:):2002222..... July 2020. ISSN 0092-8674. doi: (e.1016/f.cell.22000
Li, J., Chen, Y., Chu, H., Sun, Q.. Guan, T. Han, A., and He, Y. Dynamic Graph Representation with Knowledge- aware Attention for Histopathology Whole Slide Image Analysis, March 2024. arXiv:2403.0771 [es].
He, K., Zhang, X., Ren, S., and Sun, J. Deep Resid- ual Learning for Image Recognition, December 2015. [es].
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., and Mahmood, F. Data-efficient and weakly supervised computational pathology on whole- slide images. Nature Biomedical Engineering, 5(6): 555-570, June 2021. ISSN 2157-846X. doi: 10.1038/ $41551-020-0088 0000000..0..... Number: Publisher: Nature Publishing Group.
He, K., Girshick, R., and Dollár, P. Rethinking imagenet pre- training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4918-4921 2019.
Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J., and Zou, J. A visual-language foundation model for pathology image analysis using medical twitter. Nature medicine, 29(9):2307-2316 2023.
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., Parwani, A. V., Zhang, A., and Mahmood, F. A visual-language foundation model for computational pathology. Nature Medicinee 30(3):863 -744, March 2024.
Ilse, M., Tomezak, J. M., and Welling, M. Attention- based Deep Multiple Instance Learning, June 2018. arXiv: 1880000271111 [es].
ISSN 1546-170X. doi: 10.1038/441591-0244 Publisher: Nature Publishing Group.

---

Do MIL Models Transfer?
Mehra, A., Zhang, Y., and Hamm, J. Understanding the transferability of representations via task-relatedness. In Advances in Neural Information Processing Systems, 2024.
Data, 9(1):55, February 2022. ISSN 2052-4463. doi: 10. 10.1038/441197-022-015555 Publisher: Nature Pub- lishing Group.
Satpathy, S., Krug, K., Jean Beltran, P. M., Savage, S. R., Petralia, F., al. A proteogenomic portrait of lung squa- mous cell carcinoma. Cell, 184(16):4348 gust 2021. ISSN 0092-8674. doi: 10. 10.1016ff.ce1.2021..77 016.
Network, C. A. Brat, D., Verhaak, R., Aldape, K., Yung, W., et al. Comprchensive, Integrative Genomic Analysis of Diffuse Lower-Grade Gliomas. New Eng- land Journal of Medicine, 372(26):2481-2499 June 2015.
Au-
ISSN 0028-4793.
Shaikovski, G., Casson, A., Severson, K., Zimmermann, E., Wang, Y. K., Kunz, J. D., Retamero, A., Oakley, G., Klimstra, D., Kanan, C., et al. Prism: A multi-modal gen- erative foundation model for slide-level histopathology. arXiv preprint arXiv:2405. 10254, 2024.
Neyshabur, B., Sedghi, H., and Zhang, C. What is being transferred in transfer learning? Advances in Neural Information Processing Systems, 33:512-523, 2020.
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., et al. DI- NOv2: Learning Robust Visual Features without Super- vision, February 2024. arXiv:2304.07193.
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X., and Zhang, Y. TransMIL: Transformer based Correlated Multiple Instance I Learning for Whole Slide Image Clas- sification, October 2021. arXiv:2106.000 0.0900 [cs].
Pruksachatkun, Y., Phang, J., Liu, H., Htut, M., Zhang, X., Pang. R. Y., Vania, C., Kann, K., and Bowman, S. Intermediate-task transfer learning with pretrained lan- guage models: When and why does it work? In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguisties, pp. 5231-5247, 2020.
Song, A. H., Jaume, G., Williamson, D. F., Lu, M. Y., Vaidya, A., Miller, T. R., and Mahmood, F. Artificial intelligence for digital and computational pathology. Nu- ture Reviews Bioengineering. 1(12):930-94) 2023.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning Transfer- able Visual Models From Natural Language Supervision, February 2021. arXiv:2103.00020 [es].
Song, A. H., Chen, R. J., Ding. T., Williamson. D. F. K.,
Jaume, G., and Mahmood, F. Morphological Prototyp- ing for Unsupervised Slide Representation Learning in Computational Pathology, May 2024. 1xXiv:2405.1144
Tang, W. Zhou, F., Huang, S., Zhu, X., Zhang, Y., and Liu, B. Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology, July 2024. arXiv:2402.17228 [cs].
Raghu, M., Gilmer, J., Yosinski, J., and Sohl-Dickstein, SVCCA: Singular Vector Canonical Correlation Anal- ysis for Deep Learning Dynamics and Interpretability, November 2017. arXiv:1706.05886 [stat].
Tripuraneni, N., Jordan, M., and Jin, C. On the theory of transfer learning: The importance of task diversity. Advances in Neural Information Processing Systems, 33: 7852-7862, 2020.
Raghu, M., Zhang, C., Kleinberg, J., and Bengio, S. Trans- fusion: Understanding transfer learning for medical imag- ing, Advances in Neural Information Processing Systems, 32, 2019.
Vaidya, A., Zhang, A., Jaume, G., Song, A. H., Ding, T., Wagner, J., Lu. M. Y., Doucet, P., Robertson, H., Almagro-Perez, C., et al. Molecular-driven foun- dation model for oncologic pathology. arXiv preprint arXiv:2501.16652 2025.
Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. Do imagenet classifiers generalize imagenet? In Interna- lional conference on machine learning, pp. 5389-5400. PMLR, 2019.
Renggli, C., Pinto, A. S., Rimanie, L., Puigcerver, J., Riquelme, C., Zhang, C., and Lučić, M. Which model to transfer? finding the needle in the growing haystack. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 9205-9214, 2022.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention Is All You Need, August 2023. arXiv:1706.03766
Vorontsov, E., Bozkurt, A., Casson, A., Shaikovski, G., Zelechowski, M., et al. Virchow: A Million-Slia -Sidde Digital Pathology Foundation Model, January 2024. arXiv:2309.07778.
Dnetzer-Pejrimooskke T., Moser, A.-C., Atli, B., Vogel, C. C., Mercea, P. A., Prihoda, R., Gelpi, E., Haber- ler, C., Höftberger, R., Hainfellner, J. A., Baumann, B., Langs, G., and Wochrer, A. The Digital Brain Tu- mour Atlas, an open histopathology resource. Scientific
Wagner, S. J., Reisenbüüchler, D.,, West, N. P., Niehues, J. M., Zhu, J., et al. Transformer-bann based biomarker prediction
12

---

Do MIL Models Transfer?
from colorectal cancer histology: A large-scale multicen- tric study. Cancer Cell, 41(9):165--160 September
Zhou, H.-Y., Lu, C., Yang, S., and Yu, Y. Convnets trans- more trans- ferable? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2230-2238, 2021.
formers: Whose visual representations
2023. ISSN 15356108.
doi: 10. 10.1016fj.ceel.2223..00 202.00000......
Wang, X., Yang. S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., and Han, X. Transformer-based unsu- pervised contrastive learning for histopathological image classification. Medical Image Analysis, 81:102559, Octo- ber 2022. ISSN 1361-8415. doi: 10. 102559.
Zoph, B., Vasudevan, V., Shlens, and Le, Q. V. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 8697-8710, 2018.
Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., et al. A pathology foundation model for cancer di- agnosis and prognosis prediction. Nature, 634(8035): 970-978, October 2024. ISSN 1476-4687. doi: 10. Group.
Publisher: Nature Publishing
Xiang. and Zhang, J. Exploring low-rank property in multiple instance learning for whole slide image classi- fication. In The Eleventh International Conference Learning Representations, 2023.
Xu, F., Zhu, C., Tang, W., Wang, Y., Zhang, Y., Li, J., Jiang. H., Shi, Z., Liu, J., and Jin, M. Predicting Axillary Lymph Node Metastasis in Early Breast Can- Using Deep Learning Primary Tumor Biopsy Slides. Frontiers in Oncology, 11:759007, October 2021. ISSN 2234-943X. doi: 10.3389/fon:202217599000 arXiv:2112.02222 [physics].
Xu, H., Usuyama, N., Bagga, Zhang. S., Rao, R., Nau- mann, T., Wong, C., Gero, Z., González, J., Gu, Y. al. A whole-slide foundation model for digital pathology from real-world data. Nature, pp. 1-8, 2024a.
Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., et al. A whole-slide foundation model for digital pathology from real-world data. Nature, 630(8015) June 2024b.
ISSN 1476-4687. doi: 10.
Publisher: Nature Publishing Group.
You, K., Liu, Y., Wang, J., and Long, M. Logme: Practical assessment of pre-trained models for transfer learning. In International Conference on Machine Learning, pp. 12133-12143. PMLR, 2021.
Zhang, H.. Meng. Y., Zhao, Y., Qiao, Y., Yang, X., Cou- pland, S. E., and Zheng, Y. DTFD-MIL: Double- Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification, March 2022. arX:v:2203. 1208 [cs].
Zhao, N., Wu, Z., Lau, R. W. H., and Lin, S. What makes instance discrimination good for transfer learning? In International Conference on Learning Representations, 2021.
13

---

Do MIL Models Transfer?
A. Transfer performance across pretraining tasks
ABMIL
Transformer
BRACS-C
76-4
$LL Go
BRACS-F
31-2 COL
36-3 2L
28:3
41.7
21.7
2
28.0 31
46.1
BRACS-F
BRCA
$0.4
$G.L
85.0
BRCA HERZ
35. D3
50
60.3
39.4
04.
$3.5 47.
54.
53.59
03.2
29.5 51
530 66
BRCA HERZ
67.0
13.0 72.0
BRCA
BREA PM 550
75015
923
76-
77.5
BACA-P1
SCNN HARZ
67.6 686
BONB- H2N4
75.7
260
74.6
75.5
750
70:
40.7
RCNB-PR
MICLLC
MSCLC
05.2
03.
$5 05.0
05 60.5
05.3 65
24.7
15.0
NSCLC OGFR
91.
59.0
64
$8.0
550
75.7
NSCLC-
03.0
NSCLC KRAS
$1.7 01.0
57.7
26-4
63.0
66.
52.0
63
07.22 61
70
27.4
05.0
70
$2.0
03.0
56.5
MSCLC STKIL
7L9733
NSCLOCPSS
MSCLC TP53
75A
77.3
78.0
53
50. 22
60.0
504
60.9 85
EORAWI-C 97. 19
05.0
57.
PBRAIIS+P
71 23
71
09.5
665
45
09
66
GRML6G-C 41.2
40
91.1914
GANLGG-C
90.8
PANDA
M0. 40-4 90.0 10
90.5
50 4. 017 40.0
M14 R03
PANDA
+4.3 41.4 141
$1.
90.3 62.4
COL AVG
75.0
72.6
scoee
Rcres seaec
acRe--p P9
eecte
weick CeE
rantaes-
BRACS--
RONE HinC
NECL Eetore
NocLE
NSCL CaRas
ROCLE-STPLL MBCLE-PPP3 ERGAREEE
TransMIL
DFTD
DRACS-C
KL- S
37.1
$7
C4LD $2.4
35.7
b0 bo
X0X 64.7
50.3
86.3
50.5
BRACS-C M
65-
$4.3 60
b6.5
65.9
64.5
57.0
67
200 00
BRCA
55
12
BRCA- #F
05
05
0%
BRCA-FIKICA
56.2
91-361
95.0
BRCA-FAC3CA 42
BRCA-PR
BACA-PR
BENB-ER
90.5
00.5
SCHB ER
85.8
60.4
BCHD-HER2
71.
20.5
80.0
10.2 60.3
BCNB- HENB-HER2--50
72.3
73.6
72.7
23.3
79.
BCNO
$1.4
00.000
78.0 80.3
$7.2
BCMB-PRI
25
02.7
NSCLC
NSCLC
99.0
MSCLC-
15.3
633
NSCLC-RRAS
634
63.15
61.6
55.6
652
69.0
63.3613 63
71
61
NSCLC KRAS
$5.0
60
59
58.
60.3
99
02.9
65.0
43.0
10.2
74,6
NSCLC-FP52
763
33.1
81
76
74.2 41
79. 79
62
752
60
NSCLC-PPS2
41.
214
90.5 65
73.0 752
72.
27.0
60
760
73.8
06
70.6
00
EGRAING-F
C3.0
660
63.0 67
$1.3 012
07
27.7 67
66-4
60
EBRABIS-F
55.1 66.66
$5.5 G0
54.5 05.7
00.
66.
01.
08. 60.6
CDMLOG-F 50.3 $1.2
50.
55
52
47.2
CBNLOC
$1. 51.8
53
42. 53.6
13.1
56.2 53.0
COL AVC
72.8 23.4 73.523 25.4 72.0 75
73.6
72. 22.
22.7
25 223
Ma
COL AVG
25.8
23.2
PIL
75,7 24. 72.
N6 25.5
22.0
73.6 72.6
72.6 23,4 75. 75.6 226 72
BRACIHE e
Deca-- 2M
BRick vronz
BRCC PRECECA
BCCND
wore wEens
SCNN-P
RECLE
FESCL HACL CHERA
Gav C
PANDA
BEACE-P
Bacck weens
BeCA PRESCA
Deca- PM
MECLE-Sorr woo
NaC Cucked
Gave C
cnei crive
Peease
Figure AI. Finetuning performance with pretraining on public datasets. Performance of various MII. models initialized with different pretraining tasks (columns) each evaluation dataset (rows). Diagonals indicate performance when trained from scratch. Colormap is now-noral according to the difference between each pretrained model and the model trained from scratch, such that red corresponds to an increase in performance compared to random initialization, and blue indicates a decrease in performance. Metries reported are AUROC for binary classification tasks, weighted Cohen's 's Kappa for PANDA, and balanced accuracy for all other multiclass problems. We find that across all models and pretraining tasks, pretraining generally leads to an improvement in performance (indicated by the large proportion of red). Each model was finetuned for 10 epochs.
14

---

Do MIL Models Transfer?
Average
ABMIL
HER2
DECMPRICA,
BRCA-FIACC
BECA-PE 20331
BRCA-F
BENS-ER 772713
06.065
665
03.5
ECNB m
04 75-7
67265
135
Relative Improvement
SOND-HER 24016220 C10
DC 46R
RCME-R
EOVE-PR
RSCLE 344
17. 7E0
MOCIC
RSCLC CUTC 57445444 A
KSCUC4GI
GOMLOG-C
COMLCG
GEPLLGE
68ML0G
PANDA 35
FANDA
COL APC
$2.0 1 C 01.2 Siwe 80 5 01.3 2017 50.g 33.3 doo 12
COL AVG
20. d8 da
BRACI
ecve-se
pure stred
prts camer
Coki
pateii the
TransMIL
Transformer
BRACH-C
BRACK-
XLT 04
246
22.4
23
25.
25.8
BROA-ER 090
BROV-R
Relative Improvement
SCAB-RR
LONS-FR
NUCLE
21-433
MBCIC
NA
MACIC ROR 13 Mi
VL
MSCLC KRAS 62- 52-257
560
604
60.6
592
ASCUCHRAS 435
14
550
RSCLC-OTEES
S6G S20
55.4
K0
64.2
RSCLE-STRLL
66 500
RSCLOTPES
22
ESRAWIS-C
53.0 20.
EBRABIS
bacak staep
backk aased
Raack
Figure A2. KNN performance public datasets. Performance of various MIL models initialized with different pretraining tasks (columns) on cach evaluation dataset (rows). Colormap is row-normalized according to maximum and minimum performance on each task. Metrics reported are AUROC for binary classification tasks. weighted Cohen's Kappa for PANDA, and balanced accuracy for all other multiclass problems. We find that across transformer- based models, the PC-108 pretrained model consistently resulted in the best knn transfer across pretraining tasks. Meanwhile, the best-performing pretraining model varied.
15

---

Do MIL Models Transfer?
B. Implementation details
B.1. Multiple Instance Learning implementation
All MIL models are adapted according to their official implementation, using the default hyperparameters provided by their official codebases. For meanMIL and maxMIL,, which do not have official codebases, we first apply a linear layer paramaterized by W E Rdx511, followed by ReLU activation, onto each d-dimensional patch embedding. We then obtain slide-level prediction by feeding the average of task-specific embeddings through a classification head. For MaxMII, feed each patch through W, then pass each transformed patch embedding through a classification head, and select the patch with the highest logit as the final slide representation.
B.2. Training details
We train all models with the AdamW optimizer with a learning rate of 1 10 cosine decay scheduler, and mixed precision according to Pytorch's 's native implementation. For datasets with a validation set, we train with a maximum of 20 epochs with an early stopping patience of 5 epochs for a minimum of 10 epochs. For datasets without a validation set, we train for 10 epochs. We use cross-entropy loss with random class-weighted sampling and a batch size of 1. For regularization, we use a weight decay of 1 x 10-5, a dropout of 0.25 at every feedforward layer, and a dropout of 0.1 on the features from the pretrained encoder. Experiments were performed across four NVIDIA RTX A4000s, three NVIDIA GeForce RTX 2080 Tls GPUs, and three RTX 3090s, with single GPU used per experiment.
C. Dataset Description
We briefly describe the datasets that were used to evaluate MIL transfer.
C.1. Morphological Subtyping
EBRAINS Sobetzer-Pejrimovskk et al., 2022): We perform coarse-grained (12 classes) and fine- ane-ggriiedd (30 classes) classification of brain tumor subtypes. The dataset consisted 2,319 Hematoxylin and Eosin (H&F) Formalin-fixed and paraffin-embeddee (FFPE) Whole Slide Images (WSIs). We use label-stratified train/valltest splits (50% 25% / 25%) provided by UNI(Chen et al., 2024a) with the same folds for both coarse- and fine- ane-grained tasks. We evaluate performance using balanced accuracy.
The non-small cell lung carcinoma (NSCLC) subtyping task was binary classification problem for distinguishing lung adenocarcinoma (LUAD) and lung squamous cell carcinoma (LUSC). The training data consisted of publicly available H&E WSIs from TCGA (n = 1, 041 slides). We performed an 80% / 10% 10% train/val/test split on the TCGA dataset for training and internal validation, and evaluated the trained model on two external datasets: the Clinical Proteomic Tumor Analysis Consortium (CPTAC, n 1, 091 slides) and the National Lung Screening Trial (NLST, n = 1, 008 slides) (Campbell et al., 2016; Satpathy et al., 2021; Gillette et al., 2020). We report the AUROC averaged across the TCGA, NLST, and CPTAC test sets for all results.
NSCLC:
PANDA (Bulten et al., 2022): We used prostate cancer core needle biopsies (n 10, 616) from the Prostate Cancer Grade Assessment (PANDA) challenge to perform 6-class classification according to the prostate cancer grade. We use the same train/val/test folds (80% / 10 % / 10%) as UNI, and evaluate using Cohen's 's quadratic weighted Kappa k metric.
BRACS (Brancati et al., 2021): The BRACS subtyping task consisted of a 3-class coarse-grained classification task to distinguish benign, malignant, and atypical breast carcinoma H&E slides, as well as a fine-grained 7-class classification task that classifies benign tumors into three subtypes, atypical tumors into two subtypes, and malignant tumors as two subtypes. We use the official train/val/test folds (72% 12% / 16%). with the same folds for both coarse- and fine- fine-graine grained tasks. We evaluate performance using balanced accuracy.
Pancancer subtyping: These tasks consisted of cases collected at Brigham and Women's Hospital, from which we defined a 108-class fine-grained label and a 43-class coarse- -aarine label set. The fine-grained task sought to identify one of 108 OncoTrec codes, while PC-43 sought to identify one of 43 cancer types. The cases were divided into a ratio of 3,944 slides for training, and 1,620 slides for validation, with the same splits for both tasks. These datasets were previously named OT- and OT-43 in the UNI study (Chen al., 2024a).
16

---

Do MIL Models Transfer?
C.2. Biomarker Prediction
Lung cancer biomarkers: We evaluate H&E-stained WSIs for the binary classification task of predicting mutation status of TP53, KRAS, STK11, and EGFR in TCGA lung cancer cases (n 524 slides) (Network et al., 2015), with each task site- and label-stratified into approximate train/val/test splits (60% 20% / 20%), with the same splits for all tasks. We evaluate performance using AUROC.
Breast cancer biomarkers: We predicted the binary mutation status of ER, PR, HER2, and PIK3CA on H&E-staine WSIs from TCGA breast cancer (BRCA) cases (n = 1, 034), each site-stratified and label-stratified an approximate train/val/test splits (60% 20% / 20%). Additionally, we evaluate on breast cancer core needle biopsies (BCNB, n = 1, 058) (Xu et al., 2021) in label-stratified train/test split (90% / 10%) for ER, PR, and HER2, with the same splits for all tasks. We evaluate performance using AUROC.
GBMLGG mutational subtyping (Brennan et al., 2013; Roetzer- Pejrimovsky et al., 2022): These tasks include binary coarse-grained mutation prediction of IDHI status using the TCGA GBMLGG dataset (1,123 slides), and 5-class fine- ime-grained histomolecular subtyping. The 5-class histomolecular subtyping task was separated into the categories of Astrocytoma, IDHI -mutant,,, Glioblastoma, IDHI-mutant, Oligodendroglioma, IDHI-mutant and Ip/19q codeleted, Astrocytoma, IDHI- wildtype, and Glioblastoma, IDHI -wildtyee For training and evaluation of both tasks, we use the UNI splits, which label-stratiffed (CGA-GBMILGG into train/val/test split with 47:22:31 ratio with the same splits for both coarse- and fine-grained tasks. Additionally, perform external validation the held-out EBRAINS cohort (n 873 slides) for the cases with known IHDI status. We evaluate GBMLGGG-carse with AUROC, and GBMLGG-fine with balanced accuracy.
Shared split experiments: For the experiments depicted in Figures Al and A2, we perform transfer learning by pretraining on the training set of one dataset and evaluating on the test set a separate dataset. However, we observed that some official dataset splits (specifically, NSCLC, BRCA, and BCNB) contain overlapping samples-meante appear in the training set for one label and the test set for another label. This overlap could result in data leakage during evaluation, thereby inflating performance estimates. To mitigate this risk, we employ multi-label stratified splits for the NSCLC, BRCA, and BCNB datasets in these specific experiments. With this approach, dataset (e.g., NSCLC TP53 and NSCLC EGFR) share an identical set of training samples, eliminating any possibility of overlap between training and testing data across labels. These custom multi-label splits are publicly available at the GitHub repository. For all other experiments, we use the official or previously published dataset splits. This decision facilitates reproducibility and enables direct comparison with prior work.
that certain instances
ensure that all tasks within
17

---

Do MIL Models Transfer?
D. Expanded Results
D.I. ABMIL Scaling Implementation
Table AI. Model Configurations for ABMIL with different parameter counts with UNI patch features. Embed dim: output dimension pre are-attention linear layer. Attn Dim: Hidden dimension of attention2 embedding. Num FC Layers: number of pre-attention lincar layers.
FC Hidden Dim: hidden dimension(s) of pre-attention linear layers.
Params
In Dim
N Classes
Embed Dim
Attn Dim
Num FC Layers Hidden Dim
8,530,675
1,024
512
512
[2048, 1536, 1024, 768]
6,837,292
1,024
512
512
[2048, 1280, 768]
5.249,027
1,024
512
512
[2048, 1024]
3,084,931
1,024
512
384
[1280, 768]
1,445,507
1,024
512
384
[512, 512]
920,195
1,024
512
384
591,747
1,024
384
256
394,755
1,024
256
256
164,611
1,024
128
128
Table A2. Model Configurations for Transformer with different parameter with 1024-dimensional UNI patch features. N. Layers: Number of Transformer encoder layers. Encoder Hidden Dim: Hidden dimension of encoder feedforward network. N. FC Layers: number pre-attention linear layers. Hidden Dim: hidden dimension(s) of atteentnnion linear layers. "." indicates no feedforward network.
Params 9,984,514 14
Embed Dim
N. Layers
Encoder Hidden Dim
N. FC Layers
FC Hidden Dim
512
3
2048
6,837,555
512
1024
5,258,242
512
2
1024
[512, 512]
3,155,970
512
2
[1280, 768)
2,631,209
512
2
792,316
256
2
264,450
128
Table A3. Performance comparison of ABMIL and Transformer models without pretraining (Base). Performance is averaged across eight tasks: BRCA ER/PR, NSCLC TP53/STKI1, GBMLGG C/F, and BRACS C/F. The results show that ABMIL performance plateaus in the 5-7M parameter range before decreasing 9M. At larger sizes, Transformers benefit substantially from pretraining, outperforming ABMIL at 7M, though both models show reduced performance at 9M.
range of sizes, both with pretraining (PC-108) and
Base
PC108
Approx Params
ABMIL
Transformer
ABMIL
Transformer
0.2M
70.5
67.4
73.1
72.9
1M
70.1
68.8
72.6
72.6
2.6M
64.4
70.5
3.2M
71.3
65.2
74.0
71.2
5.2M
70.7
65.1
75.7
73.8
7M
71.3
66.3
75.4
76.8
9M
70.6
67.5
74.1
72.1
18

---

Do MIL Models Transfer?
Table Performance across random seeds. Average performance with 95% confidence interval according to runs with different random seeds. For each task, base models were randomly initialized with different weights for each random seed. PC-108 models used
the same pretrained model and were finetuned with 5 random seeds for each downstream task. Average performance and average of
standard deviation is shown in the final row and final column, Task
Initiaiizatin
ABMIL
DFTD
RRT
TransMIL
Base
2.7
6..6
65.7
63.9 6.5
BRACS-C
PC-108
68.8 2.4
63.7 5.4
68.6
63.6 2.0
A
-2.6
-1.0
+2.8
-0.3
Base
41.3 2.3
41.6119
3..4
3..6
BRACS-F
PC-108
40.9 + 1.8
12.0
36.2
40.0 +0.0-:2.2
-0.4
-2.6
+0.9
+55.4
Base
0.3
9.441044
9.4
9..0
NSCLC Morph
PC-108
95.2
0.5
94.7
95.1
A
-0.2
+0.1
+0.3
+-0.1
Base
1.5
91,1:02
90.6
9.25
BCNB-ER
PC-108
88.8 1.1
9..1
89.6
1.3
-0.4
-1.0
-1.0
-1.8
Base
1.1
0.66
70.7
6.740077
BCNB-HER2
PC-108
75.5
2.6
7..6
3.5
A
+1.0
+0.1
+1.9
-1.8
Base
0.7
83.0
8..
0.3
BCNB-PR
PC-108
1.
82.612.1
7.
8.1
0.4
-6.2
+1.0
Base
1.0
1.3
8.
8.1
BRCA-ER
PC-108
1.3
1.
1.3
3.6
A
40.0
0.4
+2.8
-122
Base
6.4
6.3
3712
6,.1
BRCA-HER2
PC-108
2.9
6.07
5.29
6..7
+2.3
42.1
-3.3
+0.6
Base
1.
64.3
2.6
4.07
BRCA-PIK3CA
PC-108
06.81144
3.3
6.10
0.9
A
4.5.2
+0.2
-02
400.1
Base
0.2
7.
7..6
7.37
BRCA-PR
PC-108
3.2
79.5
8L.5 1.0
2.22
400.7
-0.1
46.9
+2.99
Base
8.1
844
8..5
8.66
EBRAINS-C
PC-108
1.4
88. 1.8
88.5
8.84
A
1.2
+3.7
+1.0
+1.8
Base
18.81.44
6..9
0.6
1.04
EBRAINS-F
PC-108
68.8 1.0
6.
1.6
1.1
67.3 1.4
+0.0
+0.8
-0.1
Base
1.5
I.1
93.7
1.3
GBMLGG-C
PC-108
93.6 13.61. 1.3
91.7 1.1
93.0
0.5
A
0.1
0.7
-07
+0.6
Base
1.0
16.9 1 4.3
52.3 1 1.0
49.0124
GBMLGG-F
PC-108
51.6 2.0
518
2.7
55.2 3.7
43.4
A
+0.8
+4.9
+2.9
+0.9
Base
6.7312.7
3.3
6.18
1.571115
NSCLC-EGFR
PC-108
4.9
71.7
3.5
69.9
A
+9.2
+1.6
+3.3
+7.2
Base
6.7612277
12.2144
61.714.4
NSCLC-KRAS
PC-108
6.21
61.1
67.5
63.3 3.1
A
-1.2
-0.4
+5.3
+1.6
Base
1.
7..312.9
75.8
6.6417.6
NSCLC-STKI1
PC-108
3.5
8.64
8.67
75.3
+5.00
+2.1
+-6.9
+12.9
Base
7,62
80.9139
7.20112
74.0
NSCLC-TP53
PC-108
8..8
2.9
73.7
79.4
A
44.6
-0.1
-0.2
+5.4
Base
9.4
9.5
9.1
9..5
PANDA
PC-108
0.3
9..
91.7
9.4
+0.4
0.3
+0.6
+0.3
Base
2.27
74.822.14
12.91178
70.9
Average
PC-108
7
75.2 1 1.98
74.11151
7..4
A
+1.4
+0.4
+1.2
+2.6
19

---

Do MIL Models Transfer?
Table A5, Pretraining performance with different encoders for TransMIL. Performance across different tasks for TransMIL using CTransPath and ResNet patch feature encoders with PC- (C- pretraining and random initialization. Best performance between Basc and PC-108 for each encoder is bold.
CTransPuth
ResNet
Task
Base
PC-108
Base
PC-108
BRACS-C
59.1
57.6
$1.0
54.1
BRACS-F
39.0
38.6
26.1
29,9
EBRAINS-C
81.0
76.5
63.9
61.4
ERRAINS-F
53.9
54.9
46.8
42.5
PANDA
90.2
87.2
83.5
77.6
BRCA-ER
81.6
80.5
70.5
64.2
BRCA-HER?
62.9
58.6
70.5
64.9
BRCA-PIK3CA
52.8
53.1
52.4
69.5
BRCA-PR
57.7
76.3
59.6
68.3
NSCLC EGFR
59.1
69.4
48.2
66.0
NSCL.C-KRAS
59.1
65.8
65.0
67.7
NSCLC-STKII
62.4
79.0
64.3
76.9
NSCLC-TP53
60.8
GBMLGG-C C
91.4
89.6
53.4
78.7
GBMLGG-F
53.0
55.7
53.0
60.8
Average
65.1
67.94
59.5
66.3
Table A6. Performance per task with pretraining and random initialization. Results of MIL methods with random initialization (Base) and PC- 108 supervised pretraining. The number of classes specified below the task. The evaluation metries for each task are
indicated in parentheses. All models use UNI features as patch embeddings (Chen et al., 2024a). Performance on NSCLC subtyping is
averaged across the internal TCGA cohort and the external NLST and CPTAC cohorts
Task
Init.
ABMIL.
CLAM
DFTD
DSMIL
ILRA
RRT
Til.
Transformer
WIKG
mixMIl.
Meln
BRACS-C
Base
(5.3)
55.9 (5.44
64.3 (4.8)
66.8 (4.0)
(4.0
(3.9)
(4.3
(3.3
60.1
64.3 (4.00)
61.6 (4.9)
C
PC-108
68.5 (4.8)
(5.8)
(8.4
69.7 (6.9)
(4.3
(4.44
(2.8
(3.4
(4.)
58.7 (s.5)
(2.8(5.5)
BRACS-F
Base
(5.1) (3.8)
(4.5)
402 (4.1)
(3.9)
(4.6)
36.1 (3.3)
(3.3
40.4 (4.2)
(4.7
(4.2)
C 7
PC-
108
47.1 (5.3)
(4.1)
(4.5)
44.2 (4.1)
36.2 (5.0)
(5.6
(2.9
46.8 (4.7)
30.0 (1.1)
NSCLC
Base
95.3 (0.0)
91.1 (0.8)
92.1 (a.1)
94.0 (0.9)
90.4 (a.7)
93.9 (a.7)
(o.8)
93.3
91.3 (0.8)
95.9 (0.6)
91.1 (0.7)
C=2
PC-108
96.1 (0.0)
92.0 (0.8)
95.3 (0.7)
94.8 (0.7)
94.7 (0.7)
95.4 (0.7)
(0.4
(0.7)
95.4 (0.6)
92.3 (0.7)
BCNB ER
Base
85.7 (4.4)
88.1 (3.0)
85.3 (3.6)
87.4 (2.7)
87.4 (2.8)
82.1 (4.0)
67.4
81.1 (3.1)
90.1 (9.6)
88.4 (3.4)
86.3 (2.9)
C
PC-108
88.6 (3.0)
89.6 (2.7)
92.3 (3.6)
92.6 (2.7)
(2.8)
(2.9)
90.6 (3.7)
89.8 (3.m)
(3.8)
89.2 (3.0)
89.5 (2.9)
BCNB
Base
75.4 (4.8)
72.4 (5.8)
72.3 (6.0)
72.5 (a.1)
71.4 (5.6)
71.9
(5.2)
65.8 (6.2)
76.6 (4.7)
73.9 (6.2)
68.4 (8.0)
C
PC-108
79.8 (4.2)
74.4 (5.8)
79.5 (9.0)
(6.1)
71.0 (5.6)
72.6 (5.6)
71.6 (5.9)
(0.2)
75.7 (G.8)
70.3 (5.2)
70.4 (6.0)
BCNB PR C=
Base
81.5 (4.5)
81.1 (5.5)
78.3 (5.9)
79.0 (4.8)
78.1 (5.0)
80.8 (5.0)
76.1 (4.5)
80.4 (4.4)
83.6 (4
86.7 (4.8)
79.9 (6.1)
PC-108
85.2 (4.7)
83.2 (5.2)
86.4 (5.9)
82.5 (4.8)
(5.0)
75.1 (5.8)
(G.7)
85.7 (5.1)
82.5 (5 .3
89.6 (4.8)
84.4 (6.1)
ER
Base
81.4 (3.6)
80.7 (4.3)
(2.8)
82.9 (3.7)
8022 (3.8)
8188 (4.9)
76.1 (4.1)
76.0 (3.8)
83.1 (9
83.6 (3.9)
80.9 (4.4)
2
PC-108
84.2 (2.9)
81.8 (8.2)
(2.9)
(3.7)
85.0 (3.0)
86.3 (3.1)
(3.6)
(2.7)
83.3 (9 (3.8)
84.8 (2.9)
82.9 (3.6)
BRCA HER2
Base
63.2 (5.1)
61.8 (5.0)
(5.2)
61.6 (5.8)
64.4 (4.8)
55.3 (6.0)
55.8 (5.6)
64.4
$3.5 (0
$7.7 (6.6)
51.7 (6.9)
C
PC-108
(4.2)
50.6 (5.7)
54.1 (5.7)
58.3 (5.8)
(4.0)
53,9 (5.1)
(5.4)
62.7 (5.0)
58.1 (5 (5.6)
66.8 (5.6)
62.6 (6.8)
PRCCAPIKC
Base
60.9 (4.0)
(3.9)
62.9 (3.0)
62.7 (3.7)
57.2 (3.5)
57.7 (4.1)
(4.8)
44.6 (4.5)
54.5 (4
55.2
58.8 (4.4)
C=
P0.108
67.9 (4.0)
60.3 (5.8)
(3.8)
(3.7)
54.1 (5.5)
60.0 (5-0)
(4.6
63.9 (4.5)
59.9 (3.6)
61.7 (1:3)
BRCA PR
Base
68.9 (4.5)
(3.9
68.3 (2.0)
72.9 (3.9)
72.0 (4.9)
(3.5)
(1.6
73.0 (3
09.4
73.5 (9:8)
C
PC. 108
(2.7)
67.6 (3.5)
79.1 (3.2)
80.7 (2.8)
78.5 (3.2)
(3-7)
78.4 (3.5)
74.9 (3.3)
(4
72.1 (3.2)
70.9 (3-3)
EBRAINS-C
Base
86. (2.1)
86.1 (2:3)
81,2 (1.0)
86.2 (1.8)
85.0
86.7 (2.0)
(2.1)
(2.6
83.6 Ca
82.0 (9:4)
86.3 (9:3)
C
PO.108
(1.6)
87.2 (2.5)
91.1
(1.8)
89.0 (1.w)
88.5 (2.0)
(2.4)
90.7 (1.8)
83.3 (2 1)
89.0 (2:4)
85.5 (2-3)
EBRAINS-F
Base
(2.3)
(2.0)
61.5 (2.0)
64,2 (2.1)
(2.9)
(1.6
62.1 (1:0)
(2.1)
66.6 (1
64.7 (9:1)
69.3 (9:2)
PC. 108
(2.3)
73.0 (1.9)
69.4 (2.0)
66.2 (2.1)
64.6 (2.2)
(1.9)
68.9 (2.1)
(2.6
73.0 W)
70.1 (2.1)
72.3 (2.2)
GEMLGG-C
Base
92.8
91.7 (1.7)
91.7 (a.9)
93.1 (1:4)
93.6
(1.8)
(1.0
92.9
92.4 C1
94.3
91.8 (1.0)
PC.
(1.2)
92.8 (1.7)
91.7 (2.2)
95.7 (1.4)
92.9 (1.0)
93.0
94.5 (1.1)
(1.5
92.3 ()
93.4 (1.))
91.3 (1-3)
GEMLGG-F
Base
45.4 (3.1)
43,9 (3.6)
$1.3 (3.9)
(3.6
(3.3)
(3.6)
44.9
41.6 23
50.1 (3
47.2 (9:2)
108
53.7 (2.9)
50.5 (5.9)
57.1 (3.3)
(3.2)
(3.1)
55.2 (3-5)
49,2 (3.6)
48.9 (5.9)
(9
$2.5 (3.4)
47.0 (2.8)
NSCLC-EGFR
Base
63.4 (7.0)
60.6
62,9
64.8
61.6 (8.8)
62.3
68.0
63.3
60.1 (8.6)
64.0 (9.3)
63.2 (8.1)
C=2
PC-108
62.6 (7.0)
67.8 (9.6)
60.1 (0.0)
64.9 (8.5)
69.1 (8.8)
73.7 (7.0)
69.9 (7.2)
19.1
$7.8 (9.6)
64.2 (8.1)
NSCLC-KRAS
Base
60.0
63.5 (5.4)
58.1
62,5
$2.9 (8.0)
(5.1)
59.3 (6.1)
60.0
$4.0 (5
62.2 (9.3)
63.6 (4.8)
C = 2
PC-108
(5.1)
64,3 (5.4)
71.4 (6.1)
61.5 (7.4)
60.8 (9.6)
67.5 (G.2)
(6.1)
64.8 (4.6)
62.3 (G.
60.6 (G.3)
$7.3 (6.9)
NSCLCSTKII
Base
72.5 (3.0)
69.9
65.4
71.5
65.4
72.4
65.1 (7.1)
68.8 (7.8)
65.8
70.8 (0.5)
66.4 (0.4)
C =
PC-108
(8.6)
12.3 (5.9)
(9.6)
71.5 (5.9)
77.9 (9.6)
82.7 (5.0)
(6.6
73.6 (4.9)
82.1 (7.1)
61.8 (4.4)
72.9 (5.9)
NSCLC-IPS
Base
75.8 (4.0)
69,5
74,2
71.4 (4.0)
72,2 (2.1)
72.1 (4.0)
72.3
69.7 (8.0)
79.3 (4.7)
72.2 (8.0)
C =2 2
PC-108
(3.6)
75.0 (4.6)
(4.3
82.1 (5.0)
80.2 (4.6)
13.7 (5.6)
80.5 (4.6)
83.8)
76.3 (4.6)
69.5 (4.7)
80.2 (4.5)
PANDA
Base
91.6 (0.7)
(1.0)
87,9
91.2
91.5 (0.9)
91.1 (0.9)
90.5 (0.9)
(0.4
92.1 (0.8)
89.7 (0.9)
90.2 (1.0)
C=7
PC-108
(o.6)
91.8 (1.0)
93.2 (0.T)
93.5 (0.7)
91.5 (0.9)
91.5 (0.9)
89.9 (0.8)
90.7 (0.8)
93.0 (0.8)
88.9 (0.9)
89.7 (1.0)
20


Reviews:
[{"review_id": "OicUOVKB7C", "reviewer": "Reviewer_RH9m", "summary": "This work explores the transferability of multiple instance learning in computational pathology. A variety of experiments are conducted to investigate how various factors affect the transferability, filling the gap of CPath community.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their extensive suggestions on further improving the rigor and insights from our work.\n\n**Q1. Model Size**\n***\nWe add larger ABMIL models (7M and 9M parameters) and Transformers at comparable sizes, finding:\n1) ABMIL performance plateaus in the 5-7M range, with a notable decrease at 9M parameters.\n2) At smaller sizes (0.2M and 1M), ABMIL and Transformer achieve similar performance both with and without pretraining.\n3) At larger sizes, Transformers benefit substantially from pretraining, even outperforming ABMIL at 7M. Similar to ABMIL, Transformers show reduced performance at 9M.\n\nWe display the performance averaged across BRCA ER/PR, NSCLC TP53/STK11, GBM C/F, BRACS C/F.\n|Approx Params|ABMIL|Transformer|ABMIL PC108|Transformer PC108|\n|-|-|-|-|-|\n|0.2M|70.5|67.4|73.1|72.9|\n|1M|70.1|68.8|72.6|72.6|\n|2.6M|-|64.4|-|70.5|\n|3.2M|71.3|65.2|74.0|71.2|\n|5.2M|70.7|65.1|75.7|73.8|\n|7M|71.3|66.3|75.4|76.8|\n|9M|70.6|67.5|74.1|72.1|\n\n**Q2. Why does ABMIL perform best?**\n***\nABMIL is consistently effective across CPath tasks and patch encoders [1]. In the data-restricted regimes common in CPath, the default transformer configuration may be overparameterized. Our results show that transformers can achieve comparable performance to ABMIL at smaller parameter counts and through pretraining at larger model sizes. \n\n[1] Campanella, Gabriele, et al. \"A clinical benchmark of public self-supervised pathology foundation models.\" arXiv (2024).\n\n**Q3. Few-shot**\n***\nThe target tasks in Figure 3 included molecular (NSCLC TP53/STK11/EGFR, BCNB ER/PR/HER2, GBMLGG C) and morphological classification (BRACS C/F). We included BRACS C/F, as PC-108 contains only invasive carcinoma cases (a single label in BRACS F). We recognize this could raise concerns and will exclude BRACS in the revision. \n\nShown below is performance averaged over ABMIL, TransMIL, Transformer, DFTD, and CLAM for molecular tasks alone and BRACS alone, confirming that few-shot learning benefits are robust to tasks without any label overlap.\n\n|k|MOL-PC108|MOL-PC43|MOL-base|BRACS-PC108|BRACS-PC43|BRACS-base|\n|-|-|-|-|-|-|-|\n|4|57.1 ± 5.2|52.8 ± 4.6|52.7 ± 2.9|35.4 ± 5.7|35.6 ± 4.4|26.8 ± 5.1|\n|16|64.1 ± 4.9|60.2 ± 4.8|56.4 ± 3.8|45.8 ± 4.5|44.2 ± 4.3|36.2 ± 4.3|\n|32|70.1 ± 4.2|66.9 ± 5.0|61.7 ± 4.4|45.9 ± 4.0|46.8 ± 4.2|39.1 ± 5.4|\n\n**Q4. Task Difficulty**\n***\nTo compare transfer quality between easy and hard tasks, we perform a paired t-test on transfer performance between the easiest and hardest task for each dataset. Given a fixed pretraining dataset, this allows us to investigate how the difficulty of the training objective affects the transferability of the final model. For morphological datasets, we assign coarse and fine subtyping as easy and hard tasks. For molecular datasets, which typically have multiple tasks (e.g BCNB ER/PR/HER2), we compare the model performance (averaged over ABMIL, DFTD, TransMIL, Transformer trained from scratch) on each task, selecting the task with the lowest and highest AUC as the hard and easy task, respectively. This resulted in the following:\n||BRACS|EBRAINS|GBM|PC|BCNB|BRCA|NSCLC|\n|-|-|-|-|-|-|-|-|\n|Easy|C|C|C|43|ER|ER|TP53|\n|Hard|F|F|F|108|HER2|PIK3CA|KRAS|\n\nFor each pair of pretraining tasks, we compare finetuning performance across the 17 remaining evaluation tasks for four MIL models, resulting in a total of 476 paired points (4 models x 7 pretraining x 17 evaluation). \nThe result shows that **pretraining on hard tasks leads to better transfer performance, with average improvement of +0.5 (95% CI=0.1-0.9, p=0.017)**. This indicates that challenging training objectives enhance the transferability of the final model.\n\n**Q5. PC Transferability**\n***\nAlthough ABMIL shows more variability in KNN performance, Figure A2 shows that PC pretraining consistently achieves strong average performance across all models. Specifically, PC-108 ranks second-highest in ABMIL and highest in both TransMIL and Transformer KNN evaluations. Figures 2 and A2 both therefore support our conclusion that PC pretraining produces robustly transferable features.\n\nOur finetuning results in Figure A1 further validate this finding, showing that although there is some variability, PC pretraining consistently outperforms random initialization and achieves the highest average performance among pretraining tasks across all models, including ABMIL. Thus, we believe our conclusion from Figure 2 is well validated by our finetuning results. We will make this connection clearer in the final submission. \n\n**Q6. Suggestions**\n***\n*Multitask*: We agree that multi-task pretraining is a promising next step and will investigate this in further iterations. *Data Splits*: We share splits between tasks from the same dataset to ensure fair comparisons. *Patch encoders*: We will add Virchow 2G results in the final paper. *Data availability*: To facilitate further work on this topic, we will release the model weights and code for PC108/43 pretrained models.", "claims_and_evidence": "Some experimental results need further justification.\n\n1. Given that highly parameterized models, e.g. transformer-based models, benefit more from pretraining, why does ABMIL perform best?\n\n2. Although the OT-108/43 task for pretraining demonstrates good transferability in KNN-based adaptation for target tasks (indeed, this does not hold in the KNN performance of ABMIL, see Figure A2), there seems to be no similar conclusions in finetuning performance (Figure A1). As such, I am concerned about how reliable the conclusions drawn from the results are in Figure 2. Please justify it.", "methods_and_evaluation_criteria": "This research fills the gap in the field of CPath and provides valuable guidance for developing MIL approaches in the future.", "theoretical_claims": "No theoretical proofs in this work.", "experimental_designs_or_analyses": "1. Data contamination. Among 19 tasks, several tasks have the same data although labels are different. In this case, I am not sure if the test sets of target tasks are used for pretraining. For example, has the test set of NSCLC-KRAS been included in the pretrain data of NSCLC-TP53? If so, we cannot tell which factor contributes to the performance gain, pretraining itself or data contamination.\n\n2. The details of few-shot experiments should be presented. What target tasks are in Figure 3? I am concerned that if the model has already seen the samples of target classes during pretraining, which should not be viewed as few-shot learners. \n\n3. The authors claimed that “This diverse and challenging pretraining task likely promotes the learning of comparatively more detailed, generalizable slide-level representations.” Given that an individual dataset (e.g. NSCLC) has multiple tasks, have you investigated if the model can benefit more from multi-task pretraining, which is supposed to be a more challenging pretraining task?\n\n4. Given that the transferability is strongly related to the difficulty of pretrain tasks, I recommend authors to separate pretrain tasks into two groups, easy and difficult, and see if there are significant differences in the performance of the two groups.\n\n5. The authors clarified that highly parameterized architectures benefit more from pretraining. Different parameter sizes of the transformer should be investigated. I wonder if the reason why the transformer underperforms ABMIL is due to insufficient parameters.\n\n6. From Figure 4, we can see that performance continues to increase monotonically with the increase in parameters. In this case, to fully explore the scalability of model size, why not continue to increase the parameters until no further improvement is observed?", "relation_to_broader_scientific_literature": "Despite the significant research interest in the development of MIL architectures and the well-known advantages of transfer learning in general machine learning, there has been almost no investigation into the effectiveness of MIL models in transferring knowledge in CPath. This work fills this gap and provides valuable guidance for developing MIL approaches in the future.", "essential_references_not_discussed": "Regarding different patch encoders, the previous work mSTAR [1] had a similar experiment, where they claimed that the pretrained aggregator paired with the poor patch extractor benefits more from pretraining. I wonder if pretraining MIL still works when the patch extractor is strong enough, which can be validated in the patch encoder of Virchow 2G, a SOTA strong patch encoder.\n\n[1] A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model, arxiv, 2024.", "other_strengths_and_weaknesses": "## Strength\n1. Comprehensive experiments are designed for the investigation of various factors contributing to the transferability of MIL for CPath.\n2. This research fills the gap in the field of CPath and provides a valuable guidance for developing MIL approaches in the future.\n3. The paper is well-written and easy to follow.\n\n## Weakness\nplease see 'Claims And Evidence' and 'Experimental Designs Or Analyses'.", "other_comments_or_suggestions": "- There is a mistake in the order of BRACS-C and BRACS-F in Figure 2.\n- The availability of code and data PC-108/43 can contribute to the reproduction of the conclusion in this work.", "questions_for_authors": "please see 'Claims And Evidence' and 'Experimental Designs Or Analyses'.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pdW1R2ATRq", "reviewer": "Reviewer_qGZD", "summary": "The paper investigates the transfer learning capabilities of Multiple Instance Learning (MIL) models in computational pathology, evaluating 11 MIL models across 19 tasks. It finds that pretrained MIL models consistently outperform those initialized randomly, with pan-cancer pretraining tasks (such as PC-108 and PC-43) showing substantial performance gains across various downstream tasks. This pan-cancer pretraining enhances generalization across different organs and task types, often outperforming single-disease pretraining. Simple MIL architectures, like ABMIL, demonstrate high transfer performance, sometimes surpassing more complex transformer-based models. Larger models, particularly those based on transformers, benefit more from pretraining, showing significant performance improvements. The study also highlights the few-shot learning capabilities of pan-cancer pretrained models, which perform well even with limited data. Additionally, the benefits of pan-cancer pretraining are consistent across different patch encoders, indicating the robustness of the MIL framework. The findings suggest that supervised MIL models exhibit strong adaptability and transferability, with pan-cancer pretraining emerging as a highly effective strategy for enhancing performance in computational pathology.\n\n## update after rebuttal\nThanks for the authors, I am raising my score to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and for sharing your enthusiasm on the robust empirical evidence and strong performance of transferring MIL models. We have sought to address nearly all suggestions in additional experiments. Further details are provided below.\n\n**Q1. Pretraining strategies**\n***\nTo address the reviewer’s request, in addition to comparing against CHIEF, a SOTA slide foundation model (FM) trained on 60k WSIs (Table 2), we also: \n1) Compared against Gigapath, another SOTA slide FM trained on 171k WSIs. We show similar findings that ABMIL pretrained on PC-108 (3,499 WSIs) outperforms Gigapath (see BdAq Q1).\n\n2) Implemented CHIEF’s vision-language (VL) pretraining on PC-108 for fair comparison on pretraining strategy. Our results indicate conventional supervised pretraining outperforms VL pretraining on the same dataset (see BdAq Q1).\n\n\n**Q2. Limited originality**\n***\nWe respectfully disagree regarding the lack of substantial originality. Despite significant interest in MIL architectures for computational pathology (CPath) in ML/CV conferences, there is little to no investigation on MIL transfer in CPath. This is in stark contrast to progress made in general ML, with many high-impact papers using hypothesis-driven experimentation that contribute unique scientific insights (Line 077-089) without introducing new architectures [1,2]. Despite the importance of transfer learning in ML broadly, pretrained MIL model transfer remains unexplored in CPath due to limited understanding of its success conditions (Line 057-076, 102-109).\n\nAs the first work to investigate supervised MIL transferability in CPath, **we believe that this research question is not only substantially original, but also impactful by highlighting a new avenue for obtaining highly generalizable slide-level representations**. Despite progress made with self-supervised slide FMs, our alternative approach of supervised MIL model transfer has not been previously explored (Line 090-101). We show that our approach outperforms these methods while using less than 10% of the training data. Furthermore, we provide extensive insights into MIL transfer, such as how which features are transferred, how important each layer is to transfer, and how model size, patch encoders, MIL architecture, and task difficulty affect transfer.\n\n[1] Kornblith, Simon, et al. \"Do better imagenet models transfer better?\" CVPR (2019): 2661-2671 \n\n[2] Fang, Alex et al. \"Does progress on ImageNet transfer to real-world datasets? Neurips 36 (2024) \n\n[3] Ericsson et al. \"How well Do Self-Supervised Models Transfer?\" CVPR (2021): 5414-5423 \n\n**Q3. Visualization**\n***\nWe agree further interpretability is valuable. To this end, we also provided t-SNE visualizations showing pretrained slide embeddings differentiate classes better than random initialization (Figure 5). Since the scope of this work is on investigating MIL models, we preclude segmentation tasks, which focus on categorizing pixels rather than slides.\n\nMotivated by the reviewer’s suggestion, we generated additional attention-based heatmaps for ABMIL on BRACS and NSCLC subtyping, finding that pretrained models focus on diagnostically relevant tumor regions even before finetuning. This indicates that the aggregation layer is highly transferable between tasks, which we further validated through quantitative explainability experiments (see 5tA4 Q3-4). Updated visuals and discussions will be included in our final submission.\n\n**Q4. Novel architectures**\n***\nWe have conducted extensive experimentation on 11 MIL architectures, with one of them also being graph-based (WikG). Though novel architectures are not the focus of our study, we modified a few MIL architectures to further explore the impact of model size on transferability (see Rh9m Q1). Results showed increased performance with larger model sizes compared to training from scratch, encouraging exploration of new high-capacity MIL models despite limited dataset size.\n\n**Q5. Cross-domain transfer**\n***\nWhile investigating transfer from natural to pathology images is interesting, there is a scarcity of giga-pixel natural images to properly assess MIL model transfer. Instead, we had included transfer results using ResNet50 patch feature encoder (Table 3) trained on natural images, which showed significantly poorer performance than in-domain patch feature encoders.\n\n**Q6. Other comments**\n***\n*Novel Datasets*: As our work evaluates 11 MIL models across 21 pre training tasks, contributions such as novel benchmark datasets could not be fully explored, though we also note that our study is potentially one of the largest in ML/CV conferences. *Statistical tests*: We provide standard deviation via 100 trials of bootstrapping for our updated results (see Bda4 Q1, 5tA4 Q3, Rh9m Q1, Q3) and will provide confidence intervals in the main text. *Suggested reference*: We clarify that \"How well do self-supervised models transfer?\" was cited in our submission (Line 082-083).", "claims_and_evidence": "The claims in the submission are well-supported by empirical evidence. The authors demonstrate that MIL models pretrained on diverse pan-cancer tasks consistently outperform those initialized randomly across various downstream tasks, highlighting the benefits of pan-cancer pretraining for generalization and few-shot learning. Simple architectures like ABMIL show high transfer performance, while larger models, particularly transformers, benefit more from pretraining. The findings are robust across different patch encoders, indicating the MIL framework's inherent transferability. However, further investigation into comparisons with other pretraining strategies, the impact of dataset quality and diversity, and the generalizability to other domains could provide additional insights.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are highly suitable for the problem of assessing MIL model transferability in computational pathology. The comprehensive evaluation of multiple MIL models across diverse datasets and tasks effectively addresses the research questions and provides robust insights. The use of pan-cancer pretraining and standardized metrics ensures that the findings are generalizable and meaningful. While the methods are well-designed, further inclusion of self-supervised learning comparisons and clinical validation could enhance the study's comprehensiveness and practical relevance. Overall, the approach is well-aligned with the goals of the research.", "theoretical_claims": "The paper does not include any theoretical proofs, and the claims are based on extensive empirical evaluations. The empirical results are robust and provide strong support for the claims made. While theoretical analysis could further strengthen the findings, the current approach is well-aligned with the goals of the study and provides valuable insights into the transfer learning capabilities of MIL models in computational pathology.", "experimental_designs_or_analyses": "The experimental design and analysis in the paper are sound and well-structured. The authors comprehensively evaluate 11 MIL models across 19 diverse tasks and datasets, providing robust empirical evidence for the transfer learning capabilities of MIL models. The use of pan-cancer pretraining tasks (PC-43 and PC-108) is a novel approach that effectively demonstrates enhanced generalization across different organs and task types. The dual evaluation settings (end-to-end finetuning and KNN) and the inclusion of few-shot learning experiments further strengthen the findings.\n\n### Potential Issues\n\nComparison with Other Pretraining Strategies: The study focuses on supervised pretraining but lacks comparisons with self-supervised learning methods, which could provide a more comprehensive understanding.\nStatistical Significance: Detailed statistical significance tests for performance differences are missing, which could further validate the robustness of the results.\nOverall, the experimental design is robust, but addressing these potential issues could enhance the study's comprehensiveness and practical applicability.", "relation_to_broader_scientific_literature": "The key contributions of the paper are closely related to the broader scientific literature on transfer learning and pretraining strategies in computational pathology. Specifically:\n\n1.Transfer Learning in Computational Pathology: The paper investigates the transfer learning capabilities of MIL models, extending the well-established concept of transfer learning from other domains like natural language processing and computer vision to computational pathology.\n\n2.Pan-Cancer Pretraining: The use of pan-cancer pretraining tasks to enhance model performance aligns with the trend of leveraging large, diverse datasets for pretraining, similar to strategies used in models like BERT and GPT. This approach is shown to improve generalization and transferability in MIL models.\n\n3.Model Scalability: The finding that larger models, particularly transformers, benefit more from pretraining is consistent with the broader literature on model scalability. This highlights the importance of effective initializations for complex models.\n\n4.Few-Shot Learning: The paper's exploration of few-shot learning scenarios is relevant to addressing data scarcity in computational pathology, a common challenge in the field. The results demonstrate the potential of transfer learning to improve performance with limited data.\n\nOverall, the paper builds on existing knowledge in transfer learning and pretraining, providing specific insights into the application of these strategies in computational pathology using MIL models.", "essential_references_not_discussed": "In my opinion, \"How well do self-supervised models transfer?\" is worth citing.", "other_strengths_and_weaknesses": "### Strengths\n\n1. Comprehensive Evaluation: The paper thoroughly evaluates 11 MIL models across 19 diverse tasks, providing robust and generalizable findings.\n\n2. Pan-Cancer Pretraining: Introducing pan-cancer pretraining tasks (PC-43 and PC-108) is a novel approach that significantly enhances model generalization.\n\n3. Few-Shot Learning: The study explores few-shot learning scenarios, demonstrating the practical applicability of pan-cancer pretraining in data-scarce environments.\n\n4. Model Scalability: The analysis of model size and transfer performance offers valuable insights into the scalability of MIL models.\nPractical Implications: The findings have significant practical implications for improving performance in computational pathology with limited data.\n\n### Weakness:\n\n1. Limited originality: The main drawback of this paper is the lack of substantial originality. While the application of pan-cancer pretraining is novel, the overall approach largely builds on existing concepts in transfer learning and MIL.\n\n2. Single visualization scheme: the author mainly uses heat map to show the effect, but does not include the visualization results of specific tasks. An example is the effect comparison of segmentation tasks.", "other_comments_or_suggestions": "Innovation in Pretraining: Consider exploring more innovative pretraining strategies beyond pan-cancer datasets. For example, integrating multi-modal data (e.g., combining histopathology images with clinical data) could offer new insights.\n\nNovel Architectures: Investigate novel MIL architectures or modifications to existing ones that could enhance transferability. This could include exploring attention mechanisms or graph-based models.\n\nCross-Domain Transfer: Explore cross-domain transfer learning, such as transferring knowledge from natural images to histopathology images. This could provide a more comprehensive understanding of transfer learning in computational pathology.\n\nBenchmarking: Develop new benchmarks or datasets that better reflect real-world clinical scenarios. This could help in evaluating the robustness and generalizability of the proposed methods.", "questions_for_authors": "No more questions here.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9VMcMM2ttP", "reviewer": "Reviewer_BdAq", "summary": "This paper investigates transfer learning in MIL models for computational pathology. The authors test 11 MIL models across 19 pretraining tasks, showing that finetuning pretrained models significantly outperforms training from scratch, despite domain differences. Pan-cancer pretraining enables consistent generalization across organs and tasks, surpassing SOTA models. The findings highlight MIL models' adaptability and the advantages of pretraining in computational pathology.\n\n## update after rebuttal\nAfter careful consideration of all comments and the corresponding responses, I'll retain the score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable suggestions to further explore slide foundation models and clarify contributions. We provide our response below.\n\n**Q1. Retraining CHIEF**\n***\nWe train a new model (PC-CHIEF) on the PC dataset using CHIEF's training recipe, comprised of supervised contrastive loss and CLIP-based text embeddings of anatomical site.\n\nOur evaluation shows that PC-CHIEF achieves a mean performance of 69.2 across tasks, which is slightly lower than the original CHIEF's 69.8. This similar performance suggests that dataset quality is not the primary factor explaining the performance gap between the PC-108 model and CHIEF. Notably, PC-CHIEF underperforms our PC-108 model by a large margin, despite being trained on the same data samples with additional textual embeddings and contrastive loss. This result highlights our pre-training approach as a simple but effective means of developing highly transferrable MIL models. We show performance averaged across tasks within each dataset, with number of tasks indicated in parentheses. All results use CTransPath features.\n\n|Task|PC-108|CHIEF|PC-CHIEF|Base|Mean Pool|\n|-|-|-|-|-|-|\n|Avg (14)| 70.8|68.8|68.6|68.1|61.3|\n| BRACS (2)| 60.3 ± 4.5 | 58.3 ± 5.0 | 57.2 ± 5.0 | 54.4 ± 4.9 | 38.1 ± 4.3 |\n| BRCA (4)| 74.1 ± 4.0 | 71.4 ± 4.2 | 72.1 ± 3.9 | 71.5 ± 4.8 | 69.4 ± 4.7 |\n| NSCLC (4)| 71.8 ± 7.2 | 68.4 ± 6.4 | 67.9 ± 6.3 | 68.1 ± 6.9 | 62.5 ± 6.4 |\n| EBRAINS (2)| 68.7 ± 2.2 | 70.9 ± 2.2 | 70.1 ± 2.4 | 68.9 ± 2.1 | 57.2 ± 2.2 |\n| GBMLGG (2)| 74.6 ± 2.5 | 72.8 ± 2.6 | 72.8 ± 2.5 | 73.9 ± 2.5 | 69.6 ± 2.6 |\n\nTo further explore the efficacy of supervised pretraining, we extended our investigation to a different slide FM, Gigapath [1]. Unlike CHIEF, Gigapath was trained using a fully self-supervised approach, using a dataset of 171,189 WSIs. We compare finetuning performance of Gigapath with ABMIL pretrained on PC-108. All results use Gigapath patch features. \n\n|Dataset|PC-108|Gigapath|Base|Mean Pool|\n|-|-|-|-|-|\n|Average (14)|73.0|71.9|71.5|67.0|\n|BRACS (2)|58.7 ± 4.7|54.6 ± 4.3|59.3 ± 4.6|40.6 ± 4.7|\n|EBRAINS (2)|77.6 ± 2.1|79.3 ± 2.1|77.7 ± 2.0|80.9 ± 1.7|\n|GBMLGG (2)|70.9 ± 2.4|73.9 ± 2.4|70.0 ± 2.4|69.6 ± 2.6|\n|NSCLC (4)|76.4 ± 5.5|71.7 ± 6.4|70.3 ± 4.9|66.0 ± 6.2|\n|BRCA (4)|73.0 ± 4.3|73.0 ± 4.3|72.8 ± 4.4|70.7 ± 4.4|\n\nThe ABMIL pretrained model, with a pretraining set only 2% the size of Gigapath, demonstrated substantially higher performance compared to the Gigapath slide FM. These results underscore the effectiveness of our pretraining approach even when trained with a substantially less data. \n\n[1] Xu, Hanwen, et al. \"A whole-slide foundation model for digital pathology from real-world data.\" Nature 630.8015 (2024): 181-188.\n\n\n\n**Q2. Limited contributions to methodological advancements.**\n***\nWhile our work does not propose technical novelty in the form of a new MIL architecture, we respectfully disagree that our work makes limited contributions to methodological advancement. We note that in general ML, most high-impact papers on transfer learning are published using hypothesis-driven experimentation that emphasize unique scientific insights instead of methodology as a research contribution (Line 077-089). Not only is empirical investigation on transfer learning a valid and important research direction, but we also emphasize that research on MIL transfer is almost entirely absent in CPath. The vast majority of CPath studies in ML/CV conferences train MIL models from scratch, which may stem from lack of understanding on when and where these models would benefit from transfer. Furthermore, we provide an accessible, supervised alternative to slide foundation models trained on massive WSI cohorts (60-200k) via self-supervised learning, which demand substantial data and computing resources.\n\n\n**Q3. Limited Novel Insights**\n***\nThough the benefits of transfer learning are well-established in the broader ML community, the absence of existing works on MIL transfer signals a distinct gap in the literature. As the first work to investigate MIL transfer, we reveal pan-cancer pretraining (considered by qGZD as a novel technique) as a powerful and overlooked approach to improve performance across all tasks and MIL methods, allowing a pre-trained ABML model to outperform SOTA slide foundation models (CHIEF and GigaPath) while requiring only 2-7% of the training samples. In addition, we also provide practical insights specific to MIL transfer, such as the importance of different layers for transfer (see response to 5tA4 Q2-3), the effect of model size on transfer (Rh9m Q1), the effect of task difficulty (Rh9m Q4), and the transferability of models across different organs (Figure A1). \n\n**Q4. Textual edits**\n***\nWe agree that inclusion of patch foundation models will provide a comprehensive overview of related works. We will include mention of models such as Virchow in the revision. We will adjust the typesetting and reference issues in the revision.", "claims_and_evidence": "The paper’s claims are well-supported by clear and convincing evidence. The authors thoroughly evaluate 11 MIL architectures across 19 tasks. The results are presented in figures and tables demonstrate significant performance gains from pretraining.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are appropriate for the research question.", "theoretical_claims": "No theoretical claims involved.", "experimental_designs_or_analyses": "The experimental designs and analyses are sound.", "relation_to_broader_scientific_literature": "It highlights the contrast between the extensive research on MIL architecture development and the lack of investigation into MIL model transfer in computational pathology. The authors also discuss the relationship between their work and the development of slide foundation models, positioning supervised MIL transfer as a simple and effective alternative.", "essential_references_not_discussed": "The paper provides a comprehensive overview of related work in MIL and slide foundation models.  However, the inclusion of foundational models, such as Virchow, could be beneficial.", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well-written and clearly structured.\n- The experimental methodology is rigorous and comprehensive.\n- The authors provide a thorough analysis of the results and relate them to the broader literature.\n\nWeaknesses:\n- The paper primarily emphasizes empirical findings, with limited contributions to technical or methodological advancements.\n- The paper's extensive experimental results corroborate the well-established principle that initialization with pre-trained parameters outperforms random initialization.", "other_comments_or_suggestions": "The manuscript contains several typographical errors, including misaligned citations in the literature review on page 2, lines 077-089, and inaccuracies in the publication years of classic Transformer references.", "questions_for_authors": "Surprisingly, the paper's methodology outperformed the SOTA CHIEF. The authors attribute this, in part, to their meticulously curated pretraining dataset. It would be intriguing to observe the results of training CHIEF using the authors' dataset.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "899HTLrZ5l", "reviewer": "Reviewer_5tA4", "summary": "The paper presents the first comprehensive investigation into the transfer learning capabilities of MIL models in computational pathology. It evaluates 11 different MIL architectures pretrained on diverse pan‐cancer tasks (i.e., PC-108 and PC-43) across 19 downstream tasks, including cancer subtyping, grading, and biomarker prediction. The authors demonstrate that models initialized with supervised pan-cancer pre-training substantially outperform those with random initialization—even when there is a domain gap—and can even exceed the performance of state-of-the-art slide-level foundation models (e.g., CHIEF), all while using less pre-training data. The paper also explores the effect of model scale and few-shot learning performance, showing that simpler architectures (such as ABMIL) can be very competitive.\n\n## update after rebuttal\n\nAfter reviewing the authors' rebuttals, I appreciate their effort to compare with additional baselines and their insightful discussion on the beneficial transfer of features. Based on these improvements, I have raised my score to 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed feedback on areas to improve rigor and insights from our results.\n\n**Q1. Benchmarks and statistical measures**\n***\nWe have added benchmarks with finetuned GigaPath (Q1 of BdAq), as well as using CHIEF pre-training on the PC dataset (Q1 of BdAq). We show std due to character limits and will show confidence intervals in the final paper. Please see Q3 of Rh9m for std of few-shot experiments. \n \n**Q2. Baseline comparisons**\n***\nWe now include 1) mean pooling for all slide-level encoding experiments (GigaPath and CHIEF in BdAq Q1) and 2) selectively transferring early layers (motivated by [1]) Below, we report how different transfer methods affect finetuning performance on a pre-trained ABMIL model (three-layer MLP with gated attention aggregation). Following TransFusion's approach, we investigate finetuning performance as progressively fewer layers are transferred. Starting with the attention module, we re-initialize the attention layer to random weights (Reset Attn), the third linear layer + attention (Reset Lin3+), the second and third linear layer + attention (Reset Lin2+), and all weights (Reset All).\n\nWe report performance relative to full weight transfer (PC108-L). Removing the attention layer causes the largest decrease (-5.0 average). Removing the remaining MLP layers results in a smaller but still substantial decrease (additional -3.3). These results highlight that the pretrained aggregation layer is crucial for successful supervised MIL pretraining. This contrasts with [1], which found that transfer of deeper layers has minimal effect on performance, emphasizing the unique nature of MIL transfer.\n\n|Task|PC108-L|Reset Attn|Reset Lin3+|Reset Lin2+|Reset All|Mean Pool|\n|-|-|-|-|-|-|-|\n| **Avg** |73.1| -5.0 | -5.2 | -6.6 | -8.3 | -12.5 |\n| BRACS-C |71.9| -8.5 | -8.5 | -12.8 | -11.0 | -16.2 |\n| BRACS-F |53.3| -12.1 | -10.5 | -10.4 | -10.1 | -23.1 |\n| GBM C |95.4| -1.2 | -1.0 | 0.0 | -0.2 | -3.3 |\n| GBM F |51.7| 0.0 | -0.8 | -1.8 | 0.8 | -2.4 |\n| Lung EGFR |76.1| -8.6 | -10.2 | -13.1 | -12.8 | -15.3 |\n| Lung KRAS |68.4| -2.0 | -4.5 | -5.8 | -8.7 | -8.1 |\n| Lung STK11 |86.7| 0.0 | -0.6 | -8.3 | -15.0 | -20.8 |\n| Lung TP53 |81.5| -7.3 | -5.9 | -0.5 | -10.0 | -10.8 |\n\n[1] Raghu, Maithra, et al. \"Transfusion: Understanding transfer learning for medical imaging.\" Advances in neural information processing systems 32 (2019).\n\n**Q3. What features are most beneficial for transfer?**\n***\nThe experiment in Q2 suggests that the attention-based layer is the most beneficial for MIL transfer. To gain further insights, we next investigate the extent to which transferred layers change after finetuning. We follow the (SV)CCA approach used in [1], which measures the linear relationship between combinations of neuron activations across different models (0-100 scale). We compare activations before and after finetuning for each layer in standard (S) and large (L) ABMIL models on 30 randomly selected slides (45,232 patches) from the NSCLC-KRAS dataset. \n\nResults below show pretrained layers change substantially less over the course of training than randomly-initialized ones. Most notably, the third and final attention layer (abmil c), exhibits extremely low correlation (2.5 & 16.3 for ABMIL-L and -S) with its original layer activations after training with random initialization, while pretrained models maintain high similarity (88.7 & 96.6). Since abmil c is the critical attention layer responsible for converting each patch embedding into a scalar weight for slide-level aggregation, this finding suggests that MIL transfer benefits heavily from transferring learned aggregation strategies.\n\n|Layer|Base-L|PC108-L|Base-S|PC108-S|\n|-|-|-|-|-|\n|Lin 1|92.8 ± 18.3|95.5 ± 14.7|93.9 ± 16.0|97.2 ± 12.0|\n|Lin 2|81.2 ± 22.6|89.2 ± 17.9|||\n|Lin 3|47.2 ± 25.0|66.5 ± 26.3|||\n|abmil a|85.2 ± 24.7|88.6 ± 20.9|95.6 ± 13.6|92.9 ± 17.3|\n|abmil b|84.8 ± 25.1|88.7 ± 20.4|94.7 ± 14.7|96.6 ± 11.7|\n|abmil c|2.5 ± 0.0|82.7 ± 0.0|16.3 ± 0.0|97.7 ± 0.0|\n|Slide feat|37.7 ± 21.7|70.2 ± 32.9|77.8 ± 18.8|96.4 ± 4.1|\n|Average|61.6|83.1|75.7|96.2|\n\n**Q4. What do you hypothesize is being captured?**\n***\nBased on our results indicating the transferability of the attention layer, we hypothesize that pan-cancer pre-training provides a better starting point for the MIL model, by focusing on cancerous regions and disregarding regions of low diagnostic importance. The characteristics of tumor regions, such as pleomorphic nuclei, increased cellular density, and entropic cellular arrangement are common motifs across various tasks. Meanwhile, regions such as smooth muscle, processing artifacts, and red blood cells are consistently of minimal diagnostic relevance. Whereas training an MIL model from scratch leads to challenges learning to disregard these background patches, our results suggest that models trained on a large pan-cancer classification task are already equipped to prioritize tumor morphologies prior to finetuning.", "claims_and_evidence": "The paper makes several key claims:\n\n(1) pre-training improves performance: Every evaluated MIL architecture shows a notable boost when initialized from a pan-cancer pretrained model compared to random initialization.\n\n(2) Pan-cancer pre-training is robust: Models pretrained on pan-cancer datasets generalize well across different organs and task types.\n\n(3) Model scaling matters: Larger models benefit more from effective pre-training and exhibit favorable scaling properties.\n\n(4)Few-shot learning capability: Pretrained models perform exceptionally well in few-shot scenarios, indicating strong data efficiency.\n\nThese claims are backed by extensive experiments including comparisons via fine-tuning and KNN evaluations (e.g., in Table 1, Table 2, Figures 1–5), and through ablation studies that demonstrate the robustness.", "methods_and_evaluation_criteria": "The authors adopt a rigorous experimental framework:\n\n(1) pre-training and Transfer Protocol: MIL models are first trained on pan-cancer classification tasks and then evaluated on 19 downstream tasks using both end-to-end finetuning and frozen feature extraction (KNN).\n\n(2) Evaluation Metrics: They employ AUROC for binary classification, weighted kappa for grading, and balanced accuracy for multiclass tasks.\n\n(3) Diverse Datasets: The evaluation spans multiple orgons and task types, ensuring that the conclusions are not dataset-specific.\n\nOverall, the methodology is appropriate and well-suited to address the problem of transfer in data-scarce clinical environments. However, adding statistical measures such as confidence intervals could improve the robustness of the evaluation.", "theoretical_claims": "The paper is primarily empirical. There are no theoretical proofs provided.", "experimental_designs_or_analyses": "The experimental design includes:\n\n(1) Comprehensive Evaluations: Multiple MIL architectures are compared under the same conditions across a broad set of tasks.\n\n(2) Ablation Studies: The paper investigates the effects of different pre-training tasks, model scales, and even different patch encoders.\n\n(3) Few-shot Experiments: Repeated experiments with cross-validation ensure that the few-shot learning results are statistically meaningful.\n\nOne area for improvement is the inclusion of confidence intervals in key figures (e.g., Figure 3) and tables (e.g., Table 2) to assess the variability of the results.", "relation_to_broader_scientific_literature": "The paper is well-positioned within the existing literature on computational pathology and transfer learning. It draws on established MIL methods and compares its findings with those from slide-level foundation models and related transfer learning.", "essential_references_not_discussed": "One notable omission is the discussion of alternative supervised transfer baselines. For instance, incorporating insights from Raghu et al.’s “Transfusion: Understanding transfer learning for medical imaging” (NeurIPS 2019) could provide a more nuanced baseline than random initialization alone. Another baseline is simple mean-pooling to instance features. Addressing this could offer further insight into how supervised pre-training transfers to downstream tasks.", "other_strengths_and_weaknesses": "Strengths:\n\n(1) Practical Relevance: Demonstrating the effectiveness of pre-training with limited data is highly valuable for CPath.\n\nWeaknesses:\n\nInsight: The paper offers limited insight into why and how supervised pre-training leads to improved downstream performance.\n\nStatistical Reporting: Key experimental results (e.g., Table 2, Figure 3) would benefit from the inclusion of confidence intervals.\n\nBaseline Comparisons: Including comparisons with additional methods (e.g., “Transfusion”) could further contextualize the contribution.", "other_comments_or_suggestions": "(1) Table 2 Improvements: It would be valuable to include performance results for Prov-GigaPath along with confidence intervals to better capture performance variability.\n(2) Figure 3 Enhancements: Adding confidence intervals in the few-shot performance plots would improve the statistical rigor of the presented results.", "questions_for_authors": "Can you provide more insight into which features or representations are most beneficial for MIL transfer? What do you hypothesize is being captured by the pan-cancer pre-training in MIL?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper examines whether fine tuning MIL digital pathology models using transfer learning helps in other downstream tasks and provides an affirmative answer. Although the finding is not surprising, the breadth of the experiments and the number of methods tested are impressive. All reviewers recommend acceptance and the AC concurs
Decision: Accept (spotlight poster)