Bayes-MIL: A New Probabilistic Perspective on Attention-based Multiple Instance Learning for Whole Slide Images
Authors: Yufei CUI
Ziquan Liu
Xiangyu Liu
Xue Liu
Cong Wang
Tei-Wei Kuo
Chun Jason Xue
Antoni B. Chan
Venue: ICLR 2023 poster (2023)
Keywords: Multiple instance learning
medical imaging
histopathology
Bayesian neural network

Abstract: Multiple instance learning (MIL) is a popular weakly-supervised learning model on the whole slide image (WSI) for AI-assisted pathology diagnosis. The recent advance in attention-based MIL allows the model to find its region-of-interest (ROI) for interpretation by learning the attention weights for image patches of WSI slides. However, we empirically find that the interpretability of some related methods is either untrustworthy as the principle of MIL is violated or unsatisfactory as the high-attention regions are not consistent with experts' annotations. In this paper, we propose Bayes-MIL to address the problem from a probabilistic perspective. The induced patch-level uncertainty is proposed as a new measure of MIL interpretability, which outperforms previous methods in matching doctors annotations. We design a slide-dependent patch regularizer (SDPR) for the attention, imposing constraints derived from the MIL assumption, on the attention distribution. SDPR explicitly constrains the model to generate correct attention values. The spatial information is further encoded by an approximate convolutional conditional random field (CRF), for better interpretability. Experimental results show Bayes-MIL outperforms the related methods in patch-level and slide-level metrics and provides much better interpretable ROI on several large-scale WSI datasets. 

Published as a conference paper at ICLR 2023
BAYES-MIL: A NEW PROBABILISTIC PERSPECTIVE ON ATTENTION-BASER MULTIPLE INSTANCE LEARN- ING FOR WHOLE SLIDE IMAGES
Yufei Cuit, Ziquan Liu?, Xiangyu Liu
Xue Liu', Cong Wang?. Tei- Wei Kuo Chun Jason Xue?, Antoni B. Chan? MeGill University CCity University of Hong Kong BBigll Research
Naatioaal Taiwan University Mohamed bin Zayed University of Artificial Intelligence
ABSTRACT
Multiple instance learning (MIL) is a popular weakly-supervised learning model the whole slide image (WSI) for Al-assisted pathology diagnosis. The recent advance in attention-bast (ROI) for interpretation by learning the attention weights for image patches of WSI slides. However, we empirically find that the interpretability of some related methods is either untrustworthy the principle of MIL. is violated unsatisfac- tory as the high-attention regions are not consistent with experts' annotations. In this paper, we propose Bayes-MIL address the problem from a probabilistic perspective. The induced patch-level uncertainty is proposed as a new measure of MIL intepppttttt which outperforms previous methods in matching doctors an- notations. We design slide-dependent patch regularizer (SDPR) for the attention, imposing constraints derived from the MII, assumption, on the attention distribu- tion. SDPR explicitly constrains the model to generate correct attention values. The spatial information is further encoded by an approximate convolutional con- ditional random field (CRF), for better interpretability. Experimental results show Bayes-MIL outperforms the related methods in patch-level and slide-level metrics and provides much better interpretable ROI several large-scale WSI datasets.
MIL allows the model to find its region-of-interes
INTRODUCTION
In real-world applications of deep learning, data like images or texts are often associated with in- sufficient labels, due to the expensive annotation cost. For example, the whole slide images (WSI) for medical diagnosis have about 10 x 10 pixels per image, but are tagged with single categorical labels (Zhang al. 2019 Campanella et al., 2019p. Weakly-supervised learning methods are de- signed for learning representations and making decision in these cases. Multiple instance learning (MIL) a popular weakly-supervised learning model for the application of WSI recognition (Ise ct al. 2018 Lu et al. 10.1). Concretely, a large WSI slide is sliced into bag of image patches MIL builds an end-to-end parametric model that aggregates the leamnd features from instances and only learns from bag-level labels. The rule of aggregation is implementing the key principle of MIL: for binary classification, a bag is negative when all instances negative, and bag is positive when there is one more positive instance (Ilse et al. 2018).
(instances) with a moderate size I
Recent advances study the attention-
MIL for weiiihhi the instances for better performance.
This attention mechanism for MIL is extensively explored and used measure of interpretability
in various downstream tasks for medical diagnosis, like prostatic cancer ZZang et al. 2021) breast cancer Naik et al. 2020p, etc. Specifically, the high attention weights are used to indicate that its associated instances are positive instances, e.g, the cancerous image patches. However, this rule not formally justified and it not clear whether the negative instances (i.c., benign) would be assigned high attention value or the other way around. We first analyze the convergence of attention and provide validity of this rule under binary labels. Based on this rule, we conduct an empirical study on large scale WSI dataset for how the attention mechanism in the related MIL methods performs. The study shows two clear flaws of the related methods:
We define the following interchangeable terms for simplicity: "bag" and "slide"; "instance" and "patch".

---

Published as a conference paper at ICLR 2023
Image
Pre-pnssiing
Une. Vanslicatien
Site-epestod Park Regabebbr
Condelkedd Cit
Modic
Frebahilttt Medtii
CN for Aani Mamon
Neadii MkjE
Car Kerval
Parch Precision
0.71400
Patch Procision 0,745R
Patch Precision
0,8107
FROC
0.4706
Patch-FROC
0:4856
Patch-FROC
0.4918
Figure The overview of Bayes-MIL framework and zoom-in views for clear visualization of inter- pretability. (I) basic Bayes-MIL improves patch-level localization performance (Sec. 3.T). (2) The slide- dependent patch regularizer makes attention densely concentrated on the positive area, improving its inter- pretability (Sec. 3.2 (3) The convolutional CRF improves the localization by smoothing the uncertainty over different patches (Sec. 3.3). (bottom) The ablation results few metrics show the improvement of inter-
pretability, The full ablation results are Sec.
The interpretability for negative bags is untrustworthy because methods violate the key principle of MIL by placing high attention values on negative bags, thus indicating positive instances.
The interpretability for positive bags is unsatisfactory because the high attention values could not well match experts' annotations of positive instances.
In this paper, we address the problems from probabilistic perspective. First, a basic framework of Bayesian MIL (Bayes-MIL) is proposed, for inducing uncertainty over the attention weights. The uncertainty is potentially an accurate measure for guessing whether the instances are positive or negative,
replacement of attention. Second, a regularizer is designed by deduction from the MIL principle and implemented via the variational inference framework, which sets specific constraints for the attention distributions of positive and negative bags. Third, to encode the spatial information of instances for medical imaging application, we propose approximate operation to the convolutional conditional random field, which benefits the localization of the region of interest (ROI). The final classifier is modeled in Bayesian way, in order to provide calibrated uncertainty of bag-level prediction. The overview of our proposed method is shown Fig. The contributions of this paper are listed as follows:
We analyze the attention-base MIL on the interpretability-critic medical application and point out the flaws by directly using attention for interpretation.
To address these problems, we propose the first Bayesian MIL for WSI with 3 key com- ponents: probabilistic instance-wiss attention module for uncertainty visualization, the slide-dependent patch regularizer for learning the correct attention distribution, and an ap- proximate convolutional conditional random field for encoding spatial information. Our model provides well-calibrated uncertainties, which is crucial for safety in medical appli- cations.
The evaluation on large-scale MIL datasets shows Bayes-MIL outperforms the related methods in instance-level interpretation and bag-level prediction under various evaluation metrics. The visualized distribution of data uncertainty shows a strong correlation of the designed regularizer, which validates the soundness regularizer and explains why uncer- tainty is useful in MIL interpretation.
FORMULATION AND ANALYSIS OF MULTIPLE INSTANCE LEARNING
Multiple instance learning formulation We follow the standard formulation of Attention-based Multiple Instance Learning (MIL) (Ise et al. 2018 Lu et al. 202T). In MIL, the input is a bag of instances, X = [.1, ... Fx Dg E RDD. K is the number of instances, which varies for different bags. There is level label Y. We further assume the instances also have corresponding instance-level labels .yI..... ... MK pairs constituting the dataset D (Xn, Y,, JN The objective of MIL is to learn optimal function for predicting the level label with the bag of instances as input. To this end, the MIL model should be able to aggregate the information of instances {æx 1 to make the final decision. well-adooted aggregation method is the embedding approach which maps X to a beg-level representation DRD and use z predict Y. Ilse al. (0018) extends the embedding-
which are unknown during training. There are such bag-label
2

---

Published as a conference paper at ICLR 2023
aggregation approach by leveraging the attention mechanism, namely attention- deep MIL (ABMIL). First, a transformation g()) computes a low-dimensional embedding h& = ( ) R
for each instance zg. The attention module aggregates the set of embeddings { k== into a bag
level embedding 2, == = = aghe, where the attention for the k-th instance is computed via a softmax function,
exp(mm (Va (V," hu Geigmoidd V bgll] ON fx _n-1 exp(m (atammmm(ttth))t (tanh((V (V." h2 Gnigmmidd V h,)))
(I)
where the attention function fn IR + RK with parameters क = {m, V1, V2J. and fx (H), denotes the k-th output of f. H [h,,..., hg] E RDxK is the matrix of embeddings. The bag embedding z is then mapped to the logits with a feed forward layer with parameter W for the bag-level classification, u wTz. z.
Multiple instance learning for medical imaging WSI a type of high-dimensional image data format (up to 10 x 10s pixels per image) widely adopted in the medical area. Due to the scarcity of experts and the high annotation cost, only class labels (e.g., diagnosis results) are available for most WSIs. The high resolution of data and lack of precise annotations raise challenges for machine learning-assee assisted classification. To fit the data in modern computation hardware, the high-resolution image slides (slide) are partitioned into image patches (patch) before further processing. MIL fits the classification task of WSI by corresponding the slides to bags, and patches to instances. The transformation g(-) is the feature extractor from pre-trained convolutional neural network. The MIL model only predicts the slide-level label, e.g., whether a slide is cancerous or not. However, the interpretation of the patches is crucial, e.g., which patches indicate the cancer, as users always check the interpretation before trusting the prediction. ABMIL uses the attention weights to tell which patches the MIL model focuses on, and several works DSMIL (Li et al., 202T) CLAM (Lu 202T. TransMIL. (Shao et al. (021) study the variants of attention for better interpretability.
et al.
Ilse al. (0018 suggested that, with binary classification label Y E {0,1); the high attention weights in ABMIL could locate the positive area (y), 1) in an ideal case. The followup works from ABMIL use the high attention weights to indicate the positive patches in a black-box manner. However, there is no formal justification for this claim, and is still unclear whether high attention weight could also be assigned to a negative area during training. Therefore, we analyze the general attention-base based MIL framework to provide this justification.
The convergence of attention Assume that the jth input patches in ith slide is hij. the classifier weight is E IR the attention variable is a E (0, 1)xx1, The output of the network 9s and loss function C; for the i-th slide are
U = ow Haa+ + +), -Y= log (1:) - (1 - Y,) log(1 - Y.).
(2)
The H; all patches in ith slide, where the la norm of all hi upper bounded by 1, and o(r) = 1/(1 + exp(-xr) is the sigmoid function. Assume the positive and negative patches are linear separable and there is optimal w", where ||w0|| If slide H; is negative, then all patches in H, are negative, i.e., w haj 0, Vj if Y. 0. If a slide H, is positive, then the first Kp patches are positive and the last Kn patches are negative, ie.. w= "he h.,,,l > 0 and "n optimal a' so that the first Kp dimension (1 e)/K, and the last dimension is e, where e is an infinitesimal, e.g., le-5. For the initialization of a, we assume wx H.a > <>>>>>> vi. The a is output of softmax function,
1 and the margin is 7 min.,j /w" Th
[h, T K,,,
Ku,K,KK. < 0 if Y, 1. There is an
s(u), where aj ΣK_ exp(ux)"
exp(u,)
Lemma 1. If we train the w and a described in Appendix 1, the converges to the optimal w in at most 4/ max(", 2)² steps and a converges to the desired a", where the first Kp elements are large and the last Kn, elements are small.
Lemma I indicates that MIL is guaranteed to converge to the desired attention variable under the ideal condition. Note that in reality, the position of positive patches is not fixed so we use pa- rameterized function to make the attention variable a, depend provides a guarantee for the validity of visualization and the design of our framework. However, in reality, the convergence of weight and attention variables depends their initialization. Thus, isting methods may not have a good match between high-attention patches and ground-truth positive patches. The following empirical study is performed to illustrate this.
the patch feature ha.j- Lemma
3

---

Published as a conference paper at ICLR 2023
Original Image
DSMIL
CLAM
TransMIL
BMIL-APCRF
Normal Wst
10
B
D
Tumor WSI
04
82
G
H
Figure 2: The visualization of normal and tumor slides and ROIs provided by different models. The patch- level annotations for the tumor image are shown in green color in (F). The attention values a are normalized to the same range by e-ming- The mina and maxa are the same for all methods for better visualization.
-ming
Empirical study of interpretability We use the attention {ax as the patch-level predic- tion confidence and compare with the patch-level ground-truth {ym))mm using the related MIL approaches. One key discovery from the related methods is that high attention values are still generated. indicating positive patches. for negative slides, as shown in Fig. B Another dis- covery is that the high attention values are not concentrated on the doctor's annotation well during inference time, as shown in Fig. P (see numerical results in Tab.
The need for Bayesian modelling of MII. The empirical study constitutes one motivation for our probabilistic approach: bring more stochasticity to the optimization process so that the convergence of weight and attention variables does not heavily depend on the initialization. Thus, we study the probabilistic counterpart of MIL, namely Bayes-MIL, which is further potentially beneficial in the following three aspects:
Besides stochasticity in optimization, by turning a to stochastic nodes, explicit regularization could be imposed for generating correct attention (see Fig. 2 New measure of interpretability: By learning proper posterior over the param- eters p( [D), we can induce the uncertainty over patches by p( Ha, J p(aaT, H [D)dr, where H' is the testing data. Patch-level uncertainty can po- tentially indicate which patches the model is uncertain about, becoming new measure of interpretability for MIL. In other words, the patch-level uncertainty is leveraged for local- izing positive areas.
Better optimization:
Calibrated uncertainty: properly learned posterior DD) provides calibrated uncertainty on Y, which is crucial in the application of medical imaging but ignored in existing methods.
3 BAYESIAN MULTIPLE INSTANCE LEARNING
3.1 INSTANCE-LEVER DISENTANGLED UNCERTAINTY
The basic framework of Bayes-MIL is introduced in this section. To obtain uncertainty over the at- tention, the principled way is to assume a prior distribution p(TT) the parameters of attention fune- tion f (-), and let the model learn a posterior p( D). The other way is to directly learn an empirical posterior distribution (Dr e.g., ensembles. The posterior induces the uncertainty over the atten- tion by p(a)H H', D) = p(aam |r, H (p(TT) (D)dT A - fr, (H*) n. ~ p(T[D), where the at- tention function directly models the conditional distribution. i.e., p(a H) = (H) f(T,, H)).
For making p(a|m. π. H) a strict distribution, the softmax function 1 could be leveraged for nor- malization, Σ. ag tribution. However, in this case, we could only calculate the uncertainty for the single categor- ical distribution. To extract the patch-level uncertainty, there should be one probabilistic distri- bution for the attention of each patch. Therefore, we need to model the distribution for patches, p(az I,, H) fo H); & f(T, H)x and normalize for each patch. To this end, we replace the softmax function in (T) by an element-wise sigmoid function,
Then, is vector over the simplex. representing one categorical dis-
04 I+exp(-m= (tanb V ( Josigmoid
2 Σx Σ, aghi
(3)
where the parameters π = (m, Vi. V2) are shared across all patches. The second normalization when computing is for numerical stability.
4

---

Published as a conference paper at ICLR 2023
0.94
0.30
0.86
0.00
0.24
0.66
0.18
(a) Accuracy
(h) AUROC (t)
(c) NLL (1)
(d) ECE
Figure 3: The comparisons of multiplicative Gaussian, concrete dropout and BatchEnsemble for modelling
probabilistic MIL weights. The slide-level results reported. NLL is negative log likelihood and ECE is the
expected calibration error measuring uncertainty calibration.
With this treatment, the patch-level model uncertainty and data uncertainty for each patch og could be extracted by (Houlsby et al. 2011 Gal et al. 2017b):
(ax, H D] H ppppp ,,, H']]] ( H'Il
Model Uncertainty
Total Uncertainty
Data Uncertainty
We explore three different methods for modeling the probabilistic weights т:: Batch-Ensemble (Wen
al. 2020 Dusenberry et al. 2020) as the empirical posterior. Concrete Dropout (Gal et al. 201 7a)
and multplicative Gaussian noise (Kingma et al., 2015 Molchanov et al. 2017 Cui et al. 2021
that assume prior on π and derive the posterior based on variational inference. As there is no direct indication of which set of posterior and prior, we should choose, we validate them empirically with proposed framework. The experiments are conducted on CAMELYON16 dataset with 5 splits of training and validation sets. For the Batch-Enseembl, we an ensemble of size 4. For Concrete Dropout and multiplicative Gaussian noise, we take 4 samples from the learned posterior during inference time. The multiplicatiie Gaussian method is selected for the probabilistic weights due to a high accuracy and low NLL, shown in Fig.
For neural networks, computing the posterior distribution using the Bayes rule requires computing intractable integrals over т. this paper, to be consistent with the stochastic attention function modelling, use variational inference for approximating the posterior. Specifically, the pos- terior p(T is approximated by (T)),, by minimizing the Kullback-Leibler (KI.) divergence KT Tr) [|p((m (D)], where b the variational parameters. This is equivalent to maximizing the evidence lower bound:
max Tas ID(Ø) (T (D())=
-
N
n)),
(5)
where (D(()) is the expected data log-likelihood and p(T) is the prior over p(T)).
3.2 SLIDE-DEPENDENT PATCH REGULARIZER
Although the proposed model can naturally visualize different types of uncertainty at the patch level and better localize the ROI, we can further leverage the slide-level information for building strong regularization on the ROI localization. The aim is to explicitly encode the underlying logic of MIL into the training process, instead of letting the model explore implicit decision rules during training.
Recall Y is the slide-level label and { are the labels for patches which are unknown during training. The intuition is based on the logic under the MIL framework that slide is negative when all patches are negative, while a slide is positive when there are one or more positive patches. For binary label, Y = 0 iff Σx U/
and Y = otherwise. The following design principle can be drawn by simple deduction: When Y 0, the attention distributions {p(as T,, > must concentrate the negative side (as 0) to guarantee yı = O. When Y = 1, the attention distributions are free to select either the positive (aæ or negative sides. However, for precise localization, the attention distributions must concentrate on either the positive or negative side with high confidence.
This design principle is implemented by a variational inference framework by finding a regularizer on patches that is dependent on the slide label. Specifically, we choose the logit-normal distribution for ag. due its expressiveness over the simplex. The regularizer (a non-strict prior) defined as
p(ay|y Y) = (1 Y)
+ + (L
(6)
Ingis(ay)=s 26
where CN(allu, .v22 )
is the logit-normal distribution. (1o, and {F) o } are the pairs of mean and variance for the negative slides and the positive slides, respec- tively. As shown in Fig. 4 by setting the parameters for the regularizer, implement the design
5

---

Published as a conference paper at ICLR 2023
principle: p(ax|me σe) concentrates the negative side and p(ak|u1,0 allows selecting either the negative or the positive side with high confidence.
For the MIL model, we divert {ax from induced distributions stochastic nodes. The ap- proximate posterior is defined as g(ax pu, or) = [N = fu(TT, g. = fT H) and regularization term during training, which denote as the slide-dependent patch regularizer (SDPR). When a negative slide is given, the mode of the atten- tion posterior is pushed the negative side, generating low attention values over all patches. When a positive slide is given, the posterior will be trained to select a positive mode or a negative mode, with only high concentration. This produces dense localization of the ROI. The derivation of KL divergence for the regularization is in the Appendix.
Ex KL[g(ax/ (Ip((a Y)] is used
The SDPR is also beneficial for im- proving the slide-level performance of MIL. The reason it explicitly constrains the model to generate low attention values for patches from the negative slides. Note that the major obscurity in MIL comes from the positive slides, where positive patches and negative patches coexist. There is no obscurity for negative slides all patches negative by definition. However, the previous MIL models are still free to generate high attention for patches from negative slides, which neglect the underlying logic of MIL.
Proaaiity
02
0.0
0.0
0.50
0.03
Figure 4: The visualization of density (curves) of the regularizer for the (left) negative and (right) positive slides. Samples bars are visualized with color based on the attention value.
3.3 ENCODING SPATIAL INFORMATION VIA APPROXIMATION TO CONVOLUTIONAL CRF
The spatial information between patches are important for mod- elling of MIL. for WSI recognition and localization, as there exist patches are spatially correlated (Shao al. 202Tp. Here, we study how to encode the spatial information from a Bayesian perspective. The intuition is to let the neighboring attention posteriors ((ax) in- flurence each other.
SWee Deperrree Natsh Nepu kerus
One principled method for encoding the spatial information is to conditional random field (CRF). Assume a is the output attention variable of CRF and @ is the input variable in CRF, from data. With CRF, the distribution of p( a m)) could be directly modelled with m the spatial features of patches (Zheng et al. 2013 Teichmann & Cipolla 2018).
Figure The graphical repre- sentation Bayes-MII. Boxes are deterministic nodes. Circles are stochastic nodes. Dashed cir- cles nodes of induced distri- butions.
p(amm)) = Z(m)
-(ālm))
E(alm) E, wh (a + Enej (āp a, m))
where m (w. v7. a) contains the coordinates of the each patch over the slide, w [w]..... wrl" T and 7 [ ",k|"" th is the unary potential that contains information only from the single patch and dp is the wise potential that captures the pair-wise correlation between patches. How- ever, calculating the pair-wise correlation between patches has ((K2) complexity. For efficiency, we only consider the local dependency around dg., by setting the non-local pair-wise correlation to be This is equivalent to applying a softmax normalization and a convolution operation for the provided input a. We define function for the proposed convolutional CRF, a C.. (a). Specifically, à = softmax (a), à = reshape(u w, h, a), = convolution(O. KC), where KC a con- volutional kernel with predefined hyperparameters. The detailed derivations and algorithms for the full convolutional CRF for BayesMIL are in the Appendix. For a precise estimation of the variable a the following Monte-Carlo estimator is required,
E B[a- Eg ( R J E._, (a a, ((a)f Ho
(8) The full convolutional CRF has the most promising results on real-world datasets. However, the repetitive sampling makes the training less efficient. To bypass the heavy-load of sampling,

---

Published as a conference paper at ICLR 2023
first-order Taylor expansion to approximate the expectation (see Appendix). The final format of approximation could written as
q(@x j = C A (14), LNN( = C H)),, = ())
(9)
for imposing slide-level patch regularizer, as well as uncertainty disentanglement and visualization.
3.4 PUTTING IT ALL TOGETIEER
The graphical model for the Bayes-Mill Mayes-MILLin is shown in Fig. 5 For the classifier with parameter W, also use a multiplicative Gaussian modelling (Kingma et a.. 2013) as the posterior and posterior for calibrated uncertainty. The final objective is:
max 6
(Ø) (o) (a)
(10)
LD(
B wow W)), Rx, = (L W)lp((,, W)).
Rd( = KLlg(as(Cu. fol),
(t) H.))l(((s(Va.)
where RT, (() is KL. term for the probabilistic weights and Ra (d) is the SDPR for the attention. Ag and \ are the trade hyperparameters for two terms respectively.
RELATED WORK
Attention-based multiple instance learning For improving interpretability and performance of multiple instance learning (MIL), ABMIL (lse et al., 2018) first introduces the attention mecha- nism for embedding- based MIL. DSMIL (L al. 2021 considers contrastive learning for feature extraction and builds the global connection between patch attentions. TransMIL (Shao et al. 202T) proposes a correlated MIL and implements it by multi-head self-attention and spatial information encoding for full global correlation. CLAM (Lu et al. 2021) extends ABMIL to the case of multi- ple classes and builds integrated toolbox for visualizing the uncertainty. Although the attention has been extensively adopted for the MIL intrepretability. wrong attention is still being generated for the negative bags (see Fig. 2 and Fig. 6 This is not tolerable the application medical imaging, where the intrepretability of model crucial. The reason is, methods verify the convergence of attention and impose the constraint on correcting attention from the MIL principle. In this work, these issues are carefully resolved and new probabilistic framework proposed.
Orthogonal works Chen et al. (2022) proposes large-scatt scale vision transformer solution to simul- taneously learn the feature and classifier for WSL. Bayes-MIL freezes the feature extractor during training following normal setup in MII.. Zhang et al. (2022) proposes two-stage feature distil- lation MIL. framework for enhancing the performance. Bayes-MII. studies the fundamental inter- pretability problem the one-stage MIL framework. See Appendix for review of uncertainty in DNNs.
EXPERIMENTS
The proposed methods are evaluated on standard WSI datasets: CAMELYON16 (Bejnordi al. 2017) and CAMELYONI7 (Bandi et al. 2018). CAMELYON16 contains 400 hematoxylins and cosin (H&E) stained WSI of sentinel lymph node for breast cancer, labeled as normal or tumor classes. CAMELYONI7 contains 1000 WSI of the same type, labeled with normal or different stages (pN-stage) of the breast tumor. Since our paper mainly studies the interpretation of MII, we treat these stages as one class, generating binary slide-level labels (normal and tumors). We leverage the CLAM testbed for the implementation. A ResNet-50 is used for feature extraction, consistent with previous methods. Each result is obtained with 10-fold splits of training/vaiidaiionnessigg sets, which is a more thorough evaluation than previous papers. Other hyperparameters are listed in the Appendix. The codes submitted as supplemental. In our evaluation, consider 3 variants of BayesMII.: 1) Bayes-II--Vis is the basic Bayesian MIL in Sec. 3.T. 2) Bayes-MIL-SDDP is the model with slide-dependent patch regularizer from Sec. 3.2 3) Bayes-MIL--Ful is the whole model, The hyperparameters are X = 10-8 and X = 10-12 Results on CAMELYON17 show our method is better than the CLAM baseline in Patch-localization and Slide-llasification, shown in the Appendix.
Patch-level tumor region localization We first evaluate the patch-level results on tumor region localization. The tumor region localization uses the Patch-Precision, Patch-FROC and Patch-AUC

---

Published as a conference paper at ICLR 2023
Table I: Results on CAMELYONI6: (left) Patch-level localization results using Patch-FROC (P-FROC), Patch-
Precision (P-Prec.), Patch-AUROC (P-AUC); (right) Slide-level classification results using Slide-Accuracy (S-
Acc.), AUide-AUROC (S-AUC) and Cide-Calibottion (S-ECE).
Patch-level
Slide-level
P-Prec. (1)
P-FROC (1)
P-AUC (t)
S-Ace. (t)
S-AUC (?)
S-ECE (L)
DSMIL
0.1030
0,4443
0.7719
0.8682.10.05
0.379810.06 0.8944-0.06
CLAM
0.6068
0.4792
0.8839
0.8650110.66
0.9177-0.04
0.173
CLAM-T
0.7068
0,4830
0.8884
TransMIL
0.1726
0.4797
0.8644
0.8837=0.02
0.9307=0.04
0. 0.1436=#0.04
Bayes-MIL--in
0.7140
0,4797
0.8995
0.882510.005
0.9164-0.06
0. 0.1702.10.00
Payes-MIL-SDPY
0.7458
0,4856
0,9001
0.8875=00.0
0,9432=0.05
0.1021
Bayes-MIl APCRR
0.8107
0.4919
0.9129
0.900010.04
0.9479=0.05
0.122=00.44
Table 2: The patch-level localization results when using probabilistic attention values, model uncertainty and data uncertainty for our methods, evaluated on CAMELYON16.
P-Prec. (f)
P-FROC (1)
Attn
Model Une.
Data Unc.
Attn
Model Unc.
Data Unc.
Bayes-MiL-Ve MIL- Vis
0.7107
0.6999
0.7140
0.4796
0.4788
0.4797
MDPP
0,7445
0.7396
0.7458
0.4781
0.4856
0,4849
Bayes-MIL- MIL-APCRF
0,8078
0,8033
0.8107
0,4813
0,4919
0.4879
metries. The Precissoe is calculated by averaging the precision of classifying the patches. Each method provides a measure (attention ag or normalized uncertainty value 24,) for its ROL For calculating the precision, we compare the measure with three thresholds (0.1, 0.5, 0.9) for all methods and report the best results. Concretely, if as or Lis is greater than the threshold, the patch predicted as positive, otherwise negative. The Patch-FROC is defined as the average sensitivity (recall) at 6 predefined false positive rates: 1/4, 1/2, 1, 2, 4 and 8 FPs per WSI (Li et all 202T). The Patch-AUROC evaluates the averaged area under ROC over patches. The ground truth is the doctor's marking the region of the tumor.
For TransMIl., we take the diagonal of attention map from the last multi-head attention mod- ule as the measure. We add another baseline, CLAM-T, on CLAM training with temperature T' = [0,2, 0.5, 2, 5] on softmax function, a softmax(: T", as method for manually adjusting the density of localization, and select the best results for different metrics. For the Bayes-MIL- Bayes-MIL- MIL-SDRR and Bayes-MIL-APCEE methods, we take the best results from the normalized data uncertainty, the normalized model uncertainty and the probabilistic attention, with MC integra- tion over 16 samples.
The results for patch-level localization of tumor regions are shown in Tab. D (left). The Bayesian modelling of MIL (Bayes-MIL-Vis) generally improves the patch-level visualization over other methods. Tuning the temperature for MIL attention (CLAM-T) could marginally improve localiza- tion results, however, requires exhaustively tuning the temperature. For Bayes-MIL, the precision is improved by a large margin (by O. 1172-00661 1), showing the advantage of using uncertainty for localizing positive area. Including SIPR and the approximate CRF improve the patch-level results consistently, and the full model Bayes-MIL- achieves the best performance on the 3 metrics.
Tab. shows the detailed results with probabilistic attention, model uncertainty and data uncertainty for the three proposed methods. The probabilistic attention benefits from the Bayes-MIL model, making it competitive at tumor region localization. The best results for precision are from the disentangled data uncertainty, indicating the data uncertainty is able to precisely localize the tumor region. The reason that data uncertainty captures the rareness of features of positive patches indicating anomalies, such as cancerous cells, in the whole dataset. The model uncertainty is better on the P-FROC metric, which evaluates the recall different false positive rates. This shows that the uncertainty induced from parameter distributions in tends to cover the positive patches better.
Slide-level evaluation We next evaluate the slide-level classification performance. Tab. (right) shows the slide-level performance on CAMELYON16 measured by Accuracy AUC, and ECE. The results of DSMIL and TransMIL are from their papers, as their reported performance is better than our reimplementation. Compared with DSMIL and CLAM, Bayes-MIL- Vis has a higher accuracy, a lower calibration eтror and a similar AUC. When including our SDPR, Bayes-MIL-SDPP has better accuracy and AUC than MI..sMIL..
Why is Bayes-MIL good at slide classification and ROI localization? To understand the reason behind the good performance of the proposed framework, visualize attention distribution in Fig. D As shown in Fig. Joa and Fig. 6bb DSMIL and CLAM generate similar attention distributions for positive and negative slides. For the negative slides, DSMIL. still generates nearly normal
8

---

Published as a conference paper at ICLR 2023
10'
(a) DSMIL
(b) CLAM
(c) TransMIl.
19
10
10-
(d) Mayes--MlL-Vi Attn
Bayes- MII MIL-SDP Attn.
(f) Mayes-MII-APCRI Attn.
Figure 6: The log-scale histogram of attention for different methods. The pink and blue curves are for positive and negative slides, respectively. All testing images in CAMELYON16 are used, which contain 2M positive and 2M negative patches.
10-
(a) BMIL Vis M-Une.
(b) BBML--Vi D-Unc. (c) BMII- SDRR M-Une.
(d) BMIL-SDPR D-Une.
Figure 7: The log-scale histogram of normalized uncertainty for Bayes-MIL. M and stand for model and data. APCRF has similar histograms as SDPR, thus not shown for saving space.
distribution of attention, which deviates from the MIL principle. For positive slides, the densities on the positive side (a 1) of DSMIL and CLAM are low. Note that the vanilla version BMIL- Vis has different attention distributions for positive and negative slides, but does not generate high density on the positive side for positive slides. Bayes-MIL with SDPR is able push the attention of negative slides to the negative side (a ss 0, the blue curve in Fig. 6e while generating a U-shape distribution for the positive slides (red curve in Fig. 6e), which corresponds our design of SDPR with the ideal case shown in Fig. This design benefits the following aspects:
For visualization, Bayes-MIL generates concentrated high attention values for positive slides, while only generates correct values (a z 0) for negative slides.
For classification, learning correct attention guided by SDPR for the negative slides will benefit the classification of patches in the positive slides. TransMIL captures this attention distribution to some degree, so it performs well in slide-level classification.
Bayes-APCRF performs smoothing operation on the mean of the stochastic attention nodes. This pushes the histogram to have a more distinct U-shape (the U-shape curve in Fig. (i by ag- gregating neighboring positive patches, which benefits the localization and obtains the best visualization scores in Tab.
Correctly Visualized ROI Fig. shows the visualization results on the negative (normal) and positive (tumor) WSI. The related methods all generate high attention for the negative slides, which against of MIL principle. TransMIL. only generates small area high attention for negative slides, however the performance positive slides is poor. Bayes-MIL is the only method that generate correct values (a Re 0) for the negative slides, while performing the best in localization on positive slides.
Measure of uncertainty The measure of data uncertainty (Fig. 8b and Fig. 7dh naturally captures the information from the patch distribution. It shows similar distribution with the attention of Bayes-MIL-SDPR and the ideal case of SDPR in Fig. This might be the reason why data un- certainty has the best performance localizing the ROI. Furthermore, based on this observation on data uncertainty, the soundness of SDPR is empirically validated.
6 CONCLUSION
This paper analyzes the interpretability problem in existing attention based multiple instance learn- ing (MIL) models. Directly taking attention as measure of the important instances empirically violates the MIL princilee To address this problem, we propose probabilistic solution Bayes-MIL. that provides new measures for interpretability. To ensure the validity of interpretation in MIL, regularizer on attention is required. A slide-dependent patch regularizer is proposed for imposing explicit constraints to let the model learn under the MIL principle, which also improves the slide- level performance. The spatial information is further encoded, which improves both slide-level and patch-level performance. The analysis and visualization of data uncertainty distribution further val- idate the main idea and soundness of SDPR.

---

Published as a conference paper at ICLR 2023
REFERENCES
Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjac Lec, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE transactions on medical imaging. 38(2):550-555 5600 2018.
Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes Van Diest, Bram Van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen AWM Van Der Laak, Meyke Hermsen, Quirine F Manson, Maschenka Balkenhol, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama, 318((2):2199 2210, 2017.
Gabriele Campanella, Matthew G Hanna, Luke Geneslaw, Allen Miraflor, Vitor Werneck Krauss Silva, Klaus J Busam, Edi Brogi, Victor E Reuter, David S Klimstra, and Thomas Fuchs. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature medicine, 25(8):1301-1309 2019.
Richard Chen, Chengkuan Chen, Yicong Li, Tiffany Y Chen, Andrew D Trister, Rahul G Kr- ishnan, and Faisal Mahmood. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. (6144-16155, 2022.
Yufei Cui, Wuguannan Yao, Qiao Li, Antoni B Chan, and Chun Jason Xue. Accelerating monte carlo bayesian inference via approximating predictive uncertainty over simplex. arXiv preprint arXiv: 12194, 2019.
Yufei Cui, Ziquan Liu, Qiao Li, Antoni B Chan, and Chun Jason Xue. Bayesian nested neural networks for uncertainty calibration and adaptive compression. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 2392-2401, 2021.
Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snock, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran. Efficient and scalable bayesian neural nets with rank-1 factors. In International conference on machine learning, pp. 2782-282222 2792. PMLR, 2020.
Yarin Gal, Jiri Hron, and Alex Kendall. Concrete dropout. Advances neural information process- ing systems, 30, 2017a
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, pp. 1183-1192, PMLR, 2017b.
Yarin Gal et al. Uncertainty in deep learning.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint / 2011.
Maximilian Ilse, Jakub Tomezak, and Max Welling. Attention bassd deep multiple instance learn- ing. In International conference on machine learning. pp. 2127-2136. PMLR, 2018.
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameteri- zation trick. Advances in neural information processing systems, 28, 2015.
Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14318-14328, 2021.
Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology whole-slide images. Nature biomedical engineering, 5(6):555-570, 2021.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in
neural information processing systems, 31, 2018.
10

---

Published as a conference paper at ICLR 2023
Andrey Malinin, Bruno Mlodozeniee, and Mark Gales, Ensemble distribution distillation. arXiv 2019.
preprint
Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov. Variational dropout sparsifies deep neural networks. In International Conference on Machine Learning, pp. 2498-2507. PMLR, 2017.
Nikhil Naik, Ali Madani, Andre Esteva, Nitish Shirish Keskar, Michael Press, Daniel Ruderman, David B Agus, and Richard Socher. Deep learning-enaael enabled breast cancer hormonal receptor status determination from base-level h&e stains. Nature communications, 11(1):1-8, 2020.
Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Ad- vances Neural Information Processing Systems, 14:2136-21477 2021.
Marvin TT Teichmann and Roberto Cipolla. Convolutional crfs for semantic segmentation. arXiv preprint arXiv: 1805.04777 2018.
Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06715. 2020.
Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah E Coupland, and Yalin Zheng. Difd-mil: Double-tier feature distillation multiple instance learning for histopathol- ogy whole slide image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 18802-18812 2022.
Jingwei Zhang, Ke Ma, John Van Amam, Rajarsi Gupta, Joel Saltz, Maria Vakalopoulou, and Dimitris Samaras. joint spatial and magnification based attention framework for large scale histopathology classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 3776-3784, 2021.
Zizhao Zhang, Pingjun Chen, Mason McGough, Fuyong Xing, Chunbao Wang, Marilyn Bui, Yuanpu Xie, Manish Sapkota, Lei Cui, Jasreman Dhillon, et al. Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nature Machine Intelligence, 1(5):236-245, 2019.
Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes Vibhav Vineet, Zhizhong Su, Da- long Du, Chang Huang, and Philip HS Torr. Conditional random fields as recurrent neural net- works. In Proceedings of the IEEE international conference on computer vision, pp. 1529-1537, 2015.


Reviews:
[{"review_id": "iCR45Fd3_u", "reviewer": "Reviewer_nxBn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for attention-bases multi-instance-learning (MIL) on digital pathology whole slides (WSI) for tumor classification. The method improves upon SOTA MIL methods on WSI by approaching the problem using a probabilistic framework. A drawback of current attention-based WSI MIL methods is that attention is not focused on positive (tumor) instances only. The proposed approach ensures that attention is focused on positives instances, making the results more interpretable as high attention areas match tumor areas. A Bayesian modeling of MIL provides calibrated uncertainty at the patch level. A variational inference framework regularizer produces dense localization of positive patches. Finally the spatial correlations are captures by a conditional random field CRF. The empirical section compares the proposed approach to SOTA MIL WSI approaches such as DSMIL, TransMIL on two public datasets, showing significant improvements. The results are also ablated showing the contribution of individual modules.\n", "summary_of_the_review": "A well-written paper that proposes a novel way to improve on interpretability of WSI classification, while also improving SOTA accuracy on 2 large public datasets.\n", "strength_and_weaknesses": "Pros:\n\n* The method improves SOTA both on interpretability (the high attention patches correspond to tumor areas) and accuracy.\n\n* The spatial correlation of patches is learned by a CRF, which conforms to the tendency of tumors to locally grow.\n\n* The contribution of each module is clearly reported and show that each of them provides additional improvements.\n\nCons:\n\n* The feature extractor is simply a resnet50 (imagenet). For the WSI task, other works have shown that self-supervised models trained on WSI data do have superior results.\n\n* There are no speed information given for training and inference. How long does it take to classify a slide on average ? How long does it take to train the feature extractor, the MIL model ? Also a pseudo-code of the algorithm would help.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Y_y6lsPlj0", "reviewer": "Reviewer_c5qM", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes interpretable models in multiple instance learning for WSI classification. The authors identify problems in using directly attention mechanism for interpretation. The experiments conducted indicates that high attention values are still generated in negative slides, indicating positive patches. To address this issue, the authors propose Bayesian MIL incorporating 1) probabilistic instance-wise attention module for uncertainty visualization 2) slide-dependent patch regularizer for learning the correct attention distribution 3) an approximate convolutional conditional random field for encoding spatial information. The experiments on two WSI datasets showed that the proposed Bayes-MIL outperforms baselines in path and slide levels metrics. ", "summary_of_the_review": "The paper introduces a new approach for MIL problem which brings uncertainty measure in the model and improves interpretability. This is highly relevant for pathology applications where annotations are available only at slide level.", "strength_and_weaknesses": "Strength\nThe proposed approach improves upon existing MIL approaches on different aspects: It improves the performance at slide and patch level. It improves the interpretability by introducing calibrated model uncertainty.\nThe paper uses a 10-fold cross validation in the evaluation, which is not frequently seen in the literature due to the computational cost.\n\nWeakness\nThe scalability of the approach is not discussed. How much Bayes-MIL is slower than baselines in training and inference time. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "PLimfIzbZT", "reviewer": "Reviewer_ndc8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies multiple instance learning for WSI analysis. They pointed out that the interpretability of previous attention-based MIL methods for WSI diagnosis was either untrustworthy or unsatisfactory and proposed Bayes-MIL to induce patch-level uncertainty as a new measure of MIL interpretability. To impose constraints derived from the MIL assumption, they further designed a slide-dependent patch regularizer. Last, they applied an approximate convolutional conditional random field to encode the spatial information in WSI. The proposed method was evaluated on public datasets.", "summary_of_the_review": "This paper discusses an interesting problem in MIL-based WSI analysis, while the current version need further improved in regard of clarification and evaluation.\n\nAfter response: \n\nThe new clarification and experiments have partially addressed my concerns. Therefore,  I would raise my score to 6: marginally above the acceptance threshold.\n", "strength_and_weaknesses": "Strength:\n1. The discussed problem is interesting and important for AI-assisted pathology diagnosis.\n2. The proposed slide-dependent patch regularizer (SDPR) explicitly encodes the basic assumption of MIL into the training process, which is a long-ignored issue in weakly-supervised learning for WSI analysis.\n3. Good performance on two breast cancer datasets in normal/tumor classification tasks.\n\nWeakness:\n1. The authors should discuss more about the relationship between the WSI pathology diagnosis and MIL. It is easy to understand that the normal/tumor classification in WSI diagnosis followed the basic assumption of MIL and can benefit from the proposed SDPR. However, other tasks in WSI diagnosis may not meet the same requirements. For example, both tumor subtype classification with more than two classes and tumor stage classification tasks cannot be dealt with the proposed Bayes-MIL and SDPR. Does this mean that the current solution is limited in the simplest positive/negative setting in MIL?\n2. As for the tumor/normal classification, it is far more important to localize the ROI in tumor WSI instead of the normal WSI. However, in the Tumor WSIs in both Fig 2. And Fig 8., CLAM seems to provide a more accurate (the highlighted regions have better shape and is denser) visualization than the proposed method. \n3. Is it ok to conclude that in previous works, high attention weights are used to indicate it associated instances are cancerous image patches? If the slide-level prediction for a WSI is normal (without cancer), how do we interpret the high attention weights?\n4. From Table 1., we can see that without SDPR and APCRF, the proposed method is not better than previous attention-based works. Does this mean the data uncertainty is not necessary? Can we apply the SDPR and APCRF to previous attention-based works and achieve satisfactory improvement? More ablation study is needed to validate the motivation and effectiveness of the proposed Bayes-MIL.\n5. The experiments are limited to the breast cancer datasets and normal/tumor classification task.\n6. The std of the experiments is not provided in the paper.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster