Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classification
Authors: Linhao Qu
xiaoyuan Luo
Manning Wang
Zhijian Song
Venue: NeurIPS 2022 Accept (2022)
Keywords: multiple instance learning
whole slide image
knowledge distillation
weakly supervised learning

Abstract: Computer-aided pathology diagnosis based on the classification of Whole Slide Image (WSI) plays an important role in clinical practice, and it is often formulated as a weakly-supervised Multiple Instance Learning (MIL) problem. Existing methods solve this problem from either a bag classification or an instance classification perspective. In this paper, we propose an end-to-end weakly supervised knowledge distillation framework (WENO) for WSI classification, which integrates a bag classifier and an instance classifier in a knowledge distillation framework to mutually improve the performance of both classifiers. Specifically, an attention-based bag classifier is used as the teacher network, which is trained with weak bag labels, and an instance classifier is used as the student network, which is trained using the normalized attention scores obtained from the teacher network as soft pseudo labels for the instances in positive bags. An instance feature extractor is shared between the teacher and the student to further enhance the knowledge exchange between them. In addition, we propose a hard positive instance mining strategy based on the output of the student network to force the teacher network to keep mining hard positive instances. WENO is a plug-and-play framework that can be easily applied to any existing attention-based bag classification methods. Extensive experiments on five datasets demonstrate the efficiency of WENO. Code is available at https://github.com/miccaiif/WENO.

Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classification
Linhao Qu'
Xiaoyuan Luo"
Digital Medical Research Center, School of Basic Medical Science, Fudan University 1hqu208fudan edu. cn
Digital Medical Research Center, School of Basic Medical Science, Fudan University 191110100300fudan edu. cn
Manning Wang
Zhijian Song!
Digital Medical Research Center, School of Basic Medical Science, Fudan University mnwang@fudan. edu. en
Digital Medical Research Center, School of Basic Medical Science, Fudan University sjsongfuudan edu. on
Abstract
Computer-aided pathology diagnosis based on the classification of Whole Slide Image (WSI) plays important role in clinical practice, and it is often formulated as a weakly-supervised Multiple Instance Learning (MIL.) problem. Existing meth- ods solve this problem from either bag classification or an instance classification perspective. In this paper, we propose an end-to-end weakly supervised knowledge distillation framework (WENO) for WSI classification, which integrates bag clas- sifier and an instance classifier improve the performance of both classifiers. Specifically, an attention -based bag classifier is used as the teacher network, which is trained with weak bag labels, and an instance classifier is used the student network, which is trained using the normalized attention scores obtained from the teacher network as soft pseudo labels for the instances in positive bags. An instance feature extractor is shared between the teacher and the student to further enhance the knowledge exchange between them. In addition, based on the output of the student network to force the teacher network to keep mining hard positive instances. WENO is plug-and-play framework that be easily applied to any existing attention-based bag classification methods. Exten- sive experiments five datasets demonstrate the efficiency of WENO. Code is available at
knowledge distillation framework to mutually
propose hard positive instance mining strategy
Introduction
Histopathological images play an important role in cancer diagnosis and prognosis prediction 23 33
32 [20] [2] 26), and they can be scanned by digital slide scanners into Whole Slide Images (WSIs), which facilitates the development of deep learning-bart bassdd automatic analysis techniques. However, there are challenges in deep learning-based WSI analysis. First, WSIs have huge resolutions, typically reaching 100,000 x 100,000 pixels, and thus cannot directly input into deep models. For this reason, WSIs are usually tiled into many small patches for processing. Second, fine-grained
"Equal Contribution.
1 'Corresponding Authors. All authors are also from Shanghai Key Lab Medical Image Computing and Computer Assisted Intervention.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

(patch-level) annotation is very time-consuming and intee-iiteesive,, and usually pathologists can only provide slide-level labels, so traditional supervised learning methods cannot be directly used. Therefore, WSI classification is often formulated as a deep multiple instance learning (MIL.) problem, which is a weakly supervised learning paradigm [32 28
In the MIL paradigm, each WSI is considered as a bag, and the patches cut out of it are considered as its instances. If bag is negative, all the instances in are negative, while if' a bag is positive, least one positive instance exists in it. Typically, deep MIL-based WSI classification performs two main tasks: bag classification and instance classification, which are used for automatic clinical diagnosis and positive region localization, respectively.
Currently, deep MIL-based WSI classification methods can be mainly classified into sastacee-basel approach and bag-based approach. The instance-based approach [2] first trains an instance classifier and then aggregates the predictions of each instance in bag to obtain the bag prediction 7 R4 35
Because of the lack of patch-level labels, it is not known which patches are truly positive, so instance- based approach needs to select some patches from positive slides and assign them pseudo positive labels for training an instance classifier. The main problem of this approach is that the pseudo instance labels contain lot of noise, which limits the performance of the trained instance classifier, and thus leads to inaccurate instance and bag classification.
The more common bag-based approach [13 9 40 38 features of each instance in a bag and aggregates the instance features to obtain the bag feature using& trainable attention mechanism. Then, bag classifier is trained in supervised manner. During inference, the bag classifier is used to perform bag classification and the attention scores can be utilized to measure the contribution of each instance to bag classification so as perform the instance classification. However, this bag-based approach suffers from two serious problems: 1) Poor performance of instance classification. Different positive instances have different levels of difficulty of being recognized. The classifier is trained on bag-level loss, so it can correctly recognize positive bag by giving high attention weights to only few easily recognized positive instances, while the harder ones are ignored. This makes the network tend to accurately classify only a few easy positive instances in the positive bags and lack the motivation to accurately classify the hard instances [3. 2) Bias of the bag classifier. For the same reason, the trained bag- -laassassffier will have difficulty to generalize on bags that contain only hard positive instances.
18
29
19
first extracts the 30
22
To address the above issues, we propose an end-to-end weakly supervised knowledge distillation framework (WENO) for whole slide image classification. WENO integrates bag classifier and an instance classifier in a knowledge distillation framework to mutually improve the performance of both classifiers by effective knowledge transfer between them. To the best of our knowledge, the concept of weakly supervised knowledge distillation is proposed for the first time. Figure | concisely illustrates the existing knowledge distillation paradigms and the principle of WENO proposed in this paper. Specifically. WENO contains a bag classifier based on the attention mechanism as the teacher branch and an instance classifier as the student branch. To address the problem of poor instance classification performance, we use the normalized attention scores of the teacher network as the soft pseudo labels of the instances in the positive bags to train the student branch through knowledge distillation, which alleviates the noisy pseudo label problem in previous instance-based methods. At the same time, since all the instances in negative bags are negative, they are also used in training the student branch to further improve its instance classification performance. Notably, different from common knowledge distillation methods in Figure and (b), we not only train the student branch but also train the teacher branch using the bag labels of WSIs Moreover, we share the instance feature extractor of the student and the teacher to further enhance the knowledge transfer between the two branches. To further address the problem of bias in the bag classifier, we propose hard positive instance mining (HPM) strategy. In particular, we use the knowledge learned by the student instance classifier to construct bags with less easy instances, forcing the teacher network to keep mining hard positive instances in positive bags.
The ad advattages of our method are:
We propose weakly supervised knowledge distillation framework, WENO, for WSI classification. WENO integrates a bag classifier and an instance classifier in knowledge distillation framework to mutually improve the performance of both classifiers by two-way knowledge transfer between them.
2

---

(a) Supervised Knowledge Distiliation
(b) Self-Knowladg Distillation
(c) Weakly Supervsied Knowledge Distillation
Teacher
Teacher
Teacher
Stop
gradient
Beg labels
WSis (bags)
Loss /
Instance labels
EMA
Loss
Shared
Altention scores
LoSS
Student
Student
Student
Patches
(instances)
Shared": shore instance feature extractors
Figure 1: Architecture of common existing knowledge distillation frameworks and the proposed weakly supervised knowledge distillation framework. In traditional supervised knowledge distillation (a), the teacher network is trained in advance and it keeps unchanged during training the student. Knowledge distilled from the teacher to the student. In recent self-knowledge distillation (b), such DINO [31, the teacher has the same architecture with the student and it is not trainable but updated from the student. Two-way knowledge transfer exists between the teacher and the student. In comparison, in proposed framework (c), the teacher is bag-classifier and it is also trained with weak slide-level labels. Knowledge distilled from teacher to student by providing pseudo instance labels using attention scores of the teacher and knowledge transfer from the student the teacher is achieved by sharing instance feature extractors between them.
We further propose a hard positive instance mining (HPM) strategy to force the teacher network to continuously learn and mine hard positive instances in positive bags, thus alleviating the problem of focusing on easy positive instances in bag-based methods.
WENO is a plug-and-play framework that can be conveniently applied any existing bag-based methods using attention mechanism to improve their performance of both instance and bag classifica- tion. Extensive experiments five datasets show that the current SOTA methods ABMIL [13] and DSMIL [8] achieve significant improvements in both instance and bag classification performance when they combined with WENO. Code will be publicly available.
2
Related Work
2.1 Deep Multiple Instance Learning
Instance-based methods Instance-based methods focus on training an instance classifier and then aggregating the instance predictions in bag to make the bag prediction. For example, Campanella et al. [2) and Chikontwe et al. (5) iteratively trained the instance classifier by selecting key instances based the predicted probability of the instance classifier in each iteration and assigning them pseudo labels of the corresponding bags. After that, the former used Recurrent Neural Network (RNN) to aggregate instance predictions perform bag classification, and the latter proposed soft-assigmment strategy for bag inference.
Bag-based methods Bag-based methods focus aggregating instance features in bag into bag feature, and then training bag classifier with bag labels. Most them utilize attention mechanism to aggregate instance features, in which llse et al. [ is the first work of this kind. Later, Hashimoto et al. [9] used the attention mechanism to aggregate instance features at different resolutions. Yao, Zhu al. [40 38) proposed to first cluster the instances in each bag and then use the attention mechanism to aggregate the features of different clusters. Shi et al. [3I] added the instance-leve loss to the bag-level loss of [13], but the pseudo labels of each instance still came from its corresponding bag label. Recently, Li et al. 18] proposed to use non-local attention to aggregate instance features. Shao
et al. 22)] Li al. [19 proposed use Transformer to aggregate instance features.
Different from previous MIL methods, we construct a weakly supervised knowledge distillation framework to combine the training of bag classifier and instance classifier and utilize the knowledge transfer between the two branches the distillation framework mutually enhance both classifiers.

---

2.2 Knowledge Distillation
Knowledge distillation is originally method to transfer knowledge from a pre-trained complex teacher network to a simpler student network. In the deployment phase, the student network can replace the teacher network achieve model compression 25 24
6 [1] common method of distilling knowledge from the teacher network to the student network is to use the output logits of the teacher network as the soft labels for training the student network
21
12
Recently, self-distillation techniques are developed to train a student network without pre-trained teacher network [17 36 39 3) Instead, the teacher usually has the same structure the student and is updated by momentum- baseed moving average of the student network (3) Figure T (a) and (b) concisely illustrates the two main existing knowledge distillation paradigms.
Knowledge distillation is fast-developing area and we refer to 8 for comprehensive survey. In this paper, propose the concept of weakly supervised knowledge distillation for the first time.
3 Method
3.1 Problem Formulation
3.1.1 Multiple Instance Learning (MIL)
Given a dataset
(, W ., 3 consisting of WSIs, each WSI W; is tiled into non- overlapping small patches {pi.j.j = 1,2..... ...,, m,), where n; is the number of patches cut out of W.. All patches Pt,j in W; form a bag, where each patch is an instance. The bag label Y; E (0. 1).
and the instance labels {Vr.j. 1,2...., n, have the following relationship:
0.
if E, Vi. =0
Y
1,
else
That is, all instances in negative bags negative, while least one positive instance exists in positive bag. In MIL, only the labels of the training bags are available, while the labels of the instances in each positive bag are unknown. Our objective is to accurately predict both the labels of each bag in the test set (bag classification) and the labels of each instance in them (instance classification).
3.1.2 Bag-based MIL Methods
We first briefly review bag-baser methods for easier understanding of our proposed framework. These methods first use an encoder f to extract features ži, for all instances { = 1, 2, n2 in bag W. and then aggregate these instance features using a permutation invariant function g obtain the bag feature Z;
Z: = g (f (pr,1) ff (pr.2)..)
Finally, bag classifier to is utilized predict the class of the bag.
(Z:)
(3)
Traditional aggregation functions include max-pon max-pooling and mean-pooling, while ABMII. presents an attention-based trainable aggregation function:
Z a (pe.)
(4)
1=1
exp (w: tanh (VhT,))
0i.j Ey= exp T tanh (VA[))
(5)
where ai.j is the attention score predicted by the self-attention network which is parameterized by and V. The subsequent bag-based methods almost all adopt the attention-based aggregation methods,
and the difference lies in how to construct the attention weight aij. The weights a,, reflect the

---

contribution of each instance in making the bag prediction, and they can be normalized as the instance prediction for positive bags.
Dejj norm (ar.2)
The normalization function can be formulated as follows:
X Xmin
X Xma Xmin
where or, is the normalized data, is the original data, and Xmaz and Xm in are the maximum and minimum values of the original data, respectively.
3.2 Weakly Supervised Knowledge Distillation Framework
Updete (1)
HPM madule
Attention module
Update
bag feature
Update
CE Loss
Bag prediction head
Encoder
Bag predictions
Bag labels
WSis (bags)
(1)
Instance features
Attention scores
Tilin
Shared
CE Loss
Instance prediction head 9,
Encoder !
Update (2)
pdato (2)
patches
Instance fealures
Instance predictions
Soft
(instances)
pseudo
labels
Figure 2: Architecture of the proposed weakly supervised knowledge distillation (WENO) framework. WENO contains teacher branch (the red branch) and student branch (the blue branch). The teacher branch is essentially a bag-level classifier, which consists of an instance encoder, a hard positive instance mining (HPM) module, an attention module and bag prediction head. The student branch is essentially an instance-level classifier, which consists of an instance encoder and an instance prediction head. The two branches share the same instance encoder. The teacher branch is trained with bag labels, while the student branch is trained with the attention scores of the teacher branch as soft pseudo labels for the instances in positive bags. Furthermore, we propose hard positive instance mining strategy (the HPM Module) to leverage the knowledge learned by the student to help the teacher learn hard positive instances. Note that the teacher branch and the student branch are optimized alternately, where the optimization of the teacher branch is represented by update (1) in the figure and the optimization of the student branch is represented by update (2) the figure.
3.2.1 Framework Overview
Figure 2 fillustrates the overall framework of our proposed WENO, which contains a teacher branch (the red branch) and student branch (the blue branch). The whole teacher branch is a bag classifier, which consists of an instance encoder, hard positive instance mining (HPM) module, an attention module and a bag prediction head. The whole student branch is an instance classifier, which consists of an instance encoder and an instance prediction head. The encoders in the two branches share the same parameters. We directly train the bag classifier in the teacher branch with bag labels. The instance classifier in the student branch is trained with two sets of instances, in which the first set is negative instances from negative bags and the second set is the instances from positive bags with pseudo labels generated according to the normalized attention scores output by the bag classifier in the teacher branch.
Different from the traditional distillation methods in which the teacher is pre-trained or updated by momentum updates from the student, our teacher and student models are trained alternately, so that not only the teacher can transfer the knowledge learned from the bag labels to the student, but also the knowledge learned by the student can be transferred to the teacher through the shared encoder. Furthermore, we propose the hard positive instance mining strategy (the HPM Module) to help the teacher better explore hard positive instances using the knowledge learned by the student, and further improve the network's generalization ability for bag classification.
5

---

3.2.2 Teacher Network
The teacher branch is typical bag classifier and one bag is input to it at time. The instance features 3 are first extracted using the encoder ft for all instances [p...] = 1, ........ n2} within the bag We and then filtered by the hard positive instance mining (HPM) module (See Section 3.3 for details), and then the attention score Ji,j for each instance زا is obtained by the attention module A, Finally, the bag feature Zi is obtained by aggregating the instance features within the bag using attention scores, and input the bag prediction head (e to obtain the bag-level prediction Y, Since the bag label Y; is available, the teacher branch can be trained in an end-to-end manner.
Zi, fi (pis)
(8)
Oi = A (Zt.)
24,1, 24,21 7i,o))
(9)
2,
(10)
j=1
8, = 40: (Zi)
(11)
LosS
CE Yz, 8.)
(12)
The purpose of training the beg-level classifier in the teacher branch is to obtain the attention scores of each instance and use them as soft pseudo labels to train the instance-level classifier in the student branch. Note that the teacher branch can be implemented with any existing bag-based MIL. methods using attention mechanism, such as ABMIL (3). DSMIL [8), and TransMII. (29), etc., and we compare the performance of using ABMIL and DSMIL the teachers in experiments (Section 4.5
and 4.6).
3.2.3 Student Network
The student branch is an instance- classifier which consists of an encoder fa shared with the teacher branch and an instance prediction head (... We train the student branch using both the instances in positive bags with the normalized attention scores of the teacher classifier as soft pseudo labels and the instances negative bags with true negative labels. Different from the teacher branch, the inputs to the student branch are randomly selected instances, which may come from the same bags or different bags.
The loss function of the student branch is the cross-entropy between the network prediction رihة and the label y
norm (ai.s)
if Y; = else
Vii
0,
(13)
Diss 4s (21.t)
(14)
Losssrudent CE (B1,5, D1.3)
(15)
Since the student not only learns from the teacher's attention scores, but also learns from the truc negative instances directly, the instance classification ability of the student can surpass the attention scores of the teacher, which is supported by our experimental results (Section 4.3 4.6 and 5) Moreover, since the student and the teacher share the parameters of the instance encoders, the knowledge learned by the student can also improve the instance-level classification ability of the teacher, which is also validated in the ablation study (Section
In inference, we use the instance classifier in the student branch to make predictions for all instances in bag, and then use a simple max-pooling max-pooling to aggregate the predictions of instances to accomplish bag prediction.

---

3.3 Hard Positive Instance Mining Strategy (HPM)
Positive bags contain multiple positive instances, but they differ significantly in the difficulty of identification (e.g., in cancer detection, some patches contain large number of cancer cells, while some other patches contain small number of cancer cells). The loss of the bag-based methods is defined at the bag-level, so it only needs to identify at least one positive instance to classify the positive bag correctly. This makes bag-level classifiers tend to learn only easy positive instances but ignore the hard ones during training, which limits both the bag and the instance classification performance. To address this problem, we propose the hard positive instance mining strategy to force the teacher network to continuously learn and mine hard positive instances in the positive bags.
Specifically, we first train the teacher and the student models for a certain number of epochs, after which the student network has certain instance classification capability. Then, before continuing to train the teacher, we use the student classifier to predict all the instances in the input positive bags, and drop some instances for which the student outputs high positive prediction probability to construct hard pseudo bags. In this way, we can force the bag classifier to keep mining hard positive instances in positive bags to achieve better performance.
4 Experiments and Results
4.1 Datasets
We used five datasets to comprchensively evaluate the performance of WENO, including two synthetic datasets and three real-workh datasets. To explore the performance of WENO under different positive instance ratios, we used the 10-class natural image dataset CIFAR 10 15 and the 9-class pathological image dataset CRC [14 to construct synthetic WSI datasets with different positive instance ratios, and they are denoted CIFAR- 10-MIL dataset and the CRC-MIL dataset, respectively. Furthermore, we used real-world pathology datasets from three different medical centers to evaluate the performance of WENO, including a breast cancer lymph node metastasis public dataset, the Camelyon16 6 dataset [1] lung cancer diagnosis public dataset, the TCGA Lung Cancer dataset, and in-house cervical cancer lymph node metastasis dataset, the Clinical Cervical dataset. Detailed descriptions of the datasets available in the supplementary material.
4.2 Evaluation Metrics
For both instance and bag classification, we use the Area Under Curve (AUC) as the evaluation metric. We report the AUC metrics for the instance and bag classification on two synthetic datasets and the Camelyon 16 dataset. However, since the instance-level ground truth labels for the TCGA Lung Cancer dataset and the Clinical Cervical dataset are not available, we only report their AUC for bag classification.
4.3 Implementation Details
For the CIFAR-I0-MIL dataset, the encoder in Figure 2 is implemented using the AlexNet [[6] For the other datasets, the encoder is implemented using the ResNet18 [10] Both the prediction heads and the attention module are implemented using fully connected layers. No pre-training of the network parameters and no image augmentation are performed. The SGD optimizer is used to optimize the network parameters with a fixed learning rate of 0.001. For the hard positive instance mining strategy, we drop the instances with positive probability higher than threshold in positive bags. The hyperparameter thresholds vary for each dataset, and we used grid search on the validation set to determine the optimal values. In the supplementary material, we give robustness study of the threshold on the Camelyon 16 dataset. All experiments were performed using 4 Nvidia 3090 GPUs.
4.4 Comparison Methods
We compare WENO with a series of state-of-the-arr methods. For both the synthetic datasets and the real-world datasets, we use the classic ABMIL [13 and the latest DSMIL [88] as the teachers to construct the WENO frameworks and compare their instance and bag classification performance with SOTA methods. The comparison methods include instance-based methods: RNN-MIL (2) and
7

---

Chi-MIL 5I: bag-based methods: ABMIL 13 Loss-ABMIL BT and DSMIL [18.. We reproduced these methods based on the published codes, and the specific parameter settings are provided in the supplementary material. We also compare the results of using the fully supervised approaches on the synthetie datasets and the Camelyon 16 dataset, i.e., performing supervised training using the true labels of each instance and aggregating the instance predictions using max-poolle max-pooling to obtain the bag predictions.
4.5 Results on Synthetic Datasets
Table|aand Table 2 show the instance and bag classification performance of WENO on the CIFAR- 10- MIL dataset and the CRC-MIL dataset with different positive instance ratios, respectively. We use the classic ABMIL 133 and the latest DSMIL [18] as the teachers to construct the WENO frameworks. It can be seen that the WENO framework significantly improves the performance of the two original bag-based methods for both the bag classification and the instance classification tasks under all positive instance ratios. The advantage of WENO is especially significant for instance classification. In particular, DSMIL [18] does not work well in instance classification under low positive instance ratios, while the performance of DSMIL+WENG is much higher. For bag classification, as shown in [Tble[[()) DSMIL [8] does not work at positive instance ratios of 5% and 10%, while the bag classification AUC after combining the WENO framework reaches 0.9367 and 0.9900, respectively. These results show the powerful advantages of WENO: significant performance gains and easy plug-and-playy ability.
Table I: Results on the CIFAR-IO-MIL dataset.
(a) Instance classification AUC.
(b) Bag classification AUC.
Pusiiine paich
16
10%
50%
Poslive paich
5%
20%
70%
Fully seporvioll
0.9015
0.0621
0.0233
0,9540
0.0999
0,9215
Filly sxperisce
0.9331
0.902
1.00
ARMII [1]
0.8590
1,000
1.009
6.447
0.3412
0.445
WENO
1,000
1.000
+8.1174
+0.1519
DIMII
06152
0.7825
0.514
09966
1.000
MIM WEND
DSMIL
1.000
+0,3252
+0,3893
+0.3241
+0,0532
Table 2: Results on the CRC-MIL dataset.
(a) Instance classification AUC.
(b) Bag classification AUC.
Posiiin paic ralio
10%
20%
50%
70%
Puodtivc patch main
10%
2003
50%
70%
Fully segerviood
0.9978
0.9076
0.9007
0:0071
Fully supersisod
1,000
1,000
1.000
1,000
ARMIL [S
0.7410
0.8728
0.8800
017665
ARMIL. 03
0.954
1,000
0.9266
0.9894
ARMIL. + WENN
0.9625
0,0819
0,5759
0,9786
ARMIL - WEND
10000
1,000
0.9957
L,000
+0,2215
+0.1090
+0.1959
-0.1821
A
+0.02.0
-0.0151
+0.0106
DSMIL 18
0,3690
0.7008
0.4535
0.7399
DSMIL 18
1,000
0.8914
0.9997
1,009
USMIL + WENU
0.9697
0.9801
0.9817
0.9700
DSMIL WEND
1,000
1,0000
1,0000
1.400
+0.6007
+0.2793
+0.4982
-0.2361
3
+1.1086
+0.0003
4.6 Results on Real-World Datasets
Table 3 (a) shows the performance of WENO on the Camelyon16 dataset for the instance and bag classification. We still use ABMIL [3 and DSMIL [18] as the teachers to construct the WENO frameworks. It can be seen that using WENO brings significant performance improvement in both classification tasks. Notably, with only bag-level weak labels, the best instance classification AUC (0.9377) obtained with WENO lower than that of fully supervised method (0.9644) by only 0.0267; while the best bag classification AUC (0,8663) exceeds the fully supervised (0.8621) method, showing the powerful performance of WENO again.
Table 3 (b) shows the bag classification performance of WENO on the TCGA Lung Cancer dataset. We use ABMIL [13] and DSMIL [18] as the teachers to construct the WENO frameworks and achieve the best performance.
Table (c) shows the bag classification performance of WENO on the Clinical Cervical dataset. We use ABMII. [33 and DSMIL [8] as the teachers to construct the WENO frameworks and achieve the optimal performance. In contrast to previous clinical tumor recognition and classification tasks,
8

---

the prediction of lymph node metastasis according to the WSIs of primary lesion in this dataset is very challenging task in current clinical practice. Since there is no prior knowledge about what image features of the primary lesion indicate metastasis, even very experienced pathologists are unable to clearly distinguish between positive and negative instances. WENO achieves the best performance among all comparison methods, which on the one hand indicates that WENO can be applied to the prediction of metastasis using primary lesion WSIs in clinical practice, and on the other hand suggests the potential of WENO to detect underlying pathological patterns from high confident instances.
Some examples of positive and negative slides/patches in the three real-world datasets and the visualization of the heatmaps on the Camelyon 6 dataset are given in the supplementary material.
Table 3: Results on the three real- datasets.
(a) Camelyon16 Dataset.
(b) TCGA Lung Cancer Dataset, (c) Clinical Cervical Dataset.
Methols
Instasce Histasse-levelAA AUC
leve AUC
Metheds
Bap-
Methods
Heg level
Pully sipersieed
0.9644
0,8621
Mar-pooling
0.9169
ARMM-
Ano-AAMML 11
CHMILL X
0,7880
Mac-podiig
0.9014
0.9107
ABMIL
CabMIL
0.7425
06446
ABMIL [X]
0:9488
ABMIL WENO
0.8056
ABMIL
ABMIL WEND
0.9663
+0.1619
ABMIL
+0,079)
0:9271
-0J1175
DSMI [
0,8022
DSMIL WENO
RSMIL [H]
0.5568
(18401
DEMIL [1]
DSMIL + WEND
.0.0594
0.9633
0.9727
DSMIL WENO
A
+0.0300
6,822
+8,8094
5 Ablation Study
Figure shows the results of the ablation study of WENO on the CIFAR-10-MII. dataset with a positive instance ratio of 0.2. In this experiment, we use the ABMIL [3 as the teacher to construct the WENO framework. By analyzing the curves in Figure we can find: (1) Comparing the curves of the raw 'ABMIL' and ABMII+WENNO (without HPM) in panel (a) and panel (b), it can be seen that when the bag-level AUC reaches the maximum at about the 25th epoch, their instance-level classification ability is still poor. In addition, the instance-level AUC of the raw ABMIL gets worse as the training proceeds, indicating that the network gradually tends to distinguish positive bags by simple positive instances only. As shown in Figure 3 (b), when WENO is used with ABMIL, the instance- classification capability is significantly improved, and the network is still able to continuously improve the instance classification capability as the training proceeds. 2) Comparing the curves of the AABMII++ WENO (without HPM) and ABMIL+WEER
(with HPM)' in panel (b) and panel (c), it can be seen that the proposed hard positive instance mining strategy can further improve the instance classification ability of both the student and the teacher by forcing the teacher network to continuously learn and mine hard positive instances in positive bags. HPM is used after the I 150th epoch.
(4) Test Begylevel AUC by Teacher
(b) Test Instance level AUC by Thxcher
(c) Tistt Trestmon levell AUC by Sturert
alaas vetio Cotn 1996
F,M0 -2012 1900
6.0
750
750
750 29
250
Figure 3: Ablation study curves on the CIFAR- dataset.
Table 4 shows the results of the ablation study of key components of WENO on the Camelyon 16 dataset. We construct the WENO framework with ABMIL [13] as the teacher, where 'Distiliation' represents whether distillation used (without 'Distillation' denotes the raw ABMIL). 'Shared Encoder' represents whether the encoders of the teacher and the student share parameters, and 'HPM' indicates whether hard instance mining strategy is performed or not. The AUCs of both instance and bag classification indicate the effectiveness of each component of WENO.
9

---

Table Ablation study on the Camelyon16 Dataset.
Distillation
Shared Encoder
HPM
Instance-level AUC
Bag-level AUC
0,8480
0.8379
0.8574
/
0.901
0.8583
0.9271
0.8663
Visualization of the predictions on the Camelyon16 Dataset
shows some representative predictions the Camelyon16 dataset. As can be seen, using only slide-level labels, our method is able accurately predict almost all positive instances, regardless of whether there is large small proportion of positive regions in the WSIs. This intuitively demonstrates the powerful performance of WENO and its great potential for clinical applications.
Case 1
Case 2
Case 3
Case 4
Figure Visualization of the representative predictions on the Camelyon16 dataset, where four cases are included. For each case, the left image is the original image, where the contours of the ground truth positive regions are outlined using red lines and the right image the predicted image, where the predicted positive instances are marked using red squares and the predicted negative instances are not marked.
7 Conclusion
In this paper, propose WENO, end-to-end weakly supervised knowledge distillation frame- work for whole slide image classification. WENO a plug-and-play framework that can use any existing bag-based MIL methods using attention mechanism the teacher branch and improves its performance on both bag and instance classification. WENO trains the student classifier with the normalized attention scores of the teacher branch through knowledge distillation, and uses the hard positive instance mining (HPM) strategy to force the teach network to further mine and learn from hard positive instances. In experiments, WENO shows strong instance and bag classification performance on all five datasets, reaching new SOTA. WENO also has the potential to be applied other MIL problems. As for the limitations. the HPM strategy requires searching for optimal parameters on the validation set, which may prolong the training time. Deep learning- based WSI analysis has long history, so our study has the same potential negative societal impacts as existing studies.
Acknowledgments
This work was supported by National Natural Science Foundation of China under Grant 82072021.
10

---

References
[1] Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes Van Diest, Bram Van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen AWM Van Der Laak, Meyke Hermsen, Quirine F Manson, Maschenka Balkenhol, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22):2199-222 2017.
[2] Gabriele Campanella, Matthew G Hanna, Luke Geneslaw, Allen Miraflor, Vitor Werneck Krauss Silva, Klaus J Busam, Edi Brogi, Victor E Reuter, David S Klimstra, and Thomas J Fuchs, Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25(8):1301-1300 2019.
[3] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9650-9660,
2021.
[4] Veronika Cheplygina, Marleen de Bruijne, and Josien PW Pluim. Not-so-supervised: survey of semi-supervised, multi- instaieestanee, and transfer learning in medical image analysis. Medical Image Analysis, 54:280-299 296, 2019.
[5] Philip Chikontwe, Meejeong Kim, Soo Jeong Nam, Heounjeong Go, and Sang Hyun Park. Multiple instance learning with center embeddings for histopathology classification. In Interna- tional Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 519-528. Springer, 2020.
[6] Inseop Chung, SeongUk Park, Jangho Kim, and Nojun Kwak. Feature-map-level online adversarial knowledge distillation. In International Conference on Machine Learning (ICML), pages 2006-2015. PMLR, 2020.
[7] Feng and Zhi-Hua Zhou Deep miml network. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 31, 2017.
[8] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: survey. International Journal of Computer Vision, 129(6):178--188)) 1889, 2021.
[9] Noriaki Hashimoto, Daisuke Fukushima, Ryoichi Koga, Yusuke Takagi. Kaho Ko, Kei Kohno, Masato Nakaguro, Shigeo Nakamura, Hidekata Hontani, and Ichiro Takeuchi. Multi-scale domain-adversarial multiple-instance
for cancer subtype classification with unannotated histopathological images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3852-3861, 2020.
[10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016
[11] Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi. A comprehensive overhaul of feature distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1921-1930, 2019.
[12] Geoffrey Hinton, Oriol Vinyals. Jeff Dean, et al. Distilling the knowledge in neural network. arXiv preprint 150300231, 2015.
[13] Maximilian Ilse, Jakub Tomezak, and Max Welling. Attention-basee deep multiple instance learning. In International Conference on Machine Learning (ICMI.), pages 2127-2136. PMLR, 2018.
[14] Jakob Nikolas Kather, Cleo-Aron Weis, Francesco Bianconi, Susanne M Melchers, Lothar R Schad, Timo Gaiser, Alexander Marx, and Frank Gerrit Zöllner. Multi-class texture analysis in colorectal cancer histology. Scientific Reports, 6(1):1-11, 2016.
[15] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
11

---

[16] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems (NeurIPS), 25, 2012.
[17] Hankook Lee, Sung Ju Hwang, and Jinwoo Shin. Self-supervised label augmentation via imput transformations. In International Conference on Machine Learning (ICML), pages 5714-5724. PMLR, 2020.
[18] Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-super contrastive learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14318 14328, 2021.
[19] Hang Li, Fan Yang, Yu Zhao, Xiaohan Xing, Jun Zhang, Mingxuan Gao, Junzhou Huang. Liansheng Wang, and Jianhua Yao. Dt-mil: Deformable transformer for multi multi-instance learning on histopathological image. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 206-216. Springer, 2021.
[20] Xintong Li, Chen Li, Md Mamunur Rahaman, Hongzan Sun, Xiaoqi Li, Jian Wu, Yudong Yao, and Marcin Grzegorzek. A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification and detection approaches. Artificial Intelligence Review, pages 1-70, 2022.
[21] Yufan Liu, Jiajiong Cao, Bing Li, Chunfeng Yuan, Weiming Hu, Yangxi Li, and Yunqiang Duan. Knowledge distillation via instance relationship graph. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7096-7104, 2019.
[22] Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6):555-570, 2021.
[23] Andriy Myronenko, Ziyue Xu, Dong Yang. Holger R Roth, and Daguang Xu. Accounting for dependencies in deep learning based multiple instance learning for whole slide imaging. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 329-338. Springer, 2021.
[24] Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 3967-3976, 2019.
[25] Baoyun Peng, Xiao Jin, Jiaheng Liu, Dongsheng Li, Yichao Wu, Yu Liu, Shunfeng Zhou, and Zhaoning Zhang. Correlation congruence for knowledge distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5007-5016, 2019.
[26] Linhao Qu, Siyu Liu, Xiaoyu Liu, Manning Wang, and Zhijian Song. Towards label-efficient automatic diagnosis and analysis: comprehensive survey of advanced deep learning based weakly-supervised, sunisiupee and self superviee techniques in histopathological image analysis. Physics in Medicine & Biology, 2022.
[27] Linhao Qu, Xiaoyuan Luo, Shaolei Liu, Manning Wang, and Zhijian Song. Dgmil: Distribution guided multiple instance learning for whole slide image classification. In Medical Image Computing and Computer Assisted Intervention MICCAI 2022, pages 24-34, 2022.
[28] Jérôme Rony, Soufiane Belharbi, Jose Dolz, Ismail Ben Ayed, Luke McCaffrey, and Eric Granger. Deep weakly-supervised learning methods for classification and localization in histology images: a survey. arXiv preprint arXiv: 1XXi::1909003355 2019.
[29] Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang. Jian Zhang, Xiangyang Ji, et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Advances in Neural Information Processing Systems (NeurIPS), 34, 2021.
12

---

[30] Yash Sharma, Aman Shrivastava, Lubaina Ehsan, Christopher A Moskaluk, Sana Syed, and Donald Brown. Cluster-to to-coquer:: framework for end-to-end multi-instange learning for whole slide image classification. In Medical Imaging with Deep Learning, pages 682-698.
PMIR, 2021.
[3]] Xiaoshuang Shi, Fuyong Xing, Yuanpu Xie, Zizhao Zhang, Lei Cui, and Lin Yang. Loss-based attention for deep multiple instance learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 34, pages 5742-5749, 2020.
[32] Chetan Srinidhi, Ozan Ciga, and Anne L Martel. Deep neural network models for computa- tional histopathology: A survey. Medical Image Analysis, 67:101813, 2021.
[33] Xi Wang. Hao Chen, Caixia Gan, Huangjing Lin, Qi Dou, Efstratios Tsougenis, Qitao Huang, Muyan Cai, and Pheng-Ann Heng. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Transactions on Cybernetics, 50(9):35503396 2019.
[34] Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai, and Wenyu Liu. Revisiting multiple instance neural networks. Puttern Recognition, 74:15-24, 2018.
[35] Jiajun Wu, Yinan Yu, Chang Huang, and Kai Yu. Deep multiple instance learning for image classification and auto-annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1460-3469 2015.
[36] Ting-Bing Xu and Cheng-Lin Liu. Data-distortion guided self-distillation for deep neural networks. In Proceedings of the AAAI Conference Artificial Intelligence (AAAI), volume 33, pages 5565-5572, 2019.
[37] Yan Xu, Jun Yan Zhu, Eric Chang, and Zhuowen Tu. Multiple clustered instance learning for histopathology cancer image classification, segmentation and clustering. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 964-971. IEEE, 2012.
[38] Jiawen Yao, Xinliang Zhu, Jitendra Jonnagaddala, Nicholas Hawkins, and Junzhou Huang. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis, 65:101789, 2020.
[39] Sukmin Yun, Jongjin Park, Kimin Lee, and Jinwoo Shin. Regularizing class-wise predictions via self-knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13876-13885, 2020.
[40] Xinliang Zhu, Jiawen Yao, Feiyun Zhu, and Junzhou Huang. Wsisa: Making survival prediction from whole slide histopathological images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7234-7242, 2017.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 7
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section[] Z
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL.)? [No] Code will be publicly available.
13

---

(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.3
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4.3
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're
using/curating? [No]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [No]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "rtSO4vbbyxc", "reviewer": "Reviewer_S7zj", "summary": "The authors utilization whole slide image (WSI) classification as a classic instance and bag classification problem as most tasks based on whole slide images have slide-level (bag) classification labels and small patches which compose the slides (instances) for the the instance labels are unknown.  This formulation is necessary because WSI contain too much data to be process on a single GPU. Multiple instances learning paired with an aggregator function, like RNN, has been proven to deliver clinical level results, most classically for \"tumor/not tumor\" tasks.  The authors suggest that these method are limited in tasks where instance level classifications are dichotomized as challenging and not challenging and that certain bags with few levels of challenging instances can be false negatives.  All of this is a set up for their model which benefits from work in the field of knowledge distillation.   Knowledge distillation is a pairing of two models labeled student and teacher, where the teacher network is a MIL-type model using bags of instances with attention and the the student is an instance level classifier that uses all instances in negative bags as negatives and instances with \"positive attention\" from the teacher network labeled as positives (authors call them soft pseudo labels).  The authors also employ a \"Hard Positive Instance Mining\" HPM strategy whereby the instances where are predicted to be positive with high probability are excluded from the teacher training to make teacher task more challenging. The authors then implemented the strategy using synthetic datasets with CIFAR-10 and CRC-100K datasets with instance labels.  Camelyon16 was used as a real cancer dataset with both instance and bag level label. Finally, the model was trained and tested on real-world datasets without known instance level classifications, TCGA NSCLC dataset and and in house cervical cancer dataset.  Using both ABMIL and DSMIL, adding knowledge distillation improved overall bag level classification performance as measured by AUC. ", "questions": "This method appears to heighten the negative consequences of incorrectly labeled data (specifically true positives false labeled negative) because those instances labeled negative are going to adversely train the student network. Have you considered pairing this work with networks trained to catch this situation?", "limitations": "The major weakness of the paper is that the methods has not been used on well-benchmarked classification tasks.  For example, does this strategy actually enhance methodologies for which MIL+RNN already deliver clinical grade results or is this method only going to improve results for which the dataset is small and/or the task is not clinically significant. ", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This work is very strong because the authors have properly formulated the many challenges of whole slide image classification and incorporated emerging methods from the field of knowledge distillation/transfer learning to improve upon existing state of the art methodologies in WSI classifications.  From my own work, I have observed that attention maps derived from MIL training provide what appears to be very useful information but that information, as far as I know, has not been effectively harnessed in a practical way.  By formulating the problem with interactions between student and teacher networks, the bag level classifier can provide soft labels for the classification of tiles while the classification results from the student network can be harnessed to \"stress test\" the bag level classifier.  \n\nWhile the authors describe in detail the concept of HPM, the study and results are very weakly developed and presented.  Thus I think that perhaps including this as part of the work is a bit premature and the remaining work to my eyes, stands on its own two feet.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ZC6iayMFdq5", "reviewer": "Reviewer_hPPu", "summary": "This paper presents a novel method to train a teacher-student attention-MIL model with weak supervision in a way that the student can be used to improve the teacher. Dubbed WENO, the method shares an encoder between the teacher and student model and while the teacher's bag-level attention scores are used to train the student (distillation), the student's instance model is used to select hard examples for the teacher's bag. This is designed to avoid a common pitfall of MIL models where the teacher tends to use only easy-to-classify positive instances from the bag and becomes weak at classifying hard examples. The method can be added to existing attention-based MIL methods. The method is evaluated on 2 synthetic and 3 real-world datasets of WSI histology images, and is using 2 SOTA MIL methods (ABMIL and DSMIL). The method is shown to outperform SOTA by a significant margin on datasets where the prevalence of the positive class is low, and by a small margin for real-world datasets. An ablation study shows that that each key part of the method (distillation, shared-encoder and hard-example-selector) all participate in improving the baseline.\n", "questions": "no questions.", "limitations": "Beyond the author-stated limitation related to hyperparameter tuning, i see no further limitations of the approach.\n", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "strengths:\n- the method combines for the first time multi-instance-learning (MIL) with weakly-supervised knowledge distillation.\n- the method addresses the issue with attention-based MIL methods where the teacher's performance stops improving after a while and even decreases because it focuses more and more on a few easy-to-classify examples. The proposed HPM (hard-positive-mining) module to alleviate this issue is novel and uses the student's predictions after a set number of epochs to select hard examples for the teacher's bag.\n- the method improves upon SOTA for 3 public datasets of WSI histology. It comes ever-closer (within 3%) to fully-supervised models trained with instance-level labels (expensive to obtain).\n- the ablation study is useful and convincing.\n- the method is not specific to WSI histology images and can be applied to any MIL setting\n- the method does not affect inference complexity, only training is affected. This is a key factor in most real-world applications.\n- the paper is very well written and clarity is excellent\n\nweaknesses:\n- the achieved improvement over SOTA is relatively small for the real-world datasets.\n- hyperparameter tuning has to be performed for the HPM module to determine the best threshold. This is discussed in the supplemental material. However, another hyperparameter, the number of epochs after which to bring the HPM module online is not discussed.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DcxZ3g3FBiW", "reviewer": "Reviewer_oxi9", "summary": "This work presents a method to improve instance classification in whole slide images (WSI) using only class labels of WSI. To this end, the authors consider a knowledge distillation method where a teacher and a student model learn from each other alternatively. The teacher model is a bag (WSI) classifier while the student model is an instance classifier. The knowledge transfer between the two models is instance score attention which indicates the contribution of the instance to the bag prediction. Experimental validation was performed over 2 synthetic datasets and 3 real-world datasets for histology data, and the proposed method outperforms several SOTA methods.", "questions": "Can you please clarify what is the 'norm' in Equation 6?", "limitations": "Please refer to the weaknesses described in Strengths And Weaknesses.", "rating": 6, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": " Strengths:\n+ The paper is clearly written and organized. Their problem statement and the proposed approach are clearly described. \n+ The paper discusses an important application that is improving the instance classification in WSI data.\n+ The empirical results show the benefit of the method, and some ablation studies are provided.\n\nWeaknesses:\n - On page 2, lines 63-66, the authors mentioned that using soft labels as they did helps to alleviate the noisy pseudo-labels used in other methods. It is not clear how it is the case. From the description of the method, the pseudo-labels in this work change every SGD step since the network that builds them changes constantly. Therefore, they are noisier, especially at the beginning of the training. It is not clear why the authors claimed that their pseudo-labels alleviate the noise issue. In addition, there is no empirical evidence to support this claim.\n - On page 2, lines 70-72, the authors mentioned that sharing the feature extractor between student and teacher enhances knowledge distillation. It is not clear why it is the case.\n - In the entire paper, the pseudo-labels were described as the attention scores. It is not until eq.12 (page 6) that the authors describe the soft labels as being single scores that simulate probability. The text should be clear about this detail from the outset. In addition to fig.2,  there should be an operation after attention scores come out from the teacher, before being used by the student’s CE loss.\n - Hard positive instance mining (hpm) proposed in this paper is similar to hard mining in class-activation maps (CAMs). In the latter, strong activations are suppressed, thereby pushing the network to seek more positive regions. The same here, the authors drop high positive instances simulating suppression. Therefore, the proposed mining strategy is not really new; and literature discussion is necessary. See an example for CAMs ib Wei, Y., Feng, J., Liang, X., Cheng, M.-M., Zhao, Y., and Yan, S. (2017). Object region mining with adversarial erasing: A simple classification to semantic segmentation approach. CVPR 2017.\n- The main issue in this work is that it is unclear why there is an improvement? The authors are not using additional supervision. In fact, the pseudo labels are expected to be noisier and change constantly. It is unclear why training a student in parallel helps instance classification even though there are no instance labels. Authors are encouraged to provide further analysis to explain the observed improvement. If I had to guess, the main element that improves performance is in eq.12, in particular the zero-label of instances in negative bags. Authors explicitly provided instance labels but only for negative instances, giving their model a large advantage. Knowing what is not positive helps the model. \n- Finally, there is an important element missing in the experimental section: the classification accuracy per type of instance (positive/negative, ...). We do not know what pushed the AUC at instance-level to improve. Is it improving the positive instances or negative instances? My guess is the latter helps more since labels of negative instances are explicitly used.\n-  The code was not made available. Although the method is well described, there is a concern that the results in this paper would be difficult for a reader to reproduce.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept