Original:  Weakly Supervised Knowledge Distillation for Whole Slide Image ClassiﬁcationAnonymous Author(s)
Revised: Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classiﬁcation

Original: AfﬁliationAddress

Original: Abstract
Revised: Abstract

Original: Computer-aided pathology diagnosis based on the classiﬁcation of Whole Slide
Revised: Computer-aided pathology diagnosis based on the classiﬁcation of Whole Slide

Original: Image (WSI) plays an important role in clinical practice, and it is often formulatedas a weakly-supervised Multiple Instance Learning (MIL) problem.
Revised: Image (WSI) plays an important role in clinical practice, and it is often formulated as a weakly-supervised Multiple Instance Learning (MIL) problem.

Original: Existing methods solve this problem from either a bag classiﬁcation or an instance classiﬁcationperspective.
Revised: Existing methods solve this problem from either a bag classiﬁcation or an instance classiﬁcation perspective.

Original: In this paper, we propose an end-to-end weakly supervised knowledgedistillation framework (WENO) for WSI classiﬁcation, which integrates a bag classiﬁer and an instance classiﬁer in a knowledge distillation framework to mutuallyimprove the performance of both classiﬁers.
Revised: In this paper, we propose an end-to-end weakly supervised knowledge distillation framework (WENO) for WSI classiﬁcation, which integrates a bag classiﬁer and an instance classiﬁer in a knowledge distillation framework to mutually improve the performance of both classiﬁers.

Original: Speciﬁcally, an attention-based bagclassiﬁer is used as the teacher network, which is trained with weak bag labels,and an instance classiﬁer is used as the student network, which is trained usingthe attention scores obtained from the teacher network as soft pseudo labels forthe instances in positive bags.
Revised: Speciﬁcally, an attention-based bag classiﬁer is used as the teacher network, which is trained with weak bag labels, and an instance classiﬁer is used as the student network, which is trained using the normalized attention scores obtained from the teacher network as soft pseudo labels for the instances in positive bags.

Original: An instance feature extractor is shared between theteacher and the student to further enhance the knowledge exchange between them.
Revised: An instance feature extractor is shared between the teacher and the student to further enhance the knowledge exchange between them.

Original: In addition, we propose a hard positive instance mining strategy based on the outputof the student network to force the teacher network to keep mining hard positiveinstances.
Revised: In addition, we propose a hard positive instance mining strategy based on the output of the student network to force the teacher network to keep mining hard positive instances.

Original: WENO is a plug-and-play framework that can be easily applied to anyexisting attention-based bag classiﬁcation methods.
Revised: WENO is a plug-and-play framework that can be easily applied to any existing attention-based bag classiﬁcation methods.

Original: Extensive experiments on ﬁvedatasets demonstrate the efﬁciency of WENO.
Revised: Extensive experiments on ﬁve datasets demonstrate the efﬁciency of WENO.

Original: Code will be publicly available.
Revised: Code is available at https://github.com/miccaiif/WENO

Original:  Introduction
Revised:  Introduction

Original: Histopathological images play an important role in cancer diagnosis and prognosis prediction [23,31 , 30, 20] and they can be scanned by digital slide scanners into Whole Slide Images (WSIs) which facilitates the development of deep learning-based automatic analysis techniques.
Revised: Histopathological images play an important role in cancer diagnosis and prognosis prediction [23, 31, 30, 20], and they can be scanned by digital slide scanners into Whole Slide Images (WSIs), which facilitates the development of deep learning-based automatic analysis techniques.

Original: However,there are two challenges in deep learning-based WSI analysis.
Revised: However, there are two challenges in deep learning-based WSI analysis.

Original: First, WSIs have huge resolutions,typically reaching 100,000 × 100,000 pixels and thus cannot be directly input into deep models.
Revised: First, WSIs have huge resolutions, typically reaching 100,000 × 100,000 pixels, and thus cannot be directly input into deep models.

Original: For this reason, WSIs are usually tiled into many small patches for processing.
Revised: For this reason, WSIs are usually tiled into many small patches for processing.

Original: Second, ﬁne-grained(patch-level) annotation is very time-consuming and labor-intensive, and usually pathologists canonly provide slide-level labels, so traditional supervised learning methods cannot be directly used.
Revised: Second, ﬁne-grained (patch-level) annotation is very time-consuming and labor-intensive, and usually pathologists can only provide slide-level labels, so traditional supervised learning methods cannot be directly used.

Original: Therefore, WSI classiﬁcation is often formulated as a deep multiple instance learning (MIL) problem,which is a weakly supervised learning paradigm [30, 26, 4, 35].
Revised: Therefore, WSI classiﬁcation is often formulated as a deep multiple instance learning (MIL) problem, which is a weakly supervised learning paradigm [30, 26, 4, 35].

Original: In the MIL paradigm, each WSI is considered as a bag, and the patches cut out of it are considered asits instances.
Revised: In the MIL paradigm, each WSI is considered as a bag, and the patches cut out of it are considered as its instances.

Original: If a bag is negative, all the instances in it are negative, while if a bag is positive, at leastone positive instance exists in it.
Revised: If a bag is negative, all the instances in it are negative, while if a bag is positive, at least one positive instance exists in it.

Original: Typically, deep MIL-based WSI classiﬁcation performs two maintasks: bag classiﬁcation and instance classiﬁcation, which are used for automatic clinical diagnosisand positive region localization, respectively.
Revised: Typically, deep MIL-based WSI classiﬁcation performs two main tasks: bag classiﬁcation and instance classiﬁcation, which are used for automatic clinical diagnosis and positive region localization, respectively.

Original: Currently, deep MIL-based WSI classiﬁcation methods can be mainly classiﬁed into instance-basedapproach and bag-based approach.
Revised: Currently, deep MIL-based WSI classiﬁcation methods can be mainly classiﬁed into instance-based approach and bag-based approach.

Original: The instance-based approach [2, 5] ﬁrst trains an instance classiﬁer and then aggregates the predictions of each instance in a bag to obtain the bag prediction [7, 32, 33].
Revised: The instance-based approach [2, 5] ﬁrst trains an instance classiﬁer and then aggregates the predictions of each instance in a bag to obtain the bag prediction [7, 32, 33].

Original: Because of the lack of patch-level labels, it is not known which patches are truly positive, so instancebased approach needs to select some patches from positive slides and assign them pseudo positivelabels for training an instance classiﬁer.
Revised: Because of the lack of patch-level labels, it is not known which patches are truly positive, so instancebased approach needs to select some patches from positive slides and assign them pseudo positive labels for training an instance classiﬁer.

Original: The main problem of this approach is that the pseudoinstance labels contain a lot of noise , which limits the performance of the trained instance classiﬁer,and thus leads to inaccurate instance and bag classiﬁcation.
Revised: The main problem of this approach is that the pseudo instance labels contain a lot of noise , which limits the performance of the trained instance classiﬁer, and thus leads to inaccurate instance and bag classiﬁcation.

Original: The more common bag-based approach [13, 9, 38, 36, 29, 18, 27, 19, 28, 22] ﬁrst extracts thefeatures of each instance in a bag and aggregates the instance features to obtain the bag featureusing a trainable attention mechanism.
Revised: The more common bag-based approach [13, 9, 38, 36, 29, 18, 27, 19, 28, 22] ﬁrst extracts the features of each instance in a bag and aggregates the instance features to obtain the bag feature using a trainable attention mechanism.

Original: Then, a bag classiﬁer is trained in a supervised manner.
Revised: Then, a bag classiﬁer is trained in a supervised manner.

Original: During inference, the bag classiﬁer is used to perform bag classiﬁcation and the attention scorescan be utilized to measure the contribution of each instance to bag classiﬁcation so as to performthe instance classiﬁcation.
Revised: During inference, the bag classiﬁer is used to perform bag classiﬁcation and the attention scores can be utilized to measure the contribution of each instance to bag classiﬁcation so as to perform the instance classiﬁcation.

Original: However, this bag-based approach suffers from two serious problems: 1)
Revised: However, this bag-based approach suffers from two serious problems: 1)

Original: Poor performance of instance classiﬁcation.
Revised: Poor performance of instance classiﬁcation.

Original: Different positive instances have different levels ofdifﬁculty of being recognized.
Revised: Different positive instances have different levels of difﬁculty of being recognized.

Original: The classiﬁer is trained on bag-level loss, so it can correctly recognizea positive bag by giving high attention weights to only a few easily recognized positive instances,while the harder ones are ignored.
Revised: The classiﬁer is trained on bag-level loss, so it can correctly recognize a positive bag by giving high attention weights to only a few easily recognized positive instances, while the harder ones are ignored.

Original: This makes the network tend to accurately classify only a feweasy positive instances in the positive bags and lack the motivation to accurately classify the hardinstances [29].
Revised: This makes the network tend to accurately classify only a few easy positive instances in the positive bags and lack the motivation to accurately classify the hard instances [29].

Original: Bias of the bag classiﬁer.
Revised: Bias of the bag classiﬁer.

Original: For the same reason, the trained bag-classiﬁer will havedifﬁculty to generalize on bags that contain only hard positive instances.
Revised: For the same reason, the trained bag-classiﬁer will have difﬁculty to generalize on bags that contain only hard positive instances.

Original: To address the above issues, we propose an end-to-end we akly supervised k no wledge distillationframework ( WENO ) for whole slide image classiﬁcation.
Revised: To address the above issues, we propose an end-to-end we akly supervised k no wledge distillation framework ( WENO ) for whole slide image classiﬁcation.

Original: WENO integrates a bag classiﬁer and aninstance classiﬁer in a knowledge distillation framework to mutually improve the performance ofboth classiﬁers by effective knowledge transfer between them.
Revised: WENO integrates a bag classiﬁer and an instance classiﬁer in a knowledge distillation framework to mutually improve the performance of both classiﬁers by effective knowledge transfer between them.

Original: To the best of our knowledge, theconcept of weakly supervised knowledge distillation is proposed for the ﬁrst time.
Revised: To the best of our knowledge, the concept of weakly supervised knowledge distillation is proposed for the ﬁrst time.

Original: Figure 1 conciselyillustrates the existing knowledge distillation paradigms and the principle of WENO proposed in thispaper.
Revised: Figure 1 concisely illustrates the existing knowledge distillation paradigms and the principle of WENO proposed in this paper.

Original: Speciﬁcally, WENO contains a bag classiﬁer based on the attention mechanism as the teacherbranch and an instance classiﬁer as the student branch.
Revised: Speciﬁcally, WENO contains a bag classiﬁer based on the attention mechanism as the teacher branch and an instance classiﬁer as the student branch.

Original: To address the problem of poor instanceclassiﬁcation performance , we use the  attention scores of the teacher network as the soft pseudolabels of the instances in the positive bags to train the student branch through knowledge distillation,which alleviates the noisy pseudo label problem in previous instance-based methods.
Revised: To address the problem of poor instance classiﬁcation performance , we use the normalized attention scores of the teacher network as the soft pseudo labels of the instances in the positive bags to train the student branch through knowledge distillation, which alleviates the noisy pseudo label problem in previous instance-based methods.

Original: At the sametime, since all the instances in negative bags are negative, they are also used in training the studentbranch to further improve its instance classiﬁcation performance.
Revised: At the same time, since all the instances in negative bags are negative, they are also used in training the student branch to further improve its instance classiﬁcation performance.

Original: Notably, different from commonknowledge distillation methods in Figure 1 (a) and (b), we not only train the student branch butalso train the teacher branch using the bag labels of WSIs.
Revised: Notably, different from common knowledge distillation methods in Figure 1 (a) and (b), we not only train the student branch but also train the teacher branch using the bag labels of WSIs.

Original: Moreover, we share the instance featureextractor of the student and the teacher to further enhance the knowledge transfer between the twobranches.
Revised: Moreover, we share the instance feature extractor of the student and the teacher to further enhance the knowledge transfer between the two branches.

Original: To further address the problem of bias in the bag classiﬁer , we propose a hard positiveinstance mining (HPM) strategy.
Revised: To further address the problem of bias in the bag classiﬁer , we propose a hard positive instance mining (HPM) strategy.

Original: In particular, we use the knowledge learned by the student instanceclassiﬁer to construct bags with less easy instances, forcing the teacher network to keep mining hardpositive instances in positive bags.
Revised: In particular, we use the knowledge learned by the student instance classiﬁer to construct bags with less easy instances, forcing the teacher network to keep mining hard positive instances in positive bags.

Original: The advantages of our method are:
Revised: The advantages of our method are:

Original: • We propose a weakly supervised knowledge distillation framework, WENO, for WSI classiﬁcation.
Revised: • We propose a weakly supervised knowledge distillation framework, WENO, for WSI classiﬁcation.

Original: WENO integrates a bag classiﬁer and an instance classiﬁer in a knowledge distillation framework tomutually improve the performance of both classiﬁers by two-way knowledge transfer between them.• We further propose a hard positive instance mining (HPM) strategy to force the teacher network tocontinuously learn and mine hard positive instances in positive bags, thus alleviating the problem offocusing on easy positive instances in bag-based methods.• WENO is a plug-and-play framework that can be conveniently applied to any existing bag-basedmethods using attention mechanism to improve their performance of both instance and bag classiﬁcation. Extensive experiments on ﬁve datasets show that the current SOTA methods ABMIL [13] and DSMIL [18] achieve signiﬁcant improvements in both instance and bag classiﬁcation performancewhen they are combined with WENO.
Revised: WENO integrates a bag classiﬁer and an instance classiﬁer in a knowledge distillation framework to mutually improve the performance of both classiﬁers by two-way knowledge transfer between them.

Original: Code will be publicly available.

Original: Knowledge is distilled from the teacher to the student.
Revised: Knowledge is distilled from the teacher to the student.

Original: In recent self-knowledge distillation (b), such as DINO [3], the teacher has the same architecture with the student and it is not trainable but updated from the student.
Revised: In recent self-knowledge distillation (b), such as DINO [3], the teacher has the same architecture with the student and it is not trainable but updated from the student.

Original: Two-way knowledge transfer exists between the teacher and the student.
Revised: Two-way knowledge transfer exists between the teacher and the student.

Original: In comparison, in our proposed framework (c), the teacher is a bag classiﬁer and it is also trained with weak slide level labels
Revised: In comparison, in our proposed framework (c), the teacher is a bag-classiﬁer and it is also trained with weak slide-level labels.

Original: Knowledge is distilled from teacher to student by providing pseudo instance  attention  the  and knowledge transfer  the  instance feature 
Revised: Knowledge is distilled from teacher to student by providing pseudo instance labels using attention scores of the teacher and knowledge transfer from the student to the teacher is achieved by sharing instance feature extractors between them.

Original: Related Work
Revised: • We further propose a hard positive instance mining (HPM) strategy to force the teacher network to continuously learn and mine hard positive instances in positive bags, thus alleviating the problem of focusing on easy positive instances in bag-based methods. • WENO is a plug-and-play framework that can be conveniently applied to any existing bag-based methods using attention mechanism to improve their performance of both instance and bag classiﬁcation. Extensive experiments on ﬁve datasets show that the current SOTA methods ABMIL [13] and DSMIL [18] achieve signiﬁcant improvements in both instance and bag classiﬁcation performance when they are combined with WENO.

Original: [Figure]
Revised: Code will be publicly available.

Original:   2.1 Deep Multiple Instance Learning
Revised:  Related Work 2.1 Deep Multiple Instance Learning

Original: Instance-based methods Instance-based methods focus on training an instance classiﬁer and thenaggregating the instance predictions in a bag to make the bag prediction.
Revised: Instance-based methods Instance-based methods focus on training an instance classiﬁer and then aggregating the instance predictions in a bag to make the bag prediction.

Original: For example, Campanella etal.
Revised: For example, Campanella et al.

Original: [2] and Chikontwe et al.
Revised: [2] and Chikontwe et al.

Original: [5] iteratively trained the instance classiﬁer by selecting key instancesbased on the predicted probability of the instance classiﬁer in each iteration and assigning thempseudo labels of the corresponding bags.
Revised: [5] iteratively trained the instance classiﬁer by selecting key instances based on the predicted probability of the instance classiﬁer in each iteration and assigning them pseudo labels of the corresponding bags.

Original: After that, the former used Recurrent Neural Network(RNN) to aggregate instance predictions to perform bag classiﬁcation, and the latter proposed asoft-assignment strategy for bag inference.
Revised: After that, the former used Recurrent Neural Network (RNN) to aggregate instance predictions to perform bag classiﬁcation, and the latter proposed a soft-assignment strategy for bag inference.

Original: Bag-based methods Bag-based methods focus on aggregating instance features in a bag into a bagfeature, and then training a bag classiﬁer with bag labels.
Revised: Bag-based methods Bag-based methods focus on aggregating instance features in a bag into a bag feature, and then training a bag classiﬁer with bag labels.

Original: Most of them utilize attention mechanism toaggregate instance features, in which Ilse et al.
Revised: Most of them utilize attention mechanism to aggregate instance features, in which Ilse et al.

Original: [13] is the ﬁrst work of this kind.
Revised: [13] is the ﬁrst work of this kind.

Original: Later, Hashimoto etal.
Revised: Later, Hashimoto et al.

Original: [9] used the attention mechanism to aggregate instance features at different resolutions.
Revised: [9] used the attention mechanism to aggregate instance features at different resolutions.

Original: Yao, Zhuet al.
Revised: Yao, Zhu et al.

Original: [38, 36] proposed to ﬁrst cluster the instances in each bag and then use the attention mechanismto aggregate the features of different clusters.
Revised: [38, 36] proposed to ﬁrst cluster the instances in each bag and then use the attention mechanism to aggregate the features of different clusters.

Original: Shi et al.
Revised: Shi et al.

Original: [29] added the instance-level loss to thebag-level loss of [13], but the pseudo labels of each instance still came from its corresponding baglabel.
Revised: [29] added the instance-level loss to the bag-level loss of [13], but the pseudo labels of each instance still came from its corresponding bag label.

Original: Recently, Li et al.
Revised: Recently, Li et al.

Original: [18] proposed to use non-local attention to aggregate instance features.
Revised: [18] proposed to use non-local attention to aggregate instance features.

Original: Shaoet al.
Revised: Shao et al.

Original: [27], Li et al.
Revised: [27], Li et al.

Original: [19] proposed to use Transformer to aggregate instance features.
Revised: [19] proposed to use Transformer to aggregate instance features.

Original: Different from previous MIL methods, we construct a weakly supervised knowledge distillationframework to combine the training of a bag classiﬁer and an instance classiﬁer and utilize theknowledge transfer between the two branches of the distillation framework to mutually enhance bothclassiﬁers.
Revised: Different from previous MIL methods, we construct a weakly supervised knowledge distillation framework to combine the training of a bag classiﬁer and an instance classiﬁer and utilize the knowledge transfer between the two branches of the distillation framework to mutually enhance both classiﬁers.

Original: 2.2 Knowledge Distillation
Revised: 2.2 Knowledge Distillation

Original: Knowledge distillation is originally a method to transfer knowledge from a pre-trained complexteacher network to a simpler student network.
Revised: Knowledge distillation is originally a method to transfer knowledge from a pre-trained complex teacher network to a simpler student network.

Original: In the deployment phase, the student network canreplace the teacher network to achieve model compression [25, 24, 21, 6, 11].
Revised: In the deployment phase, the student network can replace the teacher network to achieve model compression [25, 24, 21, 6, 11].

Original: A common method ofdistilling knowledge from the teacher network to the student network is to use the output logits of theteacher network as the soft labels for training the student network [12].
Revised: A common method of distilling knowledge from the teacher network to the student network is to use the output logits of the teacher network as the soft labels for training the student network [12].

Original: Recently, self-distillation techniques are developed to train a student network without a pre-trainedteacher network [17, 34, 37, 3].
Revised: Recently, self-distillation techniques are developed to train a student network without a pre-trained teacher network [17, 34, 37, 3].

Original: Instead, the teacher usually has the same structure as the student and is updated by momentum-based moving average of the student network [3].
Revised: Instead, the teacher usually has the same structure as the student and is updated by momentum-based moving average of the student network [3].

Original: Figure 1 (a) and (b)concisely illustrates the two main existing knowledge distillation paradigms.
Revised: Figure 1 (a) and (b) concisely illustrates the two main existing knowledge distillation paradigms.

Original: Knowledge distillation is a fast-developing area and we refer to [8] for a comprehensive survey.
Revised: Knowledge distillation is a fast-developing area and we refer to [8] for a comprehensive survey.

Original: Inthis paper, we propose the concept of weakly supervised knowledge distillation for the ﬁrst time.
Revised: In this paper, we propose the concept of weakly supervised knowledge distillation for the ﬁrst time.

Original:  Method3.1 Problem Formulation3.1.1 Multiple Instance Learning (MIL)
Revised:  Method 3.1 Problem Formulation 3.1.1 Multiple Instance Learning (MIL)

Original: Given a dataset W =
Revised: Given a dataset W =

Original: { W 1 , W 2 , . . .
Revised: { W 1 , W 2 , . . .

Original: , W N } consisting of N WSIs, each WSI W i is tiled into nonoverlapping small patches { p i,j , j = 1 , 2 , . . .
Revised: , W N } consisting of N WSIs, each WSI W i is tiled into nonoverlapping small patches { p i,j , j = 1 , 2 , . . .

Original: , n i } , where n i is the number of patches cut out of W i .
Revised: , n i } , where n i is the number of patches cut out of W i .

Original: All patches p i,j in W i form a bag, where each patch is an instance.
Revised: All patches p i,j in W i form a bag, where each patch is an instance.

Original: The bag label Y i ∈ { 0 , 1 } , i ={ 1 , 2 , ...N } and the instance labels { y i,j , j = 1 , 2 , . . . , n i } have the following relationship:
Revised: The bag label Y i ∈ { 0 , 1 } , i = { 1 , 2 , ...N } and the instance labels { y i,j , j = 1 , 2 , . . . , n i } have the following relationship:

Original: [Equation]
Revised: [Equation]

Original: if [Equation] 1 , else (1) That is, all instances in negative bags are negative, while at least one positive instance exists in apositive bag.
Revised: That is, all instances in negative bags are negative, while at least one positive instance exists in a positive bag.

Original: In MIL, only the labels of the training bags are available, while the labels of the instancesin each positive bag are unknown.
Revised: In MIL, only the labels of the training bags are available, while the labels of the instances in each positive bag are unknown.

Original: Our objective is to accurately predict both the labels of each bag inthe test set (bag classiﬁcation) and the labels of each instance in them (instance classiﬁcation).
Revised: Our objective is to accurately predict both the labels of each bag in the test set (bag classiﬁcation) and the labels of each instance in them (instance classiﬁcation).

Original: 3.1.2 Bag-based MIL Methods
Revised: 3.1.2 Bag-based MIL Methods

Original: We ﬁrst brieﬂy review bag-based methods for easier understanding of our proposed framework.
Revised: We ﬁrst brieﬂy review bag-based methods for easier understanding of our proposed framework.

Original: Thesemethods ﬁrst use an encoder f to extract features z i,j for all instances { p i,j , j = 1 , 2 , . .
Revised: These methods ﬁrst use an encoder f to extract features z i,j for all instances { p i,j , j = 1 , 2 , . .

Original: . , n i } in bag
Revised: . , n i } in bag

Original: W i , and then aggregate these instance features using a permutation invariant function g to obtain thebag feature Z i :
Revised: W i , and then aggregate these instance features using a permutation invariant function g to obtain the bag feature Z i :

Original: [Equation]
Revised: [Equation]

Original: Finally, a bag classiﬁer ϕ is utilized to predict the class of the bag.
Revised: Finally, a bag classiﬁer ϕ is utilized to predict the class of the bag.

Original: [Equation]
Revised: [Equation]

Original: Traditional aggregation functions g include max-pooling and mean-pooling, while ABMIL [13]presents an attention-based trainable aggregation function:
Revised: Traditional aggregation functions g include max-pooling and mean-pooling, while ABMIL [13] presents an attention-based trainable aggregation function:

Original: [Equation]
Revised: [Equation]

Original: where a i,j is the attention score predicted by the self-attention network which is parameterized by wand V .
Revised: where a i,j is the attention score predicted by the self-attention network which is parameterized by w and V .

Original: The subsequent bag-based methods almost all adopt the attention-based aggregation methods,and the difference lies in how to construct the attention weight a i,j .
Revised: The subsequent bag-based methods almost all adopt the attention-based aggregation methods, and the difference lies in how to construct the attention weight a i,j .

Original: The weights a i,j reﬂect thecontribution of each instance in making the bag prediction, and they can be normalized as the instanceprediction for positive bags.
Revised: The weights a i,j reﬂect the contribution of each instance in making the bag prediction, and they can be normalized as the instance prediction for positive bags.

Revised: [Equation]

Revised: The normalization function can be formulated as follows:

Original: [Equation]
Revised: [Equation]

Revised: where X n orm is the normalized data, X is the original data, and X m ax and X m in are the maximum and minimum values of the original data, respectively.

Original: 3.2 Weakly Supervised Knowledge Distillation Framework
Revised: 3.2 Weakly Supervised Knowledge Distillation Framework

Revised: essentially a bag level classiﬁer, which consists of an instance encoder, a hard positive instance mining (HPM) module, an attention module and a bag prediction head.

Revised: The student branch is essentially an instance-level classiﬁer, which consists of an instance encoder and an instance prediction head.

Revised: The two branches share the same instance encoder.

Original:  3.2.1 Framework Overview
Revised: The teacher branch is trained with bag labels, while the student branch is trained with the attention scores of the teacher branch as soft pseudo labels for the instances in positive bags. Furthermore, we propose a hard positive instance mining strategy (the HPM Module) to leverage the knowledge learned by the student to help the teacher learn hard positive instances. Note that the teacher branch and the student branch are optimized alternately, where the optimization of the teacher branch is represented by update (1) in the and the optimization of the student branch is represented by update (2) in the 3.2.1 Framework Overview

Original: Figure 2 illustrates the overall framework of our proposed WENO, which contains a teacher branch(the red branch) and a student branch (the blue branch).
Revised: Figure 2 illustrates the overall framework of our proposed WENO, which contains a teacher branch (the red branch) and a student branch (the blue branch).

Original: The whole teacher branch is a bag classiﬁer, 
Revised: The whole teacher branch is a bag classiﬁer, which consists of an instance encoder, a hard positive instance mining (HPM) module, an attention module and a bag prediction head. The whole student branch is an instance classiﬁer, which consists of an instance encoder and an instance prediction head. The encoders in the two branches share the same parameters. We directly train the bag classiﬁer in the teacher branch with bag labels.

Revised: The instance classiﬁer in the student branch is trained with two sets of instances, in which the ﬁrst set is negative instances from negative bags and the second set is the instances from positive bags with pseudo labels generated according to the normalized attention scores output by the bag classiﬁer in the teacher branch.

Original: Different from the traditional distillation methods in which the teacher is pre-trained or updated bymomentum updates from the student, our teacher and student models are trained alternately, so thatnot only the teacher can transfer the knowledge learned from the bag labels to the student but alsothe knowledge learned by the student can be transferred to the teacher through the shared encoder.
Revised: Different from the traditional distillation methods in which the teacher is pre-trained or updated by momentum updates from the student, our teacher and student models are trained alternately, so that not only the teacher can transfer the knowledge learned from the bag labels to the student, but also the knowledge learned by the student can be transferred to the teacher through the shared encoder.

Original: Furthermore, we propose the hard positive instance mining strategy (the HPM Module) to help theteacher better explore hard positive instances using the knowledge learned by the student, and furtherimprove the network’s generalization ability for bag classiﬁcation.
Revised: Furthermore, we propose the hard positive instance mining strategy (the HPM Module) to help the teacher better explore hard positive instances using the knowledge learned by the student, and further improve the network’s generalization ability for bag classiﬁcation.

Original: 3.2.2 Teacher Network
Revised: 3.2.2 Teacher Network

Original: The teacher branch is a typical bag classiﬁer and one bag is input to it at a time.
Revised: The teacher branch is a typical bag classiﬁer and one bag is input to it at a time.

Original: The instance featuresz i,j are ﬁrst extracted using the encoder f t for all instances { p i,j , j = 1 , 2 , . . . , n i } within the bag
Revised: The instance features z i,j are ﬁrst extracted using the encoder f t for all instances { p i,j , j = 1 , 2 , . . . , n i } within the bag

Original: W i , and then ﬁltered by the hard positive instance mining (HPM) module (See Section 3.3 for details),and then the attention score a i,j for each instance z
Revised: W i , and then ﬁltered by the hard positive instance mining (HPM) module (See Section 3.3 for details), and then the attention score a i,j for each instance z

Original: i,j is obtained by the attention module A t .
Revised: i,j is obtained by the attention module A t .

Original: Finally,the bag feature Z i is obtained by aggregating the instance features within the bag using attentionscores, and input to the bag prediction head ϕ t to obtain the bag-level prediction (cid:98) Y i .
Revised: Finally, the bag feature Z i is obtained by aggregating the instance features within the bag using attention scores, and input to the bag prediction head ϕ t to obtain the bag-level prediction (cid:98) Y i .

Original: Since the baglabel Y i is available, the teacher branch can be trained in an end-to-end manner.
Revised: Since the bag label Y i is available, the teacher branch can be trained in an end-to-end manner.

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]

Original: Z i = n i

Original: [Equation]

Original: [Equation] [Equation] [Equation] The purpose of training the bag-level classiﬁer in the teacher branch is to obtain the attention scoresof each instance and use them as soft pseudo labels to train the instance-level classiﬁer in the studentbranch.
Revised: The purpose of training the bag-level classiﬁer in the teacher branch is to obtain the attention scores of each instance and use them as soft pseudo labels to train the instance-level classiﬁer in the student branch.

Original: Note that the teacher branch can be implemented with any existing bag-based MIL methodsusing attention mechanism, such as ABMIL [13], DSMIL [18], and TransMIL [27], etc., and wecompare the performance of using ABMIL and DSMIL as the teachers in experiments (Section 4.5and 4.6).
Revised: Note that the teacher branch can be implemented with any existing bag-based MIL methods using attention mechanism, such as ABMIL [13], DSMIL [18], and TransMIL [27], etc., and we compare the performance of using ABMIL and DSMIL as the teachers in experiments (Section 4. and 4.6).

Original: 3.2.3 Student Network
Revised: 3.2.3 Student Network

Original: The student branch is an instance-level classiﬁer which consists of an encoder f s shared with theteacher branch and an instance prediction head ϕ s .
Revised: The student branch is an instance-level classiﬁer which consists of an encoder f s shared with the teacher branch and an instance prediction head ϕ s .

Original: We train the student branch using both theinstances in positive bags with the  attention scores of the teacher classiﬁer as soft pseudo labelsand the instances in negative bags with true negative labels.
Revised: We train the student branch using both the instances in positive bags with the normalized attention scores of the teacher classiﬁer as soft pseudo labels and the instances in negative bags with true negative labels.

Original: Different from the teacher branch, theinputs to the student branch are randomly selected instances, which may come from the same bags ordifferent bags.
Revised: Different from the teacher branch, the inputs to the student branch are randomly selected instances, which may come from the same bags or different bags.

Original: The loss function of the student branch is the cross-entropy between the network prediction ˆ y i,j andthe label
Revised: The loss function of the student branch is the cross-entropy between the network prediction ˆ y i,j and the label

Original: y i,j :
Revised: y i,j :

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation] [Equation] [Equation] Since the student not only learns from the teacher’s attention scores, but also learns from the truenegative instances directly, the instance classiﬁcation ability of the student can surpass the attentionscores of the teacher, which is supported by our experimental results (Section 4.5, 4.6 and 5).
Revised: Since the student not only learns from the teacher’s attention scores, but also learns from the true negative instances directly, the instance classiﬁcation ability of the student can surpass the attention scores of the teacher, which is supported by our experimental results (Section 4.5, 4.6 and 5).

Original: Moreover, since the student and the teacher share the parameters of the instance encoders, theknowledge learned by the student can also improve the instance-level classiﬁcation ability of theteacher, which is also validated in the ablation study (Section 5).
Revised: Moreover, since the student and the teacher share the parameters of the instance encoders, the knowledge learned by the student can also improve the instance-level classiﬁcation ability of the teacher, which is also validated in the ablation study (Section 5).

Original: In inference, we use the instance classiﬁer in the student branch to make predictions for all instancesin a bag, and then use a simple max-pooling to aggregate the predictions of instances to accomplishbag prediction.
Revised: In inference, we use the instance classiﬁer in the student branch to make predictions for all instances in a bag, and then use a simple max-pooling to aggregate the predictions of instances to accomplish bag prediction.

Original: 3.3 Hard Positive Instance Mining Strategy (HPM)
Revised: 3.3 Hard Positive Instance Mining Strategy (HPM)

Original: Positive bags contain multiple positive instances, but they differ signiﬁcantly in the difﬁculty ofidentiﬁcation (e.g., in cancer detection, some patches contain a large number of cancer cells, whilesome other patches contain a small number of cancer cells).
Revised: Positive bags contain multiple positive instances, but they differ signiﬁcantly in the difﬁculty of identiﬁcation (e.g., in cancer detection, some patches contain a large number of cancer cells, while some other patches contain a small number of cancer cells).

Original: The loss of the bag-based methodsis deﬁned at the bag-level, so it only needs to identify at least one positive instance to classify thepositive bag correctly.
Revised: The loss of the bag-based methods is deﬁned at the bag-level, so it only needs to identify at least one positive instance to classify the positive bag correctly.

Original: This makes bag-level classiﬁers tend to learn only easy positive instancesbut ignore the hard ones during training, which limits both the bag and the instance classiﬁcationperformance.
Revised: This makes bag-level classiﬁers tend to learn only easy positive instances but ignore the hard ones during training, which limits both the bag and the instance classiﬁcation performance.

Original: To address this problem, we propose the hard positive instance mining strategy to forcethe teacher network to continuously learn and mine hard positive instances in the positive bags.
Revised: To address this problem, we propose the hard positive instance mining strategy to force the teacher network to continuously learn and mine hard positive instances in the positive bags.

Original: Speciﬁcally, we ﬁrst train the teacher and the student models for a certain number of epochs, afterwhich the student network has a certain instance classiﬁcation capability.
Revised: Speciﬁcally, we ﬁrst train the teacher and the student models for a certain number of epochs, after which the student network has a certain instance classiﬁcation capability.

Original: Then, before continuingto train the teacher, we use the student classiﬁer to predict all the instances in the input positivebags, and drop some instances for which the student outputs high positive prediction probability toconstruct hard pseudo bags.
Revised: Then, before continuing to train the teacher, we use the student classiﬁer to predict all the instances in the input positive bags, and drop some instances for which the student outputs high positive prediction probability to construct hard pseudo bags.

Original: In this way, we can force the bag classiﬁer to keep mining hard positiveinstances in positive bags to achieve better performance.
Revised: In this way, we can force the bag classiﬁer to keep mining hard positive instances in positive bags to achieve better performance.

Original:  Experiments and Results 
Revised:  Experiments and Results 4.1 Datasets

Original: 4.1 Datasets

Original: We used ﬁve datasets to comprehensively evaluate the performance of WENO, including two syntheticdatasets and three real-world datasets.
Revised: We used ﬁve datasets to comprehensively evaluate the performance of WENO, including two synthetic datasets and three real-world datasets.

Original: To explore the performance of WENO under different positiveinstance ratios, we used the 10-class natural image dataset CIFAR 10 [15] and the 9-class pathologicalimage dataset CRC [14] to construct synthetic WSI datasets with different positive instance ratios, andthey are denoted CIFAR-10-MIL dataset and the CRC-MIL dataset, respectively.
Revised: To explore the performance of WENO under different positive instance ratios, we used the 10-class natural image dataset CIFAR 10 [15] and the 9-class pathological image dataset CRC [14] to construct synthetic WSI datasets with different positive instance ratios, and they are denoted CIFAR-10-MIL dataset and the CRC-MIL dataset, respectively.

Original: Furthermore, weused real-world pathology datasets from three different medical centers to evaluate the performanceof WENO, including a breast cancer lymph node metastasis public dataset, the Camelyon16 dataset[1], a lung cancer diagnosis public dataset, the TCGA Lung Cancer dataset, and an in-house cervicalcancer lymph node metastasis dataset, the Clinical Cervical dataset.
Revised: Furthermore, we used real-world pathology datasets from three different medical centers to evaluate the performance of WENO, including a breast cancer lymph node metastasis public dataset, the Camelyon16 dataset [1], a lung cancer diagnosis public dataset, the TCGA Lung Cancer dataset, and an in-house cervical cancer lymph node metastasis dataset, the Clinical Cervical dataset.

Original: Detailed descriptions of thedatasets are available in the supplementary material.
Revised: Detailed descriptions of the datasets are available in the supplementary material.

Original: 4.2 Evaluation Metrics
Revised: 4.2 Evaluation Metrics

Original: For both instance and bag classiﬁcation, we use the Area Under Curve (AUC) as the evaluationmetric.
Revised: For both instance and bag classiﬁcation, we use the Area Under Curve (AUC) as the evaluation metric.

Original: We report the AUC metrics for the instance and bag classiﬁcation on two synthetic datasetsand the Camelyon 16 dataset.
Revised: We report the AUC metrics for the instance and bag classiﬁcation on two synthetic datasets and the Camelyon 16 dataset.

Original: However, since the instance-level ground truth labels for the TCGA
Revised: However, since the instance-level ground truth labels for the TCGA

Original: Lung Cancer dataset and the Clinical Cervical dataset are not available, we only report their AUC forbag classiﬁcation.
Revised: Lung Cancer dataset and the Clinical Cervical dataset are not available, we only report their AUC for bag classiﬁcation.

Original: 4.3 Implementation Details
Revised: 4.3 Implementation Details

Original: For the CIFAR-10-MIL dataset, the encoder in Figure 2 is implemented using the AlexNet [16].
Revised: For the CIFAR-10-MIL dataset, the encoder in Figure 2 is implemented using the AlexNet [16].

Original: For the other datasets, the encoder is implemented using the ResNet18 [10].
Revised: For the other datasets, the encoder is implemented using the ResNet18 [10].

Original: Both the predictionheads and the attention module are implemented using fully connected layers.
Revised: Both the prediction heads and the attention module are implemented using fully connected layers.

Original: No pre-training ofthe network parameters and no image augmentation are performed.
Revised: No pre-training of the network parameters and no image augmentation are performed.

Original: The SGD optimizer is used tooptimize the network parameters with a ﬁxed learning rate of 0.001.
Revised: The SGD optimizer is used to optimize the network parameters with a ﬁxed learning rate of 0.001.

Original: For the hard positive instancemining strategy, we drop the instances with positive probability higher than a threshold in positivebags.
Revised: For the hard positive instance mining strategy, we drop the instances with positive probability higher than a threshold in positive bags.

Original: The hyperparameter thresholds vary for each dataset, and we used grid search on the validationset to determine the optimal values.
Revised: The hyperparameter thresholds vary for each dataset, and we used grid search on the validation set to determine the optimal values.

Original: In the supplementary material, we give a robustness study of thethreshold on the Camelyon 16 dataset.
Revised: In the supplementary material, we give a robustness study of the threshold on the Camelyon 16 dataset.

Original: All experiments were performed using 4 Nvidia 3090 GPUs.
Revised: All experiments were performed using 4 Nvidia 3090 GPUs.

Original: 4.4 Comparison Methods
Revised: 4.4 Comparison Methods

Original: We compare WENO with a series of state-of-the-art methods.
Revised: We compare WENO with a series of state-of-the-art methods.

Original: For both the synthetic datasets andthe real-world datasets, we use the classic ABMIL [13] and the latest DSMIL [18] as the teachersto construct the WENO frameworks and compare their instance and bag classiﬁcation performancewith SOTA methods.
Revised: For both the synthetic datasets and the real-world datasets, we use the classic ABMIL [13] and the latest DSMIL [18] as the teachers to construct the WENO frameworks and compare their instance and bag classiﬁcation performance with SOTA methods.

Original: The comparison methods include instance-based methods: RNN-MIL [2] and
Revised: The comparison methods include instance-based methods: RNN-MIL [2] and

Original: Chi-MIL [5]; bag-based methods: ABMIL [13], Loss-ABMIL [29] and DSMIL [18].
Revised: Chi-MIL [5]; bag-based methods: ABMIL [13], Loss-ABMIL [29] and DSMIL [18].

Original: We reproducedthese methods based on the published codes, and the speciﬁc parameter settings are provided in thesupplementary material.
Revised: We reproduced these methods based on the published codes, and the speciﬁc parameter settings are provided in the supplementary material.

Original: We also compare the results of using the fully supervised approaches on thesynthetic datasets and the Camelyon 16 dataset, i.e., performing supervised training using the truelabels of each instance and aggregating the instance predictions using max-pooling to obtain the bagpredictions.
Revised: We also compare the results of using the fully supervised approaches on the synthetic datasets and the Camelyon 16 dataset, i.e., performing supervised training using the true labels of each instance and aggregating the instance predictions using max-pooling to obtain the bag predictions.

Original: 4.5 Results on Synthetic Datasets
Revised: 4.5 Results on Synthetic Datasets

Original: Table 1 and Table 2 show the instance and bag classiﬁcation performance of WENO on the CIFAR-10MIL dataset and the CRC-MIL dataset with different positive instance ratios, respectively.
Revised: Table 1 and Table 2 show the instance and bag classiﬁcation performance of WENO on the CIFAR-10MIL dataset and the CRC-MIL dataset with different positive instance ratios, respectively.

Original: We use theclassic ABMIL [13] and the latest DSMIL [18] as the teachers to construct the WENO frameworks.
Revised: We use the classic ABMIL [13] and the latest DSMIL [18] as the teachers to construct the WENO frameworks.

Original: It can be seen that the WENO framework signiﬁcantly improves the performance of the two originalbag-based methods for both the bag classiﬁcation and the instance classiﬁcation tasks under allpositive instance ratios.
Revised: It can be seen that the WENO framework signiﬁcantly improves the performance of the two original bag-based methods for both the bag classiﬁcation and the instance classiﬁcation tasks under all positive instance ratios.

Original: The advantage of WENO is especially signiﬁcant for instance classiﬁcation.
Revised: The advantage of WENO is especially signiﬁcant for instance classiﬁcation.

Original: In particular, DSMIL [18] does not work well in instance classiﬁcation under low positive instanceratios, while the performance of DSMIL+WENO is much higher.
Revised: In particular, DSMIL [18] does not work well in instance classiﬁcation under low positive instance ratios, while the performance of DSMIL+WENO is much higher.

Original: For bag classiﬁcation, as shownin Table 1 (b), DSMIL [18] does not work at positive instance ratios of 5% and 10%, while the bag classiﬁcation AUC after combining the WENO framework reaches 0.9367 and 0.9900, respectively.
Revised: For bag classiﬁcation, as shown in Table 1 (b), DSMIL [18] does not work at positive instance ratios of 5% and 10%, while the bag classiﬁcation AUC after combining the WENO framework reaches 0.9367 and 0.9900, respectively.

Original: These results show the powerful advantages of WENO: signiﬁcant performance gains and easyplug-and-play ability.
Revised: These results show the powerful advantages of WENO: signiﬁcant performance gains and easy plug-and-play ability.

Original: [Table]
Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Table]

Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table] [Figure]

Original: 4.6 Results on Real-World Datasets
Revised: 4.6 Results on Real-World Datasets

Original: Table 3 (a) shows the performance of WENO on the Camelyon16 dataset for the instance and bagclassiﬁcation.
Revised: Table 3 (a) shows the performance of WENO on the Camelyon16 dataset for the instance and bag classiﬁcation.

Original: We still use ABMIL [13] and DSMIL [18] as the teachers to construct the WENOframeworks.
Revised: We still use ABMIL [13] and DSMIL [18] as the teachers to construct the WENO frameworks.

Original: It can be seen that using WENO brings a signiﬁcant performance improvement inboth classiﬁcation tasks.
Revised: It can be seen that using WENO brings a signiﬁcant performance improvement in both classiﬁcation tasks.

Original: Notably, with only bag-level weak labels, the best instance classiﬁcation
Revised: Notably, with only bag-level weak labels, the best instance classiﬁcation

Original: AUC (0.9377) obtained with WENO is lower than that of fully supervised method (0.9644) by only0.0267; while the best bag classiﬁcation AUC (0.8663) exceeds the fully supervised (0.8621) method,showing the powerful performance of WENO again.
Revised: AUC (0.9377) obtained with WENO is lower than that of fully supervised method (0.9644) by only 0.0267; while the best bag classiﬁcation AUC (0.8663) exceeds the fully supervised (0.8621) method, showing the powerful performance of WENO again.

Original: Table 3 (b) shows the bag classiﬁcation performance of WENO on the TCGA Lung Cancer dataset.
Revised: Table 3 (b) shows the bag classiﬁcation performance of WENO on the TCGA Lung Cancer dataset.

Original: We use ABMIL [13] and DSMIL [18] as the teachers to construct the WENO frameworks and achievethe best performance.
Revised: We use ABMIL [13] and DSMIL [18] as the teachers to construct the WENO frameworks and achieve the best performance.

Original: Table 3 (c) shows the bag classiﬁcation performance of WENO on the Clinical Cervical dataset.
Revised: Table 3 (c) shows the bag classiﬁcation performance of WENO on the Clinical Cervical dataset.

Original: Weuse ABMIL [13] and DSMIL [18] as the teachers to construct the WENO frameworks and achievethe optimal performance.
Revised: We use ABMIL [13] and DSMIL [18] as the teachers to construct the WENO frameworks and achieve the optimal performance.

Original: In contrast to previous clinical tumor recognition and classiﬁcation tasks,the prediction of lymph node metastasis according to the WSIs of primary lesion in this dataset is avery challenging task in current clinical practice.
Revised: In contrast to previous clinical tumor recognition and classiﬁcation tasks, the prediction of lymph node metastasis according to the WSIs of primary lesion in this dataset is a very challenging task in current clinical practice.

Original: Since there is no prior knowledge about what imagefeatures of the primary lesion indicate metastasis, even very experienced pathologists are unable toclearly distinguish between positive and negative instances.
Revised: Since there is no prior knowledge about what image features of the primary lesion indicate metastasis, even very experienced pathologists are unable to clearly distinguish between positive and negative instances.

Original: WENO achieves the best performanceamong all comparison methods, which on the one hand indicates that WENO can be applied to theprediction of metastasis using primary lesion WSIs in clinical practice, and on the other hand suggeststhe potential of WENO to detect underlying pathological patterns from high conﬁdent instances.
Revised: WENO achieves the best performance among all comparison methods, which on the one hand indicates that WENO can be applied to the prediction of metastasis using primary lesion WSIs in clinical practice, and on the other hand suggests the potential of WENO to detect underlying pathological patterns from high conﬁdent instances.

Original: Some examples of positive and negative slides/patches in the three real-world datasets and thevisualization of the heatmaps on the Camelyon16 dataset are given in the supplementary material. 
Revised: Some examples of positive and negative slides/patches in the three real-world datasets and the visualization of the heatmaps on the Camelyon16 dataset are given in the supplementary material. [Table] [Figure] [Table]

Revised: [Figure]

Original: Ablation Study
Revised: [Table] [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original:  Figure 3 shows the results of the ablation study of WENO on the CIFAR-10-MIL dataset with apositive instance ratio of 0.2.
Revised: [Table] [Figure]  Ablation Study Figure 3 shows the results of the ablation study of WENO on the CIFAR-10-MIL dataset with a positive instance ratio of 0.2.

Original: In this experiment, we use the ABMIL [13] as the teacher to constructthe WENO framework.
Revised: In this experiment, we use the ABMIL [13] as the teacher to construct the WENO framework.

Original: By analyzing the curves in Figure 3, we can ﬁnd: (1) Comparing the curvesof the raw ’ABMIL’ and ’ABMIL+WENO (without HPM)’ in panel (a) and panel (b), it can be seenthat when the bag-level AUC reaches the maximum at about the 25th epoch, their instance-level[Table] classiﬁcation ability is still poor.
Revised: By analyzing the curves in Figure 3, we can ﬁnd: (1) Comparing the curves of the raw ’ABMIL’ and ’ABMIL+WENO (without HPM)’ in panel (a) and panel (b), it can be seen that when the bag-level AUC reaches the maximum at about the 25th epoch, their instance-level classiﬁcation ability is still poor.

Original: In addition, the instance-level AUC of the raw ABMIL gets worseas the training proceeds, indicating that the network gradually tends to distinguish positive bagsby simple positive instances only.
Revised: In addition, the instance-level AUC of the raw ABMIL gets worse as the training proceeds, indicating that the network gradually tends to distinguish positive bags by simple positive instances only.

Original: As shown in Figure 3 (b), when WENO is used with ABMIL,the instance-level classiﬁcation capability is signiﬁcantly improved, and the network is still able tocontinuously improve the instance classiﬁcation capability as the training proceeds.
Revised: As shown in Figure 3 (b), when WENO is used with ABMIL, the instance-level classiﬁcation capability is signiﬁcantly improved, and the network is still able to continuously improve the instance classiﬁcation capability as the training proceeds.

Original: Comparingthe curves of the ’ABMIL+WENO (without HPM)’ and ’ABMIL+WENO (with HPM)’ in panel(b) and panel (c), it can be seen that the proposed hard positive instance mining strategy can furtherimprove the instance classiﬁcation ability of both the student and the teacher by forcing the teachernetwork to continuously learn and mine hard positive instances in positive bags.
Revised: Comparing the curves of the ’ABMIL+WENO (without HPM)’ and ’ABMIL+WENO (with HPM)’ in panel (b) and panel (c), it can be seen that the proposed hard positive instance mining strategy can further improve the instance classiﬁcation ability of both the student and the teacher by forcing the teacher network to continuously learn and mine hard positive instances in positive bags.

Original: HPM is used afterthe 150th epoch.
Revised: HPM is used after the 150th epoch.

Original: [Figure]
Revised: [Figure]

Original: 308 Table 4 shows the results of the ablation study of key components of WENO on the Camelyon16dataset.
Revised: Table 4 shows the results of the ablation study of key components of WENO on the Camelyon dataset.

Original: We construct the WENO framework with ABMIL [13] as the teacher, where ’ Distillation ’represents whether distillation is used (without ’Distillation’ denotes the raw ABMIL), ’ Shared
Revised: We construct the WENO framework with ABMIL [13] as the teacher, where ’ Distillation ’ represents whether distillation is used (without ’Distillation’ denotes the raw ABMIL), ’ Shared

Original: Encoder ’ represents whether the encoders of the teacher and the student share parameters, and’ HPM ’ indicates whether hard instance mining strategy is performed or not.
Revised: Encoder ’ represents whether the encoders of the teacher and the student share parameters, and ’ HPM ’ indicates whether hard instance mining strategy is performed or not.

Original: The AUCs of bothinstance and bag classiﬁcation indicate the effectiveness of each component of WENO.
Revised: The AUCs of both instance and bag classiﬁcation indicate the effectiveness of each component of WENO.

Original: [Figure]
Revised: [Table]

Original:  Conclusion
Revised:  Visualization of the predictions on the Camelyon16 Dataset

Revised: Figure 4 shows some representative predictions on the Camelyon16 dataset. As can be seen, using only slide-level labels, our method is able to accurately predict almost all positive instances, regardless of whether there is a large or small proportion of positive regions in the WSIs. This intuitively demonstrates the powerful performance of WENO and its great potential for clinical applications.

Original: [Figure]
Revised: [Figure]

Original: In this paper, we propose WENO, an end-to-end weakly supervised knowledge distillation framework for whole slide image classiﬁcation.
Revised: In this paper, we propose WENO, an end-to-end weakly supervised knowledge distillation framework for whole slide image classiﬁcation.

Original: WENO is a plug-and-play framework that can use anyexisting bag-based MIL methods using attention mechanism as the teacher branch and improvesits performance on both bag and instance classiﬁcation.
Revised: WENO is a plug-and-play framework that can use any existing bag-based MIL methods using attention mechanism as the teacher branch and improves its performance on both bag and instance classiﬁcation.

Original: WENO trains the student classiﬁer withthe attention scores of the teacher branch through knowledge distillation, and uses the hard positiveinstance mining (HPM) strategy to force the teach network to further mine and learn from hard positive instances
Revised: WENO trains the student classiﬁer with the normalized attention scores of the teacher branch through knowledge distillation, and uses the hard positive instance mining (HPM) strategy to force the teach network to further mine and learn from hard positive instances.

Original: In experiments, WENO shows strong instance and bag classiﬁcation performance on all ﬁve datasets, reaching new SOTA.
Revised: In experiments, WENO shows strong instance and bag classiﬁcation performance on all ﬁve datasets, reaching new SOTA.

Original: WENO also has the potential to be applied to other MIL problems
Revised: WENO also has the potential to be applied to other MIL problems.

Original: As for the limitations, the HPM strategy requires searching for optimal parameters on the validation set , which may prolong the training time
Revised: As for the limitations, the HPM strategy requires searching for optimal parameters on the validation set, which may prolong the training time.

Original: Deep learning based WSI analysis has a long history, so our study has the same potential negative societal impacts as existing studies
Revised: Deep learning-based WSI analysis has a long history, so our study has the same potential negative societal impacts as existing studies.

Original: Checklist
Revised: Acknowledgments

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope? [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 6.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeSection 6.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [No] Code will be

Original: publicly available.

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [Yes] See Section 4.3.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?

Original: [No]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.3.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Section 4.1.

Original: (b) Did

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]

Original: [Table]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: [Table]

Original: using/curating?

Original: [No]

Original: [Table]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: [Table]

Original: information or offensive content?

Original: [No] [Table] If you used crowdsourcing or conducted research with human subjects... [Table]
Revised: This work was supported by National Natural Science Foundation of China under Grant 82072021.

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: [Table]

Original: applicable?

Original: [N/A]

Original: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: [Table]

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: [Table]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: [Table]

Original: spent on participant compensation?

Original: [N/A]

Original: [Table]