MHKD: Multi-step Hybrid Knowledge Distillation for Low-resolution Whole Slide Images Glomerulus Detection

Authors: xiangsen  zhang
longfei  han
chenchu  xu
zhaohui  zheng
jin  ding
xianghui  fu
dingwen  zhang
junwei  han
Venue: IEEE_org_EMBS_BHI_2024_Conference
Published: 2024-06-28 17:54:47

Abstract: Glomerulus detection is a critical component of renal histopathology assessment, essential for diagnosing glomerulonephritis. 
To mitigate the increasing workload on pathologists, AI-assisted diagnostic methods based on high-resolution digital pathology whole slide images have been developed. However, these current AI-assisted approaches are limited to high-resolution whole slide images, necessitating expensive digital scanner equipment, high image storage costs, and significant computational complexity.
To address this limitation, this paper pioneers a method for facilitating glomerulus detection in low-resolution human kidney pathology images.
Specifically, we propose a novel multi-step hybrid knowledge distillation method. 
Our method distills both the global features and the semantic information through a hybrid knowledge distillation strategy that integrates offline and online knowledge distillation, where the information from high-resolution pathological images is successively transferred to student model from the global features in the shallow network layers to the semantic information of the back-end through a multi-step training strategy.
Experimental results on two datasets show that the proposed method achieves effective detection outcomes for low-resolution kidney pathology images. Compared to other state-of-the-art detection techniques, our method achieves an $AP_{0.5:0.95}$ improvement of 23.1\% on the private LN dataset and 15.9\% on the public HUBMAP dataset.

Main paper: 
 i. introduction: 
 Glomerulonephritis is a common and potentially lifethreatening kidney disease primarily affecting the structure and function of the glomeruli. Pathological diagnosis is the gold standard for the diagnosis of glomerulonephritis, enabling the detection of diseased glomeruli and the quantitative assessment of nephritis severity. However, this diagnostic process Examples of different resolutions in the same area. The green box represents the glomerular region and the yellow box is the non-glomerular region that appears similar to a glomerulus. In low-resolution pathological images, the boundary of glomeruli in the green box area is not obvious, and the yellow box area is easily misjudged as a glomerulus.necessitates highly skilled pathologists and is labor-intensive, failing to meet the current demands. In recent years, AIassisted diagnosis has been integrated into the detection of pathological images, providing a potential solution [5] , [11] , [12] to reduce the burden on pathologists who must continuously analyze large batches of images. Various methods for glomerulus detection [2] , [3] , [4] , [30] , [31] have demonstrated commendable diagnostic performance.However, the current methods have limited application scenarios. These approaches are developed based on highresolution pathological images, necessitating costly scanners and demanding scanning accuracy. Consequently, they are only suitable for relatively advanced regions rather than less developed areas and countries. According to the survey, the price of automatic digital scanner equipment used to acquire highresolution pathology images is generally 20 to 40 times higher than the price of equipment used to acquire low-resolution pathology images, or even more. Additionally, the time and storage costs associated with obtaining high-resolution pathological images are significant. Under these circumstances, there is a great demand for pathological diagnosis based on low-resolution pathological images.Unfortunately, there are great challenges in performing glomerulus detection based on low-resolution digital images. Sensitivity to subtle structural changes and color differences in tissue is a key indicator of a pathologist's expertise and is crucial for disease diagnosis. However, such cues are often lost in low-resolution pathological images. As illustrated in Fig. 1 , the comparison between high-resolution and lowresolution kidney pathology images reveals that glomeruli, marked by green boxes, have blurred boundaries in lowresolution images, making them easily overlooked. Additionally, non-glomerular tissues, marked by yellow boxes, are often confused with glomeruli in low-resolution images due to the lack of discernible details. The nuclei of mesangial cells and the thickness of capillaries are critical indicators for assessing abnormalities, yet this information is difficult to discern in low-resolution images. Due to the aforementioned issues, applying low-resolution whole slide images (WSIs) directly to current detection methods [7] , [8] , [9] , [10] significantly degrades detection performance compared to high-resolution WSIs.To address this problem, this paper proposes a novel multistep hybrid knowledge distillation (MHKD) method. The core concept is to employ knowledge distillation to enhance the student model's capability to extract features from low-resolution pathological images by transferring information from highresolution pathological images in the teacher model to the student model. This knowledge distillation process compensates for the missing details in low-resolution pathological images, resulting in detection performance that closely approximates that of high-resolution images. However, due to the specific nature of pathological input images, conventional knowledge distillation methods do not achieve optimal results. Firstly, the high-resolution pathological image information in the teacher model is crucial for the distillation task, so we need to ensure the extraction and transfer of high-resolution information at the forward end, and on this basis to achieve efficient information transmission at other locations. In addition, different from natural images, the task-related features and discriminative small details of kidney pathological images are densely concentrated in small local areas, and there is a large information gap between pathological images of different resolutions. It is difficult to ensure the effectiveness of critical knowledge transfer by distillation of multiple positions of the model at the same time.Firstly, hybrid knowledge distillation integrates offline distillation and online distillation to optimize the distillation performance. Typically, online distillation surpasses offline distillation, but direct application in our specific task is not suitable. We discovered that training solely with online distillation would compromise the feature extraction capability of the teacher model due to difference in input pathological image resolutions. To address this issue, we exclusively employ offline distillation in the backbone part of the teacher model.Secondly, a multi-step training strategy is devised for the distillation task involving varying resolutions of input pathological images. We divide distillation locations into two main groups: global features at the shallow network layers and semantic information at the back-end. Semantic information includes local features and output. The Basic Feature Adaptive module has been designed to enhance the adaptability of basic global features, enabling them to assimilate knowledge effectively from the teacher network. To sequentially transfer information from high-resolution pathological images through basic features, adaptive features, local features, and semantic information to the student model, we adopt a multi-step training approach.Overall, our contributions are summarized as follows:• To the best of our knowledge, we are the first to propose the task of glomerulus detection on low-resolution human kidney pathology images and propose a novel multi-step hybrid knowledge distillation method for this task. In recent years, the advancement of computer technology has led to an increasing number of studies applying computer technology for the analysis and processing of digital kidney images. In the diagnosis of renal disease digital images, according to the source of the image, the diagnostic methods can be divided into US-based methods, such as [26] , CT-based methods, such as [18] , MRI-based methods, such as [27] , and WSI-based methods, such as [4] . The WSI-based approach is the one to watch due to renal pathology being considered the gold standard for diagnosing renal diseases. Evidence suggests that patients who undergo renal biopsy have higher kidney survival rates compared to those who do not [1] .There are two ways for glomerulus detection of a single WSI in the WSI-based approach: one is the slider method. Gal lego et al. [3] proposed a method that slices WSIs into patches using a fixed-size sliding window approach followed by patch classification into glomeruli or background using CNNs. The classifiers in it can be changed according to the task. The second is localization before classification. Zheng et al. [4] divided their approach for detecting glomeruli into two stages: initial localization on low-resolution WSIs followed by interception of high-resolution local areas based on this localization for subsequent classification. In addition, multimodal combination of the above methods is also helpful for glomerulus detection, such as WSIs with multiple stains. Yoshimasa Kawazoe et al. [2] employed different dyes for WSI staining and used a sliding window technique for WSI detection.However, these methods all require high-resolution WSIs. Therefore, we propose a novel task of glomerulus detection based on low-resolution human kidney digital pathology WSIs to reduce the dependence on medical resources.Knowledge distillation, a method often used for model compression, is an effective technique for transferring knowledge from large-scale teacher models to small-scale student models. At present, most knowledge distillation is applied to classification tasks [21] , [22] . It was first proposed by Hinton et al. [21] , which uses the output as a soft label to transfer the dark knowledge from a large teacher network to a small student network for classification. Recently, several works have successfully applied knowledge distillation to object detection tasks. Chen et al. [23] was the first to apply knowledge distillation to object detection through the combination of feature and prediction. Subsequently, regionspecific feature distillation has been proposed, such as Li et al. [24] proposed a scheme to imitate features within regions in Faster R-CNN. In addition, with the proposal of Deep Mutual Learning (DML) [25] and [29] , distillation can be divided into offline distillation and online distillation according to whether the teacher model updates parameters during the training process. The main advantage of the offline distillation [14] , [15] is its simplicity and ease of implementation. however, the disadvantage is that complex high-capacity teacher models and huge training time cannot be avoided. In online distillation [16] , [17] , both the teacher model and the student model are updated simultaneously. However, existing online approaches usually fail to address the problem of high-ability teachers in online Settings. In this paper, we adopt a distillation strategy combining the two.The object detection algorithm for pathological images employs a two-stage approach, where proposals are extracted from the feature maps to acquire regions of interest (ROIs), followed by classification and regression on each ROI. Consequently, the overall loss function for object detection can be formulated as follows.EQUATIONwhere L cls is all classification loss, L reg is all regression loss. Considering the strategy of knowledge distillation, the range of L detect needs to be further determined. If the strategy is offline distillation, L detect only contains the loss of student model. If the strategy is online distillation, L detect contains both the loss of student model and teacher model.To the best of our knowledge, no prior method has attempted knowledge distillation for object detection in pathological images with varying resolutions. Existing approaches to knowledge distillation in object detection tasks typically focus on selecting information from two locations: features and outputs. Hence, the formulation of the knowledge distillation loss function can be expressed as follows.EQUATIONwhere L fea is the feature distillation loss of different regions or layers and L out is the output distillation loss. The feature distillation can be further categorized based on the location of features. Additionally, L out , also known as the soft loss, is the loss computed between the output of the teacher model and the output of the student model. The overall training loss comprises the detection loss L detect and the knowledge distillation loss L KD , thus can be formulated as follows:EQUATIONwhere λ KD is a weight coefficient for knowledge distillation.In our method, we select information for distillation in two locations: global feature (basic feature and adaptive feature), semantic information (local feature and output).The proposed hybrid knowledge distillation method integrates both offline and online distillation strategies. With different resolutions of the input pathological images, hybrid knowledge distillation not only can maintain that the feature extractor in the input end of the teacher model can efficiently extract the features of high-resolution pathological images, but also ensure that the semantic information in the output end gets real-time guidance from the teacher model, so as to shorten the gap between the student model and the teacher model. As show in Fig. 2 , our proposed method is divided into two parts: Global Feature Offline Distillation for the first half and Semantic Information Online Distillation for the second half. As depicted in Fig. 2 , our method employs offline distillation on the global feature within the global feature offline distillation module. The global feature represents comprehensive information extracted by the feature extractor from the input image, serveing as a fundamental component for all tasks. Hence, we perform offline knowledge distillation ensure that the feature extraction capability of the already trained teacher model remains unaffected by the student model. By mimicking the teacher model's global features f H , our approach enables the student model to obtain the basic feature f L that is closer to the teacher model. Both the basic feature and the adaptive feature are global features of student model, and the adaptive feature will be explained in the next section. Although online distillation typically yields better results with identical inputs, it is unsuitable for our task at this time due to low-resolution student inputs that would degrade teacher performance. Consequently, we propose hybrid knowledge distillation which incorporates offline distillation specifically for global feature distillation. To elaborate further, we feed the teacher input I H into the frozen backbone of the teacher model to extract high-quality global features f H ∈ R w×h×d . Simultaneously, we pass an up-sampled version of student input I L through the student model's backbone to obtain corresponding global features f L ∈ R w×h×d . To guide pixel-level mimicry between student's global feature f L and its approximation towards f H , we employ the mean square error (MSE) loss as our chosen distillation loss function.EQUATION2) Semantic Information Online Distillation: In the semantic information online distillation phase, we optimize the local features based on the proposals provided by RPN and optimize the final output. The local features, referred to as ROI features, represent the suspected target regions. Specifically, both teacher and student models predict proposals: P roposal H and P roposal L , respectively. The teacher model generates a series of ROI features f H,H using f H and P roposal H , which are then used to predict O T . Similarly, the student model generates a series of ROI features f L,L using f L and P roposal L , which are utilized for predicting O S . To ensure that the local features of the student model imitate the target features within the same regions, we employ identical coordinates for region selection. The teacher model generates a series of f H,L based on f H and P roposal L , which represent the proposals from the student model. Subsequently, pixel-wise MSE loss is utilized to compute the losses between f H,L and f L,L .EQUATIONwhere M is the number of pixels in each ROI feature.In addition to local feature distillation, we also introduce distillation to guide the student model to produce more accurate predictions at the final prediction output. In this method, the teacher's output O T is transformed into the same data format as the ground truth (GT), replacing the role of GT in normal training. This enables the student model to learn from the teacher's directly. In the semantic online distillation part, we introduce two distillations, one for local features and another for the output. These locations contain valuable semantic information that is closely related to the final prediction. By adopting the online distillation strategy close to the output, the teacher model can better explore the relationship between teachers and students, realize realtime guidance, effectively reduce the gap between the teacher and student models, and improve the performance of the student model. Consequently, our proposed hybrid knowledge distillation method seamlessly integrates offline and online approaches while incorporating global feature extraction, local feature extraction, and output-based distillations.To improve the distillation effect, we divide the training into four steps: basic feature offline distillation, adaptive feature offline distillation, local feature online distillation, and sematic information online distillation. The reason is that progressive distillation training can effectively ensure that the high-resolution pathological image information in the teacher model can be gradually transferred to the student model.1) Concrete Steps: The hybrid knowledge distillation method we mentioned earlier divides the whole framework into two parts. In the global feature offline distillation part, the performance of the teacher model is not degraded. The semantic online distillation part can effectively reduce the gap between the teacher model and the student model.However, in the global feature offline distillation part, there is still a certain gap between global feature f L and f H after direct distillation due to significant information gaps. Therefore, we introduce adaptive feature offline distillation to divide the global feature offline distillation into two steps: basic feature offline distillation and adaptive feature offline distillation. Specifically, we distilled the basic feature f L and then performed adaptive transformation to obtain f ′ L , and then distilled it again. Consequently, the global features of the student model are adaptively enhanced and the information transfer from the teacher model is obtained twice respectively. The global features obtained by the student model exhibit a higher degree of proximity to those acquired by the teacher model.Specifically, in basic feature offline distillation, the teacher model is frozen and the student model is trained, and the loss includes the detection loss and the global feature distillation loss. In adaptive feature offline distillation, the teacher model is frozen and the student model is trained, and the loss includes detection loss and global feature distillation loss. A new module, Basic Feature Adaptive, is introduced to generate adaptive global features in student model. In semantic feature online distillation, the feature extractor of the teacher model is frozen, the rest of the teacher model is trained, and the student model is trained. the loss includes detection loss and local feature distillation loss. In semantic information online distillation, the feature extractor of the teacher model is frozen, the rest of the teacher model is trained, and the student model is trained. And the loss includes detection loss, local feature distillation loss, and output loss.2) Basic Feature Adaptive Module: To reduce the gap between the global features of the student model and the global features of the teacher model, we introduce the Basic Feature Adaptive (BFA) module after the extracted global features f L to obtain better global features. Considering that the discrepancy between the teacher and student models lies in their input image resolutions, our BFA module is modified based on SRResnet [28] , with adjustments made to parameters such as channel numbers and scaling factors.The SRResNet model is a deep learning architecture designed for image super-resolution tasks. It utilizes residual connections and convolutional neural networks (CNN) to achieve high-quality upscaling of low-resolution images, enhancing image details and clarity. By employing sub-pixel convolution, the model effectively addresses the challenge of increasing image size without compromising pixel density. Therefore, as a feature adaptive module based on this modification, we can better enrich the global feature of lowresolution images.To facilitate the exploration of the role of knowledge distillation for detection tasks on low-resolution pathological images, the dataset is downsampled so that WSIs can be directly used as input to the network in this paper.1) LN Dataset: The LN dataset contains 349 kidney biopsy WSIs from 163 patients, collected between 2011 and 2019, with varied ages, genders, and degrees of lupus nephritis. This dataset is focused on lupus nephritis and can be utilized for glomerulus detection and the diagnosis of lupus nephritis. Glomeruli were classified into five categories: 'Slight', 'Severe', 'Fibrosis', 'Incomplete', and 'Uncertain'.The data were obtained from the Chinese Systemic Lupus Erythematosus (SLE) Cohort database of Xijing Hospital and were approved by the Institutional Review Board of the Ethics Committee of Xijing Hospital (KY20223382-1) for this study.2) HuBMAP Dataset: The HuBMAP dataset [32] is a public dataset. The HuBMAP data used in this hackathon includes 11 fresh frozen and 9 Formalin Fixed Paraffin Embedded (FFPE) PAS kidney images. Glomeruli segmentation annotations exist for 15 tissue samples. Each sample is large in size and includes a substantial number of glomeruli. In this paper we divided it into 171 pathological digital imagesd for one-class glomerular detection only.We utilized a Linux server equipped with a single NVIDIA GeForce RTX 3090 to construct the trained model for glomerulus detection. The high-resolution branch, serving as the teacher model, was pre-trained and its backbone was frozen. Conversely, the low-resolution branch, acting as the student model, was not pre-trained. In order to facilitate the exploration of ways to improve the input of low-resolution images, we down sampled the input images. For LN dataset, we set the input size of the high-resolution branch to 768 × 2304 pixels while maintaining a resolution of 256 × 768 pixels for the low-resolution branch. For HUBMAP dataset, we set the input size of the high-resolution branch to 1024 × 1024 pixels while maintaining a resolution of 256 × 256 pixels for the low-resolution branch. The initial learning rate was set at 1 ×10 -3 , and every five epochs witnessed a decrease in learning rate by a factor of 0.1. The batch size is assigned as 1 to save memory. Except in the last step of multi-step training, where λ detect is set to 0.9 and λ local and λ out are set to 0.1, in the other steps, λ detect , λ basic , λ BF A and λ local are set to 1.In this section, we compare the performance of our proposed low-resolution branch network of multi-step hybrid knowledge distillation with several state-of-the-art detection methods, including Hui Y, et al.(Faster RCNN), Hui Y, et al.(Cascade RCNN), FCOS, ATSS, DAB-DETR, and DINO. All these methods trained with high-resolution input and up sample the low-resolution input during inference to match the size of the high-resolution input. These detection methods encompass both one-stage and two-stage detection approaches as well as CNN and transformer-based detectors. Fig. 3 presents a subset of the obtained detection results. We can conclude that MHKD outperforms these state-of-the-art methods through the qualitative analysis of Fig. 3 .As shown in Table I , our baseline approach with single-step offline distillation performs outperforms Hui Y, et al., Hui Y, et al., FCOS, ATSS, DAP-DETR, and DINO on LN dataset. The student method of MHKD outperforms the baseline and the direct detection methods on the low-resolution images. The mean average precision AP 0.5:0.95 achieves an 23.1% improvement compared with the direct detection methods and 15.5% improvement compared with the baseline. While the MHKD method has higher complexity in terms of parameters, its inference time remains competitive with baseline methods and fully meets the clinical needs.As shown in Table II , our baseline approach also outperforms Hui Y, et al., Hui Y, et al., FCOS, ATSS, DAP-DETR, and DINO on HUBMAP dataset. At the same time, the student method of MHKD achieves an improvement of 15.9% compared with the direct detection methods and 3.5% compared with baseline in the metric AP 0.5:0.95 . This shows that MHKD makes the student model effectively learn information from high-resolution branches, enabling it to extract more representative features from low-resolution inputs.In this section, we analyze the effect of each component in our method on LN dataset.1) Advantages of Basic Feature Adaptive Module: We adapt a multi-step hybrid knowledge distillation training strategy, which conducts three-step distillation training (basic features, local features, semantic information) and four-step distillation training (basic features, adaptive features, local features, semantic information) for whether BFA is added or not. The Table III shows the influence of the Basic Feature Adaptive module on low resolution branch detection performance. The addition of the BFA module and the distillation of adaptive features improved the AP 0.5:0.95 by 6.8%, It is proved that the BFA module greatly improves the detection performance of low-resolution pathological images.2) Advantages of Hybrid Knowledge Distillation Strategy: Online knowledge distillation and offline knowledge distillation are common knowledge distillation strategies. Here we compare our proposed hybrid knowledge distillation with online distillation and offline distillation on the basis of adding BFA and maintaining the multi-step distillation training strategy. As shown in Table IV , the detection performance of hybrid knowledge distillation is better than that of online distillation and offline distillation in all metrics. It can be seen that hybrid knowledge distillation plays a key role in our method.3) Advantages of Multi-Step Training Strategy: Table V presents the detection results for different distilled locations. We perform knowledge distillation at the three locations of basic features, local features, and final output respectively, showing better results than no knowledge distillation, proving that knowledge distillation at all three locations is effective.The training strategy is determined based on the distillation information partition. In single-step training strategy, we perform online distillation of multiple locations simultaneously. In double-step training strategy, in the first step, we perform offline distillation of global features (basic features and adaptive global features), and in the second step, we perform online distillation of semantic information. In fourstep training strategy, that is, our proposed MHKD method. As shown in Table VI, the detection performance is significantly improved with the increase of the number of steps. This proves that even if the distillation is performed at all four locations, the training step by step is more effective to improve the detection performance of low-resolution pathological images. V. CONCLUSIONIn this paper, we propose a glomerulus detection task in low-resolution human kidney pathology images, aiming to alleviate the limitations of hospitals in underdeveloped areas without sufficient cost to acquire high-resolution WSIs for kidney pathology diagnosis.To address this challenge, we introduce a novel MHKD approach for glomerulus detection in low-resolution patho- logical images. The hybrid knowledge distillation strategy combines offline and online distillation strategy, ensuring the feature extraction ability of the front-end, but also reduces the gap between the student model and the teacher model in the back-end. The multi-step training strategy facilitates the gradual transfer of high-resolution image information to the student model, from global features to semantic details. The MHKD effectively enhances the model's detection performance on low-resolution images compared to various direct detection methods, methods designed for high-resolution pathology images and simple distillation techniques. Future research will explore the integration of knowledge distillation into transformer-based object detection methods to further improve the accuracy of glomerulus detection in low-resolution pathological images and its clinical diagnostic potential. 
 
 Appendix:  



Reviews: review: title: title: Enhancing Glomerulus Detection in Low-resolution WSIs with Multi-step Hybrid Knowledge Distillation
paper_summary: paper_summary: The manuscript introduces MHKD, the first approach to address the challenge of glomerulus detection in low-resolution whole slide images for kidney pathology. This method reduces reliance on high-resolution digital pathology images, which are resource-intensive in terms of equipment, storage, and computational power. MHKD combines offline and online knowledge distillation techniques, transferring global and semantic features from a high-resolution teacher model to a low-resolution student model. By applying a multi-step training strategy, the model ensures that information from high-resolution images is gradually and effectively transferred to the student model, further enhancing the distillation effect. Results from the two independent datasets demonstrate that MHKD significantly improves detection performance compared to current state-of-the-art methods.
strength_weakness: strength_weakness: 1. The overall writing style could be improved to enhance clarity and accessibility for a broader audience.
2. The HuBMAP dataset includes only 20 kidney images, suggesting that a larger dataset would be beneficial for more robust validation.
3. While the authors mention the computational cost of analyzing kidney images, the manuscript lacks a benchmark comparison of the computational efficiency of their model.
questions: questions: 1. The MHKD framework shows promise in transferring high-resolution image labels to low-resolution images. How do you anticipate the MHKD method would perform on different types of histopathological images, such as those from other organs or diseases? Have you considered testing it on additional datasets?
2. The Basic Feature Adaptive (BFA) module appears to play a significant role in improving detection performance. Can you elaborate on the design choices behind this module and whether alternative designs were considered?
3. Given the complexity of the method, how do you envision integrating it into clinical workflows? Are there plans to develop a more streamlined or user-friendly version for clinical use?
4. Could the authors provide more details about the upsampling process mentioned in the method?
title: title: In-depth, engaging, and clinically relevant - MHKD: Low-Res Glomerulus Detection
paper_summary: paper_summary: This paper proposes a novel method called Multi-step Hybrid Knowledge Distillation (MHKD) for glomerulus detection in low-resolution kidney pathology images. The key contributions are:

1. Introducing the task of glomerulus detection on low-resolution kidney pathology images
2. Proposing a hybrid knowledge distillation approach combining offline and online distillation
3. Developing a multi-step training strategy to transfer knowledge from high to low resolution models
4. Demonstrating improved performance over state-of-the-art methods on two datasets

They have demonstrated SOTA performance in this task.
strength_weakness: strength_weakness: It would be helpful to the readers if the authors include the following additional points:

1. Inference speed
2. Lack of comparison to other knowledge distillation approaches for medical imaging - especially reference [6] that the authors have included in the references but have not used in the comparison.
3. A comment on the relatively small datasets used for evaluation - and what are the expectation in terms of generalizability of the model on larger datasets
questions: questions: Overall the paper presents a very promising research in this area of KD in histopathology. The presentation is in-depth and engages the readers very well. However, it is curious why certain other works where they have used KD in histopathology have not been compared against, especially [6] that is in the reference, but not cited elsewhere in the paper. It would be meaningful if this detail is also provided.

There are a few other questions as well:

1. How does the inference time of MHKD compare to the baseline methods? Is there a significant computational cost to the multi-step approach?

2. How sensitive is the performance to the downsampling factor between high and low resolution images? Is there an optimal ratio?
title: title: Multi-step Hybrid Knowledge Distillation for Low-resolution Whole Slide Images Glomerulus Detection
paper_summary: paper_summary: This paper proposes a novel approach for glomerulus detection based on hybrid knowledge distillation from low-resolution images.
strength_weakness: strength_weakness: The only thing the reviewer wants to see is how the complexity of the model (training times, inference times) compared to relevant studies referenced by the authors.

Based on that, we will verify whether the model can be employed to address computational resource constraints in a hospital environment.
questions: questions: 1. Should we include the comparison of model complexity between models in this study?