Improving CNN-Based Mitosis Detection through Rescanning Annotated Glass Slides and Atypical Mitosis Subtyping

Authors: rutger h j fick
christof a bertram
marc  aubreville
technische hochschule ingolstadt
Venue: MIDL_io_2024_Conference
Published: 2024-01-30 11:42:57

Abstract: The identification of mitotic figures (MFs) is a routine task in the histopathological assessment of tumor malignancy with known limitations for human observers. For a machine learning pipeline to robustly detect MFs, it must overcome a variety of conditions such as different scanners, staining protocols, tissue configurations, and organ types. 
In order to develop a deep learning-based algorithm that can cope with these challenges, there are two obstacles that need to be overcome: obtaining a large-scale dataset of MF annotations spread across different domains of interest, including whole slide images (WSIs) exhaustively annotated for MFs, and using the annotated MFs in an efficient training process to extract the most relevant features for classification.
Our work attempts to address both of these challenges and establishes an MF detection pipeline trained solely on animal data, yet competitive on the mixed human/animal MIDOG22 dataset, and, in particular, on human breast cancer.
First, we propose a processing pipeline that allows us to strengthen the true scanner robustness of our dataset by physically rescanning the glass slides of annotated WSIs and registering MF positions. To enable the use of such rescans for training, we propose a novel learning paradigm tailored for labels that match partially, which allows to account for ambiguous MF positions in the rescans caused by spurious, suboptimal fine-focus on potential MFs by the scanner. Second, we demonstrate how a multi-task learning approach for MF subtypes, including the prediction of atypical mitotic figures (AMFs), can significantly enhance a model's ability to distinguish MFs from imposters. Our algorithm, using a standard object detection pipeline, performs very competitively with an average test set F1 value across five runs of 0.80 on the MIDOG22 training set. We also demonstrate its ability to stratify overall survival on the TCGA-BRCA dataset based on mitotic density, though it falls short of reaching significance in stratifying survival based on AMFs.

Main paper: 
 introduction: 
 The identification of morphological structures of dividing cells, known as mitotic figures (MFs), is a routine task in the histopathologic assessment of tumor malignancy (Donovan et al., 2021) . MFs are morphologically heterogeneous and can be either normal or atypical. Due to their morphological complexity, MFs are prone to be missed, or mistaken for other apoptotic or necrotic cell structures during manual assessment, resulting in significant interrater disagreement (Meyer et al., 2005; Malon et al., 2012; Veta et al., 2016) . For these reasons, and also to reduce the workload of pathologists, automatic MF identification is a well-established computer vision task. The success of recent methods, which show an accurate and reproducible detection of MFs on regions of interests (ROIs) was, to a large degree, fueled by the availability of highly diverse and sufficiently large datasets, such as AMIDA13 (Veta et al., 2015) , TUPAC16 (Veta et al., 2019) , MIDOG21 (Aubreville et al., 2023b) and MIDOG22 (Aubreville et al., 2023c) . Common to those datasets, is the limitation that only annotations for a pre-selected ROI exist, that is typically selected from the area of highest MF density within the tumor. This restricts the data diversity of those datasets, in particular by not including areas that may contain cells such as aptotic or necrotic cells, that can be easily mistaken for MFs and thus can be considered hard examples for the pattern recognition problem. Practically, this means that the application to the complete whole slide image (WSI) can become an out-of-distribution problem for detectors solely trained on ROIs. However, annotation of whole tumor sections is a laborintensive task. In the field of canine histopathology, two notable datasets have been made publicly available, that, combined, provide annotations for more than fifty thousand mitotic figures, collected from 53 tumor specimens (Bertram et al., 2019; Aubreville et al., 2020) . These annotations, however, have been provided for slides scanned with a single scanner only, limiting domain generalization across scanners. Annotation of a wide range of WSIs acquired on multiple scanners is infeasible due to the high cost of skilled labor for this task. As a cost-effective alternative, our first contribution in this study is a training paradigm that allows us re-use the efforts to exhaustively annotate these WSI by rescanning their physical glass slides and filtering spurious ambiguous MFs in the rescans appropriately.Recently, it has also been reported that subtyping of MFs into normal and atypical mitotic figures (AMFs), which indicate an aberration of the normal chromosome separation process resulting in genetic alterations, might be relevant for the calculation of additional, prognostically relevant criteria in the assessment of breast cancer (Ohashi et al., 2018; Lashen et al., 2022) . Initial work on the automatic subtyping of MFs (Aubreville et al., 2023a) has found this to be a challenging task, additionally restricted by a low inter-rater agreement.In this study, we introduce a pipeline for detecting and subclassifying MFs in multiorgan, multi-scanner, multi-species, and fully WSI-based settings, demonstrating that subclassifying mitoses in multi-task learning significantly enhances the performance of MF detection. Our approach, which was trained on a diverse set of fully annotated canine samples scanned with seven different systems, uses a robust training objective that is unaffected by out-of-focus mitoses in rescans. Our main contributions can be summarized as: • We introduce a tailored training objective to counteract focusing artifacts commonly occurring when rescanning slides, allowing for the training of MF detectors with registered slides digitized by multiple scanners.• We show that adding a atypical classification subtask regularizes the mitotic figure detection task, leading to consistenly better results.• We show that our detector delivers both SOTA performance on the MIDOG22 challenge set and also stratifies survival on the BRCA-TCGA breast cancer WSI dataset. 
materials and methods: 
 In this section we first describe our MF dataset in section 2.1, followed by our WSI rescan filtering paradigm in section 2.2, and finally our model training approach in section 2.3.A clinically usable MF detection algorithm must be robust to the varying tissue composition and quality conditions found in a WSIs, as well as to domain shifts due to different scanner and organ domains. Our dataset is designed to provide robustness to these challenges:WSI Robustness We include the canine mammery carcinoma (CMC) dataset (Aubreville et al., 2020) , which consists of 21 WSIs exhaustively annotated for MFs.Scanner Robustness We rescanned a selection of the original CMC glass slides with 6 other scanners. We then transferred the MF annotations from the original WSI to the rescans using a WSI-level registration algorithm (Marzahl et al., 2021) . To remove blurred, out-of-focus or otherwise missing MFs in the rescans we designed a custom filtering approach, which we detail in section 2.2. For the rescans, we show the number of MFs remaining after our filtering scheme.Organ Robustness We acquired a secondary dataset of 159 WSIs, covering 17 different cancer types throughout different organs and animal species. As WSI and scanner robustness is obtained from the previous two datasets, here we only selected and annotated one ROI per WSI for MFs.We provide an overview of our composite dataset in Table 1 . Our hold-out test set for MF detection is the MIDOG22 training dataset (Aubreville et al., 2023c) , which consists of mitotic figure annotations in regions of interest originating from multiple species, organs and scanners. To allow for our secondary MF subtyping strategy, a trained pathologist also subtyped all annotated MFs into normal MFs or AMFs. We find that AMFs are rare, representing only about 10% of all mitotic figures. Our atypical MF test set consists of the MIDOG21 training set (Aubreville et al., 2023b) , whose mitotic figures were similarly subtyped into atypical and normal mitoses in previous work (Aubreville et al., 2023a) .Rescanning the original CMC glass slides and transfering the MF annotations to different scanners allows us to re-use the massive effort that was done to exhaustively annotate the over 14K MFs in the original scanner domain by 2+1 experts (Aubreville et al., 2020) . However, while the rescanned WSI should represent the same tissue as the original, the scanning process itself is not perfect and can result in MF annotation errors in the rescanned scanner domains. False annotations occur for various reasons: 1) MFs, which were in-focus in the original WSI are out-of-focus (OOF) in the rescan; 2) the scanning area has devicedependent limitations, leading to parts of the tissue not being scanned in areas of registered MFs; 3) the scanner's stitching algorithm can locally cut out mitotic figures if they live at the intersection between two stitched patches. Missing annotations in the rescan occur when MFs that were not visible in the original scanner domain are now in-focus in the rescan, but are not annotated. We show examples of false annotations in Figure 2 . Given the scale of the MF annotations in our CMC rescan dataset, it is infeasible to manually verify whether each registered MF is still interpretable as such in the rescan. To clean our dataset for such errors without expert intervention, we propose a filtering approach based on the premise that false annotations MFs in rescanned images are caused by spurious scanner artifacts that have a random distribution. We trained 10 different classification architectures on all scanner domains and merge them into one big ensemble. Knowing that this ensemble will be reasonably robust to different scanners and label noise, we apply this ensemble to the rescanned WSI and apply a conservative threshold to mask any uncertain MF. During the detector training, when such masked objects reside in the (larger) patch that is sampled, we opt to draw white circles over the suppressed MFs so the training ignores uncertain objects. We chose to not use this approach to pseudo-label possible missed MFs annotations in the rescans as we felt we might end up validating MF lookalikes.Our detection pipeline for MF detection is combination of a detector network to propose MF candidates, and an ensemble of classifiers that refines the selected candidates, which has proven to be successful in many MF approaches (e.g. Li et al., 2018; Piansaddhayanaon et al., 2023) . Our detector network is a YOLOR-D6 (Wang et al., 2021) and our ensemble of two classifiers consists of a DenseNet201 (Huang et al., 2017) and an EfficientNetB4 (Tan and Le, 2019) . We chose these architectures heuristically as those contributing most to the MF detection F1 score. For the classifier decoder heads, in addition to the standard binary MF/non-MF head we add a secondary head for binary normal/atypical MF classification, realized by a DenseNet201 network. We resampled all WSIs to 25 µm px and determined all thresholds on the validation set.Detector Training and Data Sampling Since we have such a large-scale, diverse and unbalanced dataset for organ types (see Table 1 and Supplementary Table S1 ), it is important to guide the model training to not focus only on the majority groups. Therefore, we adjusted the sampling probabilities to 50% from the base CMC dataset, and 25% each from the rescanned CMC and multi-organ datasets, and sampling with equal probability between the subgroups of those sets. We maintained a 50-50 split between MF annotations and negative annotations while maintaining the natural atypical/normal mitosis distribution. This strategy encourages model robustness w.r.t. different scanners in the rescanned dataset and to varying organs/cancer types in the multi-organ ROIs while leveraging the trustworthy annotations of the CMC base dataset. We trained on 1024 × 1024 px images until convergence was observed using the F1 score on the validation set, for which we assigned two cases from the CMC sets and 17 ROIs (one per tumor type) from the multi tumor set. We used an open-source library for HE-based data augmentation (Faryna et al., 2021) .Classifier Training and Data Sampling We adopted a similar domain sampling distribution for our classifier networks as for the detector. To train these networks, we used false positives generated by the trained detector model as negative examples, effectively utilizing the network as refining model (Li et al., 2018) . We optimized the ensemble weighting and decision threshold for each validation step using grid search, using the best performing model per architecture (on the validation set) out of five runs. The second classifier head only handles the normal or atypical MF subtyping. For this reason, we created a separate dataloader that samples only MFs and samples normal and atypical mitoses equally. Within each category, we sampled the subtypes equally. Given that the primary head is used for early stoppping, it is not expected that the second head is optimal at the same time. For this reason, after convergence of the MF classification head, we froze the encoder and trained the secondary head until convergence.To investigate the WSI-based performance, we evaluated our detection pipeline on the breast cancer cohort (BRCA) of The Cancer Genome Atlas (TCGA) project. Following the diagnostic grading process of breast cancer (Fitzgibbons and Connolly, 2023), we selected the area with the highest mitotic count (MC), using a circular field of view spanning 2 mm 2 . Since the classification head for atypical/normal shows only average accuracy, we selected only the predictions with the highest confidence (p > 0.9) for either class, reducing the number of detected MFs utilized for the next step by 42.84%. We then calculated the AMF to normal MF (NMF) ratio per case, similar to (Lashen et al., 2022) . Due to the expected high level of inaccuracies in the AMF/NMF decision, we elected to calculate the ratio not only per ROI but also per WSI in an effort to decrease the SNR of the metric. We then fitted the survival data provided in the dataset to find the threshold which provided the strongest predictive value, as indicated by the p value of the Cox linear hazard model and fitted a Kaplan Meier estimator on this optimal threshold. 
results: 
 Comparing the pure classification performance (second stage of the approach) for different network architectures, we find that adding the atypical subtyping task consistently improves the performance (see Table 2 ). We find this to also hold true in the end-to-end performance Mitosis Detection F1 Classifier Model w/o filtering, w/o MF subtyping w/ filtering, w/o MF subtyping w/ filtering, w/ MF subtyping DenseNet201 (Huang et al., 2017) 0.72 ± 0.022 0.74 ± 0.011 0.77 ± 0.025 EfficientNet B4 (Tan and Le, 2019) 0.69 ± 0.030 0.72 ± 0.013 0.75 ± 0.020 ResNet50 (He et al., 2016) 0.70 ± 0.016 0.71 ± 0.020 0.74 ± 0.014 on CMC, CMC Rescan, our multi-tumor dataset and MIDOG22 (see Table 3 ). Moreover, we find that AMF classification is significantly more challenging for our model, as shown in the third and fourth column of Table 3 . Note that the evaluations given in this table assume perfect recognition of MF and only evaluate the subtask of AMF classification. Our evaluation of the precision-recall-curves in the supplementary Figure S2 reveals a similar performance across scanners for breast cancer, demonstrating the scanner-robustness of our scheme, a notably high performance for the MF recognition in mast cell tumor, and a deterioration of performance on the neuroendocrine and lung cancer tumors. Lastly, as shown in the supplemantary Figure S1 , there is a strong correlation between the predicted MC and the respective ground truth, and a less significant correlation for the AMF/NMF ratio.When predicting on the TCGA-BRCA dataset, we found an average count of mitoses per ROIs of 26.67 and an average count of atypical mitoses within the ROI of 1.01. The AMF/NMF ratio on WSIs had a mean value of 0.10, whereas the same metric, when evaluated on the ROIs of the MC was 0.13, indicating a slightly higher rate of AMFs within the ROIs. The survival prediction, shown in Figure 3 , shows that the MC stratified survival into two groups. The Chi-squared test indicates significant difference (p < 0.01) between the groups. On the other hand, even for an optimized cutoff value of 0.11, the test did not indicate significant differences between the groups stratified by the AMF/NMF ratio. We found the AMF/NMF ratio calculated on the hotspot ROI did not stratify survival at all.  
discussion and conclusion: 
 Our work shows that by using animal histopathology data, we can train a highly competitive MF detector for human breast cancer. The training scheme, utilizing registered rescans of the glass slides, incorporated a high degree of scanner robustness, as our results show. While the AMF detection itself yielded only mediocre classification rates, likely caused by the high inter-rater variability and the difficulty of the problem, it consistently regularlized the model and improved performance on the primary MF detection task.We note that although the training paradigm introduced in our work reduced the number of false annotations in rescanned slides, it did not aim to recover MFs that were not annotated in the primary dataset, thus increasing the risk of false negatives in the rescanned slides. Moreover, the measurable effect of rescan filtering depends on how often the rescans are sampled during training. The use of immunohistochemistry in a restaining procedure could help in the identification of OOF mitoses in future work (Tellez et al., 2018) .The prediction of survival on the external BRCA dataset underscores the robustness of our pipeline. While we chose the threshold for subgroup separation in the survival analysis post-hoc, the value of 11 mitoses per 2 mm 2 is well within expectations given the current CAP guidelines (Fitzgibbons and Connolly, 2023) . In contrast to the findings of Lashen et al. (Lashen et al., 2022) , we did not find the AMF/NMF ratio to be significantly stratifying for overall survival, even for an optimized threshold value. While this could be linked to the performance of our classifier, we also observe a striking difference in the apparent perception of mitoses being atypical, which is expressed by a mean AMF/NMF ratio of 0.2 on TCGA-BRCA and a mean count of 2 AMFs per ROI in the original work (Lashen et al., 2022) , where our overall estimates are significantly lower.In conclusion, we demonstrated that our dual-pronged approach of using rescanned slides and MF subtyping allowed us to train a highly competitive MF detection approach using animal data only, as benchmarked on the multi-species MIDOG22 dataset. We envision that future improvements to the AMF subtyping task will allow us to find statistically significant and clinically meaningful uses for detecting atypical mitoses. 
 
 Appendix:  


Decision: The paper presents a pipeline for detecting and subclassifying mitotic figures (MFs) in histopathological images, with a focus on canine and human breast cancer datasets. It introduces a composite dataset and a training pipeline that includes preprocessing and a filtering approach to improve annotation quality. While the paper demonstrates improvements in MF detection and subtyping, reviewers highlight several weaknesses, including limited novelty, lack of comparison with existing approaches, and insufficient clarity in certain aspects.

While the paper presents a comprehensive pipeline for MF detection and subtyping, addressing the weaknesses highlighted by reviewers could significantly strengthen its contribution and impact. Suggestions include conducting an ablation study for the filtering approach, comparing with existing methods, and clarifying certain aspects of the methodology and results presentation.

Reviews: review: paper_summary: paper_summary: The article proposes an approach for identification of mitotic figures in WSIs. The contributions are two folds: a composite dataset and a training pipeline. The composite dataset is collected from three different sources: two public and one in-house. The training pipeline includes the preprocessing of the dataset: annotations from the original WSI are transferred to the rescans using a WSI-level registration algorithm and a filtering approach to remove the ambiguity. The filtering approach is based on ensembling of 10 different classification architectures which are used to mask the uncertain annotations based on the threshold. Mitosis detection pipeline consists of a detection network and two classification networks (an additional loss is used for classification: atypical MF subtyping).
main_review: main_review: The authors have provided some relevant arguments related to the raised concerns. They have also included the requested changes. Based on these inputs, the preliminary rating is revised in favour of acceptance.
strength_weakness: strength_weakness: 1. Writing:
+ The writing can be improved by including a graphical abstract summarizing the overall pipeline. It will improve the readability of the paper. 

2. Novelty and Results:
+ The novelty aspect of the paper is very limited.
     + The major contributions come from the aggregation of the dataset and training process. 
     + The architecture and other algorithms (WSI registration) are utilized from the existing work (Li et al., 2018; Piansaddhayanaon et al., 2023, Marzahl et al., 2021). 
+ Authors have proposed a filtering approach to improve the annotation. 
     + However, there is no ablation study to highlight its significance. 
+ There is no comparison with the existing approaches:
     * The existing methods should also be trained with the new composite dataset for the fair comparison
questions: questions: + The ablation study showing the impact of filtering in data preprocessing
+ Comparison with existing methods (trained with proposed dataset)
+ Do authors plan to release the dataset?
paper_summary: paper_summary: The paper introduces a pipeline for detecting and subclassifying mitotic figures (MFs) in histopathological images, particularly focusing on canine and human breast cancer datasets. It highlights the challenges in accurate MF identification and proposes solutions such as rescanning slides and incorporating mitosis subtyping. The proposed pipeline demonstrates improved performance in MF detection and subtyping, as well as survival prediction in breast cancer patients.
main_review: main_review: It is a good-quality study. The paper is well-structured, easy to follow, and addresses a relevant issue. The authors addressed my comments. Based on these inputs, the preliminary rating is preserved (Weak accept).
strength_weakness: strength_weakness: In the submission, the authors do not report plans on making data or code public. Open-sourcing the code with weights/It data would strongly increase the value of this work to the digital pathology research community.
questions: questions: Points mentioned in Weaknesses and Detailed Comments.
paper_summary: paper_summary: In this paper, the authors develop a deep learning detection and classification of mitotic figures (MF). They create a dataset with the goal of robustness against organs/species, scanners, and WSI (robust against bias in ROI selection). The detection pipeline consists of a detector to propose MF candidates and a classifier to accept or reject the proposals. The main focus of the paper is on the classifier and the data. The authors show that the classifier can be consistently improved when training it in a multi-task-learning (MTL) fashion together with classification of normal (NMF) vs atypical (AMF) MF. Further, survival stratification is predicted using either the number of MF in a hotspot region or the NMF vs AMF ratio. However, the latter approach did not yield significant findings.
main_review: main_review: The paper addresses the relevant issue of robustness by creating a very diverse dataset. It also once again shows that MTL can improve DL models. The paper is stringently structured, clearly written and easy to follow.
strength_weakness: strength_weakness: * cut-off is chosen post-hoc. Instead, the threshold could have been optimized using the raining or validation dataset
* Also, introducing this threshold may introduce a bias as a e.g. certain type of AMF could be more difficult to predict and is thus more likely to stay below the set threshold
* The discussion contains a statement about why MFs were not tried to be recovered on the original scan and that the thresholds were set post-hoc. This is both important information that should have been positioned in sections 2.2 and 2.4, respectively
questions: questions: * It is not really clear how the removal of uncertain MF classifications as described in section 2.2 affects the training of the classifiers. The authors should comment on this and why they think that removing uncertain MFs does not reduce the significance or their results.
* The authors should include a statement why they do not balance NMF vs AMF
* would be very interesting to see whether an optimization of the thresholds using training and/or validation data would result in similar values and how this affects the accuracy of the ratios
* Figure 3b is not described in the text

Comments: title: End of review period
Overall comments on the review
Our response to the review
Our comments to the review
Response to the review
comment: Dear area chair,

We appreciate the constructive feedback and positive evaluation of our paper. Regrettably, none of the reviewers have actively participated in the discussion phase. Nevertheless, we are confident that we have addressed all the issues raised in the initial review. We hope the lack of engagement as an indication that the reviewers are in agreement with our responses.

Warm regards,

Marc Aubreville (on behalf of the authors)
We sincerely thank all three of the reviewers for providing insightful and constructive criticism of our paper. We think that the new, revised manuscript has sincerely benefitted from the review. The main points that we updated were:
- We provide a new main Figure, serving as graphical abstract, as requested by Reviewer 3zQm.
- We ran an ablation study on the automatic removal of uncertain mitotic figures, as requested by reviewers 3zQm and gJtB.
- We improved the overall phraseology according to the reviewers’ comments.

Please find the detailed comments in the responses to each reviewer.
We appreciate the in-depth and very supportive comments of the reviewer and we have revised our manuscript according to the comments, which, we hope, will find the reviewer’s appreciation. Here are our replies to the points raised by the reviewer:

> cut-off is chosen post-hoc. Instead, the threshold could have been optimized using the raining or validation dataset

We apologize for the misconception that we determined the cut-off values post-hoc for the mitotic figure detection, which might have been caused by presenting the F1 score dependency on the cutoff for each subgroup in Figure 3. However, it's important to clarify that we did, indeed, optimize the cutoff based on the validation set, a point we elaborate on in the revised document.

Regarding the survival prediction on TCGA, we did employ a post-hoc determination of an optimal subgroup-dividing threshold. Such an approach aligns well with other comparable recent works [1,2,3]. Subdividing the TCGA data for the specific purpose of defining and validating a cut-off would not only impair the comparability to other studies, but also severely reduce the statistical power of the sample, which is crucial for survival analysis. We, therefore, chose not to take this route.

[1] 10.1038/s41379-022-01080-0

[2] 10.21203/rs.3.rs-2408990/v1

[3] 10.1038/s41598-023-40042-7

> Also, introducing this threshold may introduce a bias as a e.g. certain type of AMF could be more difficult to predict and is thus more likely to stay below the set threshold

As with any detection task employing subgroups, we can expect a dependency of the optimal threshold on the subgroup. However, analysis of this is, in our view, out of scope for the current work, since we did not employ a subclassification of AMFs in our pipeline in this work. To clarify, we also exchanged Figure 1, which might have given rise to this impression.

> The discussion contains a statement about why MFs were not tried to be recovered on the original scan and that the thresholds were set post-hoc. This is both important information that should have been positioned in sections 2.2 and 2.4, respectively

We acknowledge your point, and appreciate your suggestion on the placement of these critical pieces of information. Still, we feel it is important to clarify, once again, as stated earlier, that the cutoff for mitosis detection was not a post-hoc decision. We have revised the manuscript to underscore this in section 2.2. As for the statement regarding survival analysis, we reformulated this in section 2.4 for more clarity.

> In table 1 the resolution of the resulting images could be added to see whether resolution differences affect the models' performance

Thanks for this comment. We added this information to Table 1. We resampled all images to 0.25 microns/px for processing, which we also now added to the manuscript.

> In section 2.2, it says "We show examples of false positive cases in Figure 2", probably "false negatives" is meant here

We acknowledge that the term “false positive” might be misleading for an annotation. Hence, we switched our terminology to “false annotations” and “missing annotations”. Figure 2 shows false annotations, since the annotation exists but the object is no longer recognizable as a mitotic figure in the rescan.

> It is not really clear how the removal of uncertain MF classifications as described in section 2.2 affects the training of the classifiers. The authors should comment on this and why they think that removing uncertain MFs does not reduce the significance or their results.

We appreciate this comment and ran an ablation study on this question, which the reviewer will find in the revised manuscript in Table 2. In fact, as the results show, the filtering approach does improve the classifier performance consistently on all of the evaluated models. We should note that, given the sampling scheme only samples 25% from the rescanned slides, the effect would likely be even more pronounced when only training on the rescans. 

> The authors should include a statement why they do not balance NMF vs AMF

The reviewer is right and ideally you would go even further to sample the mitotic subtypes and phases equally. However, we found in preliminary experiments that this did not benefit the overall mitotic detection performance. We hypothesize that our support for AMF in the dataset is still not big enough over all domains such that the model does not overtrain on the few ones available in some domains. We will explore this in future work with further dataset extensions.

> Figure 3b is not described in the text

Thanks for pointing this out. Ultimately, we decided to move Figure 3b to the supplementary material, as also the peer review required us to make space for additions requested by the reviewers.
We appreciate the constructive comments of the reviewer and we have revised our manuscript according to the comments, which, we hope, will find the reviewer’s appreciation. These are our responses to the points raised by the reviewer:

> In the submission, the authors do not report plans on making data or code public. Open-sourcing the code with weights/It data would strongly increase the value of this work to the digital pathology research community.

We are preparing the release of the dataset, however, since the dataself is of substantial size and complexity and the publication warrants an in-depth description, it is not possible within the review time span. Additionally, at this point alterations to the co-authors list for this paper are no longer feasible. There were several other researchers who played a role in data preparation, and we firmly believe in appropriately acknowledging all contributions in scientific endeavors. Hence, we feel it's crucial that all parties involved receive proper recognition.

Regarding the release of the detector source code: As reviewer 3zQm pointed out, there is no new model architecture involved, and our paper is based on well-known application of object detectors, so we see no added value in releasing yet another very similar pipeline to the suite of already available code bases for this purpose, given the pace of software and library development in our field, would soon be outdated.

> In the abstract, the 0.80 value is reported without a metric specified. I assume it is F1 score?

We are sorry for not stating this clearly. The reviewer is correct: It is the F1 score, we have fixed this in the revised abstract.
We appreciate the constructive comments of the reviewer and we have revised our manuscript according to the comments, which, we hope, will find the reviewer’s appreciation. Here's our reply to the points raised by the reviewer:

> The writing can be improved by including a graphical abstract summarizing the overall pipeline. It will improve the readability of the paper.

We agree with the reviewer on this, and replaced the previous Figure 1 with a completely new Figure showcasing the overview of our method. 


> The novelty aspect of the paper is very limited. The major contributions come from the aggregation of the dataset and training process. The architecture and other algorithms (WSI registration) are utilized from the existing work (Li et al., 2018; Piansaddhayanaon et al., 2023, Marzahl et al., 2021).

We view this as an asset, rather than a liability of our paper: We utilize our comprehensive understanding of the issue at hand to engineer a robust solution, applicable to a vast number and variety of architectures. Our approach enhances and complements existing research in the domain. Reusing glass slides to embed characteristics of different scanners into a dataset was never done before by any other group, despite the advantages being obvious and clearly shown by our work. As our results demonstrate, the efficiency of a pipeline hinges substantially (if not more so) on the data loading strategy and dataset attributes rather than merely on the model architecture.


> Authors have proposed a filtering approach to improve the annotation. However, there is no ablation study to highlight its significance.

We agree with the reviewer and provide the ablation study as an additional column of Table 2 in the revised manuscript.

> There is no comparison with the existing approaches. The existing methods should also be trained with the new composite dataset for the fair comparison

We respectfully disagree with the reviewer on this point. Our paper does not primarily present a new architecture, making model comparisons outside the scope of our paper. We draw attention to Table 2 in our paper where we have indeed offered a comparative study of three cutting-edge classification architectures - DenseNet, EfficientNet and ResNet - in relation to our task, only to find negligible variation in the results. Given our candidate detection task is undertaken with a set operating point targeting a 99% recall rate, virtually any state-of-the-art object detection would suffice. Hence, we respectfully argue that an additional comparison at this stage of the architecture would not contribute any new insights to our paper.

> Do authors plan to release the dataset?

We are preparing the release of the dataset, however, since the dataself is of substantial size and complexity and the publication warrants an in-depth description, it is not possible within the review time span. Additionally, at this point alterations to the co-authors list for this paper are no longer feasible. There were several other researchers who played a role in data preparation, and we firmly believe in appropriately acknowledging all contributions in scientific endeavors. Hence, we feel it's crucial that all parties involved receive proper recognition.