IHCScoreGAN: An unsupervised generative adversarial network for end-to-end ki67 scoring for clinical breast cancer diagnosis

Authors: carl p molnar
thomas e tavolara
christopher a garcia
david s mcclintock
mark d zarella
wenchao  han
p  molnar
w  zarella
  han
Venue: MIDL_io_2024_Conference
Published: 2024-01-31 23:54:16

Abstract: Ki67 is a biomarker whose activity is routinely measured and scored by pathologists through  immunohistochemistry (IHC) staining, which informs clinicians of patient prognosis and guides treatment. Currently, most clinical laboratories rely on a tedious, inconsistent manual scoring process to quantify the percentage of Ki67-positive cells. While many works have shown promise for Ki67 quantification using computational approaches, the current state-of-the-art methods have limited real-world feasibility: they either require large datasets of meticulous cell-level ground truth labels to train, or they provide pre-trained weights that may not generalize well to in-house data. To overcome these challenges, we propose IHCScoreGAN, the first unsupervised deep learning framework for end-to-end Ki67 scoring without the need for any ground truth labels. IHCScoreGAN only requires IHC image samples and unpaired synthetic data, yet it learns to generate colored cell segmentation masks while simultaneously predicting cell center point and biomarker expressions for Ki67 scoring, made possible through our novel dual-branch generator structure. We validated our framework on a large cohort of 2,136 clinically signed-out cases, yielding an accuracy of 0.97 and an F1-score of 0.95 and demonstrating substantially better performance than a pre-trained state-of-the-art supervised model. By removing ground truth requirements, our unsupervised technique constitutes an important step towards easily-trained Ki67 scoring solutions which can train on out-of-domain data in an unsupervised manner.

Main paper: 
 introduction: 
 Ki67 is a protein biomarker whose percentage of Ki67-positive tumor cells has shown to be effective for indicating tumor proliferation in breast cancer cases (Davey et al., 2021) . In a clinical setting, the Ki67 score is reported routinely for breast cancer IHC tissue samples to aid in diagnosis and guide treatment decisions (Faneyte et al., 2003; Chang et al., 2000; Petit et al., 2004) . Currently, in most centers, Ki67 scoring is manually performed by pathologists, which is time-consuming and subject to inter-observer variability (Reisenbichler et al., 2020) . Therefore, automatic Ki67 scoring for breast tissue samples are extremely desirable.Unfortunately, automatic stain scoring is challenging due to the prohibitive data demands for training or fine-tuning supervised deep learning-based IHC scoring models. These models require pixel-level annotations of hundreds of thousands of cells, painstakingly labeled by highly trained technicians (Abousamra et al., 2020; Fassler et al., 2020; Graham et al., 2019; Wen et al., 2023; Van Eycke et al., 2018; Priego-Torres et al., 2022; Zhang et al., 2020) . DeepLIIF (Ghahremani et al., 2022) is a supervised framework which instead trains using multiple co-registered multiplex immunofluorescence (mpIF) images as its ground truth to eliminate cell annotation errors. However, mpIF assays are expensive and not widely available. On the other hand, pre-trained supervised models are rarely publicly available and not always applicable to in-house stain data: tissue stains are collected from a particular area of the body with distinct tissue features, using a unique arrangement of best practices, tissue scanners, chemical mixtures, and tissue qualities, resulting in a unique in-house data distribution (Wagner et al., 2024; Therrien and Doyle, 2018) .Recently, several studies have emerged in an attempt to completely eliminate the need for ground truth labels. To this end, unsupervised deep learning methods typically leverage CycleGAN (Zhu et al., 2017) , a well-known framework for unpaired and unsupervised domain transfer. One such example is automatic stain transfer, where the goal is to transfer IHC stain images into hematoxylin-eosin (H&E) stain images in an unsupervised manner (Liu et al., 2021; Trullo et al., 2022; Lin et al., 2023) . However, performing a stain transfer from IHC to H&E loses the biomarker expression information associated with the IHC stain, making IHC scoring impossible. Unsupervised nuclear semantic segmentation (Le Bescond et al., 2022) and instance segmentation (Wang et al., 2023) were recently shown to be possible by leveraging binary cell segmentation masks created from H&E images; however, both of these approaches lack the ability to retain vital IHC biomarker expression information. In the field, there remains an unmet need for an end-to-end IHC scoring method which can avoid the need for ground truth labeling.In this paper, we propose IHCScoreGAN, the first unsupervised deep learning framework to provide end-to-end scoring for IHC sample images. We leverage a domain transfer strategy to formulate a novel learning task, along with a novel model architecture which facilitates this task with a dual-branch generator. Our learning task seeks to transfer Ki67 images into synthetic cell segmentation masks extracted from an unpaired public dataset of H&E tissue images, instilling cell center points and synthetic cell colors. Significantly, our model automatically learns the correspondence between the Ki67-positive and -negative cells and the synthetic cell colors while also generating center point predictions, which together provides a predicted Ki67 score. We validated our framework on a large cohort of clinical cases sourced from Mayo Clinic, demonstrating strong performance on 2,126 clinically signed-out breast cancer cases collected over the span of 11 years, and further validated on an external dataset for cell counting, yielding competitive performance against supervised methods. Our proposed contributions are as follows:• We propose a novel domain transfer strategy for unsupervised, unpaired end-to-end IHC scoring, where the learning task is constructed from synthetic colored cell segmentation masks and center points easily extracted from public H&E images. • We propose a novel model architecture to accommodate our learning task, which can generate colored cell segmentation masks as a proxy task while simultaneously generating supplementary information for IHC scoring during inference. 
methodology: 
 Our proposed framework, IHCScoreGAN, is trained by leveraging a novel learning task constructed from unpaired, publicly-sourced H&E data. It does not require any preprocessing or postprocessing of IHC stain images (Figure 1a ). To facilitate our learning task, we build a novel model architecture which achieves end-to-end Ki67 scoring by splitting our learning task into two parts: 1) its primary goal is to generate a colored cell segmentation mask, which serves as a proxy task for 2) predicting cell center points and biomarker expressions (Figure 1b ). From this information we can extract a Ki67 score, which is aggregated per slide and evaluated against clinically-derived scores (Figure 1c ). S Ihc = {x i } N Ihc i=1, where x ∈ R 256×256×3 is a Ki67 stain tile.We generated our target dataset using unpaired H&E-stained tissue slides sourced from The Cancer Genome Atlas (TCGA) database (Weinstein et al., 2013) , a public clinical data repository. We randomly sampled H&E tissue slides of 20× magnification from the Breast Invasive Carcinoma project (TCGA-BRCA) and manually selected slides to represent a variety of cell sizes, structures, and shapes (i.e., to discourage the model from using these features to discriminate real vs. fake data), resulting in 23 slides. We then divided each slide into tiles of size 256×256 and sampled N Mask = 10, 000 total tiles. Next, we passed the H&E tiles through a publicly available HoVerNet (Graham et al., 2019) model, pre-trained on the CPM-17 dataset (Vu et al., 2019) to predict cell instance contours.We generated synthetic colored cell segmentation masks for each H&E tile by assembling the predicted cell instance contours into a cell segmentation mask, drawing a ratio from a uniform distribution U [0,1] , and then randomly picking cells to color green based on the ratio and coloring the rest red. We also draw a value from a uniform distribution U [0,1] for each cell and multiply the cell's pixel intensities by this value, in order to emulate different IHC biomarker expression intensities (see "Real Y" in Figure A.1). We then generated distinct binary masks for our two synthetic cell colors, which detach cell expression prediction from the intensity values. Finally, we generated cell center point distance maps, instilling 2-norm distances between each pixel in each cell and its corresponding predicted center point.We formally define our external target dataset asS Mask = {(y i , k i )} N Mask i=1, where y ∈ R 256×256×3 is a synthetic colored segmentation mask tile and k ∈ R 256×256×3 is its matching cell center point distance map and two binary cell expression masks. Unsupervised nuclear cell segmentation and biomarker expression identification is a proxy task for our model, which builds off CycleGAN (Zhu et al., 2017) . We train our model using a domain transfer strategy to learn how to convincingly transform our Ki67 stain input tiles x into synthetic colored segmentation masks y.The objective of our model is to learn an unpaired domain transfer between domains X and Y, where we seek to train a generator G : X → Y which can transform an input x ∈ X following ŷ = G(x), where ŷ ∈ Ŷ. To enforce a robust mapping onto the distribution of Y, an inverse generator F : Y → X is simultaneously trained to maintain "cycle consistency". Generator F can transform an input y ∈ Y following x = F (y), where x ∈ X . The idea is that an input x should be penalized for being different from its reconstruction x = F (G(x)) through a cycle consistency loss, and similarly for an input y in the other mapping direction:L Cyc X = E x [||F (G(x)) -x|| 1 ] L Cyc Y = E y [||G(F (y)) -y|| 1 ] (1)Next, the generated domain Ŷ is constrained to be indistinguishable from domain Y, which is estimated by training discriminators D Y and D X , which produce logits predicting whether an input is from Y or Ŷ, or from X or X , respectively, through an adversarial loss:L Gan X = E x [log(1 -D Y (G(x))] + E x [log(D X (x))] L Gan Y = E y [log(1 -D X (F (y))] + E y [log(D Y (y))](2)where generators G and F are updated such that L Gan X and L Gan Y are minimized, and discriminators D X and D Y are updated such that L Gan X and L Gan Y are maximized.Finally, an input x should be identical to itself after mapping onto its own distribution x = F (x), and similarly for y, constrained through an identity loss:L Idt X = E x [||F (x) -x|| 1 ] L Idt Y = E y [||G(y) -y|| 1 ](3) Our model's generators are each composed of encoder-decoder architectures similar to a four-block U-Net (Ronneberger et al., 2015) . Our discriminators each resemble a contracting convolutional neural network. Network details are further elaborated in Appendix A.1.We previously defined our target dataset asS Mask = {(y i , k i )} N Mask i=1, where y is a synthetic colored segmentation mask tile and k is its matching cell center point distance map and binary cell expression masks. In this section, we aim to predict k ∈ K from a given Ki67 input tile x, which contains the critical information for achieving simple end-to-end quantification.To generate k, we add a second decoder branch Dec k to our generator G so that it generates outputs {(ŷ, k)} = G(x) through its main and secondary branches simultaneously. For clarity, we hereafter denote the output of G through Dec k as k = G K (x), where k ∈ K. Our generator thus resembles G : X → {(Y, K)}. Our second decoder branch Dec k follows the same structure and input as its main decoder branch Dec y , but we detach the input embeddings from the gradient graph in this branch, since the proxy task already embeds all necessary information for Dec k in its encoder embeddings (i.e., cell expression and shape).To train Dec k , we add three learning objectives to the model. First, we introduce an additional cycle consistency loss between real k ∈ K and reconstructed k ∈ K:EQUATIONTo encourage generation of realistic cell center points and expressions, we add a new discriminator D K which produces logits predicting whether an input is from domain K or K, along with a new adversarial loss:EQUATIONwhere generator G K is updated such that L Gan K is minimized, and discriminator D K is updated such that L Gan K is maximized. Discriminator D K follows the same internal structure as discriminators D X and D Y . Finally, we tie our identity constraint to the encoding of y by evaluating k = G K (y) and comparing it with its matching k:EQUATIONThe model architecture and loss constraints introduced in this section encourage generation of convincing k through k = G K (x), which is a simple, single-stage process.Our model's overall objective function combines Equations 1 through 6:L model = λL Cyc X + βL Gan X + γL Idt X +λL Cyc Y + βL Gan Y + γL Idt Y +λL Cyc K + βL Gan K + γL Idt K (7)where λ, β, and γ are scaling terms. We use λ = 10, β = 1, and γ = 10 in this paper, following CycleGAN (Zhu et al., 2017) . L model is simultaneously minimized with respect to generators G and F and maximized with respect to discriminators D X , D Y , and D K .During inference, we extract predicted cell center points and cell expressions from the generated k = G K (x) using a local maxima algorithm (see Appendix A.2). We then aggregate the cell counts at the slide level and calculate the slide-level scoring followingKi67 + Ki67 + +Ki67 -. 
experiment design: 
 Model Comparison We compared our method against a state-of-the-art supervised model, DeepLIIF (Ghahremani et al., 2022) , and a supervised U-Net (Ronneberger et al., 2015) , both trained on the public Breast Tumor Cell Dataset (BCData) (Huang et al., 2020) using identical settings as their original paper. Dataset details are provided in Section 3.4.Error Metrics Mayo Clinic pathologists used a 20% cut-off for diagnosing Ki67-high vs. Ki67-low cases based on the ASCO guideline (Giordano et al., 2022) in the clinical diagnosis reports. Therefore, in our internal data experiments, we used the same cut-off value to classify a case from each method's case-level Ki67 score prediction. We then compared results by evaluating precision, recall, accuracy, and F1 score. For our experiments on the public BCData dataset, we instead evaluated mean absolute error (MAE) of cell counts across images, used in the BCData paper. In the tables, arrows represent the direction of optimal performance for each metric. Error metric details are provided in Appendix A.3.We performed a two-split experiment to reflect the performance for the model in a general use-case, where we randomly drew 1,532 cases in the train split and 594 cases in the test split, with tiles corresponding to the cases aggregated into their split. Our framework outperformed both supervised models on all error metrics except precision (Table 1 ).We also evaluated the clinical use case of our proposed framework, mimicking deployment in a real-world setting where the model was trained on older data collected from 2012 to 2018 and then evaluated on cases collected after 2018. This division resulted in 1,532 cases in the train split and 594 cases in the test split, with tiles corresponding to the cases aggregated into their split. Each model exhibited a reduction in performance in this experiment -we observed that the after-2018 samples were more challenging due to an increase in background noise, such as darker stroma staining (Table 2 and Figure 2 ). In practice, cases very close to the classification cutoff may be equivocal. Using the results in the clinical simulation experiment in Section 3.2, we relaxed the classification of equivocal cases by using each interval from ±1% to ±10% around the 20% ASCO cutoff point (Figure 3 ) and classifying cases both predicted and labeled within the interval as correct. Each model improved as the cutoff interval was relaxed, but we note that IHCScoreGAN exhibited near-optimal performance when relaxing the cutoff interval by just ±2%, reporting 0.93, 0.99, 0.98, and 0.96 in precision, recall, accuracy and F1 score, respectively (see Appendix B.1 for precise values). We validated our framework on a public breast cancer dataset, BCData, which features 1,338 Ki67 stain images taken at 40× magnification by a Motic BA600-4 scanner, along with 181,074 manually annotated cell center points and cell types (Huang et al., 2020) . BCData splits 803, 133, and 402 images for training, validation, and testing, respectively. We compared cell counting of our framework against two supervised models, both of which were trained using BCData's training images and cell annotation labels. The results are in Table 3 , where 'MP', 'MN', and 'MA' represents the MAE of positive, negative, and the average value, respectively. Our unsupervised framework did not outperform the supervised state-of-the-art DeepLIIF, yet still achieved comparable performance without needing the annotation labels. Both ours and DeepLIIF outperformed all supervised baseline models reported in (Huang et al., 2020) . It is possible that the training sample size limited our framework's performance, suggested by our sample size experiments in Appendix B.3.  
discussion: 
 In this work, we proposed the first unsupervised framework, IHCScoreGAN, for end-toend Ki67 scoring. We validated our method on 2,126 breast cancer cases and showed high agreement with clinical diagnoses provided by our pathologists. Experimental comparison against pre-trained supervised methods on our internal dataset showed the significant advantage of our framework; pre-trained DeepLIIF often erroneously segmented stroma tissue and missed/misclassified challenging cells, likely resulting from differences from its training domain (Figure 2 ). On external data, we yielded close cell counting performance to the fully-supervised state-of-the-art DeepLIIF, and superior performance to other supervised models, without needing the training annotations. This work is limited by lacking comparisons with supervised models which are fine-tuned on our dataset, which is out of the scope of this work due to the complexity of fairly assessing such an experiment. We do not consider inter-observer variability in our clinical diagnoses in this work.blocks, each consisting of a convolution layer (with a kernel size of 4, stride of 2, and padding of 1), followed by a LeakyReLU activation layer and an instance normalization layer. There are 64 convolution channels in the first encoder block, which increase by a factor of 2 in each block (up to 512 channels). The encoders have 8 additional residual blocks, each consisting of an in-place convolution (with kernel size 3), again followed by a LeakyReLU activation layer and an instance normalization layer. The decoders consist of 8 residual blocks of the same definition, followed by four expanding "up-convolution" blocks, each consisting of a transposed convolution layer (with a kernel size of 4, stride of 2, and padding of 1), followed by a LeakyReLU activation layer and an instance normalization layer. There are 512 convolution channels in the first decoder block, which decrease by a factor of 2 in the first three blocks (down to 64 channels) and end with a convolution onto 3 channels in the last decoder block. The outputs of each down-convolution block in the encoder is given a skip-connection into the corresponding up-convolution block in the decoder (i.e., the output of the third down-convolution block is concatenated with the output of the first up-convolution block, which is the input of the second up-convolution block). Finally, we perform a tanh activation on the outputs of the last decoder block, representing the final model prediction.As mentioned in Section 2.2.2, generator G additionally has a second decoder which follows identical structure to the main decoder described above, including identical skipconnections coming from the encoder, except that its gradients do not flow into the encoder during backpropagation.Our discriminators D X , D Y , and D K consist of similar four-block encoders, described above, followed by a convolution layer (with a kernel size of 4, stride of 1, and padding of 1) onto a single output channel. The result is a patch-wise real/fake logit prediction.During inference, we achieve end-to-end scoring from the generated k = G K (x) by using a simple local maxima algorithm on the cell center point maps for instance detection, followed by an arg max operation for each detected instance across the binary type maps for cell expression prediction. We stitch together k before we run the extraction algorithm, which resolves prediction artifacts at the tile borders.To formally define the local maxima algorithm, we further define {( m, b)} ∈ k, where m is the center point distance map and b is the group of binary type segmentation masks. The local maxima algorithm can then be defined as:EQUATION)where δ is a given neighborhood size, ω is a given threshold, and (i, j) are indices of pixels within the center point distance map m. This is a common, simple 2D local maxima algorithm (e.g., (Brieu et al., 2019) ). We used the maximum_filter function implemented in the scipy package in Python and assigned a pixel as a cell instance if it is its own local maxima (within δ pixels) and if it exceeds the threshold ω. In our experiments, we used a neighborhood size δ of 25 and a threshold ω of 0.5.  
 
 Appendix:  


Decision: This paper received the final ratings with 3 weak accept. The authors addressed most of the comments during the rebuttal. Given the consensus of reviewers, AC recommends to accept the paper and strongly suggest the authors take the reviewers’ comments into the final revision.

Reviews: review: paper_summary: paper_summary: This paper aims to quantify percentage of ki67-positive cells in IHC slide by training a CycleGAN to convert samples between IHC tiles to/from synthetic data generated from H&E slides and cell segmentation labels. The authors demonstrated their approach is better than one unsupervised baseline in Ki67 scoring.
main_review: main_review: - The authors clarified that the main contribution is not a robust method that adapts to datasets from different centers, but instead an unsupervised solution that does not rely on annotations for good performance. This is a totally valid point that I misunderstood.
- The authors also added experiments that demonstrate the proposed approach is worse but comparable to DeepLIIF on an external datasets where annotations are available. This shows promise.
- The authors also point out the paper's clinical utility which I think is super important that is not in the paper but good to know such context.  I would think it's hard to know whether the proposed method is useful for other people, I would still want to encourage this kind of research where the project is motivated by a clear clinical need.

Given the above points, I raise the rating to 4.
strength_weakness: strength_weakness: + The paper over-claims that its approach can adapt to real-world imaging variabilities. The authors should add experiments demonstrating the approach works well on a wide range of IHC datasets, e.g., collected from other institutions, with different scanners, varying tissue qualities and staining protocols. If these experiments could not be done, the authors should avoid claiming that this is a main contribution of the paper.
+ Another concern is the lack of comparisons to supervised learning baseline, or any baseline that is finetuned on the in-house dataset that is used to train IHCScoreGAN. The authors should perhaps try to do such comparison on open source datasets where labels are available and provide additional baselines that at least is finetuned on the in-house data for fair comparison to the proposed approach.
questions: questions: Try to address points made in weaknesses.
paper_summary: paper_summary: Deep learning models that have been trained in a fully supervised fashion for Ki67 quantification rely on hard-to-get annotations and may be prone to overfitting on training data distribution. The authors propose a method to overcome these issues : using unlabeled IHC image samples and unpaired synthetic data, they train a Cycle-GAN-inspired network to precict cell expression. Their method outperforms a fully supervised baseline trained on another dataset and tested in a generalization setup, that is on their in-house data.
main_review: main_review: I gave the paper a rating of 4, which I will leave as it, with more confidence than before. I would like to emphasize that the authors put a great effort into answering the question that were addressed to them. They have added experiments and details that have strenghtened the quality of their work.
strength_weakness: strength_weakness: - The method is only evaluated on a private dataset, which restricts the reproducibility of the experiments.

- The proposed model is only compare to a single supervised other model from the litterature, in a generalization setup. It does not include comparison with fine tuning on few samples or comparison with other adaptation techniques.

- The authors insist that the model does not require annotations, and is thus adapted to overcome the limit of having distinct training and testing data distrubtions. However, it still required a collection of over 1,500 images to be trained, which could limit the use of the model to centers that have not acquired as much data.
questions: questions: - How were the values of the $\lambda$, $\beta$ and $\gamma$ loss scaling parameters determined ? Were they taken from a previous or determined by a grid search ? The information does not appear in the main text or in the appendices, although it seems important to me, given the number of terms in the cost function and the known instability of GANs training.

- You compare your method with a pre-trained fully supervised baseline, but not with any other domain adaptation methods. Other paradigms of training could be used to overcome the issue that you're tackling of obtaining annotations (few-shot/unsupervised/self-supervised adaptation, domain alignment, weak adaptations, image-level labels, etc.).

- You evaluate the performance of your model using two-split validation. Have you tried evaluating the robustness and uncertaincy of your model to multiple training ?
paper_summary: paper_summary: The authors propose a framework for label-free Ki67 assessment based on a GAN framework. This approach is claimed to significantly outperform a supervised state-of-the-art baseline (DeepLIIF). The authors provide additional experiments to further demonstrate the capabilities of their approach.
main_review: main_review: The authors have clarified all major points that were raised by my review. I still think that the paper could profit from a more thorough investigation against more competitive approaches, and some minor doubts remain. But the paper in it's current is sufficient to be presented at MIDL.
strength_weakness: strength_weakness: The main weakness of the paper is the evaluation. Given the strong claims made by the authors, I would like to see a more comprehensive evaluation, in particular:
- The baseline against which the authors compare their algorithm is the DeepLIIF algorithm by Ghahremani et al. which is essentially an IHC segmentation model that, unlike the authors' method, has been trained in a supervised fashion. This model is applied to the test data, which is an in-house dataset of the authors. Therefore, no adaptation to the target domain could take place and the domain gap between training and target domain is likely to influence the results. However, the authors' method was trained on the target domain. For a fair comparison, either both models should be trained and tested on the same domain, or the authors could use a second domain as a hold out test set.
- To substantiate the authors' results, I would like to see an evaluation based on public datasets. The paper on the baseline approach refers to such a dataset, which also contains cell-level annotations (https://link.springer.com/chapter/10.1007/978-3-030-59722-1_28). Perhaps the authors could use this dataset to train a supervised baseline as well.
- Furthermore, I would like the authors to perform the training of their algorithm in a cross validation scheme to provide more information about the robustness of the algorithm. So far, only results on a training and test split have been provided. A cross-validation would dispel the doubt that the model performance depends on the distribution of the training and test data. 

In the discussion, the authors could also elaborate on the results of Experiment 3.3. Why did the results of the baseline deteriorate even though it was not trained on the target domain?

Another weakness is the lack of details on the experimental design. How many pathologists were involved in the IHC scoring of the in-house dataset? Did they have to come to an agreement to establish a ground truth label? How were the 23 HE images for the target dataset selected from the TCGA?
questions: questions: - Please provide further evaluation on a second set of data.
- How does the authors' model perform on unseen data?
- If possible, please perform cross validation instead of single shot training.

Comments: title: Response
Response to the comments
Response to the review
Reviewer PGy6: Adding cross validation and external dataset experiments, adding baselines, and adding clarifications to the text.
Reviewer 5rTw: Adding sample size, multi-run and external dataset experiments, adding baselines, and addressing questions.
Reviewer kX4u: Adding baselines, external dataset experiments, and claim clarifications.
comment: I would like to thank the authors for having taken into account my recommandations and addressed my concerns with carefulness and quality.
I value the authors' responsiveness to the criticisms. I acknowledge that the paper has significantly improved during the review process and I will incorporate this assessment into my final evaluation.
*We again sincerely thank the reviewer for their useful and valuable comments.*

## RE: Question 1

We appreciate the reviewer’s comment and agree that this information is missing in the current version.  

Our internal testing data refers to a separate dataset that is independent of the dataset we used in our experiments. Therefore, our internal testing data does not overlap with any data used in our experiments.  

We propose to clarify this point in the camera-ready version: **“Each value was selected through a multi-run grid search in order to optimize the model’s quantitative performance on a separate internal test dataset which is distinct from the data used in the experiments in this work.”**

## RE: Question 2 

We sincerely appreciate reviewer’s input on our paper. We would like to further discuss our thoughts as below: 

We value the reviewer’s suggestion, therefore we added the 5-fold cross-validation results to our paper. Regarding the two-split experiments, we believe that it has very important for potential clinical applications: The first (i.e., random two-split) is to be used as a performance baseline while the second (chronological two-split) reflects the data drift issue that is common in a real clinical setting (this is based on the guidance from our expert pathologist). We want to show how our model and other pre-trained models perform in such a situation. 

We believe that both the two-split and cross-validation experiments are very valuable, so we included both. Per the reviewer’s further suggestion in this round, we are working on generating results for the cross-validation for the supervised baselines and will include the results in the camera-ready version of our paper upon acceptance.  

## RE: Question 3

We appreciate the reviewer's comments and we would like to clarify as below: 

We randomly selected those WSIs from the Breast Invasive Carcinoma (TCGA-BRCA) project’s data. The 23 were manually chosen to include a variety of cell sizes, structures, and shapes. Visually, we found those images varied to a certain extent, but we cannot confirm if they are from different laboratories based on the information from TCGA, and we did not select images with consideration to image variability or laboratory source. We qualitatively assessed the quality of the generated target dataset. TCGA-BRCA consists of 1,098 cases, each of which may consist of multiple tissue slides. Considering the training efficiency and usability of our method, we try to use a small sample size for generating the target dataset.  

To address the reviewer’s concerns, enhance our method’s reproducibility, and explain the selection process in more detail, we suggest providing target masks along with the released code, and additionally propose an update to the camera-ready version upon paper acceptance to the following:  

**“We randomly sampled H\&E tissue slides of $20\times$ magnification from the Breast Invasive Carcinoma project (TCGA-BRCA) and manually selected slides to represent a variety of cell sizes, structures, and shapes (i.e., to discourage the model from using these features to discriminate real vs. fake data), resulting in 23 slides.”**
Dear authors,

Thank you for the detailed responses to my comments. However, I still have some questions regarding the submission.

- Regarding the selection of hyperparameters in Appendix A.4, you state that “ Each value was selected through a multi-run grid search in order to optimize the model’s quantitative performance on our internal testing data.” Can you please clarify what you mean by internal testing data? Did you ensure that this data did not overlap with your actual test data split to prevent your model/hyperparameters from overfitting the test data? 
- Why do the authors still report the results of the two-split experiment (3.2) while they already have done a five-fold cross validation (B.4)? The results of the cross validation have a much higher statistical informativeness as the results are far less likely to be influenced by the data distribution of the training and test splits. The authors could also report the mean scores of the supervised baselines on the test sets of the five fold cross validation. 
- Regarding the selection of the 23 images from the TCGA, I see that you have added some information. However, I'm still concerned that the section (2.1.2) is missing crucial information. Since the TCGA is a large data repository with many datasets from different types of tumors, which datasets did you use to select the images? Did the images have a consistent visual appearance or did they appear to come from multiple laboratories? Why did the authors use only 23 images instead of an entire dataset? What was the motivation for using exactly 23 images?
*We would like to express our sincere acknowledgments and appreciation for reviewer’s efforts to review our paper. We value all the points made by the reviewer and addressed each of reviewer’s comments by adding multiple experiments, adding/editing content in the paper, providing further clarification in response.*

## RE: Weakness 1, 2; detailed comments 2; questions to address in rebuttal 1, 2. 

**The reviewer’s concerns are raised regarding lack of experimental comparisons by training and testing with sufficient baselines.**

We addressed reviewer’s concern by:
1) Adding experiments (see section 3.4, results in Table 3) with two fully supervised baselines, DeepLIIF (state-of-the-art) and UNet (one of the most commonly used) on the public dataset (i.e., BCData dataset, suggested by the reviewer), 
2) Adding results in Table 3 for our model trained in internal dataset and tested on the unseen data, 
3) Re-performing validation using the BCData-trained supervised models (DeepLIIF and the added baseline, UNet) for comparison on our internal dataset (see updated results in Table 1 and 2). 

Our method demonstrated far superior performance on our internal-dataset at the case-level than the two supervised baselines. In the external dataset cell counting experiments, although IHCScoreGAN does not perform better than the state-of-the-art supervised model DeepLIIF, the results are comparable; additionally, both ours and DeepLIIF outperform all other supervised baseline models reported in the BCData paper. In Appendix B.3., we added additional sample size experiments which suggest that our method may not show its optimal performance considering the small patch-level sample size provided in the external dataset. In comparison, the supervised models were given more than 100,000 annotated cell instances, and DeepLIIF also required synthetic MxIF images in order to train. 

## RE: Weakness 3, Question to address in the rebuttal 3. 

We addressed reviewer’s comments by adding five-fold cross-validation experiments; the results are shown in Appendix B.4 (to comply with the requirements for main body page limit of 8 pages). Considering that the experiments in Table 1 and 2 consist of different splits, and by adding the cross-validation experiments, we can confidently rule out the fact that our model performance is significantly dependent on the data distribution. 

## RE: Detailed comments 1. 

As per the reviewer’s comment, we have included all choices of hyperparameters, as well as the selection method, in Appendix A.4. 

## RE: "In the discussion, the authors could also elaborate on the results of Experiment 3.3. Why did the results of the baseline deteriorate even though it was not trained on the target domain?"

**Please note that the previous Section 3.3 has been merged into Section 3.2 due to space constraints caused by the additional experiments.** 

We observed that the results deteriorate because the images after 2018 are more darkly stained, which is a more significant difference from the BCData dataset (which the supervised baselines trained on) than in images before 2018. We addressed reviewer’s comment by adding our observation in the text: 

1) **Section 3.2:**  ‘...we observed that the after-2018 samples were more challenging due to an increase in background noise, such as darker stroma staining...’  

2) **Section 4:**  ‘We observed that the pre-trained DeepLIIF often erroneously segmented  
stroma tissue and missed/misclassified challenging cells on our dataset, likely resulting from  
the differences in its training data (Figure 2).’ 

## RE:  "Another weakness is the lack of details on the experimental design. How many pathologists were involved in the IHC scoring of the in-house dataset? Did they have to come to an agreement to establish a ground truth label? How were the 23 HE images for the target dataset selected from the TCGA?"

We added/edited the content to address reviewer’s comments:  
1) **Section 2.1.1:**  “All cases in this study are sourced from Mayo Clinic and collected from 2012 through 2023 primarily by 6 pathologists. The Ki67 slides were scanned at 20× (0.5μm/pixel) resolution by Aperio® AT scanners. Clinical diagnoses were performed on selected invasive tumor regions. Typically, one pathologist is involved in the clinical diagnosis for each case.” 
 
2) **Section 2.1.2:** “We selected 23 H&E tissue slides of 20× resolution such that a variety of cell sizes, structures, and shapes were represented, divided them into tiles of size 256 × 256, and sampled N_Mask = 10, 000 tiles from the tissue regions.”
## RE: Weakness 1, 2 and Question address in rebuttal 2 

To provide fair comparison with more baselines and improve reproducibility, we addressed the reviewer’s concerns by: 
1) Adding/updating our experiments, and 
2) Providing intention for publishing our code.  

We performed experiments on a public dataset instead of performing domain adaptation on our internal dataset since this provides better transparency and reproducibility alongside the publication of our code. 

We added an experiment using a public external breast cancer dataset, BCData, where we trained and evaluated our framework against two fully supervised baselines, DeepLIIF and U-Net, for cell counting (see Section 3.4, results in Table 3). We also added/updated experiments on our internal dataset: we updated the results in Tables 1 and 2 for DeepLIIF, now trained on BCData, for consistency across all of our experiments. We also added another supervised baseline, U-Net, also trained on BCData, to Tables 1 and 2 in order to provide more baselines to address reviewer’s concerns. Our method demonstrated far superior performance on our internal-dataset at the case-level than the two supervised baselines. In the external dataset cell counting experiments, although IHCScoreGAN does not perform better than the state-of-the-art supervised model DeepLIIF, the results are comparable; additionally, both ours and DeepLIIF outperform all baseline supervised models reported in the BCData paper. In Appendix B.3., we added additional sample size experiments which suggest that our method may not show its optimal performance considering the small patch-level sample size provided in the external dataset.

We also plan to publish our code, and added a placeholder link to the abstract: 

'Upon publication, our code will be made available at www.github.com.' 

This should significantly strengthen our reproducibility. Given the unsupervised nature of our model, any researcher with access to our codebase can effortlessly train our model on their own dataset, which offers another layer of transparency. 

## RE: Question address in rebuttal 1. 

We addressed this comment by: 
1) Adding the information in the paper, and 
2) Providing intention for publishing our code (see above). 

We added the following information to the main text:
1) **Section 2.3.:** "We use $\lambda=10$, $\beta=1$, and $\gamma=10$ in this paper, following CycleGAN."
2) **Appendix A.4.:** "Each value was selected through a multi-run grid search in order to optimize the model's quantitative performance on our internal testing data."

## RE: weakness 3, Question address in rebuttal 3: 

We addressed reviewer’s comments by:
1) Providing clarification, and 
2) Adding additional sample size experiments (see appendix B.3 and results in Table 6, appendix B.4).  

We regret that we were not clear about the data splitting. Our chronological data splits happened to consist of 1,532 cases before 2018, and we wanted to make the two-split experiment comparable, thus we split the data similarly. With added sample size experiment in appendix section B.3, Table 6, we provided a multi-run sample size experiment to demonstrate that we only need 1,000-2,000 images to achieve strong and relatively stable performance. In addition, the added 5-fold cross validation results in section B.4  also reflect the evaluation of the robustness and uncertainty of our model. We observed small standard deviation in both experiments in general. 

## Re: detailed comments1: 

The two branches constitute different tasks: the proxy task branch wants to construct a mask so that the original image can be reconstructed, while the secondary branch wants to find cell center points and cell expressions. If we do not detach the gradients of Decoder K, the reconstruction task becomes much more difficult due to competing goals; e.g., the proxy task branch wants to encode cell intensity to aid reconstruction, while the secondary branch does not want this information since it is orthogonal to cell expression prediction. 

## Re: detailed comments2: 

We performed new experiments in Table 3, where our model trained with internal data and test on the unseen public external data, as the reviewer suggested. Our results show our model performs moderately. To avoid confusion, we removed our speculation in the paper. 

## Re: detailed comments3: 

In our framework, we do not use an overlap, but instead keep the stride equal to the tile size; instead, we stitch together cell center point maps and cell expression masks before running our center point extraction algorithm, since nearby predictions from adjacent tiles are resolved by the local maxima algorithm. 

We added the following information to the paper to clarify:
1) **Appendix A.2.:** "We stitch together ˆk according to their spatial position before we run the extraction algorithm, which resolves prediction artifacts at the tile borders."
*We would like to express our sincere acknowledgments and appreciation for reviewer’s efforts to review our paper. We value all the points made by the reviewer and addressed each of reviewer’s comments.*

## RE: Weakness No.1: 

To address reviewer’s concern, we:
1) Clarified our statement, and 
2) Added experiments that validates our model on an external public dataset. 

**Clarification**

This is the specific statement we wrote originally: 

*"By removing ground truth requirements, our unsupervised technique constitutes an important step towards easily-trained, generalizable Ki67 scoring solutions that can adapt to real-world imaging variability and be further adapted to other cancer types or protein biomarkers."*

We did not claim that our method has already achieved such achievement, but instead said 'important step towards'. We intended to say that our method provided a new unsupervised approach without needing labels (i.e., domain adaptation in this case). To make this point clearer, we edited our statement in the paper to address reviewer’s concern regarding the overclaim.  

*"By removing ground truth requirements, our unsupervised technique constitutes an important step towards easily-trained, generalizable Ki67 scoring solutions which can train on out of domain data in an unsupervised manner."*

Although we insist that our original statement was not an overclaim, we agree with the reviewer that additional experiments would add value to our work. As such, we added substantial experiments to address reviewer’s concern on this point (see section 3.4; results in Table 3) using a public external dataset, for which all procedures for obtaining the image are drastically different from our internal dataset. Our model shows comparable performance with the state-of-the-art fully supervised method and substantially better performance than U-Net. In Appendix B.3., we added additional sample size experiments, which suggest that our method may not show its optimal performance considering the small patch-level sample size provided in the external dataset. In comparison, the supervised models were trained on more than 100,000 cell instances, and DeepLIIF also required synthetic MxIF images in order to train. We hope that these experiments aptly address the concerns raised by the reviewer. 

## RE: Weakness No.2: 

We addressed reviewer’s concern by following reviewer’s advice/suggestions, which by adding experiments (Section 3.4 and Table 3) on a public (i.e., open source) external dataset with two fully supervised models/baselines: DeepLIIF, which is the state-of-the-art on the BCData dataset; and U-Net, which is one of the most commonly used and rigorously studied models for cell segmentation for biomedical images.  

We also added/edited experiments for internal dataset, we updated the results in Table 1, 2 and 3 for DeepLIIF with model trained on the breast cancer dataset (BCData). We also added U-Net results, also trained on BCData, in Tables 1-3 to provide more baselines to address reviewer’s concern. Our method demonstrated far superior performance on our internal-dataset at case-level that the two supervised baselines; it also showed comparable performance to the fully supervised state-of-the-art and substantially better performance than the U-Net baseline on the open source dataset at cell-counting level; although, again, our method may not show its optimal performance considering its unsupervised methodology. 

Importantly, the focus of our paper is the clinical oriented task, so we focused on a single clinical test with the intention for integration for our clinical workflow (see paper title and content of our paper). We have no intention to showcase/prove that our model is the best model/a better model than other models after all. Instead, our experiments intend to show the gap between a pre-trained model for potential clinical use vs. our unsupervised approach. Based on the instruction from MIDL website, MIDL appreciates clinical use cases, and our paper links a strongly-performing method with substantial novelty to a clinical use case validation, thus the validation should be sound  on its own considering its excellent performance on a substantially large clinical dataset (also supported by the comment from reviewer No.2's comments on the strengths). In this case, our work should not be punished by its validation. However, to make sure we address reviewer’s concern, we still added substantial experiments (for both internal and external datasets) with strong performance.   

We also plan to publish our code; we added a placeholder link to the abstract: 

'Upon publication, our code will be made available at www.github.com.' 

Given the unsupervised nature of our model, any researcher with access to our codebase can effortlessly train our model on their own dataset, which offers another layer of transparency.