Original: Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological ImagesNAME MAIL ADDR
Revised: Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images

Original: MAIL

Original: ADDR

Original: Editor: Editor
Revised: Editor: TBA

Original: Abstract
Revised: Abstract

Original: Digital pathology is an emerging topic in the analysis of pathologic tissue samples.
Revised: Digital pathology is an emerging topic in the analysis of pathologic tissue samples.

Original: It includes providing the tools towards more automated workﬂows to derive clinically relevant information.
Revised: It includes providing the tools towards more automated workﬂows to derive clinically relevant information.

Original: Digitization and storage of whole slide images have become commonplace and allow modern image analysis methods to be used.
Revised: Digitization and storage of whole slide images have become commonplace and allow modern image analysis methods to be used.

Original: In recent years, computer-based segmentation of cell nuclei has gathered considerable attention in the development of highly specialized algorithms.
Revised: In recent years, computer-based segmentation of cell nuclei has gathered considerable attention in the development of highly specialized algorithms.

Original: Currently, most of these algorithms are based on performing semantic segmentation of all cell nuclei and separating overlapping instances in a post-processing step.
Revised: Currently, most of these algorithms are based on performing semantic segmentation of all cell nuclei and separating overlapping instances in a post-processing step.

Original: Recently, instance-aware segmentation methods such as Mask R-CNN have been proposed to enable uniﬁed instance detection and segmentation, even in overlapping cases.
Revised: Recently, instance-aware segmentation methods such as Mask R-CNN have been proposed to enable uniﬁed instance detection and segmentation, even in overlapping cases.

Original: In this work, we propose a modiﬁed Mask R-CNN-based approach by incorporating distance maps of instances and hematoxylin-stain intensities as extra input channels to the model.
Revised: In this work, we propose a modiﬁed Mask R-CNN-based approach by incorporating distance maps of instances and hematoxylin-stain intensities as extra input channels to the model.

Original: Moreover, we explore the impact of three well-known inference strategies, namely test-time augmentation, ensembling, and knowledge transfer through pre-training on the segmentation performance.
Revised: Moreover, we explore the impact of three well-known inference strategies, namely test-time augmentation, ensembling, and knowledge transfer through pre-training on the segmentation performance.

Original: We perform extensive ablation experiments  to quantitatively deﬁne the most optimal inference strategy in the proposed Mask R-CNN algorithm.
Revised: We perform extensive ablation experiments across multiple runs to quantitatively deﬁne the most optimal inference strategy in the proposed Mask R-CNN algorithm.

Original: Our results show that  instance segmentation improvements of up to 3.4% and 3.6% based on Aggregate Jaccard Index and Panoptic Quality score can be obtained, respectively, using the proposed techniques in comparison to a standard Mask R-CNN model.
Revised: Our results show that average instance segmentation improvements of up to 3.5% and 4.1% based on Aggregate Jaccard Index and Panoptic Quality score can be obtained, respectively, using the proposed techniques in comparison to a standard Mask R-CNN model.

Original: Our ﬁndings conﬁrm the eﬀectiveness of aggregating information at the network input stage and optimizing inference workﬂows using minimal eﬀort.
Revised: Our ﬁndings conﬁrm the eﬀectiveness of aggregating information at the network input stage and optimizing inference workﬂows using minimal eﬀort.

Original: Implemented modiﬁcations and codes are publicly available through a GitHub repository under: [LINK WITHHELD FOR REVIEW COPY].
Revised: Implemented modiﬁcations and codes are publicly available through a GitHub repository under: https://github.com/bbanc/Improved-Mask-R-CNN-for-nuclei-segmentation .

Original: 1. Introduction
Revised: 1. Introduction

Original: Digital pathology has a wide range of applications in the computer-based analysis of histological samples.
Revised: Digital pathology has a wide range of applications in the computer-based analysis of histological samples.

Original: For instance hematoxylin and eosin (H&E-) stained images are routinely digitized to be more easily accessible and to allow the use of image analysis methods to fa- cilitate diagnosis (Pallua et al.
Revised: For instance hematoxylin and eosin (H&E-) stained images are routinely digitized to be more easily accessible and to allow the use of image analysis methods to facilitate diagnosis (Pallua et al.

Original: ; Zarella et al.
Revised: ; Zarella et al.

Original: These semi- or fully-automatic methods can be used for various applications including nuclei instance segmentation.
Revised: These semi- or fully-automatic methods can be used for various applications including nuclei instance segmentation.

Original: Nuclei density, count, and the ratio of nucleus to cytoplasm are some of the main features used when performing cancer grading and deriving treatment plans in routine pathology (Aeﬀner et al.
Revised: Nuclei density, count, and the ratio of nucleus to cytoplasm are some of the main features used when performing cancer grading and deriving treatment plans in routine pathology (Aeﬀner et al.

Original: ; Skinner and Johnson (2017)).
Revised: ; Skinner and Johnson (2017)).

Original: Performing accurate and automatic nuclei instance segmentation in digitized histological images is a challenging task due to several reasons.
Revised: Performing accurate and automatic nuclei instance segmentation in digitized histological images is a challenging task due to several reasons.

Original: The existence of touching, or even heavily overlapping nuclei, inconsistent nuclei size and shape, and signiﬁcant variations in color distributions of nuclei in various samples are among the most common issues for the development of any automatic nuclei segmentation model (Kumar et al.
Revised: The existence of touching, or even heavily overlapping nuclei, inconsistent nuclei size and shape, and signiﬁcant variations in color distributions of nuclei in various samples are among the most common issues for the development of any automatic nuclei segmentation model (Kumar et al.

Original: ; Salvi et al.
Revised: ; Salvi et al.

Original: Besides these challenges, to use supervised machine learning approaches, ground truth data at pixel level are required to train the models.
Revised: Besides these challenges, to use supervised machine learning approaches, ground truth data at pixel level are required to train the models.

Original: Deriving these ground truth annotations is a non-trivial task due to both inter- and intra-observer variability in manually created annotations (Aeﬀner et al.
Revised: Deriving these ground truth annotations is a non-trivial task due to both inter- and intra-observer variability in manually created annotations (Aeﬀner et al.

Original: Kumar et al.
Revised: Kumar et al.

Original: ; Mahbod et al.
Revised: ; Mahbod et al.

Original: Recently, deep learning-based image analysis methods have shown promising success in the ﬁeld.
Revised: Recently, deep learning-based image analysis methods have shown promising success in the ﬁeld.

Original: Moreover, multiple nuclei instance segmentation (or classiﬁcation) challenges such as MoNuSeg2018 (Kumar et al.
Revised: Moreover, multiple nuclei instance segmentation (or classiﬁcation) challenges such as MoNuSeg2018 (Kumar et al.

Original: (2020)), Kaggle DSB2018 (Caicedo et al.
Revised: (2020)), Kaggle DSB2018 (Caicedo et al.

Original: and MoNuSAC2020 (Verma et al.
Revised: and MoNuSAC2020 (Verma et al.

Original: (2021)) challenges helped to create publicly available datasets and standard benchmarking for nuclei instance segmentation.
Revised: (2021)) challenges helped to create publicly available datasets and standard benchmarking for nuclei instance segmentation.

Original: In these challenges, highly specialized methods have been proposed to bridge the gap between purely semantic and instance-based segmentation methods.
Revised: In these challenges, highly specialized methods have been proposed to bridge the gap between purely semantic and instance-based segmentation methods.

Original: Many of these approaches are based on a deep learning encoder-decoder-based semantic segmentation algorithm and aim to generate additional information to then separate overlapping nuclei in the ﬁnal inference.
Revised: Many of these approaches are based on a deep learning encoder-decoder-based semantic segmentation algorithm and aim to generate additional information to then separate overlapping nuclei in the ﬁnal inference.

Original: Diﬀerent approaches to models include regression-based models (Graham et al.
Revised: Diﬀerent approaches to models include regression-based models (Graham et al.

Original: ; Naylor et al.
Revised: ; Naylor et al.

Original: ; Mahbod et al.
Revised: ; Mahbod et al.

Original: or ternary segmentation-based models (Chen et al.
Revised: or ternary segmentation-based models (Chen et al.

Original: Oda et al.
Revised: Oda et al.

Original: ; Zhou et al.
Revised: ; Zhou et al.

Original: In contrast,detection, and instance aware-based models directly detect and segment nuclei.
Revised: In contrast, instance aware-based models directly detect and segment nuclei.

Original: Mask R-CNN( He et al.
Revised: Mask RCNN (He et al.

Original:  one of the most well-known detection-based models is comprised of two main stages.
Revised: (2020)), one of the most well-known detection-based models, is comprised of two main stages.

Original: The ﬁrst performs object detection and localization, and the second stage uses the features of the detected regions of interest to jointly perform classiﬁcation, ﬁnal localization, and segmentation.
Revised: The ﬁrst performs object detection and localization, and the second stage uses the features of the detected regions of interest to jointly perform classiﬁcation, ﬁnal localization, and segmentation.

Original: A few adapted Mask R-CNN-based approaches have beenalso introduced in the literature to perform nuclei instance segmentation.
Revised: A few adapted Mask R-CNN-based approaches have been introduced in the literature for nuclei instance segmentation.

Original: Johnson (2018) adapted Mask R-CNN for the Kaggle DSB2018 and noted the impact of optimized training procedures and the use of deeper network backbones.
Revised: Johnson (2018) adapted Mask R-CNN for the Kaggle DSB2018 and noted the impact of optimized training procedures and the use of deeper network backbones.

Original: MACD-RCNN (Ma et al.
Revised: MACD-RCNN (Ma et al.

Original: (2020)) is an adaptation for abnormal nuclei detection using ﬁxed-size region proposals and an attention mechanism.
Revised: (2020)) is an adaptation for abnormal nuclei detection using ﬁxed-size region proposals and an attention mechanism.

Original: Nuclei R-CNN (Lv et al.
Revised: Nuclei R-CNN (Lv et al.

Original: (2019)) employs Mask R-CNN using a tile-based instance merging approach to infer on large image patches.
Revised: (2019)) employs Mask R-CNN using a tile-based instance merging approach to infer on large image patches.

Original: Vuola et al.
Revised: Vuola et al.

Original: (2019) proposed ensembling Mask R-CNN with U-Net as a way of harnessing the strengths of each of the two architectures.
Revised: (2019) proposed ensembling Mask R-CNN with U-Net as a way of harnessing the strengths of each of the two architectures.

Original: Recently, Mask R-CNN has also been shown to signiﬁcantly beneﬁt from the use of test time augmentation for instance segmentation (Moshkov et al.
Revised: Recently, Mask R-CNN has also been shown to signiﬁcantly beneﬁt from the use of test time augmentation for instance segmentation (Moshkov et al.

Original: In this work, we made use of the original Mask R-CNN architecture (He et al.
Revised: We modiﬁed the original Mask R-CNN architecture (He et al.

Original: and modiﬁed it for the nuclei instance segmentation task.
Revised: for the nuclei instance segmentation task.

Original: Our main contributions can be summarised as follows:
Revised: Our main contributions can be summarised as follows:

Original: (cid:136)
Revised: (cid:136)

Original: We propose a modiﬁed Mask R-CNN-based approach by incorporating the distance maps of instances and
Revised: We propose a modiﬁed Mask R-CNN-based approach by incorporating the distance maps of instances and

Original: hematoxylin-stain (H-stain) intensities as extra input channels.
Revised: hematoxylin-stain (H-stain) intensities as extra input channels.

Original: (cid:136)
Revised: (cid:136)

Original: We exploited three well-known inference strategies, namely test-time augmentation (TTA), ensembling, and knowledge transfer through pre-training to improve the segmentation performance.
Revised: We exploit three well-known inference strategies, namely test-time augmentation (TTA), ensembling, and knowledge transfer through pre-training to improve the segmentation performance.

Original: (cid:136)
Revised: (cid:136)

Original: We showcase an optimized use of Mask R-CNN in nuclei segmentation with minimal addition to the original Mask R-CNN architecture.
Revised: We showcase an optimized use of Mask R-CNN in nuclei segmentation with minimal addition to the original Mask R-CNN architecture.

Original: (cid:136)
Revised: (cid:136)

Original: We provide quantitative comparisons of the baseline Mask R-CNN architecture to its improved versions.
Revised: We provide quantitative comparisons of the baseline Mask R-CNN architecture to its improved versions.

Original: (cid:136)
Revised: (cid:136)

Original: We perform extensive ablations to accurately measure the impact of various changes on the Mask R-CNN segmentation performance.
Revised: We perform extensive ablations to accurately measure the impact of various changes on the Mask R-CNN segmentation performance.

Original: 2. Methods 2.1 Datasets
Revised: 2. Methods 2.1. Datasets

Original: To assess the quantitative performance of Mask R-CNN and its modiﬁed versions, we used the MoNuSeg2018 dataset (Kumar et al.
Revised: To assess the quantitative performance of Mask R-CNN and its modiﬁed versions, we used the MoNuSeg2018 dataset (Kumar et al.

Original: This dataset is composed of 30 training and 14 test images with close to 22,000 training and 7,000 test nuclei instances.
Revised: This dataset is composed of 30 training and 14 test images with close to 22,000 training and 7,000 test nuclei instances.

Original: All images are H&E-stained, acquired at 40 × magniﬁcation, and are fully manually annotated.
Revised: All images are H&E-stained, acquired at 40 × magniﬁcation, and are fully manually annotated.

Original: The dataset is composed of 1000 × 1000 pixel image tiles extracted from whole slide images of nine diﬀerent biopsy sites (breast, kidney, liver, prostate, bladder, colon, and stomach in the training set; breast, kidney, prostate, bladder, colon, lung, and brain in the test set).
Revised: The dataset is composed of 1000 × 1000 pixel image tiles extracted from whole slide images of nine diﬀerent biopsy sites (breast, kidney, liver, prostate, bladder, colon, and stomach in the training set; breast, kidney, prostate, bladder, colon, lung, and brain in the test set).

Original: In addition to the MoNuSeg2018 dataset, model pre-training and knowledge transfer were explored through the PanNuke dataset (Gamper et al.
Revised: In addition to the MoNuSeg2018 dataset, model pre-training and knowledge transfer were explored through the PanNuke dataset (Gamper et al.

Original: PanNuke is a semi-automatically annotated segmentation and classiﬁcation dataset consisting of over 200,000 nuclei across 19 diﬀerent organ sites (bladder, ovaries, pancreas, thyroid, liver, testes, prostate, stomach, kidney, adrenal gland, skin, head and neck, cervix, lung, uterus, esophagus, bile-duct, colon, and breast) at 40 × magniﬁcation.
Revised: PanNuke is a semi-automatically annotated segmentation and classiﬁcation dataset consisting of over 200,000 nuclei across 19 diﬀerent organ sites (bladder, ovaries, pancreas, thyroid, liver, testes, prostate, stomach, kidney, adrenal gland, skin, head and neck, cervix, lung, uterus, esophagus, bile-duct, colon, and breast) at 40 × magniﬁcation.

Original: It contains more than 7, image patches with a ﬁxed size of 256 × 256 pixels.
Revised: It contains more than 7, image patches with a ﬁxed size of 256 × 256 pixels.

Original: Further details of the MoNuSeg and PanNuke datasets can be found in their respective publications.
Revised: Further details of the MoNuSeg and PanNuke datasets can be found in their respective publications.

Original: 2.2 Modifying Mask R-CNN
Revised: 2.2. Modifying Mask R-CNN

Original: An emerging theme in current literature is the use of additionally derived information such as distance (Graham et al.
Revised: An emerging theme in current literature is the use of additionally derived information such as distance (Graham et al.

Original: ; Mahbod et al.
Revised: ; Mahbod et al.

Original: or hematoxylin channel (Hchannel)
Revised: or hematoxylin channel (Hchannel)

Original: (Zhao et al.
Revised: (Zhao et al.

Original: information to improve model performance.
Revised: information to improve model performance.

Original: Inspired by these trends, we propose to extend Mask R-CNN by adding extra input channels to allow the model to make use of H-stain intensity information and estimated distance maps derived from a Euclidean distance transform.
Revised: Inspired by these trends, we propose to extend Mask R-CNN by adding extra input channels to allow the model to make use of H-stain intensity information and estimated distance maps derived from a Euclidean distance transform.

Original: All our experiments were based on the open-source
Revised: All our experiments were based on the open-source

Original: Matterport implementation of Mask R-CNN (Abdulla (2017)).
Revised: Matterport implementation of Mask R-CNN (Abdulla (2017)).

Original: H-stain intensity was estimated by using the color deconvolution approach proposed by (Ruifrok et al.
Revised: H-stain intensity was estimated by using the color deconvolution approach proposed by (Ruifrok et al.

Original: Inspired by the Beer-Lambert law of light absorption it aims to estimate the relative contribution of a certain stain in an RGB image and separates the various stain contributions to the observed color using color deconvolutions.
Revised: Inspired by the Beer-Lambert law of light absorption it aims to estimate the relative contribution of a certain stain in an RGB image and separates the various stain contributions to the observed color using color deconvolutions.

Revised: It captures the contributions of the hematoxylin component of the stain and therefore yields a clear intensity map encoding the positions and shapes of nuclei within the original RGB-image.

Original: The Euclidean distance map was estimated using a distance U-Net similar to the model proposed in (Mahbod et al.
Revised: The Euclidean distance map was estimated using a distance U-Net similar to the model proposed in (Mahbod et al.

Original: In analogy to (Mahbod et al.
Revised: In analogy to (Mahbod et al.

Original: , we used a network with ﬁve max-pooling layers in the encoder part and ﬁve transposed convolutional layers in the decoder part.
Revised: , we used a network with ﬁve max-pooling layers in the encoder part and ﬁve transposed convolutional layers in the decoder part.

Original: Moreover, we used dropout layers between all convolutional layers in the extracting and expanding paths with a probability of 0.1.
Revised: Moreover, we used dropout layers between all convolutional layers in the extracting and expanding paths with a probability of 0.1.

Original: Except for the last layer where a linear activation function was used, ReLu activations were utilized for all other convolutional layers.
Revised: Except for the last layer where a linear activation function was used, ReLu activations were utilized for all other convolutional layers.

Revised: The distance map captures the distance of a pixel to its nearest nonzero pixel in a binary mask.

Revised: Previous research has shown such contextual information to be a powerful tool in nuclei instance segmentation.

Original: The additional estimated H-channel and Euclidean distance information were added as prior information to the RGB channels of the input image.
Revised: The additional estimated H-channel and Euclidean distance information were added as prior information to the RGB channels of the input image.

Original: This ﬁve-channel input data was then projected onto a standard three-channel input using a 1 × 1 convolutional layer.
Revised: This ﬁve-channel input data was then projected onto a standard three-channel input using a 1 × 1 convolutional layer.

Original: We refer to this extension of Mask R-CNN as Mask R-CNN+ henceforth.
Revised: We refer to this extension of Mask R-CNN as Mask R-CNN+ henceforth.

Original: The generic ﬂowchart of Mask R-CNN+ is depicted in Fig.
Revised: The generic ﬂowchart of Mask R-CNN+ is depicted in Figure 1.

Original: 2.3 Incorporating three inference strategies
Revised: 2.3. Incorporating three inference strategies

Original: We explored and quantitatively assessed the impact of three inference strategies on the
Revised: We explored and quantitatively assessed the impact of three inference strategies on the

Original: Mask R-CNN+ nuclei instance segmentation performance.
Revised: Mask R-CNN+ nuclei instance segmentation performance.

Original: We used TTA, ensembling, and knowledge transfer through pre-training.
Revised: We used TTA, ensembling, and knowledge transfer through pre-training.

Original: Both TTA and ensembling are multiple inference methods and require their respective outputs to be merged into a single uniﬁed inference.
Revised: Both TTA and ensembling are multiple inference methods and require their respective outputs to be merged into a single uniﬁed inference.

Original: This is a non-trivial task in instance segmentation (Kumar et al.
Revised: This is a non-trivial task in instance segmentation (Kumar et al.

Original: Traditionally, it is performed using a non-maximum suppression scheme.
Revised: Traditionally, it is performed using a non-maximum suppression scheme.

Original: Non-maximum suppression in instance segmentation can be formulated as an algorithm, which ﬁrst identiﬁes which segmentations belong to the same detection (e.g. according to ground truth criteria, such as using an intersection over union metric), then uses a score to rank the matching segmentations and ﬁnally removes all but the best scoring one.
Revised: Non-maximum suppression in instance segmentation can be formulated as an algorithm, which ﬁrst identiﬁes which segmentations belong to the same detection (e.g. according to ground truth criteria, such as using an intersection over union metric), then uses a score to rank the matching segmentations and ﬁnally removes all but the best scoring one.

Original: We adapted this algorithm to suppress all singular detections (e.g. during ensembling a detection is suppressed if it is only seen by a single model) and merged all segmentations belonging to the same detection using average voting, as is commonly used in semantic segmentation ensembling.
Revised: We adapted this algorithm to suppress all singular detections (e.g. during ensembling a detection is suppressed if it is only seen by a single model) and merged all segmentations belonging to the same detection using average voting, as is commonly used in semantic segmentation ensembling.

Original: Ensembling was set up to be organ-based by splitting the full MoNuSeg2018 training dataset into folds by their primary tumor site.
Revised: Ensembling was set up to be organ-based by splitting the full MoNuSeg2018 training dataset into folds by their primary tumor site.

Original: This resulted in ﬁve distinct models trained on a combination of breast, kidney, liver, prostate, and a mixture of bladder, colon, and stomach samples.
Revised: This resulted in ﬁve distinct models trained on a combination of breast, kidney, liver, prostate, and a mixture of bladder, colon, and stomach samples.

Original: Their inferences were then aggregated using our adapted instance merging scheme.
Revised: Their inferences were then aggregated using our adapted instance merging scheme.

Original: The overview of the utilized ensembling approach is shown in Fig.
Revised: The overview of the utilized ensembling approach is shown in Figure 2.

Original: Recently, TTA has been shown to consistently improve model performance in nuclei segmentation (Moshkov et al.
Revised: Recently, TTA has been shown to consistently improve model performance in nuclei segmentation (Moshkov et al.

Original: Here, TTA was performed by augmenting the original input three times.
Revised: Here, TTA was performed by augmenting the original input three times.

Original: Horizontal and vertical ﬂips, as well as a Hue saturation augmented image were used as augmented images.
Revised: Horizontal and vertical ﬂips, as well as a Hue saturation augmented image were used as augmented images.

Original: The resulting instance segmentations were then merged using our adapted instance merging scheme.
Revised: The resulting instance segmentations were then merged using our adapted instance merging scheme.

Original: An overview of TTA is depicted in Fig.
Revised: An overview of TTA is depicted in Figure 3.

Original: Model pre-training and knowledge transfer were explored through the PanNuke dataset.
Revised: Model pre-training and knowledge transfer were explored through the PanNuke dataset.

Original: To make full use of the information provided, we trained a standard Mask R-CNN model on the joint segmentation and classiﬁcation task.
Revised: To make full use of the information provided, we trained a standard Mask R-CNN model on the joint segmentation and classiﬁcation task.

Original: Model weights acquired through pre-training on the natural image COCO (Lin et al.
Revised: Model weights acquired through pre-training on the natural image COCO (Lin et al.

Original: dataset were further trained for 30 epochs on the entire PanNuke dataset and later ﬁne-tuned on the MoNuSeg2018 training set.
Revised: dataset were further trained for 30 epochs on the entire PanNuke dataset and later ﬁne-tuned on the MoNuSeg2018 training set.

Original: The overall pre-training and ﬁne-tuning scheme is depicted in Fig.
Revised: The overall pre-training and ﬁne-tuning scheme is depicted in Figure

Original: 2.4 Experimental Setup
Revised: 2.4. Experimental Setup

Original: To measure the instance segmentation performance, we used Aggregate Jaccard Index (AJI),
Revised: To measure the instance segmentation performance, we used Aggregate Jaccard Index (AJI),

Original: Panoptic Quality (PQ) score, Detection Quality (DQ) score (or F1), Segmentation Quality (SQ) score (average SQ of all true positive detections), and standard Dice score (Kumar et al.
Revised: Panoptic Quality (PQ) score, Detection Quality (DQ) score (or F1), Segmentation Quality (SQ) score (average SQ of all true positive detections), and standard Dice score (Kumar et al.

Original: ; Kirillov et al.
Revised: ; Kirillov et al.

Revised: In order to capture model performance and measure variability in training we performed the same experiment ﬁve times and report the average results across these runs.

Original: To train the distance U-Net model (see Section 2.2), we used randomly initiated weights using the He normal initializer (He et al.
Revised: To train the distance U-Net model (see Section 2.2), we used randomly initiated weights using the He normal initializer (He et al.

Original: As we dealt with a regression task, we used a mean square error loss function.
Revised: As we dealt with a regression task, we used a mean square error loss function.

Original: Adam optimizer (Kingma and Ba (2015)) was employed to train the network.
Revised: Adam optimizer (Kingma and Ba (2015)) was employed to train the network.

Original: The initial learning rate was set to 0.001 with a learning scheduler that divided the learning rate by 10 after every 25 epochs.
Revised: The initial learning rate was set to 0.001 with a learning scheduler that divided the learning rate by 10 after every 25 epochs.

Original: We trained the model for 80 epochs with a batch size of 2 to ﬁt the full resolution images to the model.
Revised: We trained the model for 80 epochs with a batch size of 2 to ﬁt the full resolution images to the model.

Original: We used the entire MoNuSeg2018 training set for training and predicted the distance maps for the
Revised: We used the entire MoNuSeg2018 training set for training and predicted the distance maps for the

Original: MoNuseg2018 test images.
Revised: MoNuseg2018 test images.

Original: To train Mask R-CNN-based models, we used specialized training schedules as suggested in (Hollandi et al.
Revised: To train Mask R-CNN-based models, we used specialized training schedules as suggested in (Hollandi et al.

Original: The utilized training schedule for Mask R-CNN and Mask RCNN+ targets diﬀerent stages of the network at diﬀerent stages of the training.
Revised: The utilized training schedule for Mask R-CNN and Mask RCNN+ targets diﬀerent stages of the network at diﬀerent stages of the training.

Original: We adopted this idea and therefore trained our models in four stages:
Revised: We adopted this idea and therefore trained our models in four stages:

Original: The ﬁrst stage only tunes the 1 × 1 input convolutional layer and output layers while keeping the backbone ﬁxed to enable knowledge transfer.
Revised: The ﬁrst stage only tunes the 1 × 1 input convolutional layer and output layers while keeping the backbone ﬁxed to enable knowledge transfer.

Original: The second is the main training stage optimizing all network parameters.
Revised: The second is the main training stage optimizing all network parameters.

Original: The ﬁnal two stages consist of ﬁne-tuning the ﬁnal back bone-stages and output layers, and ﬁnally training the output layers only.
Revised: The ﬁnal two stages consist of ﬁne-tuning the ﬁnal back bone-stages and output layers, and ﬁnally training the output layers only.

Original: For all Mask R-CNN models, we used a network pre-trained on natural images from the
Revised: For all Mask R-CNN models, we used a network pre-trained on natural images from the

Original: COCO dataset (Lin et al.
Revised: COCO dataset (Lin et al.

Original: The loss was the same as proposed in the original Mask
Revised: The loss was the same as proposed in the original Mask

Original: R-CNN implementation (He et al.
Revised: R-CNN implementation (He et al.

Original: We used an Adam optimiser (Kingma and Ba (2015)) with an initial learning rate of 0.0001.
Revised: We used an Adam optimiser (Kingma and Ba (2015)) with an initial learning rate of 0.0001.

Original: For optimized learning, this learning rate was multiplied by 2, 1, 0.5, and 0.1 for their respective learning stages.
Revised: For optimized learning, this learning rate was multiplied by 2, 1, 0.5, and 0.1 for their respective learning stages.

Original: The ground truth distance maps were used during Mask R-CNN training.
Revised: The ground truth distance maps were used during Mask R-CNN training.

Original: During inference, the estimated distance maps were combined with the standard input data and extracted H-channel information for the ﬁve-channel image input.
Revised: During inference, the estimated distance maps were combined with the standard input data and extracted H-channel information for the ﬁve-channel image input.

Original: Data augmentation during training included random brightness contrast changes, random rotations, as well as horizontal and vertical ﬂips, and random crops of 256 × 256 pixel to ﬁt the GPU-memory. 
Revised: Data augmentation during training included random brightness contrast changes, random rotations, as well as horizontal and vertical ﬂips, and random crops of 256 × 256 pixels to ﬁt the GPU-memory. We performed a

Revised: Wilcoxon signed-rank test to deﬁne statistical signiﬁcance between our best implementation and Mask R-CNN (Gibbons and Chakraborti (2011)).

Original: 3. Results
Revised: 3. Results

Original: We propose an improved Mask R-CNN-based approach by incorporating distance and Hstain intensities and oﬀer comparisons to a baseline Mask R-CNN approach.
Revised: We propose an improved Mask R-CNN-based approach by incorporating distance and Hstain intensities and oﬀer comparisons to a baseline Mask R-CNN approach.

Original: Toquantitatively evaluate the impact of our alternate inference strategies, we ﬁrst perform singular tests and then used extensive ablations to deﬁne the best combination for the optimized segmentation performance.
Revised: To evaluate the impact of our alternate inference strategies, we ﬁrst perform singular tests and then used extensive ablations to deﬁne the best combination for the optimized segmentation performance.

Original: In all experiments, the models were trained on the MoNuSeg training set and evaluated on the test set.
Revised: In all experiments, the models were trained on the MoNuSeg2018 training set and evaluated on the test set.

Original: We also provide the average values for seen and unseen organ types separately to better assess the model’s generalization capabilities in the 
Revised: We also provide the average values for seen and unseen organ types separately to better assess the model’s generalization capabilities in the Appendix.

Original: Appendix.
Revised: Table 1 shows the average results across ﬁve runs of the proposed methods and inference adaptations on the entire test set in terms of AJI, PQ, DQ, SQ, and DICE scores. Figure 6 shows a graphic representation of these results.

Original: Table 1 shows the average results of the proposed methods and inference adaptations on the entire test set in terms of AJI, PQ, DQ, SQ, and DICE scores.
Revised: The best method shown in bold was found to be statistically diﬀerent to Mask R-CNN using the Wilcoxon signed-rank test (AJI: p = 0 . 002, PQ: p = 0 . 001, DQ: p = 0 . 008, SQ and DICE:

Original:  [Table]
Revised: p < 0 . [Table] [Table] [Table]

Original: CNN
Revised: [Table] [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: Mask R-CNN+ \ w Ensemble
Revised: [Table] [Table] [Table] [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: Mask [Table]
Revised: [Table]

Original: TTA

Original: [Table]
Revised: [Table]

Original: Mask

Original: [Table]
Revised: [Table]

Original: CNN

Original: [Table][Table]
Revised: [Table]

Original: PanNuke

Original: [Table]
Revised: [Table]

Original: Mask

Original: [Table]
Revised: [Table]

Original: CNN

Original: [Table]
Revised: [Table]

Original: w PanNuke

Original: [Table]
Revised: [Table]

Original: Ensemble

Original: [Table]
Revised: [Table]

Original: Mask

Original: [Table]
Revised: [Table]

Original: CNN

Original: [Table]
Revised: [Table]

Original: \ w PanNuke + TTA

Original: [Table]
Revised: [Table]

Original: Mask

Original: [Table]
Revised: [Table]

Original: TTA

Original: [Table]
Revised: [Table]

Original: Mask R

Original: [Table]
Revised: [Table]

Original: CNN+ \ w Ensemble + TTA + PanNuke

Original: [Table]
Revised: [Table]

Original: 4. Discussion
Revised: 4. Discussion

Original: From the results in Table 1, we conclude that providing additional information and adapting inference methods in Mask R-CNN improves performance in terms of AJI, PQ, and DICE. Analyzing the components of the PQ-score, i.e. DQ and SQ, reveals that a major part of the improvements comes from an optimization of the nuclei detection (DQ/F1-score), while the SQ stays near constant.
Revised: From the results in Table 1, we conclude that providing additional information and adapting inference methods in Mask R-CNN improves performance in terms of all measured scores. In the Appendix, Figure 6 shows considerable variability in model performance due to random initialization of untrained weights, random batches, and augmentation eﬀects on training convergence.

Original: In terms of adapted inference methods, we observed improvements across all explored modalities.
Revised: In terms of adapted inference methods, we observed improvements across all explored modalities.

Original: Model pre-training and knowledge transfer were found to be the most impactful adaptation to our inference scheme.
Revised: A combination of model pre-training, knowledge transfer, and TTA were found to be the most impactful adaptation to our inference scheme.

Original: Ensembling through utilizing a set of organ-based folds and TTA both require multiple inference and merging of detected nuclei at the instance level.
Revised: Ensembling through utilizing a set of organ-based folds and TTA both require multiple inference and merging of detected nuclei at the instance level.

Original: Out of these two methods, TTA was found to be more robust, as it increases the DQ across all experiments. The highest PQ was achieved using a combination ofall inference improvements (model pre-training, TTA, and ensembling), while this combination’s AJIis markedly lower when compared to the PanNuke and TTA combined one.
Revised: Out of these two methods, ensembling increased the instance segmentation performance more drastically, both in terms of AJI and PQ score.

Original: This marks the importance of relying across multiple measurements to accurately assess model performance without becoming overreliant on a single measurement.
Revised: The PQ score is most notably improved by using multiple inference methods.

Original: The best performing model of the MoNuSeg2018 challenge achieved an AJI of 69.07%.
Revised: The best performing model of the MoNuSeg2018 challenge achieved an AJI of 69.07%.

Original: This is a superior result to our proposed optimized method, which reached a score of 64.45%.
Revised: This is a superior result to our proposed optimized method, which reached an average score of 64.43%.

Revised: To test for statistical signiﬁcance we used a Wilcoxon signed-rank test and found our method to be statistically diﬀerent on average.

Original: The superior methods in the MoNuSeg2018 challenge include highly specialized algorithms and deep learning architectures, such as the winner of the challenge (Zhou et al.
Revised: The superior methods in the MoNuSeg2018 challenge include highly specialized algorithms and deep learning architectures, such as the winner of the challenge (Zhou et al.

Original: Their architecture introduces several new and highly adapted workﬂows to reach a ﬁnal instance segmentation result, including customized loss functions to better incorporate the rare highly complex cases, new encoder block paths, and information aggregation modules combined in a novel architecture.
Revised: Their architecture introduces several new and highly adapted workﬂows to reach a ﬁnal instance segmentation result, including customized loss functions to better incorporate the rare highly complex cases, new encoder block paths, and information aggregation modules combined in a novel architecture.

Original: Our scores, however, were achieved using Mask R-CNN with minimal adaptations to the original architecture and an optimized inference strategy.
Revised: Our scores, however, were achieved using Mask R-CNN with minimal adaptations to the original architecture and an optimized inference strategy.

Original: It is worth mentioning that the reported inter-observer variability of the challenge was 65.3% (95%CI 63.9-66.7), which is very close to the best results derived in our proposed approach.
Revised: It is worth mentioning that the reported inter-observer variability of the challenge was 65.3% (95%CI 63.9-66.7), which is very close to the best results derived in our proposed approach.

Original: To further improve the segmentation performance, a number of additional techniques can be used that are not addressed in this study.
Revised: To further improve the segmentation performance, a number of additional techniques can be used, that are not addressed in this study.

Original: These include adapting Mask R-CNN’s loss functions to better reﬂect the unique complexities in nuclei segmentation.
Revised: These include adapting Mask R-CNN’s loss functions to better reﬂect the unique complexities in nuclei segmentation.

Original: Specialized approaches, that have been utilized for semantic segmentation networks, include penalizing sharp edges with high curvature to help segment common round nuclei shapes (Wang et al.
Revised: Specialized approaches for semantic segmentation networks include penalizing sharp edges with high curvature to help segment common round nuclei shapes (Wang et al.

Original: (2020)) or using an adapted loss function to put more emphasis on diﬃcult or rare sample cases (Zhou et al.
Revised: or using an adapted loss function to put more emphasis on diﬃcult or rare sample cases (Zhou et al.

Original: Other algorithms such as Graham et al.
Revised: Graham et al.

Original: (2019) propose a tiling-based strategy to perform inference on overlapping image tiles.
Revised: (2019) propose a tiling-based strategy to perform inference on overlapping image tiles.

Original: During this process, a large input image is tiled into multiple overlapping smaller ones and only the detection in the non-overlap regions are kept during inference.
Revised: During this process, a large input image is tiled into multiple overlapping smaller ones and only the detection in the non-overlap regions are kept during inference.

Original: This, however, was found to not increase model performance in our experiments.
Revised: This, however, was found to not increase model performance in our experiments.

Original: Another highly debated topic in the analysis of histological images is the use of stain normalization or color normalization.
Revised: Another highly debated topic in the analysis of histological images is the use of stain normalization or color normalization.

Original: Salvi et al.
Revised: Salvi et al.

Original: (2021) concluded an improvement of model performance for the use in classiﬁcation and detection tasks.
Revised: (2021) concluded an improvement of model performance for the use in classiﬁcation and detection tasks.

Original: Although in this work, we did not perform extensive experiments for color normalization, a standard stain normalization technique (Macenko et al.
Revised: Although in this work, we did not perform extensive experiments for color normalization, a standard stain normalization technique (Macenko et al.

Original: (2009)) did not yield a performance improvement.
Revised: (2009)) did not yield a performance improvement.

Original: Inference time is a metric not included in our experiments.
Revised: Inference time is a metric not included in our experiments.

Original: It is however highly important when deploying segmentation algorithms into a digital pathology-based workﬂow.
Revised: It is however highly important when deploying segmentation algorithms into a digital pathology-based workﬂow.

Original: For this consideration, multiple inference techniques, such as ensembling or TTA need to be carefully considered, as they may lead to a signiﬁcant increase in the inference time on the same hardware.
Revised: For this consideration, multiple inference techniques, such as ensembling or TTA need to be carefully considered, as they may lead to a signiﬁcant increase in the inference time on the same hardware.

Original: However, if additional hardware is available, multiple inference lends itself well to parallelization leaving only the result aggregation as a ﬁnal task.
Revised: However, if additional hardware is available, multiple inference lends itself well to parallelization leaving only the result aggregation as a ﬁnal task.

Original: 5. Conclusion
Revised: 5. Conclusion

Original: We conclude that Mask R-CNN is a powerful algorithm for nuclei instance segmentation.
Revised: Mask R-CNN is a powerful algorithm for nuclei instance segmentation.

Original: It can strongly beneﬁt from additional information at the input stage and optimized  workﬂows such as TTA and pre-training.
Revised: It can strongly beneﬁt from additional information at the input stage and optimized inference workﬂows such as TTA, ensembling, and pre-training.

Original: Further research is needed to continue to explore and improve the performance of detection and instance-aware models such as Mask R-CNN. 
Revised: Further research is needed to continue to explore and improve the performance of detection and instance-aware models such as Mask R-CNN. 6. Acknoledgements

Revised: The authors would like to thank the Austrian Research Promotion Agency ( Forschungsf¨orderungsgesellschaft - FFG) for funding this research as part of the Deep nuclei analysis project under FFG No.

Revised: The authors are grateful for the help of the Research & Development team at

Revised: TissueGnostics GmbH.

Original: Appendix A. Appendix
Revised: Appendix A. Appendix

Original: Table 2 shows the generalization capabilities of Mask R-CNN+ split on seen and unseen test organs.
Revised: Table 2 shows the generalization capabilities of Mask R-CNN+ split on seen and unseen test organs.

Original: AJI
Revised: [Table]

Original: PQ
Revised: Standard Mask R-CNN Seen average 60.

Original: DQ/F SQ DICE
Revised: [Table] [Table]

Revised: Unseen average 61.

Original: Mask R-CNN+
Revised: Mask R-CNN+

Original: Unseen avg 61. Seen avg 60.
Revised: Seen average 61. Unseen average 64.

Original: Ensemble
Revised: Ensemble

Original: Unseen avg 61. Seen avg 63.
Revised: Seen average 61. Unseen average 65.

Original: TTA
Revised: TTA

Original: Unseen avg 62. Seen avg 61. 
Revised: Seen average 61. Unseen average 65.

Original: PanNuke
Revised: PanNuke

Original:  Unseen avg 64.Seen avg 63.
Revised: Seen average 62. Unseen average 64.

Original: Ensemble + Pannuke
Revised: Ensemble + Pannuke

Original: Unseen avg 63. Seen avg 64.
Revised: Seen average 63. Unseen average 66.

Original: Ensemble + TTA
Revised: Ensemble + TTA

Original: Unseen avg 63. Seen avg 61.
Revised: Seen average 62. Unseen average 66.

Original: TTA + PanNuke
Revised: TTA + PanNuke

Original: Unseen avg 65.

Original: Seen avg 64.
Revised: Seen average 62.

Original: Ensemble + PanNuke + TTA

Original: Unseen avg 64.

Original: Seen avg 63.

Original: [Table] [Table]
Revised: [Table] [Figure]

Original: [Table]
Revised: Figure 5 shows ablations to derive the use of additional information in the form of extracted H-stain and estimated distance information. Figure 6 shows the results of the 5 runs in a chart with standard deviation across runs.

Original: [Table]
Revised: Figure 7 shows sample results of inferences on the validation set.