Nuclei Segmentation in Histopathological Images with  Enhanced U-Net3+

Authors: bishal ranjan swain
kyung joo cheoi
jaepil  ko
r  swain
cheoi  swain
  ko
Venue: MIDL_io_2024_Conference
Published: 2024-01-30 14:55:54

Abstract: In the rapidly evolving field of nuclei segmentation, there is an increasing trend towards developing a universal segmentation model capable of delivering top-tier results across diverse datasets. While achieving this is the ultimate goal, we argue that such a model should also outperform dataset-specific specialized models. To this end, we propose a task-specific feature sensitive U-Net model, that sets a baseline standard in segmentation of nuclei in histopathological images. We meticulously select and optimize the underlying U-Net3+ model, using adaptive feature selection to capture both short- and long-range dependencies. Max blur pooling is included to achieve scale and position invariance, while DropBlock is utilized to mitigate overfitting by selectively obscuring feature map regions. Additionally, a Guided Filter Block is employed to delineate fine-grained details in nuclei structures. Furthermore, we apply various data augmentation techniques, along with stain normalization, to reduce inconsistencies and thus resulting in significantly outperforming the state-of-the-art performance and paving the way for precise nuclear segmentation essential for cancer diagnosis and possible treatment strategies.

Main paper: 
 introduction: 
 Nuclei Segmentation of histopathological images play an important role in medical diagnostics of Hematoxylin and Eosin (H&E) stained tissues as it aids pathologists in understanding morphology of cellular structures (Clayton, 1991; Elston and Ellis, 1991) . This not only helps in cancer grading but also in predicting the effectiveness of various treatments (Kumar et al., 2017) . Finding and segmenting individual nuclei in histopathological images is one of the crucial steps involved in numerous analytical procedures of cancer diagnosis (Cui et al., 2019; Imtiaz et al., 2023) . However, segmentation of nuclei in histopathological images is challenging as there are large variations in color, texture, and shape. Moreover, the variability in staining procedures used during the image procurement processes further increase inconsistencies in the images (Sampias and Rolls) .Recent advances have seen the rise of convolutional neural network (CNN)-based deep learning models that have started to gain attention due to their robust and consistent performance in semantic segmentation (Zhang et al., 2020) . Among these, U-Net (Ronneberger et al., 2015) has emerged as the most influential architecture, introducing encoder-decoder pooling (Zhang, 2019) to solve scale and position invariance and anti-aliasing problems that are inherently found in encoder-decoder architectures, utilize DropBlock (Ghiasi et al., 2018) to mitigate overfitting by selectively obscuring feature map regions and using a Guided Filter Block (Wu et al., 2019) to delineate fine-grained details in nuclei structures. Furthermore, we perform several pre-processing on the data including stain normalization to account for staining inconsistencies and data augmentation techniques that are meticulously tailored to address the inherent variability in the pathological H&E stained images. Through this specialized approach, our model outperforms the current state-of-the-art models by some margin in nuclei segmentation of histopathological images. 
dataset: 
 This study utilizes the MoNuSeg2018 dataset (Kumar et al., 2017) for training and evaluating our model. The dataset includes 30 training images and 14 test images, encompassing a total of 21,623 manually annotated nuclei for training and 7,223 for testing. These images were derived from H&E-stained whole slide images, featuring tissues from breast, kidney, liver, prostate, bladder, colon, and stomach in the training set, and an additional inclusion of lung and brain tissues in the testing set. All images were acquired at a 40x magnification, offering high-resolution insights into the cellular structures. In addition to MoNuSeg2018, the study further uses the CPM-17 (Vu et al., 2018) and CoNSEP (Graham et al., 2019) datasets for comparative and comprehensive evaluation.Stain normalization addresses the variability in histopathological images due to differences in H&E staining procedures, which affects the appearance of cellular structures. Variations in hues and intensities arise from disparate staining protocols, lighting, and imaging equipment, posing challenges for models trained on specific staining conditions. To mitigate these issues, we adopted a computationally efficient stain normalization approach, following the method described by (Mahbod et al., 2019) . A reference image from the training set was selected for Reinhard normalization, aligning all dataset images to this standard (Reinhard et al., 2001) . Reinhard normalization was chosen for its proven effectiveness in previous related researches (Patil et al., 2021) .Histopathology images inherently contain a high degree of variability, not only in the stains but also in the positions and morphological characteristics of cellular structures. Keeping in consideration, various photometric and geometric augmentations were performed to handle limited number of samples in the datasets.Geometric and Photometric Augmentations: Performing augmentations that alter the angular positions and shapes of the nuclei can differ the training samples significantly. Basic geometric augmentations such as rotations, scaling, flips were performed to obtain samples from different perspectives and scales. Additionally, elastic deformations that mimic the natural deformations in biological tissues were performed that helped in enhancing the model's ability to handle non-rigid transformations and complex variations. Photometric augmentations like Gamma and intensity level transformations along with contrast limited adaptive histogram equalization were performed.Perlin Simplex Noise: The Perlin noise algorithm is employed to generate a pseudorandom gradient vector on each corner on a given grid. Next, it calculates the distance vectors from a given position to the surrounding corners on the grid. Then, it takes the dot product between the distance vector and the gradient vector, thereby obtaining influence values. The dot product becomes positive if the two vectors are pointing in the same direction, and it becomes negative if the two vectors are pointing in opposite directions. In the final step, interpolation is performed between these influence values to construct smooth patterns within the grid. In our experiments, we generated 2D Perlin noise with the same size of the image patches and concatenated them to produce noisy images.We implement a modified and refined U-Net3+ model for nuclei segmentation inspired from the full-scale skip connection implementation. We selected U-Net3+ as the baseline as it performed better on our initial experiments as detailed in Appendix. Key adjustments to U-Net3+ (Huang et al., 2020) include reducing the model's depth to mitigate overfitting on small datasets, optimizing feature channels to capture essential details without excessive complexity and using GLU for gated feature selection. Appropriate Feature Channels: Feature channels help in capturing specific characteristics of the input, such as edges, corners, or textures. The initial layer of the encoder captures the low-level features like edges and corners and the deeper layers capture high-level abstract features. The specific arrangement of 32, 64, 128 and 128 was chosen as it empirically performed better than other configurations as in Table 4 .The importance of features can vary significantly across different regions in a histopathological image. Certain areas may contain more nucleus and might require more nuanced feature extraction, while others may be relatively homogeneous like cytoplasm and might require less detail. While traditional activation functions like ReLU (Agarap, 2019), apply the same transformation across all features, GLUs can learn to 'gate' certain features selectively. A GLU takes its input and splits it into two halves. One half is transformed linearly (much like a standard unit), and the other is transformed via a sigmoid activation function.The output of the sigmoid is used as a gate to control the information flow from the linear half. Mathematically, for an input x,GLU (x) = sigmoid(W a • x + b a ) ⊙ (W i • x + b l ) (1)Here, ⊙ denotes the element wise multiplication, W a , b a are the weights and biases for the gated mechanism, and W l , b l are the weights and biases for the linear transformation.Shift Invariant Convolutions: To address aliasing artifacts from max pooling, we implemented max blur pooling. This approach, involving a blurring step post-pooling, smooths feature maps to reduce aliasing and enhance the model's ability to capture fine details.DropBlock Regularization: DropBlock is a structured form of dropout that removes contiguous regions from the feature maps. By enforcing the model to fill in missing regions based on surrounding context, DropBlock encourages the network to learn more robust and generalized feature representations.Trainable Guided Filter: The trainable guided filter (Wu et al., 2019) offers a sophisticated mechanism for refining segmentation masks by leveraging edge-preserving filtering. It takes the coarse segmentation map produced by the network and refines it using the original input image as a guide. The guided filter performs smoothing within regions while preserving edges, effectively capturing both the global structure and local details of the targeted nuclei without introducing the halo artifacts.The model is trained using combined loss function by integrating the weighted average dice loss and focal loss (Lin et al., 2018) . Weighted dice loss is commonly used in biomedical imaging applications where there is infrequency in occurrence of certain regions or features.The focal loss on the other hand focuses more on the hard negatives and down-weights the easy examples, thereby allowing the model to focus on more challenging regions. L seg is the loss used in the experiments where, α and β are the controlling parameters of the respective loss components.L seg = αL dice (y, p(y)) + βL f ocal (y, p(y))(2) 
experiments and results: 
 The experiments were conducted on NVIDIA A6000 GPU, Intel i7 6700 CPU, running on the Ubuntu 22.10 operating system and using PyTorch framework. The images in training set was split into training and validation sets with ratio of 8:2 and then were augmented. The batch size was set to be 16 and number of epochs to be 50. Several experiments were conducted and through empirical results the optimal learning rate was found to be 1e-6. For the performance evaluation of the model, widely used evaluation criterion of Intersectionover-Union (IoU) (Rezatofighi et al., 2019) and Dice-Score (Eelbode et al., 2020) were used across experiments. The Dice Score computes the overlap between the predicted segmentation mask (A) and the ground truth mask (B). The IoU metric evaluates the quality of the object-level segmentation by calculating the overlap between the predicted and ground truth masks for each class and then averaging these overlaps.Our experiments demonstrate a significant improvement in nuclei segmentation performance across various datasets, as evidenced by Table 1 and Table 2 . Our optimized model, showcases superior results compared to previous and present existing state-of-the-art methods, including U-Net, U-Net++, CFNet, MBUTransNet, nnUNet, SAM, and AutoSAM. Specifically, on the MoNuSeg test images, our model achieves the highest Dice coefficient of 0.8902 and mIoU of 0.7924, outperforming other methods. This performance is not isolated to the MoNuSeg dataset but extends across CPM-17 and CoNSep datasets, with our model outperforming other approaches with Dice scores of 0.9325 and 0.8172, and mIoU scores of 0.8776 and 0.7257, respectively. The model was trained individually across the datasets. Method Dice mIoU U-Net (Ronneberger et al., 2015) 0.7943 0.6599 U-Net++ (Zhou et al., 2018) 0.7949 0.6604 CFNet (Zhan et al., 2023) 0.7987 0.6668 MBUTransNet (Qiao et al., 2023) 0.8160 0.6902 U-Net3+ (Huang et al., 2020) 0.8260 0.7039 nnUNet (Isensee et al., 2018) 0.8031 0.6781 SAM (Kirillov et al., 2023) 0.6950 0.6187 AutoSAM (Shaharabany et al., 2023) The results underscore the effectiveness of our approach, which incorporates techniques such as stain normalization, simplex noise, GLU, max blur pooling, dropblock and trainable guided filter. These enhancements contribute to the model's ability to more accurately distinguish between nuclei and surrounding tissue, even in the presence of staining variability and complex tissue morphology. Table 1 and Table 2 draw the comparative quantitative analysis while Figure 3 draws the qualitative superiority of our approach. We also performed statistical analysis, using a two-sample t-test (Welch's t-test for unequal variances), got a pvalue of approximately 0.0134. This result of p-value < 0.05 suggests that the performance difference observed between your model and the base model is unlikely to have occurred by chance and that our model has a statistically significant improvement (Fu et al., 2024) .  
conclusion: 
 In this study, we introduced a nuclei segmentation-specific enhanced U-Net3+ model in H&E stained histopathology images. Our implementation included -enhancing the performance of U-Net3+ model by implementing a series of optimizations including adaptive feature selection through Gated Linear Units (GLUs), max blur pooling for scale and position invariance, DropBlock regularization to mitigate overfitting and fast trainable guided filter for efficient learning. Furthermore, we applied stain normalization to achieve consistency across images and utilized advanced data augmentation techniques to expand the training dataset. Our methodology significantly outperformed previous methods including the existing state-of-the-art models. This was achieved by addressing key challenges such as data imbalance with a hybrid combined loss function that enhanced the model's sensitivity to varying object sizes and class imbalances. Despite the promising results, the segmentation of overlapping and clumped nuclei in H&E stained images remains a challenge. Future work will focus on developing a more intricate model architecture to capture that takes advantage of the foundational and task-specific models using instance segmentation. Additionally, we plan to explore the potential of transformers and promptable segmentations for further advancements in nuclei segmentation. Our study demonstrates that strategic optimizations can lead to significant improvements in histopathological image analysis, thereby laying a critical groundwork for more accurate cancer diagnosis and informing potential treatment pathways.conducted for the selection of baseline and similarly section A.2., A.3., A.4 ., detail about the effectiveness of feature channel configuration, dropblock + max blur pooling, and normalization respectively. A.5. and A.6., detail the experiments and verify the effectiveness of the activation and loss functions implemented. Finally A.7. shows the effectiveness of the fast trainable guided filter in the model. 
 
 Appendix: The baseline performance comparison between U-Net3+, ELU-Net (Deng et al., 2022) and nnUNet (Isensee et al., 2018) on MoNuSeg test images establishes U-Net3+ as superior, with an average Dice score of 0.826 compared to ELU-Net's 0.808 and nnUNet's 0.796. Table 3 illustrates the performance of the models across all the test image cases. This highlighted U-Net3+'s efficiency in capturing nuanced features across various tissue types compared to ELU-Net and nnUNet (Isensee et al., 2018) , setting a robust foundation as baseline for further enhancements. [32, 64, 128, 128] was found to be most effective which means that the first layer has 32 feature channels, second layer has 64, third layer has 128 and the fourth layer has 128. The optimal configuration demonstrates that a strategic increase in channel depth at later stages can significantly enhance segmentation accuracy without excessively inflating the model's parameter count. This suggests an effective method for maximizing performance while maintaining computational efficiency. The analysis of stain normalization techniques underscores the critical role of selecting an appropriate target image for both macenko (Macenko et al., 2009) and reinhard normalization as shown in Figure 6 . The target image was chosen after meticulous evaluation of the dataset to ensure compatibility with the majority of images. Figure 7 illustrates the process, showing the original, target, and Reinhard-normalized images. This approach emphasizes the importance of a careful selection process to maintain consistency across We perform several experiments to compare the effectiveness of GLU activation compared to ReLU, Leaky ReLU, and Swish in terms of Dice and mIoU scores. Superior performance of GLU shows its ability to adaptively filter and propagate relevant features through the network underlines its utility in complex segmentation tasks, suggesting a promising direction for enhancing model sensitivity and specificity. The sigmoid gating mechanism in GLU can learn to turn features on and off adaptively to learn the features that are important and focus on them. It makes the feature maps sparse by not focusing on less important features (Liu et al., 2022) . In the encoder-decoder like architecture in U-Net3+, the earlier layers capture local features and the deeper layer capture global features. GLUs are placed across multiple layers to adaptively learn to gate features at different scales and balance the need between local and global details. It  


Decision: Two of the three reviewers support accepting this work as a poster and believe it will generate meaningful discussions. I believe the rebuttal has addressed most of the important comments. The authors are requested to revise the camera-ready version in accordance with the feedback provided in the rebuttal.

Reviews: review: paper_summary: paper_summary: The authors provide a study to segment cell nuclei in histopathology images. In particular, they mine the U-Net architecture and combine several well-established techniques to outperform baselines, such as U-Net, U-Net++ and SAM. This fact relies on a couple of architectural designs, such as Gated Linear Units (GLUs), improved skip connects (inherited from U-Net++), and state-of-the-art loss combinations (Focal and Dice loss).
main_review: main_review: I don't know if the appendix was as strong as before, but I believe that the authors did a good job in incorporating the other reviewers' comments and updated the paper accordingly, as well as the incorporation of the nnUnet baseline. I think I was a little harsh on the authors and apologize.
strength_weakness: strength_weakness: In general, I would actually like to see the author's final model, and then do an ablation strategy. It seems like the design choices are independent, and then the best of all worlds is taken. However, I believe the authors can make stronger claims on their design choice if they do provide their model, ablate or change the design choice (e.g. from GLU back to ReLU, Dice Loss), and then show clearly, that only their combination is strong. This would give very much insight into the complex field of architectural design strategies. 

Further, claims such as "especially in conjunction with Max Blur Pooling" are very strong, despite the fact that adding MBP only has marginal benefits (Appendix A.3). I recommend toning down a lot of the statements and adjust them according to the authors' findings. Similar is for example the stain normalization. Table 6 shows no improvement upon stain normalization.

This problem (nuclei segmentation) is not only semantic, but also related to instance segmentation. I am missing the context if the authors model would bring benefits in this regard.

Finally, I am missing very strong baselines. For example, nnU-Net has been shown several times, that the original U-Net is very strong. For example, using the same loss and GLUs in the normal U-Net and compare it to nnU-Net would largely improve the statements of the paper.
questions: questions: * Ablation study and real impact of their design choices
* Strong baselines
* and commenting on the remaining comments
paper_summary: paper_summary: The authors describe a model for the common task of nuclei segmentation from pathology images. They have conducted careful experiments on several enhancements to the existing UNet3+ and benchmarked them. The authors arrive upon a combination of enhancements that improve model performance quite significantly over baselines.
main_review: main_review: While a direct response to my comments would have been very useful, it does appear that the authors have addressed all the comments in their revised manuscript (except the one about SAM). I do think the paper is stronger for having included nnUnet (and shown superiority over it). My opinion of the paper has therefore gone up to "strong accept".

Responding to other reviewers' comments: I am rather baffled at the other reviewers' criticism that an ablation study is needed. The entire appendix is a (very thorough) ablation study. Perhaps they simply didn't read the appendix? Though a statistical test would be nice, it is very common not to do this within the ML, rightly or wrongly, so it seems harsh to reject the paper on this basis alone (EDIT: I see from a comment on the thread above that the results are statistically significant according to a t-test, but that this has not made it into the paper). Lack of consideration of instance segmentation, while a valid idea, is rather off topic and not a reason to reject this paper. Semantic segmentation is a common and valid way to perform cell segmentation and a paper considering only semantic segmentation has value. Consequently I quite strongly disagree with reviewer eYtt's criticisms, especially with the nnUnet comparison now in place.
strength_weakness: strength_weakness: There are no technically novel aspects presented in this paper. All enhancements have previously been published separately, and this paper just brings them together and performs an ablation study. This paper requires a long appendix that contains all the ablation experiments, and  in my opinion the most interesting part of the paper is in the appendix, perhaps suggesting that a MIDL conference paper is not the best place to present this work.
questions: questions: No particularly pressing points to address, but see list of questions in the detailed comments.
paper_summary: paper_summary: This study introduces an enhanced U-Net3+ model for the semantic segmentation of nuclei in histopathological images, aiming to establish a new baseline in the field. By integrating adaptive feature selection, max blur pooling, DropBlock, and a Guided Filter Block, the model seeks to capture both short- and long-range dependencies, achieve scale and position invariance, mitigate overfitting, and delineate fine-grained details in nuclei structures. Additionally, data augmentation techniques and stain normalization are applied to reduce inconsistencies across datasets. The authors state that their approach significantly outperforms current state-of-the-art models, thereby contributing to the precision of nuclear segmentation crucial for cancer diagnosis and treatment strategies.
main_review: main_review: The authors have answered all my questions and addressed all concerns. The overall quality of the work was improved. I'm still wandering about the missing comments to the other reviewers concerns. Therefore I updated my final rating to 3a.
strength_weakness: strength_weakness: - The paper does not discuss the potential of nnUnet [1], which is a notable omission given its performance parity with or superiority to U-Net++ in similar tasks.
- The lack of consideration for instance segmentation approaches, particularly in the context of overlapping and clumped nuclei, is a missed opportunity to address a critical challenge in the field. Although mentioned in the discussion, this is crucial for real application.
- Evaluating the performance on different cell complexity levels (e.g. difficulty levels in terms of annotation) would be very interesting to see e.g. using the data set provided in [2].
- The paper's claims of significant performance improvements are not substantiated with statistical proof, weakening the strength of these assertions.
- A combined ablation study to quantify the contribution of each proposed enhancement to the overall performance improvement is missing, which would provide valuable insights into the efficacy of the individual components.
- The results' presentation in Tables 1 and 2 could be improved for clarity, specifically by explicitly noting the significance of bolded and underlined results in the table captions.

[1] Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring 
method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 203-211.
[2] Kromp, F., Bozsaky, E., Rifatbegovic, F. et al. An annotated fluorescence image dataset for training nuclear segmentation methods. Sci Data 7, 262 (2020).
questions: questions: - Could the authors elaborate on their choice of U-Net3+ over e.g. nnUnet or similar, particularly in relation to their respective performances in similar segmentation tasks?
- What are the reasons behind not considering instance segmentation methods for addressing the challenge of overlapping and clumped nuclei?
- Can statistical evidence be provided to substantiate the claims of significant performance improvements achieved by the proposed model?
- Would the authors be willing to include a combined ablation study that clearly delineates the contribution of each model enhancement to the overall performance?
- How do the authors justify the computational complexity and practical applicability of the enhanced U-Net3+ model compared to other state-of-the-art models?

Comments: title: Adiitional Response
Additional Response to Reviewer EpDY
Additional Response to Reviewer eYtt
Additional Reponse to Reviewer rzgC
paper is open for discussions
comment: We greatly appreciate your further comments.

Regarding your concerns about the claim of significant improvements without statistical proof, we performed a two-sample t-test  to compare the Dice scores of our proposed model against the base model (U-Net3+). The analysis yielded a p-value of approximately 0.0134. The p-value is less than the conventional threshold of 0.05, indicating that there is a statistically significant difference between the mean Dice scores of the two models. This result suggests that the performance difference observed between our proposed model and the base model is unlikely to have occurred by chance, supporting the conclusion that our model has a statistically significant improvement in Dice scores compared to the base model. ​

We have made the following changes to the manuscript.
1. We have now included the nnU-Net results in the state-of-the-art section of our work, as suggested. 
2. We have added the result of two-sample t-test in the section 3 - Experiments and Results.
3. We removed the equations for the evaluation metrics while citing them in references (as suggested by Reviewer EpDY ) to adhere to the page limit.
Thank you for providing these comments to my questions!

I guess my fellow reviewers would like to have their questions answered too.

The addition of the nnunet results is appreciated. It would be great, if you would also add it to the state of the art section of you work.

Furthermore I still have an issue with claiming significant improvements without a statistical proof.
We have list up the potential questions from the review and have addressed in below.

**Question 1:  Are there errors in Table 4 and 6?**

Response 1: We extend our apologies for the errors in Tables 4 and 6 and appreciate your understanding. The corrections have been made to accurately display the parameter counts and experiment results. Additionally, we have now included the results for the configuration (32, 64, 128, 256) in Table 4, providing a more comprehensive view of the models' performances.

**Question 2: Is mIoU the same as IoU since number of classes is one?**

Response 2: Yes, in the context of our binary segmentation task, where nuclei regions are labeled as 1 and non-nuclei regions as 0, the mean Intersection over Union (mIoU) is effectively equivalent to the standard IoU.

**Question 3: How did inference using SAM was performed?**

Response 3: For SAM, we utilized image masks as input prompts to guide the inference process. It's worth noting that SAM was not re-trained but was used solely for inference purposes.

**Question 4: Whether model was trained on other datasets ?**

Response 4: Yes, our model was individually trained on each dataset, and the performance metrics were computed using their respective test sets. This has now been clearly stated in Section 3 of our manuscript.

**Question 5: Did any other in-network normalization was performed?**

Response 5: We did not experiment with alternative in-network normalization methods in this study, but we recognize the value of such investigations and have noted this as a potential direction for future research.

**Question 6: Why is there absence of ablation on trainable guided filter?**

Response 6: We have included the trainable guided filter section in the ablation studies under section - A.7. We have also added a comparative qualitative visualization of effects of the guided filter in Figure 8.

We hope these adjustments and clarifications address your concerns and we look forward to any further comments or suggestions
We have list up the potential questions from the review and have addressed in below.

**Question 1: What are the impacts of design choices and ablation study?**

Response 1: The ablation study was structured to sequentially assess the impact of our design choices. While it doesn’t quantify the exact extent of each choice's contribution, it does clearly demonstrate their effectiveness. By following a sequential ablation approach and observing consistent improvements in model accuracy, we establish that each modification indeed contributes positively towards achieving state-of-the-art results. This is effectively communicated through the overview diagram in Figure 4, where the incremental benefits of our design choices are made evident.

**Question 2:  How are the baselines selected?**

Response 2: In identifying a strong baseline for our study, we prioritized simplicity and efficacy in managing both local and long-range dependencies within the histopathological  context. We selected three state-of-the-art models—U-Net3+, ELU-Net, and nnUNet—due to their demonstrated capabilities. Each model was trained on our dataset without  modifications, and their performances were tabulated in Table 3 of the ablation study. U-Net3+ emerged as the most suitable baseline for further enhancements based on its initial performance, setting the foundation for our subsequent optimizations.

**Question 3: There are some strong claims in manuscript despite having marginal benefits. Is it possible to tone down the statements based on findings?**

Response 3: Acknowledging the reviewer's feedback, we have carefully reviewed and revised our manuscript to ensure that all claims are accurately represented and fully supported by our data. We addressed discrepancies in Table 4 and Table 6 concerning feature channel configurations and stain normalization effects. These tables have been updated to correctly reflect the outcomes of our experiments, thus ensuring our statements accurately mirror the empirical evidence.

**Question 4: Is it possible to insert insets for performance visualization?**

Response 4: In response to this valuable suggestion, we have updated Figures 3 and 8 to include insets. These insets offer a clearer and more detailed visualization of performance enhancements of the model.

**Question 5: Why there is no comparison of design choices on other models (especially original U-Net) for better evaluation ?**

Response 5: We acknowledge the importance of comparing our design choices across different networks, such as the original U-Net, to comprehensively evaluate their effectiveness. While our current study focused on achieving SOTA results using a lightweight model architecture by selectively incorporating a few promising components tailored specifically nuclei segmentation task. We recognize the potential of extending our proposed enhancements to other models.

We hope these adjustments and clarifications address your concerns and we look forward to any further comments or suggestions
We would like to extend our sincerest apologies for any oversight in directly addressing the questions and concerns. We have listed them below with our remarks.  
**Question 1: Could the authors elaborate on their choice of U-Net3+ over e.g. nnUnet or similar, particularly in relation to their respective performances in similar segmentation tasks?**

Response 1 Our preliminary experiments indicated that U-Net3+ achieved superior Dice scores over nnUNet and ELU-Net, leading to its selection as our baseline model. In this rebuttal we updated the results of nnUNet in Table 3.

**Question 2: What are the reasons behind not considering instance segmentation methods for addressing the challenge of overlapping and clumped nuclei?**

Response 2: We acknowledge the significance of instance segmentation in addressing the challenges in nuclei segmentation. However, our study focused on improving the existing segmentation model by applying various techniques and observing the presence of their effectiveness. There is still a surge in active research for nuclei binary segmentation and our study could help potential future research to opting for certain design choices in their model architecture.

**Question 3: Can statistical evidence be provided to substantiate the claims of significant performance improvements achieved by the proposed model?**

Response 3: Indeed, the focus of our research is to present that SOTA results are attainable using a lightweight model architecture by selectively incorporating a few promising components tailored specifically for the task of nuclei segmentation. Our ablation study does shed light on the effectiveness of our design choices, showing incremental improvements in model performance, finally leading to over six percent increase over the base model. Moreover, we perform experiments with the same model and same hyperparameter values on other datasets - CPM-17 and CoNSep and the proposed model shows performance gains in terms of Dice scores.

**Question 4: Would the authors be willing to include a combined ablation study that clearly delineates the contribution of each model enhancement to the overall performance?**

Response 4: Indeed, we have added an overview of the ablation study which breaks it into three enhancements as shown in Figure 4 where the enhancements sequentially raised the Dice score from 0.826 to 0.8902.

**Question 5: How do the authors justify the computational complexity and practical applicability of the enhanced U-Net3+ model compared to other state-of-the-art models?**

Response 5: Our experiments show that the proposed design choices along with other enhancements increase the overall performance of the baseline model and achieves the SOTA performance. The enhanced U-Net3+ model, while not using pre-trained weights and being trained from scratch with limited data, is able to perform considerably better. While more experiments are still required to ascertain the claims of the design choices of our model, our study does show potential improvements and practical applicability, especially in settings with limited resources.

We hope that the clarifications address your concerns.We thank you once again for your patience and guidance and look forward to any further comments or suggestions you may have.
Sorry but it is very hard to consider this rebuttal, as the authors neither responded to the raised questions and concerns here nor highlighted the changes in their revised document.

Can we expect some comments from the authors?

Thank you!
Dear Reviewers The authors have submitted their rebuttal addressing the raised questions. The paper remains open for further discussion and engagement.
Thank you very much for your insightful and constructive feedback on our manuscript. We greatly appreciate the time and effort you have invested in providing a detailed review. 

We acknowledge the exceptional performance of nnUnet in various segmentation tasks. Our choice to build upon U-Net3+ was primarily driven by its architectural simplicity, and preliminary experiment results as now detailed in section A. 1. Our preliminary experiment showed that U-Net3+ outperformed nnUNet and ELU-Net and thus we selected it as the base model for potential improvements. Moreover, the simplistic approach was also the reason we went for normal segmentation instead of instance segmentation. We agree that adopting instance segmentation could potentially alleviate some of the inherent problems in nuclei segmentation and increase the accuracy of the model and recognize this as an avenue for future research.
Upon reevaluation of our manuscript, we found some inaccuracies in Tables 4 and 6 which we have corrected and have also included insets in our figures to enhance their visibility. We have also added experiment results of configuration (32. 64, 128, 256) to table 4 and have included the guided filter experiment result in the ablation section A.7. Furthermore, we also added an ablation overview diagram for better interpretation of the ablation study section and show the improvements in the model performance through the enhancements. 

The ablation and almost all parts of the experiments were conducted on MoNuSeg dataset and to justify that the model doesn’t only perform well on MoNuSeg data but performs well on nuclei segmentation, we performed experiments across CPM-17 and CoNSep dataset (Table 2) and the model achieved the state-of-the-art results without any individual optimizations. This demonstrates the ability of the model for nuclei segmentation. Furthermore, we also will be providing the code used for the experiments upon selection for clarity and further research. 

To summarize, while there is room for further innovation and refinement in our study, we firmly believe that our research makes a meaningful contribution to the field. By leveraging and optimizing existing architectures, we have advanced the state of the art, underscoring the potential of task-specific models to capitalize on the existing research. Our findings demonstrate that with careful tuning, even established methods can yield superior results. 
We hope that the updates and clarifications provided will satisfy the concerns and render the study a valuable contribution to the field of nuclei segmentation. Thank you for spending valuable time to review our paper and provide meaningful guidance.
Thank you very much for your insightful and constructive feedback on our manuscript. We greatly appreciate the time and effort you have invested in providing a detailed review. Your acknowledgment of the thorough empirical evaluation and the significant improvement our model demonstrates over existing baselines is encouraging.
We acknowledge your concerns and do agree that there was an oversight on our part in both Tables 4 and 6. These errors were corrected to accurately reflect the parameter counts and the experimental outcomes. We have also added experiment results of configuration (32. 64, 128, 256) to Table 4 and have included the guided filter experiment result in the ablation section A.7. We have further added an ablation overview diagram (Figure 4) for better interpretation of the ablation study section and modified Figure 3, Figure 7 to include insets for better visualization and readability. 
For SAM prediction, we used image masks as prompts for the prediction results. (SAM was not retained but only used for inference)
The model was trained individually across the datasets and we've also made it explicit in the manuscript as well.
While we did not experiment with alternative in-network normalization methods in this study, we recognize the value of such investigations and have noted this as a potential direction for future research. Moreover, regarding our choice of baseline for the experiments, our preliminary results indicated that U-Net3+ offered a compelling balance of simplicity and performance, which informed our decision to select it over other architectures like nnU-Net (now presented in A.1.). This simplistic approach was also the reason we went for normal segmentation instead of instance segmentation. However, we accept the value of instance segmentation in enhancing model precision and recognize this as an avenue for future research.

Your constructive criticism and insightful questions have been invaluable in refining our manuscript. We hope these revisions and clarifications address your concerns satisfactorily.  Thank you for spending valuable time to review our paper and provide meaningful guidance.
Thank you sincerely for your thoughtful and thorough review of our manuscript. Your expertise and detailed analysis have provided invaluable insights to our research. Your detailed feedback is invaluable, and we're particularly encouraged by your appreciation of the manuscript's clarity.

We acknowledge your concerns about the necessity of ablation study for the validation of our architectural decisions. Indeed, our approach of comparing incremental model modifications to a baseline did not fully exploit the required potential. While our current experiments reveal the impact of our design choices, they fall short of illustrating their independent effectiveness.  We should have fixed a final model and then ablate to find the presence and impact of certain design choices ( such as GLU, Loss, etc. ), but we performed experiments where we compared the results with the baseline. Nevertheless, these exploratory analyses did reveal the influence of each design choice on our model's performance. To elucidate these findings, we have now included an overview diagram of the ablation study (Figure 4)  in the revised manuscript, which clarifies the contribution of each enhancement to the overall effectiveness of our approach.

We have taken care to address the overstatements in our initial manuscript and ensure that our conclusions are firmly grounded in the data we have presented. Upon reevaluation of our manuscript, we found some inaccuracies in Table 4 and 6 which we have corrected and have also included insets in our figures (Figure 3, Figure 7) to enhance their visibility.

Regarding our choice of baseline for the experiments, our preliminary results indicated that U-Net3+ offered a compelling balance of simplicity and performance, which informed our decision to select it over other architectures like nnU-Net (now presented in A.1.). This simplistic approach was also the reason we went for normal segmentation instead of instance segmentation. However, we accept the value of instance segmentation in enhancing model precision and recognize this as an avenue for future research.

To summarize, while there is room for further innovation and refinement in our study, we firmly believe that our research makes a meaningful contribution to the field. By leveraging and optimizing existing architectures, we have advanced the state of the art, underscoring the potential of task-specific models to capitalize on the existing research. Our findings demonstrate that with careful tuning, even established methods can yield superior results. 
We hope that the updates and clarifications provided will satisfy the concerns and render the study a valuable contribution to the field of nuclei segmentation. Thank you for spending valuable time to review our paper and provide meaningful guidance.