GeNSeg-Net: A General Segmentation Framework for Any Nucleus in Immunohistochemistry Images

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-13 14:50:17

Abstract: Immunohistochemistry (IHC) plays a crucial role in understanding disease mechanisms, diagnosing pathology and guiding treatment decisions. The precise analysis heavily depends on accurate nucleus segmentation. However, segmentation is challenging due to significant inter- and intra-nucleus variability in morphology and distribution, stemming from inherent characteristics, imaging techniques, tissue differences and other factors. While current deep learning-based methods have shown promising results, their generalization performance is limited, inevitably requiring specific training data. To address the problem, we propose a novel General framework for Nucleus Segmentation in IHC images (GeNSeg-Net). GeNSeg-Net effectively segments nuclei across diverse tissue types and imaging techniques with high variability using a small subset for training. It comprises an enhancement model and a segmentation model. Initially, all nuclei are enhanced to a uniform morphology with distinct features by the enhancement model through generation. The subsequent segmentation task is thereby simplified, leading to higher accuracy. We design a lightweight generator and discriminator to improve both enhancement quality and computational efficiency. Extensive experiments demonstrate the effectiveness of each component within GeNSeg-Net. Compared to existing methods, GeNSeg-Net achieves state-of-the-art (SOTA) segmentation accuracy and generalization performance on both private and public datasets, while maintaining highly competitive processing speed. Code will be available for research and clinical purposes.

Main paper: 
 introduction: 
 Immunohistochemistry (IHC) staining employs the specificity of antigen-antibody reactions to visualize specific protein expression within tissues or cells. Its high specificity and sensitivity, along with the ability for multi-target quantification and localization, have contributed to its growing popularity [2, 12, 35, 37] . It encompasses two major imaging techniques: brightfield and fluorescence. Accurate nucleus segmentation in IHC images, involving detection and delineation of each nucleus, is vital for disease diagnosis and treatment [4] . For instance, the classification and grading of cancers heavily rely on the rich spatial and morphological information of nuclei [25] . Manual nucleus segmentation is time-consuming, which means the increasing need for automated segmentation. Automated nucleus segmentation remains challenging due to the inherent characteristics of nuclei, i.e., high density and adhesive edges, as depicted in Fig. 1 , resulting in under-or over-segmentation. Furthermore, complex factors such as variable shapes and sizes, blurred edges, overlapping cell clusters, uneven staining and imaging condition variations lead to high error rates [11] . A large number of traditional and deep learning-based methods have been proposed to address above problems [5, 9, 15, 23, 27, 34] . In IHC images, there are different imaging techniques, i.e., brightfield and fluorescence, and significant morphological variations originating from diverse tissues. Existing methods often struggle with generalization ability. They exhibit poor performance on nucleus segmentation for tissue types not included in the training data. Since encompassing all morphological variations during training is impractical, our aim is to conduct a pilot study to develop a general framework for nucleus segmentation in IHC images. By training on several tissue types captured via brightfield and fluorescence imaging, the method achieves accurate nucleus segmentation for various types, regardless of their presence during training.Fig. 1 showcases significant variations in nucleus morphology across diverse tissue types in both IHC brightfield and fluorescence images. For instance, in brightfield images, nuclei in image B1 exhibit a solid texture while those in B4 display a hollow texture. Nuclei in B4 are relatively regular ellipses, whereas those in B1, B2 and B3 show irregular shapes. Hence, we ponder: Is it possible to minimize these morphological variations arising from various factors mentioned before? If so, the subsequent segmentation task is simplified when dealing with nuclei of uniform morphology. This leads to higher segmentation accuracy across various tissue types, thereby improving the method's generalization performance.In this paper, we propose a novel two-stage method as illustrated in Fig. 1 . It deviates from the common approach of existing twostage methods [1, 10, 16] , which involves initial nucleus detection followed by precise segmentation. Inspired by the advancements in Artificial Intelligence Generated Content (AIGC) [6, 45] , nuclei of various tissue types and imaging techniques are enhanced to an "ideal" morphology characterized by clear centers, solid filling and well-defined edges through generation in enhancement model. Subsequently, the nuclei with distinct features in texture and edges are input into the segmentation model to obtain precise contours.In contrast to existing one-stage [9, 15, 33] and two-stage [1, 10, 16] deep learning methods with task-specific network designs, a classic image-to-image translation generative adversarial network (GAN) [14] performs enhancement alongside a conventional network for segmentation. Since two-stage methods which are not designed to be end-to-end increase complexity to the entire framework [9] , we develop a lightweight generator and discriminator based on ResNet and transformer [38] , respectively, to improve both enhancement quality and computational efficiency.The main contributions of this paper are as follows: • This paper is the pioneering study on the general framework for nucleus segmentation in IHC images, with a focus on high generalization ability. We aim to accurately segment nuclei across various tissue types in both brightfield and fluorescence images by training on a small subset of types. • We propose a novel two-stage method, GeNSeg-Net, which first enhances nuclei followed by segmentation. This process generates nuclei of various tissues with clearer texture and edges, effectively mitigating segmentation challenges. • We design a lightweight generator and discriminator to improve enhancement quality by emphasizing the semantic relationship, texture, shape and size of generated nuclei further, while ensuring computational efficiency. • We conduct experiments on a private systematic dataset of nuclei, covering diverse tissues with rich stain colors in both brightfield and fluorescence imaging, as well as public datasets DSB2018 [4] and BBBC006v1 [24] which include fluorescence images. Our method, GeNSeg-Net, exhibits SOTA performance in accuracy. 
related work: 
 In this section, we divide current nucleus segmentation methods into traditional methods and deep learning-based methods. Many traditional methods are based on watershed [27, 36, 39, 46] . For instance, Malpica et al. [27] introduced a morphological watershed-based method that utilizes both intensity and morphological information for nucleus segmentation. However, it tends to result in under-or over-segmentation when dealing with nucleus adhesion [36, 46] . Yang et al. [46] proposed a novel marker extraction method based on condition erosion to mitigate over-segmentation. Additionally, numerous other methods exist, such as thresholdbased [5] , contour-based [28] , graph-based [3] , region-based [40] and others [26, 34, 43] . A common drawback of traditional methods is their reliance on manual feature extraction, which leads to good performance solely on specific datasets with rich features.In recent years, deep learning-based methods have gained prominence. They can be categorized into one-stage [7, 9, 15, 22, 33] and two-stage [1, 10, 16, 23] methods. One-stage methods employ a single network and use post-processing to obtain precise contours, which can be further divided into classification-based and regression-based methods. Classification-based methods output classification probability maps. For example, DCAN [7] is a deep contour-aware network that predicts nuclei and boundaries simultaneously through two branches: a semantic segmentation branch and a boundary detection branch. BES-Net [29] and CIA-Net [49] establish connections between the two branches, further improving effectiveness. HARU-Net [8] , the latest advancement, utilizes a hybrid attention-based residual U-blocks network to predict foreground regions and boundaries simultaneously. These methods derive final nucleus instances by subtracting boundaries from foreground regions, potentially resulting in certain loss of segmentation accuracy. Regression-based methods output regression maps. For instance, HoVer-Net [15] predicts the distances between nucleus pixels and their centroids in both the vertical and horizontal directions, followed by watershed post-processing. StarDist [33] predicts centroid probability maps and distances from each foreground pixel to its instance boundary along pre-defined directions. However, StarDist may loss information for large nuclei due to its reliance on only central pixel features in post-processing, which is addressed by CPP-Net [9] 's optimization. Two-stage methods typically involve a detection stage followed by a segmentation stage, i.e., first locating nucleus instances and then predicting precise masks. For example, BRP-Net [10] generates nucleus proposals based on instance boundaries and then refines the foreground masks. SAM [20] represents the emergence of large-scale segmentation models, excelling in natural image segmentation tasks, but its performance on special objects like nucleus is yet to be verified. 
methodology: 
 The overall architecture of GeNSeg-Net is depicted in Fig. 2 . It consists of two stages: an enhancement stage and a segmentation stage. In the first stage, the texture and edges of all nuclei are enhanced to obtain a uniform morphology with distinct features. The lightweight generator and discriminator in enhancement model improves both the quality of enhancement and computational efficiency. In the second stage, we perform class prediction and post-processing for segmentation.Data pre-processing is illustrated in the red area of Fig. 2 . Considering texture and edge variations among nuclei, we initially enhance them through GAN [14, 17] , achieving consistent "ideal" nuclei with clear centers, solid filling and well-defined edges, referred to as "nucleus enhancement". To obtain the ground-truth ideal nuclei, we calculate the pixel-to-boundary Euclidean distance for each nucleus in the annotated images. This categorizes each pixel into three classes: nucleus bodies, pending edges and background. More specifically, we adjust the staining intensity of each pixel within a The overview of our proposed GeNSeg-Net. We take high-resolution images sized at 512 × 512 pixels as a typical example to illustrate our method. "WS" denotes watershed segmentation. For clarity, we display images with a size of 256 × 256 pixels and bold the precise contours generated by watershed.nucleus based on its distance from the centroid, gradually decreasing towards the edge. Shared boundaries among nuclei receive a same staining intensity, termed pending edges. The transformed data serves as ground truth for the first stage and as input for the second stage.3.2.1 Lightweight Generator. For the relatively straightforward generation task, we design a lightweight network as depicted in Fig. 2 . Initially, the high-resolution image X is transformed into a feature map F(X) with 𝐶 channels using the convolution operation. Subsequently, spatial information of each nucleus is aggregated at multiple levels through downsampling. The aggregation effectively consolidates nucleus pixels while eliminating isolated background noise. Simultaneously, applying subsequent operations to the downsampled feature map improves computational efficiency. The feature map then passes through 9 custom-designed Res-CBAM blocks. In Res-CBAM blocks, channel attention and spatial attention are integrated after convolution to eliminate irrelevant information and noise [8, 44] . Finally, the feature map is gradually upsampled to the original size, yielding the enhanced image Y. The process mitigates the loss of fine details caused by direct upsampling at large ratios. In the experiment, we meticulously assess the impact of downsampling and Res-CBAM on model's performance.Discriminator. In modern GANs [31] , the discriminator typically employs convolutional neural networks (CNN) as backbone. While it facilitates stable training for high-resolution images, the convolutional operation has a limited local receptive field. In cases with extensive nucleus adhesion or large nuclei, insufficient network depth fails to capture long-range dependencies. However, increasing network depth can lead to feature and detail loss, complicating optimization and reducing computational efficiency, as demonstrated in previous studies [41, 47] . Therefore, we deviate from conventional CNN models while conducting a comparative analysis in our experiments.In addition, we consider the following factors: (1) A nucleus covers multiple pixels, especially in cases of adhesive nuclei that span a significant area. Hence, rather than pixel-level discrimination, we opt for a coarser patch level. (2) Large patches in high-resolution images tend to lose low-level texture details, while small patches significantly increase computational demands and memory usage.(3) The adhesion and size of nuclei vary, making it challenging to adopt a fixed patch size for the entire image, which can affect model generalization.Inspired by TransGAN [18] , we introduce a transformer-based multi-scale lightweight discriminator, employing the transformer encoder [38] as the basic block. Our multi-scale discriminator handles local semantic relationships among nuclei, whether they are isolated or adhesive, regardless of their size. It also considers lowlevel texture, as well as global shape and size features by employing patches of varying sizes at different scales. In high-resolution feature maps, correlating two distant positions is unnecessary. Thus, we integrate grid self-attention [18] into the transformer block to improve computational efficiency.In Fig. ( 𝐻 8𝑃 × 𝑊 8𝑃 ) × 𝐶 2 .These sequences, combined with learnable positional encoding, serve as inputs to the transformer blocks. After each block, features are upsampled via average pooling and concatenated with the new sequence. A class token is introduced prior to the final transformer. It passes through the transformer block to determine whether Y is real or fake.Grid self-attention [18] is adopted in high-resolution stages (resolution higher than 16 × 16). In contrast to conventional transformer blocks [13] that involve interactions between the individual token and all others, we partition the entire feature map into grids based on a pre-defined window size. Attention operation is then performed within each grid, tailored to our requirements.In the second stage, class prediction is conducted firstly by U-Net [32] , followed by watershed post-processing. To be specific, U-Net categorizes the image context into three classes: nucleus bodies, coarse edges (approximately 4 pixels wide) and background. The watershed algorithm then defines contours within the coarse edge regions. Our segmentation model follows a classic approach with a simple network design. It achieves both high accuracy and computational efficiency. Since the segmentation model operates independently of the enhancement model, GeNSeg-Net provides great flexibility, enabling the integration of more advanced segmentation methods. Despite such a classic method within our framework, the segmentation accuracy has already achieved SOTA performance.Our training data consists of several tissue types, while the testing data includes nuclei from both the same tissue types as in the training and different tissue types. To focus the networks' attention on nucleus morphology and minimize interference from imaging techniques and other factors, both the training and testing data are uniformly converted into grayscale images, as shown in the gray area of Fig. 2 . The enhancement model and segmentation model are trained individually using their respective paired data for full supervision.During the training of our enhancement model, the loss function for the image-to-image translation GAN is defined as:EQUATIONwhere 𝐺 and 𝐷 refer to the generator and discriminator, while 𝑥 and 𝑦 represent the input and ground truth, respectively. Previous research suggests that introducing noise to the input can prevent deterministic outputs and address issues related to fitting restricted distributions [42] . However, nucleus images inherently contain background noise due to various factors. The intensity and distribution of pixel values are crucial for the model to learn localization and texture, with subtle variations playing a significant role during learning. Therefore, we refrain from introducing additional information besides the original input. Nevertheless, incorporating a traditional loss, such as L1, has proven beneficial for model training [30] . While the discriminator's task remains unchanged, the generator now has a dual objective: to generate images with realism in the semantic relationship, texture, shape and size, and to approach the ground truth in an L1 sense. The overall loss function of enhancement model is:EQUATION𝐿 𝑒𝑛ℎ (𝐺, 𝐷) = 𝐿 𝑇 (𝐺) + 𝜆𝐿 𝐺𝐴𝑁 (𝐺, 𝐷)In the fully supervised U-Net training, a combined Cross Entropy and Dice loss function is utilized for predicting three classes. The segmentation model's loss function is formulated as:EQUATIONThe weight of each term is controlled by 𝛼 and 𝛽.During inference, test images of diverse tissue types initially undergo enhancement by the trained generator. Subsequently, the trained U-Net model predicts the three classes. Precise contours are then obtained by watershed post-processing. 
experimental setup 4.1 dataset: 
 DSB2018 and BBBC006v1. Data Science Bowl 2018 (DSB2018) [4] is a competition focused on nucleus detection and segmentation. It comprises 670 images with manual annotations, sized from 256×256 to 520×696 pixels. Fluorescence imaging is utilized, encompassing DAPI and Hoechst 33342 stains. We follow the dataset division protocol outlined in [9] , allocating 380 images for training, 67 for validation and 50 for testing. BBBC006v1 [24] consists of 768 fluorescence images containing Hoechst 33342-stained U2OS cells, each with the size 696×520 pixels. The dataset is randomly divided into 462 training images, 153 validation images and 153 testing images.Private Systematic Dataset. To address the lack of a systematic public IHC dataset, we build a nucleus dataset encompassing diverse tissue types and quantities. It consists of IHC images acquired through two primary imaging techniques: brightfield and fluorescence. In brightfield imaging, hematoxylin stains nuclei, yielding vibrant images under transmitted light. Fluorescence imaging employs DAPI to label nuclei. The nuclei emit fluorescence signal when excited at specific wavelengths, which is then captured using a fluorescence microscope. To emulate real-world scenarios, we refrain from strictly controlling staining and imaging conditions, allowing for natural variations introduced by experts and environments. In clinical and industrial settings, pseudo-colors are commonly applied to fluorescence images for better visualization. Hence, we randomly assign pseudo-colors (e.g. red, blue and orange) while ensuring consistent color assignment within the same tissue. All stained whole slide images are segmented into patches of 512×512 pixels. Five pathologists with over three years of clinical experience provided pixel-wise annotations, as illustrated in Fig. 3 . In the training dataset, images are annotated into three classes: nucleus bodies, edges and background. To mitigate annotation discrepancies among experts, edges are required to be coarsely annotated with a 4-pixel width, covering the entire actual edges. Two pixels are within the nuclei, while the other two pixels lie outside. In the testing dataset, each instance is precisely annotated.The dataset composition and division are summarized in supplementary materials. Training and validation data originates from four tissue types: lung, liver, colon and stomach cancers, each with 60 images, alongside 20 testing images per tissue. Additionally, the testing data includes four different tissues: cervix, osteosarcoma, tuberculoma and lymph cancers, each comprising 20 images. They serve to evaluate the generalization ability of segmentation methods. During training, both brightfield and fluorescence data are combined, resulting in a total of 480 images. Among these, 384 images are randomly selected for training, with the remaining 96 reserved for validation.It's noteworthy that our private dataset, compared to DSB2018 [9] and BBBC006v1 [24] , contains a larger number of nuclei with more diverse morphology and more complex environment, as depicted in Fig. 4 and 5 . As a result, the segmentation task is significantly more challenging.We employ six evaluation metrics to fully assess method performance: (1) Dice: It evaluates the separation of nuclei from the background. (2) Aggregated Jaccard Index (AJI) [21] and AJI+: AJI is based on instance-wise IoU between predictions and ground truth [10, 15, 48] . AJI+ improves AJI by ensuring maximal unique pairing to obtain overall intersection, thus addressing AJI's overpenalisation. (3) Panoptic Quality (PQ) [19] : It has been widely employed in panoptic segmentation tasks [19] . PQ was first introduced into nucleus segmentation by [15] as the most comprehensive and persuasive metric. It evaluates both detection quality (DQ) and instance-wise segmentation quality (SQ) comprehensively.All methods are implemented using PyTorch 1.12.1 on a system equipped with a single NVIDIA GeForce RTX 3090 GPU. During training, we maintain the original image size and apply basic data augmentation techniques for all methods, including flip, rotation and brightness adjustment. For GeNSeg-Net, the enhancement model adopts Adam optimization with a learning rate of 10 -4 . A linear decay strategy is employed in the latter half of epochs. Batch size is set to 2 and training proceeds for 400 epochs. Considering both nucleus size and computational efficiency, the discriminator's window size is configured to 8. The loss weight coefficient 𝜆 is empirically set to 0.01. The enhancement model's backbone is Pix2pix [17] without any condition. In the segmentation model, the U-Net architecture is standard as proposed in [32] , with a fixed learning rate of 10 -2 . 𝛼 and 𝛽 are respectively set to 1 and 3. For other methods, we adhere to the settings in their respective papers. 
experiments and analysis 5.1 comparisons with the sota methods: 
 To evaluate both the segmentation accuracy and speed of our proposed GeNSeg-Net, we conduct experiments on our private systematic dataset, as well as public datasets DSB2018 [4] and BBBC006v1 [24] . GeNSeg-Net is compared against a large-scale segmentation model and SOTA nucleus segmentation methods, i.e., SAM [20] , NucleiSegNet [22] , HoVer-Net [15] , StarDist [33] and CPP-Net [9] . To ensure fair comparison, we follow the same data split protocolAccurately annotated edge as outlined in Section 4.1 and apply consistent data augmentation techniques for all methods. During training, for methods other than GeNSeg-Net, we adopt the midpoint position of the 4-pixel width annotation as ground truth since they require precise edge annotation.In Table 1 , the results of brightfield same tissue testing are shown. Notably, NucleiSegNet [22] , HoVer-Net [15] , StarDist [33] and CPP-Net [9] exhibit robust performance. SAM [20] exhibits advantages in natural images but tends to under-segment due to the significant adhesion of nuclei and minimal target variations in our task. In brightfield different tissue testing, notable variations in nucleus morphology not presented during training pose segmentation challenges. Testing metrics are presented in Table 1 . NucleiSegNet [22] and HoVer-Net [15] exhibit poor segmentation performance. In contrast, StarDist [33] and CPP-Net [9] show superior generalization ability, albeit with a performance decrease ranging from 4% to 13% across metrics. The decline is mainly due to their difficulty in detecting nuclei with different texture, shapes, sizes and signal strength. CPP-Net's reliance on similar nucleus shapes as a design prior further constrains its performance on datasets with diverse shapes. GeNSeg-Net maintains robustness and demonstrates superior results across all metrics compared to current SOTA methods.In fluorescence tissue testing, most methods surpass their brightfield counterparts as shown in Table 2 . This superiority arises from differences in staining and imaging, which result in fluorescence images with reduced background noise, heightened signal intensity and clearer nucleus edges compared to brightfield images. As a result, while maintaining consistent trends with brightfield testing, [9] 0.695 0.736 0.875 0.889 0.816 0.725 GeNSeg-Net (ours) 0.721 0.766 0.891 0.900 0.848 0.763 metrics in fluorescence different tissue testing exhibit no significant decrease compared to those in fluorescence same tissue testing. Overall, GeNSeg-Net maintains an advantage over other methods.Dataset. In contrast to our private dataset, images in DSB2018 [4] and BBBC006v1 [24] feature nuclei with lower density and more consistent morphology. This leads to the decent performance of HoVer-Net [15] , StarDist [33] and CPP-Net [9] , as presented in Table 3 . Compared to the testing results of private fluorescence data, nearly all metrics show improvement. However, upon closer examination, we notice the presence of adhesive nuclei that these methods fail to fully distinguish. Our method effectively reduces false positives and negatives, achieving higher segmentation accuracy. methods. For instance, in the DSB2018→BBBC006v1 task, GeNSeg-Net exhibits improvements of 1.9%, 2.3%, 0.8% and 1.9% in AJI, AJI+, Dice and PQ, respectively. It's noteworthy that GeNSeg-Net significantly improves all metrics in the BBBC006v1→DSB2018 task, though the scores remain below those in the Private→DSB2018 task. This highlights the importance of increasing the diversity of tissue types in the training data, even if the types in testing are not present during training.5.1.4 Inference Time. We evaluate the inference speed of all methods on our private dataset using the same computer as outlined in Section 4.1 and 4.3. Applying same post-processing method to 512 × 512 images, the average inference time per image for SAM [20] , NucleiSegNet [22] , HoVer-Net [15] , StarDist [33] , CPP-Net [9] and GeNSeg-Net is 1.938, 0.548, 1.863, 0.203, 0.353 and 0.242 seconds, respectively, covering both prediction and post-processing time. Obviously, StarDist exhibits the fastest inference speed. Despite being a two-stage method, GeNSeg-Net's lightweight design enables faster processing compared to most existing methods.In this section, we qualitatively compare all methods on our private dataset, DSB2018 [4] and BBBC006v1 [24] . Results for the private dataset are shown in Fig. 4 , while those for DSB2018 and BBBC006v1 are in Fig. 5 , in line with our previous analysis. GeNSeg-Net, incorporating the enhancement model, excels in identifying weakly stained nuclei, as indicated by the red boxes in Fig. 4 . It also effectively distinguishes adhesive nuclei and reduces over-segmentation, as shown by the purple boxes in Fig. 4 and 5 . Furthermore, thanks to our segmentation model, GeNSeg-Net produces more natural contours, better capturing the true shapes of nuclei. We conduct ablation studies on all brightfield and fluorescence testing data in our private dataset. Initially, the efficacy of the enhancement model is demonstrated. We then assess the discriminator's effectiveness and determine the generator's structure. Qualitative comparisons depicted in Fig. 6 show enhanced images in the first row, predicted class maps in the second row and segmentation results in the final row, with the top-left image representing the ground truth. Evaluation metrics from the ablation study of each component in GeNSeg-Net and the generator are presented in Table 5 and 6, where "gen", "dis", "enh model", "seg model", "ds" and "us" represent the generator, discriminator, enhancement model, segmentation model, downsampling and upsampling, respectively.Component in GeNSeg-Net. When solely relying on the segmentation model, it effectively identifies foreground nucleus regions, as indicated by the relatively high Dice. However, due to the network's general design and lack of prior information, it struggles to distinguish adhesive nuclei, resulting in noticeable under-segmentation, as shown in the first column of Fig. 6 . To address this issue, we introduce an enhancement model aimed at enhancing morphological features, i.e., ensuring clear centers, solid filling and well-defined edges to simplify subsequent segmentation. Initially, the enhancement model employs ResNet as the generator's backbone, comprising 9 blocks, denoted as "basic" in Table 5 and 6 . We adopt a CNN-based discriminator proposed in [17] , which discriminates at the patch level, as the "basic" discriminator. Integrating the enhancement model significantly improves segmentation performance, with AJI, AJI+, Dice, DQ, SQ and PQ reaching 0.682, 0.715, 0.844, 0.853, 0.835 and 0.712, respectively, slightly surpassing CPP-Net [9] . In our straightforward generation task, increasing the number of blocks adds complexity to training, while reducing blocks leads to insufficient model learning, reflected in declining evaluation metrics. We further attempt to employ a standard U-Net [32] as the generator's backbone, but it results in enhanced images with increased artifacts and noise. It performs worse in distinguishing the two foreground classes, i.e., nucleus bodies and pending edges, leading to increased adhesion, as illustrated in the third column of Fig. 6 . When both the generator and discriminator follow a "basic" structure, as seen in the red box in the second column of Fig. 6 , the model struggles to differentiate the two foreground classes, resulting in erroneous enhancement. Abrupt texture and noise is obvious. Our designed discriminator focuses more on the semantic relationships among classes, enabling better differentiation. It also improves texture and shape in the enhanced image. With our discriminator, AJI, AJI+, Dice and PQ are increased by 0.3%, 1.7%, 1.1% and 3.1%. Compared to the basic generator, our generator improves all metrics to optimal levels, achieving scores of 0.721, 0.759, 0.860, 0.912, 0.853 and 0.778, respectively.Study on the Generator in Enhancement Model. We demonstrate each component's role in the generator of the enhancement model. Integrating downsampling into the basic generator effectively aggregates nucleus pixel features, leading to a clearer separation between classes, as evidenced by the purple boxes in the fourth and fifth columns of Fig. 6 . The improved differentiation directly reduces false positives and false negatives. Notably, Table 6 shows that SQ remains unchanged while DQ exhibits a 3.6% improvement, which validates our findings. Additionally, the introduction of the Res-CBAM module further suppresses irrelevant information and noise. Improvement in non-nucleus positions can be observed from the orange boxes in the fifth and sixth columns of Fig. 6 . All evaluation metrics for this structure reach optimal levels. 
conclusion: 
 In this study, we propose a general framework for nucleus segmentation in IHC images. It comprises two stages: initial enhancement followed by segmentation. A classic generative architecture and segmentation network demonstrate the effectiveness of our framework. In the enhancement model, we design a lightweight generator and discriminator to improve both enhancement robustness and computational efficiency. Comparisons against existing methods on both the private and public datasets highlight our method's SOTA accuracy. Cross-dataset evaluation further validates its superior generalization ability. Additionally, our method exhibits highly competitive processing speed. Further discussions on methodology and future work are presented in supplementary materials.  
 
 Appendix:  


Decision: This paper proposed a general segmentation for any nucleus in immunohistochemistry images. Three reviewers confirmed the merits of this paper and recommended acceptance with 1 weak accept and two borderline accepts. Given the consensus of reviewers, a decision of accept is recommended. I do suggest the authors revise the manuscript by taking all the reviewers’ comments into consideration.

Reviews: review: paper_summary: paper_summary: This paper introduces GeNSeg-Net, a novel framework for Immunohistochemistry (IHC) that effectively enhances and segments nuclei with minimal data, addressing the challenges of variability from several factors. It improves accuracy and generalization across different conditions, outperforming existing methods, and will be available for both research and clinical use.
strength_weakness: strength_weakness: 1.High Generalization Ability: GeNSeg-Net can effectively segment nuclei across diverse tissue types and imaging techniques, even with high variability and limited training data. This broad applicability is essential for practical deployment in varied medical settings.
2.Enhanced Accuracy: The two-stage process involving an enhancement model and a segmentation model simplifies the segmentation task and leads to higher accuracy. This is achieved by standardizing nucleus morphology before segmentation.
limitations: limitations: 1.Dependency on Specific Enhancements: The success of the segmentation relies heavily on the performance of the enhancement stage. Inaccuracies or failures in enhancement can propagate errors to the segmentation output.
paper_summary: paper_summary: This paper presents a nuclei segmentation framework by combining GAN and U-Net. The GAN model can enhance input images while the U-Net can use enhanced input to predict segmentation maps.
strength_weakness: strength_weakness: 1) Using GAN to enhance image quality first is novel.

2) The proposed framework is simple and easy to catch. Experiments demonstrate the effectiveness of the proposed GeNSeg-Net.
limitations: limitations: 1) There are some other nuclei datasets, such as MoNuSeg (A Multi-Organ Nucleus Segmentation Challenge), TNBC (Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map) and CPM-17 (Methods for segmentation and classification of digital microscopy tissue images). Are these images included in the DSB2018 and BBBC006v1 datasets? If not, it would be better to provide more experiment results. 

2) Since this paper states the generator and discriminator are lightweight networks, could this paper provide a complexity analysis of GeNSeg-Net? For example, the params, FLOP and running time of GeNSeg-Net with and without this generator. How much did training costs increase by utilizing this GAN model?  How about the complexity of other compared methods? 

3) Could this paper provide more details on how to calculate pixel-to-boundary Euclidean distance to further categorise each pixel and how to adjust their stain intensity?

4) As shown in Figure 2, the function of Generator is to generate initial segmentstion maps and followed by an U–Net to revise this segmentation maps. Based on this function, is this framework actually a two–step segmentation method?

5) Why this paper utilise a Generator in here instead of any segmentation networks? 

6) What are the inputs of the U–Net?
paper_summary: paper_summary: The paper proposes a nuclei segmentation framework for IHC images called GeNSeg-Net. The framework includes an enhancement model, which is a GAN, and a segmentation model, which is a U-Net. The GAN model generates synthetic nuclei images from various types of IHC images. The U-Net model uses these synthetic nuclei labels to enhance its segmentation ability. The segmentation results show that the proposed framework surpasses some of the existing methods.
strength_weakness: strength_weakness: 1. The framework is effective, efficient, and demonstrates strong cross-type generalization ability.
2. The paper is well-organized, and the experiments are sufficient.
limitations: limitations: 1. In Figure 2, the detailed explanations of the 'Copy' and 'Transformation' between ideal nuclei and annotations are missing. Additionally, the usage of the enhanced image Y in U-Net is not clearly illustrated.

2. In contribution 1, what does 'training on a small subset of types' mean? Does this refer to semi-supervised learning or to generalization ability? What specialized model design have you implemented to achieve this?

3. Additional nuclei segmentation methods need to be compared to adequately demonstrate the effectiveness of the proposed method:
[1] Chen, Qi, et al. "PCTrans: Position-Guided Transformer with Query Contrast for Biological Instance Segmentation." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.
[2] Hörst, Fabian, et al. "Cellvit: Vision transformers for precise cell segmentation and classification." Medical Image Analysis 94 (2024): 103143.