GuidedNet: Semi-Supervised Multi-Organ Segmentation via Labeled Data Guide Unlabeled Data

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 13:33:53

Abstract: Semi-supervised multi-organ medical image segmentation aids physicians in improving disease diagnosis and treatment planning and reduces the time and effort required for organ annotation. Existing state-of-the-art methods train the labeled data with ground truths and train the unlabeled data with pseudo-labels. However, the two training flows are separate, which does not reflect the interrelationship between labeled and unlabeled data. To address this issue, we propose a semi-supervised multi-organ segmentation method called GuidedNet, which leverages the knowledge from labeled data to guide the training of unlabeled data. The primary goals of this study are to improve the quality of pseudo-labels for unlabeled data and to enhance the network's learning capability for both small and complex organs. A key concept is that voxel features from labeled and unlabeled data that are close to each other in the feature space are more likely to belong to the same class. On this basis, a 3D Consistent Gaussian Mixture Model (3D-CGMM) is designed to leverage the feature distributions from labeled data to rectify the generated pseudo-labels. Furthermore, we introduce a Knowledge Transfer Cross Pseudo Supervision (KT-CPS) strategy, which leverages the prior knowledge obtained from the labeled data to guide the training of the unlabeled data, thereby improving the segmentation accuracy for both small and complex organs. Extensive experiments on two public datasets, FLARE22 and AMOS, demonstrated that GuidedNet is capable of achieving state-of-the-art performance.

Main paper: 
 related work 2.1 semi-supervised medical image segmentation: 
 Semi-supervised segmentation methods applicable to medical images can be classified into types: consistency learning-based [2, 12, 31] and pseudo-labeling methods. One example of a consistency learning-based method is the mean teacher (MT) method [32] , which employs both teacher and student models and enforces consistency loss based on perturbations in the unlabeled data. A number of studies refined the MT method by exploring various perturbation techniques and strategies [5, 25, 29] . In contrast, pseudolabeling methods generate pseudo-labels for unlabeled data to guide the learning process [9, 19, 33] . The combination of consistency regularization and pseudo-labeling has become a leading approach for the semi-supervised semantic segmentation of medical images [26, 30] . Huang et al. [11] developed a novel convolutional neural network (CNN)-Transformer learning framework that effectively segmented medical images by producing complementary and reliable features and pseudo-labeling with bi-level uncertainty. Yuan et al. [42] proposed a new consistency regularization framework called mutual knowledge distillation (MKD), which was combined with data and feature augmentation. Recently, an increasing number of methods are focusing on enhancing the quality of pseudo-labels. For example, Zhao et al. [48] introduced rectified pseudo supervision to improve robustness under variant appearances in the image space. In addition, Chen et al. [5] utilized automatic clustering to model multiple prototypes, which helped alleviate the confirmation bias arising from noise and false labels. Accurately segmenting multiple organs is crucial for a variety of medical procedures [8] . Zhao et al. [47] proposed a lightweight network designed for multi-organ segmentation of abdominal computed tomography (CT) scans called LCOV-Net, which can segment 16 organs. LCOV-Net is a supervised network that relies on the quantity and quality of labeled data. Recently, a number of semi-supervised learning methods were developed for multi-organ segmentation in abdominal CT scans [23] . For example, Zhou et al. [49] proposed a deep multi-planar co-training (DMPCT) method that extracts consensus information from multiple views. In addition, Xia et al. [39] designed a Uncertainty-aware multi-view cotraining (UMCT) method that corrected the prediction consistency between different scales and estimated the uncertainty of each view to generate reliable pseudo-labels. To mitigate the class-imbalance problem, Wang et al. [34] proposed a dual-debiased heterogeneous co-training (DHC) framework. Furthermore, Chen et al. [3] designed a data augmentation strategy (MagicNet) that corrected the original pseudo-label via cube-wise pseudo-label blending that incorporated crucial local attributes for identifying targets, especially small organs. The abovementioned semi-supervised learning methods improved the segmentation accuracy by utilizing unlabeled data; however, they failed to fully consider the underlying relationships between unlabeled and labeled images. 
method 3.1 overview: 
 An overview of the proposed GuidedNet is illustrated in Fig. 3 , which is based on the CPS framework. This framework encompasses two parallel segmentation networks, 𝑚𝑜𝑑𝑒𝑙 𝐴 and 𝑚𝑜𝑑𝑒𝑙 𝐵.These networks share a common structure, but they are initialized differently; however, both are trained on the same input image. The purpose of this approach is to ensure that both networks produce consistent outputs. For the unlabeled data, each segmentation network can generate a pseudo-label, which serves as an additional signal to supervise the other segmentation network.The training dataset, denoted as S, is a union of labeled and unlabeled data. It is represented as S = S 𝑙 ∪ S 𝑢 , where S 𝑙 = {(𝑥 𝑖 , 𝑦 𝑖 )} 𝑁 𝑖=1 represents the labeled data, S 𝑢 = {𝑥 𝑖 } 𝑀 𝑖=1 represents the unlabeled data, and 𝑁 and 𝑀 indicate the number of labeled and unlabeled data, respectively (𝑀 ≫ 𝑁 in most cases). In this context, 𝑥 𝑖 ∈ R 𝐷 ×𝐻 ×𝑊 denotes the input volume and 𝑦 𝑖 ∈ R 𝐾 ×𝐷 ×𝐻 ×𝑊 denotes the ground truth mask, where 𝐾 is the number of classes (including the background) and 𝐻 , 𝑊 , and 𝐷 represent the height, width, and depth of the input medical volume, respectively. The goal is to train a segmentation network based on S 𝑙 and S 𝑢 that correctly predicts labels for unseen data.At each training step, labeled data (denoted as B 𝑙 ) and unlabeled data (denoted as B 𝑢 ) are sampled and fed into 𝑚𝑜𝑑𝑒𝑙 𝐴 and 𝑚𝑜𝑑𝑒𝑙 𝐵, respectively. The supervised loss function is applied to the labeled data, guiding each segmentation head to generate a prediction mask that closely aligns with the ground-truth mask viaEQUATIONwhere L 𝑠 = 1 2 [L 𝐷𝑖𝑐𝑒 +L 𝑐𝑒 ]; L 𝐷𝑖𝑐𝑒 and L 𝑐𝑒 represent the Dice and cross-entropy losses, respectively; and 𝑝 (•) 𝑖 is the output probability map. For all data, the KT-CPS loss and the CGMM loss are employed.The optimization target is to minimize the overall loss, which can be formulated asEQUATIONwhere L 𝑠𝑢𝑝 , L 𝑘𝑡 -𝑐𝑝𝑠 , and L 𝑐𝑔𝑚𝑚 denote the supervised, KT-CPS, and CGMM losses, respectively, and 𝜆 𝑢 and 𝜆 𝑔 are the hyperparameters that can adjust the influence of each loss component within the model. We empirically set 𝜆 𝑢 to 0.1 and used the CPS weight rampup function [41] to gradually enlarge the L 𝑘𝑡 -𝑐𝑝𝑠 ratio. In existing SSL segmentation methods, the training flows for labeled and unlabeled data are entirely separate, which overlooks the interrelationship between them. This oversight may lead to errors when generating pseudolabels.To rectify the generation of pseudo-labels for unlabeled data and guide the prediction process for unlabeled data, we introduce a 3D-CGMM that models the feature distribution of each class from labeled data. This approach ensures that the predictions for the unlabeled data do not solely depend on their own data but are also guided by the knowledge encapsulated in the labeled data, thereby enhancing the reliability and accuracy of the pseudo-labels.3D-CGMM Formulation. Given input data with 𝐾 annotated classes, a 3D-CGMM with 𝐾 Gaussian mixtures is constructed. The center of the features from different categories are regarded as the centroids of the distinct Gaussian mixtures. For the 𝑘 𝑡ℎ Gaussian mixture, the mean features 𝜇 𝑘 of labeled voxels 𝑥 𝑖 belonging to the 𝑘 𝑡ℎ class are initially computed, as represented byEQUATIONwhere 𝑓 (𝑥 𝑖 ) are the deep features of labeled voxels 𝑥 𝑖 , which are produced using the features before the classification layer of the segmentation model. After obtaining 𝜇 𝑘 , the variance 𝜎 𝑘 of the 𝑘 𝑡ℎ Gaussian mixture can be calculated asEQUATIONwhere 𝑃 𝑘 denotes the segmentation prediction scores of the 𝑘 𝑡ℎ category. Each voxel 𝑥 𝑖 follows the probability density of the 𝑘 𝑡ℎ Gaussian mixture, which is calculated using the Gaussian probability density function:EQUATIONThen, following Bayes' rule [16] , the posterior is derived asEQUATIONwhere 𝑃 (𝑥 𝑖 | 𝑘 ) denotes the posterior probability that voxel 𝑥 𝑖 belongs to the 𝑘 𝑡ℎ Gaussian mixture, 𝜋 𝑖 denotes the mixing coefficient defined as 1 𝐾 . The CGMM predictions 𝐺 are then produced using:EQUATION3D-CGMM Training Loss. Previous methods suggest using Expectation-Maximization (EM) algorithms [16, 27, 36] to formulate the Gaussian Mixture Model (GMM). However, this approach requires prior estimates and iterative updates of the parameters. In SSL, labeled voxels with available labels can serve as precise prior information useful for formulating the GMM. To train a 3D-CGMM effectively, selecting only voxels that have labels as reliable sources of information is essential.In this paper, instead of using time-consuming EM algorithms, we leverage the reliable information of labeled data and employ an effective function L 𝑡𝑟𝑎𝑖𝑛 to adaptively optimize the 3D-CGMM. The 3D-CGMM training loss L 𝑡𝑟𝑎𝑖𝑛 contains four parts: a self-supervision loss L 𝑠𝑒𝑙 𝑓 , a ground truth loss L 𝑔𝑡 , a maximization loss L 𝑚𝑎𝑥 , and a consistency loss L 𝑐𝑜𝑛𝑠 . First, the ground truth mask 𝑦 is assigned to supervise 𝐺 according to:EQUATIONSecond, the CGMM prediction 𝐺 and output probability map 𝑃 of the model are utilized for self-supervision, employing a cross-entropy-based function to compute the loss L 𝑠𝑒𝑙 𝑓 , which is expressed asEQUATIONThird, for the purpose of learning discriminative Gaussian mixtures, a maximization loss L 𝑚𝑎𝑥 is used to enlarge the distance between the centroids of different Gaussian mixtures:EQUATION)Additionally, in the semi-supervised framework, both 𝑚𝑜𝑑𝑒𝑙 𝐴 and 𝑚𝑜𝑑𝑒𝑙 𝐵 generate predictions for the unlabeled data, which are fed into the 3D-CGMM. The aim for the 3D-CGMM is to maintain consistency between the CGMM predictions 𝐺 ′𝐴 and 𝐺 ′𝐵 from 𝑚𝑜𝑑𝑒𝑙 𝐴 and 𝑚𝑜𝑑𝑒𝑙 𝐵, respectively, as much as possible. Therefore, we employ the mean squared error (MSE) loss [35] , which is expressed asEQUATION)The total loss function L 𝑡𝑟𝑎𝑖𝑛 for training the 3D-CGMM is expressed asEQUATIONwhere 𝜆 𝑐 is a hyperparameter used to balance the intensity of L 𝑐𝑜𝑛𝑠 .Rectify Pseudo-labels. The 3D-CGMM simultaneously generates CGMM predictions 𝐺 ′ for the unlabeled data during the process of training the 3D-CGMM. Thus, 𝐺 ′ serves as a supervisory signal for the initial predictions of the unlabeled data, thereby facilitating the refinement of the pseudo-labels under the guidance of the labeled data. This methodology leverages the intrinsic data distribution captured by the 3D-CGMM and harnesses the predictive power of 𝐺 ′ to optimize the pseudo-labels, integrating the reliability of the labeled data to inform the processing of the unlabeled data viaEQUATION)Overall Loss. The total loss function L 𝑐𝑔𝑚𝑚 for the 3D-CGMM is expressed asEQUATION)As mentioned in the previous section, significant disparities in organ sizes have been observed within multi-organ datasets containing labeled data.Organ size disparities constitute one of many aspects that need to be considered in organ segmentation. Many complexities associated with this process also need to be taken into account. Therefore, organs that are difficult to segment remain the primary focus of this study.To address this challenge, we propose a strategy known as KT-CPS. This approach, which is guided by the prior knowledge obtained from the labeled data, forces the model to pay more attention to the organs that are either small or challenging to segment, thereby rebalancing the learning process for unlabeled data.We denote the softmax probabilities of the 𝑗 𝑡ℎ voxel of the 𝑖 𝑡ℎ data generated by 𝑚𝑜𝑑𝑒𝑙 𝐴 and 𝑚𝑜𝑑𝑒𝑙 𝐵 as p 𝐴 𝑖 𝑗 ∈ R 𝐾 and p 𝐵 𝑖 𝑗 ∈ R 𝐾 , respectively, where 𝐾 is the number of classes (including the background). Specifically, pseudo-labels are generated viaEQUATIONwhere ŷ𝐴 𝑖 𝑗 and ŷ𝐵 𝑖 𝑗 are the pseudo-labels of the 𝑗 𝑡ℎ voxel of the 𝑖 𝑡ℎ data given by 𝑚𝑜𝑑𝑒𝑙 𝐴 and 𝑚𝑜𝑑𝑒𝑙 𝐵, respectively, and 𝑝 𝐴 𝑖 𝑗 (𝑘 ) and 𝑝 𝐵 𝑖 𝑗 (𝑘 ) are the p 𝐴 𝑖 𝑗 and p 𝐵 𝑖 𝑗 values for the 𝑘 𝑡ℎ dimension, respectively. The predicted categories from the model are compared to the true categories within a batch of labeled data usingR 𝐴 𝑘 = 𝑁 𝐴 𝑘 max 𝑁 𝐴 𝑘 𝐾 𝑘=0 , R 𝐵 𝑘 = 𝑁 𝐵 𝑘 max 𝑁 𝐵 𝑘 𝐾 𝑘=0, 𝑘 = 0, 1, . . . , 𝐾,where𝑁 (•)𝑘 denotes the voxel number of the 𝑘 𝑡ℎ organ in labeled images in which the predicted category matches the actual label category. Based on the voxel proportions, the weight of each organ is calculated via:𝑤 𝐴 𝑘 = max log R 𝐴 𝑘 𝐾 𝑘=0 log R 𝐴 𝑘 , 𝑤 𝐵 𝑘 = max log R 𝐵 𝑘 𝐾 𝑘=0 log R 𝐵 𝑘, 𝑘 = 0, 1, . . . , 𝐾 .(17) During training, the exponential moving average (EMA) of these parameters is leveraged to enhance the prediction stability after each training round viaEQUATIONwhere 𝑚 is the momentum parameter, which is experimentally determined to be 0.99. Based on the designed weights, the KT-CPS loss is defined asEQUATIONwhere B = B 𝑙 ∪ B 𝑢 signifies that the batch B consists of both labeled (B 𝑙 ) and unlabeled (B 𝑢 )data. This combined batch is utilized in the calculation of the KT-CPS loss. 
experiments 4.1 experimental setup: 
 Datasets. We evaluate GuidedNet using two classical abdominal multiorgan segmentation datasets: FLARE22 [24] and AMOS [13] . The FLARE22 dataset consists of 13 classes of organs (with one background): the liver (Liv), spleen (Spl), pancreas (Pan), right kidney (R.kid), left kidney (L.kid), stomach (Sto), gallbladder (Gal), esophagus (Eso), aorta (Aor), inferior vena cava (IVC), right adrenal gland (RAG), left adrenal gland (LAG), and duodenum (Duo).The dataset also includes 2000 unlabeled 3D CT volumes. To partition the dataset, we divide the labeled images into training, validation, and test sets using a ratio of 6:2:2. In addition, we incorporate the unlabeled images into the training set, resulting in labeled data proportions of 50% (42 labeled cases and 42 unlabeled cases) and 10% (42 labeled cases and 378 unlabeled cases) within two training sets. The AMOS dataset is comprised of 300 CT images, which are annotated at the pixel level for 15 distinct abdominal organs, including two additional organs not found in the FLARE22 dataset: the bladder (Bl) and prostate/uterus (P/U). In our experiments, we divide the dataset into training, validation, and test sets using a ratio of 6:2:2. For the training set, the proportion of labeled data is set to 10% and 50%. Data Preprocessing. For both datasets, the data are preprocessed before the network is trained. Specifically, the orientations of all the CT scans are standardized in the left-posterior-inferior (LPI) direction. Additionally, the following three preprocessing procedures are applied. (1) The voxel values are clipped to the range of [-325, 325] Hounsfield units (HU) to enhance the contrast of the foreground organs and suppress the background interference. (2) The voxel spacing is standardized to [1.25, 1.25, 2.5 ]. (3) Min-max normalization is implemented via (𝑥 -𝑥 0.5 )/(𝑥 99.5 -𝑥 0.5 ), where 𝑥 0.5 and 𝑥 99.5 represent the 0.5th and 99.5th percentiles of 𝑥, respectively.The regions enclosed by the dashed yellow boxes indicate misclassification executed by the model; our method corrects these errors within these regions.Implementation Details. Following the settings used in previous studies [34] , multiple data augmentation methods, i.e., random crop and random flip are adopted. The random crop size is set to 64 × 128 × 128. The batch size is set to eight, comprising four labeled and four unlabeled data. For training, we employ the stochastic gradient descent (SGD) optimizer for training with a weight decay of 0.0001 and momentum of 0.9 [22] . For both the FLARE22 and AMOS datasets, all the methods are trained using 20000 iterations and an initial learning rate of 0.1. During the network training, a polynomial learning rate policy is employed during network training to decrease the learning rate according to the formula (1 -𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛 𝑚𝑎𝑥 _𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛 ) 0.9 [3] , where 𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛 and 𝑚𝑎𝑥_𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛 denote the current iteration and the total number of iterations, respectively. During the inference process, the final volumetric segmentation is generated using a sliding-window strategy, with a stride of 32 × 80 × 80 and the sliding-window approach employs a patch size of 64 × 160 × 160. We conduct the experiments on Pytorch [28] with two NVIDIA A100 GPUs.Evaluation Metrics. In our experiments, the segmentation performances of the different methods are evaluated using two standard evaluation metrics: the Dice and Jaccard indices (referred to as Jaccard). The Dice and Jaccard values range from 0 to 1, with higher scores indicating more accurate segmentation. To reduce the randomness of the network training, Table 1 : Quantitative results (mean Dice for each organ, mean and SD of Dice, and mean and SD of Jaccard) for different methods applied to the FLARE22 dataset. 'Sup only' denotes supervised 3D U-Net [6] . The bold and underlined text denote the best and second best performances, respectively. To further assess the performance of GuidedNet, we compare our method to nine state-of-the-art semi-supervised segmentation methods: (1) deep adversarial networks (DAN) [46] , (2) MT [32] , (3) uncertainty-aware mean teacher (UA-MT) [41] , (4) the shape-aware semi-supervised network (SASSnet) [15] , (5) the dual-task consistency (DTC) framework [21] , (6) CPS [4] , (7) calibrating label distribution (CLD) for segmentation [17] , (8) the dual-debiased heterogeneous co-training (DHC) framework [34] , (9) MagicNet [3] and the fully supervised 3D U-Net [6] . For all semi-supervised methods, we utilize 3D U-Net as the backbone. Comparison experiments are performed on the FLARE22 and AMOS datasets. FLARE22. For the FLARE22 dataset, we train the semi-supervised models using training sets with labeled data proportions of 50% and 10%. The supervised 3D U-Net is trained using all 42 labeled data cases. The quantitative results obtained using the different methods are listed in Table .1. Compared to the supervised 3D U-Net, all semi-supervised methods achieve higher mean Dice and mean Jaccard by utilizing the unlabeled data. Guid-edNet significantly outperforms all the other methods, achieving a superior state-of-the-art performance. With only 50% labeled data, its mean Dice is 82.13% and its mean Jaccard is 72.47%, surpassing other semi-supervised methods by 1.90%-7.27% and 3.39%-8.74%, respectively. GuidedNet also attains the highest mean Dice and mean Jaccard for the training set with a labeled data proportion of 10%. Additionally, GuidedNet exhibits excellent performance in terms of the mean Dice of large organs (i.e., Liv, L.Kid, and R.kid), with significant improvements in the Dice of smaller organs (i.e., Gal, RAG, and LAG), and complex organs (i.e., Sto and Pan). In terms of Dice, the results for the smallest organ (i.e., LAG) surpasses those of other semi-supervised methods by 7.44%-26.51%, and the results for the most complex organ (i.e., Sto) also outperforms those of other semi-supervised methods by 1.68%-16.07%. These results validate the superiority of the 3D multi-organ segmentation performance of GuidedNet, particularly for small and complex organs. Fig. 4 illustrates the qualitative results of different methods when using 378 unlabeled images from the FLARE22 dataset. The figure shows that GuidedNet accurately segments organs of various sizes and shapes. Furthermore, compared to other methods, GuidedNet generates clearer and more well-defined organ boundaries, mitigating issues with abnormal organ segmentation.To further validate GuidedNet, we conduct experiments on the AMOS dataset. The results demonstrate that our method outperforms limited supervision (Sup only) by a significant margin, with mean Dice improvements of 23.95% and 11.65% for 18 and 90 labeled samples, respectively. Additionally, our method has shown greater robustness compared to other semi-supervised methods on the AMOS dataset. For both training sets with 10% and 50% labeled data, CLD achieves a higher mean Dice, and MigicNet achieves a higher mean Jaccard than the other methods. Under 10% labeled data, our approach exhibits a 3.38% improvement in Dice and a 2.10% improvement in Jaccard compared to the current top-ranked method. Similarly, with 50% labeled data, our approach demonstrates a 3.85% improvement in Dice and a 3.20% improvement in Jaccard compared to the current top-ranked method. To provide a qualitative comparison, the visualization results for one test case of three transverse sections, along with 3D views, are shown in Fig. 5 . Our approach is consistent with the quantitative In our ablation study, we investigate several aspects that can impact the performance of GuidedNet, including each component, the utilization of various loss terms in the 3D-CGMM training loss, and the settings of the hyperparameters. We perform the ablation studies on the FLARE22 dataset with a training set comprised of 42 labeled and 378 unlabeled cases.The Effectiveness of Each Component in GuidedNet. We conduct ablation studies to show the impact of each component in GuidedNet, the results of which are shown in Table 3 . For a fair comparison, we evaluate one component per experiment while keeping the others fixed. The first row in Table 3 represents the CPS baseline, on which our method is based. Compared to the baseline, employing the KT-CPS strategy and 3D-CGMM individually yields improvements in the mean Dice of 1.96% and 2.68%, respectively. Additionally, integrating the KT-CPS and 3D-CGMM together yields the highest mean Dice (83.28%) and mean Jaccard (72.58%), outperforming the baseline by 3.96% and 4.44%, respectively. For the segmentation of small and complex organs, the proposed KT-CPS strategy achieves more accurate segmentation results than the baseline. Specifically, the baseline with the KT-CPS strategy achieves a higher Dice for the segmentation of small organs (i.e., LAG, 62.53%; RAG, 66.50%; Gal, 75.61%; and Duo, 70.43%) and complex organs (i.e., Pan, 73.53%; and Sto, 81.53%). When the baseline is combined with the 3D-CGMM, improvements in the Dice are observed for all organs except the Spl and R.kid. This indicates that the 3D-CGMM effectively enhances the quality of pseudo-labels,thereby improving the accuracy of semi-supervised multi-organ segmentation. The individual components employed in our GuidedNet demonstrate standalone benefits, and therefore combining them led to significantly improved optimization.Design Choices of 3D-CGMM Training Loss . Compared to the baseline method that used L 𝑔𝑡 only, employing the self-supervision loss L 𝑠𝑒𝑙 𝑓 and maximization loss L 𝑚𝑎𝑥 individually yields improvements in the mean Dice of 1.79% and 1.73%, respectively. Integrating the self-supervision loss L 𝑠𝑒𝑙 𝑓 and maximization loss L 𝑚𝑎𝑥 together yields 83.28% for the mean Dice and 72.58% for the mean Jaccard. Adopting the consistency loss L 𝑐𝑜𝑛𝑠 yields the highest mean Dice (83.64%) and mean Jaccard (73.08%), outperforming the baseline by 2.35% and 2.32%, respectively. The consistency loss L 𝑐𝑜𝑛𝑠 also improved the performances, especially for SSL.Ablation Study on HyperParameters. To validate the robustness of GuidedNet, we conduct ablation studies on the hyperparameters, including the 3D-CGMM training loss weight 𝜆 𝑔 , and a selection of layer features in the decoder used to train the 3D-CGMM. The quantitative results for different hyperparameters are presented in Fig. 6 .(a) The loss weight 𝜆 𝑔 : 𝜆 𝑔 determines the contribution of L 𝑐𝑔𝑚𝑚 to the total loss. As shown in Fig. 6 (a), 𝜆 𝑔 = 0.3 results in the highest mean Dice and mean Jaccard.(b) Selection of Feature Layers: Layers 1-4 represent the first to fourth layers of the decoder network (from deep to shallow). Utilizing the features from Layer 4 to model the 3D-CGMM produced the best results, but the effectiveness decreased with shallower layers. This can be attributed to the excessive abstraction in fitting the GMM in high-dimensional space, which may lead to overfitting or instability.  
conclusion: 
 In this paper, we propose a simple yet effective framework for semi-supervised multi-organ segmentation called GuidedNet. It leverages the knowledge obtained from labeled data to guide the training of unlabeled data, which improves the quality of pseudo-labels for unlabeled data and enhances the network's learning capability for both small and complex organs. Two essential components are proposed: (1) 3D-CGMM models the feature distribution of each class from the labeled data to guide the prediction process for the unlabeled data, and (2) KT-CPS re-weights the pseudo supervised loss terms based on the prior knowledge obtained from the labeled data, forcing the model to pay more attention to organs that are either small or challenging to segment. Comparative experiments against state-of-the-art methods and extensive ablation studies demonstrate the effectiveness of GuidedNet. The visualization results demonstrate that GuidedNet performs well and is effective.  
: 
 does not reflect the interrelationship between labeled and unlabeled data (i.e., CPS [4] , ARCO [40] , UCMT [30] ). (b) In the FLARE22 dataset, rankings are provided for both the sizes of all foreground organs and the Dice. (c) Our GuidedNet, which is comprised of two components: 3D-CGMM and KT-CPS.in obtaining labeled images, researchers have attempted to utilize semi-supervised learning (SSL) [5, 18, 43] , which entails training segmentation models using a small number of labeled images and a large number of unlabeled images.The continuous development of SSL techniques has facilitated significant achievements in the field of organ segmentation. One important technique is pseudo-labeling [10] , which generates pseudolabels from unlabeled images for subsequent training [4, 14, 20, 30] . The most representative method is Cross Pseudo Supervision (CPS) [4] , which imposes the consistency on two segmentation networks perturbed with different initialization for the same input image. Labeled data is supervised by ground truths in both segmentation networks, while unlabeled data from one perturbed segmentation network is supervised by pseudo-labels generated by the other segmentation network.However, the processes employed to train the labeled and unlabeled data in these methods are separated, as depicted in Fig. 1(a) . This separation causes a significant drawback: the quality of the pseudo-labels generated during training depends exclusively on the unlabeled data and the segmentation performance of the network, with no regard for the interrelationship between the labeled and unlabeled data. This oversight leads to low-quality pseudolabels, which adversely impacts the overall performance of the model. Additionally, widely used SSL multi-organ medical image segmentation methods focus on enhancing the learning ability of small organs due to the prevalent issue of class imbalance in medical image datasets. Specifically, the inherent complexity of organ segmentation must not be overlooked. For instance, the stomach, which is the second largest organ in abdominal multi-organ dataset, ranks only eleventh in terms of segmentation Dice similarity coefficient (referred to as Dice), as shown in Fig. 1(b ). This discrepancy highlights that the stomach's substantial size does not mitigate the segmentation difficulties posed by its complexity. Therefore, multi-organ segmentation is a complex task that requires careful consideration of the varying size and complexity levels of different organs. [7] between the feature centers of the labeled and unlabeled data for each organ.To address the above issue, we propose a novel semi-supervised multi-organ segmentation method named GuidedNet (as depicted in Fig. 1(c) ). The main objective of GuidedNet is to leverage the knowledge obtained from labeled data to guide the training of unlabeled data. To accomplish this, GuidedNet is designed with two main components: a 3D Consistent Gaussian Mixture Model (3D-CGMM) and Knowledge Transfer Cross Pseudo Supervision (KT-CPS) strategy. As illustrated in Fig. 2 , when a smaller Euclidean distance [7] exists between the feature centers of labeled and unlabeled data in a particular category, the Dice for that category is usually higher. Thus, when a high degree of similarity exists between the feature spaces of labeled and unlabeled voxels, the likelihood that they contain the same semantic information is high; hence, they should share their label. On this basis, we develope a 3D-CGMM that models the labeled data distribution of each class in the feature space using Gaussian mixture. During training, the 3D-CGMM is optimized using a training loss to ensure that it can adapt in real-time. Subsequently, the 3D-CGMM is used to align the features of unlabeled voxels with the appropriate category-specific Gaussian mixture, which produces CGMM predictions. These predictions are then used to supervise segmentation predictions of the unlabeled data, leveraging guidance from the labeled data. This provides an additional training signal that assists in rectifying the pseudo-labels generated for the unlabeled data.To address the challenging task due to differences in size and complexity among the different organs, we also implement the KT-CPS strategy, which measures organ learning challenges by comparing voxel predictions for the labeled data with the corresponding ground truth. Furthermore, we re-weight the pseudo supervised loss terms based on the learning challenges of different organs to overcome the segmentation difficulties posed by class imbalance and organ complexity.The primary contributions of this study are as follows:(1) We propose to leverage the knowledge from labeled data to rectify the generated pseudo-labels by using a 3D-CGMM that utilizes the feature distribution of the labeled data to generate CGMM predictions to guide the learning of unlabeled data.(2) We design a KT-CPS strategy, which guides the training of unlabeled data to learn prior knowledge from the labeled data and enhances the learning ability of small and complex organs.(3) Extensive experiments have been conducted to validate the effectiveness of our proposed GuidedNet. The results of these experiments have demonstrated solid performance gains across two public datasets. 
 
 Appendix:  


Decision: All the reviewers agreed to accept this paper after rebuttal. I, therefore, would recommend to accept it.

Reviews: review: paper_summary: paper_summary: This paper introduces a 3D Consistent Gaussian Mixture Model (3D-CGMM) tailored to utilize feature distributions from labeled data to refine generated pseudo-labels. Additionally, it proposes a Knowledge Transfer Cross Pseudo Supervision (KT-CPS) strategy, which leverages prior knowledge gleaned from labeled data to enhance the training of unlabeled data. This approach boosts segmentation accuracy for both small and intricate organs.
strength_weakness: strength_weakness: This paper is well written and the figures are great
limitations: limitations: Making the information of labeled and unlabeled data interact with each other during the training process is not novel for semi-supervised medical image segmentation. The authors should add more explanation to interpret why the proposed method is different with these relavant models like BCP [1]. Moreover, the proposed model should compare with these models.
[1] Y. Bai, D. Chen, Q. Li, W. Shen, and Y. Wang, “Bidirectional copy-paste for semi-supervised medical image segmentation,” in CVPR, pp. 11514–11524, 2023.
paper_summary: paper_summary: Regarding semi-supervised multi-organ medical image segmentation, the paper proposes a method called GuidedNet, which uses the knowledge in the labeled data to guide the training of unlabeled data, aiming to improve the segmentation quality of unlabeled data. A 3D-CGMM method is proposed, which uses the feature distribution of the labeled data to correct the generated pseudo-labels; a KT-CPS strategy is proposed, which uses the prior knowledge obtained from the labeled data to guide the training of the unlabeled data, and enhances the network's learning ability for small and complex organs. Experiments on FLARE22 and AMOS show that GuidedNet can achieve advanced performance.
strength_weakness: strength_weakness: 1. The analysis of the problem is novel and uniquely considers the issue of poorer segmentation effects for small and complex organs in multi-organ medical image segmentation (using the stomach category in FLARE22 as an example).
2. State-of-the-art performance was achieved on the FLARE22 and AMOS datasets.
limitations: limitations: 1. The overall innovation of the paper is insufficient. Using labeled data to guide unlabeled data seems more like a combination of several papers (CAML[1], PRCL[2]), and it does not consider the uniqueness of medical organ segmentation using GMM.
2. The authors could consider showing the visualization of the stomach as a complex organ in Figure 1 or Figure 2. How did the authors define the stomach as a complex organ? Maybe it's just because the Dice is poor?
3. The order of Figure 2 could be sorted according to Dice or Euclidean distance, which would present a better visualization.
4. The representations in lines 1, 3, 4, and 13 of Algorithm 1 can be more accurate. The result can only be obtained after $x_i$ goes through the model.
5. In 3D-CGMM, I hope the authors can provide relevant explanations: What are the advantages of modeling the feature space in the form of Gaussian distribution compared to Prototype and probabilistic representation?
6. The definition given by KT-CPS in the formula is to add a weight mechanism in CPS cross supervision, but the direction of knowledge in Figure 3 (from the green feature map to the blue feature map) is not clear enough.
7. The weight mechanism used in KT-CPS is the ratio of voxel numbers, and the paper lacks an explanation of the role of KT-CPS in dealing with complex organs.
8. The experiment lacks ablation of $L_{rectify}$.
9. The paper needs to add ablation experiments for the hyperparameter $\lambda_u$.

Reference

[1] Gao S, Zhang Z, Ma J, et al. Correlation-aware mutual learning for semi-supervised medical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2023: 98-108.

[2] Xie H, Wang C, Zheng M, et al. Boosting semi-supervised semantic segmentation with probabilistic representations[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(3): 2938-2946.
paper_summary: paper_summary: The paper introduces GuidedNet, a semi-supervised framework for multi-organ segmentation in medical images. The method leverages labeled data to guide the training of unlabeled data, improving pseudo-label quality and segmentation accuracy, particularly for small and complex organs. Key components include:

1. 3D Consistent Gaussian Mixture Model (3D-CGMM): Utilizes feature distributions from labeled data to refine pseudo-labels.
2. Knowledge Transfer Cross Pseudo Supervision (KT-CPS): Uses prior knowledge from labeled data to enhance the training process of unlabeled data.

GuidedNet's effectiveness is demonstrated through extensive experiments on the FLARE22 and AMOS datasets, outperforming state-of-the-art methods in terms of segmentation accuracy.
strength_weakness: strength_weakness: 1. Innovative Use of Labeled Data: The integration of 3D-CGMM and KT-CPS uniquely leverages labeled data to guide the segmentation process of unlabeled data, leading to higher accuracy.
2. Enhanced Segmentation for Challenging Organs: Demonstrates significant improvements in segmenting small and complex organs, which are typically difficult for existing methods.
3. Comprehensive Experiments: Extensive evaluations on multiple public datasets (FLARE22 and AMOS) show superior performance over nine state-of-the-art methods.
4. Robust Ablation Studies: Detailed ablation studies highlight the effectiveness of each component and the chosen hyperparameters, reinforcing the robustness of the framework.
limitations: limitations: The narration, drawings and experiments of the whole article are quite reasonable, but I have the following questions:
1. As for the motivation and design of KT-CPS, I know that CPS is an innovative work with good results. Could you please introduce the differences between CPS and KT-CPS in rebuttal?
2. In Table 1 and Table.2, Can you explain why the network does not achieve optimal results in some organs?
3. The combined use of 3D-CGMM and KT-CPS, along with specific hyperparameters, might complicate the implementation and tuning process for practical applications.
4. The GMM model takes a long time to compute, and 3DGMM is even longer. Does it take into account time and storage costs? The approach may face scalability issues when applied to extremely large datasets or when dealing with real-time applications due to its computational requirements.
5. Is any experiments conducted at 1% labeled Settings?