Cross-View Mutual Learning for Semi-Supervised Medical Image Segmentation

Authors: song  wu
xiaoyu  wei
xinyue  chen
jing  he
xiaorong  pu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-01 12:53:40

Abstract: Semi-supervised medical image segmentation has gained increasing attention due to its potential to alleviate the manual annotation burden. Mainstream methods typically involve two subnets, and conduct a consistency objective to ensure them producing consistent predictions for unlabeled data. However, they often ignore that the complementarity of model predictions is equally crucial. To realize the potential of the multi-subnet architecture, we propose a novel cross-view mutual learning method with a two-branch co-training framework. Specifically, we first introduce a novel conflict-based feature learning (CFL) that encourages the two subnets to learn distinct features from the same input. These distinct features are then decoded into complementary model predictions, allowing both subnets to understand the input from different views. More importantly, we propose a cross-view mutual learning (CML) to maximize the effectiveness of CFL. This approach requires only modifications to the model inputs and supervisory signals, and implements a heterogeneous consistency objective to fully explore the complementarity of model predictions. Consequently, the aggregated predictions can effectively capture both consistency and complementarity across two subnets. Experimental results on three public datasets demonstrate the superiority of CML over previous SoTA methods. Code is available at https://github.com/SongwuJob/CML.

Main paper: 
 introduction: 
 Accurate identification of tissue structures or lesion regions is an essential task in medical image processing, and has achieved remarkable strides with the advance of deep learning [11, 12, 37, 40] . However, most current approaches generally rely on a large amount of meticulously annotated data, which is quite time-consuming, requiring expertise [8, 10] . To alleviate tedious manual labeling, semi-supervised medical image segmentation (SSMIS) methods [7, 17, 28, 32] have gained prominence, where they require only a limited amount of labeled data, and utilize abundant unlabeled data to achieve satisfactory effectiveness.In SSMIS, the effective utilization of unlabeled data is an alwayson topic [35] . Mainstream methods typically involve two similar networks, and conduct pseudo-label supervision [23] or consistency regularization [13] to make two subnets producing consistent predictions. Mean Teacher (MT) is a typical framework and propels a series of SSMIS works [19] . One of the most representative works is FixMatch [18] , which employs the Teacher network to generate pseudo-labels on weakly-augmented unlabeled images, and supervises the Student network learning the strongly-augmented version. Based on this, BCP [1] copy-pastes labeled and unlabeled data bidirectionally for alleviating empirical distribution mismatch. Despite the superior results achieved by these methods, the limitation imposed by Exponential Moving Average (EMA) hinders the Teacher network to reason individually about inputs, which wastes the potential of the multi-subnet architecture [25] .The co-training framework is a promising method to this end, which allows the two subnets to reason the same inputs from different views, and transfer the knowledge learned from one view to another through cross supervision [26] . For instance, MC-Net [31] employs different decoder networks, and conducts the consistency objective to make all decoders producing consistent predictions. Based on this, MC-Net+ [29] further expands the type of decoder networks to explore multi-view consistency. CAML [6] based on mutual learning paradigm transfers the labeled prior knowledge to unlabeled data. Nevertheless, we observe that most previous cotraining methods typically lack an explicit constraint to encourage different subnets to learn complementary semantics. And we point out that such complementary information is essential to realize the potential of the multi-subnet architecture. In other words, most SS-MIS methods focus on two subnets making consistent predictions, ignoring that the complementarity of model predictions is equally crucial for learning unlabeled data.In order to exhibit the effectiveness of cross-view complementary information, we conduct the training process analysis for different frameworks, i.e., MT and co-training methods, as shown in Fig. 1 .There are three observations: (1) As shown in Fig. 2 (a), limited by EMA, the performance of Teacher network is largely determined by the Student network, wasting its learning capacity. (2) As shown in Fig. 2 (b), the co-training method can allow the two subnet to reason the input individually. However, due to the lack of explicit constraints, the two models gradually become consistent under cross supervision. (3) The proposed CML method encourages to make complementary model predictions. As shown in Fig. 2(c ), the performance of the two subnets presents an adversarial rise, and the aggregation results benefited from cross-view complementary semantics, thereby achieving superior effectiveness.In summary, our goal is to enable both subnets reasoning the input from different views, and the aggregated predictions can fully exploit cross-view semantics. To this end, we propose a simple yet effective cross-view mutual learning method, which is consisted of two main parts: (1) Conflict-based feature learning (CFL): First, we impose a strong feature-level constraint to maximize the discrepancy between the feature extracted by two subnets, allowing two decoders to produce complementary predictions. Based on this, the cross supervision is conducted to learn more precise predictions for unlabeled data. (2) Cross-view mutual learning (CML): Considering that direct cross supervision may compel two subnets to discard complementary information as Fig. 2 (b), we propose to apply the CutMix operation to modify the inputs and supervisory signals. Meanwhile, we conduct a heterogeneous consistency objective, which combines two subnets' pseudo-labels, to supervise unlabeled data. The final results are obtained through aggregating predictions from the two subnets. The proposed CML method also requires implementing the consistency objective, but we encourage the exploration of complementary semantics across different views to learn unlabeled data more effectively. The contributions of our work can be summarized as below:• We propose a novel conflict-based feature learning (CFL) using a co-training framework, which maximizes the discrepancy between the features extracted from the two subnets, thereby enabling them to be decoded as complementary model predictions for the same input.• We further propose a cross-view mutual learning (CML) method based on the CFL, which only requires modifying the inputs and supervisory signals for exploiting cross-view complementarity and consistency.• The proposed CML method does not change the original network structure, and thus can be simply integrated into different segmentation models. Extensive experiments are conducted on three public datasets, demonstrating the superior segmentation performance. 
related work: 
 Semi-Supervised Medical Image Segmentation. Many advances have been made in semi-supervised medical image segmentation over the past decade. In this field, pseudo-label supervision [2, 9, 23] and consistency regularization [13, 30, 34] are two widely-used strategies. Meanwhile, many works effectively utilize unlabeled data in diverse ways. Specifically, Yu et al. [38] utilize uncertainty information to guide student network learning highly confident targets from teacher network. Wang et al. [23] use neighbor matching to generate reliable pseudo-labels, fully exploring the embedding similarity with neighboring labeled data. Furthermore, some works promote consistency in the feature space to effectively leverage unlabeled data. In specific, Wu et al. [30] explore the pixel-level smoothness, and then encourage the inter-class separation at the feature space. Basak et al. [2] introduce a patch-based contrastive learning framework, which can impose intra-class compactness and inter-class separability. Zhang et al. [42] propose a self-aware and cross-sample prototypical learning, which employs featureprototype similarity to enrich the diversity of prediction.Pseudo-Label Learning. In the realm of semi-supervised medical image segmentation, pseudo-labeling methods have been widely explored. These approaches [22, 27, 36] incorporate pseudo-labels of unlabeled data into the training process, augmenting the available supervisory information. Among them, a key point is how to reduce the impact of label noise [14] . Specifically, Yao et al. [36] propose a confidence-aware cross supervision network, which calculates the KL divergence between original and transformed image predictions, utilizing it as variance in their confidence-aware cross loss. Wang et al. [24] introduce a trust module to reassess pseudo-labels from model outputs, employing a threshold to select high-confidence values. Moreover, apart from integrating confidence-aware modules, some approaches focus on enhancing the quality of pseudo-labels. Li et al. [9] propose a self-ensembling strategy to construct reliable predictions through exponential moving average, mitigating noise and unstable pseudo-labels.Mutual Learning. The co-training framework is a promising method for semi-supervised medical image segmentation, typically involving two distinct subnets with similar structures but not sharing parameters [6, 25] . Particularly, they often conduct the mutual learning paradigm to transfer useful semantics from one subnet to another via pseudo-labeling, which allows them to provide different and complementary information for each other [26, 29] . Specifically, Chen et al. [5] propose a novel cross pseudo supervision, which trains two subnets with different initialization, and pseudo labels output from one perturbed segmentation network is used to supervise the other segmentation network. Zhang et al. [39] propose a semi-supervised contrastive mutual learning segmentation framework, which effectively leverages the cross-modal information and prediction consistency between different modalities to conduct contrastive mutual learning. Moreover, Wang et al. [25] further introduce a contrastive difference review module to mitigate the impact of network bias correction. 
method: 
 In semi-supervised scenario, the training set D consists of a small labeled set D 𝑙 = {(𝑋 𝑙 𝑘 , 𝑌 𝑙 𝑘 )} 𝑁 𝑘=1 , and a large unlabeled setD 𝑢 = {𝑋 𝑢 𝑘 } 𝑀+𝑁 𝑘=𝑁 +1, where 𝑁 ≪ 𝑀. Specifically, 𝑋 𝑘 ∈ R 𝐻 ×𝑊 ×𝐿 represents the 3D medical volume and 𝑌 𝑘 ∈ {0, 1, . . . , 𝐶 -1} 𝐻 ×𝑊 ×𝐿 is ground truth with 𝐶 categories. Our goal is to train a medical image segmentation model by fully leveraging both a labeled set D 𝑙 and a much larger unlabeled set D 𝑢 .The overall pipeline of our proposed method is depicted in Fig. 3 , with a co-training framework like MC-Net [31] to conduct two parallel networks to predict the supervisory signals of its counterpart. First, we conduct a novel conflict-based feature learning (CFL) paradigm that encourages the two subnets to reason the input from two different views, which will be introduced in Sec. 3.1. Second, we propose a simple yet effective cross-view mutual learning (CML) method that does not need the elaborate pseudo-label correction strategies, and instead constructs heterogeneous consistency objective to explore cross-view consistency and complementarity, as further discussed in Sec. 3.2. In the end, we detail the overall learning objective in Sec. 3.3.In this section, we introduce a novel conflict-based feature learning (CFL) paradigm with a co-training framework. As shown in Fig. 3 , we employ two subnets that share a similar architecture but have non-shared parameters to conduct mutual learning. Here, each subnet is divided into an encoder 𝐸 𝑖 and a decoder 𝐷 𝑖 , where 𝑖 ∈ {0, 1}, denoting the first or second subnet, respectively. Considering that the objective of CFL is to enable the two subnets to capture different features from the same input, so the outputs from different encoder networks should be distinct. In light of this, we denote the outputs from encoder networks as 𝑓 𝛼 𝑖 = 𝐸 𝑖 (𝑋 𝛼 ), where 𝛼 ∈ {𝑙, 𝑢} represents for the labeled data and the unlabeled data, respectively. Then, we impose a strong constraint L 𝛼 𝑑𝑖𝑠 in the feature space, which minimizes the cosine similarity between the latent features 𝑓 𝛼 𝑖 in a conflict learning manner. In this way, the two subnets are able to encode the same input from different views, thereby capturing different semantic information. The feature-level discrepancy loss L 𝛼 𝑑𝑖𝑠 can be formulated as:EQUATIONwhere f 𝛼 (1-𝑖 ) is a copy of the feature 𝑓 𝛼 (1-𝑖 ) without gradients, avoiding interference with parameter updates in the other subnet. The coefficient 1 is to ensure that the value of L 𝛼 𝑑𝑖𝑠 is always non-negative. Note that we need to conduct conflict-based feature learning on both of the two subnets, so the overall feature-level discrepancy loss can be expressed as L 𝛼 𝑑𝑖𝑠 = L 𝛼 𝑑𝑖𝑠,0 + L 𝛼 𝑑𝑖𝑠,1 . Meanwhile, we conduct the supervision objective to enable both subnets making precise predictions. For the labeled images, we use the ground truths to supervise the model training. Specifically, the supervision objective L 𝑙 𝑠𝑢𝑝 is a linear combination of Dice loss and Cross-entropy loss, which can be formulated as:EQUATIONwhere 𝑃 𝑙 𝑖 = 𝐷 𝑖 • 𝐸 𝑖 (𝑋 𝑙 ) represents the probability outputs from the 𝑖-th subnet. Particularly, the ground truths are used to supervise two subnets, and thus the supervision objective for labeled images can be defined as L 𝑙 𝑠𝑢𝑝 = L 𝑙 𝑠𝑢𝑝,0 + L 𝑙 𝑠𝑢𝑝,1 . For unlabeled images, we implement a mutual learning paradigm to guide each subnet in making reliable predictions, using pseudolabels generated by the other subnet. Specifically, we first compute the probability outputs for unlabeled images: 𝑃 𝑢 𝑖 = 𝐷 𝑖 • 𝐸 𝑖 (𝑋 𝑢 ), where 𝑖 ∈ {0, 1} represents two different subnets. Then, the pseudolabels are generated by 𝑎𝑟𝑔𝑚𝑎𝑥 operation, i.e., Ỹ𝑢 𝑖 = 𝑎𝑟𝑔𝑚𝑎𝑥 (𝑃 𝑢 𝑖 ). Following pairwise pseudo-labels, we supervise one subnet's probability output with the other subnet's predictions. Then, the consistency objective for unlabeled images is:EQUATIONIn conclusion, the feature-level discrepancy loss L 𝑑𝑖𝑠 allows the two subnets to encode different semantics from the same inputs. The supervision loss L 𝑠𝑢𝑝 transfers useful knowledge from one subnet to another through pseudo-labeling, and thus makes more precise predictions for unlabeled data. Overall, the mutual learning objective based on CFL method is:EQUATIONwhere L 𝑑𝑖𝑠 = L 𝑢 𝑑𝑖𝑠 + L 𝑙 𝑑𝑖𝑠 is the feature-level discrepancy objective for both labeled and unlabeled images. 𝜆 1 and 𝜆 2 are the trade-off coefficients.Recall that the proposed CFL method in Sec. 3.1 aims to reason the same inputs from different views, but the previous mutual learning methods, e.g., Eq. ( 3), conduct cross supervision to align the predictions of the two subnets. There are two issues: (1) Aligning the predictions of the two subnets might weaken the effectiveness of the CFL method. In other words, the two subnets might degrade to the extent where they reason the input from the same view. (2) The cross supervision paradigm would make two subnets ignoring meaningful semantics from their own view. To this end, we propose a novel cross-view mutual learning (CML) method to maximize the effectiveness of CFL. Note that the proposed CML method does not require changing the original network structure and the learning objective in Eq. ( 4). Instead, it only requires a simple yet effective modification of the input images and supervisory signals to achieve superior semi-supervised segmentation effectiveness.Specifically, we first apply the CutMix operation to modify the input images. There are two advantages: (1) CutMix is a strong data augmentation that helps the two subnets further learning extra meaningful semantics, and (2) CutMix allows us to generate heterogeneous supervisory signals, which splice pseudo-labels from two subnets, to maintain own perspectives of each subnet while fully exploiting complementary semantics from the other. For labeled images, we randomly select two labeled images (𝑋 𝑙 𝑢 , 𝑋 𝑙 𝑣 ) along with their corresponding ground truths (𝑌 𝑙 𝑢 , 𝑌 𝑙 𝑣 ). Then, we paste a random crop from X 𝑙 𝑣 into labeled image X 𝑙 𝑢 . The mixed image-label pairs for labeled data can be expressed as:EQUATIONEQUATIONwhere 1 ∈ {1} 𝐻 ×𝑊 ×𝐿 , and M ∈ {0, 1} 𝐻 ×𝑊 ×𝐿 denotes a zerocentered mask, where the size of the zero-value region is 𝛽𝐻 × 𝛽𝑊 × 𝛽𝐿, with 𝛽 ∈ (0, 1). The symbol ⊙ represents element-wise multiplication.Intuitively, we have no need to adjust the ground truths to supervise labeled images, so the mixed label 𝑌 𝑙 𝑚𝑖𝑥 is conducted to directly supervise the model training. The supervision objective for labeled images can be redefined as:EQUATIONwhere 𝑃 𝑙 𝑚𝑖𝑥,𝑖 = 𝐷 𝑖 • 𝐸 𝑖 (𝑋 𝑙 𝑚𝑖𝑥 ) represents the probability outputs from the 𝑖-th network. 𝑖 ∈ {0, 1} denotes two different subnets.Considering that the CFL method in Eq. ( 1) makes the two subnets reasoning the same inputs from different views, the outputs are thus complementary. In light of this, we aim to modify the supervisory signals for unlabeled images, which contain both pseudo-labels from the own perspective and complementary predictions from the other subnet. In specific, we apply the similar CutMix operation in Eq. ( 5) for unlabeled inputs. Differently, we implement heterogeneous mutual learning to make the two subnets acquiring useful semantics from unlabeled images. Given two unlabeled images (𝑋 𝑢 𝑝 , 𝑋 𝑢 𝑞 ), the mixed image-label pairs for unlabeled data can be expressed as:EQUATIONEQUATIONwhere Ŷ𝑢 𝑝,(1-𝑖 ) and Ŷ𝑢 𝑞,𝑖 are pseudo-labels for the unlabeled images 𝑋 𝑢 𝑝 and 𝑋 𝑢 𝑞 , obtained from the (1 -𝑖)-th subnet and the 𝑖-th subnet, respectively. Taking Ŷ𝑢 𝑞,𝑖 as example, we first input 𝑋 𝑢 𝑞 to the 𝑖-th subnet for predicting its probability output: 𝑃 𝑢 𝑞,𝑖 = 𝐷 𝑖 • 𝐸 𝑖 (𝑋 𝑢 𝑞 ). Then, we conduct 𝑎𝑟𝑔𝑚𝑎𝑥 operation: Ỹ𝑢 𝑞,𝑖 = 𝑎𝑟𝑔𝑚𝑎𝑥 (𝑃 𝑢 𝑞,𝑖 ), and employ non-maximum suppression (NMS) [4] on Ỹ𝑢 𝑞,𝑖 to get the final pseudo-labels Ŷ𝑢 𝑞,𝑖 . Note that we conduct the same operation for unlabeled image 𝑋 𝑢 𝑝 to generate the pseudo-labels Ŷ𝑢 𝑝,(1-𝑖 ) . Following pairwise heterogeneous pseudo-labels in Eq. ( 9), we implement the consistency objective to guide each subnet in capturing useful semantics from unlabeled images. Since we employ a mixed and different supervisory signal, the two subnets are capable of fully exploiting the cross-view complementary information from the CFL method in Sec. 3.1, while maintaining that of their own perspectives. Overall, the consistency objective for unlabeled images can be redefined as:L 𝑢 𝑠𝑢𝑝 = L 𝑢 𝑠𝑢𝑝,0 (𝑃 𝑢 𝑚𝑖𝑥,0 , 𝑌 𝑢 𝑚𝑖𝑥,0 ) + L 𝑢 𝑠𝑢𝑝,1 (𝑃 𝑢 𝑚𝑖𝑥,1 , 𝑌 𝑢 𝑚𝑖𝑥,1), (10) where 𝑃 𝑢 𝑚𝑖𝑥,𝑖 = 𝐷 𝑖 • 𝐸 𝑖 (𝑋 𝑢 𝑚𝑖𝑥 ) represents the probability outputs from the 𝑖-th network.Since the feature-level discrepancy loss L 𝑑𝑖𝑠 is an unsupervised loss to maximize discrepancy in feature space, it is not related to the supervisory signals. Hence, we do not need to change the form of L 𝑑𝑖𝑠 . Note that 𝑓 𝛼 𝑖 = 𝐸 𝑖 (𝑋 𝛼 𝑚𝑖𝑥 ) in the CML method, where 𝛼 ∈ {𝑙, 𝑢} represents for the labeled data and the unlabeled data, respectively. Besides, the proposed CML method only changes the input images and supervisory signals, and thus the overall learning objective is in form equal to Eq. ( 4), using the redefined supervision objective L 𝑙 𝑠𝑢𝑝 and L 𝑢 𝑠𝑢𝑝 in Eqs. (7, 10) .Inspired by previous work [1] , we apply the CutMix augmentation on labeled data for training a supervised model during pre-training. Meanwhile, we conduct the feature-level discrepancy objective to encourage the two subnets capturing different semantics from the same inputs. Then, the entire learning objective during pre-training can be formulated as:EQUATIONwhere 𝜆 2 is a trade-off coefficient in Eq. ( 4).During self-training, we initialize two subnets using the pretrained weights from Eq. (11) . Then, we jointly learn the supervision objectives L 𝑙 𝑠𝑢𝑝 and L 𝑢 𝑠𝑢𝑝 , and feature-level discrepancy objective L 𝑑𝑖𝑠 , the total loss is in form equal to Eq. ( 4). Note that we redefine L 𝑙 𝑠𝑢𝑝 and L 𝑢 𝑠𝑢𝑝 in Sec. 3.2 to fully exploit cross-view semantics. In the testing stage, given a test image 𝑋 𝑡𝑒𝑠𝑡 , we first compute the probability maps from the two subnets: 𝑃 𝑡𝑒𝑠𝑡,𝑖 = 𝐷 𝑖 • 𝐸 𝑖 (𝑋 𝑡𝑒𝑠𝑡 ). And the final prediction maps can be obtained in an equal-sum manner by: 𝑌 𝑡𝑒𝑠𝑡 = 𝑎𝑟𝑔𝑚𝑎𝑥 ((𝑃 𝑡𝑒𝑠𝑡,0 + 𝑃 𝑡𝑒𝑠𝑡,1 )/2). [10] 81.60 16.16 3.58 DTC [13] 81.25 14.90 3.99 URPC [15] 82.48 14.65 3.65 MC-Net [31] 83.59 14.07 2.70 SS-Net [30] 86 LA dataset. The LA dataset [33] is the benchmark dataset for the 2018 Atrial Segmentation Challenge, which comprises 100 gadolinium-enhanced MR imaging scans (GE-MRIs) and corresponding ground truths, with an isotropic resolution of 0.625 × 0.625 × 0.625 mm 3 . ACDC dataset. The ACDC dataset [3] is the benchmark dataset for the Automated Cardiac Diagnosis Challenge, which contains 100 short axis cine-MRIs, and expert annotations are provided for three classes: left and right ventricle (LV, RV), and myocardium (MYO). BraTS2019 dataset. The BraTS2019 dataset [16] is a whole brain tumor segmentation dataset, which contains 335 scans with four modalities (FLAIR, T1, T1ce, and T2), and each sequence with an isotropic resolution of 1 mm 3 . Metrics. Three widely-used metrics are used to evaluate the model performance, including Dice score (%), 95% Hausdorff Distance (95HD) in voxel, and Average Surface Distance (ASD) in voxel. 
implementation details: 
 The proposed CML method is implemented in PyTorch and executed on an NVIDIA GeForce RTX 3090 GPU. Following previous works [13, 15, 30] , we apply random cropping, flipping, and rotation to augment the training dataset. All the segmentation tasks are optimized using an SGD optimizer with an initial learning rate of 0.01. For LA and BraTS2019 datasets, we set 𝜆 1 = 1 and 𝜆 2 = 0.2 to implement cross-view mutual learning. Inspired by [1] , the zerocentered mask M is used for CutMix operation, where we set 𝛽 = 2 3 . Additionally, we follow [15, 30] to randomly crop 112 × 112 × 80 patches for the LA dataset, and 96 × 96 × 96 for the BraTS2019 dataset. The 3D V-Net is chosen as the backbone, and the batch size is 8, including four labeled patches and four unlabeled patches. Particularly, the pre-training and self-training iterations on LA dataset [10] 57.77 20.05 6.06 DTC [13] 56.90 23.36 7.39 URPC [15] 55.87 13.60 3.74 MC-Net [31] 62.85 7.62 2.33 SS-Net [30] 65.82 6.67 2.28 MCF [25] 82.37 are set as 2k and 15k, and on BraTS2019 dataset are set as 10k and 30k. For ACDC dataset, we set 𝜆 1 = 0.5, 𝜆 2 = 0.1 for CML, and 𝛽 = 1 2 for CutMix operation. Following [30] , we use a 2D U-Net as the backbone. The input patch size is set as 256 × 256 (2D slices), and the batch size is 24. The iterations of pre-training and self-training are set as 10k and 30k, respectively.In this section, we conduct experiments with varying ratios of labeled data, and compare the proposed CML with other SSMIS methods. Specifically, for LA and ACDC datasets, we evaluate our method against seven SoTA methods, including UA-MT [38] , SASSNet [10] , DTC [13] , URPC [15] , MC-Net [31] , SS-Net [30] , and MCF [25] . To ensure a fair comparison, we report the performance of all competitors under identical experimental settings in SS-Net [30] across different labeled ratios (i.e., 5% and 10%).Tabs. 1 and 2 show that the proposed CML method outperforms previous SSMIS methods, and achieves results very close to the fully-supervised counterpart. There are three observations: (1) Most previous methods, like UA-MT, effectively exploit unlabeled data with a Mean Teacher framework. However, the model performance is limited by EMA, wasting the potential of multi-subnet architecture. The proposed CML method significantly improves Dice by 5.37% and 2.57% compared to UA-MT on LA dataset with 5% and 10% labeled data. (2) MC-Net and MCF exploit unlabeled data with a co-training framework, where they conduct the mutual learning to make two subnets producing consistent predictions, which are close to our approach. However, they lack an explicit constraint to explore the complementarity of model predictions. In contrast, CML introduces a new conflict-based feature learning, which encourages the two subnets reasoning the same input from different views. Notably, the proposed CML improves 95HD by 4.08 and 2.47, compared to MC-Net and MCF on ACDC dataset with 10% labeled data. (3) The CML method can fully exploit cross-view semantics to make more precise predictions for unlabeled data. As presented in Fig. 4 , our method excels in accurately segmenting intricate details of the target organ, particularly in cases where edges are prone to misidentification, as highlighted by green circles.For BraTS2019 dataset, we compare the proposed CML method with MT [19] , UA-MT [38] , ICT [20] , EM [21] , URPC [15] , and MCF [25] in Tab. 3. All methods strictly follow the experimental setting in URPC [15] using 10% and 20% labeled data. Similarly, Our CML outperforms the previous SSMIS methods on the BraTS2019 dataset. In particular, URPC employs uncertainty estimation to implement multi-layer consistency learning. However, they ignore that the complementarity of model predictions is equally crucial, which would limit the SSMIS performance. The proposed CML achieves significant improvement in Dice, 95HD, and ASD terms, surpassing the URPC by 1.1%, 1.93, and 0.8 under 10% labeled data.To understand the effectiveness of loss components in CML, we remove each loss individually to observe the corresponding changes in performance. Specifically, (A) L 𝑙 𝑠𝑢𝑝 is a supervision loss to learn labeled data, preventing model collapse. (B) L 𝑑𝑖𝑠 is a feature-level discrepancy loss that encourages the two subnets to reason the same input from different views. (C) L 𝑢 𝑠𝑢𝑝 is a heterogeneous consistency loss to learn unlabeled data, fully exploring the complementarity of model predictions. As shown in Tab. 4, L 𝑢 𝑠𝑢𝑝 plays a crucial role in the proposed CML method, effectively learning useful semantics from unlabeled data through cross-view mutual learning. Under 10% labeled data, the model with L 𝑢 𝑠𝑢𝑝 improves 5.42% and 8.34% in Dice term compared to single L 𝑙 𝑠𝑢𝑝 on the ACDC and LA datasets, respectively. Moreover, L 𝑑𝑖𝑠 ensures that different encoders output distinct features, and thus capture cross-view complementary semantics. Specifically, L 𝑑𝑖𝑠 further improves 2.2% and 1.67% in Dice term on the ACDC and LA datasets, respectively. Notably, the complete CML further achieves superior edge segmentation performance, improving the model without L 𝑑𝑖𝑠 by 3.77 in 95HD term, 0.64 in ASD term on LA dataset, and 4.93 in 95HD term, 1.19 in ASD term on ACDC dataset, respectively.In this section, we conduct the model parameter sensitivity analysis for 𝜆 1 , 𝜆 2 , and 𝛽. In detail, 𝜆 1 and 𝜆 2 are the balance weights used to control the influence of different loss compositions. We set the hyperparameters 𝜆 1 and 𝜆 2 in the range of {0.1, 0.2, 0.5, 1}. As shown in Figs. 5(a ) and 5(b), the proposed CML is not sensitive when 𝜆 1 changes from 0.2 to 1, and 𝜆 2 changes from 0.1 to 0.5. Empirically, we set 𝜆 1 = 1 and 𝜆 2 = 0.2 for LA and BraTS2019 datasets, and 𝜆 1 = 0.5 and 𝜆 2 = 0.1 for ACDC dataset.In addition, we further explore the optimal setting for zerocentered mask M, i.e., the parameter 𝛽. Specifically, we set the hyperparameters 𝛽 = { 1 3 , 1 2 , 2 3 , 5 6 } to observe how the performance changes. As depicted in Fig. 5 (c), the performance of the model exhibits a clear decline in Dice score when 𝛽 is set as a larger or smaller value. This might be because a larger or smaller zero-centerd mask could hinder the effectiveness of CutMix operation, making it difficult to exploit the heterogeneous consistency objective to transfer useful semantics from one subnet to another.Recalling our proposed method, we first suggest employing cotraining framework to release the potential of multi-subnet architectures, where the cross pseudo-supervision is used to learn from unlabeled data, denoted as CPS. Second, we introduce a new conflictbased feature learning (CFL), which imposes a strong feature-level constraint based on CPS to encourage the two subnets reasoning about the same input from different views. In the end, building upon CFL, we further propose to implement a heterogeneous consistency objective through cross-view mutual learning (CML), fully exploring the complementarity of model predictions. To evaluate the effectiveness of CML, we conduct a comparison experiment among CPS, CFL, and CML methods with a co-training framework. As shown in Fig. 6 , CML achieves superior performance under 10% labeled data. Particularly, for the LA and ACDC datasets, CML with the CutMix operation improves Dice score by 1.8% and 1.3% compared to CFL. This is because the CutMix operation can effectively enrich the sample space for medical image datasets. Meanwhile, the proposed CML aggregates the complementary predictions from two subnets, thereby learning unlabeled data robustly. Overall, both the proposed CFL and CML methods contribute to improving the performance of co-training framework. 
conclusion: 
 In this work, we propose a novel cross-view mutual learning (CML) method for semi-supervised medical image segmentation. Specifically, we first introduce a conflict-based feature learning (CFL) paradigm that imposes a strong feature-level constraint to encourage both subnets encoding cross-view complementary semantics. Building upon CFL, we further employ the CutMix operation to construct a heterogeneous consistency objective, allowing us to fully explore the complementarity of model predictions. Note that the proposed CML method does not alter the origin network, and thus can be simply integrated into different segmentation models. Extensive experiments conducted on three datasets demonstrate the SoTA segmentation performance of CML. 
 
 Appendix:  


Decision: The paper introduces a conflict-based feature learning (CFL) paradigm and a cross-view mutual learning (CML) method, enhancing the performance of semi-supervised medical image segmentation. Reviewers highlighted its strengths in exploring cross-view semantics, effectively using CutMix, and achieving improved performance on public datasets. Concerns about the comparison with CutMix-only paradigms and potential model degradation due to adversarial loss were discussed in the rebuttal. Although the reviewers' discussions on the paper's limitations were somewhat brief, they unanimously leaned towards acceptance, acknowledging the contributions and clear writing. Considering the paper's incremental contributions and improved performance, I decide to follow the reviewers' recommendations.

Reviews: review: paper_summary: paper_summary: This paper proposes a novel Cross-view Mutual Learning (CML) method with a two-branch co-training framework to realize the potential of the multi-subnet architecture. Specifically, CML encourages learning different latent features from two subnets, thereby producing complementary predictions. Meanwhile, CML constructs heterogeneous consistency objective via CutMix operation to explore cross-view semantics.
strength_weakness: strength_weakness: (1) The authors detail the inability of previous works to realize the potential of multi-subnet architectures and conduct comparative experiments. This observation is reasonable and, based on it, the proposed CFL is used to learn cross-view semantics.
(2) The use of CutMix operation is reasonable. The authors employ CutMix to construct heterogeneous supervisory signals, effectively preventing the two subnets from degrading to reasoning from the same perspective.
(3) The proposed CML achieves superior performance on three publicly available datasets, and the authors release their codes and datasets.
limitations: limitations: (1) CutMix is a popular data enhancement technique and is also used in the semi-supervised domain. What are the differences between the proposed CML and previous works?
(2) The proposed conflict-based feature learning (CFL) encourages the model to learn distinct features from different views by introducing adversarial loss. May this adversarial loss interfere with other loss terms, and lead to model degradation problem?
paper_summary: paper_summary: To address the issue that existing Semi-supervised medical image segmentation (SSMIS) often ignore that the complementarity of model predictions is equally crucial, this paper imposes a strong feature-level constraint to make the two subnets producing the complementary predictions. A novel conflict-based feature learning (CFL) method and a cross-view mutual learning (CML) method based on CFL are proposed.
strength_weakness: strength_weakness: 1) The complementarity of model predictions is an important factor for semi-supervised learning. The authors have fully explored cross-view semantic information from both feature-level and image-level dimensions, thus the model can learn unlabeled data effectively.
2) Extensive experimental results demonstrate the superiority of the proposed CML method. The code is also provided.
limitations: limitations: CutMix is a strong data augmentation method that can improve the performance of semi-supervised learning to some extent. Although the authors innovatively use CutMix to construct heterogeneous supervision objective, a comparison with a CutMix-only paradigm is needed to illustrate the effectiveness of CML.
paper_summary: paper_summary: This work addresses the discrepancies among networks and proposes a cross-view mutual learning approach to maximize the effectiveness of conflict-based feature learning. This ensures that the aggregated predictions capture both consistency and complementarity across all views. Extensive experiments demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1.The paper is well-written and is with good organization.
2.Unlike previous works that focus on achieving consistent results from two subnetworks, the authors propose to focus on the complementarity of model predictions, which is an interesting idea.
3.Compared to existing state-of-the-art methods, the proposed CML method achieves excellent performance.
limitations: limitations: 1.	The model has three hyperparameters. Finding the optimal settings for these parameters might require additional tuning.
2.	Regarding the experimental evaluation, please clarify the working principles of the comparison methods to better highlight the contribution of the proposed method.
paper_summary: paper_summary: This paper introduces a novel approach to semi-supervised medical image segmentation (SSMIS) by leveraging a multi-subnet architecture with cross-view mutual learning. Traditional methods in SSMIS focus on consistency between subnets' predictions for unlabeled data but overlook the importance of complementarity in model predictions.

The proposed method consists of two key components: conflict-based feature learning (CFL) and cross-view mutual learning (CML). CFL encourages each subnet to learn distinct features from the same input, which are then decoded into complementary model predictions. This allows both subnets to understand the input from different perspectives. CML maximizes the effectiveness of CFL by implementing a heterogeneous consistency objective, which exploits the complementarity of model predictions across different views.
strength_weakness: strength_weakness: 1. The writing is excellent—clear, concise, and easy to understand.
2. It's reasonable to emphasis the importance of complementarity in model predictions. This approach, which enhances the Conflict-Based Feature Learning (CFL) paradigm, encourages each subnet to interpret the input from unique perspectives, fostering more robust and diverse feature extraction.
limitations: limitations: 1. In figure1, the demonstration of (b) and (c) seem to be similar, it would be better to emphasis the distinction of proposed CML framework. 
2. The introduction to the related work on semi-supervised segmentation could be more comprehensive. These studies represent different branches of semi-supervision for medical image segmentation: 
[1] Peng J, Wang P, Desrosiers C, et al. Self-paced contrastive learning for semi-supervised medical image segmentation with meta-labels[J]. Advances in Neural Information Processing Systems, 2021, 34: 16686-16699.
[2] Zeng D, Wu Y, Hu X, et al. Positional contrastive learning for volumetric medical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24. Springer International Publishing, 2021: 221-230.
[3] Wu Y, Chen J, Yan J, et al. GCL: Gradient-Guided Contrastive Learning for Medical Image Segmentation with Multi-Perspective Meta Labels[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 463-471.