ROAM: Random layer mixup for semi‐supervised learning in medical images

IET Image ProcessingVolume 16, Issue 10 p. 2593-2608 ORIGINAL RESEARCHOpen Access ROAM: Random layer mixup for semi-supervised learning in medical images Tariq Bdair, Corresponding Author Tariq Bdair [email protected] orcid.org/0000-0003-2049-2113 Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, Munich, Germany Correspondence Tariq Bdair, Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, 85748 Munich, Germany. Email: [email protected]Search for more papers by this authorBenedikt Wiestler, Benedikt Wiestler orcid.org/0000-0002-2963-7772 Department of Neuroradiology, Technical University of Munich, Munich, GermanySearch for more papers by this authorNassir Navab, Nassir Navab orcid.org/0000-0002-6032-5611 Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, Munich, Germany Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USASearch for more papers by this authorShadi Albarqouni, Shadi Albarqouni orcid.org/0000-0003-2157-2211 Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, Munich, Germany Helmholtz AI, Helmholtz Center Munich, Neuherberg, Germany Clinic for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, Bonn, GermanySearch for more papers by this author Tariq Bdair, Corresponding Author Tariq Bdair [email protected] orcid.org/0000-0003-2049-2113 Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, Munich, Germany Correspondence Tariq Bdair, Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, 85748 Munich, Germany. Email: [email protected]Search for more papers by this authorBenedikt Wiestler, Benedikt Wiestler orcid.org/0000-0002-2963-7772 Department of Neuroradiology, Technical University of Munich, Munich, GermanySearch for more papers by this authorNassir Navab, Nassir Navab orcid.org/0000-0002-6032-5611 Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, Munich, Germany Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USASearch for more papers by this authorShadi Albarqouni, Shadi Albarqouni orcid.org/0000-0003-2157-2211 Chair for Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, Munich, Germany Helmholtz AI, Helmholtz Center Munich, Neuherberg, Germany Clinic for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, Bonn, GermanySearch for more papers by this author First published: 02 May 2022 https://doi.org/10.1049/ipr2.12511AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Medical image segmentation is one of the major challenges addressed by machine learning methods. However, these methods profoundly depend on a large amount of annotated data, which is expensive and time-consuming. Semi-supervised learning (SSL) approaches this by leveraging an abundant amount of unlabeled data. Recently, MixUp regularizer has been introduced to SSL methods by augmenting the model with new data points through linear interpolation at the input space. While this provides the model with new data, it is limited and may lead to inconsistent soft labels. It is argued that the linear interpolation at different representations provides the network with novel training signals and overcomes the inconsistency of the soft labels. This paper proposes ROAM as an SSL method that explores the manifold and performs linear interpolation on randomly selected layers to generate virtual data that has never been seen before, which encourages the network to be less confident for interpolated points. Hence it avoids overfitting, enhances the generalization, and shows less sensitivity to the domain shift. Extensive experiments are conducted on publicl datasets on whole-brain and lung segmentation. ROAM achieves state-of-the-art results in fully supervised (89.5%) and semi-supervised (87.0%) settings with relative improvements up to 2.40% and 16.50%, respectively. 1 INTRODUCTION Medical imaging is defined as the process of capturing the interior of a body for clinical use, medical intervention, as well as visual monitoring of the function of some organs or tissues [1]. Medical image annotation plays a fundamental role in the medical field [2] since it provides a tool to examine different diseases [3], quantify human organs [4], therapy planning [5], tumor development monitoring [6, 7], diagnostic aid systems [8], and intra-operative assistance [9]. Nevertheless, manual segmentation is a tedious task that requires highly experienced physicians [10] and is subject to intra-/inter-observer variability [11]. That led to a great interest in automated segmentation methods estimated at 70% of international image analysis challenges in the medical domain [12]. Recently, deep learning-based methods have achieved state-of-the-art performance in medical image segmentation [13-15], and shown their applicability to a wide range of datasets without requiring human expert [16]. However, one major drawback of this approach is the necessity for a huge amount of annotated data which is oftentimes not available in medical images. Fortunately, the semi-supervised learning (SSL) framework provides the tool to alleviate this problem by utilizing a huge amount of unlabeled data along with a few annotated ones in intelligent and efficient ways. Thus, SSL methods have proved their benefits to real cases which fit the nature of medical data, where the scarcity of labeled data is the main characteristic. While surveying SSL methods is out of the scope of this paper, we refer the reader to references [17, 18] for a useful introduction. In general, SSL methods can be grouped into four main categories; (i) generative model, (ii) graph-based methods, (iii) entropy minimization, and (iv) consistency regularization. Next, we introduce briefly these methods with the focus on SSL works on medical images. Generative models have been extensively used in the past few years to estimate the density distribution of the data using the concept of adversarial learning [19]. Specifically, two networks were used in the training process, namely the generator and the discriminator networks. The goal of the generator is to produce fake data of parallel high quality to that of the original data, while the goal of the discriminator is to distinguish between the fake and the original data. This idea has been utilized by Zhang et al. [20] for gland image segmentation by encouraging the discriminator to distinguish between the segmentation results of unlabeled and labeled images while encouraging the segmenter (generator) to produce results fooling the discriminator. Nie et al. [21] utilized attention-based approach, based on the confidence map from the confidence network (discriminator), to include the unlabeled data in the adversarial training for pelvic organ segmentation. Chen et al. [22] encouraged the model to learn discriminative features for segmentation from unlabeled images, using an autoencoder trained to synthetic segmentation labels, to segment tumor and white matter hyperintensities in the brain. SCLLD [23] proposed GAN-based architecture consisting of two training phases to detect COVID-19 infection. First, the weights of generator and discriminator networks are initialized using the unlabeled data, then fine-tuned by exploiting the labeled ones. VTGAN [24], however, proposed a semi-supervised GAN-based method to synthesize retinal vascular structure angiograms from fundus images while detecting healthy and abnormal retina. Transformer-based discriminators take the original and generated images then produce feature maps used for disease classification. Major drawbacks of these approaches include the computation overhead and the complexity in the architecture. For instance, reference [23] involves two training stages, while VTGAN [24] consists of four networks; two generators and two transformer-based discriminators. In contrast, our method is easy to implement, consists of one backbone network, and requires no additional training phases. Graph-based methods represent the data, both labeled and unlabeled, in a graph structure, where the nodes represent the data points, the edges represent the connectivity, and the weights represent the distance between the nodes. Graphs can be used to propagate the labels from the labeled data to the unlabeled ones based on the connectivity and similarity. Baur et al. [25] introduced this concept as a regularization term to the main objective function for MS lesion segmentation. The term is based on the Laplacian graphs and attempts to minimize the distance between similar unlabeled and labeled data points in the hidden space. Ganaye et al. [26] took the advantages of the invariant nature of the brain structure to build an adjacency graph of the brain structures acting as a constraint to refine the predicted segmentation of the unlabeled data. Graph convolutional networks (GCNs)-based approaches have been proposed to handle the unstructured format of some medical data. For instance, GKD [27] distilled the knowledge from teacher to a student model. The teacher graph injects the available information into soft pseudo labels. Then, the pseudo labels are used to train a student graph for Autism spectrum disorder or Alzheimer's disease prediction. RA-GCN [28], on the other hand, addressed the imbalanced class distribution in the medical data by representing each class by a graph-based neural network responsible for the weighting of class samples. The whole architecture, then, is trained in an adversarial manner such that the classifier adapts itself with the attention to rare cases. The previous methods have shown their benefits to unstructured data, yet, it suffers the burden of graph construction and weighing steps. Moreover, graph-based approaches are optimized on the unlabeled data, which results in a lack of scalability. In contrast, our method is optimized on the whole input space and relaxes the need for any previous steps. Entropy minimization forces the decision boundary to pass through low-density regions to minimize the entropy of the predictions. One way to achieve this, in SSL setting, is to generate pseudo labels for the unlabeled data using a model trained on the labeled data. Next, the training process is repeated using both labeled and pseudo-labeled data [29]. This approach has been employed by Bai et al. [30] for cardiac image segmentation, where the pseudo labels were additionally fine-tuned using the conditional random field (CRF) method. Close to pseudo labeling is co-training [31] where confident predictions from separate models, trained using different views of the data, are utilized to enhance the training. Xia et al. [32] utilized co-training by enforcing multi-view consistency of the unlabeled data for pancreas and multi-organ segmentation. PLAT [33] exploited an adaptive threshold that avoids the noisy signals and generates more accurate pseudo labels to detect the cells in microscopic and stained histology images. Reference [34] proposed a pseudo labeling approach, namely self-loop uncertainty that exploited self-supervised learning sub-task that solves Jigsaw puzzles to mine the information from the unlabeled data to help in the training. While the FCN-based network is optimized to solve Jigsaw puzzles, it produces different segmentation predictions (corresponding to each stage). Then, these predictions are averaged and used as uncertainty estimation yielded by ensembling multiple models to improve the segmentation accuracy in stained tissue and skin lesion images. Our method is similar to the previous ones in the pseudo labeling step. However, we are different by two folds. First, the aforementioned methods generate pseudo labels for the unlabeled data only. Yet, our method, in addition to that, generates virtual data points and their corresponding pseudo labels from linear interpolation at a random layer of the input data. This process augments the model with novel training signals that have never been seen before, see Section 2.3 for more details. Second, the previous methods utilize different post-processing steps to enhance the quality of the pseudo labels, yet, none of them used a sharpening operation that pushes the pseudo labels into more confident regions, which was adopted by our method, check Section 2.3 and Figure 1 for more details. FIGURE 1Open in figure viewerPowerPoint Illustrative example. (a, b) Input Mixup: Shows the inconsistency of the generated soft label of grey-dot resulted from two different linear interpolations of inputs. (c) Manifold Mixup: The learned hidden states are better organized in local regions leading to the consistency of the soft labels. (d) Sharpening operation (red arrow) pushes the soft label of to a more confident region Consistency regularization methods train the model to predict the same output for different perturbations or augmentations of the input data. Mean-Teacher [35], one of the most successful method of consistency regularization, has been employed by Cui et al. [36] for brain lesion segmentation. They simply introduce a segmentation consistency loss to minimize the discrepancy between the outputs of unlabeled data under different perturbations. A similar approach was utilized by Bortsova et al. [37] for chest x-ray images segmentation. Yu et al. [38] included the uncertainty information to enable the student model to learn from the reliable targets for left atrium segmentation. Li et al. [39] utilized transformation-consistency to enhance the regularization on the pixel level. Interesting results were demonstrated on skin lesion, optic disk, and liver segmentation. UDC-Net [40] forced the so-called dual consistency between the predictions of unlabeled images on one side and the predictions of its transformed version and auxiliary decoders on the other side. Further, the consistency is guided by uncertainty measures and applied for COVID-19 lesion segmentation in the CT scans. UATS [41] applied the consistency between the current prediction of unlabeled images and its ensemble predictions from previous epochs for prostate segmentation. Yet, Wang et al. [42], in addition to the consistency between different augmentations of the unlabeled images, forced consistency between the input images and their adversarial direction to classify breast cancer in ultrasound images and ophthalmic disease in the OCT scans. Except for UDC-Net, all the previous methods applied data augmentation at the input space to force the consistency loss. In contrast, our approach augments the images at the input and the hidden layers. Although UDC-Net proposed the perturbations at the features level, they introduced a sophisticated augmentation process consisting of seven decoders. In contrast, our method handles that by simply utilizing a linear interpolation. Moreover, UDC-Net used seven decoders fixed at one hidden space to create different variations. However, our method overcomes this limitation by randomly selecting the hidden spaces on which the augmentation is performed. Modern regularization methods such as Input MixUp [43], and Manifold Mixup [44] have been recently introduced to avoid overfitting by encouraging the model to be less confident for interpolated data points at the input space or the latent space, respectively. Both methods have been successfully employed for fully supervised segmentation frameworks; for example, cardiac image segmentation [45], brain tumor segmentation [46], knee segmentation [47], and prostate cancer segmentation [48]. While the previous works have shown the effectiveness of MixUp over standard data augmentation methods in medical images, they depend heavily on fully labeled datasets, which usually are expensive and not available. Yet, this paper addresses the scarcity of the labeled data by proposing a SSL approach. Recently, MixMatch [49], that inspired our work, introduced Input MixUp to the SSL paradigm achieving SOTA results in image classification. MixMatch augments the model with interpolated data between labeled and unlabeled images at the input space. While this approach is interesting and indeed provides the model with diverse data points, it is rather limited, and suffers from inconsistent soft labels for the interpolated data points. We argue that performing the mixup operation at randomly selected input and hidden representations of the labeled and the unlabeled data provides the network with novel representations and additional training signals that suit the complexity of medical image segmentation tasks. Moreover, it provides stable soft labels of the augmented samples. Our method takes the advantages of both MixMatch and Manifold Mixup to boost the performance of the model leading to better generalizability. Thus, our contributions can be listed as follows: Proposing RandOm lAyer Mixup (ROAM) that explores the manifold by randomly selecting a subset of input and hidden layers to perform a linear interpolation of labeled and unlabeled data points and generate virtual data that fits the complexity of medical imaging segmentation in both fully and semi-supervised settings. ROAM overcomes the limitations of the previous methods by encouraging the network to be less confident for interpolated data points and reducing overfitting and generalizing well to unseen data. Performing a comprehensive ablation study showing the importance of our design choices. Further, we discuss employing the Manifold Mixup with the presence of skip connections in U-Net-like architectures. Extensive experiments are performed, following the recommendations of Oliver et al. [50], to evaluate our method under the presence of domain shift, class mismatch, and different amounts of un-/labeled data. We empirically show the effectiveness of ROAM by demonstrating a SOTA performance in both supervised and semi-supervised settings in the whole-brain image segmentation and beating the baseline models in COVID-19 infection and lung segmentation. A unified architecture is utilized to implement different SSL methods for the sake of fair comparison. The code is made publicly available for benchmarking. 2 METHODOLOGY SSL paradigm In the SSL, we are given a set of labeled S L = { X L , Y L } $\mathcal {S}_L = \lbrace \mathcal {X}_{L}, \mathcal {Y}_{L} \rbrace$ and unlabeled data S U = { X U } $\mathcal {S}_{U} = \lbrace \mathcal {X}_{U}\rbrace$ , where { X L , X U } = { x 1 , … , x L , x L + 1 , … , x L + U } $\lbrace \mathcal {X}_{L}, \mathcal {X}_{U} \rbrace = \lbrace {\bm x}_1,\ldots , {\bm x}_L, {\bm x}_{L+1},\ldots , {\bm x}_{L+U}\rbrace$ are input images, x ∈ R H × W ${\bm x}\in \mathbb {R}^{H\times W}$ , and Y L = { y 1 , … , y L } $\mathcal {Y}_{L} = \lbrace {\bm y}_1,\ldots ,{\bm y}_L\rbrace$ are the segmentation maps, y ∈ R H × W × C ${\bm y}\in \mathbb {R}^{H\times W \times C}$ , for C organs. Our goal is to build a model F ( x ; Θ ) $\mathcal {F}({\bm x};\Theta )$ that takes input image x ${\bm x}$ and outputs its segmentation map y ̂ $\hat{{\bm y}}$ . To leverage both labeled and unlabeled data in SSL paradigm, the objective function takes the form L Total = L Supervised + β L Unsupervised , \begin{equation} \mathcal {L}_{\text{Total}} = \mathcal {L}_{\text{Supervised}}+ \beta \mathcal {L}_{\text{Unsupervised}}, \end{equation} (1)where L Supervised $\mathcal {L}_{\text{Supervised}}$ denotes the supervised loss and trained using labeled data S L $\mathcal {S}_L$ , L Unsupervised $L_{\text{Unsupervised}}$ denotes the unsupervised loss and trained on the unlabeled data S U $\mathcal {S}_{U}$ , and β is a weighing factor that controls the contribution of the unsupervised loss. The unsupervised loss can have different forms depending on the employed SSL aforementioned approaches. Here, we will focus on the consistency regularization approach, where its goal is to minimize the distance between the feature representations of the input data point x and its perturbed version x ̂ $\hat{x}$ . Formally, L Unsupervised = d ( f θ ( x ) $L_{\text{Unsupervised}}=d(f_{\theta }(x)$ , f θ ( x ̂ ) ) $f_{\theta }(\hat{x}))$ , where d ( · , · ) $d(\cdot ,\cdot )$ is a distance metric. Preliminaries Input Mixup [43] is a simple data augmentation method that generates new data points ( x k , y k ) $(x_{k}, y_{k})$ through a linear interpolation between a pair of training examples ( x i , y i ) $(x_{i}, y_{i})$ and ( x j , y j ) $(x_{j}, y_{j})$ , x k = λ x i + ( 1 − λ ) x j , \begin{align} x_{k} = \lambda x_{i}+ (1-\lambda ) x_{j},\end{align} (2) y k = λ y i + ( 1 − λ ) y j , \begin{align} y_{k} = \lambda y_{i}+ (1-\lambda ) y_{j}, \end{align} (3)where λ ∈ [ 0 , 1 ] $\lambda \in [0, 1]$ . Mixup is considered as a type of data augmentation where the newly generated data points extend the training dataset following the cluster and manifold assumptions [51] that linear interpolations of input examples should lead to linear interpolations of the corresponding labels. One major drawback of this approach is that the interpolations between two samples may lead to inconsistent soft labels at interpolated points. Thus Input Mixup can suffer from underfitting and high loss. This can be better understood by examples. Figure 1 shows an illustrative example, where the red and the blue circles represent two classes. In Figure 1a, the grey-dot is generated by the linear combinations of a blue labeled example (X1) and a red unlabeled example (U2). Since the grey-dot is located in the middle distance between the two classes, based on the mixing factor λ, the generated soft label has an equal probability of blue and red classes (50% each). In contrast, in Figure 1b, the same data point (grey-dot) has been generated from a combination of X2 (red class) and U1 (blue class) with probability of 90% of being blue and 10% of being red, as it is located closer to U1, which leads to the inconsistency of the generated soft labels between the different scenarios. Manifold Mixup [44], on the other hand, overcomes the above limitations by performing the mixup operation at the hidden layers. Thus, training is carried out on the convex combinations of the hidden representations of data samples. The learned representations lead to better organization of the hidden state for each class, where it is more concentrated and organized. As a result, the inconsistency of soft labels at interpolated points can be avoided. This can be shown in Figure 1c, where the generated soft label of grey-dot is consistent, with an equal probability of each class, regardless of the interpolated data points (X1 and U2 or X2 and U1). ROAM The core components of our method are (a) Pseudo labeling: Given a pre-trained model for a few epochs on labeled data, the initial labels for the unlabeled batch were produced, then refined by applying a sharpening operation. (b) ROAM: The labeled batch and the unlabeled batch were concatenated, then passed to the network as normal. Then, a mixup operation is applied at a random layer, where the paired examples are randomly selected. At the same time, a mixup operation is applied to the corresponding labels. Finally, the process is continued from that layer to the output layer. In the following sections, we illustrate our methodology in detail, while the entire framework and the algorithm are shown in Figure 2 and Algorithm 1, respectively. FIGURE 2Open in figure viewerPowerPoint Illustration of ROAM. (a) First, initial labels for the unlabeled batch are produced from a pre-trained model, then, a sharpening step is applied to fine-tune the labels. (b) Second, the labeled and unlabeled batches are fed to the network, and mixed at a random layer. Both models in (a) and (b) are the same, yet we freeze the parameters in step (a) 2.3.1 Pseudo labels First, the unlabeled data along with the labeled set are leveraged using two steps: (i) sharpening the initial predictions for unlabeled data to minimize its entropy following reference [49], and (ii) mixup the labeled and unlabeled data at random layers following reference [44]. The unlabeled data are first fed to the model outputting the initial predictions y ̂ i = F ( x i ; Θ ) ; where x i ∈ X U , \begin{equation} \hat{{\bm y}}_{i} = \mathcal {F}({\bm x}_{i}; \Theta );\hspace{10.0pt}\text{where}\hspace{10.0pt} {\bm x}_{i} \in \mathcal {X}_{U}, \end{equation} (4)before being post-processed by a sharpening operation, parameterized with T, which is highly inspired by the entropy minimization literature [49, 52]. The pseudo label set is then defined as Y ∼ U = { y ∼ i , … , y ∼ U } $ \mathcal {\tilde{Y}}_{U} = \lbrace \tilde{{\bm y}}_{i},\ldots ,\tilde{{\bm y}}_{U}\rbrace$ , where y ∼ i = Sharpening ( y ̂ i , T ) j : = y ̂ i j 1 T / ∑ j = 1 C y ̂ i j 1 T , \begin{equation} \tilde{{\bm y}}_{i} = {\tt Sharpening}(\hat{{\bm y}}_{i}, T)_{j} := \hat{{\bm y}}_{ij} ^{\frac{1}{T}} \Big / \sum _{j = 1}^{C} \hat{{\bm y}}_{ij} ^{\frac{1}{T}}, \end{equation} (5)where y ̂ i $\hat{{\bm y}}_{i}$ is given by Equation (4), j ∈ C $j \in C$ , and C is the total number of classes. Note that as T → 0 $T \rightarrow 0$ , y i $y_{i}$ approaches one-hot encoding. Applying the sharpening operation to the initial labels produces more stable predictions through pushing the labels away from the decision boundaries, specifically, to more confident regions for each class by minimizing its entropy. This effect can be easily seen in Figure 1d where the unlabeled data point U1 is moved closer to the right distribution. 2.3.2 Random layer mixup Given the unlabeled data X U $\mathcal {X}_U$ and its pseudo labels Y ∼ U $\mathcal {\tilde{Y}}_{U}$ , along with the labeled data X L $\mathcal {X}_L$ and its one-hot encoding labels Y L $\mathcal {Y}_{L}$ , the two sets are concatenated as X = { X L , X U } , Y = { Y L , Y ∼ U } $\mathcal {X}= \lbrace \mathcal {X}_{L}, \mathcal {X}_{U}\rbrace , \mathcal {Y}= \lbrace \mathcal {Y}_{L}, \mathcal {\tilde{Y}}_{U}\rbrace$ . To enable running the mixup operation at randomly selected latent space, we first define ( H , Y ) $(\mathcal {H, Y})$ , where H = X , κ = 0 F κ ( X ) , otherwise , \begin{align} \mathcal {H} = {\left\lbrace \def\eqcellsep{&}\begin{array}{llllll}\mathcal {X}, & & & & & \kappa = 0 \\[3pt] \mathcal {F}_{\kappa }(\mathcal {X}), & & & & & \text{otherwise} \end{array} \right.}, \end{align} (6)where F κ ( · ) $\mathcal {F}_{\kappa }(\cdot )$ is the hidden representation of the input data at layer κ. Note that the input data is selected when κ = 0 $\kappa = 0$ . To introduce a noisy interpolated data, a permuted version of the original data is created H ∼ , Y ∼ = Permute ( H , Y ) $\tilde{\mathcal {H}}, \tilde{\mathcal {Y}} = {\tt Permute}(\mathcal {H, Y})$ , and fed to the MixUp ${\tt MixUp}$ operation as H ′ = λ ′ H + ( 1 − λ ′ ) H ∼ , \begin{align} \mathcal {H}^\prime = \lambda ^\prime \mathcal {H} + (1-\lambda ^\prime ) \tilde{\mathcal {H}},\end{align} (7) Y ′ = λ ′ Y + ( 1 − λ ′ ) Y ∼ , \begin{align} \mathcal {Y}^\prime = \lambda ^\prime \mathcal {Y} + (1-\lambda ^\prime ) \tilde{\mathcal {Y}}, \end{align} (8)where Permute ( . ) ${\tt Permute}(.)$ randomly shuffles the data, H ′ $\mathcal {H}^\prime$ and Y ′ $\mathcal {Y}^\prime$ are the interpolated mixed-up data, where the paired examples are selected randomly. To favour the original data over the permuted one, λ ′ $\lambda ^\prime$ is set to max ( λ , 1 − λ ) $\mathrm{max}(\lambda , 1- \lambda )$ , where λ ∈ [ 0 , 1 ] $\lambda \in [0, 1]$ is sampled from a Beta ( α , α ) $\mathrm{Beta}(\alpha ,\alpha )$ distribution with α as a hyper-parameter. Further, to keep the flow of the original data, we run some experiments without the mixup operation, and denoted as κ = Φ $\kappa = \Phi$ . In practice, this can be achieved by setting κ and λ ′ $\lambda ^\prime$ to 0 and 1, respectively. To this end, the mixed-up data H ′ $\mathcal {H}^\prime$ are fed to the model from layer κ along the way to the output layer at which the segmentation maps are predicted P $\mathcal {P}$ . Eventually, P $\mathcal {P}$ is split back into labeled and unlabeled predictions P = { P L , P U } $\mathcal {P} = \lbrace \mathcal {P}_{L}, \mathcal {P}_{U}\rbrace$ , and similarly Y ′ $\mathcal {Y}^\prime$ into Y L ′ $\mathcal {Y}_{L}^\prime$ and Y U ′ $\mathcal {Y}_{U}^\prime$ . ALGORITHM 1. ROAM: Random Layer MixUp for SSL Require: pre-trained model F ( · ; Θ ( 0 ) ) $\mathcal {F}(\cdot ;\Theta ^{(0)})$ , labeled dataset S L $\mathcal {S}_{L}$ , unlabeled dataset