CDEA: Context- and Detail-Enhanced Unsupervised Learning for Domain Adaptive Semantic Segmentation

Authors: shuyuan  wen
bingrui  hu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-06 11:52:22

Abstract: Unsupervised domain adaptation (UDA) aims to adapt a model trained on the source domain (e.g. synthetic data) to the target domain (e.g. real-world data) without requiring further annotations on the target domain.
Most previous UDA methods for semantic segmentation focus on minimizing the domain discrepancy of various levels, e.g., pixels and features, for extracting domain-invariant knowledge.  
However, the primary domain knowledge, such as context and detail correlation, remains underexplored. 
To address this problem, we propose a context- and detail-enhanced unsupervised learning framework, called CDEA, for domain adaptive semantic segmentation that facilitates image detail correlations and contexts semantic consistency.
Firstly, we propose an adaptive masked image consistency module to enhance UDA by learning spatial context relations of the target domain, which enforces the consistency between predictions and masked target images.
Secondly, we propose a detail extraction module to enhance UDA by integrating the learning of spatial information into low-level layers, which fuses the low-level detail features with deep semantic features.
Extensive experiments verify the effectiveness of the proposed method and demonstrate the superiority of our approach over state-of-the-art methods.

Main paper: 
 introduction: 
 Deep neural networks have achieved significant success in recent years across various multimedia tasks [1] , such as cross-modal retrieval, image captioning, etc. However, their training often requires a large amount of annotated data. Providing annotations is particularly labor-intensive for specific tasks, such as semantic segmentation [2] . Besides, pixel-wise labels of the entire image are necessary, and it can take more than one hour per image [3] . However, a network trained on such a source dataset usually performs worse when applied to the target dataset [4] , as neural networks are sensitive to domain gaps. To minimize such a gap, researchers resort to Unsupervised Domain Adaptation (UDA) to transfer the knowledge from labeled source-domain data to the unlabeled target-domain environment.Domain adaptive semantic segmentation has got great attention and various methods have been proposed in the last few years, which can be roughly divided into two categories: adversarial training [5] [6] [7] and unsupervised training [8] [9] [10] . However, there is still a noticeable performance gap compared to supervised training [5] . A common challenge is the confusion of classes with a similar visual appearance on the target domain, such as "wall/fence" or "pedestrian/rider", The reason is that no ground truth supervision is available to learn the slight appearance differences. For example, the segmentation class of the fence in Fig. 1(a ) is segmented as a wall, probably as a result of a similar local look. In addition, we find that the edge of category prediction results is not significant and exists a large noise.To address this challenge, we propose to enhance UDA with spatial context relations as additional clues for robust visual recognition. For instance, the curb in the foreground of Fig. 1 (a) could be a crucial context relation to recognizing the fence despite the ambiguous texture correctly. Hoyer et al. [11] propose a mask patch method by learning spatial information of the target domain, which achieves robust visual recognition. Fan et al. [12] propose a short-term dense concatenate network to learn detailed information, which improves edge extraction capability. Although the used network architectures already can model context relations, previous UDA methods can still not reach the full potential of using context dependencies on the target domain as the unsupervised target losses are not powerful enough to enable effective learning of such information. Therefore, we design a method to explicitly encourage the network to learn comprehensive context relations of the target domain during UDA. In particular, we propose a novel Context-and Detail-Enhanced (CDEA) plug-in for UDA, shown as Fig. 1 (c), which can be applied to various visual recognition tasks. Considering semantic segmentation for illustration, CDEA masks out a random selection of target image patches and trains the network to predict the semantic segmentation result of the entire image, including the masked regions. Therefore, the network has to utilize the context to infer the semantics of the masked-out regions, which can increase the robustness of the network.As there are no ground truth labels for the target domain, we resort to pseudo-labels, generated by an EMA teacher [13] that uses the original, unmasked target images as input. Therefore, the teacher can utilize both context and local knowledge to generate robust pseudo-labels. During the training, different classes of objects are masked out so that the network learns to utilize different context knowledge, which further increases the robustness [11] . After UDA with Context-and Detail-Enhanced, the network can better exploit context knowledge and correctly segment difficult areas that rely on context knowledge, such as the sidewalk in Fig. 1 To the best of our knowledge, CDEA is the first UDA method to fuse masked images and edge features to facilitate learning context and detail relations on the target domain. Due to its universality and simplicity, Context-and Detail-Enhanced can be straightforwardly integrated into various UDA methods across multiple visual recognition tasks, making it highly valuable in practice. CDEA achieves significant and consistent performance improvements for different UDA methods, including adversarial training and unsupervised training on semantic segmentation tasks with domain gaps and network architectures. The main contributions of this paper can be summarized as follows:(1) Different from existing works on semantic segmentation, we focus on mining domain-invariant knowledge from the original domain in an unsupervised manner. We propose a contextand detail-enhanced unsupervised learning framework to harness both context-and detail-wise consistency against different contexts, which is well-aligned with the segmentation task. 2 Related Work 
unsupervised domain adaptation: 
 In recent years, semantic segmentation approaches have been based on deep neural networks, which can be effectively trained in an endto-end manner to perform pixel-wise classification [14] [15] [16] [17] . However, semantic segmentation approach training often requires a large amount of annotated data, which can take more than one hour per image. Unsupervised domain adaptation (UDA) enables a model to transfer scalable knowledge from a label-rich source domain to a label-scarce target domain, which improves the adaptation performance. Several strategies have been proposed to adapt a semantic segmentation network to the target domain, which can be grouped into adversarial training [5] [6] [7] and unsupervised training [8] [9] [10] 18] .Adversarial training methods aim to learn domain-invariant knowledge based on adversarial domain alignment. Wang et al. [19] propose a fine-grained adversarial learning strategy for class-level feature alignment while preserving the internal structure of semantics across domains. Shan et al. [7] propose to conduct an auxiliary adversarial training on the fused multi-level CNN features. However, unstable adversarial training methods usually lead to suboptimal performance. Unsupervised training methods aim to create pseudo labels for the target domain images using the model trained by labeled source domain data, and then the model is re-trained by pseudo labels. He et al. [20] build a dynamic dictionary with a queue and a moving-averaged encoder to build a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. Van et al. [21] adopts a predetermined mid-level prior in a contrastive optimization objective to learn pixel embeddings. Different from the above-mentioned works, we focus on learning the domain knowledge of context and detail in an unsupervised manner. The proposed method is complementary to the existing approach to further boost the result. Several UDA methods propose network components, such as crossdomain attention [7] , or context-aware feature fusion [22] can capture contexts to facilitate learning domain-robust context dependencies. Although these network modules can capture context, the unsupervised loss on the target domain does not provide sufficient supervision to learn all relevant target context and detail relations. For example, the segmentation performs poorly in details and edges as a result of a similar local look, shown in Fig. 1(a) . Research has shown that the domain knowledge of context and detail Has a significant impact on performance in semantic segmentation [23] . Therefore, we focus on learning the domain knowledge of context and detail in an unsupervised manner. Recently, semantic segmentation approaches have been based on deep neural networks. Hoyer et al. [11] propose a mask patch method by learning spatial information of the target domain, which achieves robust visual recognition. However, these mechanisms can not capture all relevant context information and the prediction results perform poorly in details and edges. Fan et al. [12] propose a short-term dense concatenate network to learn detailed information, which improves edge extraction capability. Different from the abovementioned works [11] , The proposed Context-and Detail-Enhanced can learn more extensive multi-scale context information due to the random size of patch masking during training. Moreover, we introduce a detail-enhanced module in an unsupervised manner according to [12] , which can improve the segmentation performance in details and edges. Therefore, the multi-grained unsupervised learning on both context and detail are complementary to each other and can learn the domain-invariant context feature. 
approach: 
 In this section, We first introduce the UDA definition and segmentation losses for semantic segmentation domain adaptation in Sec. 3.1. Then, we introduce an adaptive masked image consistency module, which enforces the consistency between predictions and masked target images in Sec. 3.2. Besides, we introduce the detail-enhanced module, leading to more precise preservation of spatial details in low-level layers in Sec. 3.3. We finally discuss the mechanism of the proposed method in Sec. 3.4. Our proposed method is designed to be applicable to common network architectures and can be combined with existing UDA methods.The UDA neural network can be trained on the source domain synthetic dataX S = {x S k } N S k =1 labeled by Y S = {y S k } N S k =1and the unlabelled target domain real-world data X T = {x T k } N T k=1 with a supervised source loss and adaptation target loss, where N S and N T are the numbers of images in the source and target domain, respectively. Domain adaptive semantic segmentation intends to learn a mapping function that projects the input image x T to the segmentation prediction ỹT in the target domain. Source Domain Losses. The specific source loss depends on the computer vision task. For semantic segmentation, the pixel-wise cross-entropy is typically used.EQUATIONwhere H,W is the height and width of the input image. The label y S belongs to C categories. p i jc is the one-hot vector of label y i jc , and the value p i jc (c) equals to 1 if c == y i jc otherwise 0. We learn the basic source domain knowledge by adopting the segmentation loss on the source domain, which can be formulated as:EQUATIONwhere we utilize the visual backbone and 2-layer Multi-Layer Perceptrons (MLPs) as f θ for segmentation category prediction. Target Domain Loss. However, a model trained on the source domain usually experiences a performance drop when applied to another domain. Considering that there are no labels for the targetdomain data, we generate pseudo labels ỸT = { ỹT k } for the target domain data X T by a teacher network д ϕ during training to learn the knowledge from the target domain, where ỹT k = arдmax(д ϕ (x T k )). Therefore, UDA methods can use unlabeled images from the target domainX T = {x T k } N T k =1to adapt the network, the segmentation loss on the target domain can be formulated as:EQUATIONwhere ỹT k is the pseudo label of x T k . In practice, the teacher network д ϕ is set as the Exponential Moving Average (EMA) of the weights of the student network f θ after each training iteration. Therefore, the unsupervised loss for the target domain L T is added to the optimization problem with a weight λEQUATIONRecently, many network architectures can integrate both local and context information in their features to recognize an object [22, 24] . Although the learning of context relations can be guided by ground truth in supervised learning, without ground supervision is available for the target domain in UDA. However, the existing unsupervised losses are not consequential enough to enable effective learning of context information, such as in Fig. 1 . Therefore, we propose encouraging the learning performance of context relations on the target domain to provide context information for robust recognition of classes with similar local appearances. To improve the learning context relations in the target domain, we introduce a Context Enhance Module (CEM), which can be easily plugged into various existing UDA methods. The domain adaptation process with the CEM is illustrated in Fig. 3 and explained below. The CEM withholds local information by randomly masking out Algorithm 1 CDEA algorithm Input: Source domain data X S with labels Y S , Target domain data X T , segmentation network f θ with detail head H Detail , segmentation teacher network д ϕ , the total iteration number T total . 1: Initialize network parameter θ with ImageNet pre-trained parameters. Initialize teacher network ϕ randomly. 2: for iteration = 1 → T total do 3:x S , y S ∼ U ., x T ∼ V .x T M ← Mask(x T , r, s). Generate masked image from param mask-ratio r and mask-size s.Generate pseudo labels. ỹT ← arдmax(д ϕ (x T )).Compute predictions. ŷS ← arдmax(f θ (x S )).7: Compute detail. D ← arдmax(Laplacian(f θ (x T ))), DM ← arдmax(H Det ail (f θ (x T M ))). 8:Compute Loss for the mini-batch L total .Compute ∇ θ L total by backpropagation.10:Perform stochastic gradient descent.11:Update teacher network ϕ with θ . 12: end for 13: return Segmentation network f θ . patches of the target image. Therefore, a patch mask M is randomly sampled from a uniform distributionEQUATIONwhere v ∼ U(0, 1), s is the mask-size, r is the mask-ratio, m ∈ [0 . . .W /s -1], n ∈ [0 . . . H /s -1] the patch indices, and [•] denotes the Iverson bracket.EQUATIONwhere the target image x T . The masked target image x T M is obtained by element-wise multiplication ⊙ of mask and image, shown as Fig. 3 .The masked target prediction ŷT M can use the unmasked image regions' limited information to learn the context information. Therefore, we use the remaining context information to reconstruct the correct label without access to the entire image, the Context Enhance loss L M is introducedEQUATIONwhere ỹT is a pseudo-label and λ T is quality weight. The pseudolabel ỹT is the prediction of a teacher network д ϕ of the complete target image x T . The pixel-wise cross-entropy H (•) be shown in Eq. ( 1). The existing study found that there are potential faults in pseudolabels, especially at the beginning of the training [11] . Therefore, we design a weight of quality estimate λ T for the Context Enhance loss and utilize the ratio of pixels exceeding a threshold τ of the maximum softmax probability.EQUATIONCDEA: Context-and Detail-Enhanced Unsupervised Learning for Domain Adaptive Semantic Segmentation. MM '24, October 28-November 1, 2024, Melbourne, VIC, AustraliaAlthough existing network architectures can already model detailed information in supervised learning, previous UDA methods can still not reach the full potential of using detailed dependencies on the target domain. As shown in Fig. 1 , the existing unsupervised methods easily neglect the detailed information about boundaries because they do not learn detailed information effectively. Therefore, we propose a Detail Enhance Module (DEM) to manually guide the low-level layers in learning the spatial details.Detail Head: Compared with the high-level semantic features, the low-level features of neural networks can encode more spatial details [25] . Thus, we insert the Detail Head into the low-level backbone layer to generate the detailed feature map, shown in Fig. 2 .We first generate the detailed feature map ground truth from the segmentation ground truth by the Laplacian operator. Then, we use the detailed ground truth as the guide of a detailed feature map to guide the low-level layers in learning the features of spatial details. Finally, the learned detail features are fused with the context features from the deep block of the decoder for segmentation prediction. As shown in Fig. 2 , the Detail Extract introduces the generating process of detail ground truth.We use the Detail Head to produce the detailed map, which guides the low-level layer to encode spatial information, as shown in Fig. 2 . The Detail Head includes a 3 × 3 Conv-BN-ReLU operator followed by a 1 × 1 convolution to get the output detail map. Then, we use a bilinear interpolation to up-sample the feature map to the size of ground truth. In the experiment, the Detail Head effectively enhances the feature representation. Therefore, this detail-information can easily boost the accuracy of the segmentation task without any cost in inference. Detail Enhance Loss: We model the detail prediction as a binary segmentation task. We generate the binary detail ground truth from the semantic segmentation ground truth. Considering the number of detail pixels is much less than the non-detail pixels, detail prediction is a class-imbalance problem, which often uses weighted crossentropy to solve [26] . However, the weighted cross-entropy always leads to coarse results. Dice loss [27] is insensitive to the number of foreground/background pixels, which can measure the overlap between predicted maps and ground truth, thus can alleviate the class imbalance problem [28] .Therefore, we adopt binary cross-entropy H (•) and Dice loss L Dice to jointly optimize detail learning. So for the predicted detail map with height H and width W , the Detail loss L Det ail is formulated as follows:EQUATIONwhere D ∈ R H ×W denotes the pseudo detail ground-truth and DM ∈ R H ×W denotes the masked predicted detail. The binary crossentropy H (•) be shown in Eq. ( 1). The Dice loss L Dice is given as follows:EQUATIONwhere i denotes the i-th pixel and ϵ is a Laplace smoothing item to avoid zero division. In this paper, we set ϵ = 1.Total Loss. The teacher network д ϕ is implemented as an Exponential Moving Average (EMA) teacher. It is a common strategy used in UDA [13] . Its weights are the EMA of the weights of f θ with smoothing factor αEQUATION)where t is a training step. The EMA teacher realizes a temporal ensemble of previous student models f θ , which increases the robustness and temporal stability of pseudo-labels. As the teacher is updated based on the student f θ , it will gradually obtain the context enhanced learning capability from f θ . In contrast to the student f θ , the teacher д ϕ has privileged access to the original image x T (see Eq. 8). Thus, it can utilize the context and the detial information to generate higher-quality pseudo-labels.The overall training objective is the combination of pixel-level cross-entropy loss and the proposed CDEA:EQUATION)We summarize the overall training process of our CDEA framework in Algorithm 1.Both context and detail enhance are derived from different levels of learning, and they work at different effect regions, pixel-wise and patch-wise. However, detail enhance explores the correlation of different edge categories over the whole image, while context enhance imposes regularization on the mask patches from a local semantic perspective. Therefore, the two enhanced methods are complementary and can learn the intra-domain inherent context and detail within the data. 2. What is the advantage of the proposed approach? Previous UDA methods mainly focused on designing the module for feature learning in the target data. Differently, we are motivated by the objectives of UDA semantic segmentation in a manner and thus leverage context and detail correlations in the target data to facilitate key features and knowledge learning. By explicitly regularizing the feature space via CDEA, we enable the model to explore the inherent intra-domain context and detail information in unsupervised learning, pixel-wise and patch-wise, without extra parameters or annotations. Therefore, CDEA can be easily integrated into existing UDA approaches to achieve better results without extra work during inference. 3. Difference from semantic segmentation for UDA. Conventional contrastive learning methods tend to perform contrast in the instance or pixel level alone. We formulate context and detail enhance in a similar format but focus on the local effect regions within the images, which aligns well with the local-focused segmentation task. We show that the proposed context and detail enhance, and regularize the domain adaptation training and guide the model to shed more light on the intra-domain context. The experiment result verifies our innovation that context and detail enhance and improve smooth edges between different categories and output a higher accuracy on small-object categories. , DAFormer [47] + Ours (CDEA), HRDA [23] , HRDA [23] + Ours (CDEA). We deploy the white dash boxes to highlight different prediction parts. 
experiments 4.1 implementation details: 
 We train the network with batch size 2 for 60k iterations with a single NVIDIA Tesla 4*V100 GPU server. We adopt AdamW [51] as the optimizer, a learning rate of 6 × 10-5, a linear learning rate warmup of 1.5k iterations, and a weight decay of 0.01. The input image is resized to 1280 × 720 for GTA and 1280 × 760 for SYNTHIA, with a random crop size of 640 × 640. We adopt various data augmentation during training, such as ClassMix [52] , Gaussian blur and color jitter. We design CDEA based on a DAFormer network [47] . The teacher network is randomly initialized and the student network is pre-trained on ImageNet1k. The exponential moving average parameter α of the teacher network is 0.999 [13] . The hyperparameters of the loss function are chosen empirically λ T = 2, λ M = 1, λ D = 1 and the ratio of pixels exceeding a threshold τ = 0.5. We report our results for 19 classes on GTA→Cityscapes and both 13 and 16 classes on SYNTHIA→Cityscapes.Datasets. In this paper, we use three commonly used datasets in UDA. GTA5 is a synthetic dataset containing 24,966 high-resolution images collected from game video, and the corresponding groundtruth segmentation map can be generated by computer graphics.SYNTHIA is also a synthetic dataset, which contains 9,400 images. It shares 16 common classes with Cityscapes dataset. Cityscapes is a real-world dataset collected for autonomous driving scenarios from 50 cities around the world. It contains 2,975 and 500 images for training and validation, respectively. Evaluation Metrics. For segmentation evaluation, we adopt the mean of class-wise intersection over union (mIoU) as the evaluation metrics.Reproducibility. The code is based on Pytorch and MMSegmentation. We will make our code open-source for reproducing all results.We compare CDEA with several competitive UDA methods on GTA → Cityscapes and SYNTHIA → Cityscapes, respectively. The quantitative comparisons are shown in Tab. 1 and Tab. 2, respectively. We highlight the best accuracy in every column in bold. Meanwhile, we visualize the visual result between the proposed method and the other two state-of-the-art Transformer methods [23, 47] in Fig. 3 . GTA → Cityscapes. We visualize all experiment results on GTA → Cityscapes in Tab. 1. It could be observed that Transformer methods significantly surpass the CNN methods by a large margin since DAFormer [47] . The results show that our CDEA performed a remarkable improvement over the existing SOTA transformer models DAFormer [47] and HRDA [23] . Meanwhile, CDEA achieves 72.5 mIoU, which outperforms DAFormer [47] by a considerable margin of +4.2 mIoU. In addition, we improve +2.5 mIoU and achieve the SOTA performance of 76.3 mIoU, verifying the effectiveness of the CDEA that introduces a unified and multi-grained unsupervised learning algorithm in UDA task, when applying CDEA to HRDA [23] Moreover, CDEA achieves leading IoU of almost all classes on GTA → Cityscapes, including several small-object categories such as "fence", "pole", "training light" and "training sign". SYNTHIA → Cityscapes. As shown in Tab. 2, CDEA also achieves significant mIoU and mIoU* (13 most common categories) performance on SYNTHIA → Cityscapes, increasing +1.9 and +2.6 mIoU compared with DAFormer [47] and HRDA [23] , respectively. It is noticeable that our CDEA remains competitive in segmenting each individual class including small-scale objectives and yields the best IoU score in almost all categories. The IoU performance of CDEA verifies our innovation that the exploration of the inherent structures of intra-domain images indeed helps category recognition, especially for challenging small objectives. Qualitative Results. As shown in Fig. 3 , we show the segmentation prediction results of CDEA and the comparison with existing SOTA methods DAFormer [47] , HRDA [23] , and the original image with ground truth on both GTA → Cityscapes and SYNTHIA → Cityscapes benchmarks. The results show that CDEA can segment the detail of edge and minor categories, such as "fence", "wall", "traffic sign" and "traffic light", which are highlighted by white bounding boxes. It is also noticeable that CDEA predicts smoother edges between different categories, "traffic sign" and "person" in the fourth row of Fig. 3 . The reason is that the proposed detail-enhanced MM '24, October 28-November 1, 2024, Melbourne, VIC, Australia Shuyuan Wen, Bingrui Hu and Wenchao Li 模图像的多粒度目标检测 8In this section, we introduce the ablation experiments to validate the effectiveness of each component in our method. We verify the effectiveness of the two proposed components, Context Guidance and Detail Guidance in the proposed CDEA and investigate how set the parameter of two Guidance contributes to the final performance on GTA → Cityscapes. For a fair comparison, we apply the same experimental environment and data. Effectiveness of Context Guidance. For context guidance, the ratio and size of the mask also affect performance and training difficulty. We evaluated the performance of both fixed and random types. As shown in Tab. 3, we could observe: The best performance is 72.5 mIoU when the mask size range is 32∼128 and the mask ratio range is 0.2∼0.7 by random type. The best performance of fixed type is 71.3 mIoU when the mask size is 128×128 and the mask ratio is 0.5 by fixed type. The experimental results show that the random type is obviously better than the fixed method, indicating that the random type can improve context awareness and accuracy by random mask.Effectiveness of Detail Guidance. For detailed guidance, The different scales of Up/DownSample also affect the accuracy and performance [12] . As shown in Tab. 4, we gradually utilize and combine the different scales of Up/DownSample. We observe that larger patch generally obtain better performance since it contains more diverse contexts and is close to the test size. The best performance is 72.5 mIoU when the mask size range is 32∼128 and the mask ratio range is 0.2∼0.  
conclusion: 
 In this paper, we presented Context-and Detail-Enhanced UDA method (CDEA) to improve the learning of target domain context relations and detail feature. We target to learn a feature space that enables discriminative context features and the robust feature learning of the detail feature against variant contexts. CDEA can promote the model to extract the inherent contextual and detail feature, which is domain invariant. In comprehensive experiments, we have shown that CDEA achieves significant performance improvements in all of these UDA tasks. For instance, CDEA respectively improves the SOTA performance by +2.5 and +2.6 on GTA→Cityscapes and SYNTHIA → Cityscapes. Since the simplicity of CDEA, it can be combined with other existing methods to further facilitate the intradomain knowledge learning. In addition, We hope that CDEA can be used as part of future UDA methods to narrow the gap between UDA learning. 
: 
 ACM Reference Format: Shuyuan Wen, Bingrui Hu, and Wenchao Li. 2024. CDEA: Context-and Detail-Enhanced Unsupervised Learning for Domain Adaptive Semantic Segmentation. In Proceedings of the 32nd ACM International Conference 
 
 Appendix:  


Decision: All reviewers give positive suggestions. 

Using a spatial context relation to improve the accuracy is interesting.

The paper also presents good ablation studies and reports competitive experimental results to support the claims.

Reviews: review: paper_summary: paper_summary: This paper studies Unsupervised Domain Adaptation (UDA) in semantic segmentation and introduces CDEA, a Context- and Detail-Enhanced Unsupervised Learning framework. CDEA improves UDA by firstly introducing an Adaptive Masked Image Consistency module, which learns spatial context relations in the target domain, enforcing consistency between predictions and masked target images. Secondly, a Detail Extraction module integrates spatial information into low-level layers, combining low-level detail features with deep semantic features. Extensive experiments validate the effectiveness of CDEA, showcasing its superiority over state-of-the-art methods.
strength_weakness: strength_weakness: 1. The paper is well organized, the figures are readable and understandable.
2. The proposed method looks logical and technically sound.
3. The experimental results are good. Consistent improvements have been shown in different benchmarks.
limitations: limitations: 1. Concerns on the novelty: Previous work (MIC [1]) has also explored mask image consistency between predictions and masked target images. Directly applying a similar idea seems does not introduce sufficient novelties. Please the authors make extensive discussions on the differences.

[1]. MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation, CVPR 2023;

2. Insufficient comparison results:  1) The authors only conduct experimental comparisons on synthetic-to-real adaptation. Existing papers [1-2] also perform Day-to-Nighttime adaptation (Cityscapes→DarkZurich) and Clear-to-Adverse-Weather adaptation (Cityscapes→ACDC). The reviewer wonders whether are there any results on such benchmarks that could verify the effectiveness of the proposed method. 2) The authors only verify the effectiveness of CDEA using DAFormer and HRDA on domain adaptation benchmarks. In addition to them does the presented method also work on the MIC [1]? What are the improvements on the superior baseline of MIC[1]?

[2]. Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation, IEEE TPAMI 2023;



3. Insufficient experimental analysis: There's a lack of comprehensive experimental analysis, including TSNE feature visualization, visualization of source and target domains, class visualization, and the distribution of high-level and low-level features. These analyses are crucial for understanding the model's behavior and performance, identifying areas for improvement, and assessing how the model performs in different environments.

4. Lack of citations. Some important references [3-11] that also target context-aware domain adaptation [3-6] and Mean Teacher architecture for domain adaptive semantic segmentation [7-11] are missing. These works should be briefly reviewed in the related works. In addition, some recent works should be compared in tables.

[3]. Context-Aware Domain Adaptation in Semantic Segmentation, WACV 2021;

[4]. Domain Adaptive Semantic Segmentation with Regional Contrastive Consistency Regularization. ICME 2022;

[5]. Context-Aware Pseudo-label Refinement for Source-Free Domain Adaptive Fundus Image Segmentation, MICCAI 2023;

[6]. Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation, ECCV 2020;

[7].Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation. AAAI 2019;

[8]. Uncertainty-Aware Source-Free Domain Adaptive Semantic Segmentation, IEEE TIP 2023;

[9]. Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation. CVIU 2022;

[10]. Towards Better Stability and Adaptability: Improve Online Self-Training for Model Adaptation in Semantic Segmentation, CVPR 2023;

[11]. DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation, CVPR 2023;
paper_summary: paper_summary: This paper introduces a context- and detail-enhanced unsupervised learning framework (CDEA) for domain adaptive semantic segmentation. In this framework, it involves Context Enhancement and Detail Enhancement. For Context Enhancement,  it proposes an adaptive masked image consistency module to improve UDA by learning spatial context relations in the target domain, ensuring consistency between predictions and masked target images. For Detail Enhancement, it introduces a detail extraction module that integrates spatial information learning into low-level layers, fusing low-level detail features with deep semantic features.
strength_weakness: strength_weakness: 1. The paper is well-written with thorough attention to detail.
2. While the contributions of the paper may not be entirely novel, the authors address practical concerns and provide extensive design considerations.
3. The experiments are comprehensive, demonstrating significant advantages in performance.
limitations: limitations: Certainly, I don't have many concerns. I rated the paper as Borderline because the concepts of mask and low-level detail used in the paper are not entirely novel, but I can appreciate incremental innovation. This isn't my main issue, so there's no need for the authors to defend themselves. Here are my questions:

1. While the detail provided by the authors resembles edge maps, have they attempted to directly use edge extraction as ground truth for supervision?
2. Have the authors considered trying multiple masking approaches and conducting ablation experiments to determine the best mask?
3. Is it possible that the symbols for predicted detail and label detail in Figure 2 are reversed?
paper_summary: paper_summary: This paper proposes an unsupervised learning framework that integrates context and detail enhancement. On one hand, the study introduces an adaptive masked image consistency module to learn spatial context information in the target domain, thereby improving the performance of UDA. On the other hand, the study proposes a detail extraction module that integrates the learning of spatial information at lower levels with deep semantic features. Additionally, the proposed method can be effectively embedded into other models. The method has been validated across multiple datasets, demonstrating strong competitiveness and providing a powerful tool for semantic segmentation tasks.
strength_weakness: strength_weakness: Novelty: This paper propose a context and detail-enhanced unsupervised learning framework to harness both context- and detail-wise consistency against different contexts, which is well-aligned with the segmentation task.
Theoretical Approach: Utilizing spatial contextual information and detail information to improve the performance of semantic segmentation may be theoretically reasonable.
Evaluation: The performance improvement is demonstrated through comparisons with existing methods on two sets of datasets, highlighting the effectiveness of the proposed method.
Clarity: The structure of the paper is relatively clear.
limitations: limitations: 1.Please code the references in order of citation, as done with reference 40 in the introduction.
2.In Section 3.2, right column, ". The domain adaptation process with the CEM is illustrated in Fig. 3 and explained below." There is an incorrect citation of Fig.3.
3.In Section 3.2, right column, the meaning of the letter μ in formula (7) is not explained, resulting in an error in the description.
4.Please add the display and analysis of failure cases.
paper_summary: paper_summary: This paper introduces an innovative approach to enhance unsupervised domain adaptive semantic segmentation by incorporating spatial context relations as supplementary information. The study proposes a context- and detail-enhanced unsupervised learning framework designed to leverage both context- and detail-wise consistency across varying contexts. This framework aims to improve the robustness and accuracy of semantic segmentation models by effectively utilizing spatial context relations to better understand and adapt to new, unseen domains. Extensive experiments are conducted on two popular datasets to validate the effectiveness of the proposed method. The results demonstrate significant improvements in model performance, highlighting the potential of spatial context relations in enhancing domain adaptability and segmentation precision in unsupervised settings.
strength_weakness: strength_weakness: 1. The figures included in the manuscript are of high quality and the writing is clear and well-structured.

2. Using a spatial context relation to improve the accuracy is interesting.

3. The paper also presents good ablation studies and reports competitive experimental results to support the claims.
limitations: limitations: 1. The template used by the author is incorrect; the layout and related content need to be checked.  Like " Conference’17, July 2017, Washington, DC, USA " is wrong.

2. How does the proposed method address the issue of "similar local looking" by utilizing spatial context relations? Many other works [1][2] also utilize context- and detail-wise consistency to address this issue. What sets this paper's approach apart from others?

[1] MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation.
[2] Cross-level Contrastive Learning and Consistency Constraint for Semi-Supervised Medical Image Segmentation.

3. Is the problem shown in Fig. 1(a) solved via more spatial context relations? How does spatial context knowledge solve the problem of similar local looking? Is that solved due to the introduction of relative positions and relationships in the scene?

4. Recent method also enhances the context understanding via multi-modal fusion of visual and depth knowledge, the difference and similarity between CDEA and [3] should be discussed.
[3] Transferring to Real-World Layouts: A Depth-aware Framework for Scene Adaptation.

5.  How do you choose different weight values in the total loss function (Eqn. 12)? Is there any sensitivity analysis?