Domain Adaptation and Generalization: A Low-Complexity Approach
Authors: Joshua Niemeijer
Jörg Peter Schäfer
Venue: CoRL 2022 Poster (2022)
Keywords: unsupervised domain adaptation
semantic segmentation
domain generalization

Abstract: Well-performing deep learning methods are essential in today's perception of robotic systems such as autonomous driving vehicles.
Ongoing research is due to the real-life demands for robust deep learning models against numerous domain changes and cheap training processes to avoid costly manual-labeling efforts.
These requirements are addressed by unsupervised domain adaptation methods, in particular for synthetic to real-world domain changes.
Recent top-performing approaches are hybrids consisting of multiple adaptation technologies and complex training processes.

In contrast, this work proposes EasyAdap, a simple and easy-to-use unsupervised domain adaptation method achieving near state-of-the-art performance on the synthetic to real-world domain change.
Our evaluation consists of a comparison to numerous top-performing methods, and it shows the competitiveness and further potential of domain adaptation and domain generalization capabilities of our method.
We contribute and focus on an extensive discussion revealing possible reasons for domain generalization capabilities, which is necessary to satisfy real-life application's demands.


Domain Adaptation and Generalization: A Low-Complexity Approach
Joshua Niemeijer, Jörg P. Schäfer
German Aerospace Center (DLR)
Germany
Joshua Noshua.Niemeijee... www.eiimeeeeiiil. ,de
Joerg Schaefer@drr. .e
Abstract: Well-performing deep learning methods are essential in today's 's percep- tion of robotic systems such as autonomous driving vehicles. Ongoing research is due to the real-life demands for robust deep learning models against numerous do- main changes and cheap training processes to avoid costly manual- labellng efforts. These requirements are addressed by unsupervised domain adaptation methods, in particular for synthetic to real- world domain changes. Recent top perfforrrnnng ap- proaches are hybrids consisting of multiple adaptation technologies and complex training processes.
In contrast, this work proposes EasyAdap, simple and easy-to-use unsuper- vised domain adaptation method achieving near state-of-the-ee performance on the synthetic to real-world domain change. Our evaluation consists of compar- ison numerous top-performing methods, and it shows the competitiveness and further potential of domain adaptation and domain generalization capabilities of our method. We contribute and focus on an extensive discussion revealing pos- sible reasons for domain generalization capabilities, which is necessary to satisfy real-life application's demands.
Keywords: unsupervised domain adaptation, semantic segmentation, domain generalization
Introduction
Numerous applications have become feasible due to deep learning methods for computer vision, including the environment perception of autonomous vehicles. In particular, architectures emerged to deliver high-quality results for semantic segmentation tasks, i i.e., a pixel-wise classification of images. Unfortunately, it is in the nature of deep learning models to be sensitive domain changes, g.g., domain changes from day to night, from telescope to wide-angle lenses, or from rural to urban areas. An extensive training effort is necessary to keep the high-quality on the target domain, i. c., to adapt the model from the source to the target domain. Moreover, supervised training methods for the target domains are infeasible due the high costs of manually labeling data' and the virtually endless number of target domains?.
Several approaches exist, minimizing domain data. For example, Active learning approaches implement a human-in-the-loog training process, which asks human provide labels for selection of samples, in order to achieve high- quality training results with minimal manual labeling effort segmentation tasks remains huge challenge, it also still requires a certain amount of manual effort [3, 4, 5]. Unsupervised domain adaptation methods completely avoid the usage of manually labeled target-domain data. Instead, they align the distributions of source- and larget-domain data in the input [6, 7, 8J, feature [9, 10, 11, 12], output space [13, 14, 15], or combination of those [16, 17, 18]. The latter is complex in design, and therefore, hard reproduce.
completely avoiding labeling costs for the target-
[2]. While active learning for semantic
'The manual pixel-wise annotation of an image takes about 90 minutes. [1]] RReal-lit life applications demand adaptation every domain they might encounter.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

Training Phaso
Pscudo Labels
Seltouueev sion
Treined Model
Model(O)
Reinitaalie Model
Splcianin
Model(N)
Figure I: An overview of EasyAdapt's internal dependencies (dashed lines): The source-domain training depends on the source-domain data and labels; the self-training of the target-domain training depends the target-domain data and its previously created pseudo-labelt the self- self-supervision depends on the target-domain data and the feature clusters the source-doman data.
The usage of synthetic datasets as source domain completely abandons the need for manual labeling effort [19, 20]. However, the large domain gap between synthetic and real-world domains makes an adaptation between these domains increasingly difficult. Several aspects cause this domain gap, such as a decreased variety synthetic data and the differences regarding geometry, texture, and lighting conditions. Due to these impediments, models trained on synthetic datasets poorly perform on real-world data. On the other hand, real-life applications demand models that are robust against variety of domain changes. But performing domain adaptation large number of domains is not feasible due to the immense training effort. Desirable domain adaptation methods achieve a domain generalization. These methods train models not only to improve on the source target domain but also on unseen domains [21, 22, 23].
This work addresses the problem of unsupervised domain adaptation from synthetic to real-world domains. We propose EasyAdap, an iterative extension of our work in [11], which conducts a distri- bution alignment in the feature- and output-space (see Figure 1 for an overview). While achieving near state-of-the-ar performance on the GTA5 [19] to Cityscapes [1] domain change, our method is still simple and easy-to-tune due to its simple architecture Our feature-space distribution align- ment is based on a self-supervision through a clustering loss of the target-domain features towards class-specific source-domain feature centroids. A self-training addresses the alignment of the distri- bution the output space. We study the interplay between this self-supervision and self-training and show that synergistic effect exists (see section 4.4). We discuss the reasons for this effect in Sec- tion 4.5, Overall, our work contributes EasyAdap, an easy-to-use unsupervised domain adaptation method that yields near state-of- the-art performance. We contribute extensive study of various unsupervised domain adaptation methods regarding domain generalization.
Section 2 discusses domain adaptation methods related to our work. While Section 3 describes our method, Section 4 evaluates our approach and compares it to other domain adaptation methods w.r.t the performance on the target domain and unseen domains. This work concludes with Section 5.
2 Related Work
We categorize related unsupervised domain adaptation methods for semantic segmentation tasks re- garding their adaptation in the input, feature, and output space and hybrid methods thereof. The latter (hybrids) comprise methods from the different categories, making up the majority of the re- cent perforrrnnnng publications. While domain adaptation is broad field (e. g.. which includes learning new classes), we concentrate on adaptation approaches that align the distributions in the input, feature, and output space. The following subsections collect and briefly describe examples for these categories.
Input-Space Adaptation. Aligning the data distribution in the input space usually resembles a style transfer from the source domain to the target domain. Since a style transfer only changes the textures while keeping the geometry of the images, the ground truth is also valid for the style- transferred images. Now, training a model on labeled style- trae-trraneerree images from the source domain is close to training model on the target domain. The most successful approaches for style
PURL: http::///giiiioo.oo/////iiiiiias

---

transfer implement Cycle-GANs [6. 7], Fourier transformation [8, 24] or global image operators [16].
Feature-Space Adaptation. The adaptation in the feature space of deep learning model usually relies on adversarial training or self-suuervision and aims at distribution alignment. When the pre- logit feature distributions of the source and target domain match, the classifier in the last layer classifies the source- and target-domain images in the same way. Adversarial training is based on a feature extractor and domain discriminator network, classifying the feature space into source and larget domains. The optimization goal is to generate feature space that is not discriminable into the source and target domain while containing relevant representations for the semantic segmentation [9, 25]. A recent approach implements a contrastive loss* on class-wise features [10] attracting features of the same class while repulsing features of different classes independent of their domain. The approaches presented in [12] and our approach [11] apply semantic self-supervision to align the distribution in the feature space by attracting the target-domain features in the pre-logit feature space towards class-specifee source- domain centroids. In contrary to the contrastive self-supprvvsion presented in [10] the attraction mechanism used in [11, 12] does not rely on pseudo-labele for the target domain. Our work advances the approach of [11].
Output-Space Adaptation. Distribution aligmment approaches in the output space either imple- ment an adversarial training or self-training. Adversarial approaches consider the structure of the output space, i. e., they discriminate images between the source and target domain based on the (spa- tial) distribution of the output of the segmentation model [13, 14]. Aiming at an easily reproducible approach, we avoid adversarial approaches in our work. Most of the current top-performing meth- ods implement& a self-training on the target domain, which a training of the model on self-inferred labels (pseudo-labels). The effect of self-training depends on the quality of the pseudo-labels, i. c., these methods require strategies for the mitigation of false pseudo-labels. Common approaches ig- nore low-confident pseudo-labels using a fixed confidence threshold [27, 28]. Some approaches filter low-confident pseudo-labels dynamically [15, 17, 29].
Hybrid Methods. Most of the top performing approaches are hybrid methods combining adap- tions methods working in the input, feature, and output space, notably [16, 17, 18). The approach presented in [18] performs distribution alignment in the feature (clustering) and output space (self- training). The approach in [16] even combines adaptation methods in the input, feature, and output space. The currently best performing approach presented in [17] applies methods for distribution alignment in feature and output space and additionally performs three structurally different adapta- tion stages. Compared to these currently top-perfore perpeffooming methods, our approach comes with a much lower complexity regarding design and implementation effort.
3 Method
We introduce nested iterative training approach for domain adaptation based on repeated appli- cation of self-training and semantic clustering (see Figure 1). With this, we advance our approach in [11] from two stage model to an iterative model. We initialize the process with training on the source domain (see Section 3.1). The inner loop of the training process establishes a training on the source and target domain, stopping early after a short number of epochs. The training on the target domain comprises a semantic clustering and a self-training which implemented as described in [1]]. Upon finishing such inner iteration, the outer loop of the training process creates new pseudo-labels and repeats the process. Section 3.2 describes the algorithm for the training process.
3.1 Building Blocks
Source Training Self-training approaches require models with a sufficient understanding of the data to provide valuable pseudo-labels. To that, our approach initializes the model with sophis- ticated supervised training method on the source domain, including a rich data augmentation and a class-uniform sampling strategy. Zhu et al. [30] showed significant improvements in supervised training when applying class-uniform sampling strategy. We adapt their class-uniform sampling in
*CF. [26] for overview of contrastive learning.
3

---

the following way. We first gather a list of objects from the dataset, i.e., list of polygons', each of which has a class attribute and centroid. We sample from this list regarding uniformly distributed class attributes during the training. Eventually, each training batch combines randomly chosen crops and crops that are centered around these polygons. A parameter 0 S S controls the ratio of random samples and llss-unnformll selected samples. We pair this uniform sampling with a strong data augmentation consisting of Gaussian blurring, color jittering, and random scaling".
Semantic Clustering When training on the source- and target-domain data, our approach for clus- tering the feature space inspired by the image-Classffication method presented in [12] and advances our work in [11]². We compute the class centroids on the source-domain data as running averages of class-specific feature representations in the pre-logit feature space. We then compute a similar- ity matrix between these class centroids and the feature representations of the target-domain data. After weighing each entry of the similarity matrix with a constant that scales with the certainty of the classification on the target-domain data, our loss function computes the entropy of the weighted similarity matrix. This process implements a clustering algorithm that executes through the back- propagation. For details, please refer to the description in [11] or the supplementary material. The process of Semantic Clustering a form of self-supervision.
Training Our process applies a simple self-training approach. Before training a model on the target domain, we create pseudo-labels for the target-domain images given the current model. To mitigate the influence of false pseudo-labels, we exclude labels from the self-training that yield a prediction-vectee entropy exceeding the threshold 3 log K, where K is the number of classes and 3 > 0. Improving the quality and impact of the pseudo-labels for self-training requires implement- ing an iterative adaptation process. I. e., each adaptation step first needs to create new pseudo-labels before resetting the model's weights and (re-) starting the training.
3.2 Aap: Assembling the Bricks
Figure 1 and Algorithm 1 in the supplementary material describe our training process. First, we perform the supervised training on the source-domain data with class-uniform sampling and a strong data augmentation
gain an initial model M(). Then, the process enters the domain adaptation loop. Each adaption step first (re-) creates the pseudo-labels using the current available model M (n) (i. e., M (0) in the first iteration). After resetting the model's weights (lo weights pre-trained on ImageNet(31], D. the adaptation step starts training a new model. We rebuild the list of samples regarding random and class-uniform samples in each epoch of the training. While iterating over the smaller (source- or dorget-domain) dataset, we sample as many batches as the larger dataset allows, i.c., we restart sampling from the smaller dataset until we finish processing the larger dataset. We compute the semantic segmentation loss for each training batch on the source- and target- domain data using the ground-truth and lseudo-labels, respectively. We use the features of the source- domain data to update the running average of the class centroids. The similarities between these centroids and the target-domain features yield the clustering loss. We update the model's weights using both loss functions.
4 Evaluation and Discussion
This section evaluates our approach, EasyAdap, regarding domain adaptation and domain general- ization quality. To that, we evaluate its performance on the domain shift from synthetic to real-world data via adaptations from GTAS [19] and Synthia [20] to Cityscapes [1], respectively.
We compare our approach against numerous top-performing approaches from the literature in Sec- tion 4.1. Since real applications require domain generalization instead of specific domain shifts, we
$ 'The COCO format stores segmentation masks as polygons.
"These simple augmentations are available in the Tensorflow and PyTorch API.
In contrast to [12]. the method in [1I] solves additional problems regarding memory and computational complexity due to the image-segmennttion task.
'Note, that we define clustering as pulling elements towards defined cluster centers instead of finding a partition of the elements.

---

discuss and compare our approach regarding domain generalization against existing domain adap- tation approaches in Section 4.2. Section 4.3 evaluates the impact of our data augmentation and the class-uniform sampling. In Section 4.4, we show the effects of the self-training and self-supervision. We conclude our evaluation with discussion in Section 4.5
All of our experiments are based on an implementation of DeepLabV3+ with a WideResNe138 [32] backbone. We scaled all target-domain training images match the size of the source-domain images. While we train our model on crops of 400 x 400 pixels in batches of 24 source- and 24 target-domain images, we validate on the original sized images. The training process implements a stochastic gradient descent optimizer with momentum 0.9 and a weight decay of 10 - While the source-only setups trained for 45 epochs, our iterative adaptation steps stopped early after 20 epochs. We decay the initial learning rate of 0.007 with a factor 1 epoch
epoch after each epoch. We configure our domain adaptation approach as proposed in [11] except for the self-training threshold, which we now set to 3 To (see Section 3.1).
4.1 Comparison to Existing Approaches
Table shows the performance of state-of-the-art unsupervised domain adaptation methods from GTA5 to the Cityscapes dataset. This table includes methods aligning distributions on the input space [6, 16], feature space [33, 34, 35], output space (c. g., via self-training [15, 27)), and hybrid methods [16, 17, 18] (see Section 2 for a description of this categorization).
Table I: GTA5 to Cityscapes
1250
4120
45.
2001
11.3
24.5
20.3
53.5
2544
471
3507
Kiggieg [19]
15:88
19.1
1003
7176
15.1
11.8
8T.
15.8
7130
17.5
15-3
10.1
10%
131
103
11.3
21.6
EAEAAI PN)
86.7
11.5
80
19.4
11.5
14.0
19.9
41.5
87.7
37.0
73.6
6..99
19:0
65.0
13.0
11
ATTIDA [36]
$5.6
16.0
15.5
26.8
14.5
19699
77.9
10.3
K0
94.6
Pachhlier 1771
02.1
11:3
R21
74.247
15.1
24.5
15.6
11.0
13.6
10.6
27:3
15.4
46.3
2.
79
11.3
16
ARWENT
89-4
11.3
810
23-3
34.7
16.7
177
11.6
11.4
FEAL scEnua [1I]
30.
CBST (27)
91.5
$3.5
50.5
32.
21.0
3400
38.9
30.4
83.0
34.2
90.9
53.1
24.0
83.7
30.5
35.9
16.0
25.9
41.8
45.
MRKLD [58]
91.0
15.4
20.0
33.7
31.4
373
12.9
34.5
85.0
34.1
20.8
57.7
24.6
26.1
22.6
30.
26.
26.0
22.3
47.1
6.5
CAS
83.8
38.4
85.4
Unoenaary (15)
50.6
31.2
851
36.9
25.6
37.5
48.5
48.5
853
34.5
81.1
64.
36.8
96.3
34.9
52.3
1.7
190
446
503
(18T
92.8
13.5
86.
39.1
36.1
43.6
394
84.
15.7
28.1
639
38.3
86.0
41.6
506
m1
304
5.7
516
Coureifice
92.5
16.3
27.4
36.1
43.5
91.8
Poola (17)
$9.6
55.9
79.7
45.3
44.
45.
53.5
51.5
88.6
45.2
82.1
70.7
36.2
89.8
45.5
59A
48.9
56,4
$7.
11.15
R22
1-1
15715
12:15
1830
433
72.76
1:73
0315
3001
7241
55.55
2K11
411
275
178
25:33
souee-oally sng
44.62
36.36
36.44
14.13
29:34
2:07
somoo-asll usi 100%
76.00
34.02
75.52
26.15
2501
46.55
45.91
82.52
21.47
76.8
69.51
34.56
56.14
25.96
2814
32.56
23.95
445
654
10/90
60185
11
11:11
18.18
11.06
87
11.00
7m
10.14
1081
30.3
1200
17.9
1177
17.0
160
4618
Eacy Adsp
K7.60
16.1
65
17.31
40.03
19
86.0
89
41.1
82 12
61.5
40.13
saget
7593
Our method AdsyAdap achieves near state-of-the performance in terms of mloU on the GTAS to Cityscapes domain change: We achieve 56.32% mloU, which is comparable to the currently best approach ProDA with 57.5% mloU. Similar to ProDA, we gain our improvement over other approaches from classes like traffic light, traffic sign. fence, rider, and motorbike, which are difficult to learn. Our method outperforms approaches that have similarities to our method (see Section 2) on this domain change. In particular, this includes CLST and CAG with 50.2% and 51.6% mIoU, respectively. Except for ProDA, the same holds for all hybrid methods. ProDA, however, is hard-to- tune due to its complex architecture (four structurally different training stages) and --rrmee-a tuning. Advancing our method from [11] into an iterative domain adaptation process improves the results from 46.4% mloU to 56.32% due to synergistic effects between our self supervuperiiion and self-training. We further discuss these effects in Section 4.5.
Table 2 shows the performance of unsupervised domain adaptation methods from Synthia to the Cityscapes dataset. Without changing any hyper-paramette tuned for the GTA5 to Cityscapes do- main change, our approach still performs well, ranking behind ProDA, SAC, and CLST, all of which are also well performin in the adaptation from GTA5 to the Cityscapes dataset.
'Implementation based https: http:///e//D.eeeetteeeeeeettttte
5

---

Table 2: Synthia to Cityscapes
sallle light
regeaation
Bobubill
moll
Sallic
20500
64.5
[3.
51.4
7.0
22.3
05.1
67.6
-3.2
19.9
73.1
15.3
10.5
18.5
34.0
411
AcheSey (SS]
70.2
37.2
18.5
0.99
10.3
15.2
811.5
53..5
13.6
07.00
20.5
21.6
31.3
45.0
PacciAiign (37)
R2.4
35.0
73.6
K7
0.6
26.0
3.9
11.1
75.5
84.6
53.5
21.6
71.4
32.6
19.3
31.7
40.0
46.5
CLAN [35)
K1.5
31.0
80.1
10.1
13.3
76.2
81.5
53.4
21:2
73.0
30.9
22.6
10.7
47.8
APODA (16)
86-4
22.0
81.00
56.0
23.0
64.1
5%.1
ADVENT [14]
42.2
5.7
57.9
23.8
36.4
79.2
54.1
42.2
53.5
CBST
10.8
33.9
77.6
23.5
30.5
MRKLD
10.7
80.8
25.0
45.3
50.1
CAO UDA [18]
54.7
40.8
7.8
35.1
13.3
22.7
84.5
64.2
19.7
48.3
44.5
51.5
Sey- Uneeaarrant [15]
K7.6
41.9
K3.1
14.7
1.7
$6.3
31.3
19.0
81.6
80.6
61.0
21.8
66.3
40.7
25.
53.1
47.9
549
2Fine []]
75.7
50.0
61.9
11.5
2.5
55.3
18.0
12.7
86.2
90.1
65.1
33.2
K3.3
96.5
35.3
54.3
48.2
55.5
CLST (10)
85.0
49.2
62.2
16.3
Q.4
29.2
31.8
23.0
84.1
88.0
99.1
27.2
K5.5
46.6
28.5
56.5
49.8
578
SAC (29)
89..5
47.2
55.5
25.5
1.5
43.0
45.5
32.0
87.1
89.3
83.6
25.
96.9
35.6
30.4
53.0
52.6
59.3
PRODA (17)
$5.88
45.7
54.5
$7.1
0.6
64.0
56.6
32.0
88.1
86.3
74.3
24.5
89.3
51.1
40.3
45.6
55.5
620
sunce-only
8.6
11:58
53.01
1.45
90.25
18.55
9.1
74.57
58.74
56.83
8.16
66.98
12.05
3.41
9.69
26.31
29.34
euerronlyy any
55.14
10.54
69.12
4.69
40.3 85
28.78
23.55
80.15
76.93
61.90
21.59
40.78
18.6
17.08
22.6
17.15
42.31
source only uni 50%
64.39
28.34
72.2
3:35
1.19
40.09
27.51
20.18
79483
59.2
64.77
24.07
79.04
24.1
17.26
20.48
39.2
43.22
source only uni 100%
79.0
33.18
64.75
5.15
0.38
42.07
25.79
26.05
76.67
76.76
60.28
24.76
60.72
26.64
18.75
28.94
42 18
48.41
Lany Mep
$4.28
16
74.99
0.15
0.04
07.14
19.77
31.98
77884
55.11
75.33
36.14
56
56
26.05
28,99
23301
19.45
57.25
05.01
34.41
02.07
40.65
59.60
64.43
06.75
16.23
92.35
06.3
02.5
94.32
85.76
6539
16.03
76.12
72.77
Table 3: Domain generalization: Methods trained on GTAS (source-only) and methods adapted from GTA5 to Cityscapes tested on different real-world domains
Model
Cityscaes
BDD
rain
Eog
snow
night
DLy2 [1.3]
36.6
36.5
33.6
40.2
33.4
DLy2 (29)
408
31.3
soune-ooly Dlx3-
25.58
28.45
25.
souns-orly any
11.3
33.02
5.75
source
Dlv33
36.77
33.27
7.63
AAapSSg [13]
Sep-Uhoentainyy [15]
50.3
35.7
4144
14.0
SAC (29]
156
ProDA (17)
57.5
47.5
43.1
49.2
80.7
15.4
BasyAdap
56.6
46.09
43.02
48.56
38.87
14.38
target (Cllyscapes model
77.35
46.26
43.36
61.23
47.65
17.87
4.2 Domain Generalization
This section evaluates the domain generalization capabilities of our domain adaptation method and compares it with several other approaches. To that, Table shows the performance several models on the datasets Cityscapes, BDD [39], and four different domains of the dataset ACDC [40]. No model has seen samples of BDD or ACDC during the training. Furthermore, the source-only models did not the Cityscapes samples during training.
BDD differs from the target domain of the adaptation (Cityscapes) by containing a diverse set of weather and lighting conditions and by being recorded across the USA instead of Germany. ACDC, which was recorded in Europe, defines the specific partitions ruin, fog, snow, and night. Hence, regarding the models trained on GTA5 (and Cityscapes without ground-truth), these datasets include domain shifts regarding the environment, weather, and sensors.
The improvements of our source-only models due to data augmentation and additional class-uniform sampling also transfer to the unseen domains of BDD and ACDC. Considering the DeepLabV2 [29, 13] and DeepLab source-only models, there is no clear superior model. Note that the target model (trained on the Cityscapes dataset) yields similar results on BDD as the adaptation approaches ProDA and EasyAdap from GTA5 to Cityscapes. Apart from that, the target model always outperforms the domain adaptation approaches significantly. While ProDA leads the board of domain adaptation approaches, Easy Adap achieves very close results. Furthermore, Easy Adap outperforms the other domain adaptation approaches (see table 3).
4.3 Source-Only Training
The quality of our source-only training is crucial for the later creation of the first pseudo-labels. To study the design decision our source-only training, Table and 2 show the performance of different source-only models: trained without any augmentation (source-only); trained with color jitter, Gaussian blurring, and random scaling (source-only aug); and trained with augmentation and class-uniform sampling (source-only uni 50% and source-only uni 100%). On both datascts, GTA5

---

and Synthia, the augmentation improves the plain source-only model's quality by over 10% mloU. An additional unassom sampling improves the model's quality further by over 5% mloU. Sam- pling the complete batch via class-uniform sampling (source-only uni 100%) outperforms mixed batch of random and class-unifornly sampled images (source-only uni 50%).
4.4 Impact of Adaptation Features
We conducted several experiments to study the impact of Easy'sap's features on the model's quality. Figure 2 shows the best mloU values on the validation set per adaptation step, which consists of Ntrain = 20 training epochs (see Algorithm 1). We successively enabled self- training, class-uniform sampling"", and the semantic self-supervision on the target domain (see Section 6.1 and 3.1). Figure shows that self-training yields similar improvements bination with an additional class-uniform sampling on the target domain. A further self-supervision shows a steeper learning curve in the first few adaptation steps and yields a more durable learning behavior. e... the two learning processes without ceffssupervision stagnate after four adaptation steps, but the model trained with additional self self-suuerviiion still increases in quality.
com-
Source Only
TattTraining
EasyAdop
036
8M4
0.2-
8.0
0.6
64
e.Seffirain source unform 50% tureet soure Caayyldaa
0.12
disemanik salf superiiion
enfuure untfomm compling 13%
Adpptation
Figure 2: Left: Best mIoU per adaptation step: self-training (a); self-training-uunffrm sampling on target domain (b); colftrainingg-umifrmm sampling+semann self-supervision (c); uniform sam- pling+semantic self-supervision (d); uniform sampling source only (e). Right: Qualitative examples
4.5 Discussion and Limitation
Does domain adaptation transfer knowledge? Table 3 shows that the target model trained on Cityscapes (Cityscapes model) achieves similar performance on BDD as the ProDA and EasyAday models adapted to the Cityscapes dataset. While the Cityscapes model has overcome the domain change from sunny to rainy weather included in BDD, the adaptation methods need to overcome the domain change from synthetic to real-world The Cityscapes model also achieves similar perfor- mance on ACDC's rain domain as the ProDA and Easy Adap models. In this case, the target domain consists of rainy images only, which challenges the Cityscapes model even more, while the adapted models from GTA5 seem to play out their advantage of having seen synthetic rainy images during their training. On the other hand, the Cityscapes model outperforms the adapted models on the foggy and snowy domains, which are unseen for all adapted models. We think that the additional domain gap from the synthetic the real world for the adapted models is the reason for the performance drop of the adapted models compared to the Cityscapes model. Hence, the adapted models show the behavior of knowledge transfer (here, rainy images) from synthetic real-world domains. This observation supports approaches adapting from synthetic to real-world domains. Moreover, it sup- ports the demand for richer synthetic datasets regarding different domains, such sensor, weather, lighting, and environmental domains.
How support domain generalization through adaptation? Since domain adaptation every pos- sible encountered domain is infeasible in real-life applications, this work explicitly addresses and
From here on, always apply class-uniform sampling on the source domain.
7

---

studies domain generalization effects of unsupervised domain adaptation methods, including our own method EasyAdap. Section 4.2 and Table show stronger generalization effects for some of the adaptation methods. In particular, SAC, ProDA, and EasyAdap show significant improvements in the unseen domains of BDD and ACDC. These methods apply self-training while AdapSeg does not, and Seg-Uncene eeg-Uncertainty only applies self-training without improving the pseudo-labels through recreation. These observations open the question of whether self-training, which mimics a super- vised training both domains, yields these strong generalization effects.
What are the reasons for synergistic effects? The experiments in Section 4.4 and Figure show synergistic effect since enabling self- self-training and self-supervision gains 15% mloU while enabling only one of the mechanisms gains only 8% and 4%, respectively. Hence, simultaneously enabling both mechanisms gains another 3% atop each single mechanism. We argue that there is synergy between self-training and semantic self-supervision, e., they support each other. On the one hand, self-training aims to mimic supervised training on the source and target domain, which helps the training process to generalize upon both domains. This generalization effect yields a better alignment of the feature distribution in both domains, which in turn helps the self-supervision to attract the target-domain features to the correct source-domain class centroids. On the other hand, the self-supervision reduces the number of correct but ignored pseudo-labels by sharpening the feature space and therefore reducing the uncertainty correctly classified pixels.
How does the sampling strategy affect gemeralization? Sections 4.2 and 4.3 show that our source- only models generalize well unseen real-world data. We argue that combining a strong data augmentation with a class-uniform sampling improves the domain generalization by avoiding over- fitting. On the one hand, seldom classes gain weight in the training by oversampling, but the strong augmentation avoids overfitting. On the other hand, large-area classes such vegetation and terrain lose weight compared to seldom classes, which again avoids overfitting the training data.
Conclusion
We propose the easy-to-use unsupervised domain adaptation method EasyAdap with near state-of- the-art performance based on a self-supervision and self-training strategy. Our evaluation shows that the quality of our method comes from combination of improvements, including preliminary training on the source domain with increased domain generalization capabilities due to strong data augmentation and a class-uniform sampling; a self-training that yields strong domain generalization capabilities; and a synergy of our self-supervision and self-training. Our evaluation includes an ex- tensive discussion of domain adaptation methods at hand regarding knowledge transfer and domain generalization capabilities. Our discussion includes several research questions regarding domain adaptation and domain generalization of training and adaptation methods.
Acknowledgement
The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Climate Action" within the project "KI Delta Learning" (Förderkennzeichen 19A19013K). The authors would like to thank the consortium for the successful cooperation.
References
[1] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. CoRR, abs/1604.01685, 2016.
[2] B. Settles. Active learning literature survey. Computer Sciences Technical Report 1648, Uni- versity of -isconsin-Maddisnn, 2009.
[3] L. Cai, X. Xu, J. H. Liew, and C. S. Foo. Revisiting superpixels for active learning in semantic segmentation with realistic annotation costs. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 19988-10997, June 2021.
[4] Kasarla, G. Nagendar, G. M. Hegde, Balasubramanian, and Jawahar. Region- based active learning for efficient labeling in semantic segmentation. In 2019 IEEE Win-

---

ter Conference on Applications of Computer Vision (WACV), pages 1109-1111) 2019. doi: 10.100/WAACV2209.0001...
[5] S. Xie, Z. Feng, Y. Chen, S. Sun, C. Ma, and M. Song. Deal: Difficulty- active learn- ing for semantic segmentation. In Proceedings of the Asian Conference on Computer Vision (ACCV), November 2020.
[6] J. Hoffman, E. Tzeng, T. Park. J.-Y. Zhu, P. Isola. K. Saenko, A. Efros, and T. Darrell. Cy- cada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning, pages 1989-1998. PMLR, 2018.
[7] Y. Li, L. Yuan, and N. Vasconcelos. Bidirectional Learning for Domain Adaptation Semantic Segmentation. In Proc. of CVPR, pages 6936-6945 6945, Long Beach, CA, USA, June 2019.
[8] Y. Yang and S. Soatto. FDA: Fourier Domain Adaptation for Semantic Segmentation. In Proc. of CVPR, pages 4085-4095, Seattle, WA, USA, June 2020.
[9] J. Hoffman, D. Wang, F. Yu, and T. Darrell. Fens in the wild: Pixel-level adversarial and constraint- based adaptation. arXiv preprint arXiv:1612.02644 2016. URL https: org/pdf/ /112. 02649.22.99... pdf.
[10] R. A. Marsden, A. Bartler, M. Döbler, and B. Yang. Contrastive learning and self-training for unsupervised domain adaptation in semantic segmentation, 2021. URL https: //arxvv...vv org/pdf / 2205. 02001.pdr
[11] Niemeijer and P. J. Schäfer. Combining semantic sulf-supnriision and self self-training for do- main udaptation in semantic segmentation. In IV2021: Workshop on Autonomy @ Scale, Juli 2021. URL https: //elib.///b///e///eeeeeeeeeee dlr www.d.dd....... de/143158/
[12] K. Saito, D. Kim, S. Sclaroff, and K. Saenko. Universal domain adaptation through self super- vision. H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, page 16282-16292. Curran Associates, Inc., 2020.
[13] Y.-H. Tsai, W.-C. Hung. S. Schulter, Sohn, M.-H. Yang, and M. Chandraker. Learning to Adapt Structured Output Space for Semantic Segmentation. In Proc. of CVPR, pages 7472- 7481, Salt Lake City, UT, USA, June 2018
[14] T.H.. Vu, H. Jain, M. Bucher, M. Cord, and P. Pérez. Advent: Adversarial entropy minimization for domain adaptation semantic segmentation. IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2517-2522 2019. URL attps://openacceet wtwecceessssssttteeeeeeeeeeeeeee thecvf. com/ conteent_vPPeeeec twtp://.19///pprrs////ooooaaaaee Ent.rrraaalnttttooaaiaeeiieeee Siimentation_CCPP_220111
Proceedings of the
TeaaerssVu__ADVe Mor_Domminnn.teeeeeeeeeeereeeeee www.aaaiianaaaaiiiio.o.oo in in__emmatiic__ .df..
[15] Z. Zheng and Y. Yang. Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. International Journal of Computer Vision, pages 1-15, 2021. URL https: //apx:///axxxvv org/ wttgg//a////// /2003. 03773. .df..
[16] H. Ma. X. Lin, Z. Wu, and Y. Yu. Coarse-to- tine domain adaptive semantic segmentation with photometric alignment and category-center regularization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4051-4060, June 2021.
[17] Zhang, B. Zhang, T. Zhang, D. Chen, Y. Wang. and F. Wen. Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. CoRR, abs/2101.10979, 2021. URL https: //arxvvv..////////////aa//aaaavl org/ orgg/bs////0/ abs//aa... 10979.
[18] Q. Zhang, J. Zhang, W. Liu, and D. Tao. Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation. In Proc. of NeurlPS, pages 433 443, Vancouver, Canada, Dec. 2019.

---

[19] S. R. Richter, V. Vineet, S. Roth, and V. Koltun. Playing for data: Ground truth from com- puter games, In B. Leibe, Matas, N. Sebe, and M. Welling, editors, European Conference on Computer Vision (ECCV), volume 9906 of LNCS, pages 102-118. Springer International Publishing, 2016.
[20] G. Ros, L. Sellart, J. Matarzynska, D. Vazquez, and A. M. Lopez. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[21] S. Choi, S. Jung, H. Yun, J. T. Kim, S. Kim, and J. Choo. Robustnet: Improving domain generalization in urban-scene segmentation via instance selective whitening. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11580-1 1590, June 2021.
[22] X. Pan, P. Luo, J. Shi, and X. Tang. Two at once: Enhancing learning and generalization ca- pacities via ibn-net. In Proceedings of the European Conference Computer Vision (ECCV), September 2018.
[23] X. Pan, X. Zhan, J. Shi, X. Tang, and P. Luo. Switchable whitening for deep representa- tion learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[24] J.-A.
Termöhlen, M. Klingner, L. J. Brettin, N. M. Schmidt, and T. Fingscheidt. Continual unsupervised domain adaptation for semantic segmentation by online frequency domain style transfer. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 2881-2888, 2021. 110.1.11990TTTS497822220202001
[25] Y.-H. Chen, W.-Y. Chen, Y.-T. Chen, B.-C. Tsai, Y.-C. Frank Wang, and M. Sun. No more discrimination: Cross city adaptation of road scene segmenters. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[26] A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon. A survey on con- trastive self-supervised learning. Technologies, 91), 2021. ISSN 2227-7080. doi:10.3390/ technologies9010000. URL https: /tt::/wwww http//wwwpppp... com 2200//2277770///////
[27] Y. Zou, Z. Yu, B. Kumar, and J. Wang. Unsupervised domain adaptation for semantic segmentation via blass-balanced self-training. In Proceedings of the Eu- ropean conference on computer vision (ECCV), pages 289-305, Munich, Germany, 2018. URL https://ttopaaee http::///pccccccceeeo thecvf.ee. com/ content, @CCV__2011 hatpppper//aaaa Zou_ Zou_ Unsupervised Doomain_Aaapaatt
BCCV_ 2_11__21_______ 1at1-_ppp_____ .df.
[28] Y. Zou, Z. Yu, X. Liu, B. V. Kumar, and Wang. Confidence regularized self-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[29] N. Araslanov and S. Roth. Self-supervised augmentation consistency for adapting semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15384-15333 15394, June 2021.
[30] Y. Zhu, K. Sapra, F. A. Reda, K. J. Shih, S. Newsam, A. Tao, and B. Catanzaro. Improv- ing semantic segmentation via video propagation and label relaxation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[31] J. Deng, W. Dong, R. Socher, L.JJ. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hicrar- chical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 148-255. Ieee, 2009.
[32] Z. Wu, C. Shen, and A. van den Hengel. Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 90:119-133, 2019. ISSN 0031-3203.
[33] Y. Luo, L. Zheng, T. Guan, J. Yu, and Y. Yang. Taking Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation. In Proc. of CVPR, pages 2507-2516, Long Beach, CA, USA, June 2019.
10

---

[34] R. A. Marsden, A. Bartler, M. Döbler, and B. Yang. Contrastive learning and self-training for unsupervised domain adaptation in semantic segmentation, 2021. URL https: //rxxva..vvv/ org/pdf/ 2105. 02001.
[35] H. Wang, T. Shen, W. Zhang, L.-Y. Duan, and T. Mei. Classes matter: A fine-grained adversar- ial approach to cross-domain semantic segmentation. In European Conference on Computer Vision, pages 442.659. Springer, 2020. URL https:// :/ /rrxvv..vvv////////////////x/a org/pdf/ htpp///000/// 2 pdf.
[36] J. Yang, R. Xu, R. Li, X. Qi. X. Shen, G. Li, and L. Lin. An adversarial perturbation oriented domain adaptation approach for semantic segmentation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pages 12613-12620, 2020. URL https://oj] aaai wwa..org/aaaaaaaa....a
php/ AAA.Tartiilee/bomllaadaaa
[37] Y.-H. Tsai, K. Sohn, S. Schulter, and M. Chandraker, Domain adaptation for structured output via discriminative patch representations. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 1456-1465, 2019. URL https: /otpaaccs//seccsss..... thh.cc.. thecvf. com/ com/content.... cwrpattee_tccveeeeeee ICCV (CCCc/2201//pppere/eeeee /sai http://aaaa_eeeieaaaaaattttiiiia
Structured_ _tr
w hiwiaaDiisiiiiiaiieeeeeeeee htt.hcccheeeeeeeettttttoonnnnooo _CcCV__01______e_0 _apper_pppp... paaper.pdf..
Output_
ICC__2099_paprrr
[38] Y. Zou, Z. Yu, X. Liu, B. Kumar, and J. Wang. Confidence regularized self-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5982- 5991, 2019. URL https: batps://oa //txxv/xxvvv////////////////xx wtg/pdf//ee /998/190... 09822v3. pdf.
[39] F. Yu, W. Xian, Y. Chen, F. Liu, M. Liao, V. Madhavan, and T. Darrell. BDDI0OK: A diverse driving video database with scalable annotation tooling. CoRR, abs//805.00 006877 2018. URL http://arxiv. argg/bs//800/ 104e/8880000077.
[40] C. Sakaridis, D. Dai, and L. Van Gool. Acde: The adverse conditions dataset with correspon- dences for semantic driving scene understanding. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 10765- October 2021.


Reviews:
[{"review_id": "_0HKur8eAem", "reviewer": "Reviewer_vMEa", "summary": "The authors present a method for unsupervised domain adaptation. Many methods already exist which perform domain adaptation by aligning the distributions of the source and target data in the input space, feature space, output space, or a combination of them. The authors focus on the feature space and provide a straightforward method which uses multiple iterations self-training and semantic clustering to perform well on a target domain. The authors compare with many other methods and show strong performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- The authors compare to a wide variety of other methods and show strong performance in multiple settings. \n- The authors provide an interesting and comprehensive discussion, talking about the reasons domain adaptation works and the impacts of factors such as the sampling strategy and differing target/source domains. \n\nWeaknesses:\n- The main contribution of the paper EasyAdap is a very slight extension of the method in [10]. EasyAdap runs multiple iterations of self-training and semantic clustering instead of a single iteration in [10], and I do not see this as particularly novel.  \n- I am not convinced the methods are easy to use. Specifically, for good performance, potentially expensive and complicated data augmentation may be required. E.g. How does someone who wants to use the methods decide on how to augment the data? How do they know when there is enough augmentation? This would significantly complicate the implementation of the methods. \n- Even if the method is easy to implement, the authors do little work in ensuring that others can use their methods. E.g. there is no code to reproduce the results in the paper, the body of the paper does not explain EasyAdap thoroughly, and the methods are missing other important details which would allow for the use and understanding of the methods (e.g. precise details about the source training steps, and why the weights of source-trained model are reset).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The authors compare their method to a large variety of methods and show strong performance in multiple settings. However, the presentation needs a lot of work and there is minimal algorithmic contribution since the method is so similar to [10]. I think the paper would need a significant amount of re-writing and I cannot currently recommend acceptance. \nI do hope the authors address the comments and issues in the revision or submit a revised version of the paper in the future.\n\n---\n\n**Update after revisions and replies**: The authors have improved the paper and clarified a few important issues. The focus has shifted slightly away from a broadly easy to use method to a lower-complexity method, which I think improves the story of the paper. I also appreciate that the authors are willing to include a link to code if the paper is published. My main reservations are the similarity to [10] and the potential difficulty in deciding how to augment the training dataset. However, the performance of EasyAdap provides evidence that the extension from [10] improves performance significantly. Additionally, code and concrete implementation examples including a good set of augmentations provides a good starting point if others want to use this method. I will therefore raise my recommendation.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "Content Issues:\n- L133: How is this work inspired by [11]?\n- Sec 3. I am confused as to why the methods are so lightly discussed in the body of the paper as the authors mention EasyAdap as a key contribution of the work.\n- The authors claim their methods are easy to implement, however;\n\t- The authors do not formalize the domain adaptation problem well in Sec. 3. They offload much of the formalization to another paper [10] and only elaborate slightly in the appendix. E.g., there is a lack of information about the sampling and augmentation done in algorithm 1.\n\t- The importance of good source training seems to limit the applicability of the proposed method. It may take someone a long time to determine a good set of augmentations for a single problem. I see this counter to the idea of easy implementation. \n\t- The authors do not provide code for reproducing their results.\n- Please discuss the limitation of strong source training on the performance of the proposed domain adaptation method.\n- Section 4.3. Some of the methods in Table 1 and 2 are not mentioned anywhere else in the paper, e.g. PatchAlign [36], APODA [35]. Additionally, methods such as CLAN [32] and FADA [34] are left vaguely as domain adaptation methods which align the feature space. The authors should expand on the explanation of many of comparison methods. If necessary, this could be in the appendix.\n- Algorithm 1, L6: I am confused as what is meant by “reset(M)”. Do you reset the weights of M to untrained, and then need to do supervised source-only training again? If so, why is the source-only training necessary again? If not, what is M reset to and I am still not understanding why is it reset?  \n\n\nEditorial issues: \n- The in-line reference, equation, and section hyper-links are not working for me. \n- I am not certain the authors are using “cf.” correctly. E.g. in L113-L114: “We initialize the process with a training on the source domain (cf. Section 3.1).” I would interpret “(cf. Section 3.1)” as “compare to Section 3.1”, but it seems like they mean “see Section 3.1”. I believe the authors make this mistake frequently throughout the paper. \n- Algorithm 1, L18 - L22: Equations references are not referenced correctly\n- In Table 1, method “ADVENT” has reference [13] and in table 2 it has reference [35]. Is the [35] reference a mistake? If so please ensure the methods in tables are cited correctly. "}}, {"review_id": "Lk-azuCYGMV", "reviewer": "Reviewer_fxUx", "summary": "The authors propose EasyAdap method. The consists of the following three modules: \n(a)Source Training: use supervised learning method to train on data from source domain. It is designed to pretrain a good enough feature extractor and classifier for the following module.  \n(b)Semantic Clustering: this module is used to evaluate the similarity between target domain and source domain.  \n(c) Self training: pseudo label for data from target domain is generated and added for supervised training.\n\nThe authors evaluate their method on GTA5, Synthia, and Cityscapes dataset, and has a similar performance as the best SOTA method. \nThe authors include further discussion after their experiments.  ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The design of the framework is clearly written.\n2.  The authors have included complete ablative analysis in their experiments\n\nWeakness:\n1. EasyAdap doesn't seem to have a better performance than the SOTA baseline, e.g., ProDA\n2.  I think the experiments in the paper do not give an intuition why their methods could have a good performance over lots of baselines. For example, why do we need to add target domain data into the supervised training set and train again? Without such an intuition, I am not clear about how much distribution shift EasyAdap can handle. The authors only listed numbers and stats, but didn't explain that is happening inside their algorithm. For example, It would be very benefitial if the authors show some examples of the pseudo labels and how it improves during the iterative training procedure.  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The authors propose EasyAdap with great clarity and have sufficient experiments. However, the authors have to explain more about why and how their method helps. And another major issue is that their method doesn't beat the SOTA baseline.  ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "1. Figure 1 is extremely hard to read. It is very hard for someone who is not familiar with the specific dataset to understand what the pseudo-labeling is doing. In addition, it is also not clear what is happening in the self-supervision module.\n2. The authors should add more mathematical equations in their paper to make it more clear about what their method is doing. I noticed that there is some equations in the supplementary materials, but it would be nice if it could also be included in the main text. \n3. The authors should propose more qualitative analysis, e.g., what the pseudo labelling is doing. How the performance change with the iterative training. "}}, {"review_id": "7BydDzkd1_", "reviewer": "Reviewer_geXK", "summary": "The paper proposes an unsupervised domain adaptation method for semantic segmentation named \"EasyAdap\". The method is evaluated on autonomous driving datasets, with the goal of transferring a semantic segmentation model for traffic scene segmentation from a synthetic to a real (e.g. Cityscapes) domain. The method studies both domain adaptation (unlabeled target domain data available at training time) and the domain generalization setting (no target domain data available at training time). The method's key idea is to combine feature-space distribution alignment (similar classes have similar features across domains) and output-distribution alignment (model produce similar output distributions on source & target domain). Feature-distribution alignment is addressed through self-supervision using a clustering-like loss, output-distribution alignment through self-training on the target domain using pseudo-labels generated on the source domain.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The paper is mostly well-written\n- The method is technically sound and mostly understandable\n- The results show that the proposed methods reaches near state-of-the-art performance, and provide a thorough ablation study. I particularly liked the discussion on the interplay between self-training and self-supervision.\n- The related work section gives a very good overview of SoTA approaches to domain adaptation.\n\n\n## Weaknesses\n\n### Writing\n\nThe paper is mostly well-written, but I have some concerns regarding the clarity of the technical explanation.\n\n#### Motivation\n- I would like to see a better motivation for the way the method is put together in the introduction. The paper makes a convincing argument that self-training and self-supervision seem to have synergistic effects (discussion section). I would suggest to make this the main hypothesis of the paper, introduce and motivate it in the introduction, and use it as a guiding principle through the explanation of the paper.\n\n#### Algorithm explanation\n- While reading, I repeatedly got confused by self-training vs. self-supervision - these terms are very similar and easy to confound. Also, to me the clustering loss could equally be called an \"unsupervised\" loss. I would suggest to very clearly articulate (maybe even in a table) that the feature-distribution adaptation is achieved through self-training and the output-distribution adaptation through self-supervision.\n\n- 3.2 I think this paragraph could be improved by referring more explicitly to Figure 1, and by being more precise and consistent about the wording. For example, I would make more explicit which model is trained on which domain (e.g. that in 1 only source domain) with which labels (in 1 source domain comes with \"free\" labels, right?). Also, I got confused about \"smaller vs. larger\" dataset in line 156. This seems to be an important detail, but what's the purpose of this, why is it important?\n\n#### Experimental results\n- Section ordering. I would suggest to reorder the sections. The paper currently starts with ablation studies (4.1, 4.2), then moves on to domain adaptation results (4.3) and then domain generalization (4.4). I would start with the most important results first (4.3) and also rename the section to \"Domain Adaptation\", then 4.4, and followed by 4.1 and 4.2 where I would suggest using \"Ablation study\" in the titles.\n\n- Baseline difference discussion [10]. The paper explains that it extends the two-step method [10], which is also included in the table. I would have expected a section explaining the performance differences between the two methods as part of the experimental results, and analyze how and why the iterative setup improves results.\n\n- Clarity of Table 1 and Table 2. I would like to see a better explanation of tables 1 and 2. I was struggling to understanding initially what the \"source\" vs. \"target\" rows where and the latter is not explained in the paper (I figured it's the upper bound training on target domain data - but how is it trained?).\n\n- It would be nice to structure explicitly say which baselines are feature-based, output-based and hybrid\n\n#### \"Easy\"\n\nI have concerns regarding claiming the method is \"easy\". There is no concentrated place in the paper where the claim is justified. The abstract mentions that the method is \"easy to implement\", which  I find a very subjective claim - also does it matter in times where models are on github? The introduction claims \"easy to reproduce\", which is not explained at all -- again, if you fix random seeds and use the same code, isn't reproducibility a no-brainer? I would find \"easy to use\" (no extensive hyperparameter tuning, etc.) the most convincing interpretation of \"easy\", but no results nor discussion is provided in the paper (apart from stating that the same hyperparameters where used for domain adaptation and generalization).\n\nAlso, the paper seems to contradict itself, e.g. when claiming using a \"sophisticated\" model initialization in line 123. \n\nHowever, I think the paper still makes a contribution even if it was not \"easy\". I would suggest to focus on the iterative combination of self-training and -supervision and makes this the central theme and title of the paper.\n\n### Misc\n\n- Related work. It would be great to repeat here that the proposed method falls into the hybrid category (it does, right?)\n- Footnote 1: This claim is wrong as it makes a general statement about any pixel-wise domain. The authors should clarify on what dataset these 90 minutes apply and provide a source. I find 90 minutes extremely hard to believe. I would claim trained annotators using good tools can label complex Cityscapes scenes within minutes.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper tackles an important problem and shows convincing results and progress. The paper is mostly understandable but I would suggest to improve the clarity of the technical writing and the experimental section. Finally, I am challenging calling the method \"easy\" since I do not see convincing evidence for claiming this to be the case nor the key feature of the method.\n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "General:\n  - Make iterative self-training and self-supervision + its synergistic effect the central theme of the paper\n  - Drop or properly defend the \"easy\" claim of EasyAdap\n\nMethods:\n  - Improve the algorithm explanation by using more consistent and precise language (see explanation under strengths)\n  - Explain the point about smaller vs. larger dataset priority\n  \nExperimental section: \n  - Reorder sections from key result to ablation study (4.3, 4.4, 4.1+4.2)\n  - Baseline difference. Explain why the paper improves upon it's predecessor [10].\n\nMisc:\n  - Related work. Repeat here that the proposed method falls into the hybrid category (it does, right?)\n  - Footnote 1: Fix or substantiate this claim with references"}}]

Author rebuttal:

Decision: Accept (poster)