Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
Authors: Viraj Uday Prabhu
David Acuna
Rafid Mahmood
Marc T. Law
Yuan-Hong Liao
Judy Hoffman
Sanja Fidler
James Lucas
Venue: Accepted by TMLR ()

Abstract: Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domain. However, for high-stakes applications (e.g. autonomous driving), it is common to have a modest amount of human-labeled real data in addition to plentiful auto-labeled source data (e.g. from a driving simulator). 

We study this setting of supervised sim2real DA applied to 2D object detection. We propose Domain Translation via Conditional Alignment and Reweighting (CARE) a novel algorithm that systematically exploits target labels to explicitly close the sim2real appearance and content gaps. We present an analytical justification of our algorithm and demonstrate strong gains over competing methods on standard benchmarks.

Published in Transactions on Machine Learning Research (08/2023)
Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
Viraj Prabhu
David Acuna
Yuan-Hong Liao²
Rafid Mahmood
Marc T. Law3
Judy Hoffman
Sanja Fidler23
James Lucas
GGorgga Instiuute of Technology
University of Toronto
NVIDIA
University of Ottawa
Reviewed on OpenReview:
Abstract
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from labeled synthetic source domain to an unlabeled or sparsely labeled real target domain. However, for high-stakes applications (e.g. autonomous driving), it is common to have a modest amount human-labeled real data in addition to plentiful auto-labeled source data (e.g. from a driving simulator). We study this setting of supervised sim2real DA applied to 2D object detection. We propose Domain Translation via Conditional Alignment and Reweighting (CARE) novel algorithm that systematically exploits target labels to explicitly close the sim1reel appearance and content gaps. We present an analytical justification of our algorithm and demonstrate strong gains over competing methods on standard benchmarks.
1
Introduction
Driving
Source (sim. labeled)
simulator
Method
Target labels
mAP@50 (f)
labels
Source
Target
Human
UDA Khindkar al
2022
53.1
annotators
Target (real labeled)
FDA
Wang et al 2121
65.2
Mixing
Kishore er Mn
Seq. F
Tremblay
2018
06.4
Ours
CARE
68.1
Figure 1: Tradiiional sim2real domain adaptation (DA) assumes access to very few or no target labels, which is unrealistic in high-stakes applications like self-driving. We study the practical setting of supervised Sim2Real DA ap- plied to 2D object detection, wherein seek to maximize target performance given human-labeled target data and additional large set of machino-labeled simulated data.
Table 1: Car detection adaptation from Sim10K (Johnson
Roberson al. 2017 Cityscapes [Cordts al. [2016]: combining labeled source and target data using pro- posed method (CARE, gray row) improves over using a single data source (rows 1-2) well naïve combina- tions (rows 3-6).
Domain Adaptation (DA) is framework that seeks to overcome shifts in data distributions between training and testing. Typically, DA methods assume access to a large amount of labeled data from the training (source) distribution, and unlabeled or sparingly labeled data from the test (target) distribution (Saenko et al. 2010 Ganin & Lempitsky 2015 Tzeng et al. (017). DA has been extensively studied in computer vision for applications where annotating target data is expensive (Csurka al. 2022).
* Work done when while V.P.
NVIDIA. Correspondence virajpagatech.edu

---

Published in Transactions on Machine Learning Research (08/2023)
As annotation costs decrease, it becomes increasingly practical to annotate more target data, especially in high-stakes industrial applications such as autonomous driving (Mahmood et. al. 2022). A common practice in this field is to augment à target dataset of real driving scenarios with an additional labeled dataset generated in simulation (Karpathy 2021 Kishore et al.) 2021). Simulated data may be particularly useful to improve performance on the long-tail of driving scenarios for which it may be challenging to collect real labeled data (Rempe et al.) 2022; Resnick et al., 2022). In this paper, we formulate this setting as supervised Sim2Real DA problem. We use simulated, machine-labeled source data, and real, human-labeled target data (see Fig. D and ask: in this label-privilege setting, what would be the most effective way to combine simulated and real data to improve target performance?
Surprisingly, this practical setting has received little interest in recent domain adaptation literature, which focuses on unsupervised adaptation (no target labels, Chen et al. (2018); Acuna et al. (2021a); Li et al. (2022b)). ) and few-shot and semi-supervised adaptation (few target labels, Donahue et al. (2013); Wang et al. (0199]); Saito et al. (2019ah: [aang et al. (2020)). Although such methods could be extended to the supervisod setting, e.g. by adding supervised target loss to an off-the-shelf unsupervised DA method, we find this to be sub-optimal in practice (see Table ! since these straightforward extensions do not exploit large-scale target labels and their statisties for domain alignment. Similarly, few-shot and semi-supervised adaptation methods assume access to limited target labels (e.g. 8 labeled images per class for object detection, Wang et al. (2019a)) that are insufficient. for reliably estimating target statistics. Facing this research gap, industry practitioners may resort to naively combining labeled source and target data via mixing (Kishore et al.) 2021 (i.e. training on combined source and target data) or sequential fine-tuning (Tremblay et al.] 2018 Prakash et al. hcuristics do not address the domain gap between simulation and rcality.
2019: 2 (i.e. training on source data followed by finetuning on target data). However, these simple
This paper addresses the research- practice gap to show that systematically combining the two labeled data sets can significantly improve performance over competing methods (see Table D. We propose a general framework called Domain Translation via Conditional Alignment and Reweighting (CARE) for supervised Sim2Real DA. CARE builds on commonny-used baselines and off-the-shelf adaptation methods but explicitly leverages existing labels in the target domain to minimize both appearance gaps (pixel and instance-level visual disparity) and content gaps (disparities in task label distributions and scene layout). Specifically, we overcome the appearance gap by explicitly using ground-truth labels to conditionally align intermediate instance representations. To overcome the content gap, we conditionally reweight the importance of samples using estimated spatial, size, and categorical distributions. We formalize our setting using the joint risk minimization framework and provide thcorctical insights for our design choices. Finally, we apply our framework to the challenging task of 2D object detection. We make the following contributions:
(1) We present the first detailed study of supervised Sim2Real object detection adaptation, addressing large rescarch-vs-pre vsspraccicce gap between unsupervised and few-shot domain adaptation with an industry-standard practice of combining labeled data from simulated and real domains.
(2) We show that existing adaptation methods yield sub-optimal performance in this sctting by not adcquately exploiting larget labels. We propose CARE, an algorithm that systematically exploits target labels to bridge the sim2real domain gap by performing conditional alignment and reweighting. On three Sim2Real benchmarks for detection adaptation, CARE strongly outperforms competing methods (e.g. boosting mAP@50 by as much as 25% on Seuscapes-CCitysappee
(3) We formalize our setting using the joint risk minimization framework and provide theoretical insights into our design choices.
2 Related work
To our knowledge, supervised domain adaptation (SDA) for object detection has not seen recent work in
computer vision. Early DA works (Sacnko et al. 2010 Kulis et al., 2011 Hoffman et al.] 2013 Tsai et al. 2016) have studied the SDA setting applied to image classification, proposing contrastive-style approaches based metric learning with cross-domain pairwise consts. However, these works predate deep learning
2

---

Published in Transactions on Machine Learning Research (08/2023)
and do not study complex tasks like object detection. Below, we summarize lines of work in the related areas of unsupervised and few-shot adaptation.
Unsupervised domain adaptation (UDA). The DA literature primarily focuses on unsupervised adapta-
tion from a labeled source setting to an unlabeled target domain (Saenko et al. 2010 Ganin & Lempitsky
2015 Hoffman et al., 2018 Zhang et al. 2020). Successful UDA approaches have employed different strategies
ranging from domain adversarial learning Long et al. (2015); AAcna et al. (2021b) to domain discrepancy
minimization [Long et al. 2018), image translation (Hoffman et al.; 2018), and self-training (Prabhu et al.
2021 Li et al. 2022b). Cross-domain object detection has also seen recent work, based multi-level domain adversarial learning (Chen et al. 2018), strong-weak distribution aligmment of local and global features (Saito et al. 2019b), and domain adversarial learning weighted by region discriminativeness (Zhu et al.) 2019).
Alternatively, RoyChowdhury et al. (2019): Li et al. (2022b) self-train with refined pseudo-labels, and Kim
ct al. 2019) use background regularization.
Importantly, due to the absence of larget labels, UDA methods resort to approximations based on marginal alignment or pseudo-labels. In this paper, we instead consider supervised sim2real adaptation where ground- truth labels are provided for the target dataset during training. To compare against our approach, we benchmark supervised extensions of existing UDA methods as basclines in our paper. While our setting bears similarity to general transfer learning (Zhai et al.! 2019), it still makes the additional standard DA assumption of (large to full) label set overlap between train and test distributions. As a result, in DA such transfer can be realized by directly aligning class features across domains.
Few-shot (FDA) and Semi-supervised Domain Adaptation (SSDA). Closer to our setting are
Few-shot DA learning (FDA, Wang ct al. (2019a); Gao et al. 2022 : Zhong al. (2022) : Ramamonjison
et al. (2021) and Semi-supervised DA (SSDA, Donahue et al. 2013 : Yao et al. (2015); Saito et al. (2019a)), )
which differ in important ways. FDA assumes a very small amount of labeled target data is available (e.g. 8 images per class for detection in Wang et al. (2019a)). Such methods employ source feature-regularizztion
with instance-level adversarial learning (Wang et al. 2019a), point-wise distribution aligument (Zhong et al. 2022), and multi-level domain-aware data augmentation ion (Gao et al.] 2022). SSDA also assumes limited target labels (e.g. 1 to 3 images per category for image classification (Saito et al., 2019a)), but additionally leverages large set of unlabeled target data, making use of min-max entropy optimization (Saito et al. 2019a) student-Leacher learning frameworks (Li el al., 2022b). Closest to our work is Motiian et al. 2011 7 which also considers supervised adaptation of image classifiers by minimizing appearance gap via cross cross-domale domain semantic alignment, focusing on the setting wherein target data is sparingly labeled (e.g. 3 examples haapplls//caa claas on Office-31 (Saenko et al.) 2010)). In contrast, we study supervised DA in the context of object detection, which introduces an additional content gap arising from changes in scene layout and task label distributions. However, we assume access to a substantial amount of labeled target data in addition to a large (in theory, possibly infinite) amount of labeled simulated data. This assumption uniquely permits reliable estimation of target statistics. Our algorithm leverages these statistics and target labels to systematically close the sim2rcal domain gap.
3 Approach
In this section, we first introduce the supervised Sim2Real detection adaptation problem (Section 3.1 J. We
characterize two primary aspects of the Sim2Rcal domain gap: an appearance gap and a content gap (Section Finally, we introduce our method, CARE, that leverages a labeled larget dataset to close this domain gap (Section 3.3 and provide an analytical justification of the algorithm (Section 3.4).
3.2
3.1 Problem Formulation
Let. X' and y denote input and output spaces. In object detection, x E X' are images (X C Wxxx and y == (B, C) E y are K-class labels with C' E {} and bounding boxes B C [(w, h, x) e R*} (comprising the width w, height h, and centre coordinates (x,y), respectively). Let h(x:) == (o(g be an object detector composed of a feature extractor g(x) and a classifier h(g(z)) that are parameterized by o and 0, respectively. Matching prior object detection work Khindkar et. al. 2022: Wang et al.) 2021) we
3

---

Published in Transactions on Machine Learning Research (08/2023)
Appearance gap
Content gap
Fevel-level
Instancc-level
Task-level
Source (simulated) laced)
Box location be /E carl
Targer (real)
Yetarchela
Figure 2: The domain between simulated source and a real target domain can be decomposed into an appearance gap and content gap. The appearance gap corresponds to pixel-level differences (e.g. texture and lighting) Instance-level differences (e.g. vehicle design). The content gap consists of differences in label distributions due to different class frequencies and bounding box sizes and locations. Right. Column 1: Task label histograms. Column 2: Empirical distribution of "car" box sizes. Column 3: Empirical distribution of "car" box locations.
region proposal network that receives features generated by backbone network and passes them through a Region of interest (ROI) align layer to obtain ROI features; these are then passed through final box predictor. We let B, arg max h(g(z)) be bounding box coordinates and object class predicted by the model for input image æ. In sim2real SDA, we are given two labeled data sets representing a (simulated) source distribution Ps and a (real) target distribution Pr. Our goal is to minimize the expected risk of a detection loss consisting of a classification loss lets and bounding box regression loss loox:
design (g(x)) via Faster R-CNN (Ren et al. 2015), which
te (h(g(z)), B,C) = (b, lato(C, CC
(1)
over target domain rr Br,BBcrP [dde B,)).
3.2
Characterizing the Sim2Real Domain Gap
Leveraging the source distribution improve performance on the largel challenging due the domain gop which exists in both the image and label distributions. We partition this gap into two categories: appearance gap and content gap (Kar et al. (2199) and characterize these in detail, using the Synscapes (Wrenninge & Unger 2018
Cityscapes (Cordts et al. 2016 shift for object detection adaptation
example.
The appearance gap consists of visual disparities between images from the two domains (see Fig left). For example, a pixel-level appearance gap may be due to differences in lighting betwoen real and simulated images (Chattopadhyay et al. 2022), while an instance-level gap may be due to differences the appearance of synthesized versus real objects. We characterize the appearance gap as the dissimi- larity D(. in the probabilities between source and target distributions when conditioned on the label (e.g. Pr C))).
The content gap can be decomposed into scenc-level changes in the layout of objects (e.g. size and spatial distribution) as well as shifts in the task label distributions and the frequencies of classes (see Fig 2 right). We characterize the scene-level changes as the dissimilarity in the probabilities of object bounding boxes when conditioned on the class (BIC( Pr((()) and the task-level class frequency gap as the dissimilarity in class probabilitics D(Ps(C), PrC)).
3.3 Bridging the domain gap with CARE
To close the sim2rcal gap, Conditional Alignment and Reweighting (CARE) minimizes the effect of both the appearance and the content gaps via feature alignment and importance reweighing. Let ws(C)= I/)s(C), WT(C) 1/Pr(C) be the inverse class frequency for each domain and let v( B CB ==
4

---

Published in Transactions on Machine Learning Research (08/2023)
Source (sim)
Target (read)
Source (sim)
Target (real)
Figure 3: Visualization of cross-domain cycle consistency matching with CARE on Sim10K Cityssapps.. CARE embeed similar-looking cars closer to minimize the appearance gap.
Pr(BIC)/Ps(BIC) be the inverse ratio of the scene-level bounding box frequency gap. These reweighting factors ensure that the learned classifier considers that the source and target data sets follow the same distribution during training. In CARE, we minimize the following domain translation loss:
min Er ws( (c(CCu(BCC))a
+ B ( (h(a B3, C*)] ( C²
(2)
+ at 2.8 C.B.CPP
where Eatign is defined in Equation 3 and X 2 is regularization parameter. The above loss minimizes three terms, where the first term is reweighted detection loss over the source dataset and the second loss is class-balanced detection loss over the target dataset. The third term aligns the encoded features g(x) and g("") of similar cross-domain instance embeddings belonging to the same class. We now elaborate upon each.
3.3.1 Bridging appearance gap with cross-domain cycle consistency
To minimize the appearance gap, fatign performs class-and-box conditional feature alignment strategy by optimizing a cross-domain cycle consistency objective. Specifically, we extract ROI fcatures corresponding to the ground truth bounding box coordinates of both source and larget images and match similar cross-domain instance features belonging to the same class. Fig. illustrates this intuition.
Matching source to target. For given class, suppose we are given ground truth bounding boxes from the source and larget domains each. For each instance, our encoder extracts d-dimensional ROT features: let n' E Rd and v Rd denote features extracted from the i-th source and j-th target instances. For source instance i, we compute sofl-matching target features v similarity to u'. To measure pairwise cross-domain similarity Siff we employ negative squared Euclidean distance between features. Such soft-matching has been shown to be more stable than matching to a single target instance (Dwibedi et al. 2019 Wang et al. 2019b).
average of target features v weighted by their
v',,', where asj =
and 81,j := ||u v
> Em e
Matching back to source. Next, form cross-domain cycle by matching soft target features back to the source. As before, we first compute pairwise similarity scores for backward matching â,,i - || v+ and assemble similarity score vector s' = [$........ 8m. Sm.....,&...l E I We treat this vector as logits and minimize the following cross-entropy loss:
latign (u, 81)
Σ 1m=i (log (softmax(s))m)) where 1 1m is the indicator function. m=1
33)
5

---

Published in Transactions on Machine Learning Research (08/2023)
Source (sim)
Backbone RRN- Align
Encoder
Source feats
l
align
Target feats
Bactbonn-RRPN--CC
Box predictor
Target (real)
Source
w(C)u(BIC)
Target
w(C)
Classes
Figure 4: Conditional Alignment and Reweighting (CARE) exploits target labels to estimate and bridge cross-domain appearance gaps (via cycle consistency-based conditional feature alignment. objective) and content gaps (via importance reweighting).
Intuitively, this self-supervised cycle consistency objective encourages aligning feature representations for visually similar cross-domain instances belonging to the same class. Further, in the supervised DA setting, access labels in both source and targets allows us condition on ROI features corresponding to ground-truth (rather than proposed) bounding boxes, which ensures that the features being aligned are well-localized.
The above approach is inspired by temporal cycle confusion objective proposed for robust object detection from videos (Wang et al.) 2021). However, we make several fundamental modifications: (i) CARE matches instances across domains (rather than time), (ii) CARE matches instances from the same class, whereas Wang ct al. 2021) study a 1 class sctting, (iii) CARE aligns features corresponding to ground truth (rather than predicted) ground truth boxes, and (iv) we match instances that are close (rather than far) in feature space. Of these, (i) is necessary for use in domain adaptation, whereas (fi-iv) are additional changes we propose simplify learning and account for the large visual discrepancy across classes and domains. In Section 4.5 demonstrate that our modifications improve performance. Our alignment loss reduces to the classification of the soft nearest neighbors and therefore tends to be robust to label noise (Dwibedi et al. 2019).
We remark here that despite these novel modifications, our primary contribution is actually theoretically- motivated joint framework for supervised domain adaptation that combines conditional alignment and importance reweighting and is agnostic to the underlying feature alignment strategy. To validate this, Section 4.5 we present adaptation results by swapping out only the feature alignment strategy from cycle versions of two popularly used domain adaptation strategies and
consistency to www-aadd-boooootieee demonstrate consistent improvements.
3.3.2
Bridging content gap with importance reweighting
To close the task label distribution content gap, apply inverse frequency reweighing simulate balanced label distribution in the source and target domains. For each domain {S,T), we reweigh instances of class C' via multiplicative class weights (C) & ICN=(C), where NC(C) is the number of training examples in domain w.
We approximate the class-conditional box ratios as follows:
Pr(B Pr(w, b/C) Pr(x,y(C) (B( ~
Ps(w, Ps(x.y(C) =: eBBCC
(4)
Intuitively, this ratio up-weights boxes of class that are of a size and location relatively more represented in the larget than in the source. Note that the first approximate equality s is due to an assumption of independence between (w, h) and (x,y), which we assume to simplify computations. We estimate each probability component via class-conditional Gaussian kernel density estimation (KDE) (Scott (015)) fitted the ground truth bounding box locations and sizes respectively. In Appendix A.2] we include details of this est imatton including appropriate smoothing and thresholding to handle regions with low target support.

---

Published in Transactions on Machine Learning Research (08/2023)
Note that while class-rebalancing is well-studied for class-imbalanced problems, to knowledge, are the first to propose class-conditional box reweighting for object detection adaptation to mitigate sim2real content gap. In Section 4.5 we demonstrate that such reweighting consistently improves performance.
3.4 Analytical justification
We now analyze our loss function in Equation 2 to develop a theoretical intuition for its effectiveness. Let us rewrite the first term in the loss as follows:
Ps(a, B,C) Pr(x, B. (B(() (5)
Eps
ws (B(C)u(BBC)) (.). = Enr
Ps(C)
x Ps(BIC)
B,(x
B.C]
= EEr
x Pr(x CC
x
(6)
Pr(C)
(B(BC)
A Above, Equation 5 follows from importance reweighting, and Equation follows from Bayes rule. Next, recall that ws(C)) 1/Ps(C)) and v(BIC) = (B C). Substituting these two, we obtain
Ps [= (x Pr(C) (c)()
Eq.
Epr
(7)
Finally, recall our feature alignment component, which is designed to minimize the distance between encoded features of the same class and box statistics. Successfully minimizing the third term in Equation 2 obtain Bs(g(z)|B,C) = Pr(g(=) C). Using this, we obtain
should
Ps(g(z) Pr(g(x)\B,C) Pr(C) Pr(C) (hellll))))
Eq. ~ Epr
Err
Pr(C) (dea(((((())) B.C)
where the first line follows from the assumption that feature-level distances should reflect image appearance distances, and the second line follows from minimizing fatign. Overall, Equation S and the second term in Equation 2 minimize class-weighted version of the expected risk rr. In our case, the target metric is mean AP, which values performance on all classes equally. Since in practice target distributions often feature imbalanced classes, this modified risk simulates balanced label distribution and better maximizes mAP.
The steps above follow from several assumptions including independence of box position and size, equivalence between the ratios of feature-level probabilities and appearance probabilities, and that the target support is a subset of that the source. Further, it relies on successfully minimizing this feature-level gap. Nonetheless, as we show in the next section, our method demonstrates powerful empirical performance in the target domain.
4 Results
We now describe experimental setup for object detection adaptation: datasets and metrics (Section 4.1 implementation details (Section 4.2), and baselines (Section 4.3 We then present our results (Section 4.4 and ablate (Section 4.5) and analyze our approach (Section 4.6) Our experiments reveal that CARE consistently oulperforms alternatives including training with only target data, naïve stralegies for combining data sets, as well as modifications of vanilla UDA strategies. Moreover, these improvements are consistent across classes, box sizes, and crror types. Finally, our strong performance necessitates both the feature alignment and the class rebalancing techniques combined.
4.1 Datasets and metrics
We perform domain adaptation from three different source data sets of synthetic images, Sim10K (Johnson Roberson et al.] 2017), Synscapes (Wrenninge & Unger 2018), and internal data set. denote by

---

Published in Transactions on Machine Learning Research (08/2023)
Internal-Sim. Sim10K contains 10,000 images of 1914x1052 resolution with pixel-level annotations extracted from the game GTA-5. Synscapes is a photo-realistic dataset of 25,000 synthetic driving scenes of 1440 720 resolution. Finally, Internal-Sim is a private syuthetic data set of 48,000 photo-realistic driving scenes. Synscapes and Internal-Sim exhibit a long-tailed category distribution (sec Fig. 2 For each source, we train
an object detector to adapt to our targel, Cityscapes (Cordts et al. 2016 which a data set of 2,500 real driving images.
For our primary evaluations, use 25% of labeled target data (625 images Cityscapes) and the entire labeled source set (10k/25k/48) respectively), to mirror the practical scenario of having access to modest amount of labeled real data in addition to an order of magnitude more labeled simulated data. We find that this regime is where the conventional practitioner baselines of mixing and sequential fine-tuning show the largest improvement. On the other hand, if the target data set size is as large as the source size, we observe minimal improvements with conventional strategies to be minimal. While we use this as our primary evaluation sctting to compare against basclines, we further validate our strategy on the full ranges of target data to get complete analysis; find that. using CARE has a large positive impact over all baselines no matler we focus object
how much source versus target data is available (see Fig 5). For - detection for single class (i.e. car) to better compare against prior Sim2Rcal domain adaptation methods and Internal-Sime -ii--iiiimmmaaaaaaeeeaee CityScapes, we evaluate object
KKindkar et al. 2022). For Sereeeapes--ittscaapea detection for the subset of classes that are shared between source and target, corresponding to eight and three classes respectively. To evaluate all models, we match prior work Chen et al. (2018); Khindkar et al. (2022] ; Wang al. (2021) and report per-category Average Precision (AP) and its mean across classes at an ToU threshold of 50% (mAP@50), over the target test set.
4.2
Implementation details
We use a Faster-RCNN Ren al. (2015) architecture with ResNett-50 He et al. (2016) backbone. We run 10k iterations of SGD with learning rate of 0.01, momentum of 0.9, weight decay of 10 and learning rate warmup matching Wang et al. (2021). We set À = 0.1 in Equation 2 We use 8 NVIDIA V100 GPUs with a Per-GPU batch size of 4 and maintain 1:1 within-batch cource-to-target ratio across experiments. For more details see the appendix.
4.3 Baselines
We compare the performance of CARE to diverse range of basclines designed for the unsupervised, fow-shot, and supervised DA regimes, in addition to proposing two new baselines for supervised DA.
Control. As controls, we benchmark the following single-domain baselines:
(1) Source only: Supervised learning using only the labeled source dataset.
(2) Target only: Supervised learning using only the labeled target dataset.
Unsupervised DA. We compare against recently proposed method that only uses labeled source and unlabeled target data:
(3) ILLUME (Khindkar et al. 2022): For completeness, we copy results on Siml0K of state-of oft-rr--ttt-tttoe UDA method that uses labeled source and unlabeled target data.
Few-shot DA. We compare against a recently proposed method designed for use in the labeled source and few-shot target regime:
(4) TEA WWag et al. 2020): TFA is a two-stage finetuning algorithm proposed for few-shot object detection that updates all parameters on source (base) data followed by finetuning only the final layer (box regressor and classifier) on a balanced dataset of source and target data. However, we observe relatively low performance with finetuning only the last. layer (despite using a lower learning rate as recommended and both with and without weight re-initialization). Instead, we report results without freezing weights in the second phase. Note that we provide TFA with an identical amount of labelod target data as our method for fair comparison.
Supervised DA. We benchmark the following supervised DA baselines:
8

---

Published in Transactions on Machine Learning Research (08/2023)
Table 2: Results for supervised sim2real object detection adaptation on target. We compare CARE to source and
target only training, state-of-the-eee the-rrtt unsupervised DA method (ILLUME (ILLUMB|Khindie)) Khindkar et al. (2022) naive sem+reeal
combinations (mixing Kishore et al. (0221) and sequential finetuning Tremblay et al. (0188)
supervised extensions of
popular UDA methods (DANN Gamin & Lempitsky (2015 and MMD Long al. 2015 and recently proposed few-shot detection strategy Wang et al. 2000
Method mAP@50 (1)
Source
41.8
Method
mAP@50 (t)
Method
mAP@50 (t)
UDA
53.1
Target
62.1
Source
19.2
Source
22.5
Mixing
64.8
Target
34.2
Target
45.2
Seq. FT
66.4
Mixing
39.0
Mixing
49.3
S-MMD
65.8
Seq. FT
39.8
Seq. FT
45.4
S-DANN
65.3
S-MMD
40.0
S-MMD
50.6
FDA
65.2
S-DANN
40.8
S-DANN
49.8
CARE (ours)
68.1
CARE (ours)
48.5
CARE (ours)
53.7
(a) Sim10K -itsse-C (1-way)
(b) Synscapes Synscapes-+Cityscale Cittsscapes (8-way)
(c) Internal-Sim Intemal-Sim---CtyySappee (3-way)
(5) Mixing Kishore et al. 2021 Supervised learning on the combined source and target data sets, while maintaining a 1:1 ratio within batches (we ablate this mixing ratio in appendix).
(6) Sequential Finetuning (Tremblay et al. 2018): Supervised learning on the source dataset followed by finctuning all layers of the model with the target dataset.
We also propose and benchmark supervised extensions of two popular UDA strategies. Similar CARE, both methods first extract RO1 features corresponding to ground truth bounding box coordinates for all source and target instances within a batch, but differ in their feature alignment objective:
(7) S-MMD: A class and box-conditional supervised version of Maximum Mean Discrepancy (Long et al., 2013). S-MMD computes the per-class cross-domain maximum mean discrepancy (MMD) in feature space and then minimizes its average across classes as a domain alignment loss using lincar kernel.
(8) S-DANN: A class and box-conditional supervised version of DANN (Ganin & Lempitsky 2015 S-DANN trains a domain classifier to distinguish between per-class source and target box features and optimizes domain adversarial loss that updates the feature extractor to fool this domain classifier between cross-domain box features corresponding to the same class, similar to Chen et al. (0018).
4.4 Main Results
Table 2 summarizes our results. We find:
Combining simulated and labeled real data improves performance. We first confirm that supervised learning using only the target data outperforms the sctting of using only source data (c.g. +20.3 on -inl0O--Ciyyscapes)) Further, across all shifts, supervised baselines that naïvely combine simulated and real data (i.e. mixing and sequential finetuning) outperform training using only the target data. This shows that additional simulated data is helpful. Interestingly, find that sequential finctuning outperforms mixing two of three shifts.
CARE outperforms competing methods for unsupervised DA and few-shot DA. First, we note on Sim10K-+e + Cityscapes, CARE strongly outperforms state-of-the-art methods for unsupervised DA (Khindkar
et al.
2022 that use only labeled source and unlabeled target data (+14.4), and few-shot DA (Wang et al,
2020 that use labeled source and target data fataT++0 +2.9).
For fair comparison, assume that both FDA access the exact same amount of labeled source and target data.

---

Published in Transactions on Machine Learning Research (08/2023)
70.9
70.9
69.6
70.2
Target mAp@50
70
68.1
63.4
Target mAP 25
50
48.5
69.4
46.5
66.3.1
68.1
45
3.33
43.0
6:34
85:2
66.2
40
39.8
69
Target only Seq. Finetuning
19.0
Target only
82.1
35
33.5
334.2
Seq. Finetuning
60
Mixing
Mixing
54.8
30
CARE (Ours)
CARE (Ours)
57.9
26.9
25
25
50
100.0
12.5
25
100.0 o
% of target data
% of target data
Figure 5: Plotting the scaling behavior of various methods using 100% of labeled source data and a varying amount of labeled target data, on -iitta-- (left) and Saascapes-CCitycaapp (right). CARE consistently dominates the baselines across settings, often by significant margins (+3.3 and +9.7 mAP@50 over the next best method with 12.5% of the larget domain labeled).
Method
FRCNN w/ Swin-T
VitDet
Target
44.8
42.6
Mixing
44.9
37.9
CARE (Ours)
53.5
54.9
Table 3: Target mAP@50 for target only training, mixing, and CARE on 8-way Synscapes- CCityscapes adaplation
with i) Paster-RCNy (Ren et al. 2015) with Swin-T (Liu et al. 2021 backbone and ii) ViTDet (Li et al. 2022a
CARE outperforms naïve strategies that combine source and target data. Finally, we find that across all shifts, CARE outperforms stratogics based on based on mixing (Kishore et al.) 2021) (+3.3, +9.5, +4.4 mAP@50) and sequential finetuning (Tremblay et al. 2018) (+1.7, +8.7, +8.3 mAP@50). This suggests that the Sim2Real domain gap barrier to effective mixing, and systematically mitigating it using target labels is beneficial. Most importantly, we outperform each benchmarked supervised extension of UDA all shifts. This result validates the research-practice gap by showing that UDA cannot be casily extended to the practical setting of labeled target data, thereby necessitating CARE in supervised domain adaptation.
Gains with CARE are consistent across target labeling budgets. While we focus on the setting wherein a modest amount of target data is labeled (e.g. 25%-625 images on Cityscapes). in addition to a large amount of labeled source data (10K images from Sim10K) a natural question that arises is the versatility of CARE across varying amounts of labeled target data. To study this, we benchmark the performance of supervised DA strategies we vary the amount of labeled larget data available (and 100% of source data wherever applicable). Figure 5 shows results the Siml0K-+Cityscappe (left) and Synscapes-+ Citysscppss (right). As seen, CARE consistently dominates competing methods across labeling budgets, with gains being particularly significant in the low target data regime (+3.3 and +9.7 mAP@50 over the next best method with 12.5% of the target domain labeled). More generally, we find that methods that combine simulated and real data show strong improvements over target-only training in the low-target data regime and diminishing returns thercafter.
Gains with CARE consistent across backbones. In addition to the standard Faster R-CNN with Re backbone that employ across detection experiments for fair comparison to prior work (Chen ct al. 2018
Wang et al. 2021 Li et al. 2022b we now test generality by running CARE with two additional
2We note that Sim10K ,Cityscapes, even performance with target-only training saturates higher target label budgets, and it is unclear whether the diminishing returns are due performance saturation rather than the reduced utility of simulation.
10

---

Published in Transactions on Machine Learning Research (08/2023)
Table 4: Ablating CARE on all three shifts. Our method is in gray with the improvement versus mixing in small font.
P(g())|B,C) aligmment
P(C) P(B(O)
mAP@50 (t)
#
rewt.
rewt.
Sim10k
Synscapes
Internal-Sim
1
(Mixing baseline)
64.8
39.0
49.3
2
S-MMD
65.8
40.0
50.6
3
S-DANN
65.3
40.8
49.8
Cycle Consistency
67.2
41.8
50.8
5
None (Mixing baseline)
64.8
46.1
51.8
6
Cycle Consistency
67.2
46.6
52.5
7
Cycle Consistency
68.1 195.1+3.11 48.5+9.5
53.7+4.4
Table 5:
Ablating our proposed conditional reweighting strategies on Synscapes + Cityscapes.
Method
w/o CB
w/ CB
Method
mAP@50
Source
19.2
20.0
P(w, h, x, y[C)
48.5
Target
34.2
40.0
Only y()
46.7
Mixing
39.0
46.1
Only P(w,hIC)
48.3
Seq. FT
39.8
44.9
None
46.6
(a) Ablating P(C) reweighting.
(b) Ablating P(B[C) rewcighting.
architectures: (i) Replacing the ResNet-50 (He al. 2016) backbone in Faster R-CNN (Ren el al. 2015 Swin-T (Liu et al. 20210: Swin is hierarchical vision transformer architecture that uses shifted window attention across non-overlapping local windows which limits computation complexity be linear with respect to image size, which improves efficiency and in turn allows for modeling multi-scale imagery. (fi) VitDet (Li et al. 2022ab: ViTDet employs a plain, non-hierarchical vision transformer backbone (initialized with MAE (He et al. 2022) pretraining) with simple feature pyramid network and standard window attention with a few cross window propagation blocks. Despite its simple design, ViTDet achieves strong detection performance.
Note that both of these make of Vision Transformer Dosovitskiy et al. 2020) backbones, and ViTDet. additionally benefits from self-supervised pretraining with MAF He et al. (2022). We report larget mAP@50 for target-only training, mixing, and CARE on 8-way Synscapes-Ciitssaapes adaptation. As seen, even with recent vision transformer-basee backbones and detectors CARE provides consistent and strong improvements over the baseline (eg. absolute gain of +8.6 with Swin-T, +17 with VitDet, over mixing).
4.5 Ablation study
In Table 4
we ablate the various components of CARE. We find that:
Class-and-box conditional feature alignment is necessary (Rows 2-4 vs. 1). Regardless of the specific feature alignment strategy (i.e. S-MMD, S-DANN, and our proposed cross-domain Cycle Consistency), additional feature alignment improves performance.
We also remark that during model design, tested variations of Cycle Consisteney-based alignment Sim10K->Cityseap by i) conditioning on predicted rather than ground truth class and box coordinates (66.1 m50@ -1.1 compared to our method), and ii) conditioning on predicted box coordinates and ignoring class predictions (64.9 mA P@050, roughly on par with mixing). These two settings yielded 66.1 mAP@ (-1.1 versus Row 4) and 64.9 mAP@50 (-2.3 versus Row 4), respectively.
Class rebalancing with P(C)) reweighting is highly effective (Row vs. 1). Particularly on multi- class source settings (e.g. Synscapes and Internal-Sim), P(C) reweighting considerably boosts performance
11

---

Published in Transactions on Machine Learning Research (08/2023)
A target AP (lower is better)
15
Mixing CARE
14. 14,015
target mAP@O. 0.5:0.95
0.4
source target mixing Seq. FT
10
0.3
7
0.2
Ours
1.4 8.8
1.8 Background Missed
0.1
0.70. Duplicate
Classification
Localization
Cls+Loc
0.0
person
rider
car
truck
bus
train
motorcycle bicycle
Error type
Figure 6: (left) Per-class performance comparison of CARE to baselines Syuscapes Cityesssssissssssss (right) Visualizing change dAP (lower is better) Bolya al. (2020) for errors different types using CARE, over a mixing baseline.
(e.g. by +7.1 mAP@50 Synscapes Sanscapes->Citysceee Cityscapess. Further, Table (a) shows that class balancing naturally improves the baselines as well, due to mAP evaluating classes equally.
Importance reweighting based on estimated P(B[C) statistics further improves performance (Row vs. 6). Finally, we show that our novel proposal of additional class-conditional box reweighting consistently improves performance across all shifts. Table (b) presents results for different formulations of P(BIC). It validates reweighting scheme which decomposes box size with P(w, C[C, and location with P(). Capturing both is better than using only one (+1.8, +0.2 mAP@50) or neither (+1.9 mAP@50).
Our proposed feature alignment objective outperforms prior work. Recall that we propose to align similar rather than dissimilar cross-domain instances belonging to the same class using a cycle consistency objective, unlike prior work that uses cycle confusion (Wang et al., 2021). To validate this choice, we compare Ciiyyscpess adaptation performance with both objectives on top of standard mixing objective. We find that cycle confusion underperforms against our proposed cycle consistency objective by 1.7 (i.e., 65.5 versus 67.2), and consequently adopt cycle consistency throughout. With additional importance reweighting, find that mAP@50 with cycle consistency further improves to 68.1.
CARE is agnostic to the underlying feature alignment strategy. Our primary contribution is not a particular domain adaptation approach, but rather a novel framework designed for supervised domain adaptation, which, unlike prior work, explicitly leverages target labels components conditional feature alignment, class-reweig,ting, and class-conditional box-reweighting, leading strong improvements.
To validate this, we present target mAP@50 for
adaptation by swapping out only the feature alignment strategy from cycle consistency to class-and-box-coone nox-cconditional versions of two popularly used domain adaptations strategics: maximum mean discropancy (MMD) Long et al. (2015) minimization, and domain adversarial learning (DAL) Ganin & Lempitsky 2015 We find that both of these still significantly improve performance over no additional feature alignment 2.2 with MMD, +3.9 with DAL).
4.6
CARE: Fine- Fine-grained performance analysis
Using Synseapes--Citysaapee outperforms baselines in all settings and not just in aggregate.
analyze content-specifie metrics to demonstrate that CARE consistently
CARE improves over baselines on all classes. Fig. 6 (left) studies per-class performance improvements
with our proposed method against baselines. Our method outperforms each baseline for every class.
2020) evaluate specific error types of our mixing baseline and CARE models (lower is better). Fig. 6 (right) shows that CARE reduces classification, localization, and duplicate errors, while slightly worsening joint. dussification++lcalll errors.
Fino-grained error analysis. We use TIDE Bolya
12

---

Published in Transactions on Machine Learning Research (08/2023)
v(w, h C=car)
1.2
A target mAp@O 50.55.0.. .5
w/ P(B|C) reweighting
10.0
car
20
bus
1.1
7.5
height (%)
bike
40
5.0
1.0
60
2.5
0.9
80
0.0
all
100
10a 10
10* 10s
10a 10
0.8
102
0
25
50
75
width (%)
Box area (# pixels)
Figure 7: Visualizing P(w, h[C) reweighting on Synseapes-CCitssappee (left) Visualizing o(w, hlC = car). (right) Visualizing change in mAP after P(w, h|C') reweighting for three categories (car, bus, bike).
CARE improves per-class performance across box sizes. Fig. 7 (left) illustrates bounding box frequency ratio weights (w.. h (C) for the "car" class estimated via the first term of Eq. equation 4 Matching our intuition (see Fig. 2 right), these ratios up-weight target cars sizes that are relatively less frequent in the source domain. Fig. (right) illustrates the change mAP as a result of our reweighing for three categories over boxes of different sizes. Here, reweighing consistently improves mAP and can yield up to +10 mAP improvement for large objects such buses. We these trends to hold for the remaining categories.
Visualizing matching with cycle consistency. Fig. provides qualitative visualization of the matching behavior of our proposed cycle consistency approach, for two pairs of source and larget images. For each example, we estimate the Euclidean distance in feature space between all cross-domain instance pairs in the aligned feature space of our CARE model and visualize the closest pair of car instances for each example. As expected, we find that our method embeds similar-looking cars closer feature space.
5 Discussion
We study supervised Sim2Real adaplation applied to object delection and propose strategy that exploits target labels to explicitly estimate and bridge the sim2real appearance and content gaps. Our method possesses a clear thcoretical intuition and our empirical analyses validate our improvements in every sctting that we tested, for example by boosting mAP@50 by as much as 25%.. Most importantly, this paper tackles a large research-practice gap by bridging the literature on unsupervised and few-shot domain adaptation with an industry-standart practice of combining labeled data from both simulated and real domains. With this, envision renewed future methodological interest in SDA.
Limitations. Despite its strong performance, our method makes few assumptions: First, it requires sufficient labeled data in source and target domains to reliably estimate dataset-level statistics. Further, our formulation assumes conditional independence of box sizes and locations as well as an equivalence between pixel-level and feature-level distributions. We also rely on successful cross-domain alignment. These assumptions may be violated to varying degrees in practice. Further, we do not consider an unlabeled portion of the (arget domain and leave that exploration future work.
Prior work in visual domain adaptation primarily focuses the tasks of image classification, semantic segmentation, and object detection. Of these, we focus on 2D detection as it is the most complex and practically applicable (for autonomous driving applications) of the three. Further, we explicitly address this additional complexity in our approach, e.g. by correcting for the sim2real shift in class-conditional size and location distributions using importance reweighting. However, while in theory such reweighting should be directly extensible related tasks (e.g. 3D object detection) and settings (e.g. general transfer lcarning), such experiments outside the scope of this paper.
13

---

Published in Transactions on Machine Learning Research (08/2023)
References
David Acuna, Jonah Philion, and Sanja Fidler. Towards optimal strategies for training self-driving perception models in simulation. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (cds.). Advances in Newral Information Processing Systems, 2021a. URL httts::/ http:/eeee..e//foouuei/o/22//0 IO21FFY
David Acuna, Guojun Zhang, Mare T. Law, and Sanja Fidler. f-domain adversarial learning: Theory and algorithms. In Marina Meila and Tong Zhang (eds.): Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 66 75. PMLR, 18 24 Jul 2021b.
Daniel Bolya, Sean Foley, James Hays, and Judy Hoffman. Tide: general toolbox for identifying object detection errois. In Computer Aision-ECCi 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pp. 558-573. Springer, 2020.
Prithvijit Chattopadhyay, Kartik Sarangmath, Vivek Vijaykumar, and Judy Hoffman. Pasta: Propor- tional amplitude spectrum training augmentation for syn-to-rcal domain generalization. arXiv preprint arXiv:2212.0097 00999 2022.
Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cmn for object detection in the wild. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 3339 3348, 2018.
Marius Cordts, Mohamed Omran, Scbastian Ramos, Timo Rchfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Gabriela Csurka, Riccardo Volpi, and Boris Chidlovskii. Unsupervised domain adaptation for semantic image segmentation: a comprehensive survey. Foundations and Trends in Computer Graphics and Vision, 2022.
Jeff Donahue, Judy Hoffman, Erik Rodner, Kate Saenko, and Trevor Darrell. Semi-supervised domain adaptation with instance constraints. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 668-675, 2013.
Alexcy Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Representations, 2020.
Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Picrre Sermanct, and Andrew Zisserman. Temporal cycle-consisterncy learning. Proceedings of the IEEE/CVF conference on computer vision and pollern recognition, pp. 1801 1810, 2019.
Yaroslav Ganin and Victor Lempitsky, Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning, pp. 1180 1189, 2015.
Yipeng Gao, Lingxiao Yang, Yummu Huang, Song Xie, Shiyong Li, and Wei-Shi Zheng. Acrofod: An adaptive method for cross-domain few-shot object detection. In Computer Vision-ECCV 2022: 17th Europeon Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIII, pp. 673 690. Springer, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference on computer vision and pottern recognition, pp. 16000 16009, 2022.
Judy Hoffman, Erik Rodner, Jeff Donahue, Trevor Darrell, and Kate Saenko. Efficient learning of domain- invariant image representations. ar preprint Y3011301.390 2013.
14

---

Published in Transactions on Machine Learning Research (08/2023)
Judy Hoffman, Eric Tzeng, Tacsung Park, Jun-Yan Zhu, Phillip Isola, Kate Sacnko, Alexci Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International conference. on machine learning, pp. 1989 1998. Pmlr, 2018.
Matthew Johnson-Roberson, Charles Barto, Rounak Mehta, Sharath Nittur Sridhar, Karl Rosaen, and Ram Vasudevan. Driving in the matrix: Can virtual worlds replace tasks? In 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2017.
annotations for real world
IEEE,
Amlan Kar, Aayush Prakash, Ming- Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, and Sanja Fidler. Meta-sim: Learning to generate synthetic datasets. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 4551-4560, 2019.
Andrej Karpathy. Tesla ai day 2021 -simulation, 2021. URL https://www.yutttt cww com/watch?v=
jo24Fwecy4Mtt-56522
Vaishnavi Khindkar, Chetan Arora, Vineeth N Balasubramanian, Anbumani Subramanian, Rohit Saluja, and CV Jawahar. To miss-attend is to misalign! residual self-attentive feature aligmment for adapting object detectors. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3632-3642, 2022.
Seunghyeon Kim, Jaehoon Choi, Taekyung Kim, and Changick Kim. Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection. In Proceedings of the LEEE/CVE International Conference on Computer Vision, pp. 6092-6101, 2019.
Aman Kishore, Tae Eun Choe, Junghyun Kwon, Minwoo Park, Pengfei Hao, and Akshita Mittel. Synthetic data generation using imitation training. In Proceedings of the IEEE/CVES /CFF International Conference on Computer Vision, pp. 3078 3086, 2021.
Brian Kulis, Kate Saenko, and Trevor Darrell. What you saw is not what you get: Domain adaplation using asymmetric kernel transforms. In CVPR 2011, pp. 1785 1792. IEEE, 2011.
Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In European Conference on Computer Vision, pp. 280-296. Springer, 2022a.
Yu-Jhe Li, Xiaoliang Dai, Chih- Yao Ma, Yen-Cheng Liu, Kan Chen, Bichen Wu, Zijian He, Kris Kitani, and Peter Vajda. Cross-domain adaptive teacher for object detection. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 7581 7581-7590, 2022b.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012 10022, 2021.
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International Conference on Machine Learning, pp. 97-105, 2015.
Mingsheng Long, Zhangjic Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018.
Rafid Mahmood, James Lucas, Jose M Alvarez, Sanja Fidler, and Marc T Law. Optimizing data collection for machine learning. arXiv preprint orXiv:2210.01234, 2022.
Saeid Motiaan, Marco Piecirilli, Donald A Adjeroh, and Gianfranco Doretto. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE international conference on computer vision, pp. 5715-5725, 2017.
Viraj Prablu, Shivam Khare, Docksha Kartik, and Judy Hoffman. Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Compuler Vision, pp. 8558 8567, 2021.
15

---

Published in Transactions on Machine Learning Research (08/2023)
Aayush Prakash, Shaad Boochoon, Mark Brophy, David Acuna, Eric Cameracci, Gavriel State, Omer Shapira, and Stan Birchfield. Structured domain randomization: Bridging the reality gap by context-aware synthetic data. In 2019 International Conference on Robotics and Automation (ICRA), pp. 7249 7255. IEEE, 2019.
Aayush Prakash, Shoubhik Debnath, Jean-Francois Lafleche, Eric Cameracci, Stan Birchfield, Marc T Law, ct al. Sclf-supervised real-to-sim scene generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16044-16054, 2021.
Rindra Ramamonjison, Amin Banitalebi-Dehkord,, Xinyu Kang, Xiaolong Bai, and Yong Zhang. Simrod: A simple adaptation method for robust object detection. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 3570 3579, 2021.
Davis Rempe, Jonah Philion, Leonidas J Guibas, Sanja Fidler, and Or Litany. Generating useful accident- prone driving scenarios via a learned traffic prior. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition, pp. 17305-17315, 2022.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 2015.
Cinjon Resnick, Or Litany, Amlan Kar, Karsten Kreis, James Lucas, Kyunghyun Cho, and Sanja Fidler. Causal scene bert: Improving object detection by searching for challenging groups of data, or preprint (44::222200655. 2022.
Aruni RoyChowdlurr, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, and Erik Learned-Miller. Automatic adaptation of object detectors to new domains using self-training. In Proceedings of the IEEE//VF Conference on Computer Vision and Pattern Recognition, pp. 780-790, 2019.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European conference on computer vision, pp. 213-226. Springer, 2010.
Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE International Conference on Compuler Vision, pp. 8050-8058, 2019a.
Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Strong-weak distribution alignment for adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8956-6965, 2019b.
David W Scott. Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons, 2015.
Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield. Training deep networks with synthetic data: Bridging the reality gap by domain randomization. In Proceedings of the IEEE conference computer vision and pottern recognition workshops, pp. 969-977, 2018.
Yao Vao-Hung Hubert Tsai, Yi-Ren Ych, and Yu-Chiang Frank Wang. Learning cross-domain landmarks for heterogeneous domain adaptation. In Proceedings of the IEEE conference on compuler vision and paltern recognition, pp. 5081-5090, 2016.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. (167-7176, 2017.
Tao Wang, Xiaopeng Zhang, Li Yuan, and Jiashi Feng. Few-shot adaptive faster r-cnn. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 7173-7182, 2019a.
Xiaolong Wang, Allan Jabri, and Alexci A Efros. Learning correspondence from the cycle-consistency of time. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2566-2511 2266,,27, 2019b.
16

---

Published in Transactions on Machine Learning Research (08/2023)
Xin Wang, Thomas Huang, Joseph Gonzalez, Trevor Darrell, and Fisher Yu. Frustratingly simple few-shot object detection. In International Conference on Machine Learning, pp. 9919-9928 PMLR, 2020.
Xin Wang, Thomas E Huang, Benlin Liu, Fisher Yu, Xiaolong Wang, Joseph Gonzalez, and Trevor Darrell. Robust object detection via instanco-level temporal cycle confusion. In Proceedings of the TEEE/CVF International Conference on Computer Vision, pp. 9143 9152, 2021.
Magnus Wrenninge and Jonas Unger. Synscapes: A photorcalistic synthetic dataset for street scene parsing. arviv preprinl 1rxiv:880.000 08700, 2018.
Ting Yao, Yingwei Pan, Chong- Wah Ngo, Houqiang Li, and Tao Mei. Semi-supervised domain adaptation with subspace learning for visual recognition. In CVPR, 2015.
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint wrXi:1110.044877 2019.
Yabin Zhang, Bin Deng, Hui Tang, Lei Zhang, and Kui Jía. Unsupervised multi-class domain adaptation: Theory, algorithms, and practice. IEEE Transoctions Pattern Analysis and Machine Intelligence, 44(5): 2775 2792, 2020.
Chaoliang Zhong, Jie Wang, Cheng Feng, Ying Zhang, Jun Sun, and Yasuto Yokota. Pica: Point-wise instance and centroid aligument based few-shot domain adaptive object detection with loose annotations. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pp. 2329-2222222 2338, 2022.
Xinge Zhu, Jiangmiao Pang, Ceyuan Yang, Jianping Shi, and Dahua Lin. Adapting object detectors via selective cross-domain aligument. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 687-696, 2019.
A Appendix
A.1 Varying mixing ratio
The mixing ratio controls the proportion of source and target data that are contained within a mini-batch of fixed size. In Fig. Sa we vary the mixing ratio of real: simulated data and measure the subsequent performance on the Cityscapes test set. We use all source and larget data for these experiments. We observed that the performance is fairly stable beyond ratio of 50% and, for simplicity, we adopt this mixing ratio for all experiments unless otherwise stated.
log Ps(w, h|C car)
log Pr(w, hlc car)
72
target mAP@50
69.9
2
70
69.4
60.4
height (%)
68.4
40
68
60
60
66,7
66
80
80
12.5
25
37.5
50
62.5
75
87.5 100
0
25
50
25
50
75
Within-batch target ratio (%)
width (%)
width (%)
(a) Sim10K -Cttysapsss (100% target data): (b) Visualizing log PDF values KDE densities fitted bounding Varying aithin-batel real: sim ratio for mixing. box size on Synscapes >Cityscapes for the "car" class.
Figure 8: Additional empirical analysis of CARE.
17

---

Published in Transactions on Machine Learning Research (08/2023)
A.2 Additional details on importance reweighting
When applying bounding-box importance reweighting we introduce a smoothing mechanism to ensure bounded loss values and thresholding to handle areas with low target support. Specifically, we compute:
ao
Pr(BIc) Cs
+3
if Pr
CBIC)
1.0
otherwise
where a are scaling paramcters (that we set 20, -9, effectively bounding loss weights between and 11). denoles the sigmoid operator, and is probability threshold that set to 0.1. For boxes with very small target support, simply set weights to floor value of 1.0.
A.3 Additional CARE analysis
Visualizing KDE estimates. In Figure we visualize log PDF values from KDE estimates fit to bounding box width and heights on both the source (Synscapes) and target (Cityscapes) domains. As seen, the KDE is able to capture the difference in box size distributions across domains: car class sizes vary significantly It is intuitive that with appropriate importance reweighting for source boxes as proposed in Sec. 3 we can improve overall performance across categories.
more in the target domain, consistent with our observation in Fig. 2
Complexity analysis. CARE does not add a substantial performance overhead over baselines. In fact, it is more efficient than sequential finetuning: we run CARE for 10k iterations, while sequential finetuning requires 20k (10k iterations for pretraining source and 10k for finetuning target). On the other hand, CARE and mixing have similar computational efficiency, with CARE being slightly slower due to additional class-conditional feature aligmment, which necessitates learning additional shallow encoder. Finally, we note that the per-class weights and class-conditional box weights are precomputed oflline and so do not add a significant training- taining-time time overhead. Concretely, we find that training CARE for -OKK-KKKyyaappeeeee adaptation on 8 NVIDIA V100 GPUs with a per-GPU batch size of 4 takes approximately 3.5 hours to complete, as compared to 3 hours and 10 minutes for mixing.
A.4 Additional Implementation Details
Faster-RCNN training recipe. For data augmentation, we only use random flipping (with 50% probability). For optimization, we use SGD with a learning rate of 0.01, momentum of 0.9, and weight decay of 10 We train for 10k iterations and use lincar learning rate warmup for 500 iterations with warmup ratio of decay the learning rate by factor of 10 at 6k and 8k
0.001. Following prior work (Wang et al. 2021 iterations each.
Selecting A. We follow prior work (Prabhu et al. 2021 and tune loss weights (A in Equation such that the feature alignment loss of the same order of magnitude (after scaling) as the detection losses; we accordingly set A = 0.1 across experiments.
18


Reviews:
[{"review_id": "CEBIRBnpcW", "reviewer": "Reviewer_YPSN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The investigated problem is well motivated, where labeled real and synthetic data are available in practice.\n2. The appearance and content gaps are clearly presented and tackled with careful designs.\n3. The proposed method is supported by theoretical analyses. \n4. Improved performance over competitors, including UDA, FDA, and seq FT.\n\nWeakness:\n1. The proposed method is validated on a weak backbone of Faster-RCNN. It is interesting to present whether the proposed method is worked with more advanced backbone models.\n2. Some missing references on DA, e.g., supervised DA is also investigated in (a), and (b) summarized various adversarial feature alignment methods.\n    (a) Unified deep supervised domain adaptation and generalization, ICCV\n    (b) Unsupervised multi-class domain adaptation: Theory, algorithms, and practice, TPAMI", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the supervised sim2real DA problem with the 2D object detection task. To tackle this problem, the authors proposed a CARE method by minimizing both the appearance gaps and content gaps simultaneously. Specifically, the appearance gap is minimized by conditionally aligning the intermediate representations with ground-truth labels, while a reweighting scheme is proposed to overcome the content gap. Additionally, theoretical insights are provided. Experiments are conducted on the cityscape dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The effectiveness of the method can be validated more clearly with more advanced detection architecture. \n2. More related references are suggested.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PYqX9QfHbx", "reviewer": "Reviewer_jEpn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n* Clear explanation of the motivations behind the work and on why it is an interesting task to explore from an industrial perspective.\n\n* I appreciated the effort in proposing meaningful baselines to compare against. Since the authors are proposing a new protocol for evaluation they had not only to come up with the evaluation of their own solution but also of several baselines which require adapting other proposed methods.\n\n## Weakness\n\na. **Presentation**: I found section 3.3.1 not clear and the notation confusing. In particular it is not clear to me how the soft match is defined. If I understood correctly $\\hat{f}^j_T$ is supposed to be the reconstruction of $f^j_S$ as a weighted combination of all features from domain $T$. If that’s correct shouldn’t it have the subscript $S$? I found the notation quite confusing.\nAlso what’s the reason to apply this alignment only with the cycle source->target->source and not also target->source->target? In your setting both domains should have enough annotated samples to compute this regularization in a meaningful way. \n\nb. **Experimental validation**: The authors picked a quite old architecture for detection (FasterRCNN with a Resnet50) and an unusual target dataset (Cityscape, which is usually benchmarked in the context of segmentation). In these settings it is unclear if the performance achieved is competitive in absolute terms or if the baselines selected are weak. In particular it could be interesting to either benchmark on a dataset which already provides several submitted methods (e.g., KITTI) or to take more modern state of the art architectures for detection and train them to the best of their possibility on the full target datasets (e.g., DETR [1] or ViDT [2]). \n\nc. **Lack of details on the experimental evaluation**: from the current paper it is not clear why depending on the choice of source dataset the authors decide to test only on a subset of the classes in the target dataset. Is it a choice imposed by the datasets having only a subset of classes in common? Also other details missing regard the training recipe used to train the detection model. In particular it will be important to discuss the data augmentation strategy used as it has been proven to have a great impact in obtaining stronger baselines.\n\nd. **Experimental settings violates the motivations of the paper**: The motivation of the paper claims the need to study realistic settings closer to industry standards, however in the end the author's main evaluation (Tab. 2) report results using only 25% of the available target data, while Fig. 5 shows that when using 100% of the target data often the gap shrink significantly. Note that according to the motivation of the paper the 100% target data setting should be the focus of the work. Also I found it a bit hard to justify the need of a loss rebalancing to minimize the content gap between synthetic and real dataset. In the context of the paper it is needed because the authors are studying datasets proposed by independent authors in academia, but in real setting the synthetic dataset can be generate to cover the same distribution (or a distribution as close as possible) of the target dataset, therefore maybe making the need of a content alignment less important.\n\n## References\n1. Carion, Nicolas, et al. \"End-to-end object detection with transformers.\" European conference on computer vision. Cham: Springer International Publishing, 2020.\n2. Song, Hwanjun, et al. \"Vidt: An efficient and effective fully transformer-based object detector.\" arXiv preprint arXiv:2110.03921 (2021).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes to study a new setting for domain adaptation in which the task is to maximize performance on a target domain having available some annotated data from the target domain and a lot  more from a different source domain. This is similar to the semi-supervised domain adaptation methods with the main difference being the availability of many annotated images from the target domain rather than only a few. The authors claim that these settings are relevant in industry especially in the context of autonomous driving where many real (target domain) and synthetic (source domain) data are available. The practical task studied by the authors is object detection. Their proposal to fight domain shift is to use cross domain feature matching to fight appearance shift between the two domains and loss rebalancing to fight content shifts by trying to matcn labels and box distributions. With these two modifications their proposal is able to significantly improve performance wrt a baseline trained only on the target domain as well as against algorithms for unsupervised or semi-supervised domain adaptation. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please explain better the part criticized in weakness a-c-d and if possible improve the experimental validation as mentioned in weakness b.", "broader_impact_concerns": "No concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zdUvu6X2LF", "reviewer": "Reviewer_Hv1a", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is very well written and easy to follow. The authors motivate their object of study well and their approach is clear. On multiple occasions, the authors provide intuitions for their design choices which I appreciate. The Figures and Tables are very clear and well designed. The paper is well structured.\n\nI appreciate that the authors include simple baselines which are not necessarily native to domain adaptation, such as mixing or sequential finetuning of the model. \n\nThe results are presented in a very detailed way. I especially appreciate Figure 5 where different ratios of the available target data are studied. I also appreciate Fig. 6, where the authors evaluate per-class performance, thereby checking that the improvements due to CARE are consistent across all classes.\n\nOverall, the experiments have been performed carefully and the presented analyses are solid and detailed. I have some questions and suggestions for the authors which I outlined below, but did not discover any major flaws in this paper.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors study the task of Sim2Real Domain Adaptation in a novel scenario where limited labeled target data is available. This is a reasonable setting in sensitive real-world applications. The authors use simulated, machine-labeled source data, and real, human-labeled target data, and aim to combine both in the most effective way. They characterize the domain gap between the source and target domains via the appearance gap and the content gap which they treat and address separately. The authors show superior results on all studied datasets compared to the baseline methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "How was lambda in Eq. 2 set? Are the results sensitive to this hyperparameter? Please specify the model selection process here.\n\nCould you please provide a complexity analysis of your method compared to other methods? For example, sequential finetuning likely requires more gradient steps because the model is first pretrained on the source data and then finetuned on the target data, so CARE should be more efficient? How efficient is CARE compared to the other baselines?\n\nThe authors study Supervised Domain Adaptation which, as they write, has not been studied much in the field of general Domain Adaptation since Unsupervised Domain Adaptation has been more popular. The setting they study is actually closer to transfer learning where a base model is pretrained on some large dataset (e.g. ImageNet) and then finetuned on a target task, e.g. on VTAB (https://ai.googleblog.com/2019/11/the-visual-task-adaptation-benchmark.html). Could you please establish a connection to transfer learning in the related work section? I think that testing CARE in a transfer learning setting is out of scope for this work since the paper already provides very detailed results in the studied setting of SDA. I think the presented results are definitely sufficient for this publication since the claims are well-supported by the presented evidence. Do the authors think that testing their approach in transfer learning could be interesting for future work? If so, could they maybe add it in the discussion section? My thought is that if this method has not been studied in transfer learning before, then maybe this paper would also be interesting to researchers working on transfer learning.  The way that the paper is currently written, I think the target audience will be researchers working on domain adaptation (but not transfer learning).\n\nQuestion rather than a requested change: You write in the discussion: “First, it requires sufficient labeled data in source and target domains to reliably estimate dataset-level statistics.” Here, you refer to estimating the content and appearance statistics I assume. Could you please comment on aligning BatchNorm statistics between source and target and how CARE helps there? It has been shown previously (e.g. in [1]) that having separate BN layers per domain can be beneficial. Do you think CARE is implicitly aligning BN statistics of the source and target domains on the class / instance level?\n\n[1] Rebuffi et al. “Learning multiple visual domains with residual adapters”\n\n\nRelated to the point above, could you formulate a stronger limitation for CARE (or maybe a minimal set of necessary requirements), for example if the source and target datasets are sufficiently different? In order to successfully reweigh the class probabilities, do all classes need to be present in both domains or can one domain have classes not present in the other? Do the target classes need to be a subset of the source classes? Is there a similar limitation for the cross-domain cycle consistency objective? Again, going back to transfer learning, using a pretrained model is almost always beneficial over training from scratch on a small labeled target dataset even if the tasks are very different. Is this the same for CARE?\n", "broader_impact_concerns": "I do not have concerns with regards to ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All three reviewers found that the authors have solved their concerns and agree that the target problem is interesting and practical. The authors shall include the additional results (the results with two backbone architectures, ResNet-50+Swin-T and VitDet, and the results on Synscapes) in the rebuttal period and the requested information by reviewer Hv1a in the camera ready version, including
(a) a detailed complexity analysis of CARE vs the other baselines；
b) a discussion comparing the studied setting to transfer learning such e.g. fine-tuning of a pretrained model on VTAB；
c) details on how hyperparameters (such as lambda in eq. 2) were set).
Decision: Accept (poster)