Align and Distill: Unifying and Improving Domain Adaptive Object Detection
Authors: Justin Kay
Timm Haucke
Suzanne Stathatos
Siqi Deng
Erik Young
Pietro Perona
Sara Beery
Grant Van Horn
Venue: Accepted by TMLR ()

Abstract: Object detectors often perform poorly on data that differs from their training set. Domain adaptive object detection (DAOD) methods have recently demonstrated strong results on addressing this challenge. Unfortunately, we identify systemic benchmarking pitfalls that call past results into question and hamper further progress: (a) Overestimation of performance due to underpowered baselines, (b) Inconsistent implementation practices preventing transparent comparisons of methods, and (c) Lack of generality due to outdated backbones and lack of diversity in benchmarks. We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, increasing the diversity of available DAOD benchmarks, and (4) A new method, ALDI++, that achieves state-of-the-art results by a large margin. ALDI++ outperforms the previous state-of-the-art by +3.5 AP50 on Cityscapes $\rightarrow$ Foggy Cityscapes, +5.7 AP50 on Sim10k $\rightarrow$ Cityscapes (where ours is the only method to outperform a fair baseline), and +0.6 AP50 on CFC-DAOD. ALDI and ALDI++ are architecture-agnostic, setting a new state-of-the-art for YOLO and DETR-based DAOD as well without additional hyperparameter tuning. Our framework, dataset, and method offer a critical reset for DAOD and provide a strong foundation for future research.

Published in Transactions on Machine Learning Research (03/2025)
Align and Distill: Unifying and Improving Domain Adaptive Object Detection
Justin Kay', Timm Haucke', Suzanne Stathatos', Siqi Deng" Erik Young', Pietro Perona 2.3, Sara Beery!.-, Grant Van Horn
*MIT, 2 Calttec, 9 AWS, Skaait Fisheries Enhancement Group, $ UMass Amherst
kayj@mit edu
"qqual advising contribution, Work done outside AWS
Reviewed on OpenReview: https: // openreview. net/ forum? id= SSXSr2945R
Abstract
Object detectors often perform poorly on data that differs from their training set. Domain adaptive object detection (DAOD) methods have recently demonstrated strong results on addressing this challenge. Unfortunately, we identify systemic benchmarking pitffalls that call past results into question and hamper further progress: (a) Overestimation perfor- mance due underpowered basclines, (b) Inconsistent implementation practices preventing transparent comparisons of methods, and (c) Lack of generality due to outdated backbones and lack of diversity in benchmarks. We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, increasing the diversity of available DAOD bench- marks, and (4) A new method, ALDI++, that achieves stale-of-the-arl results by a large margin. ALDI++ outperforms the previous thet--of- by +3.5 AP50 on Cityscapes - Foggy Cityscapes, +5.7 AP50 Sim10k Cityscapes (where ours is the only method to outperform fair baseline), and +0.6 AP50 on CFC-DAOD. ALDI and +LDI+ are architecturee-gnosiio setting a new state-of-the-art for YOLO and DETR-based DAOD as well without additional hyperparameter tuming. Our framework datase F and method offer a critical reset for DAOD and provide a strong foundation for future rescarch.
1 Introduction
The challenge of DAOD. Modern object detector performance, though excellent across many bench- marks [Lin al.) 2014 Weinstein al. 2021b a Bondi et al. 2018 Schneider et al. 2018 Rodriguez et al. 2011), often severely degrades when test data exhibits a distribution shift with respect to training data (Oza) et al. 2023 For instance, detectors do not generalize well when deployed in new environments in envi- ronmental monitoring applications (Kay et al. 2022 Weinstein et al. 2021b). Similarly, models in medical applications perform poorly when deployed in different hospitals or on different hardware than they were trained (Xue el al. 2023 Guan & Liu (202)). Unfortunately, in real-workd applications it is often difficult, expensive, or cine-consuming to collect the additional annotations needed to address such distribution shifts in supervisod manner. An appealing option these scenarios is unsupervised domain adaptáve object de- lection (DAOD), which attempts to improve detection performance when moving from "source" domain (used for training) to "target" domain (used for testing) (Koh et al. 2021 Kalluri et al. 2023) without the use of target-domain supervision.
The current paradigm. The research community has established a set of standard benchmark datasets and methodologies that capture the deployment challenges motivating DAOD. Benchmarks consist of labeled data that divided into two sets: a source and a target, each originating from different domains. DAOD methods are trained with source-domain images and labels, as in traditional supervised learning, and have access to unlabeled target domain images. The target-domain labels not available for training.
guithub.comm//nssttnaay/lll

---

Published in Transactions on Machine Learning Research (03/2025)
To measure DAOD methods' performance, researchers use source-only models and oracle models points of reference. Source-only models-sometingg sometimes also referred to as baselines- are trained with source-domain data only, representing a lower bound for performance without domain adaptation. Oracle models are trained with supervised target-domain data, representing a fully-supervised upper bound. The goal in DAOD is to close the gap between source-only and oracle performance without largel-domain supervision.
Impediments to progress. Recently-published results indicate DAOD is exceptionally effective, doubling the performance of source-only models and even outperforming fully-supervised oracles (Li et al., 2022b;
Chen et al. 2022 Cao et al.) 2023). However, upon close examination we discover problems with current benchmarking practices that call these results into question:
Pl: Improperly constructed source-only and oracle models, leading to overestimation of performance gains. We find that source-only and oracle models are consistently constructed in a way that does not properly isolate domain-ddaptation-spccdif components, leading to misattribution of performance improvements. We show that when source-only and oracle models are fairly constructed i.e. use the same architecture and training settings as DAOD methods existing methods outperform oracles and many methods do not even outperform source-only models (Fig. D in stark contrast to claims made by recent work. These results mean we do not have an accurate mcasure of the efficacy of DAOD.
P2: Inconsistent implementation practices preventing transparent comparisons of methods. We find existing DAOD methods are built using a variety of different object detection libraries with inconsistent training settings, making difficult to determine whether performance improvements come from new DAOD methods or simply improved hyperparameters. We find that tweaking these ipperparameeerswwhoore differ between methods yet are not reported in papers-can lead to a larger change in performance than the proposed methods themselves (see Section 6.3), thus we cannot take reported advancements at face value. Without the ability to make fair comparisons we cannot transparently evaluate contributions nor make principled methodological progress.
values often
P3: (a) Lack of diverse benchmarks and (b) outdated backbone architectures, leading to overestimation of methods' generality, DAOD benchmarks have focused largely on urban driving scenarios with synthetic distribution shifts (Sakaridis et al.) 2018 Jobeson- et al.] 2016 and methods continue to use outdated detector backbones for comparison with prior work (Chen et al. 2018). We show that in fact the manking of methods changes across benchmarks and archilectures, revealing that published results may be uninformative for practitioners using modern architectures and real-world data.
A critical reset for DAOD research. DAOD has the potential for impact in a range of real-world applications, but these systemic benchmarking fitlalls impede progress. We aim to address these problems and lay a solid foundation for future progress in DAOD with the following contributions:
1. Align and Distill (ALDI), unified benchmarking and implementation framework for DAOD, In order to enable fair comparisons, we first identify key themes in prior work (Section 2 and unify common components into a single state-of-the-art framework, ALDI (Section 30. ALDI facilitates detailed study of prior art and streamlined implementation of new methods, supporting future research.
2. A fair and modern training protocol for DAOD methods, enabled by ALDI. We provide quantitative evidence of the benchmarking pitfalls we identify and propose an updated training and evaluation protocol to address them (Section 6.1 This enables us to set more realistic and challenging targets for the DAOD community and perform the first fair comparison of prior work in DAOD (Section 6.2
3. A new benchmark dataset, CFC-DAOD, sourced from a real-worle adaptation challenge in environmental monitoring (Section 5). CFC-DAOD increases the diversity of DAOD benchmarks and is notably larger than existing options. We show that the ranking of methods changes across different benchmarks (Section 6.2 thus the community will benefit from an additional point of comparison.
4. A new method, ALDI++, that achieves state-of-the-art results by large margin. Using the same model scttings across all benchmarks, ALDI++ outperforms the previous stato-of-the-art by +3.5 AP50 Cityscapes + Foggy Cityscapes, +5.7 AP50 on Sim10k + Cityscapes (where ours is the only method to outperform a fair source-only model), and +2.0 AP50 on CFC Kenai + Channel.
2

---

Published in Transactions on Machine Learning Research (03/2025)
UMT
70
Oracle
AargetAPPP AP50
SADA
Source-only
PT
60
Cityscapes Foggy Cityscapes
Target
MIC
AT
50
ALDI+
E In 100
40
Prior art
Prior art w/ ALDI
Ours
Figure 1: ALDI provides unified framework for fair comparison of domain adaptive object detection (DAOD) methods, and ALDI+ achieves state-of-the-art performance. We show: (1) Inconsistent implementation practices give the appearance of steady progress in DAOD (left bars (Deng
ct al. 2021 Chen et al.) 2021 2022 Hoyer et al. 2023 Li et al. 2022b)); reimplementation and fair com-
parison with ALDI shows less difference between methods than previously reported (middle bars); (2) A fairly constructed source-only model (blue line) outperforms many existing DAOD methods, indicating less progress has been made than previously reported; and proper oracle (orange line) outperforms all existing methods, in contrast to previously-publitte achieves state-of-the-arl performance DAOD benchmarks such Cityscapes + Foggy Cityscapes and is complementary to ongoing advances in object detection like VitDet [i et al. 2022ah.
results; and (3) Our proposed method ALDI+ (green bars)
2
Related Work
Our work concerns domain adaptive 2D object detection (DAOD). Two methodological themes have domi- nated recent DAOD research: feature alignment and self-training senmining/sslf-fiee cief-ddistilaation. We first give an overview of these themes and previous efforts to combine them, and then use commonalities to motivate our unified framework, Align and Distill, in Section
Feature alignment in DAOD. Feature alignment methods aim to make target-domain data "look like" source-domain data, roducing the magnitude of the distribution shift. The most common approach utilizes
adversarial learning objective to align the feature spaces of source and larget data (Ganin & Lempitsky 2015
Chen et al.) 2021 2018 Zhu et al.) 2019). Faster R-CNN in the Wild (Chen et al. 2018) utilizes adversarial
networks at the image and instance level. SADA (Chn et al.,
2021 extends this to multiple adversarial networks at different feature levels. Other approaches propose mining for discriminative regions (Zhu el al.
2019), weighting local and global features differently (Saito et al., 2019), incorporating uncertainty (Nguyen et al. 2020), and using attention networks (Vs et al.] 2021). Alignment at the pixel level has also been proposed using image-to-image translation techniques to modify input images directly (Deng et al. 2021
in DAOD. Self-training methods use "teacher" model to predict pseudo- labels on target-domain data that are then used as training targets for a "student" model. Self-training can be seen as a type of self- diffddistillation (Pham et al. 2022 Caron et al. 2021), which is a special case of knowledge distillation (Hinton et al. 20155: Chen et al. 2017] where the teacher and student models share the same architecture. Most recent self self-training approaches in DAOD are based on the Mcan Teacher (Tarvainen)
buffsrainiingssfff-disilllllll
& Valpola, 2017) framework, in which the teacher model updated as an exponential moving average (EMA) of the student model's parameters. Extensions to Mean Teacher for DAOD include: MTOR, which utilizes graph structure to enforce student-teacher feature consistency (Cai et al. 2019 Probabilistic Teacher (PT), which uses probabilistae localization prediction and soft distillation losses (Chen et al. 2022), and Contrastive
Mean Teacher (CMT), which uses MoCo (He et al. 2020 for student-teacher consistency [Cao el. al. 2023
Combining feature alignment and sclf-training. Several approaches utilize both feature alignment and self self-training/ self- distillation, motivating our unified framework. Unbiased Mean Teacher (UMT) (Deng al., mean teacher in combination with image-to-image translation to align source and target data
2021
3

---

Published in Transactions on Machine Learning Research (03/2025)
the pixel level. Adaptive Teacher (AT) (uue et al 2023) uses mean teacher with image-level discriminator network. Masked Image Consistency (MIC) (Hoyer et al. 2023) uses mean teacher, SADA, and masking augmentation to enforce teacher-student consistency. Because these methods were implemented in different codebases using different training recipes and hyperparameter settings, it is unclear which contributions are most, effective and to what exlent feature alignment and sell-training are complementary. We address these issues by reimplementing these approaches the ALDI framework and perform fair comparisons and ablation studies in Section G
DAOD implementations. There are two components to object detector design: the detection archi- tecture (e.g. Faster R-CNN Ren et al. (2015), YOLO Redmon et al. (2016). DETR Carion et al. (2020)) and the backbone (e.g. VGG Simonyan & Zisserman (2014), ResNet, He et al. (0116), ViT Dosovitskiy et al. (2020)) Current toate-of-the-arte methods in DAOD predominantly use Faster R-CNN architectures. DOAD
methods for YOLO and DETR backbones have recently received some attention [Zhou et al. (2023): Yu et al. (2022;: Jia et al. (2023), but have yet to surpass Faster R-CNN-basod methods' performance. For this reason, our main experiments also utilize the Faster R-CNN architecture. Existing methods differ in their choice of backbone, making comparisons difficult; we address this by consistently utilizing ResNet-50 backbones for all experiments and in our re-implementations of prior work. However, the ALDI framework is architecture and backbone agnostic, and we provide additional experiments using YOLO and DETR architectures, as well as ViT and Con Liu et al. (2022) backbones.
DAOD datasets. Cityscapes (CS) + Foggy Cityscapes (FCS) (Cordts et al. 2016: Sakaridis et al., 2018 is a popular DAOD benchmark that emulates domain shift caused by changes in weather in urban driving scenarios. The dataset contains eight vehiele and person classos. Sim10k CS (Johnson-Roberson et al. 2016) poses Rim2Real challenge, adapting from video game imagery to real-world imagery. The benchmark focuses on a single class, "car". Other common tasks include adapting from real imagery in PascalVOC (Ev- eringham et al. 2010) to clip art and watercolor imagery (Inoue et al. 2018). We report results on CS + FCS and Sim10k + CS due to their widespread popularity in the DAOD litcrature and focus on real appli- cations. We note that existing benchmarks reflect a relatively narrow set of potential DAOD applications. To study whether methods generalize outside of urban driving scenarios, in Section 5
we introduce a novel dataset sourced from real-world adaptation challenge in environmental monitoring, where imagery is much different from existing benchmarks.
3 Align and Distill (ALDI): Unifying DAOD
We first introduce Align and Distill (ALDI), a new benchmarking and implementation framework for DAOD. ALDI unifies feature alignment and self-distillation approaches in a common framework, enabling fair com- parisons and addressing P2. Inconsistent implementation practices, while also providing the foundation for development of new method ALDI++ that achieves state-the-art performance (Section 4 Section 6.2
The framework is visualized in Fig. 2 All components are ablated in Section 6.3
Data. DAOD involves two datasets: a labeled source dataset Xare and an unlabeled target datasct Xigt- Each training step, minibatch of size B is constructed containing both Bare source images and Begt target images, B = Br + Begt.
Models. ALDI is designed as a setudent-teacherr framework to facilitate algorithms utilizing self- training/nnnnn self- distillation. When enabled, both a student model ftu and a teacher model Pech are initialized with the same weights, typically obtained through supervised pretraining on ImageNet or X Pretraining on X is often referred to "burn-in." The student is trained via backpropagation, while the teacher's weights are updated each training step to be the EMA of the student's weights (Tarvainen & Valpola| 2017), d.e. buch = aftch + +1 (1- amm with a E [0, 1]. After training, we keep 0tch and discard Ostu- Algorithms that do not use self-training, sattraannnnnssel-ddsstlllllee /esss/slilisssilaatiio (e.g. SADA (Chen et al. 2021 simply disable Ozeh-
In this paper we focus predominantly on two-stage detectors based on Faster R-CNN (en et al. 2015) as they are currently the statc-of- satte-of-thc-art DAOD, though we note that our framework is architactureagnnotii and also supports YOLO and based detectors. We provide additional YOLO and DETR results in A Appendix| A.1
4

---

Published in Transactions on Machine Learning Research (03/2025)
Lesup
Liuuent
ALDI module
Example settings
Vrro
Pare
Plyt
Pryt
Network init.
InsggeNet (Chen
Dong et
2021
Hover
2023
FsG
Oui Buch
Source-donann burn-1n
Chen
2022
Leglign
Tares
Flip and scale, SimCLR angs
Chen et al.
2022
et al 2022b
IIMA
Teacher Biok
Tigs
Image masking (Hoyer al.
2023
Student
Bate
Bare Bugt
1:1
Chen al.
2021
Deny,
202
Hoyer al.
2023
2:1
Cheu 0t m.
2024
m..
A1226
T-Teat
Pseudo-label Deng al.
2021
Hover et
2023
Li et al. 2022b]
Knowledge distilaatonn
Source
Target
Adversarial
[Li et al.
20226
Chen ot al
2021
Hoyer et al. 2023
Lation
Imagr-to--nne ionoo--iieee tramslation
Dong es a.,
2021
Figure 2: (Left) The ALDI framework. Each training step (moving left to right and bottom to top): (1) Sample Bre labeled source images Farci transform by ~~~ Tacci pass to student; compute supervised loss Lup using ground-truth labels Yere- (2) Sample Begt unlabeled target images Pigti transform by t to Tigti pass to student to get preds Ptat- Compute alignment objectives Lalign using Tarc and Tegt- (3) Pass same unlabeled target data Figt, weakly transformed, to teacher; postprocess to obtain teacher predictions Pigt. Compute distillation loss Tdiisall between teacher and student predictions. Use stop gradient (SG) on teacher model; update teacher the EMA of student's weights. (Right) Example settings for each component of ALDI. ALDI supports a range of existing methods off-the-shell while providing a general implementation framework for new methods.
Training involves one or more of the following three objectives. We note that each objective is optional in order to support a range of algorithmic approaches.
1. Supervised training with source data. For each labeled source sample Taro. we apply a transfor- mation ~ Tare, where Taee is the set of possible sourco-domain transformations. The transformed sample is passed through the student model to compute the supervised loss Loup given the ground truth targets y/rc.
1 Bare
Lsup
Σ
L (fstu (f(Iarc,))) Varc.e)
(1)
Barc
where ((5-) are standard object detection loss functions, e.g. those of Faster R-CNN (Ren et al.
2015
2. Self-distillation with target data. For each unlabeled target sample Ztgt. we transform the input using f ~ Tweak (a set of weak transfornations) for the teacher model and t Tigt (stronger transformations) for the student model. The teacher's predictions Pegt,s serve distillation largets for the student's 's predictions Pigt.i, and compute distillation loss Ldistil:
Bust Caistill (Pogt.t, Pegu,s)
Pegt,i Orchh(f(ee (f(
(2)
Ptgt.i (stu(t(2 (t(Fggt)))
(3) Edistill
(4)
Begt
where teacher outputs Pigt.i are postprocossod to be cither soft (c.g., logits or softmax outputs) or hard (c.g., thresholded pseudo-label) targets and the choice of distillation loss method-specifie. This formulation unifies different distillation techniques into common objective, supporting a range of approaches.
3. Feature alignment. The source samples Tar, and target samples Figt, are optionally "aligned" using alignment objective Latign that enforces invariance across domains at either the image or feature level. This formulation is general; however, in this paper, we focus two common alignment losses: domain-adversariil training and image-to-image alignment.
Domain adversaraal training (í.e. DANN Gaani Lempitsky 2015 trains a domain classifier D to distin- guish between source and target features, while the feature extractor aims to confuse D:
B
Caliga,DANN = B
Σ i=1
Wyom, log(D(((z:)) + (1 Jdom, log(1 D(f(x:)))
(5)
5

---

Published in Transactions on Machine Learning Research (03/2025)
where ydom, is the domain label (source 0, target 1) and ((z,) is a feature representation of zę.
image-to-image alignment instead pursues domain invariance in the pixel space. Given image-to-image generative models Gere, Gug (e.g., a CycleGAN (Zhu et al.) (017)), images are "translated" (a pixel-level transformation) from the source domain to the target domain and vice versa. We then obtain Zigt-like, = Gare (Earc,t), like- Gigt (Figt.s), and substitute into Eq. 1 Eq- (2), and Eq. (3).
Unification of prior work. We demonstrate the generality of our framework by reimplementing five recently- recently-proposed methods on top of ALDI for fair comparison: UMT (Deng et al. 2021). SADA (Chen) et al. 2021), PT (Chen et al. 2022). MTC (Hoyer et al. 2023), and AT (Li et al. 2022b). We enumerate the
settings required to reproduce each method in Appendix
4 ALDI++: Improving DAOD
We next propose a set of simple but effective enhancements to the Align and Distill approach. We call the resulting method ALDI++. We show Section 6.2 that these enhancements lead to stale-of-the-ar results, and ablate each component in Section 6.3
1. Robust burn-in. A key challenge in student-teacher methods is improving target-domain pscudo-label quality. We point out that pseudo-label quality in the early stages of self-training is largely determined by the out-of- distribbitntii (OOD) gemeralization capabilities of the imitial teacher model 0init, and thus propose a pro-training ("burn-in") stratogy aimod at improving OOD generalization before self-training.
We add strong data angmentations including random resizing. color jitter, and cutout (DeVries Taylor 2017 Chen al., 2020), and keep an EMA copy of the model during burn-in, two strategies that have
previously been shown to improve OOD generalization MMorales-Brotone et al. 2024 |rpit al. 2022 Gao
et al. 2022,, i.c. we pre-train model 0 with the loss from Eq. (1), where t Tate and Loup are still the standard Faster R-CNN losses. Each iteration we update an EMA copy of the model,
gEMA agEMA a))
(6)
with a E 1]. After pre-training, we initialize Pat = Orch = gEMA, We are the first to utilize these strategies for DAOD burn-in, and we show in Section 6.3 that this pro-training stratogy leads to faster convergence time and better results.
2. Multi-task soft distillation. Most prior work utilizes confidence thresholding and max-max suppression to generate "hard" pscudo-labels from teacher predictions Pigt. However in object detection this strategy is sensitive to the confidence threshold, leading to false positive and false negative errors that harm self-training (Kay et al. 2023 RoyChhuyy et al.) 2019). Inspired by the knowledge distillation literature we propose instead using "soft" distillation losses i.e. using teacher prediction scores as targets without us to eliminate the confidence threshold hyperparameter.
Airesholiiggalllwwee
We describe here our approach for two-stage (Faster R-CNN-bascd) object detection. Distillation implemen- fation details for YOLO and DETR architectures can be found in Appendix B We distill each task of Faster Proposal Network localization (rpn) and objectness (obj), and Region-offInteresl Heads localization (roih) and classification (cls) independently. At each stage, the teacher provides distillation targets for the same set of input proposals used by the sandent-i.e. anchors A in the first stage, and student
rpm region proposals Pigt in the second stage:
Pigt roih,cle
oppn,obs stu groih
(A, zitt)
(7)
prpn,obj Orpn,obs (A, Pigt grosh,cls grosh, rpn Pigs
(8)
Pigt
0 (l ppn zege)
(9)
(pigh xzgg))
(10)
At each iteration, student distillation losses Laistall are computed as:
Irpn No An Tpn srp") + > Looy Peo Pogi)
(11)
disttl
Xa Troih (Figh proit) + Ag Tects (rige-
(12)

---

Published in Transactions on Machine Learning Research (03/2025)
Ldistiil
L
rm rosh distall distill
(13)
Where Trpn and Troih are the smooth L1 loss and Tray and Tedss are the cross-entropy loss, and Xo... = by default. See Fig. 2 for a visual depiction, and the appendix for implementation details.
One prior DAOD work, PT (Chen al. 2022), has also used soft distillation losses. Our method addresses two shortcomings: (1) PT requires a custom "Probabilistic R-CNN" architecture for distillation, while our approach is general and can work with any two-stage detector, and (2) PT uses pole as an indirect proxy for distilling ,h0 while our approach distills each task directly.
3. Revisiting DAOD training recipes. We also re-cxaminc common design choices DAOD order establish strong baseline settings for ALDI+ In particular, we find that two simple changes consistently improve domain adaptation results: (1) Using strong regularization on both target and source data during self-training, and (2) Training with equal amounts of source and target supervision in each minibatch (ú.e. Bare Bigt). We also opt to disable all feature alignment in +LDI+ to stabilize training and find that the effects on accuracy are minimal (see Section 6.3
5
The CFC-DAOD Dataset
CFC-DAOD Chamnel river:
Camera
Camera
Camera
Images per camem
per camera
BBoox per image (hist)
100k
9.8k
4k-
75k
3k-
9.4k
50K
9.2k
Ik-
9.0k
0-
CFC: Kenai
Channel
Figure 3: The CFC-DAOD benchmark focuses on detecting fish (white bounding boxes) in sonar imagery under domain shift caused by environmental differences between the training location (Kenai) and testing location (Channel). Our dataset contains 168k bounding boxes in 29k frames sampled from 150 new vidcos captured over two days from different sonar cameras on the Channel river, enabling DAOD experiments. Here visualize the distribution of images and annotations from each camera.
Next we introduce our dataset contribution, CFC-DAOD, as step toward addressing P3: (a) Lack of diverse benchmarks leading to overestimation of mothods' gencrality.
CFC. The Caltech Fish Counting Dataset (CFC) (Kay et al. 2022) domain generalization benchmark sourced from fisheries monitoring, where sonar video is used detect and count migrating salmon. The detection task consists of single class ("fish") and domain shift is caused by real-world environmental differences between camera deployments. We identify this application crality of DAOD methods due to its stark differences with existing DAOD opportunity to study the gen- specifically, sonar imagery is grayscale, has low signal-to-nois ratios, and foreground objects are difficult to distinguish from the background-howee data needed for DAOD.
CFC focuses on generalization rather than adaptation and does not include the
CFC-DAOD We introduce an extension CFC, deemed CFC-DAOD, to enable the study of DAOD in this application domain. The task is to adapt from source location-"Kenai" f.e. the default training set from CFC-te to difficult target location, "Channel". We collected additional 168k bounding box annotations in 29k frames sampled from 150 new vidcos captured over two days from 3 different sensors on the "Channel" river (see Fig. 3). For consistency, we closely followed the video sampling protocol used to collect the original CFC dataset as described by the authors (see (ay et al.) 2022)). Our addition to CFC is crucial for DAOD as it adds an unsupervised training set for domain adaptation methods and supervisod training set train oracle methods. We keep the original supervised Kenai training set from CFC (132k annotations in 70k images) and the original Channel test set (42k annotations in 13k images). We note

---

Published in Transactions on Machine Learning Research (03/2025)
this is substantially larger than existing DAOD benchmarks (CS contains 32k instances in 3.5k images, and
Sim10k contains 58k instances in 10k images). See Appendix
D for more dataset statistics and Appendix
for qualitative visualizations. We make the dataset public.
6 Experiments
Source-only and oracle models
Lisue
Lopp
Hare
Pare
ypr
Pare
Post
Mest
only
Oracko
Target AP50 (FCS)
63.5
Strong mng
EMA
8,... (Inal)
EMA
Beo. RRail
EMA
t TT And
T
53.5
Ayrd
Souree
Fare
Pegt
Target
Fix = Mer
(optional)
Poct Pas
48.5
Sourcc-only
Oracle
Source AP50 (CS)
Figure 4: Revisiting source-only and oracle models in DAOD. We argue that in order to provide a fair measure of domain adaptation performance in DAOD, source-only and oracle models must utilize the same non-adaptive architectural and training components as methods being studied. In the case of Align and Distill-basee approaches, this means source-only and oracle models must have access to the same set of source augmentation and EMA as DAOD methods. We see that. these upgrades significantly improve source-only performance on target-domain data (+7.2 AP50 on Foggy Cityscapes), even though the source- only model has never seen any target-domain data, and these upgrades also improve oracle performance. Overall, these results set more challenging and realistic performance largets for DAOD methods.
In this section we propose an updated benchmarking protocol for DAOD (Section 6.1 that allows us to fairly analyze the performance of +LDI++ compared to prior work (Section 6.2 and conduct extensive ablation studies (Section 6.3
6.1 Benchmarking Protocol
Datasets. We perform experiments on Cityscapes + Foggy Cityscapes, Sim10k + Cityscapes, and CFC Kenai > Channel. In addition to being consistent with prior work, these datascts represent three common adaptation scenarios capturing a range of real-world challenges: weather adaplation, Sim2Real, and envi- ronmental adaptation, respectively. We note that there have been inconsistencies in prior work in terms of which ground truth labels for Cityscapes used. We the Detectron2 version, which includes three intensity levels of fog {0.005, 0.01,
Metrics. For all experiments we report the PascalVOO metric of mean Average Precision with loU 2 0.5 ("AP50") (Everingham et al. 2010). This is consistent with prior work on Cityscapes, Foggy Cityscapes, Sim10k, and CFC.
Revisiting sourcc-only and oracle models. Here we address P1: Improperly constructed sourcc-only and oracle models, leading overestimation of performance gains. The goal of DAOD is develop adaptation techniques that use unlabeled target-domain data to improve target-domain performance. Thus, in order to properly isolate adaptation-specific techniques, any technique that does not need target-domain data to run should also be used by source-only and oracle models. This means that source-only and oracle models should also utilize the same strong augmentations and EMA updates as DAOD methods.
In Fig. we illustrate the resulting sourco-only and oracle models, and show that including these compo- nents significantly improves both source-only and oracle model performance (+7.2 and +2.6 AP50 on Foggy Cityscapes, respectively). This has significant implications for DAOD research: because source-only and
8

---

Published in Transactions on Machine Learning Research (03/2025)
oracle models have not been constructed with equivalent components, performance gains stemming from better generalization have until now been misaltribuled to DAOD. With properly constructed source-only and oracle models, the gains from DAOD are much more modest (see Fig, 5) Note that for clarity we compare all methods in Fig. ! and Fig. 5 against a single source-only and oracle model, however it would be more appropriate to compare each method its own bespoke source-only and oracle models that use the same training components; see Appendix A.7 for the full comparison.
Fixed training settings. Prior work has used inconsistent backbones and image sizes, making head-to- head comparisons less fair. Using ALDI we instead compare using the same training settings, offering fair comparison. As a starting point we utilize reasonably modern scttings likely to be used by practitioner: the Cityscapes defaults in the Detectron2 codebase. All methods in our comparisons, including source-only and oracle models, utilize Faster R-CNN architectures with ResNet-50 (Ren et al., 2015) backbones with FPN (Lin et al. 2017), COCO (Lin et al. 2014) pro-training, and an image size of 1024px on the shortest
side. See Appendix for more details.
6.2
Fair Comparison and State-of-the-Art Results
CS + FCS
Sim Ok - CS
CFC Kenai - Channel
Target AP50
65
Source-only
PT
80
70
Oracle
MIC
60
70
UMT
AT
SADA
ALDI+
55
60
Figure 5: Fair comparison of ALDI++ with existing state-of-the-art approaches using the ALDI framework and modern training recipes. Some prior methods show consistent benefit but others lag behind fair source-only models. Our method +LDI++ outperforms prior work on all datascts studied by a significant margin: +3.5 AP50 on CS + FCS, +5.7 AP50 on Sim10k + CS, and +0.6 AP50 on CPC Kenai Channel. Notably, ALDI+ is the only method to outperform source-only model on Sim10k - CS.
We compare + with reimplementatious of five -fate-of-the-art DAOD methods on top of our frame- work: UMT (Deng et al. 2021), SADA (Chen ct al., 2021), PT (Chen al. 2022). MIC (Hoyer et al., 2023), and AT (Li et al., 2022b); see Appendix C for the ALDI settings used to reproduce them. We use the fair benchmarking protocol propased in Section 6.1 Results are shown in Fig. All methods (including
ALDI++) use the same settings for all benchmarks.
Comparison with state-of-the-art. ALDI+ outperforms all prior work and sets a new state-of-the-art on all benchmarks studied, outperforming the next-best methods by +3.5 AP50 on CS - FCS, +5.7 AP50 on Sim10k + CS (where ours is the only method to outperform fair source-only model), and +0.6 AP50 CFC Kenai Channel. ALDI+ achieves near-oracle level performance CS + FCS and CFC Kenai Channel (0.4 and 0.9 AP50 away, respectively), while other methods close less than half the gap between source-only and oracle models.
Comparison across datasets. We compare all methods on CS + FCS, Sim10k + CS, and CFC Kenai Channel, in Fig. 5 We find the ranking of methods differs across datasets. ALDI++ MIC and AT are consistently the top-performing mothods across all datasets. MIC performs noticcably better on CFC Kenai Channel than other prior work, nearly matching the performance of ALDI++. UMT exhibits variable performance due to the differences in the difficulty of image generation across datasets (see Appendix C for examples). SADA underperforms other methods on CS + FCS and CFC Kenai + Channel, but closes this gap on the more difficult. Sim10k + CS. These results dernonstrate the utility of CFC-DAOD as another point of comparison for DAOD methods; we see that method performance on synthetic benchmarks like CS + FCS is not necessarily indicative performance on real-world domain shifts.
Comparison with fair source-only and oracle models. Re-implementing methods in ALDI improves absolute performance of most methods due to upgraded training settings; however performance decreases
9

---

Published in Transactions on Machine Learning Research (03/2025)
R:50-FPN (ImageNet pre-train) R50-FPN (COCO pre-train)
VitDet-B
Cityscapes - Foggy Cityscapes
Sourcc-only
70
Oracle
mAP@..U--05
UMT
60
SADA
PT
MIC
50
AT
ALDI+
40
Figure 6: Comparison across backbones and pre-training data. Pre-training strategy does not sig- nificantly change the relative strength of methods compared to each other nor compared source-only and oracle models. However, all models perform worse compared to source-only and oracle models when using VitDet backbones, with only ALDI++ and MIC outperforming sourco-only model.
dramatically compared to source-only and oracle models. There are several instances where modernized DAOD methods are actually worse than fair source-only model. Notably, a source-only model outperforms work on Sim10 + CS. We also see that no state-of- the-art methods
upgraded versions all previously
outperform fair oracle on any datasct, in contrast to claims made by prior work (Li et al. 2022b
Chen
et al. 2022
Cao et al. 2023
Comparison across backbones and pre-training data. We compare all methods on CS + FCS using ImageNet vs. COCO pre-training, as well as Resnet-50 FPN backbones vs. VitDet-B backbones, in Fig. G See Appendix [ for other datasets. We see that while COCO pre-training improves absolute performance of all methods, their ranking does not change significantly compared to ImageNet pre-training, nor does their performance in relation to source-only baselines. Interestingly, we find that ALDI++ outperforms an oracle CS + FCS when using ImageNet pre-training, however do not observe this trend the other datasets (see Appendix F). We hypothesize that this may be due to noise in the Foggy Cityscapes target-train labels due to their programmatic generation, and that COCO pre-training helps prevent the oracle from overfitting to these erroneous boxes. We show that only +LDI+ and MIC continue to show improvements over an upgraded VitDet source-only model (see Appendix A.2 for other datasets), with ALDI++ performing slightly better than MIC (+0.4 AP50). We see there is a larger gap between the ViT ALDI++ and the ViT oracle compared to ResNet backbones, indicating the potential for future work to improve performance. Across all experiments in Fig. 6 we see that the source-only- oracle gap shrinks as the underlying model improves due to either stronger pre-training or backbone upgrades, indicating that DAOD may offer diminishing returns with stronger models.
6.3 Ablation Studies
In this section we ablate the performance of each component of ALDI on CS + FCS.
Base settings. For cach ablation, unless otherwise specified we begin with the following training settings. We initialize Swn,, Otch with COCO pre-training followed by a burn-in phase on Xsre with weak augmentations and early stopping based on validation performance. Tare includes random horizontal flip and random scaling, Tigt includes random horizontal flip, random scaling, color jitter, and cutout. The Bare Bigt batch ratio is 1:1. Ldistill is hard pseudo-labeling with a confidence threshold of 0.8, and Lalign is disabled. Note these base settings are not necessarily those of ALDI+ but rather the most commonly chosen values in prior work for each component. Additional training settings are reported in Appendix C
Oetus 8tch Network initialization (burn-in). In Fig. 7 we analyze the effects of our proposed burn-in strategy (sce Section 4 We mcasure performance in terms of target-domain AP50 as well as convergence time, defined as the training time at which the model first exceeds 95% of its final larget-domain performance. We compare our approach with: (1) No dataset-specifie burn-in, f.e. starting with COCO weights, and (2) The approach used by past work-using fixed burn-in duration, e.g. 10k iterations. We find that our method results in significant improvements in both training speed and accuracy, leading to upwards of 10% improvements in AP50 and reducing training time by factor of 10 compared to training without burn-in.
10

---

Published in Transactions on Machine Learning Research (03/2025)
Table 1: Ablation studies. (a) Effects of target-domain augmentation on self-training. Augmentations ap- plied to student inputs (Tige in Fig. 2. Stronger augmentations improve performance considerably. (b) Ef- fects of distillation objectives on self-training. We compare hard targets- -seed by most prior art, which thresholds teacher predictions to create pscudo-labels- with proposed soft targets. Soft targets im- prove overall performance. Results are the mean and standard deviation over 3 runs. (c) Feature alignment has diminishing returns. Compared to a source-only baseline AP50 of 59.1, feature aligmment objectives Latign without self-training provides up to 2.6 AP50 benefit (first row), but diminishes to 0.2 AP50 additional gain when used alongside solf-training (last row).
(a)
(b)
(c)
Tgt
AP50pCS
AP50
Lokion
Laiistuu
AP50RCs
Weak (scale & flip)
52.6
Hard targets
63.7 = 0.1
61.7
+
Color jitter
59.0
Soft targets
64.0 = 0.4
/
63.7
+
Color jitter Erase
63.1
/
v
63.9
+
Color jitter + MIC
64.3
Tere Source augmentations. In Fig. 70 we ablate the set of source-domain data augmentations. We compare using weak augmentations (random flipping and random scaling), strong augmentations (color jitter and cutout), and a combination of weak and strong, noting that prior works differ in this regard but do not typically report the settings used. We find that using strong source augmentations on the entire source-domain training batch outperforms weak augmentations and a combination of both.
Tigt Target augmentations. In Table la we investigate the of different augmentations for target- domain inputs to the student model. (We note that weak augmentations are always used for target-domain inputs to the teacher in accordance with prior work). We see that stronger augmentations consistently improve performance, with best performance coming from the secently-proposee MIC augmentation (Hoyer
et al. 2023
Bigt/ B Batch composition. In Fig. 7 we ablate the ratio of source and target data within minibatch. We note that prior works differ in this setting but do not typically report what ratio used. We see that using equal amounts of source and target data within each minibatch leads to the best performance. Notably, we also find that the inclusion of source-domain imagery is essential to benefits from self- self-training without any source imagery, AP50Fcs drops from 64.5 to 59.3.
Ostu- Oech Burn-in: Performance train time
T src Source augmentations
Brgt B Batch composition
65
65.0
65
Target AP50 (FCS)
64
60
64.5
63
Burn-in
62
None
64.0
Fixed duration
61
Our approach
63.5
60
45
After bum--n
x After self-traninng
$9
63.0
Weak
Strong
Both
Training time (V100-hours)
Target Bege
(a)
(b)
(c)
Figure Ablation studies. (a) Our proposed burn-in stratogy (Section improves AP50rcs by +4.7 and reduces training time by 10x compared to no burn-in. (b) Strong source-data augmentations during self-training lead to better performance. (c) An equal ratio of source and target data during self-training leads to best performance.
11

---

Published in Transactions on Machine Learning Research (03/2025)
Ldistill Self-distillation. In Table 1b we analyze the effects of our proposed multi-task soft distillation approach (see Section 4. We compare our approach with the "hard" pseudo-label approach used by prior work, where teacher predictions are post-processed with non non-maximum suppression and a hard confidence threshold of 0.8 (Hoyer et al. 2023 Li ct al., 2022b; Deng et al., 2021 Liu et al., 2021). For our proposed "soft" distillation method, we first sharpen leacher predictions at both detector stages using a sigmoid for objectness predictions and softmax for classification predictions, both with default temperature of 1. We see that our proposed soft targets improve performance compared to hard targets.
Latign Feature alignment. Finally we investigate the use of feature alignment. We implement adver- sarial feature alignment approach consisting of an image-level and instance-level feature discriminator (our implementation performs on par with SADA while being simpler to train; see Appendix A.. In Table Lc we show that fcature alignment used in isolation (í.e. without self-training) offers performance gains up 2.6 AP50. However, these performance gains are smaller than those seen from self-training (AP50 of 61.7 vs. 63.7, respectively). When used in combination with self-training techniques, the additional ben- cfit of feature alignment drops to I 0.2 AP50 This suggests that self-training is currently the most promising avenue for progress and that more research is needed to develop complementary approaches. We also note that feature alignment approaches introduce training instability that may not be worth the small performance gain for practical use.
7 Discussion and Conclusions
In this work we proposed: the ALDI framework and an improved DAOD benchmarking methodology, pro- viding a critical reset for the DAOD research community; a new dataset CFC-DAOD, increasing the diversity and real-world applicability of DAOD benchmarks; and new method ALDI+ that advances the stato-of- the-art. We conclude with key findings.
Network initialization has an outsized impact. We find that general advancements in computer vision eclipse progress in DAOD: Sesnet50-FPX source-only model outperforms all VGG-based DAOD methods, and a VitDet source-only model outperforms all Resnet50-FPN based DAOD methods. Similarly, simply adding stronger augmentations and EMA to source-only models leads to better target-domain performance than some udaptation methods, and including these upgrades during network initialization (burn-in) improves adaptation performance as well.
DAOD techniques are helpful, but do not consistently achieve oracle-level performance previously claimed (Li et al., 2022b; Chen et al., 2022: Cao et al., 2023). Top-performing DAOD methods, including ALDI++, demonstrate improvements over source-only models (see Fig. | and Fig. 5 However, in contrast to previously-publliler results, no DAOD method consistently reaches oracle-level performance across datasets, architectures, and pre-training strategics, suggesting there is still room for improvement. The gap between DAOD methods and oracles even larger for stronger architectures like VitDet. This is a promising area for future research.
Benchmarks sourced from real-world domain adaptation challenges can help the community develop generally useful methods. We find that DAOD methods do not necessarily perform equivalently across datasets (see Fig. 5). Diverse benchmarks are useful to make sure we are not overfitting to the challenges of one particular use case, while exposing and supporting progress in impactful applications. Our contributed codebase and benchmark dataset provide the necessary starting point to enable this effort.
A lack of transparent comparisons has incentivized incremental progress in DAOD. Most highly-performant prior works in DAOD are some combination of DANN (Ganin et al. 2016) and Mean Teacher (Tarvainen & Valpola 2017) plus custom training techniques. Without fair comparisons it has been possible to propose near-duplicate methods that still achieve state-of-the-art performance duc to hyperpa- rameter tweaks. Our method ALDI++ establishes a strong point of comparison for Align and Distill-based approaches that will require algorithmic innovation to surpass.
Validation is the elephant in the room. All of our experiments, and all previously published work in DAOD, utilize a target-domain validation set to perform model and hyperparameter selection. This violates key assumption in unsupervised domain adaptation: that no target-domain labels available
12

---

Published in Transactions on Machine Learning Research (03/2025)
to begin with. Prior work has shown that it may not be possible to achieve performance improvements in domain adaptation at all under realistic validation conditions (uuggrav et al. 2021 2022 Kay et al. 2023). Therefore our results (as well as previously-publisiee work) can really only be seen as an upper bound on DAOD performance. While this is valuable, further research is needed to develop effective unsupervised validation procedures for DAOD.
Acknowledgments
This material is based upon work supported by: NSF CISE Graduate Fellowships Grant #2313998, MIT EECS department fellowship #4000014939, MIT J-WAFS seed grant #2040131, and Caltech Resnick Sus- tainability Institute Impact Grant "Continuous, accurate and cost-effective counting of migrating salmon for conservation and fishery management in the Pacific Northwest." Any opinions, findings, and conclusions or vecommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF, MIT, J-WAFS, Caltech, or RST. The authors acknowledge the MIT SuperCloud and Lincoln Labo- ratory Supercomputing Center for providing HPC resources Reuther al. (2018). We also thank the Alaska Department of Fish and Game for their ongoing collaboration and for providing data, and Sam Heinrich, Neha Hulkund, Kai Van Brunt, Rangel Daroya, and Mark Hamilton for helpful feedback.
References
Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. Ensemble of averages: Improving model selection and boosting performance in domain generalization. Advances in Neural Information Processing Systems, 35:8265- 8077-8277777 2022.
Elizabeth Bondi, Fei Fang, Mark Hamilton, Debarun Kar, Donnabell Dmello, Jongmoo Choi, Robert Han- naford, Arvind Iyer, Lucas Joppa, Milind Tambe, et al. Spot poachers action: Augmenting conservation drones with automatic detection near real time. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Chong-Wah Qi Cai, Yingwei Pan, Chong- Wah Ngo, Xinmei Tian, Lingyu Duan, and Ting Yao. Exploring object relation in mean teacher for cross-domain detection. In Proceedings of the IEEE/CVP Conference on Compuler Vision and Pattern Recognition, pp. (1457-11468 2019.
Shengeao Cao, Dhiraj Joshi, Liang- Yan Gui, and Yu-Xiong Wang. Contrastive mean teacher for domain adaptive object detectors. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 23839 -3844,,,,8, 2023.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213 229. Springer, 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging propertics in self-superiieed vision transformers. In Proceedings of the IEEE/CVP international conference on computer vision, pp. 9650 9660, 2021.
Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker. Learning efficient object detection models with knowledge distillation. Advances in neural information processing systems, 30, 2017.
Mcilin Chen, Woijic Chen, Shicai Yang, Jie Song, Xinchao Wang, Lei Zhang, Yunfeng Yan, Donglian Qi, Yueting Zhuang, Di Xie, al. Learning domain adaptive object detection with probabilistic teacher. arXiv preprint Xiv::2200. 06293, 2022.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
13

---

Published in Transactions on Machine Learning Research (03/2025)
Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cmn for object detection in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3339 3348, 2018.
Yuhua Chen, Haoran Wang, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Scale-aware domain adaptive faster r-cnn. International Journal of Computer Vision, 129(7)22223 2243, 2021.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understand- ing. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2016.
Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan. Unbiased mean teacher for cross-domain object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 4091 4101, 2021.
Terrance DeVrics and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1706 fo::0700,,55,, 2017.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words:
Transformers for image recognition at scale. arXiv preprint arXim:2010. 1 11929, 2020.
Mark Everingham, Lue Van Gool, Christopher KT Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303 338, 2010.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. Tn Interna- tional conference on machine learning, pp. 1180 1189. PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096 20060, 2016.
Irena Gao, Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang, Out-of-distribution robustness via targeted augmentations. In NewrIPS 2022 Workshop Distribution Shifts: Connecting Methods and Applications, 2022.
Hao Guan and Mingxia Liu. Domain adaptation for medical image analysis: a survey. IEEE Transactions on Biomedical Engineering, 69(3):1173 1185, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on compuler vision, pp. 2961 2969, 2017.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pp. 16000-16009, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in neural infor- mation processing syslems, 30, 2017.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. ar preprint 1003 2015.
14

---

Published in Transactions on Machine Learning Research (03/2025)
Lukas Hoyer, Dengxin Dai, Haoran Wang, and Luc Van Gool. Mic: Masked image consistency for context- enhanced domain adaptation. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pattern Recognition, pp. 11721 11732, 2023.
Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weakly-supervised object dctection through progressive domain adaptation. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 5001-5009, 2018.
Peidong Jia, Jiaming Liu, Senqiao Yang, Jiarui Wu, Xiaodong Xie, and Shanghang Zhang, Pm-detr: Domain adaptive prompt memory for object detection with transformers. arXiv preprint arXiv:2307.00313, 2023.
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralyties yolov8, 2023. URL https://github.com ultrallyticcsuulrraayyyee
Matthew Bohnson-Roberson Charles Barto, Rounak Mehta, Sharath Nittur Sridhar, Karl Rosaen, and Ram Vasudevan. Driving in the matrix: Can virtual worlds replace numan-generated annotations for real world tasks? arXiv preprint 0rXiv.1610.01980 2016.
Tarun Kalluri, Wangdong Xu, and Manmohan Chandraker. Geonet: Benchmarking unsupervised adaptation across geographies. CVPR, 2023.
Justin Kay, Peter Kulits, Suzanne Stathatos, Siqi Deng, Erik Young, Sara Beery, Grant Van Horn, and Pictro Perona. The caltech fish counting dataset: A benchmark for multiple-object tracking and counting. In European Conference on Computer Vision, pp. 290 311. Springer, 2022.
Justin Kay, Suzanne Stathatos, Siqi Deng, Erik Young, Pietro Perona, Sara Beery, and Grant Van Horn. Unsupervised domain adaptation in the real world: A case study in sonar video. In NewrIPS 2023 Computational Sustainabilily: Promises and Pitfalls from Theory to Deployment, 2023.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in- the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637 5664. PMLR, 2021.
Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In European Conference on Computer Vision, pp. 280-296. Springer, 2022a.
Yu-Jhe Li, Xiaoliang Dai, Chih- Yao Ma, Yen-Cheng Liu, Kan Chen, Bichen Wu, Zijian He, Kris Kitani, and Peter Vajda. Cross-domain adaptive teacher for object detection. In Proccedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognation, pp. 7581 7590, 2022b.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zätnick. Microsoft coco: Common objects in context. In European conference on compuler vision, pp. 740 100-755.27 Springer, 2014.
Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pallern recomnition, pp. 2117 2125, 2017.
Yen-Cheng Liu, Chih-Yar Ma, Zijian He, Chia-War Kuo, Kan Chen, Peizhao Zhang, Bichen Wn, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi-supervised object detection. oix preprint arxiv: 102. 09480, 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnct for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976-11986, 2022.
Daniel Morales-Brotons, Thijs Vogels, and Hadrien Hendrikx. Exponential moving average of weights in deep learning: Dynamics and benefits. Transactions Machine Learning Research, 2024.
15

---

Published in Transactions on Machine Learning Research (03/2025)
Kevin Musgrave, Serge Belongie, and Scr-Nam Lim. Unsupervised domain adaptation: A reality check. arXiv preprint Xr::::2111. 15672, 2021.
Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. Benchmarking validation methods for unsupervised domain adaptation. arXiv preprint arXiv:2208.07533 2022.
Dang- Nguyen, Wei-Lun Tseng, and Hong-Hap Han Shuai. Domain-adaptive object detection via uncerlainly-aware distribution alignment. In Proceedings of the 28th ACM international conference on 2020.
multimedia, pp.
Poojan Oza, Vishwanath A Sindagi, Vibashan Vishnukumar Sharmini, and Vishal M Patel. Unsupervised domain adaptation of object detectors: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Gaurav Parmar, Richard Zhang, and fun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11420,, 2022.
Minh Pham, Minsu Cho, Ameya Joshi, and Chinmay Hegde. Revisiting self-distillation. arXiv preprint arXiv:2206.08// 08/991 2022.
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection, 2016. URL https://arxiv.on org. wwt.:/gg///////eeeee.o aaaaabas///// /000///50...0 02640
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cmn: Towards real-time object detection with region proposal networks. Advances in neurol information processing systems, 28, 2015.
Albert Reuther, Jeremy Kepner, Chansup Byun, Siddharth Samsi, William Arcand, David Bestor, Bill Berg- eron, Vijay Gadepally, Michael Houle, Matthew Hubbell, Michael Jones, Anna Klein, Lauren Milechin, Julia Mullen, Andrew Prout, Antonio Rosa, Charles Yee, and Peter Michaleas. Interactive supercomput- ing on 40,000 cores for machine learning and data analysis. In 2018 IEEE High Performance extreme Computing Conference (HPEC), pp. 1-6. IEEE, 2018.
Mikel Rodriguez, Ivan Laptev, Josef Sivic, and Jean- Audibert. Density-aware person detection and tracking in crowds. In 2011 International Conference on Computer Vision, pp. 2423-2422222222 24300.0... TEEE, 2011.
Aruni ChoChuur,,,, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, and Erik Learned-Miller. Automatic adaptation object detectors to new domains using self- self-training. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pallern Recoguilion, pp. 780 790, 2019.
Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Strong-weat distribution alignment for adaptive object detection. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pp. 6956-6966 6965, 2019.
Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision, 126:973 992, 2018
Stefan Schneider, Graham W Taylor, and Stefan Kremer. Deep learning object detection methods for ccological camera trap data. In 2018 15th Conference computer and robot vision (CRV), pp. 321-328. IEEE, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint Xrxi::1409. 1556, 2014.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
16

---

Published in Transactions on Machine Learning Research (03/2025)
Vibashan Vs, Vikram Gupta, Poojan Oza, Vishwanath A Sindagi, and Vishal M Patel. Mega-cda: Memory guided attention for category-aware unsupervised domain adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 4516 4526, 2021.
Wen Wang, Yang Cao, Jing Zhang, Fengxiang He, Zheng-Jun Zha, Yonggang Wen, and Dacheng Tao. Exploring sequence feature alignment for domain adaptive detection transformers. In Proceedings of the 29th ACM International Conference on Mullimedia,, pp. 1730 1738, 2021.
Ben Weinstein, Lindsey Gardner, Vienna Saccomanno, Ashley Steinkraus, Andrew Ortega, Kristen Brush, Glenda Yenni, Ann E McKellar, Rowan Converse, Christopher Lippitt, et al. A general deep learning model for bird detection in high resolution airborne imagery. Rriv, 2021a.
Ben G Weinstein, Sarah J Graves, Sergio Marconi, Aditya Singh, Alina Zare, Dylan Stewart, Stephanie A Bohlman, and Ethan P White. A benchmark datasct for canopy crown detection and delincation in co- registered airborne rgb, lidar and hyperspectral imagery from the national ecological observation network. PLoS computational biology, 177():e000 2021b.
Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan- Yen Lo, and Ross Girshick. /g/thhb.. httpa/hcww.cooecoooooeeeeeeeeete 2019.
Detectron2.
https:
Zhiyun Xue, Feng Yang, Sivaramakrishnan Rajaraman, Ghada Zamzmi, and Sameer Antani. Cross dataset
analysis of domain shift in exr lung region detection. Diagnostics, 13(6):1068, 2023.
Jinze Yu, Jiaming Liu, Xiaobao Wei, Haoyi Zhou, Yohei Nakata, Denis Gudovskiy, Tomoyuki Okuno, Jianxin Li, Kurt Keutzer, and Shanghang Zhang. Mttrans: Cross-domain object detection with mean teacher transformer. In European Conference on Computer Vision, pp. 629-645. Springer, 2022.
Huayi Zhou, Fei Jiang, and Hongtao Lu. Ssda-yolo: Semi-supervised domain adaptive yolo for cross-domain object detection. Computer Vision and Image Understanding, 229:103649, 2023.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei Efros. Unpaired haage-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEFE international conference on computer vision, pp. 2223-2232, 2017.
Xinge Zhu, Jiangmiao Pang, Ceyuan Yang, Jianping Shi, and Dahua Lin. Adapting object detectors via selective cross-domain alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 687 696, 2019.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXic preprint orXiv:2010.044 2020.
17

---

Published in Transactions on Machine Learning Research (03/2025)
A Additional Experiments
A.1 YOLO and DETR Architectures
To demonstrate the architecturr-agnostiisnn of our framework and enable further rescarch, we implement ALDI for the one-stage delection architect YOLOv5 Redmon al. (2016); Jocher et al. (2023) and the transformer-based architecture Deformable DETR Carion et al. (2020] Zhu et al. (2020). Implementation
details are further described in Appendix B
Table 2: ALDI-YOLO results.
(a) + PCS
(b) Sim10k - CS
(c) CFC-DAOD
Method
AP50
Method
AP50
Method
AP50
Source-only (Y0L.v55m)
58.8
Source-only
75.0
Source-only
60.2
SSDA-YOLO Zhou et 2023 (YOLOv5I)
55.9
ALDI- ALDI-YOLO (ours)
75.0
ALDI-YOLO (ours)
52.4
ALDI-YOLO (ours, YOLOv5m)
62.5
Oracle
88.0
Oracle
76.7
Oracle (YOLOv5m)
66.3
Table 3: DETR results on Cityscapes + Foggy Cityscapes. We use 800px input size for consistency with prior work.
Method
AP50
Source-only
44.5
SFA Wang et al. (2021
41.3
MTTrans Yu et al. (2022)
43.4
PM-DETR Jia et al. 2023
44.3
ALDI-DETR (ours)
44.8
Oracle
50.0
A.2 ViT and ConvNeXt backbones
Table 4: Sim10k Cityscapes
Table 5: CFC Kenai
Channel
Method
AP50cs
Method
AP50channel
ViT-B baseline
81.7
ViT-B baseline
69.0
ALDI+ + ViT-B
81.8
ALDI+ + ViT-B
71.1
ViT-B oracle
89.8
ViT-B oracle
76.7
For completeness we show results using ALDI++ in combination with VitDet-B Li al. (2022a) in Table 4
(Sim10k - Cityscapes) and Table (CFC Kenai - Channel). We that ALDI continues demonst improvements over baselines even overall architectures get stronger, though these improvements are smaller in magnitude than VitDet-B results on the CS - FCS datasct.
We also demenstrate the performance of ALDI+ with even larger backbones to examine how performance and domain gaps change. We show results from VitDet-L in Table 0 and with ConvNeXt-L in Table 7 Results are similar our main results; we continue see improvements of basclines, oracles, and ALDI- in these settings.
A.3 Adversarial Feature Alignment
We report additional ablations for the adversarial feature alignment network(s) used, comparing our imple- mentations of image-level alignment and instance-level alignment with baseline and SADA. As see in
18

---

Published in Transactions on Machine Learning Research (03/2025)
Table 6: Cityscapes + Foggy Cityscapes
Table Cityscapes + Foggy Cityscapes
Method
AP50FCs
Method
AP50FCs
ViT-L baseline
70.2
ConvNeet-- baseline
58.9
+ + ViT-I.
76.1
ALDI+ + ConvNext-L
63.3
ViT-L oracle
77.4
ConvNext-L oracle
64.1
Table Sa. Table Sb and Table [c the best settings use differ by dataset. By default our feature alignment experiments in Sec. 6.1 of the main paper both instance and image level alignment. See Appendix B.6
below for further implementation details.
Table 8: Comparison of adversarial alignment methods. (a) Cityscapes - Foggy Cityscapes. We see that our implementations outperform SADA Chen al. (2021) while being simpler. Image-level alignment is best, followed by Image + Instance. (b) Sim10k -> Cityscapes. Instance-level alignment is best. (c) CFC Kenai - Channel. Image Instance is best. We see there is no consistently-best strategy across datasets; however, we note that for all datasets, the benefit of using adversarial feature alignment is smaller than self-training (see Sec. 6.3 of the main paper).
(b)
Method
AP50Fcs
Mothod
AP50cs
Method
AP50channel
Source-only
51.9
Source-only
70.8
Source-only
65.8
SADA
54.2
Image-level (ours)
71.8
Image-level (ours)
65.2
Image-level (ours)
55.8
Instance-level (ours)
73.3
Instance-level (ours)
66.0
Instance-level (ours)
54.3
Image Instance (ours)
71.5
Image Instance (ours)
66.9
Image Instance (ours)
54.9
A.4 Visualizing Alignment
We investigate the overlap of source and target. data in the feature space of different methods. For each method, we pool the highest-level feature maps of the backbone, either globally ("image-level") or per instance ("instanco-level".. We then embed the pooled feature vectors in 2D space using PCA for visual inspection (see Fig We also compute a dissimilarity score based on FID Heusel el. al. (2017), by fitting Gaussians the source and target features and then computing the Fréchet distance between them.
A.5 Teacher update
We compare other approaches to updating the teacher during self-training vs. using exponential moving average in Table g We that EMA significantly outperforms using fixed teacher (i.e. vanilla self- training, where pseudo-labels are generated once before training) well as using the student as its own teacher without EMA.
Table 9: Comparison of teacher update approaches on Cityscapes Foggy Cityscapes. Mean teacher greatly outperforms other options.
Method
AP50rcs
Sourcc-only baseline
51.9
No update (vanilla self self-training)
52.9
Student is teacher
53.8
EMA (mean teacher)
63.5
19

---

Published in Transactions on Machine Learning Research (03/2025)
Bascline
MeanTeacher
UMT
Adversarial Alignment
Cityscapes Foggy
v=0.22,4==0.81
0.63, 0.607
0.61 0.002
0.02
y= dp
Sim10k
Cityscapes
0.69, d, 48.30
== 0.65, dy 14.44
== 0.62, d, 6.55
== 0.59, 15.68
Kenai S1
Kenai S2
Channel
Channel $2
Channel
0=0.91.4,=6.3
d.
== 0.89, de 0.13
y= 0.77.
Figure 8: Embedding of pooled features from the final backbone layer in 2D space using PCA. We compare: (1) A source-only baseline, (2) The base settings from Section 6.3 ("Mean Teacher"), (3) UMT (which utilizes image-to-image translation), and (4) MeanTeel Teacher + adversarial feature alignment using image-level features. The ratio of variance explained by the first two PCA components is given by v and dissimilarity score between source and target features is given by dr. dr is lower than the bascline for all alignment methods and does roughly match the overall visual trend in feature overlap. In all cases, the simple MeanTeache model significantly reduces the distance between source and target data even though there is no explicit alignment criterion, even resulting in a smaller de than adversarial alignment methods for CS + FCS & CFC Kenai + Channel.
A.6 Example of FUir Comparisons
In Fig. we show a case study of why fair comparisons are impactful for DAOD rescarch. We compare two similarly-performing prior works, AT and MIC, and that implementation inconsistencies have led to nontransparent comparisons between the two methods. Notably, the originally reported results even used different ground truth test labels. When re-implemented top of the same modern framework using ALDI, we are able to fairly compare the two methods for the first time.
A.7 Method-specific Cource-Only Models
Our protocol for training fair source-only models introduced in Section 6.1 to utilize all techniques from the methods being studied that do not need target data run. For simplicity, in the main paper have only displayed the source-only model that utilizes the same components ALDI++, though these settings differ slightly from the other methods studied. In Fig. 10 we show an alternate view in which for cach method we train bespoke source-only model using the exact same training settings as the DAOD method. The main difference is the set image augmentations used, except for SADA, which also does not use EMA. We see that there is only a small variation the strength of the sourcc-only models corresponding to each method, so our choice to only visualize one in the main paper for simplicity is reasonable. The exception is SADA, whose source-only model is significantly weakened by not using FMA.
20

---

Published in Transactions on Machine Learning Research (03/2025)
AT
AP50FCS
MIC
Same ground truth + Same codebase + Modemn backbone Figure 9: Effects of fair and modernized comparison between MIC and AT. Here we show an example why fair and modern comparisons are necessary for making principled progress in DAOD. Moving left to right: (1) Published results report a difference of 3.3 AP50 on Cityscapes + Foggy Cityscapes wetween the two methods; (2) However the authors used different truth test labels, and when this is corrected we see that the originally-publise
Reported
MIC model actually outperforms the originally-publlshee AT model; (3) The authors also used different object detection libraries (Detectron2 for AT and maskrenn-benchhmate for MIC); when we re-implement them on top of ALDI (still using the VGG-16 backbones proposed in the original papers), we see that AT significantly outperforms MIC, but (4) These performance differences are less pronounced when using a modern backbone, indicating that for practical use there is less difference between these two methods than previously reported.
Method-specific source-only models
Source-only
60
UMT
TargetAPSS APS0
SADA
PT
40
MIC
CS
AT
20
ALDE
UMT
SADA
MIC
ALDI+
Figure 10: Comparing methods method-specific source-only models that use the exact same training settings the DAOD method in question. These results use ImageNel pre-training. Most of the bespoke source-only models perform very similarly, in the main paper only visualize one representative source-only model. The exception is SADA, whose corresponding source-only model performs worse due to the lack of EMA during training. See Appendix A.7
21

---

Published in Transactions on Machine Learning Research (03/2025)
B Implementation Details
B.1 ALDI-YOLO
We use Detectron2 implementation of YOLOv5t our starting point. All hyperparameter settings are identical to those of A
We implement soft distillation for YOLOv5 as follows. We compute an "objectness" bagreground/bacckgguea loss for each proposal, and compute classification and localization losses for pseudo-forrerrond labels only. Given pre-softmax student. logits and teacher logits I:
Lobj.soft (C labs)
(14)
Tids,soft CF CF(lass,a(aat))
(15)
Lioc, CCCUUCOU altoo))
(16)
saftt = Lobj.soft Leds,soft + Lloc,soft
(17)
Where BCE is the binary cross-entropy loss, CE is the cross-entropy loss, CIOU is the Complete IoU loss and a is still function of the post-softmax scores. See ?? for a visual depiction.
B.2 ALDI-DETR
We use an Detectron2 implementation of Deformable DETR as our starting point. There is not an established technique for using soft knowledge distillation in end-to-end transformer-base queries like those DETR: thus, we use hard distillation with a pseudo label threshold of 0.8. Similar to prior work Wang et al. 2021 Yu et al. 2022 Jia al. (2023). we disable the EMA update for object query parameters.
Table 10: Settings to reproduce five prior works and our method ALDI+ Burn-in: fixed duration (Fixed), our approach (Ours, Section 4h. Augs. Tares Tigt: Random flip (F), multi-scale (M). crop & pad (CP), color jitter (J), gaussian blur (B), cutout (DeVrics & Taylor 2017 (C), MIC (Hoyer et al. 2023). 1: augs used half the images in the batch. Begt,
BB: Target-domain portion of minibatch of size B. Postprocess: Processing of teacher preds before distillation: sigmoid/sofin /softmax (Sharpen), sum class preds for pseudo-objectuess (Sum), conf. thresholding (Thresh), NMS. Ldiatar: Distillation losses: hard pseudo-labels (Hard), continuous targets (Soft). Latign: Feature aligument losses: image-level adversarial (Img), instance-level adversarial (Inst)), image-to-image translation (Img2Img). t; settings used in ALDI implementation (last column) but not in the original implementation (second-to-last column). at, source- only and oracle results sourced from
Li et al. (202bb).
Method
Oxtu: Bich
T
Tigt
B.g. 19
Post- process
Liiastit
Lotiggn
Buru-in
Source-only
F, MT, Ct, BI, E
0.0
SADA
Chen
2021
F, M Mt
F, M
0.5
Img. lust
PT
Chen al.
2022
Fixed
F, M J, Ft, MT, CP, J.
Sharpen, Sum
Soft
UNTT
Deng et al.
2021
Ft, Mt
0.5
Thresh, NMS
Hard
Img21mg
MIC
Hoyer et al.
2023
F, M
F. Mr,
MIC
0.5
Thresh, NMS
Hard
Img. Inst
AT
Li et
2022b
Fixed
F, MT, J4, CA,
F, Mi, J, C,
0.3
Thresh, NMS
Hard
Img
ALDI+
Ours
F,M. C
F, M. J, B, MIC
0.5
Sharpen
Soft
Oracle
F, Mi, Jt, CI, Bt
1.0
B.3 Re-implementations Other Methods
Here we include additional details regarding our re-implementations of prior work on top of the ALDI framework. All hyperparameters are reported in Table 10 We visualize our implementations in Fig. 11
22

---

Published in Transactions on Machine Learning Research (03/2025)
B.3.1
Adaptive Teacher (Li et al., 2022b)
Adaptive Teacher (AT) uses the default settings from the base configuration Table of the main paper, plus an image-level alignment network. For fair reproduction, we used the authors' alignment network implementation instead of our own for all AT experiments.
B.3.2 MIC (Hoyer al., 2023
We reimplemented the masked image consistency augmentation as a Detectron2 Transform our framework for efficiency. We also implemented MTC's "quality weight" loss re-weighting procedure, though in our experiments found that makes performance slightly worse (AP50 Foggy Cityscapes of 62.8 vs. 63.1
without).
B.3.3 Probabilistic Teacher (Chen et al. 2022
Probabilistic Teacher (PT) utilizes: (1) custom Faster R-CNN architecture that makes localization predic- tions probabilistie, called "Gaussian R-CNN", (2) focal loss objective, (3) learnable anchors. We ported implementations of these three components to our framework. Note that we first had to burn in Gaussian R- R-CNN, so PT was not able to use the exact same starting weights other methods.
B.3.4 SADA (Chen et al., 2021
We port the official implementation of SADA Detectron2. Note that SADA does not include burn-in self-training, so the base implementation is the Detectron2 baseline config.
B.3.5
Unbiased Mean Teacher (Deng et al. 2021
Our implementation mirrors the UMTsCA configuration from
Deng et al. (2021)
B.4
Faster R-CNN Losses
Here we describe the standard Faster R-CNN losses before describing how we modify them into "soft" distillation losses. Faster R-CNN consists of two stages: region proposal network and the region-of-interest heads.
B.4.1 Region proposal network (RPN):
Inputs. The RPN takes as input:
Features extracted by a backbone network (e.g. Resnet-50 with fcature pyramid network in most of our experiments).
set of anchor boxes that represent the initial candidates for detection.
Outputs. For each anchor, the RPN predicts two things:
1. A binary classification called "objectness" representing whether the content inside the anchor box is foreground or background.
2. Regrossion targets for the anchor, reprosenting adjustments to the box to more closely enclose any foreground objects.
Computing the loss. In order evaluate these predicted proposals, each proposal matched either foreground background based its intersection-ovve--niion with the nearest. ground truth box. Based
23

---

Published in Transactions on Machine Learning Research (03/2025)
SADA
PT
Leissiall Soft
Liup
Liuu
ysrc
Pare
Pigl
ysrc
Parc
Pigt
Pigt
SG
Lalign SADA
Softmax, Sum
Student Oen
Student
Outr
EMA
Teacher Ozch
are
Fight
Fight
Flip, Jitter. Cutout
Egg
Flip
Flip
Flip
Flip. Scale
Source
Torre
Target
Fege
Source
Terc
Target
Fzgs
Fere /= Marc =24
1:od== / Meye =24
Purr - Marc 116
PFrel Megt
=32
UMT
MIC
Laistill Hard
Las Hard
Loup
Loup
Dare
Parc
Ptgt
Pegt FsG SG
Yare
Perc
Ptst
Ptgt FsG
Ladign Img22mg
Thresh, NMS
Lationn SADA
Thresh, NMS
Student
Outr
EMA
Teacher Bech
Student
Ost..
EMA
Teacher Bach
Ligl
Ligl
Figh
Flip, Jitter, MIC
lg!
CropPad. litter
Plip. Scale
Flip. Scale
Flip
Source
Ferre
Target
Source
Tore
Target
Agt
| Mare 24
Piyd- Mege
Fare More
Flyel |
AT
ALDI+
Laistan Hard
Lodistiii Soft
Teeup
Loup
Usrc
Pare
Pigt
Ptge
Yere
Porc
Ptgt
Pegt SG
SG
Tratign Image
Thresh, NMS
Softmax
Student Osca
Student Ostu
EMA
Teacher Oceb
EMA
Teacher Pcch
Figh
Flip. Jitter, Curout
Egg
Fogg
Flip. Scale, Jitter. MIC
Eigs
Flip. Scale
Flip. Scale
Flip, Jitter (1/2), Cutout (1/2)
Flip. Scale, Jitter, Cutout
Source
Kore
Target
Teat
Source
Iare
Target
Fagh
Fon to Mare 16
11gel= Mag
=32
=32
FRol= Myre 24
Mini= =24
Figure 11: Visual depiction of ALDI settings for reproducing prior work.
24

---

Published in Transactions on Machine Learning Research (03/2025)
these matches, in the Detectron2 default settings binary cross-entropy loss is computed for (1) and smooth I.1 loss is computed for (2).
A key challenge in Faster R-CNN is the severe imbalance between foreground and background anchors. To address this, a smaller number of proposals are sampled for computing the loss (256 in the default settings) with a specified foreground ratio (0.5 in the default setting). Objectness loss computed for all proposals, while the box regression loss is computed only for foreground proposals (since it is undefined how the network should regress background proposals).
B.4.2 Region of interest (ROI) heads:
Inputs. The ROI heads take input:
1. Proposals from the RPN. In training, these are sampled at desired boreground/backgroone ratio, similar to the procedure used for computing the loss in the RPN. Note, however, that these will be different proposals than those used to compute RPN loss. the Detectron2 defaults, 512 RPN proposals are sampled as inputs to the ROI heads at foreground ratio of 0.25.
Cropped backbone features, extracted using a procodure such as ROTAlign (e et al. 2017 These are the features in the backbone feature map that are "inside" each proposal.
Outputs. The RO1 heads then predict for each proposal:
1. A multi-class classification.
Regression targets for the final bounding box, representing adjustments the box to more closely enclose any foreground objects.
Computing the loss. Predicted boxes are matched with ground truth boxes again based on intersection- over-union in order to compute the loss. By default we compute cross-entrope loss for (1) and smooth L1 loss for (2). (2) is again only computed for foreground predictions.
B.5 Soft Distillation Losses for Faster R-CNN
Distillation losses are computed between teacher predictions and student predictions. One option is select the teacher's most confident predictions based on a confidence threshold parameter to be "pscudo-labels" These take the place ground truth boxes the standard Faster R-CNN losses for the student. We refer this approach as using "hard targets."
In contrast, here describe how compute "soft" losses using intermediate outputs from the teacher to guide the student without thresholding.
RPN. The teacher and student RPNs start with the same anchors. We use the same sampling procedure described in B.4.1 choosing proposals for loss computation. Importantly, we ensure the same proposals are sampled from the teacher and student so that they can be directly compared. We postprocess the teacher's objectness predictions with a sigmoid function to sharpen them. We then compute a binary cross-cntropy loss between the teacher's post-sigmoid outputs and student's objectness predictions. We also compute a smooth L1 loss between the teacher's RPN regression predictions and the student's RPN regression predictions. Regression losses are only computed on proposals where the teacher's post-sigmoid objectness score is 2 0.8.
ROI heads. The second stage of Faster R-CNN predicts a classification and regression for each RPN proposal; therefore, we need the input proposals to the student and teacher to be the same in order to directly compare their outputs. To achieve this, during soft distillation we initialize the student and teacher's ROI heads with the student's RPN proposals-intuitivee we want the teacher to tell the student "what to do with" " its proposals from the first stage.
25

---

Published in Transactions on Machine Learning Research (03/2025)
We postprocess the teacher's classification predictions with softmax to sharpen them, then compute cross- entropy loss between the teacher's post-softmax predictions and the student's classification predictions. We also compute a smooth L1 loss between the teacher's regression predictions and the student's 's regression predictions. We only compute regression losses where the teacher's top-scoring class prediction is not the background class.
B.6 Adversarial Feature Alignment
We implement two networks to perform adversarial alignment at the image level and instance (bounding box)
level. Our approach inspired by Faster R-CNN in the Wild (Chen et al. 2018) and SADA (Chen
et al.
2021
Image-level alignment. We build an adversarial discriminator network that takes in backbone features the image level. By default the "p2" layer of the feature pyramid network described in Lin et al.] 2017). We use simple convolutional head consisting of one hidden layer. Our defaults result in this torch module:
ConvDiscriminator(
(model): Sequential
(0): Conv2d 256,
xernel_size=(3, 3),
stride=(1, 1))
(1): ReLU()
(2): kdaptivekvgPPo1226oupppp siie=_s
(3): Datten(start_dim-.) and_din=-1)
(4): Linear felt_ffeature=266,
out out_featee fat_features-1,
bias-True)
)
Instance-level alignment. We also implement an instance-level adversarial alignment network that takes as input the penultimate layer of the ROI hoads classification head. By default, our instance level discrimi- nator consists of one hidden fully-connected layer. Our defaults result in this torch module:
FCDiscriminator((
(model): Sequential(
(0): aattee(start_ddmn=s and_din==1)
(1): Linear (in_ fen_feaaurre=1024,
out_features-1024,
bias=True)
(2): ReLU()
(3): Linear (in_ features=1024, out_foatures=l,
bias-True)
C Experiment Details
C.1 Backbone Pretraining
In our experiments, we evaluale two different backbones: a ResNel-50 (He el al. 2016) with Feature Pyramid Network (Lin et al., 2017), and a VIT-B (Dosovitskiy et al. 2020) with ViT (Li et al. 2022a). Both backbones are pre-trained on the ImageNet-1K classification and the COCO instance segmentation (Lin) et al.) (0144) tasks. In addition, the ViT-B backbone also pre-trained using the masked autoencoder objective proposed in He et al. (2022)
26

---

Published in Transactions on Machine Learning Research (03/2025)
CS FCS
Sim10k Cs
CFC Kenai Channel
Targpt-
Source
Target-like
Target
Storrce
Target
Souree-lile
Figure 12: Exemplary results of our CycleGAN models. Source and target are the original images. Target-like and source-like images translated by CycleGAN. Since FCS is derived from CS, CS - FCS is the only case in which we have paired images and therefore show the translation from source into target-like and from target into source-like for the same example.
C.2 Image-to-lmage Translation
In contrast to the adversarial alignment in feature space in SADA (Chen al. 2021). UMT (Deng al. 2021 aligns the domains in image (i.e. pixel) space. This is achieved by training and using unpaired amage-to-image translation model to try to transform images from the source dataset into images that look like images from the target dataset ("target-like") and vice-versa ("source-like"). We follow Deng et al. (021 by using the CycleGAN (Zhu et al. 2017) image-to-image translation model. We train the CycleGAN for 200 epochs (Cityscapes <> Foggy Cityscapes, Sim10k ++ CS) or 20 epochs (Kenai Channel) and respectively select the best model according to the average Fréchet inception distance (FID) (Heusel al. 2017 between the source & source-like and the target & target-like images in the training dataset. For FID computation, use the clean-fid implementation proposed in Parmar et al. (2022). We compute FID the training datasots as UMT only uses translated images thercof, which is why we are only interested in the best fit on the training data. We follow Deng el al. (2021) by then generating source-like and larget-like dataset using the selected model ahead of time, before the training of the main domain adaptation method. We note that tuning CycleGAN's hyperparameters or using other imago-to-image translation methods could possibly improve UMT's performance however for the fair reproduction we use the defaults, We show some exemplary results of CycleGAN models that used to train UMT (Deng et al. 2021 in Fig 12
27

---

Published in Transactions on Machine Learning Research (03/2025)
C.3 Other Training Settings
We fix the total effective batch size 48 samples for fair comparison across all experiments. For training, perform each experiments 8 Nvidia V100 (32GB) GPUs distributed over four nodes. We use the MIT Supercloud Reuther et al. 2018
D CFC-DAOD Dataset Details
Like other DAOD benchmarks, CFC-DAOD consists of data from two domains, source and larget.
D.1
Source data
In CFC-DAOD, the source-domain training set consists of training data from the original CFC data release, i.., video frames from the "Kenai left bank" location. We have used the 3-channel "Baseline+ format introduced in the original CFC paper (Kay et al. 2022). For experiments in the ALDI paper, we subsampled empty frames to be around 10% of the total data, resulting in 76,619 training images. For reproducibility, we release the exact subsampled set. When publishing results on CFC-DAOD, however, researchers are allowed to use the orignial CFC training set however they see fit and are not required to use our subsampled "Basclinc++" data.
Train:
Validation: The CFC-DAOD Kenai (source) validation set is the set. We use the 3-channel "Baseline+4" format from the original CFC paper. There are 30,454 validation images.
the original CFC validation
D.2 Target data
Train: In CFC-DAOD, the target-domain "training" set consists new data from the "Kenai Channel" location in CFC. These frames should be treated as unlabeled for DAOD methods, but labeled for Oracle methods. We also use the "Baseline+++" format, and use the authors' original code for generating the image files from the original video files for consistency. There are 29,089 target train images.
Test: The CFC-DAOD target-domain test set is the same as the "Kenai Channel" test set from CFC. We use the "Baseline++4" format. There are 13,091 target test images. Researchers should publish final mAP@lou=0.5 numbers on this data, and may use this data for model selection for fair comparison with prior methods.
E The ALDI Codebase
We release ALDI as open-source codebase built modern detector implementation. The codebase is optimized for speed, accuracy, and extensibility, training to 5x faster than existing DAOD codebases while requiring to 13x fewer lines of code. These qualities make our framework valuable for practitioners developing detection models real applications, as well as for researchers pushing the saate-of-the-ate in DAOD.
E.1 Detection Framework
We designed the ALDI codebase to be lightweight and extensible. For this reason, we build top of recent version of Detectron2 (wu et al. 2019). The last tagged relcase of Detectron2 was v0. 6 in November 2021, however there have been a number upgrades since then leading to state-of-the-art performance. Thus, we use a fixed version that we call v0. 7ish based off of an unofficial pull request for v0.7, commit 7755101 dated August 30 2023. We include this version of Detectron2 as pip-installable submodule in the ALDI codebase for now, noting that once the official version is released it will no longer need to be a submodule (ie it will be able to be directly installed through pip without cloning any code).
28

---

Published in Transactions on Machine Learning Research (03/2025)
Open-source codebases in domain adaptive object detection. Existing methods use a variety of different detector implementations, including deprecated frameworks (maskreun--ennchmrrt) and versions (Detectron2 < v0.6). In contrast, ALDI is built on lop of modern framework, optimized for training speed, and is able reproduce all five of these methods while requiring fewer lines of code (LOC) than any individual existing implementation. Our codebase can serve strong starting point for future research.
Table 11:
Faster R-CNN Implementation
Codebase
Backbone
Input size
LOC
UMT
Deng et al.
2021
faster-renn.pyton pytorch
VGG-16
600px
19k
SADA
Chen el al. 2021
maskrenn-benehhmare
Resnet50-FPN
800px
7k
PT
Chen et al. 2022
Detectron2 v0.5
VGG-16
600px
3.4k
MIC
Hoyer et al. 2023
maskrronn-benhhmarkk
Resnet50-FPN
800px
20k
AT
Li et al.)
2022b
Detectron2 v0.3
VGG-16
600px
4k
ALDI (Ours)
Detectron2 ~v0.7
Resnet50-FPN
1024px
1.5k
Our codebase makes no modifications the underlying Detectron2 code, making it a lightweight standalone framework. This is in contrast to existing DAOD codebases (see Table [1] that often duplicate and modify the underlying framework as part of their implementation. By building top of Detectron2 rather than within it, our codebase is up to 13x smaller than other DAOD codebases while providing more functionality. We note that in Table 11 other codebases implement a single method while ours supports all methods studied.
E.2 Speedups
We found significant bottlenecks in training in other Detectron2- based codebases. Notably, we found that dataloaders and transform implementations were incfficient. These included, for instance:
Converting tensors back and forth between torch, numpy, and PIL during augmentation. We ad- dressed this, reimplementing transforms as needed so that everything stays in torch.
Using the random hue transform from torchvision. We found minimal changes in performance from disabling this component of the ColorJitter transform.
Using separate dataloaders for weakly and strongly augmented imagery. We instead usee a single dat- aloader per domain, with a hook to retrieve weakly augmented imagery before strong augmentations are performed.
We reimplemented the dataloaders and augmentation strategics used by AT, MIC, and others to be more efficient, leading to 5x speedup training time per image compared to AT.
F Tabular Results
G Qualitative Results
We visualize predictions from all models Cityscapes - Foggy Cityscapes in Fig. 13 Sim10k Cityscapes in Fig. and CFC-DAOD in Fig. 15
Cityscapes + Foggy Cityscapes We choose random frame from each city in the validation set (Munster, Lindau, and Frankfurt) and show the 0.005 and 0.02 fog levels. We see that, compared to ALDI++, UMT and PT consistently suffer from more false positive detections, while other methods display both false positive and false negative detections. AT and MIC perform qualitatively similary ALDI+ many cases. AT suffers from more false positives than ALDI+ in all locations but fewer false negatives in Lindau. MTC
29

---

Published in Transactions on Machine Learning Research (03/2025)
Table 12: Results with ResNet50-FPN backbones and ImageNet pre-training. Previously- published results are shown in gray. Best results for each benchmark are in bold and second-best are underlined.
Method
CS + Foggy CS
Sim10k -> CS
CFC Kenai - Channel
Source-only
(23.5) 51.9
(35.5) 61.4
67.3
UMT
(Deng et al.
2021
(41.7) 56.7
(43.1) 41.8
64.9
SADA
CChnn al. 2021
(44.0) 42.9
55.7
61.6
PT
Chen al. 2022
(47.1) 51.5
(55.1)
64.3
67.0
MIC
Hoyer et a.. 2023
(47.6) 54.0
64.3
71.9
AT
Li ct al.
20226
(50.9) 61.0
56.7
69.2
+LDI++ (Ours)
63.9
69.1
72.6
Oracle
(42.7) 62.4
(66.4) 83.3
77.1
Table 13: Results with ResNet50-FPN backbones and COCO pro-training, Best results for cach benchmark are in bold and second-best are underlined.
Method
CS + Foggy CS
Sim10k - CS
CFC Kenai - Channel
Source-only
59.1
76.8
66.7
UMT
(Deng et al. 2021
61.4
58.7
61.2
SADA
Chen et a 202
54.2
71.8
58.9
PT
Chen et al. 2022
59.2
70.6
69.0
MIC
Hoyer et
2023
61.7
73.1
75.5
AT
Li
al.
2022b
63.3
72.0
69.1
+ (Ours)
66.8
77.8
76.1
Oracle
67.2
86.1
73.8
Table 14: Results with VitDet-B backbones and ImageNet pre-training. Best results are in bold and sccond-best are underlined.
Method
CS + Foggy CS
Source-only
68.1
UMT
Deng et al. 2021
63.9
SADA
Chen et a..
202
54.4
PT
Chen et al. 2022
65.8
MTC
Hoyer et
2023
70.6
AT
(Li al.
2022b
66.6
+LDI+ (Ours)
71.0
Oracle
74.5
performs better than ALDI+ in this randomly-selecteed Lindau frame. In the Frankfurt example, see one downside of the Foggy Cityscapes benchmark: since detection annotations are generated programatically from segmentations, strange false positives can exist in the ground truth. Our new dataset CFC-DAOD addresses this problem by focusing directly on object detection.
Sim10k - Cityscapes In Fig. 14 we compare results of all methods on the Sim10k -> Cityscapes benchmark on two random images from each location in the Cityscapes validation set. In Munster, we see that all methods, including ALDI++, struggle with differentiating overlapping cars in the first image. In the second
30

---

Published in Transactions on Machine Learning Research (03/2025)
image, ALDI and MIC outperform other methods in differentiating cars in groups with less overlap than the first example. In Lindau, most prior work exhibits significantly more false negatives than - and AT, AT, however, merges detections of multiple cars, leading to false negatives as well. In Frankfurt, most methods again exhibit significant false negatives, while ALDI++ and MIC show small false positive detections. Across all locations, MTC and SADA predict large false positives caused by foreground street lines.
CFC-DAOD In Fig. 15 compare results of all methods the CFC-DAOD Kenai + Channel benchmark. We select two random images from each camera view in the test set. These camera views represent different range windows of the sonar camera, with Stratuml the nearest-range (leading to the highest resolution and largest fish), followed by Stratum2, and finally Stratum3 is the longest-range (lowest resolution and smallest fish). We can see that each Stratum exhibits its own challenges differentiating fish from the background, dealing with low signal-to-noise ratios, and differences in target size. Stratuml, all methods except. SADA detect the easily identifiable large fish in the first frame, but all methods suffer from the same false negative in the second frame. +LDI+ and UMT predict the same false positive In Stratum2, all methods suffer from false positives caused by background texture, and UMT, SADA, and PT suffer false negatives. In Stratum3, all methods fail to detect the furthest-range fish which are highly occluded by background texture, but exhibit fewer false positives than the other strata.
the edge of the field of view.
UNT
SADAA
MIC
AUDI
Figure 13: Qualitative results Foggy Cityscapes. Best viewed maginified.
31

---

Published in Transactions on Machine Learning Research (03/2025)
UMT
103
Figure 14: Qualitative results on Sim10k + Cityscapes. Best viewed maginified.
32

---

Published in Transactions on Machine Learning Research (03/2025)
Stratum 1
Straturn2
Stratum3
UMT
SADA
PT
MIC
AT
ALDI+
GT
Figure 15: Qualitative results on the CFC-DAOD Lest set.. Best viewed maginified.
33


Reviews:
[{"review_id": "Mwihkb7T0A", "reviewer": "Reviewer_kcu1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### **Strengths**:\n**[Clear Motivation and Problem Statement]** The paper provides a straightforward and well-motivated problem statement. The figures illustrate the system architecture, helping readers quickly grasp the overall workflow, and the authors highlight known issues in DAOD benchmarks that have long been recognized in the community.\n\n**[Comprehensive Ablation Study]** In Section 6.3, the paper provides a wide-ranging ablation study for the ALDI++ model, examining factors such as source and target augmentation, batch composition ratio, multi-task soft distillation, and feature alignment. This thorough investigation reveals valuable insights into the framework’s design and is comprehensive.\n\n**[Unification of Multiple DAOD Methods]** The ALDI framework successfully integrates multiple DAOD approaches into a single pipeline for fair, streamlined benchmarking. Its core design space, including supervised training, target-domain distillation, and adversarial alignment components, is technically reasonable.\n\n**[Improved Baseline Protocol (P1)]** The authors devote significant attention to constructing a fair baseline model by comparing source-only and target-supervised methods. This step is often overlooked yet critical, as huge improvements can be observed from inadequate baselines.\n\n---\n### **Major Weaknesses:**\n\n**[Weak Claims Regarding Benchmark Diversification P3(1)]** Authors have mentioned in P3 that DAOD benchmarks have focused on urban driving scenarios, thereby giving rise to an assumption that methods generalize across different domains and architectures. While it is true that the field has largely relied on driving benchmarks (due to their availability and standardization), it is not necessarily the case that prior work assumes generalizable applicability. Furthermore, although the authors highlight this gap, they only introduce one additional dataset component -- the fish detection setting -- which may not be sufficient to fully address the stated need for broader domain coverage.\n\n**[Weak Claims on Outdated Backbones P3(2)]** Authors have claimed in P3 that reliance on older architectures hamper the perceived generality of DAOD methods. However, their own approach still uses Faster R-CNN, which is considered a somewhat dated backbone per se. Although a number of other methods have explored more advanced architectures (e.g., DETR, ViT-based detectors), the manuscript does not provide a concrete plan to move beyond the standard practice. As a result, the argument that most methods use outdated backbones remains unconvincing without a proposed alternative.\n\n**[Insufficient Visualization]** The paper doesn't include any major big figure showing the domain adaptation object detection resuls. More comprehensive visual results, possibly in supplementary material would provide clearer evidence of the system’s capabilities. Without additional visual examples, it is difficult to assess the method’s performance, generalization, and consistency in more diverse scenarios. \n\n---\n### **Minor Weaknesses**\n**[Modality-Specific Scope]** The title and introduction refer broadly to \"Domain Adaptive Object Detection,\" but the paper deals exclusively with 2D detection using camera images. Given the importance of 3D detection in many modern driving applications, the authors should explicitly state that this work focuses on image-based 2D object detection.\n\n**[Broken URLs in Abstract]** Two anonymous URLs provided in the abstract were no longer accessible at the time of review.\n\n**[Missing Failure Case Analysis]** Although the paper describes strong results, it lacks a dedicated section on failure cases. Presenting examples of challenging or problematic scenarios, along with insights into the method’s shortcomings, would have provided a more complete picture of the model's capability.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a unified benchmarking and implementation framework—Align and Distill (ALDI), to address the systemic pitfalls in domain adaptive object detection (DAOD). The authors identify that past results often overestimate performance due to underpowered baselines, inconsistent implementations, and reliance on outdated backbones. In response, they propose a modern DAOD evaluation protocol and introduce a new benchmark dataset (CFC-DAOD) that better reflects real-world diversity. They also present a new method, ALDI++, which achieves state-of-the-art results on multiple benchmarks and underscores the importance of careful model initialization and training pipelines. Despite these advances, the authors find that no DAOD method consistently reaches “oracle-level” performance across all architectures and pre-training strategies, revealing substantial room for improvement. They conclude that real-world benchmarks, transparent comparisons, and reliable validation procedures are essential for further progress, offering their framework, dataset, and approach as a critical reset for DAOD research.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Below is a list of suggested revisions**:\n1. Provide additional justification or examples to demonstrate how the new fish detection setting materially broadens the scope of DAOD benchmarks. \n2. While Faster R-CNN is a standard choice, consider adding preliminary results or discussions on more modern backbones (e.g., DETR or ViT-based) to solidify the claim that your method generalizes well beyond a traditional architecture.\n3. Comprehensive visualizations showing detection results in both typical and challenging scenarios.\n4. Failure case analysis, presenting visualizations of common failure cases.\n5. Other weaknesses mentioned in the section above.\n\nAddressing these points would significantly improve the paper’s clarity and contribution.", "broader_impact_concerns": "I do not observe obvious ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "83f9x3Nqht", "reviewer": "Reviewer_vcow", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper is well-written, clearly identifying the issue and its underlying causes.\n2. This paper effectively clarifies the inconsistencies in training and evaluation protocols used in prior works. By isolating adaptation-specific techniques to create fairer baselines, it clearly exposes the incremental progress of previous methods.  I believe such results help the community to re-think the commonly adopted methodologies and review criteria in this field.\n3. The new dataset proposed by this paper is indeed novel and enhances benchmarking diversity.\n4. This paper demonstrates that two simple techniques outperform all previous methods. The results are convincing, especially when compared against modernized baselines.\n\n---\n\nWeaknesses\n\n1. This paper claims to modernize the baseline architectures (P1), however, the detection framework (Faster-RCNN) is in fact obsolete, contradicting to its claim.\n2. $T_{strong}$ in Section 4.1 is not defined properly\n3. In Figure 6, it seems that stronger baseline architectures (ViTDet-B vs. R50-FPN) have less domain gap.  As this paper has shown that prior works may be sub-optimal, it's unclear if the proposed ALDI++ helps when using using state-of-the-art baseline architectures, such as ConvNext-L or CLIP/ViT-L.  One might conjecture that there would be very little or even no domain gap.\n4. This paper does not explain why ALDI++ outperforms oracle when using R50-FPN (ImageNet pre-train).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper identifies key benchmarking issues in Domain Adaptive Object Detection, such as outdated architectures, inconsistent training, and limited benchmark diversity, which have led to inflated performance claims and incremental progress. To address this, this paper introduces modern baseline architectures, re-implements training and evaluation protocols, and establishes a new benchmark dataset. With fair comparisons, this paper shows prior methods may be instead sub-optimal, highlighting the field's slow advancement.   Finally, this paper proposes two general techniques that significantly outperform previous works.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Authors are encouraged to conduct experiments over modern \"detection\" framework, such as DINO[1]\n2. Authors should define $T_{strong}$ properly.\n3. Authors are encouraged to conduct experiments over state-of-the-art visual backbones, such as ConvNext-L[2] or CLIP/ViT-L[3]\n4. Authors are encouraged to explain why ALDI++ outperforms oracle model when using R50-FPN (ImageNet pre-train)\n\n\n---\n\nReference:\n\n[1] DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection.  Zhang et al.\n[2] A ConvNet for the 2020s.  Liu et al.\n[3] OpenCLIP.", "broader_impact_concerns": "There's no ethical implications of the work", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8aYv5XhcVk", "reviewer": "Reviewer_6uSg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. ALDI creates a fair benchmarking framework for domain adaptive object detection (DAOD) methods, helping to understand whether performance improvements are due to domain adaptation techniques or other factors, an ambiguity that characterizes DAOD methods.\n2. The authors re-implement 5 methods for DAOD, which proves the possibility of using this framework for future work.\n3. The authors introduce ALDI++, a new method that achieves state-of-the-art performance on three benchmarks. They also introduce a new dataset for evaluating the generalizability of DAOD methods.\n4. The authors' claims are supported by extensive experiments.\n\nWeaknesses\n1. While implementing DAOD methods within a single framework like Detectron2 has its advantages, I wonder how difficult it would be to use the ALDI framework with an object detection model not supported by Detectron2, such as the one-stage object detector YOLO.\n2. The authors acknowledge using a target-domain validation set for model and hyperparameter selection, but it is unclear which dataset was used for this purpose. Additionally, it is unclear the advantage of tuning ALDI++ on this dataset, especially compared to the re-implemented methods, which were not tuned, even though some design choices (such as input resolution and batch size) were modified compared to the original implementation.\n3. It is unclear why the ablation study does not use the final model. For example, the default setting used in the ablation study involves hard thresholding, rather than soft thresholding, which is one of the advantages introduced by ALDI++.\n4. In Figure 10, the authors compare DAOD methods to their source-only model. They also mention that SADA does not use EMA during training. Given this, is it fair to compare methods that do not use EMA with the source-only baseline used in the paper?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Align and Distill (ALDI), a framework that unifies common components of domain adaptive object detection (DAOD) methods to address past benchmarking flaws, ensuring a fair comparison between methods. Specifically,  ALDI compares five DAOD methods that use self-training and feature alignment. The authors also introduce ALDI++, a new method that achieves state-of-the-art results by including three improvements: (i) a strategy to improve out-of-distribution generalization before self-training, (ii) the use of soft distillation losses instead of hard ones, and (iii)  a revised training approach with strong regularization on both target and source data, and a balanced number of source and target samples in each minibatch.\n\nThe authors show that some evaluated DAOD methods overestimate their improvement compared to their source-only model. This source-only model is specifically designed to exclude performance gains unrelated to DAOD methods by removing only the components that involve target data. To further investigate the generalizability of DAOD methods, the authors also introduce an extended version of the Caltech Fish Counting Dataset for the DAOD task.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The following proposed adjustments could further strengthen an already solid work:\n1. The authors could include a discussion on the limitations of the proposed ALDI framework, particularly regarding which methods can be re-implemented.\n2. It would be helpful to clarify whether the validation set used to tune the design choices is FCS as for the ablation study. Additionally, the authors could report the main hyperparameters of the five methods re-implemented in the ALDI framework compared to their original implementation.\n3. The authors should clarify the categories of methods that can be compared to the source-only baseline proposed in the paper.", "broader_impact_concerns": "No concern about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper delves deep into the problem of domain adaptive object detection, pointing out issues and presenting good insights with rigorous, fair benchmarking. Reviewers commended the authors’ clarity in unifying existing methods within one framework, revealing the true sources of performance gains, and demonstrating how their new fish-counting dataset can highlight inconsistencies that were overlooked by driving-centric benchmarks. They found the experiments thoroughly convincing and praised the authors’ attention to detail in ablation studies, which illustrate why certain components matter more than others.

All reviewers unanimously recommended acceptance, emphasizing the high quality and clear exposition of the work. They noted that ALDI++ achieves state-of-the-art results with a well-executed evaluation across multiple architectures and backbones, reflecting both strong methodology and the potential to inspire future research. Given these strengths, the paper will be valuable to TMLR’s audience by improving transparency and best practices in DAOD, and by offering novel empirical insights that help the community move beyond poorly established baselines.

Although the inclusion of fish counting data enhances the diversity of existing DAOD benchmarks, a possible direction for future work would be extending these efforts to other scenarios (for instance, sim-to-real generalization in robotics). This would further strengthen the paper’s contribution by exploring a broader range of real-world domains, complementing the current focus on driving scenes and underwater imagery.
Decision: Accept (poster)