Diffusion Domain Teacher: Diffusion Guided Domain Adaptive Object Detector

Authors: boyong  he
yuxiang  ji
zhuoyue  tan
liaoni  wu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 16:51:13

Abstract: Object detectors often suffer a decrease in performance due to the large domain gap between the training data (source domain) and real-world data (target domain). Diffusion-based generative models have shown remarkable abilities in generating high-quality and diverse images, suggesting their potential for extracting valuable feature from various domains. To effectively leverage the cross-domain feature representation of diffusion models, in this paper, we train a detector with frozen-weight diffusion model on the source domain, then employ it as a teacher model to generate pseudo labels on the unlabeled target domain, which are used to guide the supervised learning of the student model on the target domain. We refer to this approach as Diffusion Domain Teacher (DDT). By employing this straightforward yet potent framework, we significantly improve cross-domain object detection performance without compromising the inference speed. Our method achieves an average mAP improvement of 21.2% compared to the baseline on 6 datasets from three common cross-domain detection benchmarks (Cross-Camera, Syn2Real, Real2Artistic), surpassing the current state-of-the-art (SOTA) methods by an average of 5.7% mAP. Furthermore, extensive experiments demonstrate that our method consistently brings improvements even in more powerful and complex models, highlighting broadly applicable and effective domain adaptation capability of our DDT.

Main paper: 
 introduction: 
 Object detection is a fundamental task in computer vision, with its applications permeating an array of real-world scenarios. There have been impressive strides and significant achievements in object detection, leveraging both Convolutional Neural Networks (CNNs) [18, 42, 53, 65] and transformer-based models [4, 85] . Nonetheless, these data-driven detection algorithms wrestle with the challenging issue of domain shift: the large gap between the training data (source domain) and the testing environments (target domain) frequently results in a substantial decline in detection accuracy. This obstacle is ubiquitous across various sectors, including robotics, autonomous driving, and healthcare, and it poses a formidable barrier to the widespread applications of object detection in practice. Consequently, the deployment of domain adaptation techniques, aimed at minimizing domain disparities, has become essential to boost the robustness and generalizability of models across diverse environments.Unsupervised Domain Adaptation (UDA) methodologies have surged to the forefront of research, taking advantage of the sparse labeled data from the source domain in conjunction with copious unlabeled data from the target domain to significantly enhance cross-domain detection performance. Current UDA tactics have explored a variety of strategies including domain classifiers [10, 25, 59, 81, 84] , graph matching [36, 38, 39] , domain randomization [32] , image-to-image translation [26] , and self-training frameworks [6, 14, 41, 56] . These techniques have been crucial in achieving notable advancements in cross-domain object detection.Moreover, diffusion-based generative models [24, 54, 61] have showcased remarkable capabilities in generating high-quality and diverse images, signaling their vast potential for a spectrum of downstream applications. Some works [1, 66, 71] have already harnessed diffusion models for a breadth of tasks. This evidence suggests a promising avenue for employing these models to bolster cross-domain detection efficacy. Nevertheless, the step-by-step inference process of these models is not fast enough to meet the immediate processing needs of object detection. Although there 27.9 27.9 27.5 27.522.5 22.5 32.9 32.928. 1 Figure 1 : Evaluation results on Clipart [28] during training.It is evident that the performance of the student significantly improves after entering self-training, even surpassing the diffusion teacher, and the mean teacher exhibits better performance compared to the student.has been some effort to adapt diffusion models for image generation and manipulation, as seen with tools like LoRA [27] and Control-Net [76] , there is a lack of research on applying diffusion models to cross-domain detection. Fortunately, current UDA methods provide us with valuable insights. Specifically, we draw inspiration from previous state-of-theart (SOTA) approaches [3, 14, 41] and adopt the Mean Teacher [64] self-training framework, where the teacher model generates pseudo labels for the supervised learning of student model on the target domain. The weights of the teacher model are typically updated through Exponential Moving Average (EMA) by the student model. This consistency-based self-training approach allows the student model to progressively learn from the target domain, thereby improving the performance of the detector in cross-domain detection.In our approach, we freeze all parameters of the diffusion model and extract intermediate feature from the upsampling structure of the U-Net [55] architecture during the inversion process. These features are then passed through a bottle-neck structure to generate hierarchical features similar to a general backbone for downstream detection tasks. This enables effective training and fine-tuning of the diffusion model with a small number of parameters, and yields discriminative feature for classification and regression tasks, leading to improved performance in cross-domain detection. The mean teacher, updated through EMA from the student model, further enhances stability and generalization.Through the detector with the diffusion backbone for feature extraction struggles to match or surpass the performance of general backbones like ResNet [22] on intra-domain. However, in the target domain, the performance of the diffusion detector surpasses them greatly. This strongly confirms the diffusion model is an incredibly powerful and highly generalized feature extractor. Furthermore, it is even more remarkable that the diffusion teacher model continues to enhance the cross-domain performance of stronger backbones, all without any increase in additional inference speed.The contributions of this paper can be summarized as follows:• We introduce a frozen-weight diffusion model as backbone, which efficiently extracts highly generalized and discriminative feature for cross-domain object detection. Notably, the diffusion-based detector, trained exclusively on the source domain, demonstrates exceptional performance when applied to the target domain. • We incorporate the diffusion detector as a teacher model within the self-training framework, providing valuable guidance supervised learning of the student model on the target domain. This integration effectively enhances cross-domain detection performance without any increasing of inference time.• We achieve substantial improvements in cross-domain detection. Our method achieves an average mAP improvement of 21.2% compared with the baseline, and surpassing the current SOTA methods by 5.7% mAP. Further experiments demonstrate that the diffusion domain teacher consistently enhances cross-domain performance for detectors with stronger backbones, leading to superior results in the target domain. 
related work 2.1 object detection: 
 Object detection aims to locate and classify objects in given images. Deep convolutional neural networks [22, 60] have revolutionized this teak and is widely applied in real-world applications. Faster R-CNN [18] , a prominent two-stage detection method, employs a region proposal network (RPN) to generate candidate regions, followed by region of interest (ROI) refinement to determine the final bounding boxes and classes. Some research is focused on improving the precision and efficiency of two-stage methods [2, 51] . In addition, researchers are investigating single-stage detectors [42, 53, 67] aimed at simplifying the detection process by integrating box regression and classification. Moreover, anchor-free detectors [65, 77] that eliminate the reliance on predefined anchors have garnered significant attention. Recent research trends involve the adoption of transformer-based end-to-end detectors [4, 75, 85] , which reconceptualize the detection task as a set prediction problem, thereby obviating the necessity for traditional handcrafted components such as anchor generation and non-maximum suppression (NMS).Although object detection has made significant advancements, performance can suffer greatly due to domain shifts between training and test data. To address this issue, UDA aims to mitigate the impact of domain shifts by leveraging labeled source data and unlabeled target data. Initially, studies inspired by GANs [20] introduced domain adversarial training [17] , which minimized domain gaps by extracting invariant feature. This approach is later adapted to detection tasks [10, 25, 59, 81, 84] , resulting in notable improvements in performance on the target domain. Some methods [26] focus on reducing inter-domain differences at image level, by applying image-to-image translate like CycleGAN [83] .Recently, self-training domain adaptation methods [3, 13, 14, 41 ] have achieved better results in cross-domain object detection by optimizing the learning of pseudo labels in the target domain. For example, AT [41] combines domain adversarial training and selftraining to improve the quality of pseudo labels. UMT [13] utilizes consistency learning for a teacher model to generate high-quality pseudo labels to improve cross-domain detection. CMT [3] introduces contrastive learning to optimize the utilization and alignment of features from the source and target domain. HT [14] optimizes the generation of pseudo labels by applying consistency measures in regression and classification. Overall, self-training methods in cross-domain object detection enhance the detection performance in the target domain by improving the quality of pseudo labels, with the teacher model being updated from the student model.Diffusion models [24, 52, 54, 58, 61] have achieved impressive results in image generation, surpassing previous models like GAN [20] .With their strong generative and generalization capabilities, some research has begun to explore the potential of diffusion models in feature representation and their application to downstream tasks. For instance, DDPMSeg [1] and ODISE [71] utilize feature extracted from diffusion models for semantic and panoptic segmentation tasks, respectively. DIFT [63] and HyperFeature [49] use the diffusion model to discover correspondences in images. This inspires us to consider the application of diffusion models for improving cross-domain detection tasks. 
approach: 
 In this section, we present our Diffusion Domain Teacher (DDT) framework in detail. First, in Sec. 3.1, we review the formulation of Unsupervised Domain Adaptation Detection (UDAD). Then, in Sec. 3.2, we provide a detailed description of how the frozen-weight diffusion model serves as a feature extractor, producing hierarchical features, to adapt to the detection task. Furthermore, in Sec. 3.3, we explain the application of the diffusion teacher detector in thr self-training framework, where pseudo labels generated on the unlabeled target domain guide the supervised learning of the student model. Finally, we summarize the total training objective.To be specific, we denote a given set of 𝑁 𝑠 samples S = 𝑋 𝑖 𝑠 , 𝑌 𝑖 𝑠 𝑁 𝑠 𝑖=1as source domian, where 𝑋 𝑖 𝑠 represents an image and 𝑌 𝑖 𝑠 represents the bounding box with category labels in the respective image. Similarly, we denote the target domain data as T = 𝑋 𝑖 𝑡 𝑁 𝑠 𝑖=1 , which consists of 𝑁 𝑡 unlabeled samples. Exactly, the distribution of S and T , including the distributions of images from 𝑃 (𝑋 𝑠 ) and 𝑃 (𝑋 𝑡 ) (e.g., style, scene, weather), labels 𝑃 (𝑌 𝑠 ) and 𝑃 (𝑌 𝑡 ) (e.g., the shapes, sizes, and density of instance), and even the scales of 𝑁 𝑠 and 𝑁 𝑡 are different, denoted as 𝑃 (S) ≠ 𝑃 (T ), is what we refer to as a cross-domain detection problem. Furthermore, relying solely on supervised learning from the labeled source domain results in an inherent bias towards source domain in cross-domain detection. Domain adaptation for detection aims to improving the performance on the target domain by reducing the dissimilarity between S and T , seeking to a domain-invariant detector.Diffusion generative models [24, 54, 61] aim to minimize the discrepancy between the distribution of images generated by the model, denoted as 𝑃 𝜃 (𝑥), and the distribution of the training data, denoted as 𝑃 data (𝑥). During training, gaussian noise of varying magnitudes is added to the clean training data, commonly referred to as diffusion process. The diffusion process starts with a clean image 𝑥 0 from the training data and generates a noisy image 𝑥 𝑡 by mixing 𝑥 0 with noise of different magnitudes:EQUATIONwhere 𝜖 ∼ N (0, I) represents randomly sampled noise, and 𝑡 ∈ [0,𝑇 ] denotes the time step, where larger values correspond to adding more noise. The amount of noise added is determined by 𝛼 𝑡 , which is a predefined noise schedule, and ᾱ𝑡 = 𝛼 1 𝛼 2 . . . 𝛼 𝑡 . The model 𝑓 𝜃 is trained to predict the input noise 𝜖, given 𝑥 𝑡 and 𝑡, typically using structures like U-Net [55] .The iterative process of the diffusion model poses challenges when directly applied to downstream supervised tasks. We extract intermediate feature at a specific time step 𝑡 during the inversion process, and apply these features for regression and classification tasks in the detection task. Specifically, we append a input noise corresponding the time step 𝑡 to the input image, shift it to 𝑥 𝑡 and then input it along with 𝑡 into 𝑓 𝜃 to extract activation layers as intermediate feature. More specifically, we apply the intermediate feature from the four stages of the upsampling process in the denoise network U-Net [55] . For each input image, we concatenate multiple time step feature together and employ a bottle-neck structure to project the feature into hierarchical layers with a channel size of [256, 512, 1024, 2048], similar to the output of ResNet [22] , which is directly applied to the object detection task, as shown in the left side of Fig. 2 .We employ a detector that extracts feature using the diffusion model and trained on the source domain as the teacher model, denoted as . Subsequently, we optimize the student model using the pseudo labels. We introduce a hyperparameter 𝜎 as a threshold for the confidence scores of the output for the teacher model, enabling us to select more reliable pseudo labels.F diff .We define the supervised learning of the student model F stu on the source domain as follows:EQUATIONwhere RPN is used to generate potential candidate regions, and ROI performs classification and bounding box regression on these candidate regions to obtain more accurate class and bounding box predictions, denoted as cls and reg, respectively. Similarly, we define the learning of the student model in the target domain as follows:EQUATIONThen, we employ EMA to update a mean teacher model F 𝑚𝑒𝑎𝑛 by copying the weights from the student model. We define this process as follows:EQUATIONwhere 𝑡 and 𝑠 represent the parameters of F mean and F stu , respectively. By employing EMA to update the mean teacher model, we aim to create a more stable and robust model by gradually incorporating the knowledge learned by the student model over time. We select the output of F mean as result for predicting.We apply a hyper parameter 𝜆 to adjust the weights between L unsup and L sup . The final formulation of our comprehensive loss function is summarized as follows:EQUATIONIn our DDT framework, following [41] , we employ Weak Augmentation to provide target domain images to the diffusion teacher model for generating reliable and accurate pseudo labels. Simultaneously, we apply Strong Augmentation to the images as inputs to the student model, as illustrated in Fig. 2 . Specifically, Weak Augmentation includes random crop and random horizontal flip, while Strong Augmentation involves color transformations such as color space conversion, contrast adjustment, equalization, sharpness enhancement, and posterization, as well as spatial transformations such as rotation, shear, and translation of the position. 
experiments 4.1 datasets: 
 Cityscapes. Cityscapes [12] dataset provides a diverse of urban scenes from 50 cities. It includes 2,975 training images and 500 validation images with detailed annotations. The dataset covers 8 detection categories, using bounding boxes sourced from instance segmentation.BDD100K. BDD100K [73] dataset is a comprehensive collection of 100,000 images specifically designed for autonomous driving applications. The dataset offers detailed detection annotations with 10 categories.Sim10K. Sim10k [30] is a synthetic dataset comprising 10,000 rendered images simulated within the Grand Theft Auto gaming engine, specifically designed to facilitate the training and evaluation of object detection algorithms in autonomous driving systems.VOC. VOC [16] is a general-purpose object detection dataset that includes bounding box and class annotations for common objects across 20 categories from the real world. Following [41] , we combined the PASCAL VOC 2007 and 2012 editions, resulting in a total of 16,551 images.Clipart. Clipart [28] dataset comprises 1,000 clipart images across the same 20 categories as the VOC dataset, exhibiting significant differences from real-world images. Following [41] , we utilize 500 images each for training and testing purposes.Comic. Comic [28] dataset consists of 2,000 comic-style images, featuring 6 categories shared with the VOC dataset. Following [29] , we allocate 1,000 images each for training and testing.Watercolor. Watercolor [28] dataset contains 2,000 images in a watercolor painting style, with 6 categories shared with the VOC dataset. Following [41] , we use 1,000 images for both training and testing.Cross-Camera. We train on Cityscapes [12] (source domain) and validate on BDD100K [73] (target domain) to evaluate the cross-camera detection performance in diverse weather and scene conditions. We focus on the 7 same categories as SWDA [59] . (first row) and BDD100K [73] (second row).Synthetic to Real (Syn2Real). We train on Sim10K (source domain) and validate on Cityscapes [12] and BDD100K [73] (target domian)to validate the performance of synthetic-to-real detection. Following SWDA [59] , we focus on the shared category car.Real to Artistic. We train on the VOC [16] (source domain) and perform validation on the Clipart [28] , Comic [28] , and Watercolor [28] (target domains) to assess cross-domain detection performance from real-world images to artistic styles. Referring to AT [41] and D-ADAPT [29] , we respectively apply the 20, 6, and 6 shared categories between VOC and each of the Clipart, Comic, and Watercolor.Following [29, 41, 59] , we use Faster R-CNN [18] as the default detector with a ResNet101 [22] backbone pretrained on ImageNet [57] , implemented with MMDetection [8] . The training and testing sizes of images are set to (1333, 800) for Cityscapes, BDD100K, and Sim10K, and (1200, 600) for VOC, Clipart, Comic, and Watercolor. The models are trained with 20,000 steps on two 3090 GPUs, with a total batch size of 16. We employ the SGD optimizer with an initial learning rate of 0.02, following the default settings in MMDetection.In self-training, we refer to the settings in [14, 41] to apply both weak and strong augmentation on the unlabeled target domain. We employ the EMA update parameter 𝛼 of 0.999 for the mean teacher model updates and simply set the loss weight 𝜆 to 1. We train exclusively on the source domain for the first 12000 steps and then perform joint training on both the source and target domain for the remaining 8000 steps.For evaluation, we report the Average Precision (AP) for each object category and the mean Average Precision (mAP) across all categories, with applying an Intersection over Union (IoU) threshold of 0.5.In this section, we present the evaluation result of our DDT framework along with other SOTA approaches. Current cross-domain object detection methods employ different detectors [18, 45, 65, 85] , which we refer to as FRCNN, FCOS, SSD, and DDETR in our table. Furthermore, the backbones with varying depths, including ResNet-18, ResNet-50, ResNet-101 [22] , and VGG-16 [60] , are denoted as R18, R50, R101, and V16, respectively. To provide a comprehensive comparison, we report the results of our method with ResNet18, ResNet50, and ResNet101. The baseline refers to the results that only train on the source domain and test on the target domain.Cross-camera adaptation. Tab. 1 presents the results of the Cross-camera settings. Our DDT method achieved the best performance with mAP 43.4 on the target domain, surpassing the previous SOTA method HT [14] by 3.2 mAP and outperforming other methods by a significant margin. Notably, AT [41] and HT [14] utilize self-training framework, have demonstrated substantial performance improvements by enhancing the quality of generated pseudo labels. Leveraging the powerful feature representation capability of the diffusion model and its exceptional performance on [73] compared with baseline, respectively, surpassing the results of previous algorithms SWDA [59] and CDN [62] . Similarly, our method obtains improvements of 19.4, 19.7, and 19.3 mAP on Cityscapes [12] , respectively, surpassing all methods except HT [14] in Table 3 . It can be observed that detectors for synthetic-to-real detection, due to the significant differences between synthetic and real-world images, does not perform well with source data only, while our method significantly improves the cross-domain performance from Sim10K [30] to BDD100K [73] and Cityscapes [12] .Real to Artistic adaptation. In Tab. 4, 5, and 6, we show the results of real to artistic cross-domain object detection. Our results of Resnet50 and ResNet101 significantly surpass the previous best method AT [41] by 3.9 and 4.8 mAP, respectively. On Comic [28] , the results of ResNet18, ResNet50, and ResNet101 [22] greatly surpass the previous best result of AT [41] , by 2.0, 6.5, and 9.7 mAP, respectively. Similarly, our best result surpasses AT [41] by 3.8 mAP on Watercolor [28] as shown in Tab. 6 . In real to artistic benchmark, overall, we find that due to the significant differences between real-world images and artistic-style images, the cross-domain performance is poor. Compared to the baseline, our method exhibits an average relative improvement of 95%, 163%, and 48% on Clipart, Comic, and Watercolor, respectively. This indicates that real to artistic adaptation is a challenging task, and it also demonstrates that our approach we have successfully improved the cross-domain performance by reducing the gap between real and artistic domain.We conduct additional experiments to analyze the feature representation capabilities of different models. Specifically, we compared our diffusion model with powerful backbones, including ConvNext [48] , Swin Transformer [47] , VIT [15] , as well as the self-supervised method MAE [21] , pretrained on ImageNet [57] . Additionally, GLIP [33] , which is pretrained on a larger dataset and has shown promising performance on object detection benchmarks. Our objective is to investigate two questions:(1) Will the diffusion model offer better intra-domain and crossdomain feature representation? (2) Will the diffusion model serve as a better teacher?Ablation Study on the Intra-domain and Cross-domain Representation. To answer the first question, we evaluate the performance of seven models across different data settings in Tab. 7. ConvNext-Base [48] 43.2+19.9 60.2+16.8 54.6+20.4 R101 [22] Swin-Base [47] 35.1+6.8 57.5+14.1 52.9+18.7 R101 [22] VIT-Base [15] 37.4+9.1 54.8+11.4 51.1+16.9 R101 [22] MAE (VIT-Base) [15, 21] 35.8+7.5 59.0+16.2 53.2+19.0 R101 [22] GLIP (Swin-Tiny) [33, 47] 39.6+11.3 58.9+15.5 54.0+19.8 R101 [22] Diff. (Ours) 55.6+27.3 64.0+20.6 58.3+24.1ConvNext-Base [48] Diff. (Ours) 59.5+18.4 63.5+1.7 59.6+7.1 Swin-Base [47] Diff. (Ours) 46.6+14.4 63.6+9.9 58.6+13.8 VIT-Base [15] Diff. (Ours) 41.7+13.1 60.2+11.8 54.9+15.6 MAE (VIT-Base) [15, 21] Diff. (Ours) 43.1+16.7 64.6+6.7 57.3+16.4 GLIP (Swin-Tiny) [33, 47] Diff. (Ours) 49.5+9.7 64.0+4.7 59.5+9.4 Specifically, we compared their intra-domain performance in training and testing within the source domain (V→V, S→S) and target domain (Ca→Ca, Cs→Cs, B→B), and cross-domain testing (V→Ca, S→Cs, S→B) to assess their cross-domain feature representation capabilities. we find that our diffusion model performe poorly within the intra-domain, lagging behind the other six models.In cross-domain testing, our diffusion model outperformed other methods on Clipart [28] but remained inferior to ConvNext [48] and GLIP [33] . Additionally, we calculate the cross-domain metrics for each model and the results obtained from training and testing in the target domain to measure the relative cross-domain capabilities of each models, represented as "Rel. " in Tab. 7. We find that the diffusion model consistently achieves the best relative cross-domain performance.Overall, the answer to the first question may be disappointing, as the diffusion detector showes some improvement in cross-domain performance but still fall behind the detectors with stronger and large-dataset pretrained backbones.Ablation Study of Different Teachers. In Tab. 8, we present the performance of different teacher and student settings for crossdomain detection. First, we use ResNet101 [22] as the student and other models as teacher. We find that although the diffusion model performs worse than ConvNext [48] and GLIP [33] in cross-domain performance, it exhibits significantly better performance when used as a teacher model. Furthermore, when we use the diffusion model as the teacher and the other six models as students, it consistently brings large improvements. This answers our second question, confirming that our diffusion model is indeed a better teacher, even when faced with highly competent students and consistently improve their performance.Ablation Results on Diffusion and Mean Teacher. To better understand the significance of teachers in our DDT, we present the results of different teacher model settings in Tab. 9. The findings reveal that excluding the Mean Teacher and Diffusion Teacher from our method leads to an average decrease of 3.1 and 6.7 mAP, respectively. When all teachers are removed, the self-training performance experiences an average decline of 8.3 mAP. These results clearly demonstrate that both the diffusion teacher and mean teacher play crucial roles in our DDT and are indispensable for achieving better performance. Fig. 1 provides an intuitive illustration of the impacts of the diffusion teacher and mean teacher in training process.Ablation Results of Different Diffusion Settings. We report the results of the diffusion models with different time steps and save steps settings in Tab. 10. It is observed that in cross-domain detection with a larger domain gap (real to artistic), longer time steps and save steps show better results. We consider a trade-off between accuracy and efficiency and choose time steps 5 and save steps 5 as our default settings.Analysis of Feature Representation of Diffusion Model. The results in Tab. 7 and 8 further deepen our understanding of the feature representation of diffusion model and its advantages in cross-domain detection. In our view, the observed results can be attributed as: fully frozen weight and adaptation of the diffusion model with the light structure that aligns with the hierarchical feature outputs, limit its performance within the intro-domain compared to fully trainable models. However, when applied as a teacher, the diffusion model guides the student to achieve superior performance in cross-domain, surpassing even the teacher itself. We think that the improved cross-domain representation ability can be attributed to the inherent characteristics of the diffusion model as well as the advantages gained from supervised learning on the source domain. In contrast, other fully trainable teacher models often concentrate primarily on supervised learning on the source domain, resulting in homogeneous optimization and limited guidance for the student. As a result, it becomes challenging to enhance the performance of the students to the level achieved by the homogeneous teacher. These results provide compelling evidence for the advantages of the diffusion model in addressing cross-domain detection tasks.Error analysis. Error analysis on Clipart [28] reveals that false negatives, i.e., missed detections, are the main factor impacting the performance on the target domain as shown in Fig. 4 . Our method significantly reduces the number of missed detections, thereby greatly improving the performance of cross-domain detection. A representation of prediction results and feature visualization further corroborate this conclusion, as depicted in the Fig. 3 . 
conclusion: 
 In this paper, we propose a domain adaptive method based on the diffusion model to address the performance degradation caused by the large gap between the source and target domain. We employ a frozen-weight diffusion model as the backbone and extract intermediate feature in the inversion process for the detection task, which we refer to as the diffusion teacher. Subsequently, we apply diffusion teacher in the self-training framework to generate pseudo labels on the unlabeled target domain, guiding the learning of the student model. Our method significantly improves cross-domain detection performance on six datasets, achieving an average improvement of 21.2% mAP compared to the baseline, surpassing the current SOTA methods by an average of 5.7% mAP, without compromising the inference speed. Furthermore, we validate the consistent performance improvement of our method in more extensive experiments for detectors with more powerful backbones, demonstrating the strong and universality domain adaptive capability of our approach. 
acknowledgment: 
 The authors would like to thank Xiamen University and Unmanned Aerial Vehicle (UAV) Laboratory for the funding and providing with all the necessary technical support. 
 
 Appendix:  


Decision: This paper received two weak accept and one borderline accept final ratings from the reviewers. AC agrees that this paper benefits from good writing and interesting idea. However, the authors are encouraged to make the necessary changes to the best of their ability.

Reviews: review: paper_summary: paper_summary: This paper introduces Diffusion Domain Teacher (DDT) for improving cross-domain object detection. It uses a frozen-weight diffusion model trained on the source domain to generate pseudo labels for the target domain, guiding the student model's learning. DDT significantly enhances performance and demonstrates broad applicability and effectiveness in domain adaptation, consistently improving results even in complex models. The paper is well-organized, with rich experiments and significant performance improvements.
strength_weakness: strength_weakness: 1.The paper is well-structured and clearly written, making the methodology and results easy to understand.
2.The paper includes a comprehensive set of experiments, validating the effectiveness of the proposed method across multiple benchmarks.
3.The method shows substantial improvements in cross-domain object detection performance, highlighting its practical value.
limitations: limitations: 1. The fonts in the figures are not uniform. In Figure 2, some fonts are too dense, and many figures have fonts that are too small to be easily readable. It is recommended to adjust the figures for better readability and ensure they correspond closely with the text.
2. The tables do not highlight the best results, which would make it easier for readers to quickly identify the top-performing methods.
3. The paper does not discuss the limitations of the proposed method, which would be helpful for understanding potential challenges and areas for improvement.
paper_summary: paper_summary: This paper focus on cross-domain object detection. The method tackle this problem with a teacher-student framework and improve the quality of pseudo labels with a diffusion model.
strength_weakness: strength_weakness: 1. The motivation of proposed method is clear and straightforward.
2. The author conducted extensive comparison experiments on many datasets, and ahcieved sota performance. 
3. Extensive ablation study demonstrate the effectiveness of proposed method.
limitations: limitations: Although the proposed method achieves satisfactory performance on multiple domain adaptation scenarios, the paper not provide  a through analysis about the effectiveness. For instance, why the pre-trained diffusion model is more effective than ResNet, ViT?
paper_summary: paper_summary: Object detectors face performance challenges due to domain shifts. To address this, the authors propose Diffusion Domain Teacher (DDT), leveraging diffusion models' generative capabilities. DDT utilizes a frozen diffusion model trained on source data to generate pseudo labels for unlabeled target data, guiding supervised learning of a student model. This approach significantly improves cross-domain detection performance.
strength_weakness: strength_weakness: 1) Unique use of diffusion models for cross-domain object detection.
2) The teacher-student framework leveraging diffusion models is conceptually sound.
3) Extensive evaluation on multiple datasets shows significant improvements.
4) Well-written and easy to follow.
5) Applicable to cross-domain detection scenarios with limited labeled target data.
limitations: limitations: 1) The paper lacks a clear and explicit articulation of the challenges of introducing Diffusion for the proposed approach.
2) It is unclear whether the tested model refers to the student model alone or includes the mean teacher.
3) The paper has a relatively limited number of mathematical formulations, and the default parameter values in Equations (2) and (3) are not sufficiently justified.
4) The bounding boxes shown in Figure 3 for the BDD100k dataset appear imprecisely aligned, indicating potential issues with detection performance.
5) The paper fails to consider a tf baseline for comparison, which could provide further insights.
6) While mAP is used, additional metrics based on Intersection over Union (IoU) could provide a more comprehensive evaluation.
7) Some tables lack comparisons with methods published in recent years (2023, 2024), limiting the paper's ability to demonstrate state-of-the-art status.