SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection

Authors: yuxuan  li
xiang  li
weijie  li
qibin  hou
li  liu
ming-ming  cheng
jian  yang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 04:11:41

Abstract: Synthetic Aperture Radar (SAR) object detection has gained significant attention recently due to its irreplaceable all-weather imaging capabilities. However, this research field suffers from both limited public datasets (mostly comprising <2K images with only mono-category objects) and inaccessible source code. To tackle these challenges, we establish a new benchmark dataset and an open-source method for large-scale SAR object detection. Our dataset, SARDet-100K, is a result of intense surveying, collecting, and standardizing 10 existing SAR detection datasets, providing a large-scale and diverse dataset for research purposes. To the best of our knowledge, SARDet-100K is the first COCO-level large-scale multi-class SAR object detection dataset ever created. With this high-quality dataset, we conducted comprehensive experiments and uncovered a crucial challenge in SAR object detection: the substantial disparities between the pretraining on RGB datasets and finetuning on SAR datasets in terms of both data domain and model structure. To bridge these gaps, we propose a novel Multi-Stage with Filter Augmentation (MSFA) pretraining framework that tackles the problems from the perspective of data input, domain transition, and model migration. The proposed MSFA method significantly enhances the performance of SAR object detection models while demonstrating exceptional generalizability and flexibility across diverse models. This work aims to pave the way for further advancements in SAR object detection. The dataset and code is available at \url{https://github.com/zcablii/SARDet_100K}.

Main paper: 
 introduction: 
 Synthetic Aperture Radar (SAR) [57; 60] is a pivotal technology in remote sensing, providing numerous advantages over traditional optical sensors. Notably, SAR possesses the capability to acquire geographical images under any weather conditions, irrespective of factors such as sunlight, land cover, or certain types of camouflage, as demonstrated in Fig. 1(a) . As a consequence of these advantages, SAR has found extensive applications in critical domains, including national defence [48] , humanitarian relief [3; 68] , camouflage detection [19] , and geological exploration [51; 24] .With its invaluable benefits, the field of SAR object detection has garnered increasing attention. In recent years, there has been a substantial increase in the number of research papers focusing on this field, as illustrated in Fig. 1(b) . Despite the increasing influence, this research area has suffered from significant challenges including limited resources and transferring gaps.Limited resources. A significant obstacle in high-resolution SAR image object detection is the sensitivity of SAR images, coupled with the high costs associated with annotating these images. This severely restricts the availability of public datasets. Existing datasets, such as SAR-AIRcraft [90] , Air-SARShip [76] , SSDD [84] , and HRSID [71] , typically consist of a singular type of object against a simplistic background. Moreover, these datasets are generally limited in scale, potentially introducing bias when evaluating different methodologies. Additionally, a notable barrier to advancing research in SAR object detection is the lack of publicly accessible source code, making it challenging to reproduce previous research findings and conduct fair comparisons or build upon existing work.To address this problem, we merge the most publicly available SAR detection datasets. This effort includes a comprehensive review of current public SAR detection resources, followed by the collection and standardization of these datasets into a uniform format, creating a unified large-scale multi-class dataset for SAR object detection, named SARDet-100k. This dataset comprises approximately 117k images and 246k instances of objects across six distinct categories. To our knowledge, SARDet-100k is the first dataset of COCO-scale magnitude in this research area. It significantly contributes to overcoming the previously mentioned limitations by providing a rich resource for the development and evaluation of SAR object detection models. Moreover, the dataset and source code will be made publicly available.Transferring gaps. Through our empirical research and detailed analysis, we have identified that a principal hurdle in SAR object detection is the significant domain gap and model gap encountered when transferring a backbone network pretrained on natural RGB datasets (e.g., ImageNet [17] ), to a detection network on SAR imagery. The domain gap stems from the stark visual discrepancies between RGB and SAR imagery, whereas the model gap arises from the model differences between the pretrained backbone and the whole detection framework employed in the downstream task.To mitigate the aforementioned domain gap and model gap, we propose a novel Multi-Stage with Filter Augmentation (MSFA) pretraining framework to bridge these gaps. This framework addresses the challenge from multiple angles: data input, domain transition, and model migration, each tailored to the unique properties of the SAR image detection task. For data input: to address the input domain gap between the pretrain and finetune datasets, we employ traditional, handcrafted feature descriptors. These descriptors efficiently transform the input data from pixel space to a feature space that is not only robust to noise but also statistically narrows the gap between data from RGB and SAR modalities (see Fig. 2(a) ), thereby enhancing the transferability of pretrained knowledge. For domain transition: we propose a domain transition bridge utilizing an optical remote sensing detection dataset. This bridge connects natural RGB images through optics correlation and SAR images through object correlation, establishing a hierarchical pretraining approach that effectively closes the domain gap between RGB and SAR imagery (see Fig. 2(b) ). For model migration: to guarantee thorough training of the entire detection framework and to facilitate complete model migration for finetuning, we employ the entire detector as a bridging model throughout the multi-stage pretraining process.The MSFA framework demonstrates remarkable efficacy in reducing the substantial domain and model gaps typically encountered between the pretraining and finetuning stages. MSFA is not only effective but also general and applicable across various modern deep neural networks.Our contribution to the field of SAR object detection can be concluded into the following FOUR points:• Introduction of the first COCO-level large-scale dataset for SAR multi-category object detection.• Identification of critical gaps in traditional model pretrain and finetune approaches for SAR object detection. • Proposal of a Multi-Stage with Filter Augmentation (MSFA) pretraining framework, which demonstrates remarkable effectiveness, as well as excellent generalizability and flexibility across various deep network models. • Establishment of a new benchmark in SAR object detection by releasing the datasets and code associated with our research. This contribution is expected to foster further advancements and progress in the field.2 Related Work 
sar images and handcraft features: 
 SAR imaging often suffers from poor image quality due to multiplicative speckle noise and artifacts [57; 47] . To mitigate this, many traditional handcrafted feature descriptors have been developed or adapted to extract more discernible features from SAR images. These include Histogram of Oriented Gradients [15] (HOG), Canny Edge Detector [5] , Gradient by Ratio Edge (GRE) [29] , Haarlike [62] Feature Descriptor and Wavelet Scattering Transform [45] (WST). Early works employed traditional algorithms, such as HOG for SAR object recognition [56; 49] , Canny [28; 38] for edge detection. However, in recent years, the field of SAR image analysis has been largely dominated by deep learning approaches.While recent studies have focused on tasks related to low-level processing [69; 86] , classification [83; 85; 93; 26; 50; 78] and pretrain [29; 29] , they have attempted to integrate classic handcrafted features into modern neural networks for robust SAR image feature extraction and refinement. In contrast, our work does not simply inject such handcrafted features into networks, but explores the benefits and potentials of handcrafted features in domain adaptation and SAR object detection under modern deep neural networks. This research area remains largely unexplored, and our work aims to bridge this gap.Various popular deep learning-based object detection frameworks, including RetinaNet [33] , FCOS [59] , GFL [30] , RCNN series [53; 4] , YOLO series [52; 10] , and DETR [6] , demonstrate remarkable generalizability in the field of general object detection. Additionally, modern backbone networks such as ConvNext [42] , VAN [22] , LSKNet [31] and Swin Transformer [41] are designed to efficiently and effectively model visual features. However, SAR image object detection poses unique challenges due to factors such as small object size, speckle noise, and sparse information inherent in SAR images. As a result, recent deep learning methods for SAR object detection primarily focus on network and module design to address these challenges. Approaches like MGCAN [9] , MSSDNet [91] , and SEFEPNet [79] enhance object features through multiscale feature fusion. Quad-FPN [82] combines four distinct feature pyramid networks for thorough multiscale feature interaction to alleviate noise interferences and multi-scale object feature misalignment. PADN [88] and EWFAN [65] employ attention mechanisms to enhance object features in the presence of SAR speckle noise. CenterNet++ [21] , an extension of CenterNet [92] , incorporates feature enhancement, multi-scale fusion, and head refinement modules to improve the detector's robustness specifically for SAR images. Additionally, CRTransSar [74] , built on the high-performance Swin transformer [41] , leverages context representation learning to enhance object features.While most existing works concentrate on mitigating SAR speckle noise interference through network structure improvements, few attempts to address the issue at the level of input data. Furthermore, most studies utilize ImageNet pretrained backbones as the initialization of the detection framework, overlooking the substantial domain gap between the pretrained nature scenes dataset and the finetuned SAR dataset, as well as the model gap between the backbone and the entire detection framework. Instead, we seek to address these unique challenges through a carefully designed pretraining strategy. [12] provide access to these images, which offer a macroscopic view of various geophysical places such as cities, mountains, rivers, and cultivated land. This makes them particularly advantageous for scene classification tasks. However, the inherent low resolution of these images constrains their capability to delineate fine details of smaller objects, such as ships, cars, and airplanes. Conversely, high-resolution SAR images provide more detailed information but require significant hardware resources. Moreover, these high-definition images often encompass sensitive information, making them unsuitable for public release. Furthermore, acquiring high-resolution SAR datasets can be very expensive, posing significant challenges to their accessibility.Numerous research teams frequently encounter budgetary limitations that restrict their capacity to obtain a large and diverse collection of high-resolution SAR datasets. These financial constraints not only limit the scope of geographical areas that can be covered but also affect the variety of data sources that can be accessed. Consequently, the datasets made available by these groups often lack diversity, particularly in aspects such as spectral bands, polarization, and resolution. From a researcher's perspective, evaluating models on such small and homogeneous datasets can introduce bias and lead to unfair performance comparisons.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: Limitations are discussed in Section 6 and A.8.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
sardet-100k: 
 To address the aforementioned challenges, we undertake a thorough survey of SAR object detection datasets. As a result, we carefully collect a total of 10 publicly available high-quality datasets that are not only diverse but also have no conflicting object categories. EQUATIONWhere T i is a pre-defined transformation. The resulting Filter Augmented Input 280 to the detection model, Inp, will be a concatenation of the original single channel 281 grayscale SAR image x and the generated filter augmented feature M x i as:EQUATION)283By casting the original data input from the heterogeneous pixel space to a 284 homogeneous filter augmented feature space, the domain gaps between different 285 image domains can be greatly reduced, as illustrated in Fig. 4a .Multi-stage pretrain. We formulate the traditional pretrain schema as: EQUATIONEQUATIONQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We upload the source code as supplementary material. We will release our code and dataset after paper acceptance. Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
multi-stage with filter augmentation pretraining framework: 
 Several recent studies [26; 78; 74; 21] have demonstrated the effectiveness of mature handcrafted features and specialized network module designs in improving SAR object detection performance. However, most of these works rely on the default ImageNet pretraining approach, thus overlooking the significant domain gap between the pretrained nature scenes dataset and the finetuned SAR dataset. Additionally, they fail to address the model gap that exists between the backbone and the entire detection framework. To address these limitations, we propose a novel framework called the Multi-Stage with Filter Augmentation (MSFA) Pretraining Framework. Our framework tackles the challenges from the perspective of data input, domain transition, and model migration. MSFA comprises two core designs: the Filter Augmented Input and the Multi-Stage pretrain strategy.As discussed in the Related Work section, numerous existing handcrafted feature descriptors leverage meticulously designed filters to extract features. These features, robust and rich in information, act as augmented information derived from the original image. Thus, we propose employing such features as auxiliary information alongside the original pixel data. The filter augmented feature M of data x can be generally defined as: EQUATIONWhere T i is a pre-defined transformation. Drawing inspiration from the information residual design in ResNet [23] , we construct the Filter Augmented Input to the detection model, Inp, by concatenating the original grayscale SAR image x with the generated filter augmented feature M x i as:EQUATIONBy casting the original data input from the heterogeneous pixel space to a homogeneous filter augmented feature space, the domain gaps between different image domains can be greatly reduced, as illustrated in Fig. 2 (a).We formulate the traditional pretrain schema as:EQUATIONEQUATIONThe function Train 3)). Then finetune on the SAR dataset D SAR with pretrained backbone initialised from detection model A B (as Eq. ( 4)).Our proposed multi-stage pretrain strategy, alternatively, can be illustrated as in the Eq. (3) (5) (6) .EQUATIONEQUATIONWhere an extra second stage pretraining in Eq. ( 5) is added. We propose the utilization of a large-scale optical remote sensing dataset, D RS , as a detection pretrain for domain transit. The dataset consists of optical modal imagery, which also shares similar object shapes, scales, and categories in downstream SAR datasets. This characteristic serves as a valuable bridge between the optical distribution of natural images in ImageNet and the object distribution in SAR remote sensing images. By leveraging such second stage pretrain, the domain gap is effectively minimized, as illustrated in Fig. 2 (b).Finally, the proposed MSFA framework integrates Filter Augmented Input with Multi-Stage pretraining, as illustrated in Fig. 3 . Our MSFA framework effectively bridges the substantial domain and modal gaps between pretraining on nature images and finetuning on SAR image detection.By introducing Filter Augmented Input, we leverage mature handcrafted feature descriptors to extract noise-robust features. This also enables us to effectively transform the heterogeneous image domains of both pretraining and finetuning images into a homogeneous feature domain. By unifying the input data into a consistent feature domain, we address the disparities that exist between different types of images. Consequently, it enhances the alignment and transferability of knowledge across domains. Moreover, the incorporation of multi-stage training involves utilizing an additional large-scale optical remote sensing dataset for detection pretraining. This dataset acts as a domain bridge, connecting the domain of ImageNet's nature images with that of SAR remote sensing images. As a result, it further reduces the domain gaps, facilitating a smoother transition between the two domains. Furthermore, the detection pretraining in the second stage of the MSFA framework can also act as a model bridge.It allows for the comprehensive training of the entire detection framework, rather than solely focusing on the backbone, making the whole detection framework well-initialized to perform optimally in the SAR detection finetuning.5 Experiments and Analysis framework of our proposed MSFA method. The findings, detailed in Table 3 , indicate that incorporating these handcrafted features notably enhances the performance of the detector. Additionally, our analysis reveals that converting image pixels into handcrafted feature spaces significantly minimizes the distributional gaps between the ImageNet and the SARDet-100K datasets. This is particularly evident in the Pearson Correlation Coefficients (PCC) between the inputs of the ImageNet and SARDet-100K datasets, as illustrated in Table 4 . This underscores the efficacy of the proposed approach in bridging the domain gap between natural and SAR images, thereby enhancing the efficiency of knowledge transfer from the pretraining process.Remarkably, the Wavelet Scattering Transform (WST) feature stands out for its exceptional performance. This superiority can be attributed not just to its role in significantly narrowing the domain gap, but also to its capacity for extracting rich, multi-scale information. Such information acts as a robust auxiliary feature by mitigating noise and preserving object-related details. However, we also find that using multiple filter augmented features will not have further significant performance gain. It is possible that the existing WST already captures the essential information necessary for effective object detection, and incorporating additional ones does not provide substantial additional beneficial information.Due to the outstanding performance of WST, we employed it as the default Filter Augmented Input in our MSFA method for the remainder of the paper.  
multi-stage pretrain: 
 To evaluate the effectiveness of the proposed multi-stage pretraining approach, we conduct experiments in which we keep the input modality consistent and finetuned the detection model using various pretraining strategies on the SARDet-100K dataset. As a baseline, Exp. 1 takes single-channel SAR data as input, pretrains the backbone model on ImageNet for 100 epochs, and then directly finetune the detector on the SARDet-100K dataset (following the widely used default setting). In addition to the baseline, we perform a second stage pretrain specifically for object detection on optical remote sensing datasets, such as DOTA [73] or DIOR [27] . (Details on DOTA and DIOR datasets can be found in the Appendix). Following the second pretraining stage, we finetune the model either solely on the backbone or on the entire framework.The results of Exp. 2, 4, 6, and 8 in Table 5 prove the substantial advantages of the two-stage pretraining approach. Notably, even the relatively small-scale DIOR dataset showcases noticeable performance gains compared to the baseline (Exp. 1 and 5). This observation underscores the significance of reducing the domain gap during the pretraining phase of SAR detection.However, the DIOR dataset pretraining is not as effective as the larger-scale DOTA dataset (Exp. 2 vs 4, 6 vs 8). This comparison underscores the significance of the pretraining scale in achieving optimal results. The DOTA dataset, with its larger scale and similar average instance area to SARDet-100K, provides a more comprehensive and informative pretraining, leading to improved performance during the subsequent finetuning stage.The comparison between Exp. 3 & 4 and Exp. 7 & 8, demonstrate the superiority of pretraining the entire framework over solely the backbone, highlighting the significant impact of the model gap on the performance of SAR detection.In summary, our proposed multi-stage pretraining strategy in MSFA alleviates both the data domain gap and the model gap between pretraining and the downstream model, leading to significant enhancements in SAR detection performance. Detailed experimental results and visualization are available in the Appendix.To assess the effectiveness and generalizability of the proposed MSFA, we conduct experiments using various detectors and backbones, as presented in Fig. 4 (a) and 4(b). Significant performance improvement is observed across different frameworks (including single-stage [53; 4; 43] , twostage [33; 30; 59] , and end2end [94; 58; 39] ) and diverse backbones (including ResNets [23] , ConvNexts [42] , VANs [22] , and Swin-Transformer [41] networks). It provides strong evidence for the effectiveness and wide applicability of our proposed method. Furthermore, we observe stable performance improvements as we scale up the backbone size, as shown in Fig. 4 (b), indicating the good scalability of our proposed method.Significantly, the design of our MSFA method was developed with flexibility, generalizability and wide applicability in mind. Therefore the method can be seamlessly integrated into most existing models without any modifications. To assess the effectiveness and generalizability of the proposed MSFA, we con-388 duct experiments using various detectors and backbones, as presented in Fig. 6a 389 and 6b. Significant performance improvement is observed across different frame-390 works (including single-stage [4, 42, 51] , two-stage [29, 32, 57] , and end2end [38,56, 391 91]) and diverse backbones (including ResNets [22] , ConvNexts [41] , VANs [21] , 392 and Swin-Transformer [40] networks). It provides strong evidence for the effec-393 tiveness and wide applicability of our proposed method. Furthermore, we observe 394 stable performance improvements as we scale up the backbone size, as shown in 395 Fig. 6b , indicating the good scalability of our proposed method.Significantly, the design of our MSFA method was developed with flexibility, 397 generalizability and wide applicability in mind. Therefore the method can be 398 seamlessly integrated into most existing models without any modifications. We compare various SOTA methods, including both general object detection models [43; 53; 14; 64; 4; 87; 72] , as well as SAR object detection models [7; 25; 89; 13; 70; 40; 37; 74; 21; 20] . We evaluate their performance on the SSDD and HRSID datasets, which are the commonly used benchmarks for SAR object detection. To leverage the superior efficiency and performance of the VAN [22] backbone (as shown in Fig. 4 6 demonstrate that our MSFA method outperforms all the compared methods by a significant margin. Specifically, MSFA achieves a mAP@50 of 97.9% on the SSDD dataset and a mAP@50 of 83.7% on the HRSID dataset, setting new state-of-the-art results. It is noteworthy that our method is the only open-sourced method among the compared SAR detection SOTAs. 
limitaion and future work: 
 The scope of this paper is limited to supervised pretraining. However, considering the availability of a vast amount of unannotated SAR images, it would be valuable to explore the potential of semi-supervised, weakly-supervised or unsupervised learning methods for domain transfer in SAR object detection.While this paper aims to propose a simple, practical, effective, and generic method, it does not delve into the details of specific designs. Future work can be expanded to explore the aforementioned directions in more depth, incorporating intricate and specialized designs to enhance the performance and capabilities of SAR object detection. 
conclusion: 
 This paper presents a new benchmark for large-scale SAR object detection, introducing the SARDet-100k dataset and the Multi-Stage with Filter Augmentation (MSFA) pretrain method. Our SARDet-100k dataset comprises over 116K images spanning 6 categories, providing a large and diverse dataset for conducting SAR object detection research. To bridge the domain and model gaps between pretraining and finetuning stages in SAR object detection, we propose the MSFA pretraining framework. MSFA significantly enhances the performance of SAR object detection models, setting new state-of-the-art performance in previous benchmark datasets. Moreover, MSFA demonstrates remarkable generalizability and flexibility across various models. Our research endeavours to overcome the current obstacles prevalent in SAR object detection. We anticipate our contributions will pave the way for future research and innovations in this domain.Our research endeavours to overcome the current obstacles prevalent in SAR object detection. We anticipate our contributions will pave the way for future research and innovations in this domain. Furthermore, we convert all dataset annotations into the COCO annotation format [34] . This step ensures consistency and compatibility among the different datasets. Consequently, the merged dataset, SARDet-100K, is also standardized in the COCO format, which is readily compatible with popular open-source detection code frameworks, eliminating the need for additional manual data preprocessing. Fig. S6 (b) provides an overview of the category-level statistics for the SARDet-100K dataset. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Licenses of merged datasets are put into Table S6 . We cited relevant assets, and our proposed dataset collection gets permission from the original dataset owners.• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We provided detailed information about the proposed dataset and detailed documentation about our source code. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Histogram of Oriented Gradients [15] (HOG). HOG is a widely used local feature descriptor in image processing and computer vision. It captures and represents the image's local structure and shape information by analyzing the distribution of gradient orientations. HOG is proven to be effective for SAR image classification and object detection tasks because it is invariant to random noise in SAR images. Early works employ HOG for SAR object recognition [56; 49] , and recent studies demonstrate the effectiveness of HOG features for SAR image classification [83; 85] , as well as the model pretraining [66] in modern neural networks.Canny Edge Detector [5] . Canny is a widely used edge detection algorithm, which aims to identify significant edges in an image while minimizing noise and spurious responses. The algorithm utilizes Gaussian smoothing, pixel orientation gradient magnitude, and non-maximum suppression. Early research [28; 38] recognize the advantage of Canny for SAR image processing. Recent works also verify the effectiveness of the Canny feature in SAR image edge detection [69] , interference detection [18] , and image registration [86] . Other than Canny Edge Detector, Gradient by Ratio Edge (GRE) [29] , a recently proposed edge detector, leverages SAR-HOG [16] and SAR-SIFT [56] to achieve effective edge detection in SAR images.Haar-like [62] The detailed experimental outcomes are presented in Table S7 , Table S8 , and Table S9 , where comprehensive, fine-grained testing metrics are provided. These include AP@50, AP@75, AP@small (AP@s), AP@medium (AP@m), and AP@large (AP@l), offering deeper insights and robust results for model evaluation.Table S7 serves as an extension of Table 3 from the main paper, showcasing a comparison of various filter-augmented inputs utilizing Faster R-CNN with ResNet-50 as the detection model. To ensure a fair comparison, we evaluate MSFA and vanilla detection models under similar computational budgets. In Table S11 , we train Faster-RCNN models (with a ResNet-50 ImageNet-1K pretrained backbone) with 12 epochs of MSFA pretraining on the DOTA dataset, followed by finetuning on the SARDet-100K dataset. This is compared to directly finetuning the model for 36 epochs on the SARDet-100K dataset. With the same total number of training epochs and a slightly less total number of iterations, the proposed MSFA achieves a significantly higher (1.7% higher) mAP result on the SARDet-100K test set. MSFA pretraining incorporates general knowledge from both natural and remote sensing datasets, which allows efficient and effective finetuning and helps mitigate overfitting in downstream tasks. It is also important to note that, MSFA pretraining is a one-time effort. The pretrained MSFA models can be reused for finetuning different SAR detection datasets.To assess the quality of the proposed SARDet-100K dataset as a large-scale SAR object detection benchmark, we evaluate different models on this dataset and compare the results with those obtained from other popular benchmark datasets, such as SSDD [84] and HRSID [71] . The results are presented in Fig S9 . These results indicate that the performance of modern models on our dataset has not yet reached saturation. Among the various models reviewed, there is an 8.4% performance gap between the weakest and strongest models, whereas, for SSDD and HRSID, this gap is only 4.1% and 4.3%, respectively. This suggests that SSDD and HRSID are relatively simple for most existing models, leading to performance saturation on these smaller datasets.Furthermore, we observed that on these smaller datasets, models with a relatively large number of parameters tend to suffer from performance degradation due to overfitting. For example, ResNet-152 underperforms compared to ResNet-101 on SSDD, and ResNet-101 underperforms compared to ResNet-50 and ResNet-18 on HRSID. However, on the large-scale SARDet-100K, this issue does not arise. On our larger dataset, models continue to benefit from increased model size, indicating that our proposed dataset is suitable for developing relatively larger models for large-scale SAR object detection. For the ImageNet pretrain, we adopt a 100-epoch backbone pretraining strategy on the Imagenet-1K, with their default training strategy in MMPretrain [11] configurations.For the second stage of the proposed multistage pretrain, we select the large-scale optical remote sensing DOTA dataset as the main dataset. We also do comparison experiments on the DIOR dataset to find out the downstream performance affection factors on the second stage pretrain dataset. Their detailed information is illustrated in Table S10 .For the DOTA dataset, we perform an extra dataset preprocess because DOTA dataset images have a large range of different image resolutions. The original image contains only 1,411 training images. To have more image and multiscale instances for effective training, we follow [31] to adopt a multi-scale dataset splitting strategy by rescaling the original high-resolution images into three distinct scales (x0.5, x1.0, x1.5), and then cropping each scaled image into 1024×1024 patches with an overlap of each patch of 500 pixels, to avoid destruction division on the instance at patch borders. With the preprocessed dataset, load resize as 1024*1024, with a RandomFlip probability of 0.5. For the DIOR dataset, we train the model by resizing the images as 800*800, with a RandomFlip probability of 0.5.For finetuning on SARDet-100k, SSDD and HRSID, we train by resizing the image into 800*800, with a RandomFlip probability of 0.5. We train the model by 12 epochs on the train set and test the model on the test set with the 12th epoch checkpoint.We primarily conduct our experiments using the MMPretrain [11] and the MMDetection [8] frameworks, on 8 RTX-3090 GPUs (24G). For detailed information on the hyperparameters and training settings, please refer to Table S12 .The detection visualization results, comparing the MSFA framework with the traditional ImageNet pretraining approach, are presented in Fig. S10 . These results demonstrate that MSFA outperforms the conventional ImageNet backbone pretrain in terms of reducing missed detections, false detection, and improving localization accuracy. However, the current model is not without its drawbacks and imperfections. Fig. S11 highlights several failure scenarios. When the input SAR images lack identifiable details or contextual information, it can lead to incorrect classifications. When SAR images contain small and densely packed objects, the model may fail to detect some of them. Poor image quality characterized by fading, blurriness, or overall low resolution, further exacerbates the risk of missed detections. In challenging cases, the detector may also struggle with accurate localization.Regarding fine-grained category detection performance, Table S13 presents the detection results of Faster-RCNN [53] with ConvNext-B [42] and MSFA pretraining. Notably, the model exhibits relatively low performance for objects with:• Small sizes, such as Tank (with an average area of 461 pixels)• Large length-width ratios, such as Bridge • High appearance variability, such as Airplane However, the primary focus of this work is to address the domain and model gaps existing in SAR object detection pretraining and finetuning. Our method demonstrates excellent compatibility with most existing deep networks and can seamlessly integrate with models specifically designed to tackle the above challenging scenarios. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.38th Conference on Neural Information Processing Systems (NeurIPS 2024). 


Decision: All reviewers recognize the contribution of this submission. After rebuttal, the authors provide further clarification of the issues mentioned by the reviewers. Now, all reviewers have no further issue for the submission. The AC agrees the reviewers' comments and learns toward an acceptance for this paper.

Reviews: review: paper_summary: paper_summary: This work introduces SARDet-100K, a new large-scale, multi-category dataset for SAR object detection. It also proposes a novel Multi-Stage with Filter Augmentation pretraining framework to mitigate domain and model gaps encountered when transferring models pretrained on RGB datasets to SAR datasets. A new benchmark dataset and open-source method in SAR object detection is established.
strength_weakness: strength_weakness: The concept of the 'model gap' is not clearly defined in the current manuscript. Could you provide a more detailed explanation of what you mean by 'model gap' within the context of your study?

Instead of a time-consuming pretraining stage, why not directly train the SAR detection model using only the proposed dataset with sufficiently long iterations since the dataset already contains a sufficient amount of images and instances.
questions: questions: None
limitations: limitations: Limitations are well discussed.
paper_summary: paper_summary: This study presents the a large-scale dataset designed for SAR object detection, alongside a Multi-Stage with Filter Augmentation pretraining framework. The authors address the challenges associated with the limited availability of public SAR datasets and the lack of accessible source codes. The proposed method is effective and can be generalized to most modern backbone and detection networks.
strength_weakness: strength_weakness: Your novelty lies in incorporating handcrafted features, the introduction and related work on these features are somewhat lacking. Given the current dominance of deep learning methods, many junior researchers may not be familiar with classic handcrafted feature descriptors. Therefore, it would be beneficial to provide a more comprehensive introduction and conceptual visualization for each of the mentioned handcrafted features to enhance the paper's clarity and accessibility. 

The supplementary materials would benefit from including a clear, step-by-step guide on training and testing the models. While the code is provided, the lack of direct scripts and instructions makes it difficult to replicate the results.

The category distribution in Figure S6(b) reveals SARDet-100K is a significantly imbalanced dataset. This imbalance may lead to the long-tail problem, potentially hindering the performance of models on tail categories. Why do not you consider balancing the dataset?
questions: questions: In Figure 1, are the compared image pairs spatially aligned? It seems not. Would it be an issue?
limitations: limitations: None.
paper_summary: paper_summary: The authors establish a new benchmark SAR object detection dataset (SARDet-100K) and open-source SAR detection pretrain method (MSFA). This initiative significantly addresses the limitations posed by the scarcity of public SAR datasets and the inaccessibility of source codes, fostering further research and development in SAR object detection.
strength_weakness: strength_weakness: The paper does not clarify if the metrics reported on the SARDet-100K dataset are for the test set or the validation set. Additionally, please clarify the training setting: is the model checkpoint used for testing from the best validation or from the last epoch?

As a benchmark dataset and method, actual runtimes and memory usage should be reported. These details are currently missing from the paper.

There is a lack of clarification on the abbreviations used in Table 1 and S12.
questions: questions: Regarding the image slicing in your dataset standardization process, how do you handle objects that lie on the slicing border? Will the slicing split the objects apart?
limitations: limitations: Limitations are well discussed in Section 6 and A.8.
paper_summary: paper_summary: The authors address the limitations of existing datasets and the inaccessibility of source codes by creating a new benchmark dataset, SARDet-100K, which is a large-scale, multi-class dataset. Additionally, the paper proposes a Multi-Stage with Filter Augmentation pretraining framework designed to overcome the domain and model gaps between pretraining on RGB datasets and finetuning on SAR datasets. The MSFA method demonstrates effectiveness and generalizability across various models.
strength_weakness: strength_weakness: 1. You mentioned that optical remote sensing datasets (like DOTA) shares similar object shapes, scales, and categories in SAR datasets, therefore the downstream SAR datasets detection can benefits from the transferred knowledge. But why not discuss joint train the DOTA and SARDet-100K datasets together? In this way, the model can also learn joint representation from both DOTA and SARDet-100K therefore potentially improves the detection performance on SARDet-100K.
2. It is recommended to add a few more recent methods for comparison in Table 5.
questions: questions: See above.
limitations: limitations: Limitations are properly addressed.

Comments: comment: Thanks for your reply. The authors have addressed the issues. I tend to accept it.
The authors have solved my concerns, I am convinced to maintain my initial judgment.
The authors have addressed all the concerns, and I would like to raise my score to accept.