proposed a modular framework to implement and compare existing attention-based FSOD methods. Thorough experiments on both aerial and natural images showed the superiority of local attention mechanisms, called alignment. To reduce the performance gap between natural and aerial images, a novel alignment technique is developed within the framework to specifically address the detection of small objects. The resulting FSOD approach outperforms existing work in the literature on aerial image datasets as it improves largely the detection of small targets. Finally, Chap. 7 studied the last FSOD approach: the fine-tuning strategy. Leveraging the recent DiffusionDet detector based on diffusion models, we propose a simple fine-tuning approach that outperforms significantly previous techniques. The simplicity of this approach and its impressive detection quality allows for tackling more difficult scenarios such as Cross-Domain FSOD. Fine-tuning based FSOD is also better suited for transductive inference than metric-learning or attention approaches as it does not change the detection model much. Both transductive inference and Cross-Domain scenarios are of particular interest to COSE as they match better the real application. Our experiments in these directions show promising results and should be extended in future work. All detection models heavily rely on the Intersection over Union (IoU) for training and evaluation. However, we showed in Chap. 8 that it is not an optimal choice, especially when dealing with small objects. Thus, it becomes critical when applying FSOD to aerial images. Therefore, we proposed Scale-adaptive Intersection over Union to replace IoU. It significantly improves the training of few-shot object detectors, as it allows for a better balance between small and large objects. In addition, it aligns better with human perception and is then a better choice for the evaluation of object detectors. Then, with a more industrial mindset, we optimized and deployed several object detectors inside the CAMELEON prototype. We explained our process, the tools leveraged, and the compromises made in Chap. 9. This chapter focuses only on regular object detection as it is a first step before deploying more complex models able to generalize either to new classes or new domains.

Finally, we take a step back and compare the three approaches that we proposed in this thesis: Prototypical Faster R-CNN, XQSA and FSDiffusionDet. Each has its pros and cons, even if PFRCNN does not perform well it can be adapted at test time which is a highly desirable property. On the contrary, FSDiffusionDet is less flexible but achieves significantly higher performance. This comparison is available in Tab. 10.1, it compares the three methods on various criteria, grouped into four categories: Performance, Flexibility, Training and Deployment. These categories encompass the desired capabilities for a detector model inside the CAMELEON system.

## Future Research Tracks

In this PhD, various directions have been explored for improving Few-Shot Object Detection. We believe that we have provided significant contributions and answered relevant questions in the FSOD field. However, our analysis raises new questions and problems. First, we have proposed several improvements toward the detection of small objects in the few-shot regime. Nonetheless, there is still a significant performance gap between small and large objects and more effort should be put in this direction. To this end, we have explored the attention mechanisms and loss function design,
<table><caption>Table 10.1: Comparison table between the three proposed approaches in this thesis: Prototypical Faster R-CNN, XQSA and FSDiffusionDet. The ratings are given according to the experiments conducted throughout this project and the insights generated. Green and red colors denote pros and cons, respectively, best viewed in colors. *Episodic training is more complex and often subject to instabilities. *Technical complexity of the methods is a subjective criterion that measures how intricate a method is. †TensorRT optimization is not compatible with custom layers or complex data streams which greatly complexifies the optimization of more elaborated methods such as attention-based approaches.</caption><thead><tr><th></th><th></th><th>Prototypical Faster R-CNN</th><th>XQSA</th><th>FSDiffusionDet</th></tr><tr><th></th><th>Approach</th><th>Metric-Learning</th><th>Attention</th><th>Fine-tuning</th></tr></thead><tbody><tr><td rowspan="5">Performance</td><td>Novel classes performance</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Base classes performance</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Extremely low-shot performance</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Inference speed</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Class scalability</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td rowspan="5">Flexibility</td><td>Shot scalability</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Fixed class number</td><td>No</td><td>No</td><td>Yes</td></tr><tr><td>Test-time adaptation</td><td>Yes</td><td>No</td><td>No</td></tr><tr><td>Cross Domain Adaptation Capabilities</td><td>Not tested</td><td>Not tested</td><td>Promising results</td></tr><tr><td>Base training time</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td rowspan="4">Training</td><td>Fine-tuning required</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>Fine-tuning time</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Episodic training*</td><td>Yes</td><td>Yes</td><td>No</td></tr><tr><td>Technical complexity*</td><td>Red</td><td>Red</td><td>Red</td></tr><tr><td>Deployment</td><td>TensorRT optimization†</td><td>Red</td><td>Red</td><td>Red</td></tr></tbody></table>

but we have not looked into the training examples selection strategy even though it may have a significant influence on the small and large objects' balance during training. Incidentally, it would be relevant to replace IoU inside the example selection process with SIoU. This would favor smaller objects and certainly improves overall performance on aerial images. At least, it would offer more control over the training balance between small and large objects. SIoU is a controllable criterion, but it has a drawback, it requires the setting of two parameters and their optimal choices require some trials and errors. It would be of great use to come up with an automatic strategy that could provide the optimal parameters for a given problem or dataset.

The main motivation of Few-Shot Learning is to mimic the human ability to learn from very few examples. Of course, we are still far from solving FSOD, and a lot of efforts are still needed to close the gap with fully supervised learning, not to mention human-level perception. Nevertheless, the improvements made through this project encourage the study of more complex yet more realistic settings. One is particularly relevant for COSE's use case: Cross-Domain FSOD. This problem is still mostly untouched in the literature. We conducted some early experiments in this direction in Chap. 7 and demonstrated promising results. However, dedicated designs are required to get real
improvements as applying FSOD methods directly on this task is certainly not optimal. Many di-
rections are worth a try, taking inspiration from classification methods, e.g., generative modeling or
discrepancy-based adaptation. The transductive inference is another promising approach, it makes
some additional assumptions, but it is still realistic at least from COSE's perspective The transduc-
tive approach has not been explored for detection and could bring desirable properties to the FSOD
field, especially test-time adaptability which is still very challenging.

**Remaining Industrial Challenges**

Now from COSE's perspective, plenty of challenges are still to be addressed. First, the current
CAMELEON prototype only has a single camera but the final product is meant to have five more. It
would probably be quite difficult to achieve real-time detection on all images. Of course, we could
continue optimizing the models and find better speed/accuracy tradeoffs. The recent advances in
the YOLO family are promising, and substantial gains are achieved with newer GPUs. However,
it may not be sufficient to multiply the throughput of the detection pipeline by six. Fortunately,
as there is some overlapping between the images, it is not required to process all six data streams
independently. Instead, we could determine the overlapping areas to avoid processing them multiple
times. But that is not trivial and requires image registration techniques which are also expensive,
especially at the size of CAMELEON's images.

While the current prototype achieves satisfactory detection performance, there is still room for improvement. Our attempt to apply Knowledge Distillation (KD) to aerial images has been fruitless. Nevertheless, distillation is a promising direction to improve detection performance while maintaining a fixed computation budget. Some effort should be spent on developing novel KD methods that are also beneficial with aerial images and small objects.

Currently, only regular object detection algorithms have been deployed inside the prototype. While this is certainly a major step, it is still limited by the training classes and domain. As it is not possible to know in advance the kind of images that will be encountered during missions, mainly because of confidentiality constraints, we cannot guarantee the quality of the detections in operation. It would be extremely valuable to achieve real-time class and domain adaptation from a few examples. It would result in a single model that could be adapted on-the-fly (literally) according to the operation's needs. However, state-of-the-art FSOD is still far from this and COSE should probably not invest in this direction in the short run, especially as such adaptative methods will probably come at the cost of lower detection quality and slower inference. Instead, COSE should focus on developing an efficient fine-tuning platform that could be used by the forces to train their own models for specific missions. It could not be done during a mission, but afterward using the images collected and a few annotations. It would help to analyze the mission data faster and could produce new real-time models for subsequent similar missions. The key would be to conceive a unique application that can handle image annotation, model fine-tuning and deployment, all at once.
BIBLIOGRAPHY

[1] Tsung-Yi Lin et al. "Feature pyramid networks for object detection". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2017, pages 2117-2125.

[2] Paul Viola and Michael Jones. "Rapid object detection using a boosted cascade of simple features". In: *Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition*. CVPR 2001. Volume 1. Ieee. 2001, pages I-I.

[3] Pierre Sermanet et al. "Overfeat: Integrated recognition, localization and detection using convolutional networks". arXiv preprint arXiv:1312.6229 (2013).

[4] Ross Girshick et al. "Rich feature hierarchies for accurate object detection and semantic segmentation". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2014, pages 580-587.

[5] Mark Everingham et al. "The pascal visual object classes (voc) challenge". *International journal of computer vision* 88.2 (2010), pages 303-338.

[6] Tsung-Yi Lin et al. "Microsoft coco: Common objects in context". In: *European conference on computer vision*. Springer. 2014, pages 740-755.

[7] Gerard Salton. "Introduction to modern information retrieval". McGraw-Hill (1983).

[8] Kemal Oksuz et al. "Localization recall precision (LRP): A new performance metric for object detection". In: *Proceedings of the European conference on computer vision (ECCV)*. 2018, pages 504-519.

[9] Rohit Jena et al. "Beyond mAP: Towards better evaluation of instance segmentation". CVPR (2023).

[10] Syed Sahil Abbas Zaidi et al. "A survey of modern deep learning based object detection models". *Digital Signal Processing* (2022), page 103514.

[11] Zhengxia Zou et al. "Object detection in 20 years: A survey". *Proceedings of the IEEE* (2023).

[12] A Jay Goldstein, Leon D Harmon, and Ann B Lesk. "Identification of human faces". *Proceedings of the IEEE* 59.5 (1971), pages 748-760.

[13] A Jay Goldstein, Leon D Harmon, and Ann B Lesk. "Man-Machine Interaction in Human-Face Identification". *Bell System Technical Journal* 51.2 (1972), pages 399-427.
[14] Y Kaya and K Kobayashi. "A basic study on human face recognition". In: *Frontiers of pattern recognition*. Elsevier, 1972, pages 265–289.

[15] Alan Loddon Yuille, Peter W. Hallinan, and David S. Cohen. "Feature extraction from faces using deformable templates". *International Journal of Computer Vision* 8 (1989), pages 99–111.

[16] Matthew Turk and Alex Pentland. "Eigenfaces for recognition". *Journal of cognitive neuroscience* 3.1 (1991), pages 71–86.

[17] Martin Bichsel and Alex P Pentland. "Human face recognition and the face image Set's topology". *CVGIP: Image understanding* 59.2 (1994), pages 254–261.

[18] K-K Sung and Tomaso Poggio. "Example-based learning for view-based human face detection". *IEEE Transactions on pattern analysis and machine intelligence* 20.1 (1998), pages 39–51.

[19] Régis Vaillant, Christophe Monrocq, and Yann Le Cun. "Original approach for the localisation of objects in images". *IEE Proceedings-Vision, Image and Signal Processing* 141.4 (1994), pages 245–250.

[20] Henry A Rowley, Shumeet Baluja, and Takeo Kanade. "Neural network-based face detection". *IEEE Transactions on pattern analysis and machine intelligence* 20.1 (1998), pages 23–38.

[21] Baback Moghaddam and Alex Pentland. "Probabilistic visual learning for object detection". In: *Proceedings of IEEE international conference on computer vision*. IEEE. 1995, pages 786–793.

[22] Constantine P Papageorgiou, Michael Oren, and Tomaso Poggio. "A general framework for object detection". In: *Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271)*. IEEE. 1998, pages 555–562.

[23] Remi Ronfard, Cordelia Schmid, and Bill Triggs. "Learning to parse pictures of people". In: *Computer Vision-ECCV 2002: 7th European Conference on Computer Vision Copenhagen, Denmark, May 28–31, 2002 Proceedings, Part IV* 7. Springer. 2002, pages 700–714.

[24] Krystian Mikolajczyk, Cordelia Schmid, and Andrew Zisserman. "Human detection based on a probabilistic assembly of robust part detectors". In: *Computer Vision-ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, May 11-14, 2004. Proceedings, Part I* 8. Springer. 2004, pages 69–82.

[25] Constantine Papageorgiou and Tomaso Poggio. "A trainable system for object detection". *International journal of computer vision* 38 (2000), pages 15–33.

[26] Navneet Dalal and Bill Triggs. "Histograms of oriented gradients for human detection". In: *2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05)*. Volume 1. IEEE. 2005, pages 886–893.

[27] Pedro Felzenszwalb, David McAllester, and Deva Ramanan. "A discriminatively trained, multiscale, deformable part model". In: *2008 IEEE conference on computer vision and pattern recognition*. Ieee. 2008, pages 1–8.
[28] Ross Girshick, Pedro Felzenszwalb, and David McAllester. "Object detection with grammar models". *Advances in neural information processing systems* 24 (2011).

[29] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. "Imagenet classification with deep convolutional neural networks". *Communications of the ACM* 60.6 (2017), pages 84–90.

[30] Jasper RR Uijlings et al. "Selective search for object recognition". *International journal of computer vision* 104 (2013), pages 154–171.

[31] Ross Girshick. "Fast r-cnn". In: *Proceedings of the IEEE international conference on computer vision*. 2015, pages 1440–1448.

[32] Kaiming He et al. "Spatial pyramid pooling in deep convolutional networks for visual recognition". *IEEE transactions on pattern analysis and machine intelligence* 37.9 (2015), pages 1904–1916.

[33] Shaoqing Ren et al. "Faster r-cnn: Towards real-time object detection with region proposal networks". *Advances in neural information processing systems* 28 (2015), pages 91–99.

[34] Joseph Redmon et al. "You only look once: Unified, real-time object detection". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2016, pages 779–788.

[35] Joseph Redmon and Ali Farhadi. "YOLO9000: better, faster, stronger". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2017, pages 7263–7271.

[36] Joseph Redmon and Ali Farhadi. "Yolov3: An incremental improvement". *arXiv preprint arXiv:1804.02767* (2018).

[37] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. "Yolov4: Optimal speed and accuracy of object detection". *arXiv preprint arXiv:2004.10934* (2020).

[38] Xiang Long et al. "PP-YOLO: An effective and efficient implementation of object detector". *arXiv preprint arXiv:2007.12099* (2020).

[39] Zheng Ge et al. "YOLOX: Exceeding YOLO Series in 2021". *arXiv preprint arXiv:2107.08430* (2021).

[40] Chuyi Li et al. "YOLOv6: A single-stage object detection framework for industrial applications". *arXiv preprint arXiv:2209.02976* (2022).

[41] Chien-Yao Wang, I-Hau Yeh, and Hong-Yuan Mark Liao. "You only learn one representation: Unified network for multiple tasks". *arXiv preprint arXiv:2105.04206* (2021).

[42] Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors". *arXiv preprint arXiv:2207.02696* (2022).

[43] Hei Law and Jia Deng. "Cornernet: Detecting objects as paired keypoints". In: *Proceedings of the European conference on computer vision (ECCV)*. 2018, pages 734–750.

[44] Kaiwen Duan et al. "Centernet: Keypoint triplets for object detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 6569–6578.

[45] Zhi Tian et al. "Fcos: Fully convolutional one-stage object detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 9627–9636.
[46] Wei Liu et al. "Ssd: Single shot multibox detector". In: *Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I*. Springer. 2016, pages 21-37.

[47] Karen Simonyan and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition". *arXiv preprint arXiv:1409.1556* (2014).

[48] Kaiming He et al. "Deep residual learning for image recognition". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2016, pages 770-778.

[49] Sergey Zagoruyko and Nikos Komodakis. "Wide residual networks". *arXiv preprint arXiv:1605.07146* (2016).

[50] Saining Xie et al. "Aggregated residual transformations for deep neural networks". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2017, pages 1492-1500.

[51] Mingxing Tan and Quoc Le. "Efficientnet: Rethinking model scaling for convolutional neural networks". In: *International conference on machine learning*. PMLR. 2019, pages 6105-6114.

[52] Cheng-Yang Fu et al. "Dssd: Deconvolutional single shot detector". *arXiv preprint arXiv:1701.06659* (2017).

[53] Kaiming He et al. "Mask r-cnn". In: *Proceedings of the IEEE international conference on computer vision*. 2017, pages 2961-2969.

[54] Shu Liu et al. "Path aggregation network for instance segmentation". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 8759-8768.

[55] Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. "Nas-fpn: Learning scalable feature pyramid architecture for object detection". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2019, pages 7036-7045.

[56] Ashish Vaswani et al. "Attention is all you need". In: *Advances in neural information processing systems*. 2017, pages 5998-6008.

[57] Alexey Dosovitskiy et al. "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale". *ICLR* (2021).

[58] Niki Parmar et al. "Image transformer". In: *International conference on machine learning*. PMLR. 2018, pages 4055-4064.

[59] Nicolas Carion et al. "End-to-end object detection with transformers". In: *European Conference on Computer Vision*. Springer. 2020, pages 213-229.

[60] Xizhou Zhu et al. "Deformable {DETR}: Deformable Transformers for End-to-End Object Detection". In: *International Conference on Learning Representations*. 2021.

[61] Jifeng Dai et al. "Deformable convolutional networks". In: *Proceedings of the IEEE international conference on computer vision*. 2017, pages 764-773.

[62] Ding Jia et al. "DETRs with Hybrid Matching". *arXiv preprint arXiv:2207.13080* (2022).

[63] Hugo Touvron et al. "Training data-efficient image transformers & distillation through attention". In: *International conference on machine learning*. PMLR. 2021, pages 10347-10357.
[64] Hangbo Bao et al. "BEiT: BERT Pre-Training of Image Transformers". In: *International Conference on Learning Representations*. 2022.

[65] Stéphane d'Ascoli et al. "Convit: Improving vision transformers with soft convolutional inductive biases". In: *International Conference on Machine Learning*. PMLR. 2021, pages 2286–2296.

[66] Ze Liu et al. "Swin transformer: Hierarchical vision transformer using shifted windows". In: *Proceedings of the IEEE/CVF international conference on computer vision*. 2021, pages 10012–10022.

[67] Ze Liu et al. "Swin transformer v2: Scaling up capacity and resolution". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2022, pages 12009–12019.

[68] Mathilde Caron et al. "Emerging properties in self-supervised vision transformers". In: *Proceedings of the IEEE/CVF international conference on computer vision*. 2021, pages 9650–9660.

[69] Maxime Oquab et al. *DINOv2: Learning Robust Visual Features without Supervision*. 2023.

[70] Alexander Kirillov et al. *Segment Anything*. 2023.

[71] Zhuang Liu et al. "A convnet for the 2020s". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 11976–11986.

[72] Wenhai Wang et al. "Internimage: Exploring large-scale vision foundation models with deformable convolutions". *arXiv preprint arXiv:2211.05778* (2022).

[73] Xiyang Dai et al. "Dynamic Head: Unifying Object Detection Heads with Attentions". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 7373–7382.

[74] Shoufa Chen et al. "Diffusionet: Diffusion model for object detection". *arXiv preprint arXiv:2211.09788* (2022).

[75] Lei Liu, Zongxu Pan, and Bin Lei. "Learning a rotation invariant detector with rotatable bounding box". *arXiv preprint arXiv:1711.09405* (2017).

[76] Xingxing Xie et al. "Oriented R-CNN for object detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 3520–3529.

[77] Gui-Song Xia et al. "DOTA: A large-scale dataset for object detection in aerial images". In: *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*. 2018, pages 3974–3983.

[78] Yongchao Xu et al. "Gliding vertex on the horizontal bounding box for multi-oriented object detection". *IEEE transactions on pattern analysis and machine intelligence* 43.4 (2020), pages 1452–1459.

[79] Jian Ding et al. "Learning RoI transformer for detecting oriented objects in aerial images". *arXiv preprint arXiv:1812.00155* (2018).

[80] Wei Li, Wei Wei, and Lei Zhang. "GSDet: Object Detection in Aerial Images Based on Scale Reasoning". *IEEE Transactions on Image Processing* 30 (2021), pages 4599–4609.

[81] Chunfang Deng et al. "Extended Feature Pyramid Network for Small Object Detection". *IEEE Transactions on Multimedia* 24 (2022), pages 1968–1979.
[82] Bharat Singh, Mahyar Najibi, and Larry S Davis. "Sniper: Efficient multi-scale training". *Advances in neural information processing systems* 31 (2018).

[83] Bharat Singh et al. "Scale normalized image pyramids with AutoFocus for object detection". *IEEE Transactions on Pattern Analysis and Machine Intelligence* 44.7 (2021), pages 3749-3766.

[84] Mate Kisantal et al. "Augmentation for small object detection". *arXiv preprint arXiv:1902.07296* (2019).

[85] Jakaria Rabbi et al. "Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network". *Remote Sensing* 12.9 (2020), page 1432.

[86] Yancheng Bai et al. "SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network". In: *ECCV*. 2018.

[87] Syeda Nyma Ferdous, Moktari Mostofa, and Nasser M. Nasrabadi. "Super resolution-assisted deep aerial vehicle detection". In: *Defense + Commercial Sensing*. 2019.

[88] Chang Xu et al. "Detecting Tiny Objects in Aerial Images: A Normalized Wasserstein Distance and A New Benchmark". *ISPRS Journal of Photogrammetry and Remote Sensing* (ISPRS J P & RS) (2022).

[89] Tsung-Yi Lin et al. "Focal loss for dense object detection". In: *Proceedings of the IEEE international conference on computer vision*. 2017, pages 2980-2988.

[90] Peter J Huber. "Robust estimation of a location parameter". *Breakthroughs in statistics: Methodology and distribution* (1992), pages 492-518.

[91] Jiahui Yu et al. "Unitbox: An advanced object detection network". In: *Proceedings of the 24th ACM international conference on Multimedia*. 2016, pages 516-520.

[92] Hamid Rezatofighi et al. "Generalized intersection over union: A metric and a loss for bounding box regression". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2019, pages 658-666.

[93] Zhaohui Zheng et al. "Distance-IoU loss: Faster and better learning for bounding box regression". In: *Proceedings of the AAAI conference on artificial intelligence*. Volume 34. 07. 2020, pages 12993-13000.

[94] Jiabo He et al. "Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression". *Advances in Neural Information Processing Systems* 34 (2021), pages 20230-20242.

[95] Ke Li et al. "Object detection in optical remote sensing images: A survey and a new benchmark". *ISPRS Journal of Photogrammetry and Remote Sensing* 159 (2020), pages 296-307.

[96] Agrim Gupta, Piotr Dollar, and Ross Girshick. "LVIS: A Dataset for Large Vocabulary Instance Segmentation". In: *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*. 2019.

[97] Shuai Shao et al. "Objects365: A Large-Scale, High-Quality Dataset for Object Detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)*. 2019.
[98] Andreas Geiger, Philip Lenz, and Raquel Urtasun. "Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite". In: *Conference on Computer Vision and Pattern Recognition (CVPR)*. 2012.

[99] Fisher Yu et al. "Bdd100k: A diverse driving dataset for heterogeneous multitask learning". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2020, pages 2636-2645.

[100] Shanshan Zhang, Rodrigo Benenson, and Bernt Schiele. "CityPersons: A Diverse Dataset for Pedestrian Detection". In: *CVPR*. 2017.

[101] Xuehui Yu et al. "Scale match for tiny person detection". In: *Proceedings of the IEEE/CVF winter conference on applications of computer vision*. 2020, pages 1257-1265.

[102] Shuai Shao et al. "CrowdHuman: A Benchmark for Detecting Human in a Crowd". *arXiv preprint arXiv:1805.00123* (2018).

[103] T Nathan Mundhenk et al. "A large contextual dataset for classification, detection and counting of cars with deep learning". In: *Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14*. Springer. 2016, pages 785-800.

[104] Darius Lam et al. "xview: Objects in context in overhead imagery". *arXiv preprint arXiv:1802.07856* (2018).

[105] Xian Sun et al. "FAIR1M: A benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery". *ISPRS Journal of Photogrammetry and Remote Sensing* 184 (2022), pages 116-130.

[106] Alexander Ziller et al. *Oktoberfest Food Dataset*. 2019.

[107] Naoto Inoue et al. "Cross-domain weakly-supervised object detection through progressive domain adaptation". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 5001-5009.

[108] Jing Wang et al. "Logodet-3k: A large-scale image dataset for logo detection". *ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)* 18.1 (2022), pages 1-19.

[109] Caijing Miao et al. "Sixray: A large-scale security inspection x-ray benchmark for prohibited item discovery in overlapping images". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2019, pages 2119-2128.

[110] Yiming Sun et al. "Drone-Based RGB-Infrared Cross-Modality Vehicle Detection Via Uncertainty-Aware Learning". *IEEE Transactions on Circuits and Systems for Video Technology* 32.10 (2022), pages 6700-6713.

[111] Yann LeCun et al. "Gradient-based learning applied to document recognition". *Proceedings of the IEEE* 86.11 (1998), pages 2278-2324.

[112] Waseem Rawat and Zenghui Wang. "Deep convolutional neural networks for image classification: A comprehensive review". *Neural computation* 29.9 (2017), pages 2352-2449.
[113] Ting Chen et al. "A simple framework for contrastive learning of visual representations". In: *International conference on machine learning*. PMLR. 2020, pages 1597-1607.

[114] Jean-Bastien Grill et al. "Bootstrap your own latent-a new approach to self-supervised learning". *Advances in neural information processing systems* 33 (2020), pages 21271-21284.

[115] Kaiming He et al. "Momentum contrast for unsupervised visual representation learning". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2020, pages 9729-9738.

[116] Mathilde Caron et al. "Unsupervised learning of visual features by contrasting cluster assignments". *Advances in neural information processing systems* 33 (2020), pages 9912-9924.

[117] Yaqing Wang et al. "Generalizing from a few examples: A survey on few-shot learning". *ACM Computing Surveys (CSUR)* 53.3 (2020), pages 1-34.

[118] Yisheng Song et al. "A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities". *arXiv preprint arXiv:2205.06743* (2022).

[119] James Kirkpatrick et al. "Overcoming catastrophic forgetting in neural networks". *Proceedings of the national academy of sciences* 114.13 (2017), pages 3521-3526.

[120] Donghyun Yoo et al. "Efficient k-shot learning with regularized deep networks". In: *Proceedings of the AAAI Conference on Artificial Intelligence*. Volume 32. 1. 2018.

[121] Eleni Triantafillou, Richard Zemel, and Raquel Urtasun. "Few-shot learning through an information retrieval lens". *Advances in neural information processing systems* 30 (2017).

[122] Tamir Hazan, Joseph Keshet, and David McAllester. "Direct loss minimization for structured prediction". *Advances in neural information processing systems* 23 (2010).

[123] Jane Bromley et al. "Signature verification using a" siamese" time delay neural network". *Advances in neural information processing systems* 6 (1993), pages 737-744.

[124] Sumit Chopra, Raia Hadsell, and Yann LeCun. "Learning a similarity metric discriminatively, with application to face verification". In: *2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)*. Volume 1. IEEE. 2005, pages 539-546.

[125] Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. "Siamese neural networks for one-shot image recognition". In: *ICML deep learning workshop*. Volume 2. 1. Lille. 2015.

[126] Jake Snell, Kevin Swersky, and Richard Zemel. "Prototypical Networks for Few-shot Learning". In: *Advances in Neural Information Processing Systems*. Edited by I. Guyon et al. Volume 30. 2017.

[127] Flood Sung et al. "Learning to compare: Relation network for few-shot learning". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 1199-1208.

[128] Jinlu Liu, Liang Song, and Yongqiang Qin. "Prototype rectification for few-shot learning". In: *Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16*. Springer. 2020, pages 741-756.

[129] Mengye Ren et al. "Meta-learning for semi-supervised few-shot classification". *arXiv preprint arXiv:1803.00676* (2018).
[130] Jessica Deuschel et al. "Multi-prototype few-shot learning in histopathology". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 620–628.

[131] Oriol Vinyals et al. "Matching Networks for One Shot Learning". In: *Advances in Neural Information Processing Systems*. Edited by D. Lee et al. Volume 29. 2016.

[132] Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. "Tadam: Task dependent adaptive metric for improved few-shot learning". *Advances in neural information processing systems* 31 (2018).

[133] Jürgen Schmidhuber. "Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook". PhD thesis. Technische Universität München, 1987.

[134] Sebastian Thrun and Lorien Pratt. "Learning to learn: Introduction and overview". *Learning to learn* (1998), pages 3–17.

[135] Chelsea Finn, Pieter Abbeel, and Sergey Levine. "Model-agnostic meta-learning for fast adaptation of deep networks". In: *International Conference on Machine Learning*. PMLR. 2017, pages 1126–1135.

[136] Yoonho Lee and Seungjin Choi. "Gradient-based meta-learning with learned layerwise metric and subspace". In: *International Conference on Machine Learning*. PMLR. 2018, pages 2927–2936.

[137] Luca Franceschi et al. "Bilevel programming for hyperparameter optimization and meta-learning". In: *International Conference on Machine Learning*. PMLR. 2018, pages 1568–1577.

[138] Taesup Kim et al. "Bayesian model-agnostic meta-learning". *arXiv preprint arXiv:1806.03836* (2018).

[139] Harkirat Singh Behl, Atılım Güneş Baydin, and Philip HS Torr. "Alpha maml: Adaptive model-agnostic meta-learning". *arXiv preprint arXiv:1905.07435* (2019).

[140] Andrei A Rusu et al. "Meta-learning with latent embedding optimization". *arXiv preprint arXiv:1807.05960* (2018).

[141] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. "Generalization of model-agnostic meta-learning algorithms: Recurring and unseen tasks". *Advances in Neural Information Processing Systems* 34 (2021), pages 5469–5480.

[142] S Ravi H Larochelle. "Optimization as a model for few-shot learning". In: *5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings*. 2017.

[143] Luca Bertinetto et al. "Meta-learning with differentiable closed-form solvers". *arXiv preprint arXiv:1805.08136* (2018).

[144] Luca Bertinetto et al. "Learning feed-forward one-shot learners". In: *Advances in neural information processing systems*. 2016, pages 523–531.

[145] Nikhil Mishra et al. "A simple neural attentive meta-learner". *arXiv preprint arXiv:1707.03141* (2017).

[146] Carl Doersch, Ankush Gupta, and Andrew Zisserman. "Crosstransformers: spatially-aware few-shot transfer". *arXiv preprint arXiv:2007.11498* (2020).
[147] Fang Zhao et al. "Dynamic conditional networks for few-shot learning". In: *Proceedings of the European conference on computer vision (ECCV)*. 2018, pages 19–35.

[148] Tsendsuren Munkhdalai et al. "Rapid adaptation with conditionally shifted neurons". In: *International conference on machine learning*. PMLR. 2018, pages 3664–3673.

[149] Adam Santoro et al. "Meta-learning with memory-augmented neural networks". In: *International conference on machine learning*. PMLR. 2016, pages 1842–1850.

[150] Tiago Ramalho and Marta Garnelo. "Adaptive posterior learning: few-shot learning with a surprise-based memory module". *arXiv preprint arXiv:1902.02527* (2019).

[151] Tsendsuren Munkhdalai and Hong Yu. "Meta networks". In: *International conference on machine learning*. PMLR. 2017, pages 2554–2563.

[152] Zhongwen Xu, Linchao Zhu, and Yi Yang. "Few-shot object recognition from machine-labeled web images". In: *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*. 2017, pages 1164–1172.

[153] Łukasz Kaiser et al. "Learning to remember rare events". *arXiv preprint arXiv:1703.03129* (2017).

[154] Vladimir N Vapnik. "An overview of statistical learning theory". *IEEE transactions on neural networks* 10.5 (1999), pages 988–999.

[155] Thorsten Joachims et al. "Transductive inference for text classification using support vector machines". In: *Icml*. Volume 99. 1999, pages 200–209.

[156] Corinna Cortes and Vladimir Vapnik. "Support-vector networks". *Machine learning* 20 (1995), pages 273–297.

[157] Dengyong Zhou et al. "Learning with local and global consistency". *Advances in neural information processing systems* 16 (2003).

[158] Omer Belhasin, Guy Bar-Shalom, and Ran El-Yaniv. "TransBoost: Improving the Best ImageNet Performance using Deep Transduction". *arXiv preprint arXiv:2205.13331* (2022).

[159] Guneet S Dhillon et al. "A baseline for few-shot image classification". *arXiv preprint arXiv:1909.02729* (2019).

[160] Limeng Qiao et al. "Transductive episodic-wise adaptive metric for few-shot learning". In: *Proceedings of the IEEE/CVF international conference on computer vision*. 2019, pages 3603–3612.

[161] Malik Boudiaf et al. "Information maximization for few-shot learning". *Advances in Neural Information Processing Systems* 33 (2020), pages 2445–2457.

[162] Imtiaz Ziko et al. "Laplacian regularized few-shot learning". In: *International conference on machine learning*. PMLR. 2020, pages 11660–11670.

[163] Ruibing Hou et al. "Cross attention network for few-shot classification". *Advances in Neural Information Processing Systems* 32 (2019).

[164] Jongmin Kim et al. "Edge-labeling graph neural network for few-shot learning". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2019, pages 11–20.
[165] Yanbin Liu et al. "Learning to propagate labels: Transductive propagation network for few-shot learning". arXiv preprint arXiv:1805.10002 (2018).

[166] Yuqing Hu, Vincent Gripon, and Stéphane Pateux. "Graph-based interpolation of feature vectors for accurate few-shot classification". In: *2020 25th International Conference on Pattern Recognition (ICPR)*. IEEE. 2021, pages 8164–8171.

[167] Pau Rodríguez et al. "Embedding propagation: Smoother manifold for few-shot classification". In: *Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVI 16*. Springer. 2020, pages 121–138.

[168] Michalis Lazarou, Tania Stathaki, and Yannis Avrithis. "Iterative label cleaning for transductive and semi-supervised few-shot learning". In: *Proceedings of the iee/cvf international conference on computer vision*. 2021, pages 8751–8760.

[169] Seong Min Kye et al. "Meta-learned confidence for few-shot learning". arXiv preprint arXiv:2002.12017 (2020).

[170] Alex Nichol, Joshua Achiam, and John Schulman. "On first-order meta-learning algorithms". arXiv preprint arXiv:1803.02999 (2018).

[171] Chuanxing Geng, Sheng-jun Huang, and Songcan Chen. "Recent advances in open set recognition: A survey". *IEEE transactions on pattern analysis and machine intelligence* 43.10 (2020), pages 3614–3631.

[172] Abhijit Bendale and Terrance E Boult. "Towards open set deep networks". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2016, pages 1563–1572.

[173] Inhyuk Jo et al. "Open set recognition by regularising classifier with fake data generated by generative adversarial networks". In: *2018 IEEE international conference on acoustics, speech and signal processing (ICASSP)*. IEEE. 2018, pages 2686–2690.

[174] Malik Boudiaf et al. "Open-Set Likelihood Maximization for Few-Shot Learning". arXiv preprint arXiv:2301.08390 (2023).

[175] Ségolène Martin et al. "Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inference". *Advances in Neural Information Processing Systems* 35 (2022), pages 34677–34688.

[176] Spyros Gidaris and Nikos Komodakis. "Dynamic few-shot visual learning without forgetting". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 4367–4375.

[177] Anthony Robins. "Catastrophic forgetting, rehearsal and pseudorehearsal". *Connection Science* 7.2 (1995), pages 123–146.

[178] Xilai Li et al. "Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting". In: *International Conference on Machine Learning*. PMLR. 2019, pages 3925–3934.

[179] Mei Wang and Weihong Deng. "Deep visual domain adaptation: A survey". *Neurocomputing* 312 (2018), pages 135–153.
[180] Eric Tzeng et al. "Simultaneous deep transfer across domains and tasks". In: *Proceedings of the IEEE international conference on computer vision*. 2015, pages 4068-4076.

[181] Timnit Gebru, Judy Hoffman, and Li Fei-Fei. "Fine-grained recognition in the wild: A multi-task domain adaptation approach". In: *Proceedings of the IEEE international conference on computer vision*. 2017, pages 1349-1358.

[182] Saeid Motiian et al. "Unified deep supervised domain adaptation and generalization". In: *Proceedings of the IEEE international conference on computer vision*. 2017, pages 5715-5725.

[183] Muhammad Ghifary, W Bastiaan Kleijn, and Mengjie Zhang. "Domain adaptive neural networks for object recognition". In: *PRICAI 2014: Trends in Artificial Intelligence: 13th Pacific Rim International Conference on Artificial Intelligence, Gold Coast, QLD, Australia, December 1-5, 2014. Proceedings 13*. Springer. 2014, pages 898-904.

[184] Mingsheng Long et al. "Learning transferable features with deep adaptation networks". In: *International conference on machine learning*. PMLR. 2015, pages 97-105.

[185] Eric Tzeng et al. "Deep domain confusion: Maximizing for domain invariance". *arXiv preprint arXiv:1412.3474* (2014).

[186] Mingsheng Long et al. "Deep transfer learning with joint adaptation networks". In: *International conference on machine learning*. PMLR. 2017, pages 2208-2217.

[187] Werner Zellinger et al. "Central moment discrepancy (cmd) for domain-invariant representation learning". *arXiv preprint arXiv:1702.0881* (2017).

[188] Fuzhen Zhuang et al. "Supervised representation learning: Transfer learning with deep autoencoders". In: *Twenty-fourth international joint conference on artificial intelligence*. 2015.

[189] Baochen Sun, Jiashi Feng, and Kate Saenko. "Return of frustratingly easy domain adaptation". In: *Proceedings of the AAAI conference on artificial intelligence*. Volume 30. 1. 2016.

[190] Artem Rozantsev, Mathieu Salzmann, and Pascal Fua. "Beyond sharing weights for deep domain adaptation". *IEEE transactions on pattern analysis and machine intelligence* 41.4 (2018), pages 801-814.

[191] Yanghao Li et al. "Revisiting batch normalization for practical domain adaptation". *arXiv preprint arXiv:1603.04779* (2016).

[192] Yaroslav Ganin et al. "Domain-adversarial training of neural networks". *The journal of machine learning research* 17.1 (2016), pages 2096-2030.

[193] Eric Tzeng et al. "Adversarial discriminative domain adaptation". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2017, pages 7167-7176.

[194] Ian Goodfellow et al. "Generative adversarial networks". *Communications of the ACM* 63.11 (2020), pages 139-144.

[195] Ming-Yu Liu and Oncel Tuzel. "Coupled generative adversarial networks". *Advances in neural information processing systems* 29 (2016).

[196] Phillip Isola et al. "Image-to-image translation with conditional adversarial networks". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2017, pages 1125-1134.
[197] Jun-Yan Zhu et al. "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks". In: *Computer Vision (ICCV), 2017 IEEE International Conference on*. 2017.

[198] Judy Hoffman et al. "Cycada: Cycle-consistent adversarial domain adaptation". In: *International conference on machine learning*. Pmlr. 2018, pages 1989–1998.

[199] Jascha Sohl-Dickstein et al. "Deep unsupervised learning using nonequilibrium thermodynamics". In: *International Conference on Machine Learning*. PMLR. 2015, pages 2256–2265.

[200] Jonathan Ho, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models". *Advances in Neural Information Processing Systems* 33 (2020), pages 6840–6851.

[201] Yulong Zhang et al. "Diffusion-based Target Sampler for Unsupervised Domain Adaptation". *arXiv preprint arXiv:2303.12724* (2023).

[202] Muhammad Ghifary et al. "Deep reconstruction-classification networks for unsupervised domain adaptation". In: *Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14*. Springer. 2016, pages 597–613.

[203] Konstantinos Bousmalis et al. "Domain separation networks". *Advances in neural information processing systems* 29 (2016).

[204] Taeksoo Kim et al. "Learning to discover cross-domain relations with generative adversarial networks". In: *International conference on machine learning*. PMLR. 2017, pages 1857–1865.

[205] Eleni Triantafillou et al. "Meta-dataset: A dataset of datasets for learning to learn from few examples". *arXiv preprint arXiv:1903.03096* (2019).

[206] Yuqian Fu, Yanwei Fu, and Yu-Gang Jiang. "Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data". In: *Proceedings of the 29th ACM International Conference on Multimedia*. 2021, pages 5326–5334.

[207] Hongyi Zhang et al. "mixup: Beyond empirical risk minimization". *arXiv preprint arXiv:1710.09412* (2017).

[208] Linhai Zhuo et al. "TGDM: Target Guided Dynamic Mixup for Cross-Domain Few-Shot Learning". In: *Proceedings of the 30th ACM International Conference on Multimedia*. 2022, pages 6368–6376.

[209] An Zhao et al. "Domain-adaptive few-shot learning". In: *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2021, pages 1390–1399.

[210] Wentao Chen et al. "Cross-Domain Cross-Set Few-Shot Learning via Learning Compact and Aligned Representations". In: *Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XX*. Springer. 2022, pages 383–399.

[211] Yuqian Fu et al. "ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning". In: *Proceedings of the 30th ACM International Conference on Multimedia*. 2022, pages 6609–6617.

[212] Wei-Hong Li, Xialei Liu, and Hakan Bilen. "Universal representation learning from multiple domains for few-shot classification". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 9526–9535.
[213] Jaehoon Oh et al. "ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning". In: *Proceedings of the 31st ACM International Conference on Information & Knowledge Management*. 2022, pages 4359-4363.

[214] Fupin Yao. "Cross-domain few-shot learning with unlabelled data". *arXiv preprint arXiv:2101.07899* (2021).

[215] Wenyu Zhang et al. "Few-Shot Adaptation of Pre-Trained Networks for Domain Shift". *arXiv preprint arXiv:2205.15234* (2022).

[216] Yanxu Hu and Andy J Ma. "Adversarial Feature Augmentation for Cross-domain Few-Shot Classification". In: *Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XX*. Springer. 2022, pages 20-37.

[217] Gabriel Huang et al. "A Survey of Self-Supervised and Few-Shot Object Detection". *arXiv preprint arXiv:2110.14711* (2021).

[218] Mona Kohler, Markus Eisenbach, and Horst-Michael Gross. "Few-Shot Object Detection: A Comprehensive Survey". *IEEE transactions on neural networks and learning systems* (2021).

[219] Sixu Liu et al. "Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges". *Remote Sensing* 14.18 (2022), page 4435.

[220] Bingyi Kang et al. "Few-shot object detection via feature reweighting". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 8420-8429.

[221] Ting-I Hsieh et al. "One-Shot Object Detection with Co-Attention and Co-Excitation". In: *Advances in Neural Information Processing Systems* 32. 2019.

[222] Xiaopeng Yan et al. "Meta r-cnn: Towards general solver for instance-level low-shot learning". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 9577-9586.

[223] Xiang Li, Jingyu Deng, and Yi Fang. "Few-Shot Object Detection on Remote Sensing Images". *IEEE Transactions on Geoscience and Remote Sensing* (2021), pages 1-14.

[224] Qi Fan et al. "Few-shot object detection with attention-RPN and multi-relation detector". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2020, pages 4013-4022.

[225] Yang Xiao and Renaud Marlet. "Few-shot object detection and viewpoint estimation for objects in the wild". In: *European Conference on Computer Vision*. Springer. 2020, pages 192-210.

[226] Geonuk Kim, Hong-Gyu Jung, and Seong-Whan Lee. "Few-Shot Object Detection via Knowledge Transfer". In: *2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)*. 2020, pages 3564-3569.

[227] Xiang Li et al. "One-shot object detection without fine-tuning". *arXiv preprint arXiv:2005.03819* (2020).

[228] Juan-Manuel Perez-Rua et al. "Incremental few-shot object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2020, pages 13846-13855.
[229] Zixuan Xiao et al. "Few-Shot Object Detection With Self-Adaptive Attention Network for Remote Sensing Images". *IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing* 14 (2021), pages 4854-4865.

[230] Yuxuan Gao et al. "A Fast and Accurate Few-Shot Detector for Objects with Fewer Pixels in Drone Image". *Electronics* 10.7 (2021), page 783.

[231] Guangxing Han et al. "Meta faster r-cnn: Towards accurate few-shot object detection with attentive feature alignment". In: *Proceedings of the AAAI Conference on Artificial Intelligence*. Volume 36. 1. 2022, pages 780-789.

[232] Gongjie Zhang et al. "Meta-DETR: Image-Level Few-Shot Detection with Inter-Class Correlation Exploitation". *IEEE Transactions on Pattern Analysis and Machine Intelligence* (2022).

[233] Weijie Liu et al. "Dynamic Relevance Learning for Few-Shot Object Detection". *arXiv preprint arXiv:2108.02235* (2021).

[234] Tung-I Chen et al. "Dual-Awareness Attention for Few-Shot Object Detection". *IEEE Transactions on Multimedia* (2021), pages 1-1.

[235] Honghui Xu et al. "Few-Shot Object Detection via Sample Processing". *IEEE Access* 9 (2021), pages 29207-29221.

[236] Jinghui Chu et al. "Joint Co-Attention And Co-Reconstruction Representation Learning For One-Shot Object Detection". In: *2021 IEEE International Conference on Image Processing (ICIP)*. 2021, pages 2229-2233.

[237] Aoxue Li and Zhenguo Li. "Transformation invariant few-shot object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 3094-3102.

[238] Xu Huang et al. "Few-Shot Object Detection on Remote Sensing Images via Shared Attention Module and Balanced Fine-Tuning Strategy". *Remote Sensing* 13.19 (2021), page 3816.

[239] Guangxing Han et al. "Few-shot object detection with fully cross-transformer". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 5321-5330.

[240] Shiqi Chen et al. "Few Shot Object Detection for SAR Images via Feature Enhancement and Dynamic Relationship Modeling". *Remote Sensing* 14.15 (2022), page 3669.

[241] Yan Ouyang et al. "Few-shot object detection based on positive-sample improvement". *Defense Technology* (2022).

[242] Yizhou Zhao, Xun Guo, and Yan Lu. "Semantic-aligned fusion transformer for one-shot object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 7601-7611.

[243] Hojun Lee, Myunggi Lee, and Nojun Kwak. "Few-shot object detection by attending to per-sample-prototype". In: *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2022, pages 2445-2454.
[244] Shan Zhang et al. "Kernelized few-shot object detection with efficient integral aggregation". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 19207-19216.

[245] Zheng Zhou et al. "FSODS: A Lightweight Metalearning Method for Few-Shot Object Detection on SAR Images". *IEEE Transactions on Geoscience and Remote Sensing* 60 (2022), pages 1-17.

[246] Nanqing Liu et al. "Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images". *arXiv preprint arXiv:2303.06817* (2023).

[247] Xinyu Jiang et al. "Few-Shot Object Detection via Improved Classification Features". In: *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2023, pages 5386-5395.

[248] Gongjie Zhang et al. "PNPDet: Efficient few-shot detection without forgetting via plug-and-play sub-networks". In: *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2021, pages 3823-3832.

[249] Aming Wu et al. "Universal-Prototype Enhancing for Few-Shot Object Detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 9567-9576.

[250] Liyang Liu et al. "Gendet: Meta learning to generate detectors from few shots". *IEEE Transactions on Neural Networks and Learning Systems* (2021).

[251] Leonid Karlinsky et al. "Repmet: Representative-based metric learning for classification and few-shot object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2019, pages 5197-5206.

[252] Yukuan Yang et al. "Restoring negative information in few-shot object detection". *Advances in neural information processing systems* 33 (2020), pages 3521-3532.

[253] Xiangyun Zhao, Xu Zou, and Ying Wu. "Morphable detector for object detection on demand". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 4771-4780.

[254] Bo Sun et al. "FSCE: Few-shot object detection via contrastive proposal encoding". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 7352-7362.

[255] Aming Wu et al. "Generalized and discriminative few-shot object detection via SVD-dictionary enhancement". *Advances in Neural Information Processing Systems* 34 (2021), pages 6353-6364.

[256] Hao Chen et al. "Lstd: A low-shot transfer detector for object detection". In: *Proceedings of the AAAI Conference on Artificial Intelligence*. Volume 32. 1. 2018.

[257] Jiaxi Wu et al. "Multi-scale positive sample refinement for few-shot object detection". In: *European Conference on Computer Vision*. Springer. 2020, pages 456-472.

[258] Xin Wang et al. "Frustratingly Simple Few-Shot Object Detection". In: *International Conference on Machine Learning (ICML)*. 2020.
[259] Zhibo Fan et al. "Generalized Few-Shot Object Detection without Forgetting". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 4527-4536.

[260] Weilin Zhang and Yu-Xiong Wang. "Hallucination improves few-shot object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 13008-13017.

[261] Stefan Wolf et al. "Double Head Predictor based Few-Shot Object Detection for Aerial Imagery". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 721-731.

[262] Prannay Kaul, Weidi Xie, and Andrew Zisserman. "Label, verify, correct: A simple few shot object detection method". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2022, pages 14237-14247.

[263] Yiting Li et al. "Few-shot object detection via classification refinement and distractor retreatment". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 15395-15403.

[264] Yuhang Cao et al. "Few-Shot Object Detection via Association and DIscrimination". *Advances in Neural Information Processing Systems* 34 (2021), pages 16570-16581.

[265] Limeng Qiao et al. "Defrcn: Decoupled faster r-cnn for few-shot object detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021, pages 8681-8690.

[266] Yang Xu et al. SIMPL: Generating Synthetic Overhead Imagery to Address Zero-shot and Few-Shot Detection Problems. 2021.

[267] Amir Bar et al. "Detreg: Unsupervised pretraining with region priors for object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 14605-14615.

[268] Karim Guirguis et al. "CFA: Constraint-based Finetuning Approach for Generalized Few-Shot Object Detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 4039-4049.

[269] Yan Wang et al. "Context Information Refinement for Few-Shot Object Detection in Remote Sensing Images". *Remote Sensing* 14.14 (2022), page 3255.

[270] Weijie Liu et al. "Novel instance mining with pseudo-margin evaluation for few-shot object detection". In: *ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE. 2022, pages 2250-2254.

[271] Yihang She et al. "Fast Hierarchical Learning for Few-Shot Object Detection". In: *2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE. 2022, pages 1993-2000.

[272] Shuang Wu et al. "Multi-faceted Distillation of Base-Novel Commonality for Few-Shot Object Detection". In: *Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part IX*. Springer. 2022, pages 578-594.

[273] Bin-Bin Gao et al. "Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation". In: *Advances in Neural Information Processing Systems*. 2022.
[274] Shaobo Lin et al. "An Effective Crop-Paste Pipeline for Few-shot Object Detection". arXiv preprint arXiv:2302.14452 (2023).

[275] Na Dong et al. "Incremental-detr: Incremental few-shot object detection via self-supervised learning". arXiv preprint arXiv:2205.04042 (2022).

[276] Yu-Xiong Wang, Deva Ramanan, and Martial Hebert. "Meta-learning to detect rare objects". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 9925–9934.

[277] Li Yin, Juan M Perez-Rua, and Kevin J Liang. "Sylph: A hypernetwork framework for incremental few-shot object detection". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 9035–9045.

[278] Shafin Rahman, Salman Khan, and Nick Barnes. "Transductive learning for zero-shot object detection". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 6082–6091.

[279] Guangxing Han et al. "Multimodal few-shot object detection with meta-learning based cross-modal prompting". arXiv preprint arXiv:2204.07841 (2022).

[280] Lifan Zhao, Yunlong Meng, and Lin Xu. "OA-FSUI2IT: A Novel Few-Shot Cross Domain Object Detection Framework with Object-Aware Few-Shot Unsupervised Image-to-Image Translation". In: *Proceedings of the AAAI Conference on Artificial Intelligence*. Volume 36. 3. 2022, pages 3426–3435.

[281] Yipeng Gao et al. "AcroFOD: An Adaptive Method for Cross-Domain Few-Shot Object Detection". In: *Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIII*. Springer. 2022, pages 673–690.

[282] Yuzuru Nakamura et al. "Few-shot Adaptive Object Detection with Cross-Domain CutMix". In: *Proceedings of the Asian Conference on Computer Vision*. 2022, pages 1350–1367.

[283] Wuti Xiong and Li Liu. "CD-FSOD: A Benchmark for Cross-domain Few-shot Object Detection". arXiv preprint arXiv:2210.05311 (2022).

[284] Kibok Lee et al. "Rethinking Few-Shot Object Detection on a Multi-Domain Benchmark". In: *Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XX*. Springer. 2022, pages 366–382.

[285] Yue Lu et al. "Decoupled metric network for single-stage few-shot object detection". *IEEE Transactions on Cybernetics* 53.1 (2022), pages 514–525.

[286] Xiaopeng Yan et al. "Meta r-cnn: Towards general solver for instance-level low-shot learning". In: *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2019, pages 9577–9586.

[287] Qi Fan et al. "Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector". In: *CVPR*. 2020.

[288] Xiongwei Wu, Doyen Sahoo, and Steven Hoi. "Meta-rcnn: Meta learning for few-shot object detection". In: *Proceedings of the 28th ACM International Conference on Multimedia*. 2020, pages 1679–1687.
[289] Yong Zhou et al. "Few-shot object detection via context-aware aggregation for remote sensing images". *IEEE Geoscience and Remote Sensing Letters* 19 (2022), pages 1-5.

[290] Ankan Bansal et al. "Zero-shot object detection". In: *Proceedings of the European Conference on Computer Vision (ECCV)*. 2018, pages 384-400.

[291] Alec Radford et al. "Learning transferable visual models from natural language supervision". In: *International conference on machine learning*. PMLR. 2021, pages 8748-8763.

[292] Tao Wang et al. "Few-shot adaptive faster r-cnn". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2019, pages 7173-7182.

[293] Geir Drange. *Arthropod Taxonomy Orders Object Detection Dataset*. 2020.

[294] Lihao Jiang et al. "Underwater species detection using channel sharpening attention". In: *Proceedings of the 29th ACM International Conference on Multimedia*. 2021, pages 4259-4267.

[295] Mathilde Caron et al. "Deep clustering for unsupervised learning of visual features". In: *Proceedings of the European Conference on Computer Vision (ECCV)*. 2018, pages 132-149.

[296] Elad Hoffer and Nir Ailon. "Deep metric learning using triplet network". In: *International workshop on similarity-based pattern recognition*. Springer. 2015, pages 84-92.

[297] Li Deng. "The mnist database of handwritten digit images for machine learning research". *IEEE Signal Processing Magazine* 29.6 (2012), pages 141-142.

[298] Diederik P Kingma and Jimmy Ba. "Adam: A method for stochastic optimization". *arXiv preprint arXiv:1412.6980* (2014).

[299] Laurens Van der Maaten and Geoffrey Hinton. "Visualizing data using t-SNE." *Journal of machine learning research* 9.11 (2008).

[300] Xiaolong Wang et al. "Non-local neural networks". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 7794-7803.

[301] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. "Layer normalization". *arXiv preprint arXiv:1607.06450* (2016).

[302] Lilian Weng. "What are diffusion models?" *lilianweng.github.io* (2021).

[303] Diederik P Kingma and Max Welling. "Auto-encoding variational bayes". *arXiv preprint arXiv:1312.6114* (2013).

[304] Robin Rombach et al. "High-resolution image synthesis with latent diffusion models". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2022, pages 10684-10695.

[305] Alexander Quinn Nichol and Prafulla Dhariwal. "Improved denoising diffusion probabilistic models". In: *International Conference on Machine Learning*. PMLR. 2021, pages 8162-8171.

[306] Cheng Lu et al. "Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps". *arXiv preprint arXiv:2206.00927* (2022).

[307] Cheng Lu et al. "Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models". *arXiv preprint arXiv:2211.01095* (2022).
[308] Zhifeng Kong and Wei Ping. "On fast sampling of diffusion probabilistic models". arXiv preprint arXiv:2106.00132 (2021).

[309] Emiel Hoogeboom and Tim Salimans. "Blurring diffusion models". arXiv preprint arXiv:2209.05557 (2022).

[310] Giannis Daras et al. "Soft diffusion: Score matching for general corruptions". arXiv preprint arXiv:2209.05442 (2022).

[311] Jiaming Song, Chenlin Meng, and Stefano Ermon. "Denoising diffusion implicit models". arXiv preprint arXiv:2010.02502 (2020).

[312] Jonathan Ho et al. "Cascaded Diffusion Models for High Fidelity Image Generation." J. Mach. Learn. Res. 23.47 (2022), pages 1-33.

[313] Tero Karras, Samuli Laine, and Timo Aila. "A style-based generator architecture for generative adversarial networks". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2019, pages 4401-4410.

[314] Tero Karras et al. "Analyzing and Improving the Image Quality of StyleGAN". In: *Proc. CVPR*. 2020.

[315] Tom Brown et al. "Language models are few-shot learners". *Advances in neural information processing systems* 33 (2020), pages 1877-1901.

[316] Colin Raffel et al. "Exploring the limits of transfer learning with a unified text-to-text transformer". *The Journal of Machine Learning Research* 21.1 (2020), pages 5485-5551.

[317] Aditya Ramesh et al. "Zero-shot text-to-image generation". In: *International Conference on Machine Learning*. PMLR. 2021, pages 8821-8831.

[318] Jean-Baptiste Alayrac et al. "Flamingo: a visual language model for few-shot learning". *Advances in Neural Information Processing Systems* 35 (2022), pages 23716-23736.

[319] Chitwan Saharia et al. "Photorealistic text-to-image diffusion models with deep language understanding". *Advances in Neural Information Processing Systems* 35 (2022), pages 36479-36494.

[320] Gautam Mittal et al. "Symbolic music generation with diffusion models". arXiv preprint arXiv:2103.16091 (2021).

[321] Jinglin Liu et al. "Diffsinger: Singing voice synthesis via shallow diffusion mechanism". In: *Proceedings of the AAAI Conference on Artificial Intelligence*. Volume 36. 10. 2022, pages 11020-11028.

[322] Rongjie Huang et al. "Prodiff: Progressive fast diffusion model for high-quality text-to-speech". In: *Proceedings of the 30th ACM International Conference on Multimedia*. 2022, pages 2595-2605.

[323] Justin Lovelace et al. "Latent Diffusion for Language Generation". arXiv preprint arXiv:2212.09462 (2022).

[324] Yan Li et al. "Generative time series forecasting with diffusion, denoise, and disentanglement". *Advances in Neural Information Processing Systems* 35 (2022), pages 23009-23022.
[325] Clement Vignac et al. "DiGress: Discrete Denoising diffusion for graph generation". arXiv preprint arXiv:2209.14734 (2022).

[326] Hiroshi Sasaki, Chris G Willcocks, and Toby P Breckon. "Unit-ddpm: Unpaired image translation with denoising diffusion probabilistic models". arXiv preprint arXiv:2104.05358 (2021).

[327] Animesh Karnewar et al. "HOLODIFFUSION: Training a 3D Diffusion Model Using 2D Images". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2023, pages 18423–18433.

[328] Aimon Rahman et al. "Ambiguous Medical Image Segmentation using Diffusion Models". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2023, pages 11536–11546.

[329] Bin Yang et al. "Craft objects from images". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2016, pages 6043–6051.

[330] Spyros Gidaris and Nikos Komodakis. "Object detection via a multi-region and semantic segmentation-aware cnn model". In: *Proceedings of the IEEE international conference on computer vision*. 2015, pages 1134–1142.

[331] Zhaowei Cai and Nuno Vasconcelos. "Cascade r-cnn: Delving into high quality object detection". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 6154–6162.

[332] Spyros Gidaris et al. "Boosting few-shot visual learning with self-supervision". In: *Proceedings of the IEEE/CVF international conference on computer vision*. 2019, pages 8059–8068.

[333] Ceyuan Yang et al. "Instance localization for self-supervised detection pretraining". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2021, pages 3987–3996.

[334] Fangyun Wei et al. "Aligning pretraining for detection via object-level contrastive learning". *Advances in Neural Information Processing Systems* 34 (2021), pages 22682–22694.

[335] Varun Godbole et al. *Deep Learning Tuning Playbook*. Version 1.0. 2023.

[336] Pengfei Zhu et al. "Detection and Tracking Meet Drones Challenge". *IEEE Transactions on Pattern Analysis and Machine Intelligence* (2021), pages 1–1.

[337] Sara Beery et al. "The iwildcam 2021 competition dataset". arXiv preprint arXiv:2105.03494 (2021).

[338] Menglin Jia et al. "Fashionpedia: Ontology, segmentation, and an attribute localization dataset". In: *Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16*. Springer. 2020, pages 316–332.

[339] Shifeng Zhang et al. "Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection". In: *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2020, pages 9759–9768.

[340] Xiang Li et al. "Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection". *Advances in Neural Information Processing Systems* 33 (2020), pages 21002–21012.
[341] Lachlan Tychsen-Smith and Lars Petersson. "Improving object localization with fitness nms and bounded iou loss". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2018, pages 6877-6885.

[342] Degang Sun et al. "A scale balanced loss for bounding box regression". *IEEE Access* 8 (2020), pages 108438-108448.

[343] Ombretta Strafforello et al. "Humans disagree with the IoU for measuring object detector localization error". In: *2022 IEEE International Conference on Image Processing (ICIP)*. IEEE. 2022, pages 1261-1265.

[344] Shuangjiang Du, Baofu Zhang, and Pin Zhang. "Scale-sensitive IOU loss: An improved regression loss function in remote sensing object detection". *IEEE Access* 9 (2021), pages 141258-141272.

[345] Richard Beals and Jacek Szmigielski. *Meijer G-Functions: A Gentle Introduction*. Aug. 2013.

[346] Glenn Jocher. *YOLOv5 by Ultralytics*. Version 7.0. 2020.

[347] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network". *arXiv preprint arXiv:1503.02531* (2015).

[348] Adriana Romero et al. "FitNets: Hints for Thin Deep Nets". In: *In Proceedings of ICLR*. 2015.

[349] Wonpyo Park et al. "Relational knowledge distillation". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2019, pages 3967-3976.

[350] Xiaojie Wang et al. "Kdgan: Knowledge distillation with generative adversarial networks". *Advances in neural information processing systems* 31 (2018).

[351] Yuang Liu, Wei Zhang, and Jun Wang. "Adaptive multi-teacher multi-level knowledge distillation". *Neurocomputing* 415 (2020), pages 106-113.

[352] Tao Wang et al. "Distilling object detectors with fine-grained feature imitation". In: *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2019, pages 4933-4942.

[353] Quanquan Li, Shengying Jin, and Junjie Yan. "Mimicking very efficient network for object detection". In: *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2017, pages 6356-6364.

[354] Akshay Chawla et al. "Data-free knowledge distillation for object detection". In: *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2021, pages 3289-3298.

[355] Yi Wei et al. "Quantization mimic: Towards very tiny cnn for object detection". In: *Proceedings of the European conference on computer vision (ECCV)*. 2018, pages 267-283.

[356] Wanwei Wang et al. "Gan-knowledge distillation for one-stage object detection". *IEEE Access* 8 (2020), pages 60719-60727.

[357] Jianping Gou et al. "Knowledge distillation: A survey". *International Journal of Computer Vision* 129 (2021), pages 1789-1819.

[358] Paulius Micikevicius et al. "Mixed precision training". *arXiv preprint arXiv:1710.03740* (2017).
# PROOFS OF SIoU'S PROPERTIES

In this appendix, we provide the proofs for Properties 1 and 2, and discuss the order-preservigness of SIoU.

## Property 1 (SIoU Relaxation)

Let $b_1$ and $b_2$ be two bounding boxes and introduce $\tau = \frac{w_1h_1+w_2h_2}{2}$ their average area. SIoU preserves the behavior of IoU in certain cases such as:

$$ - \text{IoU}(b_1, b_2) = 0 \Rightarrow \text{SIoU}(b_1, b_2) = \text{IoU}(b_1, b_2) = 0 $$

$$ - \text{IoU}(b_1, b_2) = 1 \Rightarrow \text{SIoU}(b_1, b_2) = \text{IoU}(b_1, b_2) = 1 $$

$$ - \lim_{\tau \to +\infty} \text{SIoU}(b_1, b_2) = \text{IoU}(b_1, b_2) $$

$$ - \lim_{\kappa \to 0} \text{SIoU}(b_1, b_2) = \text{IoU}(b_1, b_2) $$

**Proof** First let recall the expression of SIoU, SIoU($b_1, b_2$) = IoU($b_1, b_2$)$^p$ with $p = 1 - \gamma \exp(-\frac{\sqrt{\tau}}{\kappa})$. $\tau > 0$ because boxes cannot be empty and as $\gamma \in ]-\infty, 1]$ and $\kappa \in \mathbb{R}^*_+$, we have $p > 0$.

From this, the first two items of Property 1 follow clearly.

The two other points hold because the function $f: x \mapsto \text{IoU}(b_1, b_2)^x$ is continuous on $\mathbb{R}$ for any couple of boxes $b_1$ and $b_2$ ($\text{IoU}(b_1, b_2) \in [0, 1]$) and $\lim_{\tau \to \infty} p = \lim_{\kappa \to 0} p = 1$.
**Property 2 (Loss and gradients reweighting)**

Let $\mathcal{L}_{IoU}(b_1, b_2) = 1 - \text{IoU}(b_1, b_2)$ and $\mathcal{L}_{SIoU}(b_1, b_2) = 1 - \text{SIoU}(b_1, b_2)$ be the loss functions associated respectively with IoU and SIoU. Let denote the ratio between SIoU and IoU losses by $\mathcal{W}_{\mathcal{L}}(b_1, b_2) = \frac{\mathcal{L}_{SIoU}(b_1, b_2)}{\mathcal{L}_{IoU}(b_1, b_2)}$. Similarly, $\mathcal{W}_{\nabla}(b_1, b_2) = \frac{|\nabla \mathcal{L}_{SIoU}(b_1, b_2)|}{|\nabla \mathcal{L}_{IoU}(b_1, b_2)|}$ denotes the ratio of gradients generated from SIoU and IoU losses:

$$ \mathcal{W}_{\mathcal{L}}(b_1, b_2) = \frac{1 - \text{IoU}(b_1, b_2)^p}{1 - \text{IoU}(b_1, b_2)}, \qquad (A.1) $$

$$ \mathcal{W}_{\nabla}(b_1, b_2) = p\text{IoU}(b_1, b_2)^{p-1}, \qquad (A.2) $$

$\mathcal{W}_{\mathcal{L}}$ and $\mathcal{W}_{\nabla}$ are increasing (resp. decreasing) functions of IoU when $p \ge 1$ (resp. $p < 1$) which is satisfied when $\gamma \le 0$ (resp. $\gamma > 0$). As the IoU goes to 1, $\mathcal{W}_{\mathcal{L}}$ and $\mathcal{W}_{\nabla}$ approaches $p$:

$$ \lim_{\text{IoU}(b_1,b_2) \to 1} \mathcal{W}_{\mathcal{L}}(b_1, b_2) = p, \qquad (A.3) $$

$$ \lim_{\text{IoU}(b_1,b_2) \to 1} \mathcal{W}_{\nabla}(b_1, b_2) = p. \qquad (A.4) $$

**Proof** Let denote the IoU($b_1, b_2$) by $\mu$, and define two functions $f: \mu \mapsto 1 - \mu = \mathcal{L}_{IoU}(b_1, b_2)$ and $g: \mu \mapsto 1 - \mu^p = \mathcal{L}_{SIoU}(b_1, b_2)$.

$f$ and $g$ are differentiable on $[0, 1]$ and $\lim_{\mu \to 1} f(\mu) = \lim_{\mu \to 1} g(\mu) = 0$. This holds because $p$ is independent of the IoU (i.e., $\mu$).

Therefore L'Hôpital's rule can be applied:

$$ \lim_{\mu \to 1} \mathcal{W}_{\mathcal{L}} = \lim_{\mu \to 1} \frac{g(\mu)}{f(\mu)} = \lim_{\mu \to 1} \frac{g'(\mu)}{f'(\mu)} = \lim_{\mu \to 1} p\mu^{p-1} = p. $$

The expression of the second ratio $\mathcal{W}_{\nabla}(b_1, b_2)$ follows directly as $|\nabla \mathcal{L}_{SIoU}(b_1, b_2)| = g'(\mu)$ and $|\nabla \mathcal{L}_{IoU}(b_1, b_2)| = f'(\mu)$, hence $\lim_{\mu \to 1} \mathcal{W}_{\nabla} = \lim_{\mu \to 1} p\mu^{p-1} = p$.

**Order-preservigness**

Let us take three boxes $b_1, b_2$, and $b_3$. Order-preservigness does not hold for SIoU. Therefore IoU($b_1, b_2$) $\le$ IoU($b_1, b_3$) does not imply SIoU($b_1, b_2$) $\le$ SIoU($b_1, b_3$). However, this property is often true in practice. Denoting by $\tau_{i,j}$ the average area between boxes $i$ and $j$, we can study the conditions for the order to hold. We will also note $p_{i,j} = 1 - \gamma \exp(-\frac{\sqrt{\tau_{i,j}}}{\kappa})$

Let's suppose, without loss of generality, that $\tau_{1,2} \le \tau_{1,3}$. Otherwise, cases 1 and 2 would be swapped.

**Case 1:** $\gamma \le 0$

We have $p_{1,2} > p_{1,3}$ as $\tau_{1,2} \le \tau_{1,3}$ and $\gamma \le 0$. Therefore, $p_{1,2} = p_{1,3} + \varepsilon$, with $\varepsilon > 0$.
Then,

$$
\begin{align*}
\mathrm{IoU}(b_1, b_2)^{p_{1,2}} &= \mathrm{IoU}(b_1, b_2)^{p_{1,3}+\varepsilon} \\
&= \mathrm{IoU}(b_1, b_2)^{p_{1,3}} \mathrm{IoU}(b_1, b_2)^{\varepsilon} \\
&\leq \mathrm{IoU}(b_1, b_2)^{p_{1,3}} \\
&\leq \mathrm{IoU}(b_1, b_3)^{p_{1,3}}
\end{align*}
$$

Line 3 holds because $0 < \mathrm{IoU}(b_1, b_2)^\varepsilon \le 1$. Line 4 is true because $\mathrm{IoU}(b_1, b_2) \le \mathrm{IoU}(b_1, b_3)$ and the function $h: x \mapsto x^{p_{1,3}}$ is monotonically increasing. Hence, when $\gamma \le 0$ the order is preserved.

**Case 2:** $\gamma > 0$

We have $p_{1,3} > p_{1,2}$ as $\tau_{1,3} \le \tau_{1,2}$ and $\gamma > 0$. Therefore, $p_{1,3} = p_{1,2} + \varepsilon$, with $\varepsilon > 0$.

In this case, the order does not always hold, counter-examples can be found. However, it is useful
to study the conditions for it to hold:

$$
\mathrm{IoU}(b_1, b_2)^{p_{1,2}} \le \mathrm{IoU}(b_1, b_3)^{p_{1,3}} \Leftrightarrow \frac{\ln(\mathrm{IoU}(b_1, b_2))}{\ln(\mathrm{IoU}(b_1, b_3))} \ge \frac{p_{1,3}}{p_{1,2}}
$$

In practice, the right condition is often true as $p_{1,2}$ and $p_{1,3}$ are close due to scale matching, a trick present in most detection frameworks to prevent comparison of proposals and ground truth of very different sizes. In addition, the ratio of log values gets large rapidly, even if the gap between $\mathrm{IoU}(b_1, b_2)$ and $\mathrm{IoU}(b_1, b_3)$ is small, the ratio $\frac{\ln(\mathrm{IoU}(b_1, b_2))}{\ln(\mathrm{IoU}(b_1, b_3))}$ can be large.

The order-preservigness property holds in many cases. This is sensible as IoU is still a reliable
metric that has been used extensively for the training and evaluation of detection models. However,
in the rare cases where this order is broken, the IoU(b₁, b₂) and IoU(b₁, b₃) are close, so IoU does
not discriminate much between the boxes. On the contrary, SIoU prefers the smallest one (or largest
one, according to the choice of γ). This stronger discrimination is probably beneficial for training
and evaluation.
# THEORETICAL GIoU'S DISTRIBUTION ANALYSIS

In this appendix, we give a proof of the formulas of the probability density function, expected value, and variance of GIoU. We also provide some non-closed-form expressions for other criteria.

**Proposition 1 (GIoU's distribution)**

Let $b_1 = (0, y_1, w_1, h_1)$ be a bounding box horizontally centered and $b_2 = (X, y_2, w_2, h_2)$ another bounding box randomly positioned, with $X \sim \mathcal{N}(0, \sigma^2)$ and $\sigma \in \mathbb{R}_+^*$. Let's suppose that the boxes are identical squares, shifted only horizontally (i.e., $w_1 = w_2 = h_1 = h_2$ and $y_1 = y_2$).

Let $Z = \mathcal{C}(X)$, where $\mathcal{C}$ is the generalized intersection over union. The probability density function of $Z$ is given by:

$$d_Z(z) = \frac{4\omega}{(1+z)^2\sqrt{2\pi}\sigma} \exp\left(-\frac{1}{2}\left[\frac{\omega(1-z)}{\sigma(1+z)}\right]^2\right). \qquad (B.1)$$

The two first moments of $Z$ exist and are given by:

$$\mathbb{E}[Z] = \frac{2}{\pi^{3/2}} G_{3,2}^{2,3} \left( 2a^2 \begin{vmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & 0 & 0 \end{vmatrix} \right), \qquad (B.2)$$

$$\mathbb{E}[Z^2] = 1 - \frac{8a}{\sqrt{2\pi}} + \frac{16a^2}{\pi^{3/2}} G_{3,2}^{2,3} \left( 2a^2 \begin{vmatrix} -1 & \frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & 0 & 0 \end{vmatrix} \right), \qquad (B.3)$$

where $G$ is the Meijer G-function [345] and $a = \frac{\sigma}{\omega}$.

**Proof** First, in the setup described in Proposition 1, GIoU can be expressed in terms of the width of the boxes $\omega$ and the shift in between $x$. Let's call this function $\mathcal{C}$:

$$\begin{aligned} \mathcal{C}: \mathbb{R} &\to [-1, 1] \\ x &\mapsto \frac{\omega - |x|}{\omega + |x|} \end{aligned}$$

The shifts are sampled from a Gaussian distribution: $X \sim \mathcal{N}(0, \sigma^2)$, therefore we are interested in the distribution of the variable $Z = \mathcal{C}(X)$.
The cumulative density function of Z is given by:

$$
\begin{align*}
F_Z(z) &= P(Z \le z) = P\left(\frac{\omega - |X|}{\omega + |X|} \le y\right) \\
&= P\left(\omega \frac{1-z}{1+z} \le |X|\right) \\
&= 2P\left(\omega \frac{1-z}{1+z} \le X\right) \\
&= 2\left(1 - F_X\left(\omega \frac{1-z}{1+z}\right)\right) \\
&= 2\left(1 - F_X(g(z))\right)
\end{align*}
$$

With $g(z) = \omega(\frac{1-z}{1+z})$. Hence, the density function of Z can be derived by taking the derivative of $F_Z$:

$$
\begin{align*}
d_Z(z) &= \frac{d}{dz} F_Z(z) \\
&= -2g'(z) F'_X(g(y)) \\
&= \frac{4\omega}{(1+z)^2\sqrt{2\pi}\sigma} \exp\left(-\frac{1}{2}\left[\frac{\omega(1-z)}{\sigma(1+z)}\right]^2\right)
\end{align*}
$$

To determine the first and second moments of Z, we make use of the change of variable formula:

$$
\begin{align*}
\mathbb{E}[Z] &= \mathbb{E}[\mathfrak{C}(X)] = \int_{-\infty}^{+\infty} \mathfrak{C}(x) d_X(x) dx \\
\mathbb{E}[Z^2] &= \mathbb{E}[\mathfrak{C}(X)^2] = \int_{-\infty}^{+\infty} \mathfrak{C}(x)^2 d_X(x) dx
\end{align*}
$$

Let's start with $\mathbb{E}[Z]$:

$$
\begin{align*}
\mathbb{E}[Z] &= \int_{-\infty}^{+\infty} \mathfrak{C}(x) d_X(x) dx \\
&= \int_{-\infty}^{+\infty} \frac{1}{\sqrt{2\pi}\sigma} \frac{\omega - |x|}{\omega + |x|} e^{-\frac{x^2}{2\sigma^2}} dx \\
&= \frac{2}{\sqrt{2\pi}\sigma} \int_{0}^{+\infty} \frac{\omega - x}{\omega + x} e^{-\frac{x^2}{2\sigma^2}} dx \\
&= \frac{2}{\sqrt{2\pi}\sigma} \int_{0}^{+\infty} \frac{2\omega - (\omega + x)}{\omega + x} e^{-\frac{x^2}{2\sigma^2}} dx \\
&= \frac{2}{\sqrt{2\pi}\sigma} \left[ 2 \int_{0}^{+\infty} \frac{\omega}{\omega + x} e^{-\frac{x^2}{2\sigma^2}} dx - \int_{0}^{+\infty} e^{-\frac{x^2}{2\sigma^2}} dx \right] \\
&= \frac{4}{\sqrt{2\pi}\sigma} \int_{0}^{+\infty} \frac{\omega}{\omega + x} e^{-\frac{x^2}{2\sigma^2}} dx - 1 \\
&= \frac{4}{\sqrt{2\pi}\sigma} \int_{0}^{+\infty} \frac{1}{1+u} e^{-\frac{u^2}{2a^2}} du - 1
\end{align*}
$$
$$ 
\begin{aligned}
&= \frac{4}{\sqrt{2\pi}a} \frac{\sqrt{2}a}{2\pi} G_{3,2}^{2,3} \left( 2a^2 \begin{vmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & 0 & 0 \end{vmatrix} \right) - 1 \\
&= \frac{2}{\pi^{3/2}} G_{3,2}^{2,3} \left( 2a^2 \begin{vmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & 0 & 0 \end{vmatrix} \right) - 1
\end{aligned}
$$

From line 2 to 3, we used the parity of function $\mathfrak{C}$, between 6 and 7, a change of variable $u = x/\omega$ is done and $a$ is set to $\sigma/\omega$. Finally, in the second-to-last line, we identify a Meijer-G function [345] evaluated at $2a^2$. Unfortunately, there exist no closed-form of the integral $\int_0^{+\infty} \frac{1}{1+u} e^{-\frac{u^2}{2a^2}} dx$. In this case, a Mellin transform of this integral allows recognizing a Meijer-G function. For other criteria, their first two moments cannot be expressed in a similar closed form. That is why we only provide the theoretical expressions of the expectation and variance of GIoU.

A similar derivation leads to the expression of the second moment of $Z$:

$$ 
\begin{aligned}
\mathbb{E}[Z^2] &= \int_{-\infty}^{+\infty} \mathfrak{C}(x)^2 d_X(x) dx \\
&= \int_{-\infty}^{+\infty} \frac{1}{\sqrt{2\pi}\sigma} \left( \frac{\omega - |x|}{\omega + |x|} \right)^2 e^{-\frac{x^2}{2\sigma^2}} dx \\
&= \frac{2}{\sqrt{2\pi}\sigma} \int_0^{+\infty} \left( \frac{\omega - x}{\omega + x} \right)^2 e^{-\frac{x^2}{2\sigma^2}} dx \\
&= 1 - \frac{8\omega}{\sqrt{2\pi}\sigma} \int_0^{+\infty} \frac{x}{(\omega + x)^2} e^{-\frac{x^2}{2\sigma^2}} dx \\
&= 1 - \frac{8}{\sqrt{2\pi}\sigma} \left[ a^2 - 2\sigma^2 \int_0^{+\infty} \frac{1}{(\omega + x)^3} e^{-\frac{x^2}{2\sigma^2}} dx \right] \\
&= 1 - \frac{8}{\sqrt{2\pi}\sigma} \left[ a^2 - 2a^2 \int_0^{+\infty} \frac{1}{(1+u)^3} e^{-\frac{u^2}{2a^2}} du \right] \\
&= 1 - \frac{8a}{\sqrt{2\pi}} + \frac{16a^2}{\pi^{3/2}} G_{3,2}^{2,3} \left( 2a^2 \begin{vmatrix} -1 & \frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & 0 & 0 \end{vmatrix} \right)
\end{aligned}
$$

From line 2 to 3, we again use the parity of $\mathfrak{C}$, from 4 to 5, an integration by parts is done, and finally, from 5 to 6, we apply the change of variable $u = x/\omega$. Once again, we get an integral that does not have any closed form but can be expressed by another Meijer-G function.

For completeness, we recall here the definition of the Meijer-G function:

$$ G_{p,q}^{m,n} \left( z \begin{vmatrix} a_1 & \dots & a_p \\ b_1 & \dots & b_q \end{vmatrix} \right) = \frac{1}{2\pi i} \int_L \frac{\prod_{j=1}^m \Gamma(b_j - s) \prod_{j=1}^n \Gamma(1 - a_j + s)}{\prod_{j=m+1}^q \Gamma(1 - b_j + s) \prod_{j=n+1}^p \Gamma(a_j - s)} z^s ds, \quad (B.4) $$

where $L$ is the integration path and $\Gamma$ is the gamma function. $m, n, p$ and $q$ are integers while $a_j$ and $b_j$ are real or complex numbers. There are some constraints on these parameters, but we do not detail them here, they can be found in [345].
<figure><img src="image_122.png" /></figure>