Progressive Exploration-Conformal Learning for Sparsely Annotated Object Detection in Aerial Images
Authors: Zihan Lu
Chenxu Wang
Chunyan Xu
Xiangwei Zheng
Zhen Cui
Venue: NeurIPS 2024 poster (2024)
Keywords: Aerial object detection; Sparse annotation; Conformal exploratory learning
Primary area: machine_vision
TL;DR: We address sparsely annotated aerial object detection task with a Progressive Exploration-Conformal Learning (PECL) framework.

Abstract: The ability to detect aerial objects with limited annotation is pivotal to the development of real-world aerial intelligence systems. In this work, we focus on a demanding but practical sparsely annotated object detection (SAOD) in aerial images, which encompasses a wider variety of aerial scenes with the same number of annotated objects. Although most existing SAOD methods rely on fixed thresholding to filter pseudo-labels for enhancing detector performance, adapting to aerial objects proves challenging due to the imbalanced probabilities/confidences associated with predicted aerial objects. To address this problem, we propose a novel Progressive Exploration-Conformal Learning (PECL) framework to address the SAOD task, which can adaptively perform the selection of high-quality pseudo-labels in aerial images. Specifically, the pseudo-label exploration can be formulated as a decision-making paradigm by adopting a conformal pseudo-label explorer and a multi-clue selection evaluator. The conformal pseudo-label explorer learns an adaptive policy by maximizing the cumulative reward, which can decide how to select these high-quality candidates by leveraging their essential characteristics and inter-instance contextual information. The multi-clue selection evaluator is designed to evaluate the explorer-guided pseudo-label selections by providing an instructive feedback for policy optimization. Finally, the explored pseudo-labels can be adopted to guide the optimization of aerial object detector in a closed-looping progressive fashion. Comprehensive evaluations on two public datasets demonstrate the superiority of our PECL when compared with other state-of-the-art methods in the sparsely annotated aerial object detection task.

Progressive Exploration-Conformal Learning for Sparsely Annotated Object Detection in Aerial Images
Zihan Lu', Chenxu Wang', Chunyan Xu Xiangwei Zheng?, Zhen Cui 1. PCA Lab, Key Lab of Intelligent Perception and Systems for Tlig--Dmeessionl Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology. Nanjing, China.
2. Shandong Normal University, Jinan, China. [zihanlu, chenxuwang, cyx, zhen.cui @ njjustedu.cnn xwzhengen@ @ 163.com
Abstract
The ability to detect aerial objects with limited annotation is pivotal to the de- velopment of real-world aerial intelligence systems. In this work, we focus on a demanding but practical sparsely annotated object detection (SAOD) in aerial images, which encompasses a wider variety of aerial scenes with the same num- ber of annotated objects. Although most existing SAOD methods rely fixed thresholding to filter pscudo-labels for enhancing detector performance, adapting aerial objects proves challenging due the imbalanced probabiliiiesccoffdenee associated with predicted aerial objects. To address this problem, we propose DoplooatiionConffonnn
novel Progressive dress the SAOD task, which can adaptively perform the selection of high-quality pseudo-labels in aerial images. Specifically, the pseudo-label exploration can be formulated as a decision-making paradigm by adopting a conformal pseudo-label explorer and multi-clue selection evaluator. The conformal pseudo-label ex- plorer learns an udaptive policy by maximizing the cumulative reward, which can decide how select these high-quality candidates by leveraging their essential characteristics and inter-instance contextual information. The multi-clue selection evaluator is designed to evaluate the explorer-guide pseudo-labed label selections by providing an instructive feedback for policy optimization. Finally, the explored pseudo-labels can be adopted to guide the optimization of aerial object detector a closed-loop progressive fashion. Comprehensive evaluations on two public datasets demonstrate the superiority our PECL when compared with other state-of-the-art methods in the sparsely annotated aerial object detection task. The code will be available at:
Learning (PECL) framework to ad-
Introduction
Recently, object detection has gained widespread attention, but the demand for a large amount of labeled data time-consuming and labor-intensive. To address this challenge, semi-supervised object detection (SSOD) has been proposed to enhance detection performance by utilizing limited labeled samples and a large number unlabeled samples. SSOD methods (I4, 30) 23 (36) mainly focus on general images, and cannot consider the unique characteristics of objects in aerial images, such as dense arrangements, rich contextual relationships, and large complex scenes. As shown in Figure objects in aerial images are more dense and similar compared to general images. For example, the average objects per image are 68.4 vs 7.7 in the DOTA (27) and COCO (3) datasets, respectively. Therefore, address the sparsely annotated object detection (SAOD) task in acrial images, which
"Corrsponning authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) the global aerial image
(b) the local sparsely-annotated objects
Figure I: An example of sparsely annotated objects in an aerial image. Here a small part of objects are annotated,
e.g., large vehicles with the light blue, and small vehicles with the dark blue.
labels a few part of objects in the training set. Compared with the conventional SSOD task, the SAOD can cover more diversity of aerial scenes even with the same number of annotated objects, as illustrated in Figure bb).
The SAOD task mitigates the need for expensive instance-level annotations, but also faces severe challenges of detector optimization, especially only with the limited and sparse annotation objects. In general, most existing SAOD approaches have attempted mine more confidence supervised signals (e.g., pseudo-labels) from a substantial quantity of unlabeled acrial images. For example, Yoon et al. (32) combined an anchor-free object detector with an object tracker to generate dense annotations for training images, Co-mining (26) employed a Siamese network comprising two branches that mutually predict pseudo-label sets. Niiaani et al. (18) utilized a part-aware sampling technique to handle instances within primary categories and used pseudo-labels to exclude un un-annotated regions. However, existing SAOD methods typically focus on using fixed thresholds to filter pseudo-labels lacking the ability to adaptively select high-quality pseudo-labels. Different from conventional objects, some aerial objects characterized by substantial dimensions and notable features (e.g.. planes) exhibit higher predicted probabilities, whereas smaller aerial objects like vehicles are associated with comparatively lower predicted probabilities. Consequently, the discrepancy in probabilities and confidence levels observed in the predicted results poses a severe challenge in identifying pseudo-labed instances by setting a rigorous threshold in the sparsely annotated acrial object detection process. Mcanwhile, the conformal prediction methodology, as elucidated by Shafer et al. (2008) (20), emerges as a potent approach for quantifying uncertainty by harnessing confidence levels to effectively rectify the challenges stemming from imbalanced data.
In this work, we propose a Progressive improve the SAOD performance in aerial images, which can adaptively explore more confident pseudo-labels by considering the imbalance characteristics of different categories. Specifically, we firstly pre-train the detector with these sparsely annotated objects, and employ online clustering to generate class wise knowledge to assist the pseudo-label exploration. The conformal decision problem of pseudo-label exploration is addressed with two essential components: the conformal pscudo-label explorer and multi-clue selection evaluator. The conformal pseudo-labe explorer is responsible for selecting confident pseudo-labels by considering the intrinsic and conformal characteristics of the predicted candidates: while the multi-clue selection evaluator assesses the pseudo-label exploration and provides feedback to guide the optimization of the pseudo-labed explorer. Finally, both the detector updating and the conformal pseudo-label exploration are integrated into a closed-loop. mutually reinforcing framework, promoting the SAOD task.
Innooration-Coffommaal Learning (PECL) framework to
To summarize, the contributions of this work are threefold: i) We propose a progressive exploration- conformable learning framework that integrates the detector updating and the conformal pseudo-label exploration into an iteratively co-enhancing system. ii) We perform the pscudo-label exploration to mine more high-qualit pseudo-labels by considering contextual information in large complex scenes, consisting of two modules: the conformal pseudo-label explorer and multi-clue selection evaluator. iii) Comprehensive evaluations on two public datasets, DOTA (27) and HRSC2016 (15), demonstrate the effectiveness of our PECL, which outperforms the baselines and state-of-in totte-of-th-arn the-art methods by a large margin in the sparsely annotated aerial object detection task.

---

2 Related Work
Semi-/weakly-superre wwwesssseedggggasrlaaaaaaeeeeeee annotated object detection: Due to the extensive annotations requiring a significant investment of time and labor, SSOD and SAOD have attracted considerable attention in recent years. SSOD aims at the detector optimization by using limited labeled samples and large number of unlabeled samples. For example, Pseudo labeling (TO) has been widely used in superviiee learning, which utilizes pre-trained detector to generate labels for unlabeled data. Furthermore, some methods (2) [2;; IT) combined consistency regularization with pscudo labeling. CSD (9) leveraged consistent predictions of horizontally flipped image pairs, STAC (2I) enforced consistency constraints on weakly and strongly augmented image pairs. Unbiased Teacher (14) adopted the focal loss (12) to resolve pseudo-labe bias caused by class imbalance in real labels. SOOD 7 designed rotation-awa aware adaptive weighting loss and global consistency loss for semi- supervised oriented object detection. Furthermore, considerable efforts have been made in weakly- supervised oriented bounding box detection. H2RBox (31) learned object center from horizontal box labels in weak-supervision, used scale and spatial constraints to estimate object dimensions in self-superviiion. PointOBB (16) and Point2RBox (34) used Multiple Instance Learning and Knowledge Combination learn rotated box regression from single point supervision. While SAOD labels part of instances in each image. Many researchers have explored methods to address this challenge. Niitani et al. (88 introduced part-aware sampling that leveraged the logical relationship between parts to guide the sampling process. Co-mining (26) involved a Siamese network with two branches that predicted pseudo-label sets for each other. Rambhatla et al. (19) treated SAOD as a semi- superviper problem focusing on regions, automatically identifying unlabeled foreground objects. Calibrated Teacher (25) calibrated the predicted confidence to match the actual accuracy, ensuring detectors at different training stages share similar confidence distributions and fixed threshold. In contrast, our work focuses on sparsely annotated aerial object detection by adaptively exploring contextual relationships between unlabeled instances in large complex scenes.
Deep reinforcement learning has shown promising results in many decision-making domains. It has also been widely applied in the field of computer vision. Caicedo al. (3) transformed the object detection into a Markov Decision Process to deform bounding boxes for accurate localization. Huang et al. (8) proposed an adaptive tracking method, which used reinforcement learning to select the number of network layers. DSN (37) treated video summarization frames based on the probability. Yu et al. employing a policy network to select suitable repair tools. Walid et al. ①I formulated the landmark localization in 3D medical images as a reinforcement learning problem and introduced actor-critic for localization tasks. Tian et al. (24) introduced a medical image segmentation method based on reinforcement learning, the reward showed the change between the predictions and the ground truth. IVADC-FDRL (17) empowered the agent to learn from real data by integrating detector for anomaly detection and Q-learning for anomaly classification. In this work, we build a progressive exploration- conformal learning process to adaptively select high-quality pseudo-labels, thereby enhancing the supervision signal for sparsely annotated aerial object detection.
Reinforemend/xpllratorr learning in computer vision:
sequence prediction problem, predicting frame probabilities and selecting
33 applied reinforcement learning for image restoration,
3
The Proposed Method
3.1
Problem Definition
The sparsely annotated aerial object detection is to learn a robust aerial detector by employing these sparsely annotated instances and large amount of unlabeled images/regions in the training set. Formally, we define X' = {X,, O, N as the training set, where N represents the total number of training images, X; and O; denote the i-th image sample and the corresponding sparsely annotated object set, respectively. For the i-th image, the annotated object set can be represented as O; = {a] vi 6, J where 2 vi by denote the image region, the class label, the bounding- location of the j-th annotated object, and N₂ (i.e., Na 2 0) refers to the total number of labeled instances.
In the sparsely annotated aerial object detection task, how to mine more confident supervision (e.g.. pseudo-labels) from these unlabeled images/regions critical to promote the detector optimization. In general, most existing sparsely annotated object detection approaches (10) 26 25 adopted
fixed threshold to select lseudo-labels, which can be used to provide more supervised signals. Unlike these typical objects, certain acrial objects characterized by their large, distinct features (c.g..
3

---

planes) exhibit heightened predicted probabilities, whereas aerial objects with small dimensions (e.g,, vehicles) tend have lower predicted probabilities. Therefore, the imbalance the predicted object probabilities could pose challenge in pinpointing pseudo-label samples through a strict thresholding in the sparsely annotated aerial object detection problem. Inspired by the conformal prediction technique (20), which enables uncertainty quantification and utilizes confidence levels to tackle imbalanced data, we propose a novel Progressive Exploration-Conformaae Learning (PECL) framework to address the SAOD task in aerial images.
3.2 PECL framework
Our proposed PECL framework can be built as close-loop iterative learning process between the conformal pseudo-label exploration and the detector updating. Firstly, we can obtain a pre- trained aerial detector θ by employing these sparsely annotated objects in the training set. To provide the confident guide for the subsequent process, we construct class-wise prototypes P = (Pak Rd) C,K
J [==1,km1 with the fully connected layer features of each annotated instance in (38). Here C' is the number of classes, K is the number of prototypes per class, and d the feature dimension of the prototype. We can obtain the candidate set of each input sample X,, i.e., {*1 ). where Nic represents the number of candidates. We perform the conformal pseudo-label exploration to refine and identify these high-quality candidates as pseudo-labels, which can adaptively consider the distinct characteristics of different categories in the candidate set, especially in the aerial images. Here we develop the conformal pseudo-label explorer to facilitate the creation of an adaptive exploration policy, and the multi-clue selection evaluator to appraise the effectiveness of pseudo-label selection.
3.2.1 Conformal Pseudo-label Explorer
To address the imbalanced prediction probabilities across different categories in the acrial scenes, specially design the conformal pseudo-label explorer to effectively probe these high-quality pseudo- labels. Specifically, the conformal pseudo-label explorer is a multi-layer perceptron composed of three fully connected layers. Its objective is to learn an adaptive pseudo-label exploration policy to determine whether to select the current candidate zς. The conformal pseudo-label explorer takes the current exploratory characteristic & as input, and obtains a two-dimensional selection probability distribution T(c) R22, representing the probabilities of taking different selections under the current characteristic. The selection decision at is then obtained by sampling from this distribution, i.c., af π (c)). where decision value of indicates not selecting the candidate, a value of represents selecting the candidate as a pscudo-label. To assist the conformal pseudo-label explorer ग in performing the rational decision, the important point how to design a comprehensive exploratory characteristic of by considering multiple aspects of information. Here the characteristic c = F. Ōf, - is formed by utilizing the characteristics of the sparsely annotated objects O,, the selected pscudo-labels O and the current candidate of in the current i-th acrial image. The exploratory characteristic thus provides a rich and accurate scene description for the conformal pseudo-label explorer, facilitating more effective pseudo-label selection.
Taking the candidate 7 as an example, we explain a detailed depiction of constructing the correspond- ing of at time I. The characteristic of the current candidate मद can be represented as a feature vector, denoted as ( pro((")) Jsim Ar(ū;) determines the precise level of confidence (20) for the candidate ž² to quantify the uncertainty of the predicted probability. Here Ap(z,) can be expressed as a class-conditional probability where the non-conformity scores of all learned prototypes take on a value bigger than equal to the non-conformity score of this candidate under the condition of the certain class V/. Formally:
As(i!) Pro (INC(PE))) NC(E))
ΣK [N((i)Pg))) 2 NC +1
K ++
where NC( =1 --rr((((11)) represents the non-conformity score of the candidate at with the class of and [[:] is the indicator function. A higher non-conformity score indicates greater dissimilarity between the candidate and the reference class distribution. pro(s indicates the predicted probability of candidate Zf. inferred by the optimized detector. feim (z)) computes the

---

maximum cosine similarity between the feature of the candidate x and the prototypes (pot k== of obtain current exploratory characteristic of by
the predicted class label 8f. Based on the above, concatenating the average feature of the sparsely annotated objects O₂ and the selected pseudo-labell Of with the feature of the current candidate ø The conformal exploratory characteristic enables the explorer make effective pseudo-label selection and explore the unlabeled candidates efficiently, leading to the improved performance of the detector.
3.2.2 Multi-clue Selection Evaluator
To facilitate the conformal pseudo-label explorer in learning optimal pseudo-label exploration policy, we introduce a multi-clue selection evaluator Q to assess the current exploratory characteristic of and selection policy at. providing instructive feedback for policy optimization. In essence, the selection evaluator is also constructed as a multi-layer network with three fully connected layers. It takes the current characteristic & and the decision policy af as input and estimates the cumulative reward for the characteristic-decieee
pairs (i.e.. Q(c), af signifying the effectiveness of the current policy. The cumulative reward takes into account the instant exploratory reward r accumulated at each time step during the conformal pseudo-label decision, which measures how well the conformal pseudo-label explorer x performs. A high cumulative reward indicates that the conformal pseudo-label explorer achieves positive candidate selection in decision maassinnn process, and vice versa.
To accurately estimate the cumulative reward, the instant exploratory reward r should reflect whether the current selection at is appropriate or not. We first design a reward function (z()) to evaluate the impact of selecting current candidate o from the perspectives of information entropy and confidence margin, which is specifically represented as follows:
( AH (EU(TE)
where is the weighting parameter. AH (œς) AH
represents the change in the information entropy of the current confident instances belonging to the same class if when the current candidate 74 is introduced into the collection of the sparsely annotated objects O2 and the selected pseudo-labels Ö It can be
expressed as:
AH = (p.) ((4))
( s.t. 7/ = M (m + pro(z)))
(3)
A 1 M Pmoppr( M
where H(-) calculates the information entropy of probability distribution. F.Th from the set {O, Ō U P) related to class 5f and M denotes its total number. The candidate with a high uncertainty will increase the overall entropy. (U( indicates the change in the confidence margin of the current confident instances belonging to the predicted class ÿ when the current candidate i is selected. It can be formulated as:
where (-) denotes the confidence margin, calculating the difference between the highest value and the second highest value in the prediction probability. A larger margin indicates that the candidate is more confident, leading to a positive promotion in the overall confidence margin.
By considering the reward function v²(z)) of the current candidate and the current selection at taken by the conformal pscudo-label explorer, ultimately design the instant exploratory reward rf as follows:
+1 af = ( 1, a = 1,
< Ola 0, (5))) > 0
(5)
0la = 0, (())
where the positive exploratory reward plays a constructive guiding role in the conformal pabudo-label exploration, and vice versa. Then the target value V/ can be defined as the sum of the instant exploratory reward rf and the future cumulative reward Q(c++) _at+1) ) formulated as follows:
= r +70(t)1 .(t1
(6)
5

---

Algorithm The PECL training process
Input: Training set OX {2r v. ff iif=1
Output: Detector e, conformal pseudo-label explorer x, and multi-clue selection evaluator Q
1: Pre-train e with X,, O by minimizing Cartthpor, and construct the class-wise prototypes P:
2: Initialize the experience replay pool D 0;
3: for 1, 3,... .N do
4: Obtain the candidate set (ri Nic for X, with θ;
5: Initialize the selected pseudo-label set O = 0 and the current candidate Et and then obtain the characteristic c/:
6: for t = 1, 3. Nic do
7:
Perform the conformal pseudo-label explorer
Obtain the decision of using и with current characteristic e/: Update O by taking the decision af:
9:
Conduct the multi-clue selection evaluator
Obtain next characteristic of with ,, 0 and O₁
12:
Calculate the reward rf by Eq. (5);
13:
Insert the recording (c.. a;
into D7;
14:
Sample batch from D to update and
end for
16:
Update θ with annotated objects O and the selected pseudo-labels Ô by minimizing Laet (Eq.(8)).
17:
end for
where is a discount factor. Finally, continuously update the conformal pseudo-label explorer and the multi-clue selection evaluator Q through performing gradient descent on the following objective function:
Cref = -V.--Q(c a))
Q(c of)
Appro. target
where the decision at is sampled from (cc).. The first term aims to adjust the predicted cumulative reward to approximate the target value for updating the multi-clue selection evaluator Q. The second term aspires to maximize the cumulative reward, enabling the conformal pseudo-label explorer r to refine the exploration policy. These selected high-quality candidates can in turn promote the capability of the aerial object detector. Furthermore, we use an experience replay pool to store series of data {(c), a rf. of ) and sample batches gradually update the parameters of the conformal pseudo-label explorer and the multi-clue selection evaluator (i.e., teep and Oera).
3.2.3 Progressive detector updating
The ultimate goal of our progressive exploration-conformabee learning framework is optimize an aerial detector by utilizing the sparse annotation and adaptively exploring these high-quality pseudo-labels in the training set. By employing these explored pseudo-labels Ô and the initial sparsely annotated objects O, as supervision signals, we can progressively acquire satisfactory detector O by minimizing the loss function using stochastic gradient descent, Formally,
10:1
Ldet
Cota ( (FR + (reg(a( + Epor (x),
(8)
Σ
(Cass(") 0() + Creg (x
where |O:| and (0Mic] represent the number of sparsely annotated objects and the selected pseudo- labels during the conformal pseudo-label exploration process in the i-th image. J,t and by indicate the the selected pseudo-lahels ./. Los the standard cross-entropy loss used in ReDet (6) and OR-CNN (29) or the focal loss used in $2 A-Net (5) for classification, freg is the SmoothL loss for localization. To construct representative class-wise prototypes for providing adopt the prototype loss Loot referenced in (38). To mitigate the impact of false negatives, we additionally employ the background weighting reduction strategy during the detector optimization process. Both the detector updating and the reinforced pseudo-label exploration are integrated to address the sparsely annotated aerial object detection task in closed-loop, mutually reinforcing fashion. The PECL training process is presented in Algorithm T
class label and bounding-box
confident guidance,

---

Table I: Performance comparisons of different detector baselines for the OBB task on the DOTA dataset
different label rates.
Method
Pl.
BD
BR
GTF
SV
TV
SH
TC
BC
ST
SRF
RA
HA
SP
HC
mAP(%)
5 A-Net [3)
57.27
50.85
20.13
59.11
36,98
32.58
37.97
83.75
49.15
27.89
42.48
48,
18.33
24.54
12.35
40,14
S" A-Ne w/ PECL
73.30
63.17
25.57
62.35
55.87
48.41
59.71
90.00
66.99
53.07
37.35
50.43
27.13
38.45
12.18
50.39
1%
OR-CNN 29
69.05
69.69
23.81
62.53
40.11
37.65
38.22
88
67.52
46.01
51.83
60.68
28
80
36.97
19.39
49.36
OR-CNN PECI
79.57
74.21
32.73
63,70
50.93
48.0 04
48.22
90.15
77.78
56.75
55.75
59.50
41.62
48.
26.51
56.91
ReDer [
60.36
62,98
26.16
66.08
34.24
39 99
29.35
87 42
68.58
50
49.94
56.59
30.75
40.43
18.51
48.10
ReDet PECL
86,09
72.16
38.91
65.05
67.29
72.01
75.40
89.17
78.52
70.42
56.75
57.68
54.09
53.50
18.72
63.72
8 A Net (5)
56.92
48.39
21.41
61.37
42.76
35 06
46.46
85.02
54.32
35.45
38.1 11
50.41
20.56
29.15
3.35
41,92
S A-ee PFCII
74.57
58.63
26.14
62.36
60.68
55.81
69.71
89.64
65.52
62.58
36.53
52.20
31.45
42.58
18.71
53.81
2%
OR CNN 29
68.78
71.00
24.81
63.66
47.06
37.65
45.59
87
38
70.69
51
53.54
57.26
28 20
44.52
23.40
51.65
OR-CNN w/ PECI.
79,80
72.87
31.12
67.52
58.99
51.18
66.
16
90.15
15
74,48
58.96
53.41
60.98
42.97
44.95
26.43
58.66
ReDet [X
59.73
64.67
23,65
72.47
40.07
40.
37
53.99
87 90
64. 66
48
51. 16
57.63
33 20
46.65
23.79
51.20
ReDet w/ PECI.
87.27
73.30
41.54
70.35
67.19
71 88
81.14
89.08
73.33
77.97
55.47
61.82
60.77
55.55
32.09
66.58
A-et (5)
65.85
49.72
21.97
59.97
56.63
52.17
66.53
87.14
57.84
48.72
38.07
50.47
20.78
36.08
19.75
48.78
S A-Ne w/ PECL
78.18
63.18
30.36
63.85
66.22
64.50
13.06
90.26
69.54
64.57
41.92
50.56
32.99
43.54
28.56
57.42
5%
OR-ONN (20)
71,05
24.62
64.69
57.81
57.64
70.99
88
62
53.27
46
60
68
30.71
37.51
31,46
56,30
OR-CNN w/ PECCL
86.95
73.10
30.17
67.68
68.34
69.61
80,10
90.07
74.91
71.95
80
59.82
48.26
52.43
40.06
64.08
ReDet [6]
68.73
31.49
57
63.
88.38
38
62.21
$2.09
56.34
44.69
23.52
56.06
ReDet PECI.
08.38
71.19
64.
72.50
82.51
65
75.85
76.66
50.79
60.95
61.53
61.00
42.08
67.06
5 A-Net [S]
73.92
$7.80
28. 38
62.52
63.08
64
10
71.33
88 10
59 29
60.02
42.47
$2.71
33.22
44.55
14.21
54.39
5 A-Net PECL
80.40
15.51
65.62
65.93
70.60
60
69.21
78.77
69.69
56.31
52.84
53.29
18.71
62.49
10%
OR-CNN (9)
77.51
69.97
33.06
63.23
68
25
69
09
75. 27
88
36
65.87
63.19
52 26
65.89
42.
20
45.6
51.72
62.11
OR-CNN PFCI.
87.07
71.11
37.59
65.78
69.88
72.53
53
82.4
90
77.33
75.82
53.24
65.07
54
58
59.
50.11
67.74
ReDer
[+
78.75
70.17
32.21
69.76
59.90
67.18
18
73.86
87
87.06
66.62
58.84
53.44
56.45
50.91
43.70
35.89
60.32
ReDet PECL
87.23
75,15
42.24
67.21
74.05
73,95
85.42
90.41
84.47
71.95
59.35
57.83
64.48
61.36
45.30
69.36
4 Experiments
4.1 Dataset and setup
Datasets:
We conduct experiments on two public aerial datasets, DOTA (27 and HRSC2016 (T5). to evaluate the performance of our proposed PECL. The DOTA (2) dataset consists of 2806 high- resolution acrial images, where the fully annotated 188282 instances vary in size, aspect ratio and rotation angle. The dataset has 15 object categories, including plane (PL), baseball-diamond (BD), bridge (BR), ground-track-fiele (GTF), small-vehicle (SV), large-vehicle (LV), ship (SH), soccer-ball- field (SBF), tennis-court (TC), basketball-court (BC), storage-tank (ST), roundabout (RA), harbor (HA), swimming-por pool (SP), and helicopter (HC). To meet the requirement of sparse annotation, we adopt a novel protocol where we randomly sample 1%, 2%, 5% and 10% of instances for each class for annotation. While the HRSC2016 (15) dataset contains aerial images with complex backgrounds and diverse ship objects. The training set and validation set contain 436 and 181 images, while the test set consists of 444 images. Here 1% and 2% label rates often lead to over-fitting due to the HRSC2016 dataset is with small amount of images, thus we only generate sparsely labeled versions of the dataset with 5% and 10% label rates. All evaluation experiments are conducted on the test set with the standard mean average precision (mAP) as the performance evaluation metric.
To demonstrate the generality of our PECL, we adopt two-stage detectors, including ReDet (6) and OR-CNN (29), as well as one-stage detector, S² A-Nt (5), serving as the baselines. In our experiments, we set the following hyper-parameters: K=10, f=1,,=0.9) The feature dimension d is set to 1024 for ReDet and OR-CNN, 256 for S A-Net Our models are trained with the mmdetection /4/mmrttae (39) framework on the DOTA dataset for 12 epochs, and on the HRSC2016 dataset for 36 epochs. All the experiments are conducted on two NVIDIA 2080Ti GPUs. For performing the detector optimization, we employ the SGD optimizer with an initial learning rate of 0.01. The learning rate is divided by 10 at the decay steps: 8 and 11 for DOTA, 24 and 33 for HRSC2016. The momentum and weight decay are set to 0.9 and Ic-4. The background/foreground weights are set to 0.3 and 1.0, respectively. The conformal pseudo-label explorer and multi-clue selection evaluator adopt the SGD optimization with learning rates of le-3 and 5e-4, respectively. The maximum capacity of an experience replay pool is set to 1000.
Implementation details:
4.2 Experimental results
DOTA:
We present the detection results for the oriented bounding box (OBB) task and horizontal bounding box (HBB) task on the DOTA (27) dataset, as shown in Table and Table respectively. In general, our PECL has demonstrated remarkable improvements over different basclines across all
7

---

Table Performance comparisons of different de- tector baselines for the OBB HBB task on the HRSC2016 at different label rates.
Table 2: Performance comparisons of different detec- tor baselines for the HBB task on the DOTA dataset different label rates.
mAP(S)
mAP(K)
Method
Method
1%
2%
10%
OBB
1IBB
OBB
IIBB
SeA-Net {
40.04
42.07
48.
53.00
SeA-Net PECL
50.67
53.28
57.82
62.57
Set-Net
57.90
65.09
79.23
Net w/PCCL
75,50
75.84
80.65
85.88
OR-CNN w/ PFCC.
OR-CNN 20
$7.58
49.31
52.91
57.93
57.26
63.74
63.12
67.55
OR CNN (9)
40.90
52.48
55.70
63,22
OR-CNN w/ECL
54.30
62.46
85,50
85,68
ReDet (
48.57
52.69
57.93
61.22
ReDet w/ PECL
64.70
67.46
67.76
69.95
ReDet [
58.28
69.36
79.66
83.36
RoDet wPPEC
75.10
80.42
87.29
88.57
Table 5: Performance comparisons of different strate- gies when selecting pseudo-labols on the DOTA dataset at the 1% label rate.
Table 4: Comparison with statc-of-the-ee methods for the OBB task on the DOTA dataset the 5% label rate. The detection methods are based on rotated- Faster-RCNN (+) and rotated-RetinaNet (1).
Setting
Pseudo-label explorer
Selection
Experience replay mAP(%)
evaluator
mechanism
Setting
Method
mAP(%)
10
Supervised
S'A-Net' 5
48.78
III
II
61.31
60.84
ReDet
$6.06
IV
63.72
SOOD
53.74
Semi-supervised
Table 6: Performance comparisons of different ex-
Unbiased Teacher (4
64.74
ploratory characteristic designs on the DOTA dataset at the 1% label rate.
Calibrated Teacher'
25
S'A-Net' w/PECL
57.42
BRL"
65.04
Setting
Predicted probubility
Feature similarity
Confidence level
mAP(S%)
Co-mining" 26
65.35
60.90
Sparse-umootated
Region-based [19
65.71
61.17
ReDet" w/PECL
67.06
IV
d
/
63.72
label rates. For the OBB task, PECL, achieves performance gains of 10.25%, 7.55%, and 15.62% at the 1 % label rate compared with the baselines, indicating the effectiveness of our approach on different detectors. Furthermore, it is worth noting that our PECL shows greater performance improvement in some densely distributed small objects, such as small-vehicle (SV) and ship (SH), compared to individually presented large objects, such as baseball-diamond (BD) and soccer-ball-feed (SBF) (33.05%, 46.05% vs 9.18%, 6.81% at the 1 1% label rate with ReDet (6)). This demonstrates that PECL takes full advantage of the contextual relationships between instances.
Similarly, for the HBB task, our PECL boosts the performance of the detectors at all label rates, achieving at least 4.43% improvement. Importantly, as to the ReDet baseline. our sparsely trained model could achieve competitive performance to the fully supervised one (69.95% vs 77.47%) by only using a label rate of 10%.
HRSC2016: We further conduct comparative experiments on the HRSC2016 dataset (55, and the performance comparisons of our PECL with different detector baselines for two tasks are presented in Table Overall, our method has brought great improvements across various experimental settings. At the 10% label rate, our PECL achieves mAPs of 80.65%, 85.50% and 87.29% for the OBB task, surpassing the baselines with gains of 12.23%, 29. 900% and 7.63%, respectively. Similar improvements are observed at the 5% label rate. These results underscore the effectivences of our PECL even relatively small-scale acrial dataset, further highlighting its efficacy in sparsely annotated acrial object detection task.
To demonstrate the superiority of our PECL, we further compare it with other state-of-the-an methods applied the sparsely annotated aerial object detection task. To ensure fairness, all comparative experiments are conducted with DesNee550/ReR55-Reee DOTA dataset at the 5% label rate. The results are presented Table It can be observed that our PECL achieves the best performance of 67. 17.06% with ReDet as the baseline, surpassing Unbiased Teacher, BRL, Co-mining, and Region-based by 2.32%. 2.02%, 1.71 1,, and 1.35%, respectively. When S A-Net is used as the baseline, our PECL is 3.68% and 1.61 % higher in mAP compared
cemi/sparse-supervee
on the

---

A52-1
161-1
-02
-4.
(2-
24
23
Figure 2: Performance comparisons of different reward settings at various label rates on the DOTA dataset.
Figure 3: Loss curves of conformal pseudo-label explorer and multi- clue selection evaluator under different action spaces at the 1 1% label rate on the DOTA dataset.
to SOOD and Calibrated Teacher. The reason may be that approach explore high-quality pseudo-labels by addressing the imbalanced prediction probabilities across different categories in the aerial scenes, thereby mitigating the limitation of sparse annotation and boosting the detector optimization.
4.3 Ablation study
We conduct comprehensive ablation experiments to evaluate the performance of our framework under different settings. All the experiments are conducted with the ReDet baseline on the DOTA dataset at the 1 % label rate for the OBB task, unless stated otherwise.
Effeet of each component: As shown in Table we give comparative results with three strategies when generating an optimal pseudo-label selection policy, including the conformal pseudo-label explorer, the multi-clue selection evaluator, and the experience replay mechanism. When only using the conformal pseudo-label explorer, we update the policy using the policy gradient method (22) and the detector achieves mAp of 60.84%. It already outperforms the baseline, verifying the effectiveness of the conformal pseudo-label exploration mechanism. When further adopting the multi-clue selection evaluator, we achieve more improvement with an mAP of 61. 61.31%. 1.. The
reason may be that the evaluator provides positive guidance for policy optimization. Moreover, the accumulated experience alleviate overestimation and bring the best mAP of 63.72%.
We study the effects of three factors of the designed instant exploratory reward, including the information entropy (IE), confidence margin (CM) and value binarization (VB) technique Eq. (5). As presented in Figure 2 the detection results can be improved with different factors of the exploratory reward to some extent. These results indicate that combining information entropy and confidence margin as the exploratory reward is the most effective way to measure the uncertainty/quality of candidates and evaluate the pseudo-label selection. Besides, the VB technique ensures the stable convergence for policy optimization, further enhancing the detection performance.
Reward setting the multi-clue selection evaluator:
Exploratory characteristic design in the conformal pseudo-label explorer: To validate the significance of different characteristics when forming the exploratory characteristic, T presents its comparative results with the predicted probability, feature similarity and confidence level. The predicted probability can measure the accuracy of the candidate, resulting in 1.32% performance boost. The feature similarity indicates the resemblance of the candidate to the class centers, leading to gain of 2.55%. The confidence level reflects the inconsistency between the candidate and the corresponding prototypes, contributing to 2.82% improvement. These results confirm the rationality of our characteristic design and emphasize the importance of appropriate exploratory characteristic to enhance the performance.
Action space in the conformal pseudo-label explorer: We design two sizes of action space (ASS). namely the dimension of the probability vector obtained by the pseudo-label explorer. For 'ASS=1', the explorer outputs the probability of selecting the candidate, while 'ASS=2' gives the likelihood of the candidate being selected or not. Our approach achieves performance of 60.81% ('ASS=l') vs 63.72% ('ASS=2') on the DOTA dataset at the 1% label rate. Moreover, Figure illustrates the loss curves of explorer and evaluator under different action spaces. We can observe when 'ASS=2', the loss values converge faster and reach lower convergence values compared to 'ASS=1'. The

---

reason may be that 'ASS=2' could provide larger decision space, thereby promoting more flexible decision- mocisinn-making and improving the exploration capability the pseudo-label explorer.
Conclusion and Discussion
In this paper, propose Progressive Exploration Exploration-Conformmbbl Learning (PECL) framework to address the sparsely annotated acrial object detection problem, which can adaptively perform the conformal pscudo-label exploration from the large-scals unlabeled acrial images. By introducing the conformal pseudo-label explorer and the multi-clue selection evaluator, we formulate the pseudo- label exploration as conformal decision-making problem. It can select these confident pseudo- labels by considering the specific characteristics of different categories and inter-instance contextual relationships. Extensive experiments and ablation studies demonstrate the effectiveness of our PECL framework.
Limitations. This work also has some limitations. The first limitation is that our work is built for scenes with densely distributed objects, and its performance in general scenes may be suboptimal. Secondly, the application of reinforcement learning algorithms increases the overall training time. In the future, we will continue to explore the potential sparsely annotated object detection and extend this PECL framework to other weakly supervised detection/seegmenttiion tasks in general/aerial images to further reduce the annotation costs.
Acknowledgement
This work is supported by the National Natural Science Foundation of China (Grants Nos. 62372238,,2476133), the Natural Science Foundation of Shandong Province China (Grants Nos. 7R20221.2H003, ZR2020L
References
[1] Walid Abdullah Al and Dong Yun. Partial policy-based reinforcement learning for anatom- ical landmark localization in 3d medical images. IEEE Transuctions Medical Imaging,
1245112245 2019.
[2] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach semi-supervised learning. Advances in Neural Information Processing Systems, 32, 2019.
[3] Juan C Caicedo and Svetlana Lazebnik. Active object localization with deep reinforcement learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 2015.
[4] Kai Chen, Jiaqi Wang. Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng. Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection toolbox and 2019.
benchmark. arXiv preprint
[5] Jiaming Han, Jian Ding, Jie Li, and Gui-Song Xia. Align deep features for oriented object detection. IEEE Transactions on Geoscience and Remote Sensing, 60:1-11, 2021.
[6] Jammnn Han, Jian Ding, Nan Xue, and Gui-Song Xia. Redet: A rotation-equivariant detector
for aerial object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2786-2795, 2021.
[7] Wei Hua, Dingkang Liang. Jingyu Li, Xiaolong Liu, Zhikang Zou, Xiaoqing Ye, and Xiang
Bai. Sood: Towards semi-supervised oriented object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 15558-15567, 2023.
[8] Chen Huang, Simon Lucey, and Deva Ramanan. Learning policies for adaptive tracking with deep feature cascades. In Proceedings of the IEEE International Conference on Computer Vision, pages 105-114, 2017.
[9] Jisoo Jeong, Seungeui Lee, Jeesoo Kim, and Nojun Kwak. Consistency-basee semi-supervised learning for object detection. Advances in Nearal Information Processing Systems, 32, 2019. [10] Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Proceedings of International Conference on Machine Learning, volume 3, page 896, 2013.
[11] Junnan Li, Caiming Xiong, and Steven CH Hoi. Comatch: Semi-supervised learning with contrastive graph regularization. In Proceedings of the IEEE International Conference on Computer Vision, pages 9475-9484, 2021.
10

---

[12] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 2980-2988, 2017.
[13] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In Proceedings of European Conference on Computer Vision, pages 740-755, 2014.
rision.pages [14] Yen-Cheng Liu, Chih-Yar Ma, Zijian He. Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, 7solt Kira, and Peter Vajda. Unbiased teacher for semi-supervised object detection. In Proceedings of International Conference on Learning Representations, 2021.
[15] Zikun Liu, Liu Yuan, Lubin Weng, and Yiping Yang. A high resolution optical satellite image dataset for ship recognition and some new baselines. In International Conference on Pattern Recognition Applications and Methods, volume 2. pages 324-331, 2017.
[16] Junwei Luo, Xue Yang, Yi Yu, Qingyun Li, Junchi Yan, and Yansheng Li. Pointobb: Learning
oriented object detection via single point supervision. In Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, pages 16730-16740, 2024.
[17] Romany F Mansour, José Escoreia- Escoreia-Gutierrez, Margarita Gamarra, Jair A Villanueva, and Nallig Leal. Intelligent video anomaly detection and classification using faster renn with deep reinforcement learning model. Image and Vision Computing, 1 112:104229 2021.
[18] Yusuke Niitani, Takuya Akiba, Tommi Kerola, Toru Ogawa, Shotaro Sano, and Shuji Suzuki. Sampling techniques for large-scale object detection from sparsely annotated objects. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 6510-6518, 2019.
[19] Sai Saketh Rambhatla, Saksham Suri, Rama Chellappa, and Abhinav Shrivastava. Sparsely annotated object detection: A region-based semi-supervised approach. In Proceedings of the IEFE International Conference on Computer Vision, 2023.
[20] Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine
Learning Research, 9(3), 2008.
[21] Kihyuk Sohn, Zizhao Zhang, Chun-Liang Li, Han Zhang, Chen- Yu Lee, and Tomas Pfis- ter. A simple semi-supervised learning framework for object detection. arXiv preprint wiXiv:205.04755: 2020.
[22] Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient meth- ods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 12, 1999.
[23] Yihe Tang, Weifeng Chen, Yijun Luo, and Yuting Zhang. Humble teachers teach better students for semi-supervised object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3132-3141, 2021.
[24] Zhiqiang Tian, Xiangyu Si, Yaoyue Zheng, Zhang Chen, and Xiaojian Li. Multi-step medical image segmentation based on reinforcement learning. Journal of Ambient Intelligence and Humanized Computing, pages 1-12. 2020.
[25] Haohan Wang. Liang Liu, Boshen Zhang, Jiangning Zhang, Wuhao Zhang, Zhenye Gan, Yabiao Wang, Chengjie Wang, and Haoqian Wang. Calibrated teacher for sparsely annotated object detection. In Proceedings of the ÁAAI Conference on Artificial Intelligence, 2023.
[26] Tiancai Wang, Tong Yang, Jiale Cao, and Xiangyu Zhang. Co-mining: Self-supervised learning for sparsely annotated object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2800-2808, 2021.
[27] Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie. Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang. Dota: A large-scale dataset for object detection in aerial images. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 3974-3983, 2018.
(28) Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoe Le. Unsupervised data aug- mentation for consistency training. Advances in Neural Information Processing Systems, 12625566268... 2020.
(29) Xingxing Xie, Gong Cheng, Jiahao Wang, Xiwen Yao, and Junwei Han. Oriented r-cnn for object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 3520,3529, 2021.
[30] Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In Proceedings the IEEE International Conference on Computer Vision, pages 3060-3069, 2021.
[31] Xue Yang. Gefan Zhang, Wentong Li, Xuehui Wang, Yue Zhou, and Junchi Yan. H2rbox: Horizontal box annotation is all you need for oriented object detection. arXiv preprint 10.00:2210.06742. 2022.
11

---

[32] Jihun Yoon, Seungbum Hong, and Min-Kool Choi. Semi-supervised object detection with sparsely annotated dataset. In Proceedings of the IEEE International Conference on Image Processing, pages 719-723, 2021.
[33] Ke Yu, Chao Dong, Liang Lin, and Chen Change Loy. Crafting toolchain for image restoration by deep reinforcement learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages
2018.
[34] Yi Yu, Xue Yang. Qingyun Li, Feipeng Da, Jifeng Dai, Yu Qiao, and Junchi Yan. Point2rbox: Combine knowledge from synthetic visual patterns for end-to-end oriented object detection with single point supervision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 16783-16793 2024.
[35] Han Zhang, Fangyi Chen, Zhiqiang Shen, Qiqi Hao, Chenchen Zhu, and Marios Savvides. Solving missing-annotation object detection with background recalibration loss. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1888-1892, 2020.
[36] Hongyu Zhou, Zheng Ge, Songtao Liu, Weixin Mao, Zeming Li, Haiyan Yu, and Jian Sun. Dense teacher: Dense pseudo-labels for semi-supervised object detection. Proceedings of European Conference on Computer Vision, pages 35-50, 2022.
[37] Kaiyang Zhou, Yu Qiao, and Tao Xiang. Deep reinforcement learning for unsupervised video summarization with
reward. In Proceedings of the AAAI Conference
on Artificial Intelligence, volume 32, 2018
[38] Tianfei Zhou, Wenguan Wang. Ender Konukoglu, and Luc Van Gool. Rethinking semantic segmentation: A prototype view. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pages 2582-2593, 2022.
[39] Yue Zhou, Xue Yang, Gefan Zhang, Jiabao Wang, Yanyi Liu, Liping Hou, Xue Jiang, Xingzhao Liu, Junchi Yan, Chengqi Lyu, al. Mmrotate: A rotated object detection benchmark using pytorch. In Proceedings of the 30th ACM International Conference on Multimedia, pages 7331-7334, 2022.
12

---

A Advancing together with big foundation models.
To adapt the big model era, adopt big-model CLIP for sparsely annotated object detection task. The comparisons are shown 7 We observe that direct use of CLIP for pseudo-label inference cannot cause large improvement, but additionally adding PECL boosts the performance greatly. A possible reason is the difference between aerial images and general images (trained for CLIP).
Table 7: Comparisons after adopting the CLIP model.
Method
1%
2%
5%
10%
ReDet
48. 10
51.20
56.06
60.32
ReDet w/ CLIP
54.01
56.33
60.87
62.27
ReDet w/ PECL
63.72
66.58
67.06
69.36
ReDet PECL+CLIP
64.75
67.65
68.36
69.84
B Comparison with SOTA oriented object detection methods.
We have compared with some SOTA oriented object detection methods (e.g.. A-Nt, OR-CNN and ReDet) in sparsely annotated task, as in Tables 1, of the submitted paper. Here, we add comparison with the latest oriented method LSKNet-T on the DOTA dataset at the 5% label rate, as shown in Table It indicates that our PECL can well-generalized to SOTA oriented detectors.
Table 8: Performance comparison with SOTA oriented detector baselines the 5% label rate the DOTA datasct.
Method
LSKNet-T
LSKNet-T w/ PECL
mAP(%) 58.08
67.89
C Performance comparisons with more label rates setting.
We follow the protocol of previous semi-supervised method (e.g., STAC) utilize the extreme label rates of 1%, 2%, 5%, 10%, which actually consider the abundance of objects in aerial images. For the label rates, c.g., 30%, 50%. 70%, 100%, we also report results on the DOTA dataset in Table 9 can be observed that our PECL can bring certain performance improvement under all sparse label rates, indicating the effectiveness of our method However, under the 100% label rate, the performance deteriorates, possibly because the method introduces other label noise under full supervision, which affects the performance of the detector.
sparsely annotated aerial object detection tasks.
Table Performance comparisons with more label rates.
Method
30%
70%
100%
ReDet
67.48
71.44
72.46
76.25
ReDet w/ PECL
73.03
73.95
74.05
75.18
D Visualization results
The qualitative comparisons of detection results visualized in Figure 4 We can observe that proposed PECL. outputs more meaningful and precise predictions than Region-based and the baseline ReDet on all different tasks and datasets, especially in some densely distributed small instances, i.e., plane/ship. But our method seems to be not precise enough location, which is the future research. These visualization results demonstrate the superiority of our PECL in SAAOD task.
E
Description of prototype loss
Here we describe in detail the prototype loss Topt in the Eq.(8) of the submitted paper. our proposed PECL framework, construct class-wise prototypes P to provide prior confident guidance
13

---

(a) ground truth
(b) ReDet
(c) Region- based
(d) Ours
DOTA
OBB
DOTA
HBB
HRSC2016
CBB
plane
large-vehicle
swimming-pool
basketball-court
ship
Figure 4: Some visualized detection results the 10% label rate. From left right, each column shows the results of the ground truth, supervised bascline (ReDet), Region-basce based, and our PECL. From top bottom, each row represents the results OBB task on DOTA dataset, HBB task on DOTA dataset, and OBB task on HRSC2016 datasct. The colored rectangles represent predictions. The red dashed cireles and the solid red circles indicate false and missed detections, respectively.
for the subsequent reinforced pseudo-label exploration process. In order to obtain more robust and representative prototypes, the prototype loss consists of two components, expressed as:
Cptt = Leont + Pret
where Loont represents a contrastive learning loss between sparsely annotated instances and proto- types to distinguish prototypes of different categories, expressed as:
exp(cos oi
Lcont
log
(10)
k= cxp (cos Ock:)
where cos Ook = Pok F(x))= is the cosine similarity between the feature vector of instance of and the e-th class k-th prototype, F(.) extracts the feature of the last fully connected layer. ky- arg max; {cos 0 ) )m
represents the ki-th prototype of class v/- While Cret is regularization term minimize the distance between the instances and the intra-class prototypes. The term can be formulated as follows:
Cret - (1 Pppk F(x)))?
(11)
By applying these constraints, constantly evolve the prototypes through momentum update method during the detector optimization. Formally,
Pek apak + (1 (11-a)((4 ((x))
(12)
where a E [0, 1] is a momentum coefficient.
F
Ablation for prototype number
The number K determines the number of class centers and the size of calibration set used for confidence level, greatly affecting the subsequent reinforced pseudo-label exploration process. We present the performance comparisons different prototype numbers on DOTA dataset at the 1% label rate in Table 10 It could be observed that when K 10, the detector performance reaches the best
14

---

63.72% mAP. When K=5, the performance decreases, but it is still a considerable performance. When K rises to 50, it brings a certain degree drop. We believe that may be due to the redundant class centers, or sparse annotation that cannot learn prototypes well. Therefore, K=10 can better represent the probability distribution of classes, calculate the confidence level of candidates, and construct better exploratory characteristic, bringing balance between representation and computational complexity.
Table 10: Performance comparisons with different numbers of prototypes for each class the 1 label rate on the DOTA dataset.
K
5
10
20
50
mAP(%))
61.77
63.72
62.91
63.18
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We focus on the problem of sparsely annotated aerial object detection task, i.e., the Conformal Pseudo-label Explorer and Multi-clue Selection Evaluator elaborated in Sec. [.2. and Sec. 3.2.22 We provide extensive quantitative and qualitative experimental In addition, the visualization results are B
results in Sec. 4 and Appendix
F
C
shown in Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [NA]
Justification:
The paper has no limitation.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the implementation details in Sec. 4.1 addition, the codes will be released after the paper is accepted.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
17

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: No codes are included in this submission, but the codes will be publicly available when paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://nips. cc/
pub public// guides Codeeubmme ssi asionPolicyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips. http://ccpc.c..gguu..oouu
Code StrodeSubmii www.mssssiii...... ssionnoliccyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The experimental settings are described Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We conducted a large number of statistical experiments and published the most stable results. The results of some the comparative experiments come from published work in the field. We have reproduced them and obtained consistent results, proving that our experimental results are real and credible, and that the experimental comparison fair.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computer resources are described in Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http
Answer: [Yes]
Justification:
We have reviewed the NeurlIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The open source code framework and dataset used in this study is introduced
in Sec. 4.1
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paaperswi haperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "BKf0PsX13R", "reviewer": "Reviewer_1ik3", "summary": "This paper proposes a new learning framework, PECL, for sparsely annotated object detection (SAOD) in aerial images. The framework introduces a conformal pseudo-label explorer and a multi-clue selection evaluator, which can leverage category-specific characteristics and inter-instance contextual relationships. Comprehensive experiments demonstrate the superior performance of the proposed solution compared to prior approaches. Additionally, the paper includes numerous ablation studies and case studies that examine the various properties of the PECL framework.", "questions": "Please check out the weaknesses section for details.", "limitations": "I don't see potential negative societal impact of this work.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "- Aerial imagery has many useful properties for sparely annotated object detection. It is good that this paper explores those properties to develop a better SAOD framework. \n- The paper includes a substantial number of ablation studies and case studies, which could unveil valuable insights into the proposed method.", "weaknesses": "- The paper highlights PECL's effectiveness in addressing SAOD tasks by comparing the PECL-enabled version with a simple supervised baseline. The results are appealing, which is good. However, when comparing PECL with other SAOD solutions, the improvement is modest, at 1.35%.\n\n- The SAOD task in aerial imagery does not appear to be more challenging than SAOD tasks for general imagery, as objects in aerial imagery typically have only one degree of freedom in rotation. This reduces the significance of PECL's contribution.\n\n- The writing can be improved. The paper is somewhat hard to follow, for example, lines 128-131. Sometimes, even the logic is hard to follow. For instance, in line 31, the average number of objects per image seems irrelevant, as it is a dataset statistic. Neighboring aerial images could be stitched together to form a larger image with more objects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive suggestions. Here is our detailed response.\n\n(1) Our proposed PECL has improved performance by at least 5.63% compared to supervised baselines. Compared to other state-of-the-art methods, we have also achieved at least 1.35% growth. Given the limited improvement of other methods, this achievement is already quite challenging in the current research field, and it also indicates that our method still has a lot of room for improvement.\n\n(2) Compared to SAOD task in general images, SAOD task in aerial images faces greater challenges.\n- Firstly, objects in aerial images have characteristics such as arbitrary orientation, dense distribution, and large aspect ratios.  Figure 1 of overall author rebuttal PDF file intuitively counts the number of objects in each image in the DOTA and COCO datasets, which reflects the relatively high density of objects in aerial images. Motivated by this, our proposed PECL utilizes rich contextual information between objects to mine high-quality pseudo-labels.\n- Secondly, aerial datasets often exhibit serious long tail problems, resulting in significantly imbalanced prediction results for different classes. Figure 2 of overall author rebuttal PDF file shows the statistical analysis on the selected candidates of each class. As in Figure 2, when the threshold is with 0.9, less positive/negative candidates are selected; when reducing it to 0.5, positive/negative candidates are all increased. This confirms that it is difficult to select pseudo-labels with a certainty threshold. In contrast, our PECL employs reinforcement learning to learn an adaptive policy to decide the confidence of selecting candidates.\nTherefore, SAOD in aerial images is necessary and challenging.\n- The diversity of rotation angles remains another challenge in aerial object detection. The introduction of rotation angles is not merely about adding a parameter to learn. The periodicity of angles leads to the boundary discontinuity and square-like problems[1], causing issues in the regression of rotated boxes and affecting the localization quality of rotated bound boxes. This is also an issue worth investigating in the SAOD task and can serve as the motivation for future work. \n\n(3) For minor writing/formatting errors, we will polish this paper in the revision.\n\n## Reference\n[1] Detecting Rotated Objects as Gaussian Distributions and Its 3-D Generalization", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jopns4RHUl", "reviewer": "Reviewer_RekV", "summary": "This paper propose a Progressive Exploration-Conformal Learning (PECL) framework to address the sparsely annotated object detection task, which can adaptively perform the selection of high-quality pseudo-labels.  The pseudo-label exploration are formulated as a decision-making paradigm by adopting a conformal pseudo-label explorer and a multi-clue selection evaluator. Some evaluations on two public datasets demonstrate the superiority of our PECL", "questions": "- I thought of an interesting setting, that is, Sparsely Annotated weakly-supervised roriented Object Detection, such as sparsely annotated HBox. Is it possible to add a related experiment based on H2RBox-v2 and verify it in combination with PECL techniques? Of course, this is not necessary, it's just my sudden thought. If the author can provide it, I am willing to further improve the rating.\n- I am not a professional reinforcement learning researcher, so I am worried whether the training of the entire detector will be slower after the introduction of reinforcement learning. I hope the authors can clarify this.", "limitations": "N/A", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Paper is well writen.\n- It seems interesting to simulate and learn the pseudo-label exploration process using reinforcement learning, i.e. the conformal pseudo-label explorer and the multi-clue selection evaluator.\n- The proposed method seems to work well.\n- The comparative experiments and ablation experiments are quite substantial.", "weaknesses": "- Table 4 lacks comparisons for weakly supervised oriented object detection, which are supported in MMRotate. Specifically, HBox-supervised (H2RBox, H2RBox-v2), Point-supervised (Point2RBox, PointOBB).\n- The authors mainly verified it on aerial image, but the proposed technology does not seem to be strongly related to the specific scenario, and therefore lacks verification in more convincing natural scenarios, e.g. COCO.\n- More object detection methods in aerial image need to be investigated in related work, such as OBB/HBB, full-/weakly-/semi-supervised.\n- After the introduction of reinforcement learning, the visualization of the changes of pseudo labels during the training process needs to be presented. The desired result is a significant improvement in quality.\n- The authors verified it on a CNN-based detector, and it would be more perfect if it could be verified on a DETR-based detector, e.g. ARS-DETR.\n- The left fields of Table 1 (e.g. 1%) and Table 4 (e.g. Sparse-annotated) do not seem to be centered. You can use ```\\multirow``` to adjust them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive suggestions. Here is our detailed response.\n\n(1) H2RBox[1] learns object center localization from horizontal box annotations in the weak supervision branch, utilizes scale and spatial constraints to learn object width, height, and rotation angle information in the self-supervision branch. H2RBox-v2[2] utilizes the principle of symmetry to learn flip-rotation consistency, thereby predicting the rotation angle of object. PointOBB[3] and Point2RBox[4] use Multiple Instance Learning and Knowledge Combination, respectively, to learn rotated box regression from single point supervision. We will add more weakly-/semi-supervised aerial object detection methods in the related work of the revision, which are all aimed at alleviating the time-consuming and labor-consuming of rotated annotations. \n\n(2) To prove the generality of our PECL, we try to conduct experiments on general detection dataset, COCO dataset. Following the setting of BRL[7], four sparsely annotated training sets are generated. In Table 1 of overall author rebuttal PDF file, we compare the different methods with our method trained under three annotation sets. It can be seen that under all sparse conditions, our PECL outperforms the Co-mining[8] by 0.2%, 0.6%, 0.8%, respectively, indicating that our proposed method can mine supervised signals effectively.\n\n(3) Thank you for the reminder; we will add more OBB/HBB, full-/weakly-/semi-supervised aerial object detection methods in the related work of the revision.\n\n(4) The visualization of the changes of pseudo-labels during the training process has been presented in Figure 3 of overall author rebuttal PDF file. The results indicate a significant improvement in the quality of pseudo-labels after introducing reinforcement learning.\n\n(5) We encountered some difficulties when applying our PECL to DETR-based detector, e.g. ARS-DETR[9]. Our PECL is effective in pseudo-label exploration on dense object detectors, because a sufficient number of proposals provide enough samples for statistics-based conformal prediction.However, for DETR-based detector, which has sparse proposals, our PECL cannot perform well. This idea is worth further exploration in future work.\n\n(6) For minor writing/formatting errors, we will polish this paper in the revision.\n\n(7) Sparsely Annotated Weakly-Supervised Oriented Object Detection is indeed an interesting and valuable research setting. However, we encountered difficulties when integrating our PECL with H2RBox-v2. Specifically, H2RBox-v2 is a weakly supervised method based on FCOS[5] detector, whose label assignment strategy requires the participation of all points on the feature map to classify each candidate point as a positive or negative sample. Therefore, this strategy requires ground truth boxes to have very high quality. We need to modify the FCOS label assignment strategy to allow for the assignment of ignored samples (similar to Faster R-CNN[6]), in order to reduce dependence on the quality of ground truth boxes. This involves a lot of work, and we have just proposed a preliminary idea that has not yet been implemented. We will implement this in future research work.\n \n(8) Compared with baselines, during the training process, our PECL has a higher time cost due to an iterative detector updating procedure and an extra conformal pseudo-label exploration procedure; however, in the testing process, our method takes the same time as baselines with the same detection network.\n\n\n## Reference\n[1] H2RBox: Horizonal Box Annotation is All You Need for Oriented Object Detection.\n\n[2] H2RBox-v2: Incorporating Symmetry for Boosting Horizontal Box Supervised Oriented Object Detection.\n\n[3] PointOBB: Learning Oriented Object Detection via Single Point Supervision.\n\n[4] P2RBox: A Single Point is All You Need for Oriented Object Detection.\n\n[5] FCOS: Fully Convolutional One-Stage Object Detection.\n\n[6] Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.\n\n[7] Solving missing-annotation object detection with background recalibration loss.\n\n[8] Co-mining: Self-supervised learning for sparsely annotated object detection.\n\n[9] ARS-DETR: Aspect Ratio-Sensitive Detection Transformer for Aerial Oriented Object Detection.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Sn2DsdKVOc", "reviewer": "Reviewer_hP3V", "summary": "Summary:\nThe paper addresses the challenge of sparsely annotated object detection (SAOD) in aerial images, a critical task for real-world aerial intelligence systems where annotations are limited. Acknowledging the difficulty posed by the imbalanced probabilities and confidences in predicted aerial objects, the paper proposes a novel Progressive Exploration-Conformal Learning (PECL) framework. This framework adaptively selects high-quality pseudo-labels to enhance detection performance. It comprises a conformal pseudo-label explorer and a multi-clue selection evaluator, which together form a decision-making paradigm for pseudo-label exploration. The paper also demonstrates that their method outperforms existing SOTA methods on the DOTA and HRSC2016 datasets.\nContributions:\nThe paper makes significant contributions to the field of semi-supervised aerial object detection by addressing key challenges related to sparse annotations and proposing a robust framework that improves detection performance through adaptive and progressive pseudo-label exploration.", "questions": "See the weaknesses above.", "limitations": "(1) While the paper claims state-of-the-art performance, it lacks the comparative experiments with the SOTA methods from January to May 2024. Adding several comparative experiments with SOTA methods, such as on CVPR/TGRS 2024, can be considered.\n(2) For Section Related Work, it would be better to add some latest works published in 2024. For Section The Proposed Method, it would arouse the readers’ interest to add illustration chart of the proposed SCIR framework.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The overall paper is technically sound.\n(2) For originality and significance, the proposed PECL integrates a conformal pseudo-label explorer with a multi-clue selection evaluator to adaptively select high-quality pseudo-labels, offering a new perspective on handling sparse annotations in object detection.\n(3) For quality and clarity, the methodological framework is well-structured, detailing the iterative process between pseudo-label exploration and detector updating.", "weaknesses": "(1) While the PECL framework is innovative, its complexity might pose challenges for practical implementation. The multi-layer perceptron for the conformal pseudo-label explorer and the iterative training process could be computationally intensive. Discussing the impact of these modules on the overall computational cost and inference time is supposed to consider.\n(2) Can the authors provide more details on the computational efficiency of the PECL framework? Specifically, how does the iterative process impact training time, and are there any optimizations that can be applied to improve efficiency?\n(3) How does the PECL framework specifically address the detection of small and occluded objects in aerial images? Are there any additional strategies or modifications that could further enhance performance for these challenging cases?\n(4) The writing is a little unsatisfactory, especially in the first section, where the rationale for the proposed method is not well explained, making it difficult for readers to develop interest in your approach.\n(5) In Line 71-73, it is difficult for readers to intuitively realize that experimental performance is one of the main contributions without a very convincing data.\n(6) Shown in Line 142, “which can can adaptively” is a error. Moreover, the case of the first letter of the subsection and subsubsection should be unified in Section The Proposed Method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive suggestions. Here is our detailed response.\n\n(1) Our proposed conformal pseudo-label explorer and multi-clue selection evaluator are meticulously encapsulated classes that can be directly invoked. In practical implementation, they exhibit excellent user-friendliness, with no challenges to overcome.\n\n(2) Compared with baselines, during the training process, our PECL has a higher time cost due to an iterative detector updating procedure and an extra conformal pseudo-label exploration procedure; however, in the testing process, our method takes the same time as baselines with the same detection network.\n\n(3) Compared with other semi-supervised/sparse-annotated methods, our PECL demonstrates a degree of superiority in the learning process. Specifically, the training time of our PECL, Unbiased Teacher[1], Co-mining[2] methods are 13.35h, 16.74h, 15.42h on the DOTA dataset at 5% label rate. We conduct all experiments on two NVIDIA 2080Ti gpus.\n\n(4) Additionally, we further consider accelerating the convergence speed of the explorer and evaluator by reducing the complexity of the exploratory state and action spaces, as well as the frequency of updating the target network, thereby enhancing overall computational efficiency.\n\n(5) For small and occluded objects in aerial images, our proposed conformal pseudo-label explorer designs exploratory characteristics based on the non-conformity score, which considers the imbalanced probabilities between small and large objects. In future work, we will incorporate the size of objects into the characteristic design to further enhance the performance and robustness of our algorithm.\n\n(6) In order to intuitively reflect that the experimental performance is one of our main contributions, we will revise Lines 71-73 to:``......demonstrate the effectiveness of our PECL, which outperforms the baselines and state-of-the-art methods by at least 5.63%, 1.35%, respectively......’’.\n\n(7) According to our investigation, there are only two semi/sparse-supervised methods for object detection in aerial images in 2024, Pseudo-Siamese Teacher[3](TGRS 2024), S2O-Det[4](TII 2024), and neither of them is open source. We will add them to related work, and track the comparative experiments in the future.\n\n(8) For Section The Proposed Method, it would arouse the readers’ interest to add illustration chart of the proposed SCIR framework.\n\n(9) According to your suggestion, we have presented a diagram of our proposed PECL framework in Figure 4 of overall author rebuttal PDF file.\n\n(10) For minor writing/formatting errors, we will polish this paper in the revision.\n\n## Reference\n[1] Unbiased teacher for semi-supervised object detection.\n\n[2] Co-mining: Self-supervised learning for sparsely annotated object detection.\n\n[3] Pseudo-Siamese Teacher for Semi-Supervised Oriented Object Detection.\n\n[4] S2O-Det: A Semisupervised Oriented Object Detection Network for Remote Sensing Images.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Accurate annotation is a key factor in ensuring object detection performance. However, the manual annotation process is time-consuming and labor-intensive, especially in remote sensing scenarios with densely arranged objects. 
In this paper, we propose the Sparsely Annotated Object Detection (SAOD) task, which aims to perform remote sensing object detection using sparse annotations (e.g., 5%, 10%).To address the SAOD task in aerial images, we propose an innovative Progressive Exploration-Conformal Learning (PECL) framework, which consists of a conformal pseudo-label explorer and a multi-clue selection evaluator. This framework takes into account the imbalance in predicted results between categories and the rich contextual information to mine confident pseudo-labels, thereby enhancing the performance of the detector. The conformal pseudo-label explorer learns an adaptive pseudo-label policy by maximizing cumulative rewards. The multi-clue selection evaluator aims to optimize the policy by providing guiding feedback. Ultimately, the explored pseudo-labels can be utilized to guide the closed-loop iterative optimization of the aerial detector.
We sincerely respond to each reviewer under their comments.

Meta-review: The paper addresses the challenge of sparsely annotated object detection (SAOD) in aerial images, a demanding but practical task for real-world aerial intelligence systems where there is a wider variety of aerial scenes with the same number of annotated objects. The approach can adaptively perform the selection of high-quality pseudo-labels to enhance detection performance. It comprises a conformal pseudo-label explorer and a multi-clue selection evaluator, which together form a decision-making paradigm for pseudo-label exploration. 

The paper also demonstrates that their method outperforms existing SOTA methods on the DOTA and HRSC2016 datasets. However, when comparing PECL with other SAOD solutions, the improvement is modest, at 1.35%. Also more object detection methods in aerial image need to be investigated in related work, such as OBB/HBB, full-/weakly-/semi-supervised. The authors mainly verified it on aerial image, but the proposed technology does not seem to be strongly related to the specific scenario, and therefore lacks verification in more convincing natural scenarios, e.g. COCO.
Decision: Accept (poster)