PointOBB-v2: Towards Simpler, Faster, and Stronger Single Point Supervised Oriented Object Detection

## Abstract

Abstract Single point supervised oriented object detection has gained attention and made initial progress within the community. Diverse from those approaches relying on one-shot samples or powerful pretrained models (e.g. SAM), PointOBB has shown promise due to its prior-free feature.
In this paper, we propose PointOBB-v2, a simpler, faster, and stronger method to generate pseudo rotated boxes from points without relying on any other prior.
Specifically, we first generate a Class Probability Map (CPM) by training the network with non-uniform positive and negative sampling. We show that the CPM is able to learn the approximate object regions and their contours.
Then, Principal Component Analysis (PCA) is applied to accurately estimate the orientation and the boundary of objects.
By further incorporating a separation mechanism, we resolve the confusion caused by the overlapping on the CPM, enabling its operation in high-density scenarios.
Extensive comparisons demonstrate that our method achieves a training speed 15.58 × \times × faster and an accuracy improvement of 11.60%/25.15%/21.19% on the DOTA-v1.0/v1.5/v2.0 datasets compared to the previous state-of-the-art, PointOBB. This significantly advances the cutting edge of single point supervised oriented detection in the modular track.

## 1 Introduction

Oriented object detection is essential for accurately labeling small and densely packed objects, especially in scenarios like remote sensing imagery, retail analysis, and scene text detection, where Oriented Bounding Boxes (OBBs) provide precise annotations.
However, annotating OBBs is labor-intensive and costly. Therefore, numerous weakly supervised methods have emerged in recent years, including horizontal bounding box supervision and point supervision. Representative methods for horizontal bounding box supervision include H2RBox (Yang et al., ) and H2RBox-v2 (Yu et al., ). In addition, point supervision, which only requires labeling the point and category for each object, significantly reduces the annotation cost. Notable point-supervised methods include P2RBox (Cao et al., ), Point2RBox (Yu et al., ), and PointOBB (Luo et al., ).

As illustrated in Fig. 1, existing point-supervised oriented object detection methods can be broadly categorized into three paradigms: (a) SAM-based methods (Cao et al., ; Zhang et al., ) rely on the powerful SAM (Kirillov et al., ) model, which, although effective in natural images, struggles with cross-domain tasks like aerial imagery, particularly in small object and densely packed scenarios. Additionally, SAM-based methods are slow and memory-intensive due to post-processing; (b) Prior-based Weakly-supervised Oriented Object Detection (WOOD) methods, such as Point2RBox (Yu et al., ),
integrate human priors which reduce generalizability since different datasets require distinct prior knowledge. Further, the end-to-end setup limits flexibility, preventing these methods from leveraging more powerful detectors and benefiting from their performance improvements;
(c) Modular WOOD methods (Luo et al., ) do not rely on manually designed priors and offer greater flexibility by decoupling pseudo-label generation from the detector, making them more suitable for efficient and scalable detection tasks.

As a previous state-of-the-art method, PointOBB falls under the modular WOOD category and offers a feasible solution for point-supervised detection. However, it has several practical limitations: the pseudo-label generation process is very slow, taking approximately 7-8 times longer than the subsequent detector training. Additionally, its training requires significant GPU memory due to multiple view transformations. Moreover, the variability in the number of Region of Interest (RoI) proposals can lead to out-of-memory issues, particularly in dense object scenarios. Although limiting the number of RoI proposals can mitigate this issue, it results in degraded performance.

Figure: Figure 1: Compare with existing point supervised methods, including (a) Prompt OOD (i.e. SAM based); (b-c) Weakly OOD: Prior-based and Modular. OOD means Oriented Object Detection.
Refer to caption: x1.png

Considering the aforementioned issues, our motivation is to design a simpler, faster, and stronger method, which leads to the development of PointOBB-v2. Our approach aims to retain the strengths of the modular WOOD paradigm while addressing the inefficiencies of PointOBB, particularly in terms of speed and memory consumption, making it more suitable for real-world applications.

PointOBB-v2 introduces a novel and concise pipeline that discards the teacher-student structure, achieving significant improvements in the accuracy and generation speed of the pseudo-label, and improving memory efficiency, especially in scenarios with small and dense objects. Specifically, we generate Class Probability Maps (CPM) from point annotations and design a novel sample assignment strategy to capture object contours and orientations from the CPM. Next, we apply non-uniform sampling based on the probability distribution and use Principal Component Analysis (PCA) to determine object boundaries and directions. To address dense object distributions, we design a separation mechanism to reduce confusion in pseudo-label generation caused by connected CPMs.

Experimental results demonstrate that our method consistently improves accuracy, speed, and memory efficiency across various datasets compared to PointOBB, achieving several state-of-the-art results. Specifically, on the DOTA-v1.0 dataset, our method, when using pseudo-labels for training with Rotated FCOS, improves the mAP from 30.08% (PointOBB) to 41.68%, a gain of 11.60% mAP. In more challenging datasets like DOTA-v1.5 and DOTA-v2.0, which contain a higher density of small objects, our method achieves mAP of 36.39% and 27.22%, with respective gains of 25.15% and 21.19% over PointOBB, demonstrating its robustness in handling small and densely packed objects. Furthermore, our pseudo-label generation process is 15.58 times faster, reducing the time from 22.28 hours to 1.43 hours. On the DOTA-v1.5 and DOTA-v2.0 datasets, PointOBB requires limiting the number of RoI proposals due to high memory consumption, while our method operates without such restrictions, with a memory usage of approximately 8GB.

Our contributions are summarized as follows:

- •
We propose a novel and efficient pipeline for point-supervised oriented object detection, which eliminates the time- and memory-consuming teacher-student structure, significantly improving pseudo-label generation speed and reducing memory usage.
- •
Without any additional deep network design, our method relies solely on class probability maps to generate accurate object contours, using efficient PCA to determine object directions and boundaries. We also design a vector constraint method to distinguish small objects in dense scenarios, improving detection performance.
- •
Experimental results show that our method consistently outperforms PointOBB across multiple datasets, achieving 11.60%/25.15%/21.19% gain on the DOTA-v1.0/v1.5/v2.0 datasets, with a 15.58$\times$ speedup in pseudo-label generation and memory usage reduced to approximately 8GB without limiting RoI proposals.

## 2 Related Work

In addition to horizontal detection (Zhao et al., ; Liu et al., ), oriented object detection (Yang et al., ; Wen et al., ) has received extensive attention. In this section, we first introduce oriented detection supervised by rotated boxes. Then, approaches to point-supervised oriented detection and other weakly-supervised settings are discussed.

### 2.1 RBox-supervised Oriented Detection

Representative works include anchor-based detector Rotated RetinaNet (Lin et al., ), anchor-free detector Rotated FCOS (Tian et al., ), and two-stage solutions, e.g. RoI Transformer (Ding et al., ), Oriented R-CNN (Xie et al., ), and ReDet (Han et al., ). Some research enhances the detector by exploiting alignment features, e.g. R^3Det (Yang et al., ) and S^2A-Net (Han et al., ). The angle regression may face boundary discontinuity and remedies are developed, including modulated losses (Yang et al., ; ; Qian et al., ) that alleviate loss jumps, angle coders (Yang & Yan, ; Yang et al., ; Yang & Yan, ; Yu & Da, ) that convert the angle into boundary-free coded data, and Gaussian-based losses (Yang et al., ; ; ; ; Murrugarra-Llerena et al., ) transforming rotated bounding boxes into Gaussian distributions. RepPoint-based methods (Yang et al., ; Hou et al., ; Li et al., ) provide alternatives that predict a set of sample points that bounds the spatial extent of an object.

### 2.2 Point-supervised Oriented Detection

Recently, several methods for point-supervised oriented detection have been proposed: 1) P2RBox (Cao et al., ), PMHO (Zhang et al., ) and PointSAM (Liu et al., ) propose oriented object detection with point prompts by employing the zero-shot Point-to-Mask ability of SAM (Kirillov et al., ). 2) Point2RBox (Yu et al., ) has introduced a novel end-to-end approach based on knowledge combination in this domain. 3) PointOBB (Luo et al., ) achieves point annotation based RBox generation method for oriented object detection through scale-sensitive consistency and multiple instance learning.

Among these methods, P2RBox, PMHO and PointSAM require SAM model pre-trained on massive amounts of labeled data, whereas Point2RBox requires one-shot samples (i.e. human priors) for each category. Although achieving better accuracy, they are not as general as PointOBB. Hence, we choose PointOBB as our baseline to develop a simpler, faster, and stronger method, PointOBB-v2.

### 2.3 Other Weakly-supervised Settings

Compared to the Point-to-RBox, some other weakly-supervised settings have been better studied. These methods are potentially applicable to our Point-to-RBox task setting by using a cascade pipeline, such as Point-to-HBox-to-RBox and Point-to-Mask-to-RBox. In our experiment, cascade pipelines powered by state-of-the-art weakly-supervised approaches are also adopted for comparison. Here, several representative work are introduced.

HBox-to-RBox. The seminal work H2RBox (Yang et al., ) circumvents the segmentation step and achieves RBox detection directly from HBox annotation. With HBox annotations for the same object in various orientations, the geometric constraint limits the object to a few candidate angles. Supplemented with a self-supervised branch eliminating the undesired results, an HBox-to-RBox paradigm is established. An enhanced version H2RBox-v2 (Yu et al., ) is proposed to leverage the reflection symmetry of objects to estimate their angle, further boosting the HBox-to-RBox performance. EIE-Det (Wang et al., ) uses an explicit equivariance branch for learning rotation consistency, and an implicit equivariance branch for learning position, aspect ratio, and scale consistency. Some studies (Iqbal et al., ; Sun et al., ; Zhu et al., ) use additional annotated data for training, which are also attractive but less general.

Point-to-HBox. Several related approaches have been developed, including: 1) P2BNet (Chen et al., ) samples box proposals of different sizes around the labeled point and classify them to achieve point-supervised horizontal object detection. 2) PSOD (Gao et al., ) achieves point-supervised salient object detection using an edge detector and adaptive masked flood fill.

Point-to-Mask. Point2Mask (Li et al., ) is proposed to achieve panoptic segmentation using only a single point annotation per target for training. SAM (Segment Anything Model) (Kirillov et al., ) produces object masks from input point/HBox prompts. Though RBoxes can be obtained from the segmentation mask by finding the minimum circumscribed rectangle, such a complex pipeline can be less cost-efficient and perform worse (Yang et al., ; Yu et al., ).

Figure: Figure 2: Our PointOBB-v2 first generates a Class Probability Map (CPM) with a positive and negative sample assignment strategy during training. It then applies Principal Component Analysis (PCA) to infer object orientation and boundaries to generate pseudo labels.
Refer to caption: x2.png

## 3 Method

Our task focuses on oriented object detection with single point supervision. We first utilize point annotations for each object in the training dataset to generate pseudo labels, which are then used to train an existing detector. As shown in Fig. 2, the model first generates a Class Probability Map (CPM) based on the point annotations. Specifically, during training, we devise a positive and negative sample assignment strategy, where the resulting CPM outlines the rough object contours, with higher probabilities concentrated around the point and along the object axes.

We generate pseudo oriented bounding boxes according to CPM. We perform non-uniform sampling around the point annotation of each object, guided by the probability distribution within the CPM. We convert the sampling process into a weighted probability approach, which maintains the same expected result while eliminating the variance introduced by random sampling. By applying Principal Component Analysis (PCA) to the weighted grid points, we can infer the object’s orientation. We then determine the object boundaries by combining the thresholded CPM with the inferred orientation. Furthermore, to address densely populated object scenarios, we introduce a mechanism for differentiating between closely situated objects, ensuring effective separation and accurate detection.

### 3.1 Class Probability Map Generation

The Class Probability Map (CPM) represents the per-class probability for each point on the feature map, with values ranging between [0, 1]. To generate the CPM, our model first takes an image $I$ of dimension $(C,H,W)$ as input and processes it through a ResNet-50 (He et al., ) backbone with an FPN (Lin et al., ) structure. The final class probability map is derived from the highest-resolution feature map of the FPN, which is then projected through a projection layer. The output is a map of size $(N_{\text{class}},H_{0},W_{0})$. Formally, it is defined as:

$$ $\text{CPM}=\text{Proj}(f(I)_{0}),$ (1) $$

where CPM is the class probability map, $\text{Proj}(\cdot)$ represents the projection layer, and $f(I)_{0}$ is the highest-resolution feature map from the ResNet-50 + FPN.

Figure: Figure 3: Visualization of class probability map (CPM).
Refer to caption: x3.png

### 3.2 Label Assignment

A key component of our approach is the design of a robust sample assignment strategy for both positive and negative samples. This strategy is essential for building an accurate CPM, which outlines rough object contours, concentrating higher probabilities around object centers and along their axes. To ensure reliable separation of objects, especially in densely populated scenarios, our method addresses the challenge of closely situated objects by introducing additional mechanisms for effective differentiation. We illustrate this label assignment in the upper-right part of Fig. 2. The specific details of the sample assignment process are outlined below:

Positive Label Assignment. For positive samples, we select all points within a fixed radius $b_{1}$ (set to 6 in our model) around each point. If a point lies within multiple such radii, it is assigned to the closest center. The condition for positive samples is as follows:

$$ $\begin{split}\exists\text{GT}_{i}\in\text{GT}_{1\sim N},(d(p,\text{GT}_{i})<b_ {1})\land(d(p,\text{GT}_{i})=\min(d(p,\text{GT}_{1\sim N})))\\ \Rightarrow p\text{ is positive},\text{cls}(p)=\text{cls}(\text{GT}_{i}).\end{split}$ (2) $$

Negative Label Assignment. Given $N$ ground truth objects (GT), for each $\text{GT}_{i}$, we identify its nearest neighboring object $\text{GT}_{j}$ based on the Euclidean distance. This gives us a vector $\text{dist}_{min}$ with dimension $[N]$, where each element $\text{dist}_{i}$ represents the minimum distance between $\text{GT}_{i}$ and its closest neighbor. We then draw a circle with radius $\alpha\times\text{dist}_{i}$ around $\text{GT}_{i}$, where $\alpha$ (set to 1 in our model) is a fixed proportional constant. Points outside all such circles are designated as negative samples. The negative sample condition is formulated as:

$$ $\forall\text{GT}_{i}\in\text{GT}_{1\sim N},d(p,\text{GT}_{i})>\alpha\times \text{dist}_{i}\Rightarrow p\text{ is negative}.$ (3) $$

In addition to the above defined negative labels, we also set the middle region between objects as negative to make the boundaries clearer between densely packed objects (denoted as “Neg./M” in ablation Tab. 4). For each $\text{GT}_{i}$, we identify its nearest neighbor $\text{GT}_{j}$ that belongs to the same class. A circle is drawn with a radius $b_{2}$ (set to 4 in our model) and centered at the midpoint of the line connecting $\text{GT}_{i}$ and $\text{GT}_{j}$, and points within this circle are assigned as negative samples. The condition is defined as:

$$ $\begin{split}\forall\text{GT}_{i}\in\text{GT}_{1\sim N},\exists\text{GT}_{j} \in\text{GT}_{1\sim N},d(\text{GT}_{i},\text{GT}_{j})=\min(d(\text{GT}_{i}, \text{GT}_{1\sim N}))\\ \land\text{cls}(\text{GT}_{i})=\text{cls}(\text{GT}_{j})\land d(p,(\text{GT}_{ i}+\text{GT}_{j})/2)<b_{2}\Rightarrow p\text{ is negative}.\end{split}$ (4) $$

Robustness. While we do not explicitly define positive and negative samples based on precise object contours or oriented bounding boxes, which may result in some inaccuracies during label assignment (i.e. incorrectly assigning a small portion of positive or negative samples during training), this does not significantly hinder our method’s ability to learn accurate object contours. These minor label assignment inaccuracies, particularly in densely populated regions or for objects with extreme aspect ratios, do not affect the overall robustness and effectiveness of the approach. As demonstrated in Fig. 3, our strategy is capable of learning the correct contours, even for objects with large aspect ratios and in densely packed scenarios.

### 3.3 Orientation and Boundary estimation via PCA

After obtaining the CPM, we sample points around each ground truth based on the class probabilities, and then apply Principal Component Analysis (PCA) on the sampled points to determine the object’s orientation.
As shown in bottom part of Fig. 2, we sample points around the GT based on the probabilities in the CPM for the corresponding object class. We choose a $7\times 7$ grid centered at the GT with the coordinates of the 49 integer points $z_{1\sim 49}$ as:

$$ $(x,y),x\in[-3,3],y\in[-3,3].$ (5) $$

For each grid point $z_{i}$, we can compute the CPM probability $p_{i}$ and decide whether to sample the point based on this probability. Once we have the sampled points, we apply PCA to find the primary direction of the point set, which represents the object’s orientation. While PCA provides the correct primary direction in expectation, the randomness introduced by sampling can cause variance in the result from a single pass. Although averaging over multiple sampling runs can mitigate this variance, it also increases computational cost.

To address this, we propose an equivalent method that transforms probabilistic sampling into a weighted coordinate transformation. Instead of sampling points probabilistically, we assign a weight of $p_{i}$ to each point $z_{i}$, ensuring the same expected outcome while eliminating the variance caused by random sampling. The covariance matrix is then defined as:

$$ $C_{z}=\sum\limits_{i=1}^{N}p_{i}(z_{i}-\mu_{z})^{\text{T}}(z_{i}-\mu_{z}).$ (6) $$

We then perform eigenvalue decomposition on $C_{z}$:

$$ $C_{z}v_{i}=\lambda_{i}v_{i}.$ (7) $$

The eigenvector $v_{1}$ corresponding to the largest eigenvalue $\lambda_{1}$ is chosen as the primary direction. Since $C_{z}$ is a real symmetric matrix, the secondary direction is guaranteed to be orthogonal to the primary direction. This orthogonality corresponds to the perpendicular relationship between the two adjacent sides of an oriented bounding box.

After identifying the primary and secondary directions, we determine the object boundaries along these directions. Starting from the center, we move along each direction and stop when the value at a position falls below a threshold, indicating the object boundary.

### 3.4 Object Differentiation in Dense Scenarios

In dense scenarios, objects can be difficult to distinguish on the CPM. This can affect the PCA’s ability to determine the object orientation and the boundary identification. To address this, we design a “Vector Constraint Suppression” method to resolve boundary ambiguity.

Vector Constraint Suppression.
Even after determining the correct orientation in dense scenarios, the object boundaries may still be unclear, making it difficult to precisely locate them using the probability threshold described in Sec. 3.3.
In most cases, simply distinguishing between two closely positioned objects is sufficient to define the object’s boundary.

We propose a simple constraint:
For each $\text{GT}_{i}$, we first find its nearest same-class neighbor $\text{GT}_{j}$ and compute the vector $u=\langle\text{GT}_{i},\text{GT}_{j}\rangle$ between $\text{GT}_{i}$ and $\text{GT}_{j}$.
If the angle between this vector and the primary or secondary direction is smaller than a threshold $\alpha$ (set to $\pi/6$ in our model), we consider this direction valid for boundary definition. The boundary is then constrained by the following condition:

$$ $u\cdot v_{k}<\frac{1}{2}\times d(\text{GT}_{i},\text{GT}_{j})\quad\text{if}\ \text{angle}(u,v_{k})<\alpha$ (8) $$

where $v_{k}$ is the primary or secondary direction, $\text{GT}_{j}$ is the nearest same-class object to $\text{GT}_{i}$, and $\frac{1}{2}$ means that the boundary should be closer to $\text{GT}_{i}$ than to $\text{GT}_{j}$.

## 4 Experiment

### 4.1 Datasets

DOTA (Xia et al., ) is a large-scale dataset designed for object detection in aerial images, covering various object categories and complexities. DOTA has three versions:

DOTA-v1.0 has 2,806 images with 188,282 instances across 15 categories.
The images range from 800×800 to 4,000×4,000 pixels and exhibit significant variation in scale and orientation.

DOTA-v1.5 extends DOTA-v1.0 by adding annotations for extremely small objects (less than 10 pixels) and introducing a new category, Container Crane (CC). It includes a total of 403,318 instances while retaining the same image count and dataset split as DOTA-v1.0.

DOTA-v2.0 further expands the dataset to 11,268 images and 1,793,658 instances, covering 18 categories. Two additional categories, Airport (AP) and Helipad (HP), are introduced, providing a more diverse and challenging set of aerial images.

### 4.2 Experimental Settings

Our implementation is based on the MMRotate library (Zhou et al., ). In the pseudo-label generation stage, we train the model for 6 epochs using momentum SGD as the optimizer. We set the weight decay to 1e-4, with an initial learning rate of 0.005, which decays by a factor of 10 after the $\text{4}^{\text{th}}$ epoch. The batch size for training is set to 2. For the detector training phase using pseudo-labels, we use the same detector configurations as the default settings in MMRotate. Throughout the entire training process, random flipping is employed as the only data augmentation technique. Our experiments were accelerated using two GeForce RTX 3090 GPUs.

**Table 1: Results of each category on the DOTA-v1.0 dataset. FCOSR and ORCNN refer to Rotated FCOS and Oriented R-CNN. ^∗ indicates using additional human knowledge priors.**
| Method | PL | BD | BR | GTF | SV | LV | SH | TC | BC | ST | SBF | RA | HA | SP | HC | mAP${}_{\text{50}}$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Point2Mask-RBox | 4.0 | 23.1 | 3.8 | 1.3 | 15.1 | 1.0 | 3.3 | 19.0 | 1.0 | 29.1 | 0.0 | 9.5 | 7.4 | 21.1 | 7.1 | 9.72 |
| P2BNet+H2RBox | 24.7 | 35.9 | 7.0 | 27.9 | 3.3 | 12.1 | 17.5 | 17.5 | 0.8 | 34.0 | 6.3 | 49.6 | 11.6 | 27.2 | 18.8 | 19.63 |
| P2BNet+H2RBox-v2 | 11.0 | 44.8 | 14.9 | 15.4 | 36.8 | 16.7 | 27.8 | 12.1 | 1.8 | 31.2 | 3.4 | 50.6 | 12.6 | 36.7 | 12.5 | 21.87 |
| Point2RBox-RC | 62.9 | 64.3 | 14.4 | 35.0 | 28.2 | 38.9 | 33.3 | 25.2 | 2.2 | 44.5 | 3.4 | 48.1 | 25.9 | 45.0 | 22.6 | 34.07 |
| Point2RBox-SK^∗ | 53.3 | 63.9 | 3.7 | 50.9 | 40.0 | 39.2 | 45.7 | 76.7 | 10.5 | 56.1 | 5.4 | 49.5 | 24.2 | 51.2 | 33.8 | 40.27 |
| PointOBB (FCOSR) | 26.1 | 65.7 | 9.1 | 59.4 | 65.8 | 34.9 | 29.8 | 0.5 | 2.3 | 16.7 | 0.6 | 49.0 | 21.8 | 41.0 | 36.7 | 30.08 |
| PointOBB-v2 (FCOSR) | 64.5 | 27.8 | 1.9 | 36.2 | 58.8 | 47.2 | 53.4 | 90.5 | 62.2 | 45.3 | 12.1 | 41.7 | 8.1 | 43.7 | 32.0 | 41.68 |
| PointOBB (ORCNN) | 28.3 | 70.7 | 1.5 | 64.9 | 68.8 | 46.8 | 33.9 | 9.1 | 10.0 | 20.1 | 0.2 | 47.0 | 29.7 | 38.2 | 30.6 | 33.31 |
| PointOBB-v2 (ORCNN) | 63.7 | 45.6 | 2.0 | 39.5 | 50.5 | 49.6 | 45.4 | 89.8 | 62.9 | 41.3 | 13.6 | 42.8 | 8.9 | 39.5 | 29.5 | 41.64 |
| PointOBB (ReDet) | 24.2 | 75.0 | 0.5 | 60.6 | 59.3 | 46.1 | 45.7 | 6.1 | 10.1 | 25.0 | 0.2 | 50.4 | 30.0 | 45.0 | 31.1 | 33.95 |
| PointOBB-v2 (ReDet) | 65.4 | 52.1 | 2.2 | 44.4 | 55.0 | 49.3 | 51.8 | 89.0 | 70.2 | 47.0 | 16.2 | 43.9 | 13.0 | 43.8 | 29.4 | 44.85 |

**Table 2: Results on DOTA-v1.0/v1.5/v2.0, reporting the mAP${}_{\text{50}}$ metric. FCOSR and ORCNN refer to Rotated FCOS and Oriented R-CNN. ^∗ indicates using additional human knowledge priors.**
| Method | DOTA-v1.0 | DOTA-v1.5 | DOTA-v2.0 |
| --- | --- | --- | --- |
| Point2Mask-RBox | 9.72 | - | - |
| P2BNet+H2RBox | 19.63 | - | - |
| P2BNet+H2RBox-v2 | 21.87 | - | - |
| Point2RBox-RC | 34.07 | 24.31 | 14.69 |
| Point2RBox-SK^∗ | 40.27 | 30.51 | 23.43 |
| PointOBB (FCOSR) | 30.08 | 10.66 | 5.53 |
| PointOBB-v2 (FCOSR) | 41.68 (+11.60) | 30.59 (+19.93) | 20.64 (+15.11) |
| PointOBB (ORCNN) | 33.31 | 10.92 | 6.29 |
| PointOBB-v2 (ORCNN) | 41.64 (+8.33) | 32.01 (+21.09) | 23.40 (+17.11) |
| PointOBB (ReDet) | 33.95 | 11.24 | 6.03 |
| PointOBB-v2 (ReDet) | 44.85 (+10.90) | 36.39 (+25.15) | 27.22 (+21.19) |

### 4.3 Main Results

Results on DOTA-v1.0. As shown in Tab. 1, our method achieves state-of-the-art performance compared to the previous leading approaches, i.e. PointOBB and Point2RBox. Specifically, under three different detectors, our method attains $\text{mAP}_{50}$ scores of 41.68%, 41.64%, and 44.85%, representing improvements of 11.60%, 8.33%, and 10.90% over PointOBB, respectively. Additionally, when compared to Point2RBox-RC, which does not incorporate human prior knowledge, our approach achieves a substantial gain of 10.78%. Even when compared with Point2RBox-SK, which leverages manual sketches to assist in boundary determination, our method still outperforms it by 4.58%. These results demonstrate the robustness and effectiveness of our approach, even without the need for manual prior knowledge.

Results on DOTA-v1.5/v2.0. Both DOTA-v1.5 and DOTA-v2.0 present a higher level of difficulty due to the increased number of densely packed and smaller objects. As Tab. 2 shows, our method demonstrates significant improvements over other approaches on these more challenging datasets, indicating its strength in handling small and densely distributed objects, attributed to the separation mechanism we designed. In comparison to PointOBB, our method achieves substantial gains on both DOTA-v1.5 and DOTA-v2.0, with greater absolute improvements and higher percentages. For instance, when trained with ReDet, our approach improves by 36.39% on DOTA-v1.5 and 27.22% on DOTA-v2.0, corresponding to 25.15% and 21.19% increases, which surpass the 10.90% improvement on DOTA-v1.0. Furthermore, our method consistently outperforms Point2RBox. Even when compared to Point2RBox-SK, which incorporates human prior knowledge, our method achieves improvements of 5.88%/3.79% on DOTA-v1.5/v2.0, respectively.

**Table 3: Comparison of the training time in pseudo-label generation phase and the accuracy between PointOBB and PointOBB-v2. The reported mAP${}_{\text{50}}$ is trained with Rotated FCOS.**
| Method | Epochs | Train Hours | mAP${}_{\text{50}}$ |
| --- | --- | --- | --- |
| PointOBB | 24 | 22.28 | 30.08 |
| PointOBB-v2 | 6 | 1.43 | 41.68 |

Computational Cost.
Our method is highly lightweight, primarily due to its single-branch structure, which eliminates the need for the traditional teacher-student framework. Unlike other methods, we do not require multiple image transformations or consistency constraints within the model. As shown in Tab. 4, the pseudo-label training process for our model takes only 1.43 hours, which is 15.58 times faster than the 22.28 hours required by PointOBB.

In terms of memory consumption, our approach is also more efficient. For dense object scenarios such as DOTA-v2.0, our method uses approximately 8GB of memory, making it suitable for most GPUs. In contrast, PointOBB faces out-of-memory issues when handling such dense scenes, requiring restrictions on the number of RoIs to run properly. However, this limitation severely impacts the detector’s performance, resulting in numerous small objects being undetected.

### 4.4 Ablation Studies

Label Assignment.
Tab. 4 demonstrates the impact of our three label assignment strategies on model performance. In these experiments, different label assignment strategies were used to train and generate the CPM. When a specific strategy was applied to define positive and negative samples, the remaining points were ignored during training. We observed that using a simple circular strategy to determine positive samples resulted in only 23.62%. However, by incorporating a more comprehensive strategy to identify negative samples, the performance increased significantly to 44.75%. Furthermore, assigning middle region between objects to negative (denoted as “Neg./M”) yielded a slight improvement, raising the mAP to 44.85%. This underscores the crucial role of the negative label assignment strategy, which contributes greatly to the performance gains.

PCA Sampling Strategy and Size.
We conducted ablation experiments on the PCA sampling strategy and the range of sampling sizes. As shown in Tab. 6, our weighted method for PCA calculation improves accuracy by 3.45% compared to the probabilistic method. We also found that this improvement primarily benefits classes with larger aspect ratios, such as large vehicles and harbors. This is because CPM in elongated objects exhibit significant probability variation along the short axis of their oriented bounding boxes, and probabilistic sample method introduces considerable instability.
Additionally, we evaluated the impact of the PCA sampling size. As shown in Tab. 8, our method achieves the best performance when the sampling size is set to 7.

**Table 5: Ablation study of sample methods. “Probabilistic” means sample according to the CPM probability.**
| Sample | Probabilistic | Weighted |
| --- | --- | --- |
| mAP${}_{\text{50}}$ | 41.40 | 44.85 |

**Table 7: Ablation study of the sampling size of PCA.**
| Size | mAP${}_{\text{50}}$ |
| --- | --- |
| 5 | 43.35 |
| 7 | 44.85 |
| 9 | 44.17 |
| 11 | 43.63 |

**Table 9: Comparison of pseudo-label generation quality (measured by mIoU) and detection performance (measured by mAP${}_{\text{50}}$) across different datasets. Additionally, we selected three representative categories—Small Vehicle (SV), Large Vehicle (LV), and Ship (SH)—which are abundant and exhibit characteristics of small objects and densely packed distributions. mean represents the average value across all classes in the dataset.**
| Dataset | Method | Memory<br>(GB) | mIoU | mAP${}_{\text{50}}$ |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| SV | LV | SH | mean | SV | LV | SH | mean |  |  |  |
| DOTA-v1.0 | PointOBB | $>$24^∗ | 57.06 | 49.78 | 46.59 | 44.88 | 59.32 | 46.06 | 45.71 | 33.95 |
| PointOBB-v2 | 5.99 | 52.74 | 47.65 | 48.82 | 45.40 | 55.00 | 49.27 | 51.77 | 44.85 |  |
| DOTA-v1.5 | PointOBB | $>$24^∗ | 9.69 | 26.58 | 26.49 | 30.65 | 0.01 | 4.15 | 11.40 | 11.24 |
| PointOBB-v2 | 7.35 | 40.42 | 45.62 | 49.74 | 43.65 | 20.52 | 42.97 | 48.53 | 36.39 |  |
| DOTA-v2.0 | PointOBB | $>$24^∗ | 10.01 | 24.45 | 21.33 | 26.63 | 0.49 | 1.22 | 0.36 | 6.03 |
| PointOBB-v2 | 7.67 | 42.47 | 48.14 | 46.45 | 42.91 | 20.94 | 28.76 | 22.15 | 27.22 |  |
| ^∗ The training stage requires constraining to around 70 RoIs; otherwise it results in out-of-memory errors. |  |  |  |  |  |  |  |  |  |  |

Vector Constraint.
As shown in Tab. 6, applying the vector constraint significantly improves detection performance. Through further analysis, we found that the improvement is primarily concentrated in dense object categories, such as small vehicles, large vehicles, and ships. In contrast, sparse categories like harbors and swimming pools are almost unaffected. This observation aligns with the motivation behind the design of this module, which especially addresses densely packed object scenarios.

### 4.5 Analysis

Label Accuracy.
Recognizing the potential inaccuracies in human annotations, where the center point might not be perfectly labeled, we conducted experiments by adding noise to the center points to evaluate the robustness of our model. We selected different thresholds $\sigma$ and calculated the object’s scale as $S=\sqrt{wh}$. The center points were randomly shifted along a uniformly sampled direction, with the offset distance drawn from a uniform distribution over the range $[-\sigma S,\sigma S]$. We observed a slight performance decrease as the center points were perturbed. As Tab. 8 shows, the average mAP dropped by only 2.27% with a 10% shift. Despite this decline, our method still significantly outperforms PointOBB, demonstrating the strong robustness of our model.

Quality of Pseudo Labels.
As shown in Tab. 9, our method consistently outperforms PointOBB in generating pseudo labels, with performance gains increasing in more challenging datasets like DOTA-v1.5 and DOTA-v2.0. Specifically, we observe mIoU improvements of 0.52%, 13.00%, and 16.28% across DOTA-v1.0, DOTA-v1.5, and DOTA-v2.0, respectively. Notably, although the improvement in DOTA-v1.0 is only 0.52%, training the same detector with our pseudo labels yields a nearly 10% increase in mAP compared to PointOBB. As shown in Fig. 4, the first and second columns illustrate that our model learns more accurate object scales, while the third column demonstrates that, unlike PointOBB—which produces overlapping pseudo labels for small vehicles—our method effectively distinguishes between these densely packed objects.

Dense Object Scenarios. As shown in Tab. 9, we selected three representative categories—Small Vehicle (SV), Large Vehicle (LV), and Ship (SH)—which are characterized by small and densely packed objects. Datasets like DOTA-v1.5 and DOTA-v2.0 introduce a much larger number of these densely packed objects compared to DOTA-v1.0. In these challenging scenarios, our method significantly outperforms PointOBB. For example, in DOTA-v2.0, our method achieves a mean mIoU of 42.91% and mAP of 27.22%, whereas PointOBB drops to 26.63% and 6.03%, respectively. Visualizations further confirm that our model generates better pseudo labels in dense scenes. In terms of detection results, in the last column of Fig. 4, we show a dense scene with 25 large vehicles, where our method detects all of them, while PointOBB identifies only 15.

Limitations.
(a) Our method assigns negative samples based on the minimum distance between objects, requiring at least two point annotations per image. In scenarios with extremely sparse objects, it may degrade the performance.
(b) Some hyperparameters (e.g. the radius in label assignment) are set based on the dataset. They may require adjustments when facing other scenarios.

Figure: Figure 4: Visualization of pseudo-labels and detection results from our model compared to PointOBB. For clarity, label texts for dense objects are hidden.
Refer to caption: x4.png

## 5 Conclusion

In this paper, we introduced PointOBB-v2, a simpler, faster, and stronger approach for single point-supervised oriented object detection. By employing class probability maps and Principal Component Analysis (PCA) for object orientation and boundary estimation, our method improves detection accuracy while discarding the traditional time- and memory-heavy teacher-student structure.
Experimental results demonstrate that PointOBB-v2 consistently outperforms the previous state-of-the-art across multiple datasets, achieving a training speed 15.58× faster and accuracy improvements of 11.60%/25.15%/21.19% on the DOTA-v1.0/v1.5/v2.0 datasets, with notable gains in small and densely packed object scenarios.
Our method achieves a substantial speedup and accuracy boost while using less memory, showcasing its effectiveness for real-world applications.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 3
presentation: 2
questions: - What are the annotation costs for full annotation and single point? How do they compare? Is low performance worth the reduced annotation  cost? 
- How does PointOBBv2 perform on DIOR-R and other oriented object detection datasets?


## Post-rebuttal edit: 

The authors provided satisfactory answers to my questions above. They promise to include these answers in the final version of the paper.
soundness: 3
strengths: Performance improvement over the baseline is impressive. Also, the proposed method is 15x faster to train.
summary: This paper improves PointOBB, an existing weakly supervised oriented object detection method. Oriented object detection is important for analyzing aerial and satellite imagery. The cost of annotation, especially when you want to draw oriented boxes, is high. This paper (and, previously, PointOBB) proposes to annotate an object with just a single point, which is an easier annotation method. The proposed method first generates a class probability map, which creates positive and negative regions. Then, a PCA based analysis is done to estimate principal axes of objects to generate rotated boxes as pseudo annotation. Authors report improved performance over PointBB on the DOTA datasets.
weaknesses: The proposed method (i) is a simple increment to PointOBB and (ii) involves many thresholds, indicating that it might not be an easily generalizable one: 

(i) The method boils down to class probability map (CPM) and PCA.  CPM is a simple positive/negative segmentation. 

(ii) The method has at least 6 thresholds: radius b1 (L244), radius b2 (L262), multiplier to radius (L254), and three others at lines 209, 307 and 320. This suggests that the method is fragile and not easily generalizable to new datasets. I believe it is the authors' responsibility to show that their method is generalizable. Yet, results are reported only on the DOTA dataset. 


The main claim of the paper is that they can do oriented object detection with weak supervision (single point per object). This is good but the performance (around 45)  is drastically behind the fully supervised methods (https://paperswithcode.com/sota/oriented-object-detection-on-dota-1-0), which is around high 70s. So, I think, it  is reasonable to expect a discussion on full annotation cost vs performance degradation. However, none is provided.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: See the Weaknesses section.
soundness: 4
strengths: 1. The paper is innovative, well-organized, and well-written.
2. The proposed method is an improvement of the structure of PointOBB. 
3. The authors conduct comprehensive experiments to demonstrate the effectiveness of the proposed method.
summary: In this paper, the authors propose a single point supervised oriented object detection method called PointOBB-v2, which can generate pseudo rotated boxes from points without relying on any other prior. Compared with PointOBB, PointOBB-v2 introduces a novel pipeline to improve the accuracy and generation speed of the pseudo-label. Extensive experiments on the DOTA-v1.0/v1.5/v2.0 demonstrate the effectiveness of the proposed method.
weaknesses: 1. In related work, it is better to provide a summary of the methods described in section 2.1.
2. The authors conduct experiments on DOTA datasets. How about the effectiveness of the proposed method on other datasets with different object categories and data distributions, such as DIOR?

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: Please check the weaknesses.

## Justification

This paper presents notable advancements in point-supervised oriented object detection, but key limitations reduce its impact. The explanation of the CPM’s effectiveness is incomplete, particularly regarding how it generates clear object contours from coarse labels. While strong in experimental results, the paper would benefit from clearer methodology. 

My initial rating is weak reject. I will finalize the rating after a discussion with the authors and other reviewers.
soundness: 3
strengths: ## 1. Significant Contribution to Point-Supervised Oriented Object Detection

This paper presents a notable advancement in point-supervised oriented object detection, achieving improvements in both accuracy and training efficiency.

## 2. Comprehensive Experimental Analysis

The authors provide a thorough evaluation of the proposed method, including detailed ablation studies that effectively demonstrate its efficacy. The method is validated across three aerial image datasets, further underscoring its robustness and applicability.
summary: The paper presents PointOBB-v2, a faster and more accurate method for single-point supervised oriented object detection that improves upon PointOBB. By generating a Class Probability Map (CPM) with targeted sampling and applying Principal Component Analysis (PCA), PointOBB-v2 effectively estimates object orientation and boundaries without relying on pre-trained models. An added separation mechanism enhances performance in high-density scenarios. PointOBB-v2 achieves 15.58x faster training and significant accuracy gains on DOTA datasets, marking a new state-of-the-art in this domain.
weaknesses: ## 1. Incomplete Explanation of the Class Probability Map (CPM)

- While the overall pipeline of PointOBB-v2 is generally understandable from L.198-211, the crucial workings of the CPM remain unclear. 
- Specifically, key details about CPM training are omitted, including the training objective, activation function, and handling of ignored labels.
- The current structure of the Method section adds to the confusion. Section 3.1 describes CPM generation process, followed by Section 3.2, which outlines label assignment. (The first Section 3.1 assumes that we already have the completed CPM generation model, but the readers cannot understand how it can work.)
- Reordering these sections (such as by first describing the model training or merging both subsection) could enhance clarity.


## 2. Limited Justification for the Effectiveness of CPM

- The paper lacks convincing arguments for why the CPM functions effectively. For instance, the use of a fixed radius for positive and negative sampling lacks a clear rationale, particularly in how it helps define object contours.
- Sections 3.1 and 3.2 provide limited insight into how the model generates a CPM with distinct object regions.
- Figure 2, which illustrates the “Training Target” and the “Output CPM”, raises questions about how the model achieves such precise CPM output even when trained on such coarse training labels. Additional explanation on this discrepancy would be valuable.
- Further clarity on Figure 2 would be beneficial: is the red dot an output from the model or a manual illustration? Similarly, the meanings of colors in Figure 3 (blue, pink, and orange) should be clarified.

## 3. Generalization to Other Oriented Object Detection Scenarios
- While the method appears well-suited for densely packed object scenarios (e.g., aerial or retail images), it’s uncertain whether it generalizes to other tasks, such as text detection. Despite aiming to contribute to general object detection, the experiments are limited to remote sensing images, a subset of oriented object detection.


## Additional Comment

- The concept of CPM bears resemblance to the existing method TricubeNet [A], which, although fully supervised, also represents objects within a 2D map. A brief discussion of the similarities and distinctions between these approaches could enhance the paper’s context.

[A] TricubeNet: 2D Kernel-Based Object Representation for Weakly-Occluded Oriented Object Detection (WACV 2022)

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. What does “the randomness introduced by sampling can cause variance in the result from a single pass” in Line 288 mean? Where does the randomness come from? 
2. Table 2 shows that the improvement on the challenge datasets (DOTA-v1.5/v2.0) is more pronounced compared to DOTA-v1.0. What might account for this difference?
soundness: 3
strengths: 1. The research topic of point-supervised oriented object detection is meaningful and challenging, and the paper is clear and well-organized.
2. As a new version of existing work (PointOBB), the comparison between them is essential. The experiments show the proposed method outperforms the PointOBB baselines in both training speed and accuracy by a considerable amount.
summary: In this paper, PointOBB-v2 is proposed for point-supervised oriented object detection. The authors introduce the learning of Class Probability Map and use Principal Component Analysis to determine the size and rotation of objects. By employing these paradigms, the capability of the PointOBB-v2 is enhanced and significantly outperforms the PointOBB baselines.
weaknesses: 1. Since the authors use “the highest resolution feature map of the FPN”, it is not clear whether the resolution while learning the CPM affects the performance. 
2. There are some minor issues: Please check Figure 2, Line 433, and Line 468. Some equations end with a period, while others end with a comma. Please ensure they are consistent.

---

confidence: 5
contribution: 4
initial_rating: 8
presentation: 3
questions: 1. The paper mentions improved performance in high-density scenarios, but what specific attributes of PointOBB-v2 contribute most to these improvements?
2. Why was Principal Component Analysis (PCA) specifically chosen to estimate object orientation and boundary? Could other methods, such as clustering algorithms or deep learning-based orientation estimation, have been a better alternative? What are the trade-offs?
soundness: 3
strengths: 1. The primary contributions of PointOBB-v2 are its significant speed and efficiency improvements. The pseudo-label generation process is 15.58 times faster, and memory consumption is considerably reduced, requiring only around 8GB of GPU memory, compared to the previous version that faced significant out-of-memory issues. It also achieves higher accuracy on DOTA-v1.0/1.5/2.0 compared to PointOBB,  particularly in dense scenarios. 
2. The experimental section of the paper demonstrates a comprehensive evaluation of PointOBB-v2, including comparisons across multiple versions of the DOTA dataset (v1.0, v1.5, and v2.0). The authors present quantitative results that highlight consistent improvements in accuracy and efficiency over the predecessor, PointOBB. The experimental setup effectively uses multiple detectors and datasets to validate the proposed method’s robustness and adaptability, particularly in dense object detection scenarios.
3. The manuscript is well-structured, with a logical flow that clearly explains the motivation, methodology, and outcomes. The authors effectively contextualize their work within the existing literature, highlighting the gap in current oriented object detection approaches and positioning PointOBB-v2 as an advancement in this domain.
summary: This paper presents PointOBB-v2, a novel approach for single point supervised oriented object detection, aimed at providing a simpler, faster, and stronger methodology compared to its predecessor, PointOBB. Specifically, PointOBB-v2 generates Class Probability Maps (CPM) using a positive and negative sample assignment strategy, which helps to capture the approximate contours of objects. To accurately estimate orientation and boundaries, Principal Component Analysis (PCA) is employed, providing a lightweight approach that avoids the need for prior deep network components or cumbersome teacher-student training frameworks. Experiments show that  PointOBB-v2 significantly improves pseudo-label generation efficiency, achieving a 15.58-fold increase in speed and reduced memory consumption while maintaining superior accuracy.
weaknesses: 1. The paper focuses primarily on aerial imagery datasets, and it is not entirely clear how well PointOBB-v2 would generalize to other domains such as autonomous vehicles or medical imaging. Although the aerial domain has its unique characteristics, a brief discussion or experiment on a different type of dataset could help demonstrate the method's broader applicability and flexibility.
2. There are still some grammatical issues in this paper that could be improved. In "Diverse from those approaches relying on one-shot samples," the phrase "Diverse from" is slightly awkward. Consider changing it to "Unlike approaches that rely on one-shot samples." Also, in "achieving significant improvements in the accuracy and generation speed of the pseudo-label, and improving memory efficiency," "improving" is repetitive here. Consider rewriting it as "achieving significant improvements in pseudo-label accuracy, generation speed, and memory efficiency."

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Please explicitly present the objective function of CPM.
2.  The paper adopts an independent approach for pseudo-label generation and detector training, unlike the conventional semi-supervised teacher-student framework which enables dynamic pseudo-label updates. Please explain it.
3. While identical labels are assigned to positive samples of the same target. Why does the Class Probability Map (CPM) assign higher probabilities to regions surrounding the object centers and along their axes?
4. Please explain the term 'ignored' in Figure 2.
5. Please discuss the hyper parameters ( the radius and α ) in label assignment.
soundness: 3
strengths: 1. The solution proposed by the authors is well-described and well-supported by experimental results. 
2. The paper is easy to follow.
3. The experiments are conducted four datasets of this field.
4. The paper is well-organized, with a clear flow from introduction to conclusion.
summary: This paper begins with a comprehensive review of existing point-supervised methods, examining their relative merits with particular emphasis on the limitations of the state-of-the-art approach, PointOBB in Modular WOOD. The key limitations identified include: 1) inefficient pseudo-label generation, 2) excessive GPU memory consumption, and 3) frequent memory overflow issues during RoI proposal generation.
To overcome these limitations, the authors introduce PointOBB-v2, which presents a streamlined pipeline that eliminates the traditional teacher-student architecture. Instead, it employs class probability maps exclusively for precise object contour generation, incorporating efficient Principal Component Analysis (PCA) for object orientation and boundary determination. Additionally, the study proposes a novel vector constraint mechanism for enhanced detection of small objects in high-density scenarios.
Experimental results reveal substantial improvements in pseudo-label generation speed, memory efficiency, and detection performance.
weaknesses: 1. While the paper demonstrates the empirical effectiveness of PointOBB-v2, it could benefit from providing more theoretical justification or intuition behind the design choices, such as the label assignment strategy and the selection of hyperparameters.
2. The authors acknowledge that some hyperparameters, such as the radius in label assignment, are set based on the dataset. While this approach yields strong results on the tested datasets, it may require manual tuning when applied to new scenarios, potentially limiting the method's adaptability.
3.  Although the paper discusses the advantages of modular approaches' flexibility over end-to-end architectures, this method inherently constrains the dynamic updating of pseudo-labels during detector training—a crucial technique that has demonstrated significant efficacy in semi-supervised learning frameworks.

Meta-review: (a) This paper introduces PointOBB-v2, a single-point supervised oriented object detection method that generates pseudo rotated boxes directly from point annotations without additional priors. It presents a notable advancement in point-supervised oriented object detection, achieving improvements in both accuracy and training efficiency.  

(b) The strengths of the paper are 1) well-organized, well-written writings; 2) significant speed and efficiency improvements (e.g., 15x faster generation speed and reduced memory, only 8GB); 3) sufficient comparisons and experiments which provides a thorough comparison with the original PointOBB.

(c) The first version of this paper involves extensive threshold tuning, lacks experiments on other datasets and discussions of related work, and contains some typos and issues with label assignments. These weaknesses are effectively addressed during the rebuttal.

(d) The most important reasons to accept this paper are that the authors push the boundary of the topic of ``point-supervised oriented object detection'', achieving significant speed and efficiency improvements, making the technique more practical.

Decision: Accept (Poster)