Title: H2RBox-v2: Incorporating Symmetry for Boosting Horizontal Box Supervised Oriented Object Detection
Venue: NeurIPS 2023
Accepted: yes
Keywords: Oriented object detection

self-supervised learning
Abstract: With the rapidly increasing demand for oriented object detection, e.g. in autonomous driving and remote sensing, the recently proposed paradigm involving weakly-supervised detector H2RBox for learning rotated box (RBox) from the more readily-available horizontal box (HBox) has shown promise. This paper presents H2RBox-v2, to further bridge the gap between HBox-supervised and RBox-supervised oriented object detection. Specifically, we propose to leverage the reflection symmetry via flip and rotate consistencies, using a weakly-supervised network branch similar to H2RBox, together with a novel self-supervised branch that learns orientations from the symmetry inherent in visual objects. The detector is further stabilized and enhanced by practical techniques to cope with peripheral issues e.g. angular periodicity. To our best knowledge, H2RBox-v2 is the first symmetry-aware self-supervised paradigm for oriented object detection. In particular, our method shows less susceptibility to low-quality annotation and insufficient training data compared to H2RBox. Specifically, H2RBox-v2 achieves very close performance to a rotation annotation trained counterpart -- Rotated FCOS: 1) DOTA-v1.0/1.5/2.0: 72.31%/64.76%/50.33% vs. 72.44%/64.53%/51.77%; 2) HRSC: 89.66% vs. 88.99%; 3) FAIR1M: 42.27% vs. 41.25%.

H2RBox-v2: Incorporating Symmetry for Boosting Horizontal Box Supervised Oriented Object Detection

Abstract

With the rapidly increasing demand for oriented object detection, e.g. in autonomous driving and remote sensing, the recently proposed paradigm involving weakly-supervised detector H2RBox for learning rotated box (RBox) from the more readily-available horizontal box (HBox) has shown promise. This paper presents H2RBox-v2, to further bridge the gap between HBox-supervised and RBox-supervised oriented object detection. Specifically, we propose to leverage the reflection symmetry via flip and rotate consistencies, using a weakly-supervised network branch similar to H2RBox, together with a novel self-supervised branch that learns orientations from the symmetry inherent in visual objects. The detector is further stabilized and enhanced by practical techniques to cope with peripheral issues e.g. angular periodicity. To our best knowledge, H2RBox-v2 is the first symmetry-aware self-supervised paradigm for oriented object detection. In particular, our method shows less susceptibility to low-quality annotation and insufficient training data compared to H2RBox. Specifically, H2RBox-v2 achieves very close performance to a rotation annotation trained counterpart - Rotated FCOS: 1) DOTA-v1.0/1.5/2.0: 72.31%/64.76%/50.33% vs. 72.44%/64.53%/51.77%; 2) HRSC: 89.66% vs. 88.99%; 3) FAIR1M: 42.27% vs. 41.25%.

1 Introduction

Object detection has been studied extensively, with early research focusing mainly on horizontal detection [1, 2]. When fine-grained bounding boxes are required, oriented object detection [3] is considered more preferable, especially in complex scenes such as aerial images [4-8], scene text [9-13], retail scenes [14], and industrial inspection [15, 16].

With oriented object detection being featured, some horizontal-labeled datasets have been reannotated, such as DIOR [17] to DIOR-R [18] for aerial image (192K instances) and SKU110K [19] to SKU110K-R [14] for retail scene (1,733K instances). Although re-annotation enables the training of oriented detectors, two facts cannot be ignored: 1) Horizontal boxes (HBoxes) are more readily available in existing datasets; 2) Rotated box (RBox) or Mask annotation are more expensive.

BoxInst-RBox

BoxLevelSet-RBox

H2RBox

H2RBox

SAM-RBox

SAM-RBox

SAM-RBox

SAM-RBox

H2RBox-v2

H2RBox-v2

H2RBox-v2

H2RBox-v2

: 31.4

Such a situation raises an interesting question: Can we achieve oriented object detection directly under the weak supervision of HBox annotation? Yang et al. [20] explored this new task setting and proposed the first general solution called HBox-to-RBox (H2RBox) in 2023.

H2RBox gives an effective paradigm and outperforms potential alternatives, including HBox-MaskRBox (generating RBoxes from segmentation mask) powered by BoxInst [21] and BoxLevelSet [22], the state-of-the-art HBox-supervised instance segmentation methods. Yet it is not impeccable at least in two folds: 1) H2RBox learns the angle from the geometry of circumscribed boxes, requiring high annotation quality and a large number of training data containing the same object in various orientations. 2) H2RBox requires quite restrictive working conditions. For example, it is incompatible with rotation augmentation and is sensitive to black borders on images.

If the above requirements are not met, H2RBox may perform far below expectations. As a result, H2RBox does not support some small datasets (e.g. HRSC [5]) and some augmentation methods (e.g. random rotation), leading to bottlenecks in its application scenarios and performance.

By rethinking HBox-supervised oriented object detection with a powerful yet unexplored theory, symmetry-aware self-supervision, we present H2RBox-v2, a new version of H2RBox that exploits the symmetry of objects and solves the above issues.

Motivation: Symmetry is a natural and popular property in vision [24]. For example, in the DOTA dataset [4], many categories (planes, courts, vehicles, ships, etc.) show significant reflection symmetry. For RBox annotation, symmetry is also an important consideration-Humans can intuitively take the direction of the symmetrical axis as the orientation, even without annotation guidance. Is there a way to use symmetry as a supervising tool in deep neural networks for oriented object detection? Will this technique lead to better performance? These are the questions for which this paper is intended.

What is new in H2RBox-v2? 1) A new self-supervised branch leaning angle from symmetry. H2RBox-v2 directly learns angles from the image through the symmetry of objects. That means it is capable of learning the angles correctly even if the HBox annotations are inaccurate in size (not

1 WS: Weakly-supervised, SS: Self-supervised. 2 Based on whether the training converges on the dataset.

precisely the circumscribed rectangle, see Table 9) or when the training data is relatively insufficient (see Table 10). 2) A newly designed CircumIoU loss in the weakly-supervised branch. With this amendment, H2RBox-v2 is now compatible with random rotation augmentation. 3) As a result, H2RBox-v2 gives a higher performance (as is displayed in Fig. 1 and Table 1), further bridging the gap between HBox-supervised and RBox-supervised oriented object detection.

Contributions: 1) This work is the first attempt to explore symmetry for angle regression in oriented object detection, showing that symmetry in images can help learn the orientation of objects in a self-supervised manner. 2) A new training paradigm incorporating symmetry in images is elaborated on, and an integral and stable implementation is provided in this paper. 3) The proposed methods are evaluated via extensive experiments, showing their capability of learning the orientation via symmetry, achieving a higher accuracy than H2RBox. The source code is publicly available.

2 Related Work

RBox-supervised oriented object detection: Representative works include anchor-based detector Rotated RetinaNet [25], anchor-free detector Rotated FCOS [26], and two-stage detectors such as RoI Transformer [27], Oriented R-CNN [28] and ReDet [29]. Besides, R 3 Det [30] and S 2 A-Net [31] improve the performance by exploiting alignment features. Most of the above methods directly perform angle regression, which may face loss discontinuity and regression inconsistency induced by the periodicity of the angle. Remedies have been thus developed, including modulated losses [32, 33] that alleviate loss jumps, angle coders [34-36] that convert the angle into boundary-free coded data, and Gaussian-based losses [37-40] that transform rotated bounding boxes into Gaussian distributions. Additionally, RepPoint-based methods [41-43] provide new alternatives for oriented object detection, which predict a set of sample points that bounds the spatial extent of an object.

HBox-supervised instance segmentation: Compared with HBox-supervised oriented object detection, HBox-supervised instance segmentation, a similar task also belonging to weakly-supervised learning, has been better studied in the literature. For instance, SDI [44] refines the segmentation through an iterative training process; BBTP [45] formulates the HBox-supervised instance segmentation into a multiple-instance learning problem based on Mask R-CNN [46]; BoxInst [21] uses the color-pairwise affinity with box constraint under an efficient RoI-free CondInst [47]; BoxLevelSet [22] introduces an energy function to predict the instance-aware mask as the level set; SAM (Segment Anything Model) [23] produces object masks from input prompts such as points or HBoxes.

Most importantly, these HBox-supervised instance segmentation methods are potentially applicable to HBox-supervised oriented object detection by finding the minimum circumscribed rectangle of the segmentation mask. Such an HBox-Mask-RBox paradigm is a potential alternative for the task we are aiming at and is thus added to our experiments for comparison.

HBox-supervised oriented object detection: Though the oriented bounding box can be obtained from the segmentation mask, such an HBox-Mask-RBox pipeline can be less cost-effective. The seminal work H2RBox [20] circumvents the segmentation step and achieves RBox detection directly from HBox annotation. With HBox annotations for the same object in various orientations, the geometric constraint limits the object to a few candidate angles. Supplemented with a self-supervised branch eliminating the undesired results, an HBox-to-RBox paradigm is established.

Some similar studies use additional annotated data for training, which are also attractive but less general than H2RBox: 1) OAOD [48] is proposed for weakly-supervised oriented object detection. But in fact, it uses HBox along with an object angle as annotation, which is just 'slightly weaker' than RBox supervision. Such an annotation manner is not common, and OAOD is only verified

Backbone

Neck

H

×

W

×256

H

×

W

×3

Angle

AngleCoder

H

×

W

×1

Backbone

Neck

H

×

W

×256

AngleCoder

H

×

W

×1

Angle

H

×

W

×3

Backbone

Neck

H

×

W

×256

Angle

H

×

W

×3

AngleCoder

H

×

W

×1

I

flp

I

I

rot

Backbone

Neck

H

×

W

×256

I

rot

Flipped view

Flip consistency

Rotate consistency

HBox supervision

RBox predictions

Rotated view

• Random rotation

• Reflection padding

Shared

(Shared)

Shared

L

cn

L

cls

BoxCoder

H

×

W

×5

Regression

H

×

W

×4

Classification

H

×

W

×

C

Center-ness

H

×

W

×1

(c) Weakly-supervised (WS) branch

(b) Self-supervised (SS) branch

(a) View generation

Angles learned from symmetry

flp

I

on their self-collected ITU Firearm dataset. 2) Sun et al. [49] proposes a two-stage framework: i) training detector with the annotated horizontal and vertical objects. ii) mining the rotation objects by rotating the training image to align the oriented objects as horizontally or vertically as possible. 3) KCR [50] combines RBox-annotated source datasets with HBox-annotated target datasets, and achieves HBox-supervised oriented detection on the target datasets via transfer learning.

Symmetry detection: Detecting the symmetry (e.g. reflection) has been a long-standing research topic in vision [24], with approaches including key point matching [51, 52], iterative optimization [53, 54], and deep learning [55, 56]. These methods are aimed at tasks quite different from this paper, and we are the first to introduce symmetry-aware learning for oriented object detection.

3 Proposed Method

An overview of the proposed H2RBox-v2 is given in Fig. 2, which consists of a self-supervised (SS) branch (Sec. 3.2) and a weakly-supervised (WS) branch (Sec. 3.3).

3.1 Preliminaries and Approach Overview

In H2RBox-v2, SS branch is designed to learn the orientation of objects from symmetry via consistencies between several views of training images, whilst WS branch learns other properties from the HBox annotation. The core idea of symmetry-aware SS learning is described below.

Definition of reflection symmetry: An object has reflectional symmetry if there is a line going through it which divides it into two pieces that are mirror images of each other [57].

Assume there is a neural network f nn ( · ) that maps a symmetric image I to a real number θ :

To exploit reflection symmetry, we endow the function with two new properties: flip consistency and rotate consistency.

Property I: Flip consistency. With an input image vertically flipped, f nn ( · ) gives an opposite output:

I'

I

0

θ

f

nn

(

I

0

) =

θ

pred

f

nn

(

I

1

) = -

θ

pred

f

nn

(

I'

) =

θ

pred

f

nn

(

I

2

) = -

θ

pred

+

2

θ

I

2

I

1

Flipped

along φ

=

θ

Flipped

vertically

Rotated

by

2

θ

Way 2

Way 1

Axis of symmetry: φ

=

θ

where flp ( I ) is an operator of vertically flipping the image I .

Property II: Rotate consistency. With an input rotated by R , the output of f nn ( · ) also rotates by R :

where rot ( I, R ) is an operator that clockwise rotates the image I by R .

Now we consider that given an image I 0 symmetric about φ = θ sym, assuming the corresponding output is θ pred = f nn ( I 0 ) , the image can be transformed in two ways as shown in Fig. 3:

Way 1 : Flipping I 0 along line φ = θ sym. According to the above definition of reflection symmetry, the output remains the same, i.e. f nn ( I ′ ) = f nn ( I 0 ) = θ pred .

Way2 : Flipping I 0 vertically to obtain I 1 first, and then rotating I 1 by 2 θ sym to obtain I 2 . According to flip and rotate consistencies, the output is supposed to be f nn ( I 2 ) = -θ pred +2 θ sym.

On the ground that ways 1 and 2 are equivalent, the transformed images I ′ and I 2 are identical. And thus f nn ( I ′ ) = f nn ( I 2 ) , finally leading to θ pred = θ sym.

From the above analysis, it arrives at a conclusion that if image I 0 is symmetric about line φ = θ sym and function f nn ( · ) subjects to both flip and rotate consistencies, then f nn ( I 0 ) must be equal to θ sym, so the orientation is obtained. In the following, we further devise two improvements.

Handling the angle periodicity: For higher conciseness, the periodicity is not included in the above formula. To allow images to be rotated into another cycle, the consistencies are modified as:

where k is an integer to keep left and right in the same cycle. This problem is coped with using the snap loss in Sec. 3.4. Meanwhile, the conclusion should be amended as: f nn ( I 0 ) = θ sym + kπ/ 2 , meaning that the network outputs either the axis of symmetry or a perpendicular one.

Extending to the general case of multiple objects: Strictly speaking, our above discussion is limited to the setting when image I 0 contains only one symmetric object. In detail implementation, we use an assigner to match the center of objects in different views, and the consistency losses are calculated between these matched center points. We empirically show on several representative datasets (see Sec. 4.2) that our above design is applicable for multiple object detection with objects not perfectly symmetric, where an approximate axis of each object can be found via learning.

3.2 Self-supervised (SS) Branch

The above analysis suggests that the network can learn the angle of objects from symmetry through the flip and rotate consistencies. A self-supervised branch is accordingly designed.

During the training process, we perform vertical flip and random rotation to generate two transformed views, I flp and I rot, of the input image I , as shown in Fig. 2 (a). The blank border area induced by rotation is filled with reflection padding. Afterward, the three views are fed into three parametershared branches of the network, where ResNet50 [58] and FPN [59] are used as the backbone and the neck, respectively. The random rotation is in range π/ 4 ∼ 3 π/ 4 (according to Table 7).

Similar to H2RBox, a label assigner is required in the SS branch to match the objects in different views. In H2RBox-v2, we calculate the average angle features on all sample points for each object and eliminate those objects without correspondence (some objects may be lost during rotation).

Following the assigner, an angle coder PSC [36] is further adopted to cope with the boundary problem. Table 4 empirically shows that the impact of boundary problem in our self-supervised setting could

Periodic targets

(a) Snap loss: Snapping predictions to

the nearest cycle of targets

θ

(2

k

+1)π+

θ

target

2

k

π+

θ

target

θ

pred(2)

θ

pred(3)

Loss

(1)

Loss

(2)

/

Loss

(3)

-

+

Projecting

B

pred

to

B

gt

's direction

Projected box

Predicted box

Ground-truth box

Intersection area

(b) CircumIoU loss: Between RBox predictions

and circumscribed RBox GTs

be much greater than that in the supervised case, especially in terms of stability. The decoded angles of the original, flipped, and rotated views are denoted as θ , θ flp, and θ rot .

Finally, with formula in Sec. 3.4, the loss L flp can be calculated between θ and θ flp, whereas L rot between θ and θ rot. By minimizing L flp and L rot, the network learns to conform with flip and rotate consistencies and gains the ability of angle prediction through self-supervision.

3.3 Weakly-supervised (WS) Branch

The SS branch above provides the angle of objects. To predict other properties of the bounding box (position, size, category, etc.), a weakly-supervised branch using HBox supervision is further introduced. The WS branch in H2RBox-v2 is inherited from H2RBox, but has some differences:

In H2RBox, the angle is primarily learned by the WS branch, with an SS branch eliminating the undesired results. Comparably, H2RBox-v2 has a powerful SS branch driven by symmetry that can learn the angle independently, and thus the angle subnet is deleted from the WS branch.

H2RBox converts B pred to an HBox to calculate the IoU loss [60] with the HBox B gt, which cannot work with random rotation where B gt becomes an RBox. To solve this problem, CircumIoU loss (detailed in Sec. 3.4) is proposed in H2RBox-v2 to directly calculate L box between B pred and B gt , so that B gt is allowed to be an RBox circumscribed to B pred, as is shown in Fig. 2 (c).

3.4 Loss Functions

Loss for SS branch: As described in Eq. (4), the consistencies encounter the periodicity problem in rotation. To cope with this problem, the snap loss ℓ s 3 is proposed in this paper:

where an illustration is displayed in Fig. 4 (a). The snap loss limits the difference between predicted and target angles to π/ 2 so that the loss calculation upon the two decoded angles will not induce boundary discontinuity. The snap loss is used in both L flp and L rot as:

where L flp is the loss for flip consistency, and L rot for rotate consistency. θ , θ flp, and θ rot are the outputs of the three views described in Sec. 3.2. R is the angle in generating rotated view.

With the above definitions, the loss of the SS branch can be expressed as:

where λ is the weight set to 0.05 according to the ablation study in Table 6.

Loss for the WS branch: The losses in the WS branch are mainly defined by the backbone FCOS detector, including L cls for classification and L cn for center-ness. Different from RBox-supervised

methods, B gt in our task is an RBox circumscribed to B pred, so we re-define the loss for box regression L box with CircumIoU loss as:

where B proj is the dashed box in Fig. 4 (b), obtained by projecting the predicted box B pred to the direction of ground-truth box B gt .

The CircumIoU loss enables H2RBox-v2 to use random rotation augmentation to further improve the performance (see Table 2), which is not supported by H2RBox.

Finally, the loss of the WS branch can be expressed as:

where the hyper-parameters are set to µ cn = 1 and µ box = 1 by default.

Overall loss: The overall loss of the proposed network is the sum of the WS loss and the SS loss:

where µ ss is the weight of SS branch set to 1 by default.

3.5 Inference Phase

Although using a multi-branch paradigm in training (as shown in Fig. 2), H2RBox-v2 does not require the multi-branch or the view generation during inference.

Due to the parameter sharing, the inference only involves the forward propagation of backbone, angle head (from SS), and other heads (i.e. regression, classification, and center-ness from WS). The only additional cost of H2RBox-v2 during inference is the PSC decoding (compared to H2RBox). Thus, FCOS/H2RBox/H2RBox-v2 have similar inference speed (see Table 2).

4 Experiments

Using PyTorch 1.13.1 [61] and the rotation detection tool kits: MMRotate 1.0.0 [62], experiments are carried out. The performance comparisons are obtained by using the same platforms (i.e. PyTorch/MMRotate version) and hyper-parameters (learning rate, batch size, optimizer, etc.).

4.1 Datasets and Settings

DOTA [4]: DOTA-v1.0 contains 2,806 aerial images-1,411 for training, 937 for validation, and 458 for testing, as annotated using 15 categories with 188,282 instances in total. DOTA-v1.5/2.0 are the extended version of v1.0. We follow the default preprocessing in MMRotate: The high-resolution images are split into 1,024 × 1,024 patches with an overlap of 200 pixels for training, and the detection results of all patches are merged to evaluate the performance.

HRSC [5]: It contains ship instances both on the sea and inshore, with arbitrary orientations. The training, validation, and testing set includes 436, 181, and 444 images, respectively. With preprocessing by MMRotate, images are scaled to 800 × 800 for training/testing.

FAIR1M [66]: It contains more than 1 million instances and more than 40,000 images for finegrained object recognition in high-resolution remote sensing imagery. The dataset is annotated with five categories and 37 fine-grained subcategories. We split the images into 1,024 × 1,024 patches with an overlap of 200 pixels and a scale rate of 1.5 and merge the results for testing. The performance is evaluated on the FAIR1M-1.0 server.

Experimental settings: We adopt the FCOS [26] detector with ResNet50 [58] backbone and FPN [59] neck as the baseline method, based on which we develop our H2RBox-v2. We choose average precision (AP) as the primary metric to compare with existing literature. For a fair comparison, all the listed models are configured based on ResNet50 [58] backbone and trained on NVIDIA RTX3090/4090 GPUs. All models are trained with AdamW [67], with an initial learning rate of 5e-5 and a mini-batch size of 2. Besides, we adopt a learning rate warm-up for 500 iterations, and the learning rate is divided by ten at each decay step. '1x', '3x', and '6x' schedules indicate 12, 36, and 72 epochs for training. 'MS' and 'RR' denote multi-scale technique [62] and random rotation augmentation. Unless otherwise specified, '6x' is used for HRSC and '1x' for the other datasets, while random flipping is the only augmentation that is always adopted by default.

4.2 Main Results

DOTA-v1.0: Table 2 shows that H2RBox-v2 outperforms HBox-Mask-Rbox methods in both accuracy and speed. Taking BoxLevelSet-RBox [22] as an example, H2RBox-v2 gives an accuracy of 15.02% higher, and a x7 speed faster by avoiding the time-consuming post-processing (i.e. minimum circumscribed rectangle operation). In particular, the recent foundation model for segmentation i.e. SAM [23] has shown strong zero-shot capabilities by training on the largest segmentation dataset to date. Thus, we use a trained horizontal FCOS detector to provide HBoxes into SAM as prompts, so that the corresponding masks can be generated by zero-shot, and finally the rotated RBoxes are obtained by performing the minimum circumscribed rectangle operation on the predicted Masks. Thanks to the powerful zero-shot capability, SAM-RBox based on ViT-B [68] in Table 2 has achieved 63.94%. However, it is also limited to the additional mask prediction step and the time-consuming post-processing, only 1.7 FPS during inference.

In comparison with the current state-of-the-art method H2RBox, to make it fair, we use the reproduced result of H2RBox, which achieves 70.05%, 2.23% higher than the original paper [20]. In this fair comparison, our method outperforms H2RBox by 2.26% (72.31% vs. 70.05%, both w/o MS). When MS is applied, the improvement is 2.62% (75.35% vs. 77.97%, both w/ MS).

Furthermore, the performance gap between our method and the RBox-supervised FCOS baseline is only 0.13% (w/o MS and RR) and 0.46% (w/ RR). When MS and RR are both applied, our method outperforms RBox-supervised FCOS by 0.57% (78.25% vs. 77.68%), proving that supplemented with symmetry-aware learning, the weakly-supervised learning can achieve performance on a par with the fully-supervised one upon the same backbone neural network. Finally, H2RBox-v2 obtains 80.61% on DOTA-v1.0 by further utilizing a stronger backbone.

Table 5: Ablation with different WS losses.

DOTA-v1.5/2.0: As extended versions of DOTA-v1.0, these two datasets are more challenging, while the results present a similar trend. Still, H2RBox-v2 shows considerable advantages over H2RBox, with an improvement of 3.06% on DOTA-v1.5 and 1.65% on DOTA-v2.0. The results on DOTA-v1.5/2.0, HRSC, and FAIR1M are shown in Table 3.

HRSC: H2RBox [20] can hardly learn angle information from small datasets like HRSC, resulting in deficient performance. Contrarily, H2RBox-v2 is good at this kind of task, giving a performance comparable to fully-supervised methods. Compared to KCR [50] that uses transfer learning from RBox-supervised DOTA to HBox-supervised HRSC, our method, merely using HBox-supervised HRSC, outperforms KCR by 10.56% (89.66% vs. 79.10%).

FAIR1M: This dataset contains a large number of planes, vehicles, and courts, which are more perfectly symmetric than objects like bridges and harbors in DOTA. This may explain the observation that H2RBox-v2, learning from symmetry, outperforms H2RBox by a more considerable margin of 6.33% (42.27% vs. 35.94%). In this case, H2RBox-v2 even performs superior to the fully-supervised FCOS that H2RBox-v2 is based on by 1.02% (42.27% vs. 41.25%).

4.3 Ablation Studies

Loss in SS branch: Table 4 studies the impact of using the snap loss (see Sec. 3.4) and the angle coder. Column 'PSC' indicates using PSC angle coder [36] and "w/o PSC" means the conv layer directly outputs the angle. Column ' ℓ s' with check mark denotes using snap loss (otherwise using smooth L1 loss). Without these two modules handling boundary discontinuity, we empirically find that the loss could fluctuate in a wide range, even failure in convergence (see the much lower results in Table 4). In comparison, when both PSC and snap loss are used, the training is stable.

Loss in WS branch: Table 5 shows that CircumIoU loss with random rotation can further improve the performance, which H2RBox is incapable of. ' ℓ p' means using CircumIoU loss in Sec. 3.4, and otherwise, IoU loss [60] is used following a conversion from RBox to HBox (see H2RBox [20]).

Weights between L flp and L rot : Table 6 shows that on both DOTA and HRSC datasets, λ = 0 . 05 could be the best choice under AP 50 metric, whereas λ = 0 . 1 under AP 75 . Hence in most experiments, we choose λ = 0 . 05 , except for Table 7 where λ = 0 . 1 is used.

Range of view generation: When the rotation angle R is close to 0, the SS branch could fall into a sick state. This may explain the fluctuation of losses under the random rotation within -π ∼ π , leading to training instability. According to Table 7, π/ 4 ∼ 3 π/ 4 is more suitable.

∗ Not stable, occasionally be much lower.

p

100%

30%

10%

H2RBox

AP

50

AP

70.05

55.73

37.71

75

38.38

20.14

6.98

H2RBox-v2

AP

50

AP

75

72.31

61.25

44.61

39.49

27.91

14.97

Branch multiplexing: An additional experiment that randomly selects from 5% flip or 95% rotation in only one branch (the proportion based on λ = 0 . 05 in Table 6) shows AP 50 /AP 75 : 72.24%/39.51% (DOTA w/o MS) while reducing the training time and the RAM usage to H2RBox's level.

Padding strategies: Compared to the performance loss of more than 10% for H2RBox without reflection padding, Table 8 shows that H2RBox-v2 is less sensitive to black borders.

Annotation noise: Table 9 multiplies the height and width of annotated HBox by a noise from the uniform distribution (1 -σ, 1 + σ ) . When σ = 30% , the AP 50 of H2RBox-v2 drops by only 1.2%, less than H2RBox (2.69%), which demonstrates the better robustness of our method.

Training data volume: Table 10 displays that the gap between H2RBox and H2RBox-v2 becomes larger on the sampled version of DOTA, where p denotes the sampling percentage.

5 Conclusion

This paper presents H2RBox-v2, a weakly-supervised detector that learns the RBox from the HBox annotation. Unlike the previous version H2RBox, H2RBox-v2 learns the angle directly from the image of the objects through a powerful symmetry-aware self-supervised branch, which further bridges the gap between HBox-supervised and RBox-supervised oriented object detection.

Extensive experiments are then carried out with the following observations: 1) Compared to H2RBox, H2RBox-v2 achieves higher accuracy on various datasets, with an improvement of 2.32% on average over three versions of DOTA, and 6.33% on the FAIR1M dataset. 2) H2RBox-v2 is less susceptible to low annotation quality and insufficient training data. As a result, it is compatible with small datasets such as HRSC, which H2RBox cannot handle. 3) Even compared to fully-supervised counterpart (i.e. Rotated FCOS), H2RBox-v2 still shows quite competitive performance, proving the effectiveness and the potential of symmetry-aware self-supervision for rotating detection.

Broader impacts. Oriented object detection can be used for military purposes e.g. by remote sensing.

Limitations. There can be cases when the objects are not symmetric in appearance. This may also hold when the objects are partially occluded even from the top view. Moreover, it becomes more challenging to explore the symmetry in 3-D object rotation detection due to occlusion, which yet has been well explored in [39] for autonomous driving by the H2RBox (v1) method [20].

References

Zhong-Qiu Zhao, Peng Zheng, Shou-Tao Xu, and Xindong Wu. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems , 30(11):3212-3232, 2019.

Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A survey. International Journal of Computer Vision , 128(2):261-318, 2020.

Long Wen, Yu Cheng, Yi Fang, and Xinyu Li. A comprehensive survey of oriented object detection in remote sensing images. Expert Systems with Applications , page 119960, 2023.

Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang. Dota: A large-scale dataset for object detection in aerial images. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3974-3983, 2018.

Zikun Liu, Liu Yuan, Lubin Weng, and Yiping Yang. A high resolution optical satellite image dataset for ship recognition and some new baselines. In Proceedings of the International Conference on Pattern Recognition Applications and Methods , volume 2, pages 324-331, 2017.

Xue Yang, Hao Sun, Kun Fu, Jirui Yang, Xian Sun, Menglong Yan, and Zhi Guo. Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sensing , 10(1), 2018.

Xue Yang and Junchi Yan. On the arbitrary-oriented object detection: Classification based approaches revisited. International Journal of Computer Vision , 130:1340-1365, 2022.

Kun Fu, Zhonghan Chang, Yue Zhang, Guangluan Xu, Keshu Zhang, and Xian Sun. Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing , 161:294-308, 2020.

Nibal Nayef, Fei Yin, Imen Bizid, Hyunsoo Choi, Yuan Feng, Dimosthenis Karatzas, Zhenbo Luo, Umapada Pal, Christophe Rigaud, Joseph Chazalon, Wafa Khlif, Muhammad Muzzamil Luqman, JeanChristophe Burie, Cheng-lin Liu, and Jean-Marc Ogier. Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification - rrc-mlt. In IAPR International Conference on Document Analysis and Recognition , volume 01, pages 1454-1459, 2017.

Jianqi Ma, Weiyuan Shao, Hao Ye, Li Wang, Hong Wang, Yingbin Zheng, and Xiangyang Xue. Arbitraryoriented scene text detection via rotation proposals. IEEE Transactions on Multimedia , 20(11):3111-3122, 2018.

Minghui Liao, Zhen Zhu, Baoguang Shi, Gui-Song Xia, and Xiang Bai. Rotation-sensitive regression for oriented scene text detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5909-5918, 2018.

Xuebo Liu, Ding Liang, Shi Yan, Dagui Chen, Yu Qiao, and Junjie Yan. Fots: Fast oriented text spotting with a unified network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5676-5685, 2018.

Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. East: An efficient and accurate scene text detector. In IEEE Conference on Computer Vision and Pattern Recognition , pages 2642-2651, 2017.

Xingjia Pan, Yuqiang Ren, Kekai Sheng, Weiming Dong, Haolei Yuan, Xiaowei Guo, Chongyang Ma, and Changsheng Xu. Dynamic refinement network for oriented and densely packed object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11207-11216, 2020.

Yuekai Liu, Hongli Gao, Liang Guo, Aoping Qin, Canyu Cai, and Zhichao You. A data-flow oriented deep ensemble learning method for real-time surface defect inspection. IEEE Transactions on Instrumentation and Measurement , 69(7):4681-4691, 2020.

Hongjin Wu, Ruoshan Lei, and Yibing Peng. Pcbnet: A lightweight convolutional neural network for defect inspection in surface mount technology. IEEE Transactions on Instrumentation and Measurement , 71:1-14, 2022.

Ke Li, Gang Wan, Gong Cheng, Liqiu Meng, and Junwei Han. Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing , 159:296-307, 2020.

Gong Cheng, Jiabao Wang, Ke Li, Xingxing Xie, Chunbo Lang, Yanqing Yao, and Junwei Han. Anchorfree oriented proposal generator for object detection. IEEE Transactions on Geoscience and Remote Sensing , 2022.

Eran Goldman, Roei Herzig, Aviv Eisenschtat, Jacob Goldberger, and Tal Hassner. Precise detection in densely packed scenes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5227-5236, 2019.

Xue Yang, Gefan Zhang, Wentong Li, Xuehui Wang, Yue Zhou, and Junchi Yan. H2rbox: Horizontal box annotation is all you need for oriented object detection. International Conference on Learning Representations , 2023.

Zhi Tian, Chunhua Shen, Xinlong Wang, and Hao Chen. Boxinst: High-performance instance segmentation with box annotations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5443-5452, 2021.

Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Xiansheng Hua, and Lei Zhang. Box-supervised instance segmentation with level set evolution. In European Conference on Computer Vision , 2022.

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643 , 2023.

Christopher Funk, Seungkyu Lee, Martin R. Oswald, Stavros Tsogkas, Wei Shen, Andrea Cohen, Sven Dickinson, and Yanxi Liu. 2017 iccv challenge: Detecting symmetry in the wild. In IEEE International Conference on Computer Vision Workshops , pages 1692-1701, 2017.

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence , 42(2):318-327, 2020.

Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object detection. In IEEE/CVF International Conference on Computer Vision , pages 9626-9635, 2019.

Jian Ding, Nan Xue, Yang Long, Gui-Song Xia, and Qikai Lu. Learning roi transformer for oriented object detection in aerial images. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2849-2858, 2019.

Xingxing Xie, Gong Cheng, Jiabao Wang, Xiwen Yao, and Junwei Han. Oriented r-cnn for object detection. In IEEE/CVF International Conference on Computer Vision , pages 3520-3529, 2021.

Jiaming Han, Jian Ding, Nan Xue, and Gui-Song Xia. Redet: A rotation-equivariant detector for aerial object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2785-2794, 2021.

Xue Yang, Junchi Yan, Ziming Feng, and Tao He. R3det: Refined single-stage detector with feature refinement for rotating object. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 3163-3171, 2021.

Jiaming Han, Jian Ding, Jie Li, and Gui-Song Xia. Align deep features for oriented object detection. IEEE Transactions on Geoscience and Remote Sensing , 60:1-11, 2022.

Xue Yang, Jirui Yang, Junchi Yan, Yue Zhang, Tengfei Zhang, Zhi Guo, Xian Sun, and Kun Fu. Scrdet: Towards more robust detection for small, cluttered and rotated objects. In IEEE/CVF International Conference on Computer Vision , pages 8231-8240, 2019.

Wen Qian, Xue Yang, Silong Peng, Junchi Yan, and Yue Guo. Learning modulated loss for rotated object detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 2458-2466, 2021.

Xue Yang and Junchi Yan. Arbitrary-oriented object detection with circular smooth label. In European Conference on Computer Vision , pages 677-694, 2020.

Xue Yang, Liping Hou, Yue Zhou, Wentao Wang, and Junchi Yan. Dense label encoding for boundary discontinuity free rotation detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15814-15824, 2021.

Yi Yu and Feipeng Da. Phase-shifting coder: Predicting accurate orientation in oriented object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023.

Xue Yang, Junchi Yan, Ming Qi, Wentao Wang, Xiaopeng Zhang, and Tian Qi. Rethinking rotated object detection with gaussian wasserstein distance loss. In Proceedings of the 38th International Conference on Machine Learning , volume 139, pages 11830-11841, 2021.

Xue Yang, Xiaojiang Yang, Jirui Yang, Qi Ming, Wentao Wang, Qi Tian, and Junchi Yan. Learning high-precision bounding box for rotated object detection via kullback-leibler divergence. In Advances in Neural Information Processing Systems , volume 34, pages 18381-18394, 2021.

Xue Yang, Gefan Zhang, Xiaojiang Yang, Yue Zhou, Wentao Wang, Jin Tang, Tao He, and Junchi Yan. Detecting rotated objects as gaussian distributions and its 3-d generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4335-4354, 2023.

Xue Yang, Yue Zhou, Gefan Zhang, Jirui Yang, Wentao Wang, Junchi Yan, Xiaopeng Zhang, and Qi Tian. The kfiou loss for rotated object detection. In International Conference on Learning Representations , 2023.

Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. Reppoints: Point set representation for object detection. In IEEE/CVF International Conference on Computer Vision , pages 9656-9665, 2019.

Liping Hou, Ke Lu, Xue Yang, Yuqiu Li, and Jian Xue. G-rep: Gaussian representation for arbitraryoriented object detection. arXiv preprint arXiv:2205.11796 , 2022.

Wentong Li, Yijie Chen, Kaixuan Hu, and Jianke Zhu. Oriented reppoints for aerial object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1829-1838, 2022.

Anna Khoreva, Rodrigo Benenson, Jan Hosang, Matthias Hein, and Bernt Schiele. Simple does it: Weakly supervised instance and semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition , pages 876-885, 2017.

Cheng-Chun Hsu, Kuang-Jui Hsu, Chung-Chi Tsai, Yen-Yu Lin, and Yung-Yu Chuang. Weakly supervised instance segmentation using the bounding box tightness prior. Advances in Neural Information Processing Systems , 32, 2019.

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In IEEE International Conference on Computer Vision , pages 2961-2969, 2017.

Zhi Tian, Chunhua Shen, and Hao Chen. Conditional convolutions for instance segmentation. In European Conference on Computer Vision , pages 282-298, 2020.

Javed Iqbal, Muhammad Akhtar Munir, Arif Mahmood, Afsheen Rafaqat Ali, and Mohsen Ali. Leveraging orientation for weakly supervised object detection with application to firearm localization. Neurocomputing , 440:310-320, 2021.

Yongqing Sun, Jie Ran, Feng Yang, Chenqiang Gao, Takayuki Kurozumi, Hideaki Kimata, and Ziqi Ye. Oriented object detection for remote sensing images based on weakly supervised learning. In 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) , pages 1-6. IEEE, 2021.

Tianyu Zhu, Bryce Ferenczi, Pulak Purkait, Tom Drummond, Hamid Rezatofighi, and Anton van den Hengel. Knowledge combination to learn rotated detection without rotated annotation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023.

Seungkyu Lee and Yanxi Liu. Curved glide-reflection symmetry detection. In IEEE Conference on Computer Vision and Pattern Recognition , pages 1046-1053, 2009.

Tianshu YU, Junchi Yan, Jieyi Zhao, and Baochun Li. Joint cuts and matching of partitions in one graph. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2018.

Rajendra Nagar and Shanmuganathan Raman. Detecting approximate reflection symmetry in a point set using optimization on manifold. IEEE Transactions on Signal Processing , 67(6):1582-1595, 2019.

Rajendra Nagar and Shanmuganathan Raman. 3dsymm: Robust and accurate 3d reflection symmetry detection. Pattern Recognition , 107:107483, 2020.

Ahyun Seo, Byungjin Kim, Suha Kwak, and Minsu Cho. Reflection and rotation symmetry detection via equivariant learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9529-9538, 2022.

Yifei Shi, Xin Xu, Junhua Xi, Xiaochang Hu, Dewen Hu, and Kai Xu. Learning to detect 3d symmetry from single-view rgb-d images with weak supervision. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4882-4896, 2023.

Yanxi Liu. Computational symmetry. In Computer Vision: A Reference Guide . Springer International Publishing, 2020.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition , pages 770-778, 2016.

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In IEEE Conference on Computer Vision and Pattern Recognition , pages 936-944, 2017.

Jiahui Yu, Yuning Jiang, Zhangyang Wang, Zhimin Cao, and Thomas Huang. Unitbox: An advanced object detection network. In Proceedings of the 24th ACM International Conference on Multimedia , pages 516-520, 2016.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems , volume 32, pages 8024-8035, 2019.

Yue Zhou, Xue Yang, Gefan Zhang, Jiabao Wang, Yanyi Liu, Liping Hou, Xue Jiang, Xingzhao Liu, Junchi Yan, Chengqi Lyu, Wenwei Zhang, and Kai Chen. Mmrotate: A rotated object detection benchmark using pytorch. In Proceedings of the 30th ACM International Conference on Multimedia , 2022.

Liping Hou, Ke Lu, Jian Xue, and Yuqiu Li. Shape-adaptive selection and measurement for oriented object detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 923-932, 2022.

Zonghao Guo, Chang Liu, Xiaosong Zhang, Jianbin Jiao, Xiangyang Ji, and Qixiang Ye. Beyond boundingbox: Convex-hull feature adaptation for oriented and densely packed object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8788-8797, 2021.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In IEEE/CVF International Conference on Computer Vision , pages 9992-10002, 2021.

Xian Sun, Peijin Wang, Zhiyuan Yan, Feng Xu, Ruiping Wang, Wenhui Diao, Jin Chen, Jihao Li, Yingchao Feng, Tao Xu, Martin Weinmann, Stefan Hinz, Cheng Wang, and Kun Fu. Fair1m: A benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing , 184:116-130, 2022.

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations , 2018.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations , 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to exploit the reflection symmetry as a new supervision to HBox-supervised oriented object detection. Several modifications are made to adapt the new self-supervised (SS) branch, including removing the angle subnet in the weakly-supervised (WS) branch and a CircumIoU loss for box regression. Experiments show a clear state-of-the-art result to previous Hbox-supervised oriented object detectors.\n\nSTRENGTHS:\n1. The paper tries to exploit the reflection symmetry to improve the Hbox-supervised object detector, which is interesting.\n\n2. The paper is well-motivated and easy to understand.\n\n3. The proposed H2Rbox-v2 achieves state-of-the-art performance in the Hbox-supervised oriented object detectors.\n\nWEAKNESSES:\nMajor concern:\n\n1. The soundness of using reflection symmetry is somewhat low. According to page 5 line 130, if the network $f_{nn}(\\cdot)$ subjects to both flip and rotate consistencies, then $f_{nn}(I_0)=\\theta_{sym}$. I don't know why this equation holds. Clearly, we don't know the symmetrical axis of the image $I_0$. And if we rotate the image by a random angle $\\theta$ in the Way2 of Fig.3, then $f_{nn}(I_2)$ would never be equal to $\\theta_{sym}$. Even if the symmetrical axis is known, this reflection symmetry learning can only theoretically and ideally work under the setting of a single symmetric object. As a matter of fact, it would be more rational that the converse proposition holds - \"If the network $f_{nn}(\\cdot)$ can always predict the symmetrical axis $\\theta_{sym}$ of the image, then, it must subject to both flip and rotate consistencies.\" I suggest the authors reconsider the logic here.\n\n2. Due to weakness 1, the novelty is also limited since one can easily conclude that H2Rbox-v2 is to add one more additional branch with a new view generation upon H2Rbox, i.e., the left column of Fig. 2(b) (\"vertical flipping\"). The right column is random rotation which is similar to the SS branch of H2Rbox. This is what I believe the model brings us performance gains, but not the so-called reflection symmetry learning.  The experiments cannot support that the reflection symmetry is important, but can only verify the effectiveness of adding the vertical flipping branch, which is supported by Tab. 6. It is still unclear for the relationship between reflection symmetry learning and multi-branch self-supervised learning.\n\n3. The ablation studies provide a weak explanation for the effectiveness of the proposed components. In Tab. 4, adding the $l_s$ loss only produces near zero mAP on DOTA, while it is much higher on HRSC. Why did such drastic fluctuations occur? The PSC coder is just an angle encoder-decoder module, which is originally proposed to improve the performance of oriented object detectors. It would not be a critical factor affecting the performance, as demonstrated by the PSC paper, ~2 mAP at most. This paper shows PSC is crucial in the proposed method. That is to say, without PSC, the proposed method produces much poor performance. This largely weakens the soundness of the designs in the SS branch. Additionally, PSC imposes the loss on two encoded phase-shifting patterns of angles, which inherently solves the boundary discontinuity, while the proposed SS branch decodes the phase-shifting patterns to angle and then calculates the loss on two angles, which introduces the boundary discontinuity again, thus undermining the contribution of the snap loss.\n\n4. It is confusing to me that Fig.2 shows the angle is predicted by the SS branch, while all the other properties are predicted by the WS branch. That is to say, the inference time of H2Rbox-v2 (WS+SS) should be double of the WS branch, while H2Rbox only needs the WS branch for inference. I'm concerned about why the FPS of H2Rbox-v2 is equal to H2Rbox as shown in Tab. 2. The authors need to describe the inference process clearly.\n\nMinor concern:\n\nThe baseline model is unclear in the ablation studies. In Tab. 4, if the baseline model is \"w/o PSC and $l_s$\", then what is the angle coder used in the baseline? And what are the angles in calculating Smooth-L1 loss? The same question emerges in Tab. 5 and Tab. 6. Is that mean all the baseline models of the tables adopt the optimal strategies of the other tables? If so, then the best results of Tab.6 seem to be not in line with the others.\n\nOverall:\n\nWhile the proposed method achieves state-of-the-art performance, my initial rating score is 4. I have some concerns about the soundness of the reflection symmetry learning and the ablation studies. It is of vital importance to give an explanation and analysis of the rationality of the proposed method.\n\nQUESTIONS:\nSuggestions:\n\n1. For the weaknesses 1 and 2, I suggest the author to rethink the logic of the analysis of Sec. 3.1. It would be better if we could restate the analysis in page 5 line 130 to the converse proposition - \"If the network $f_{nn}(\\cdot)$ can always predict the symmetrical axis $\\theta_{sym}$ of the image, then, it must subject to both flip and rotate consistencies.\" And then the contrapositive - \"if the network subjects to neither flip consistency nor rotate consistency, it cannot predict the symmetrical axis of the image.\" Therefore, we need to enhance both flip consistency and rotate consistency. H2Rbox considers the rotate consistency only, which leads to sub-optimal results.\n\n2. There is a missing experiment, i.e., $\\lambda=0$ in Tab. 6.\n\nLIMITATIONS:\nsee Weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an advance solution for using horizontal bounding boxes as supervision to learn oriented object detectors. \nThe proposed method H2RBox-V2, which is a modification of the recent work H2RBox, has some technique novelty and contribution as it jointly uses the weakly- and self-supervised branches to learn the rotation angle. \nExperiments are validated on DOTA, HRSC and FAIR1M, and show state-of-the-art performance.\n\nSTRENGTHS:\n+ The overall idea to learn the rotated angles from both weakly- and self- supervised branch is very interesting.\n\n+ The task itself, to learn rotated bounding boxes from horizontal bounding boxes, is a new setting for the rotated object detection community. \n\n+ The performance improvement against prior arts is significant.\n\nWEAKNESSES:\n- Unclear methodology design and presentation.\n\n(1) The authors claim to propose a new CircumIoU loss for this framework. Unfortunately, in the methodology section, especially Sec3.4, there is no term ‘CircumIoU loss’, and it is impossible to know which loss is the claimed novelty. \n\n(2) Besides, given this unclear description, it is very difficult to know its difference or novelty against some prior works such as [a], which also optimizes the detector from rotation angles.\n\n[a] Arbitrary-Oriented Object Detection with Circular Smooth Label. ECCV 2020.\n\nOther minor issues and comments for improvement:\n\n- Please provide some rotation angle distribution visualizations from the proposed framework.\n\n- Fig.1, column2 and column4. The predictions from the proposed method and SAM-RBox seem not to have much difference. Please consider use more representative figures.\n\n- Table 1, the performance gap. Please explictly mention which dataset it is that leads to 3.41% drop and 0.07% improvement.\n\n- Fig.2 looks very crowded. It can be better polished, and the size of each content can be made more fit.\n\n- For loss function, please use \\mathcal{} to distinguish it from scalars.\n\nQUESTIONS:\nThe authors have addressed my questions in the rebuttal well.\n\n\n______ before rebuttal _____________\n\n\nQ1: Issues on the flaw of the technique framework. Both experimentally and theoretically.\n\nQ2: The details, design and clear presentation of the CircumIoU loss.\n\nQ3: systematically address the unfair experimental comparison issue.\n\nLIMITATIONS:\nThe authors have addressed my concerns well.\n\n______________________before rebuttal ___________\nThe limitations are not properly discussed. The reviewer believes the technique flaws as mentioned in weakness part is more critical to the proposed framework.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces H2RBox-v2, an innovative approach to further bridge the gap between HBox-supervised and RBox-supervised oriented object detection. It seeks to address the limitations of the original H2RBox model, which required high-quality annotations and large training datasets, and was incompatible with rotation augmentation. H2RBox-v2 augments the original model by adding a self-supervised branch that learns object orientations from inherent visual symmetry, and a weakly-supervised branch that incorporates a new CircumIoU loss to allow for random rotation augmentation.\n\nSTRENGTHS:\n- **Originality:** While the paper builds upon existing work (specifically H2RBox), it infuses new concepts, the most significant being the utilization of symmetry for angle regression. This concept is highly innovative and introduces a novel angle of approach for oriented object detection. This creativity in applying the natural property of symmetry to enhance detection accuracy sets this work apart from previous methods and broadens the boundary of the field.\n\n- **Quality:** The proposed H2RBox-v2 exhibits enhanced performance on various datasets compared to its predecessor, extensive ablation experiments have demonstrated the importance of each module.\n\n- **Clarity:** The paper is well-structured and clearly presents the methodology and results, making the contributions of this research easily understandable.\n\n- **Significance:** Bridging the gap between HBox-supervised and RBox-supervised oriented object detection significantly reduce the annotation costs.\n\n- H2RBox-v2 has shown improved performance on various datasets and is specifically designed to cope with situations where H2RBox-v1 may underperform. This makes it a valuable contribution to real-world applications of oriented object detection.\n\nWEAKNESSES:\n- **Theory is not fully comprehensive:** *L138-142* only discussed the case where a single instance is contained in an image. However, in reality, remote sensing images can contain many objects, especially in dense scenarios. Despite the theoretical proof being not fully comprehensive, their simplicity of idea and empirical effectiveness seem to be sufficient according to me.\n\n- **Training overhead not detailed:** It is crucial that the model does not introduce additional overhead during the testing phase, but the paper does not discuss the computational overhead and the time taken for the model training process in detail\n\nQUESTIONS:\n- Would it be more effective to conduct the comparative experiments on the DOTA dataset, regarding the assertion that H2Rbox-v1 requires more training data than v2? Specifically, by training both v1 and v2 with varying percentages of DOTA data (namely 10%, 20%, and 50%), we could gain a more robust comparison of their respective performances.\n\n- When the object is not in the center of the image, are Way1 and Way2 described in L124-132 still equivalent?\n\n- In remote sensing scenarios, some objects may not possess symmetry, such as swimming pools and harbors. What impact does this have on the theory?\n\nLIMITATIONS:\nNothing to report.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new horizontal box-supervised rotation object detection detector. The proposed detector consists of two modules: a self-supervised regression branch for angle regression and a weakly supervised branch for horizontal box regression. This method is more simpler and shows clear improvement over the previous one.\n\nSTRENGTHS:\n- The paper fully considers the independence and correlation between angle and horizontal box in rotation object detection. It only uses angle regression in self-supervised regression while using Circumscribed RBox IoU to associate angle regression and horizontal box regression in weakly supervised regression to obtain the rotated box. \n-  The paper greatly improves the performance of the hbox-supervised detector, approaching or even reaching the performance of some rbox-supervised detectors.\n- The paper is well written, and the method is clear and well described.\n\nWEAKNESSES:\n- Figure 2 seems a bit complicated, it would be better to highlight the main points to make it clearer.\n- Although this article has achieved impressive results, it is important to consider the robustness of the model when horizontal bounding box annotations are not accurate enough, the article's experiments do not explicitly demonstrate the robustness towards inaccuracies in horizontal bounding box annotations.\n\nQUESTIONS:\n- I have a question about the self-supervised branch in the proposed method. Why was it designed with two perspectives instead of using one perspective and randomly selecting multiple variations under that perspective? Additionally, would using a more perspective, such as different rotation angles, combinations of rotation and symmetries, or scaling variations, lead to performance improvements?\n\n- Regarding Table 4, I noticed that the experiments on DOTA and HRSC datasets show inconsistent results. Specifically, in the Dota dataset, not using PSC will result in an incorrect result, while in the HRSC dataset, not using PSC only results in a performance loss. This raises the question of whether PSC only improves performance or if it has an indispensable impact on the model.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}