H2RBox: Horizontal Box Annotation is All You Need for Oriented Object Detection
Authors: Xue Yang
Gefan Zhang
Wentong Li
Yue Zhou
Xuehui Wang
Junchi Yan
Venue: ICLR 2023 poster (2023)
Keywords: Oriented Object Detection
Rotated Object Detection

Abstract: Oriented object detection emerges in many applications from aerial images to autonomous driving, while many existing detection benchmarks are annotated with horizontal bounding box only which is also less costive than fine-grained rotated box, leading to a gap between the readily available training corpus and the rising demand for oriented object detection.  This paper proposes a simple yet effective oriented object detection approach called H2RBox merely using horizontal box annotation for weakly-supervised training, which closes the above gap and shows competitive performance even against those trained with rotated boxes.  The cores of our method are weakly- and self-supervised learning, which predicts the angle of the object by learning the consistency of two different views. To our best knowledge, H2RBox is the first horizontal box annotation-based oriented object detector. Compared to an alternative i.e. horizontal box-supervised instance segmentation with our post adaption to oriented object detection, our approach is not susceptible to the prediction quality of mask and can perform more robustly in complex scenes containing a large number of dense objects and outliers. Experimental results show that H2RBox has significant performance and speed advantages over horizontal box-supervised instance segmentation methods, as well as lower memory requirements. While compared to rotated box-supervised oriented object detectors, our method shows very close performance and speed. The source code is available at PyTorch-based \href{https://github.com/yangxue0827/h2rbox-mmrotate}{MMRotate} and Jittor-based \href{https://github.com/yangxue0827/h2rbox-jittor}{JDet}.

Published as a conference paper at ICLR 2023
H2RBOX: HORIZONTAL Box ANNOTATION IS ALL YOU NEED FOR ORIENTED OBJECT DETECTION
Xue Yang', Gefan Zhang' Wentong Li", Xuehui Wang', Yue Zhou', Junchi Yan
'MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University
"COWAROBOT Co. Ltd. *Zhejiang University "Shanghai Al Laboratory
(rangxuc-2019-250 liciu,iizonn www.:iizzzzzeeccee ftt.eessttuuuuuu...mmmmmm wwwcntongezju.coo edu.
(wangxuchui, sjtu.zy, yan. jan
edu.
Jittor Code: https:/ betps://githube com/ wowp:/emmxxeee2/////bbboooooaaaa
PyTorch Code: https: /ttps://gite /ttp//ggtttt.tttt/eeee con www/yaagggee0827/ http://///////bbb///// amrootte
ABSTRACT
Oriented object detection emerges in many applications from aerial images to au- tonomous driving, while many existing detection benchmarks are annotated with horizontal bounding box only which is also less costive than fine-grained rotated box, leading to gap between the readily available training corpus and the rising demand for oriented object detection. This paper proposes a simple yet effective oriented object detection approach called H2RBox merely using horizontal box annotation for -sparly-uperrvieed training, which closes the above gap and shows competitive performance even against those trained with rotated boxes. The cores of our method are weakly- and self-supervised learning, which predicts the an- gle of the object by learning the consistency of two different views. To our best knowledge, H2RBox is the first horizontal box annotation- based oriented object detector. Compared to an alternative i.e. horizontal bopeerped instance seg- mentation with our post adaption to oriented object detection, our approach is not susceptible to the prediction quality of mask and can perform more robustly complex scenes containing large number of dense objects and outliers. Exper- imental results show that H2RBox has significant performance and speed advan- tages over horizontal bux-super instance segmentation methods, as well lower memory requirements. While compared to rotated box-supervised oriented object detectors, our method shows very close performance and speed. The source code is available at Py HyTchh---bae MMRotate and
JDet.
INTRODUCTION
In addition to the relatively matured area of horizontal object detection (Liu et al., 2020), oriented ob- ject detection has received extensive attention, especially for complex scenes, whereby fine-grained bounding box (e.g. rotated/quadrilateral bounding box) is needed, e.g. aerial images (Ding et al., 2019: Yang et al., 2019a), scene text (Zhou et al., 2017), retail scenes (Pan et al., 2020) etc.
Despite the increasing popularity of oriented object detection, many existing datasets are annotated with horizontal boxes (HBox) which may not be compatible (at least on the surface) for training an oriented detector. Hence labor-intensive re-annotation' have been performed on existing horizontal- annotated datasets. For example, DIOR-R (Cheng al., 2022) and SKU1 10K-R (Pan et al., 2020) rotated box (RBox) annotations of the acrial image dataset DIOR (192K instances) (Li et al., 2020) and the retail scene SKUI 1OK (1,733K instances) (Goldman et al., 2019), respectively.
One attractive question arises that if one can achieve weakly supervised learning for oriented object detection by only using (the more readily available) HBox annotations than RBox ones. One poten-
"Conresoondence author is Junchi Yan. The work in part supported by National Key Research and De- velopment Program of China A01076000, National Natural Science Foundation of China (62222607), Shanghai Municipal Science and Technology Major Project (2021SHZIZX0102),
'The annotation cost (in price) of the RBox is about 36.5% (S86 vs. $63) higher than that of the HBox according https::t////...... /tttps::/coou/ googl com/ -plpppl-- p1 form dat Tabel ing/pricing,

---

Published as a conference paper at ICLR 2023
(a) BoxInst-RBox
(b) BoxLevelSet-RBox
the & ERINT
MEL
(c) H2RBox
Figure 1: Visual comparison of three HBox-superviee rotated detectors on aircraft detection (Wei et al., 2020), ship detection (Yang et al., 2018), vehicle detection (Azimi et al., 2021), etc. The Sroxx-Mask-aee
style methods, i.e. -Box (Tian et al., 2021) and BoxLevelSet-RBox (Li et al., 2022b), perform not well in complex and object-cluttered scenes.
tial and verified technique in experiments is supervid instance segmentation, concern- ing with BoxInst (Tian et al., 2021). BoxLevelSee (Li et al., 2022b), etc. Based on the segmentation mask by these methods, one can readily obtain the final RBox by finding its minimum cireumscribed rectangle. and we term the above procedure as HBox-Mask- -Box style methods i.e. BoxInst-RBBl -Box and BoxLevelSet-RBox in this paper. Yet it in fact involves a potentially more challenging task i.c. instance segmentation whose quality can be sensitive to the background noise, and it can influence heavily on the subsequent RBox detection step, especially given complex scenes (in Fig. 1()) and the objects are crowded (in Fig. 1(b)). Also, involving segmentation is often more computational costive and the whole procedure can be time consuming (see Tab. 1-2).
In this paper, we propose simple yet effective approach, dubbed as HBox-to-RBox (H2RBox), which achieves close performance to those RBox annotation supervised methods e.g. (Han et al., 2021b; Yang et al., 2023a) by only using HBox annotations, and even outperforms in considerable amount of cases as shown in our experiments. The cores of our method are weakly- and self- supervised learning, which predicts the angle of the object by learning the enforced consistency be- tween two different views. Specifically, we predict five offsets in the regression sub-network based on FCOS (Tian et al., 2019) in the WS branch (see Fig. 2 left) that the final decoded outputs are RBoxes. Since we only have horizontal box annotations, we use the horizontal circumscribed rectangle of the predicted RBox when computing the regression loss. Ideally, predicted RBoxes and corresponding ground truth (GT) RBoxes (unlabeled) have highly overlapping horizontal circum- scribed rectangles. In the SS branch (see Fig. 2 right), we rotate the input image by a randomly angle and predict the corresponding RBox through regression sub-network. Then, the consistency of RBoxess between the two branches, including scale consistency and spatial location consistency, are learned to eliminate the undesired cases to ensure the reliability of the WS branch. Our main contributions are as follows:
1) To our best knowledge, we propose the first HBox annotation- oriented object detector. Specifically, a weakly- and self-supervised angle learning paradigm is devised which closes the gap between HBox training and RBox testing, and it serve plugin for existing detectors.
2) We prove through geometric equations that the predicted RBox is the correct GT RBox under our designed pipeline and consistency loss, and does not rely not-fully- hallyyverified/ad-hoo assumptions, e.g. color-pairwiss affinity in BoxInst or additional intermediate results whose quality cannot be ensured, e.g. feature map used by many weakly supervised methods (Wang et al., 2022).
3) Compared with the potential alternatives e.g. 1Box-Mask-RRe whose instance segmenta- tion part is fulfilled by the state-of-the-art BoxInst, our H2RBox outperforms by about 14% mAP
2

---

Published as a conference paper at ICLR 2023
0.707s
RandomHotation CemerCrapl Reladion Padding
CemerCrop
FiterObjest
View
View 2
Rotation Matrix:
Imagel
Original Ground GI
Image2
GT
Backbone & Neck
Backbone
Shered
Nock
Weakly-auperrised (WS) Branch
Sele-supervised (SS) Branch
Regression
Regreasion
HieWell
Dacodar
Angla HaWWt
RBox
Angle HoWWc
Feg. Subnet
Reg.
Shared Head (reg.)
Classi ficaliiaion HtWhCC
Label Assigner
Label
Re Re-Assigner
Center ness
HeWet
Lon
GT
Shared Head (cls. reg.)
Figure 2: Our H2RBox consists of two branches respectively fed with two augmented views (View and View 2) of the input image. The left Weakly-supervised Branch in general can be any rotated object detector (FCOS here) for RBox prediction, whose cireumscribed HBox is used for supervised learning given the GT HBox label in the sense
learning. This branch is also used for test-stage inference. The right Self-supervised Branch tires achieve RBox prediction consistency of the two views with self-supervised learning. Image is from the DIOR-R dataset.
(67.90% vs. 53.59% on DOTA-v1.0 dataset, requiring only third of its computational resources (6.25 GB vs. 19.93 GB), and being around 12x faster in inference (31.6 fps vs. 2.7 fps).
Compared with the fully RBox annotation-supervised rotation detector FCOS, H2RBox only 0.91% (74.40% vs. 75.31%) %) and 1.01 % (33. 15% vs. 14.16% behind on DOTA-v1.0 and DIOR-R, respectively. Furthermore, we do not add extra computation in the inference stage, thus maintaining a comparable detection speed, about 29.1 FPS vs. 29.5 FPS on DOTA-v1.0.
2 RELATED WORK
RBox-supervised Oriented Object Detection. Oriented object detection in visual images has
ceived increasing attention across different areas e.g. aerial image (Xu et al., 2020; Yang et al., 2022; 2023a; Hou al., 2023), scene text (Zhou al., 2017: Liao al., 2018), retail (Pan al., 2020; Chen et al., 2020), etc. Earlier methods including RRPN (Ma et al., 2018), ROI-Transformer (Ding al., 2019) and ReDet (Han et al., 2021b) directly perform angle regression. To address the loss discontinuity and regression inconsistency due to periodicity of angle, subsequent works ei- ther convert the parameterization of the rotated bounding box into 2-D Gaussian distributions (Yang et al., 202 lc;d) or transform the angle regression to classification (Yang et al., 2021a; Yang & Yan, 2022). (Hou al., 2022; Li al., 2022a) introduce the adaptive point set for object representation to mitigate the angle regression sensitivity and meanwhile captures instances' semantic information.
HBox-supervised Instance Segmentation and Its Potential for Oriented Object Detection. The bold idea of purely using HBox-annotation
train rotated object detector is attractive yet still rarely studied literature, which can be seen as a weakly supervise (WS) learning paradigm for oriented object detection. A related and better-studied technique is HBox-supervised instance seg- mentation, which tries to segment instance based on the HBox annotations for WS training. For instance, SDI (Khoreva et al., 2017) relies the region proposals generated MCG (Pont-Tuset
3

---

Published as a conference paper at ICLR 2023
One
one-to-many
CTyy View 1
rbox -prior View
label re-assignment across views
(a) 020 by zeros padding. (b) O2M by reflect padding.
(c) The two re-assignment strategies.
Figure 3: Comparison of different padding methods (Sec. 3.1) and re-assignment strategies
(Sec. 3.4). Green and red RBox represent the target rsox and rbox* respectively.
al., 2016) and uses iterative training process to refine the segmentation. BBTP (Hsu al., 2019) formulates the Huox-supervised instance segmentation into a multiple instance learning prob- lem based on Mask R-CNN (He al., 2017). BoxInst (Tian al., 2021) uses the color-pairwise affinity with box constraint under an efficient Rol-free CondInst (Tian et al., 2020). BoxLevelSet (Li al., 2022b) introduces an energy function to predict the instance-awan aware mask as the level set.
Though one can obtain the final object orientation by certain means based on the segmentation mask from the above instance segmentation methods, c.g. by finding the minimum cireumscribed rectangle, we argue and show in our experiments that such an HBox-Mask-RRas complex (segmentation can be even more difficult than rotation detection see Fig. 1) and expensive in the presence of dense objects and background noises. Hence we aim to skip the segmentation step and build an HBox-to-RBae paradigm which has not been studied before to our best knowledge.
pipeline can be
3 PROPOSED METHOD
The overview of the H2RBox is shown in Fig. 2. Two augmented views are generated and infor- mation leakage is avoided for training overfitting. There are two branches. One branch is used for weakly-supervised (WS) learning where the supervision is the GT HBox from the training data, and the regression loss is calculated between the circumscribed HBox derived from the predicted RBox by this branch and GT HBox. The other branch trained by self self-supervised (SS) learning that involves two augmented views of the raw input image, which encourages to obtain the consistent RBox prediction between the two views. The final loss is the weighted sum of the WS loss and SS loss. Note that the test-stage prediction is concerned only with the WS branch.
3.1 AUGMENTED VIEW GENERATION
In line with the general idea of self-supervised learning by data augmentation, given the input image, we perform random rotation to generate View while keeping View I consistent with the input image, as shown in Fig. However, rotation transformation will geometrically and inevitably introduce an artificial black border area and leads to the risk of GT angle information leakage. We provide two available techniques to resolve this issue:
v2, area² in the center of the image. 1) Center Region Cropping: Crop Ls x 7
2) Reflection Padding: Fill the black border area by reflection padding.
If the Center Region Cropping is used in View 2, View also needs to perform the same operation and filter the corresponding ground truth. In contrast, Reflection Padding works better than Center Region Cropping because it preserves as much of the area as possible while maintaining a higher image resolution. Fig. 3(a) and Fig. 3(b) compare zeros padding and reflection padding, Note that the black border area does not participate in the regression loss calculation in the SS branch, so it does not matter that this region is filled with unlabeled foreground objects by reflection padding.
2 When the rotation angle a multiple of 45", the black border area reaches its peak, so the side length of the largest crop area x/2 of the side length of the original image (s), refer the View 2 in Fig. 2.
4

---

Published as a conference paper at ICLR 2023
(a) WS loss only
(b) WS + SS loss
Figure 4: Visual comparison of our methods with and without the SS loss used in the SS branch. can help learn the scale and spatial location consistency between the two branches.
3.2 THE WEAKLY-SUPERVIE (WS) BRANCH
The two generated views (View 1 and View 2) are respectively fed into
the two branches with the parameter- shared backbone and neck, specified as ResNet (He et al., 2016) and FPN (Lin et al., 2017a) as shown in Fig. 2. The WS branch here is specified by a FCOS- -aosed rotated object detector, as involved for both training and infer- ence. This branch contains regression and classification sub-networks to pre- dict RBox, category, and center-ness. Recall that we can not use the pre- dicted RBox to calculate the final re- gression directly as there is no RBox annotation but HBox only. There- fore, we first convert the predicted RBox into the corresponding mini- mum horizontal circumscribed rectan- gle, for calculating the regression loss between the derived HBox and the GT Hbox annotation (we defer the details of the loss formula- tion to Sec. 3.5). As the network is better trained, an indirect connection (horizontal cireumscribed rectangle constraint) occurs between predicted RBox and GT RBox (unlabeled): No matter how an object is rotated, their corresponding horizontal circumscribed rectangles are always highly over- lapping. However, as shown in Fig, 4(a), only using WS loss can only localize the objects, while still not effective enough for accurate rotation estimation.
h
cos 0
sin Whne
w
cos yp sin up
Wax
sim 0 h
cos
sin ypl + h
cos 4p
h.s
Figure 5: Proof of the relationship between predicted RBox and GT RBox under horizontal cireumscribed rectangle constraint and scale constraint. Green and orange RBoxes represent correct coincident prediction B and undesired symmetric prediction B*
3.3 THE SELF-SUPERVISED (SS) BRANCI
As complementary to the WS loss, we further introduce the SS loss. The SS branch only contains one regression sub-network for predicting RBox in the rotated View 2. Given a (random) rotation transformation R (with degree A!) as adopted View 2, the relationship between location (æ, y) of View I in the WS branch and location (x" yy") )') of View 2 with rotation R in the SS branch is:
cos A0 sin A0
)
(α',")) ==))=(0 (x (-- xe,
Be R + (Ie, Ve), R
(1)
sin A0
cas AO
where (wes yc) the rotation center (i.e. image center). Recall the label of the black border area (in Fig. 3) in the SS branch is set as invalid and negative samples, which will not participate in the subsequent losses designed below.
Specifically, a scale loss Lugh accounts for the scale consistency to enhance the indirect connection described above: For augmented objects obtained from the same object through different rotations, set of RBoxes of the same scale predicted by the detector, and these predicted RBoxes and corresponding GT RBoxes (unlabeled) shall have highly overlapping horizontal circumscribed rect- angles. With such enhanced indirect connection, including horizontal circumscribed rectangle
5

---

Published as a conference paper at ICLR 2023
constraint and scale constraint, we can limit the prediction results to a limited number of feasible cases, explained as follows:
Fig. 5 shows two cases based on the above enhanced indirect connection, and lists four different expressions for the four variables (w, h, 0, :p). Due to the periodicity of the angles, there are only two feasible solutions the four equations within the angle definition, i.e. the green GT RBox and the orange symmetric RBox. In other words, with such a strengthened indirect connection, the relationship between predicted RBox and GT RBox is coincident (sw,, h, 0) or symmetrical about the center of the object B (w, , - 0). It can be seen from Fig. 4(a) that there are still many bad cases with extremely inaccurate angles after using Lwk- Interestingly, if we make a symmetry transformation of these bad cases with their center point, the result becomes much better. When generating views, a geometric prior can be obtained, that is, the spatial transformation relationship between the two views, denoted as R. in Eq. 1. Thus, we can get the following four transformation relationships, marked as T(Bwe: Bee), between the two branches:
TB B = (R), T(B B²x) = (R, = (S.
(2)
T( B = (R J, T(B B²,) = (R = RS
where B and B represent the coincident bounding box predicted in WS branch and the symmet- ric bounding box predicted in SS branch, respectively. Here S denotes symmetric transformation.
Take T(B ) {R. S} as an example, it means Bº S(R B
Therefore, an effective way to eliminate the symmetric case is to let the model know that the relation- ship between the RBoxes predicted by the two branches can only be R. Inspired by above analysis, spatial location loss is used to construct the spatial transformation relationship R of RBoxes pre- dicted by two branches. Specifically, the RBox predicted by WS branch is first transformed by R, and then several losses (e.g. center point loss Lzy and angle loss Lg) are used to measure its lo- cation consistency with the RBox predicted by SS branch. In fact, the spatial location consistency, especially the angle loss, provides fifth angle constraint equation (y A Ae + 0) so that the system of equations in Fig. 5 have a unique solution (i.e. the predicted RBox is the GT RBox) with non-strict proof, because system of equations are nonlinear. The final SS learning consists of coale-consistent and peatial-location consistent learning:
Sim(R- Bux, B..) =1
(3)
Fig. 4(b) shows the visualization by using the SS loss, with accurate predictions. The appendix shows visualizations of feasible solutions for different combinations of constraints.
3.4 LABEL RE-ASSIGNER
Since the consistency of the prediction results of the two branches needs to be calculated, the la- bels need to be re-assigned in the SS branch. Specifically, the labels at the location (a", . y²) of the SS branch, including center-ness (en*), target category (c*) and target GT HBox (gthoz"*), are the same as in the location (a, y) of the WS branch. Besides, we also need to assign the rbox"" (Iwee Bwas Wws, hws, Ows) predicted by the WS branch as the target RBox of the SS branch to calculate the SS loss. We propose two reassignment strategies:
1) One-to-one (020) assignment: With en, and gtboz", the rboz predicted at location (x, y) in the WS branch is used as the target RBox at (x", .y") ) of the SS branch (see Fig. 3(a)).
2) One-to-many (O2M) assignment: Use the rboz. closest to the center point of the gtbor (x.)) as the target RBox at location (x*,y") of SS branch, as shown in Fig. 3(b).
Fig. 3(c) visualizes the difference between the two re-assignment strategies. After re-assigning, we need to perform an rotation transformation on the rboa wi to get the rbou wa* (Niwer Vwss w Mi ('r)) for calculating the SS loss according to Eq. 3:
Vire) (Iam Ez, Vns- ) (Eer Be)s (wwshhiee (Wws, kww), 8 = P.. + A0 (4)
The the visualized label assignment in Fig. further shows that the SS loss effectively eliminates prediction of the undesired case. The label reassignment of different detectors may require different strategies. The key is to design a suitable matching strategy for the prediction results of the two views, which can allow the network to learn the consistency better.

---

Published as a conference paper at ICLR 2023
3.5 THE OVERALL LOSs BY COMBINING THE WS AND SS LOSSES
Since the WS branch is a rotated object detector based on FCOS, the losses in this part mainly include the regression Lreg. classification Letss and center-ness Len- We define the WS loss in the WS branch as follows:
Luoe
141
Σ
Lede ((aw)
122
Σ (#.1)
Lon Len(eni (T.w))
Nyoc (2.20)
Npos
(S)
M3
Σ 200 (r2h(rbox( (x.3))
un gtbouia..)
Σ CRpos
where Lots is the focal loss (Lin al., 2017b), Len cross-entropy loss, and Lreg is IoU loss (Yu et al., 2016). Noss denotes the number of positive samples. p and с denote the probability distribution of various classes calculated by Sigmoid function and target category. rboz and gtbor" represent the predicted RBox in the WS branch and horizontal GT box, respectively. en' and en indicate the predicted and target center-ness. 1 (e is the indicator function, being 1 if ((x.y) > 0 and 0 otherwise. The r2h(-) function converts the RBox to its corresponding horizontal circumscribed rectangle. We set the hyperparameters μu = 1, P2 = 1 and μ3 by default.
Then, the SS loss between rboz (xiss Viwss Wass huss m=)) and rbox (Z Year Weey hess 20.) predicted by the SS branch is:
Toss Σ erppux
Σ (x"1²*)
1 (of we y0) enis wn Tree (rbon w) rboz(z= wv1))
(6)
Leeg (rbox"* use ,rbox*") yi Vilyy + Yalwhe, Lay Σ
1 ( tas)
tE(r,))
Lune = m Liou (Bus, Ble) + sin - Oos)l, (Bwe, By,)) + cos(6 Oes)1}
where Bioe (-e(- W hi:: wws h(w)) B3_ (-wes- - Wsa: has) and B3, (-hes. W heer Wss). We set = 0.15 and 72 = by default. Luhe takes into account the loss discontinuity caused by the boundary issues (Yang et al., 2021c), such as periodicity of angle and exchangeability of edges.
The overall loss is weighted sum of the WS loss and the SS loss where we set À 0.4 by default.
Leoral
Lugs
AL,s
(8)
4 EXPERIMENTS
4.1 DATASETS AND IMPLEMENTATION DETAILS
DOTA-v1.0 (Xia et al., 2018) is one of the largest datasets for oriented object detection in aerial images, which contains challenging cases, e.g. large-scale dense scenes and complex background. contains 15 categories, 2,806 images and 188,282 instances with both RBox and HBox annotations, and the latter are directly derived from the former one. The proportion of the training set, validation set, and testing set is 1/2, 1/6, and 1/3, respectively. For training and testing, we follow standard protocol by cropping images into 1,024x ,0224 patches with a stride of 824. DIOR-R (Cheng et al., 2022) is an aerial image dataset annotated by RBoxes based on its horizontal annotation version DIOR (Li et al., 2020). There are 23,463 images and 190,288 instances with 20 classes.
Methods are implemented both by PyTorch (Paszke et al., 2019)-based framework MMRotate (Zhou et al., 2022) and Jittor (Hu et al., 2020)-based framework JDet. We adopt the FCOS (Tian et al., 2019) with ResNet50 (He al., 2016) backbone and FPN neck (Lin al., 2017a) the baseline method and building block based on which we develop our approach (see Fig. 1). To implement the weakly-supervised
-Mask-BBox alternatives for comparison, we use two strong HBox annotation-based instance segmentation methods: BoxInst and BoxLevelSet, followed by finding its minimum compact surrounding rectangle as the detected RBox and we dub them BoxInst-RBoo and LeLLeveSSet-Roo respectively. All models are trained with AdamW (Loshchilov & Hutter, 2018) on GeForce RTX 3090 GPU, except BoxLevelSet (Li et al., 2022b) which requires NVIDIA V100 with larger memory. The initial learning rate is 10 with 2 images per mini-batch The weight decay is 0.05. Besides, we adopt learning rate warm-up for 500 iterations, and the learning rate is divided by 10 at each decay step. Random flipping is adopted without any additional tricks.
7

---

Published as a conference paper at ICLR 2023
Table 1: Results of box the default 50 (%) the DOTA-v1.0 All models trained with ResNet50. 'Ix' and '3x" schedules indicate 12 epochs and 36 epochs for training. "idicatees using
NV VI00 GPU with more memory. MS denotes multi-scale (Zhou et al., 2022) training and testing.
See the appendix for performance of specific categories.
Method
Sched.
MS
Size
Mem. (GB)
FPS
AP
(Box-supervisedd
RepPoints (Yang et al., 2019b)
1x
1,024
3.44
24.5
64.18
RetinaNet (Lin al., 2017b)
1x
1,024
3.61
25.4
67.83
RetinaNet (Lin et al., 2017b)
1x
1,024
4.17
73.30
CSL (Yang & Yan, 2020)
1x
1,024
3,93
24.6
68.26
GWD (Yang al., 2021c)
1,024
3.61
25.4
69.25
KLD (Yang ct al., 2021d)
1x
1,024
3.61
25,4
69.64
KFIoU (Yang et 2023b)
1X
1,024
3.61
25.4
70.05
SASM (Hou al., 2022)
1x
1,024
3.69
24.4
70.35
Det (Yang et al., 2021b)
1x
1,024
3.78
20.0
71.17
S A-Net (Han et al., 202
1x
1,024
3.37
23.3
74.13
FCOS (Tian et al., 2019)
1x
1,024
4.66
29.5
70.78
FCOS (Tian et al., 2019)
3x
1,024
4.66
29.5
72.22
FCOS (Tian et al., 2019)
1x
1,024
6.23
75.31
HBox- -Boo-superiised:
Box (Tian al., 2021)
1x
960
19.93
2.7
53.59
BoxLevelSet-RBox (Li et al., 2022b)
1x
960
26.81
4.7
56.44
H2RBox (FCOS-based)
1x
960
6.25
31.6
67.90
H2RBox (FCOS-based)
1,024
7.02
29.1
67.82
H2RBox (FCOS-based)
3x
960
6.25
31.6
70.73
H2RBox (FCOS-based)
3x
1,024
7.02
29.1
70.41
H2RBox (FCOS-based)
1,024
8.58
74.40
Table Results of box (%) on the DIOR-R test. models are trained with ResNer50. The input image size is 800 800. 'Ix' and '3x' schedules indicate 12 epochs and 36 epochs. indicates using NV V100 GPU with more memory.
Method
Sched.
Mem. (GB)
FPS
AP
AP 50
AP75
BBoxssuperiseed:
RetinaNet (Lin al., 2017b)
1x
2.48
33.3
33.47
54.60
33.80
KLD (Yang et al., 2021d)
1x
2.48
33.3
35.77
58.00
37.00
GWD (Yang al., 2021c)
1x
2.48
33.3
37.01
57.80
38.20
FCOS (Tian et al., 2019)
1x
3.06
40.8
34.16
58.60
31.90
HBox-vupervised.
BoxLevelSet-RBox" (Li et al., 2022b)
1x
11.44
29.96
56.56
24.36
BoxInsi-RBox (Tian et al., 2021)
Ix
9.23
3.1
31.73
57.40
28.10
H2RBox (FCOS-basse (COOS-base))
1x
4.52
34.9
33.15
57.00
32.60
4.2
MAIN RESULTS
Results on DOTA-y1.0. As shown in Tab. our method significantly outperforms BoxInst-RBox and BoxLevelSet-RBor by 14.31% and 11.46% in terms of AP30, respectively, Moreover, our meth- ods are also more memory and inference efficient. Specifically, compared BoxInst, we only need less than one- third of its memory (6.25 GB vs. 19.93 GB) and have about 12x speed advantage (31.6 fps vs. 2.7 fps). In contrast to BoxLevelSet. our memory costs only quarter of its memory (6.25 GB vs. 26.81 GB), and inference is about times faster (31.6 fps vs. 4.7 fps). In fact, the main cost of the -BBxx methods come from the costive post-processing step for find the compact surrounding box as RBox which is fulfilled by calling OpenCV function in implementation. Even compared with RBox-supervised methods, our method has outperformed several methods, such RepPoints and RetinaNet. Under the 'Ix' and '3x' training schedules, our method slightly lags behind the baseline method, i.e. FCOS (recall After using multi-scal training and testing, the gap is reduced only 0.91% % (75.31% % 74.40%).
(Box-supervised), by 2.96% and 1.81% %.
Results on DIOR-R. Note that some categories this dataset including Chimney, Wind mill, Air- port, Golf field, all forcefully annotated by horizontal boxes though the objects not exactly
8

---

Published as a conference paper at ICLR 2023
Table 3: Ablation for H2RBox with different bor- der effect dismissing strategies for view genera- tion by padding/cropping on DOTA-vI.0.
'Table 4: Ablation with different label re-
assignment strategies. O2M and 020 rep-
resent one-to-many and one-to-one.
Padding
Cropping
AP
APso
AP75
Dataset
Assigner
AP
AP
AP
Zeros
20.17
51.76
12.91
O2M
21.60
53.96
14.14
DOTA
Zeros
33.72
63,95
30.00
020
35,92
67.31
32.78
Reflection
35.92
67.31
32.78
O2M
31.10
56,00
29.80
DIOR-R
Reflection
33.60
64.09
30.02
020
33.15
57.00
32.60
Table Ablation with two strategies S1, S2 dealing with circular category: ST & RA on DOTA-vI.00
Table 6: Ablation with using SS loss (Lxz) or not on DOTA-v1.0 and DIOR-R.
S1
S2
ST
RA
AP
AP
AP75
Dataset
Lge
AP
AP's 50
AP 75
69.82
38.87
31.90
64.52
27.11
DOTA
12.63
37.13
7.54
85.29
64.04
36.36
67.25
33.26
35.92
67.31
32.78
84.58
65.98
35.92
67.31
32.78
DIOR-R
15.27
29.60
13.60
85.41
63.38
36.41
67.22
33.40
33.15
57.00
32.60
horizontal, which may affect the learning and the final results. As shown in Tab. 2, compared with DOTA-v1.0, DIOR-R is less challenging for the instance segmentation methods. This may explain the observation that the performance of H2RBox and BoxInst-RBox on AP is close. Yet for high-precision detection i.c with high APrs that requires more accurate segmentation, H2RBox outperforms BoxLevelSet-RBor and BoxInst-RBox on AP7s by 8.24% (32.6% vs. 24.36%) and 4.50% (32.6% vs. 28. 10%), and with lower memory and high inference speed. Similarly, H2RBox performs slightly inferior than the RBox-supervised FCOS: 33.15% vs. 34.16%.
4.3 ABLATION STUDIES
The ablation study is performed on the proposed H2RBox with 12 training epochs.
Border effect elimination for view generation. Tab. 3 studies the impact of different border effect elimination strategies for view generation. in terms of padding and/or cropping (see Sec. 3.1). Such techniques are essential avoid ground truth angle information leakage otherwise the model will suffer overfitting and leads to significant performance drop verified in the first row of the table. Note that when both reflection padding and cropping are applied the AP slightly drops from 35.92% to 33.60% compared with only using reflection padding. The reason may be due to that reduced size of input image by cropping. Hence in all other experiments we always use reflection padding alone.
Label re-assignment. Tab. 4 shows the one-to-one strategy outperforms one-to-many strategy.
Strategies for dealing with sotropic circular object classes. For circular objects like Storage Tank (ST) and Roundabout (RA), the self-supervised loss takes effect as it is insensitive to isotropic information. We take two treatments to handle such circular objects. S1: for training, we mask the SS loss for circular classes. S2: for testing, the horizontal cireumscribed rectangle of the circular category taken as the final output. Tab. 5 shows that, when either or both strategies is used, the performance can be greatly improved, about 15% ST and about 25% on RA.
Self-supervised loss. Without using SS loss, Tab. 6 shows that our method only achieves 12.63% and 15.27% DOTA-v1.0 10 and DIOR-R, respectively. In contrast, the use of SS loss leads substantial increase in overall performance, reaching 35.92% and 35%. Figure 4(b) also shows that the SS loss can effectively help the model learn the correct object angle information.
CONCLUSION
This paper presents H2RBox, the first (to the best of our knowledge) HBox-supervised oriented ob- ject detector. H2RBox learns the rotation via self-supervised learning, whose loss measures the con- sistency of the predicted angles in two different views. Compared to the alternative HBox- -upeevuue instance segmentation methods, H2RBox achieves much higher detection accuracy especially for complex scenes, yet with lower memory and higher speed. Compared with fully RBox -upx-super algorithms, our method still shows competitive.

---

Published as a conference paper at ICLR 2023
REFERENCES
Seyed Majid Azimi, Reza Bahmanyar, Corentin Henry, and Franz Kurz. Eagle: Large-scale vehicle detection dataset in real-world scenarios using aerial imagery. In 25th International Conference on Pattern Recognition, pp. 6920-6927. IEEE, 2021.
Zhiming Chen, Kean Chen, Weiyao Lin, John See, Hui Yu, Yan Ke, and Cong Yang. Piou loss: Towards accurate oriented object detection in complex environments. In European Conference on Computer Vision, pp. 195-211, 2020.
Gong Cheng, Jiabao Wang, Ke Li, Xingxing Xie, Chunbo Lang, Yanqing Yao, and Junwei Han. Anchor-free free oriented proposal generator for object detection. IEEE Transactions on Geoscience and Remole Sensing, 2022.
Jian Ding, Nan Xue, Yang Long, Gui-Song Xia, and Qikai Lu. Learning roi transformer for oriented object detection in aerial images. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2849-2858, 2019.
Eran Goldman, Roci Herzig, Aviv Eisenschtat, Jacob Goldberger, and Tal Hassner. Precise detection in densely packed scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5227-5236, 2019.
Jiaming Han, Jian Ding, Jie Li, and Gui- Gui-Song Xia. Align deep features for oriented object detection. IEEE Transactions Geoscience and Remote Sensing, 60:1-11, 2021a.
Jiaming Han, Jian Ding, Nan Xue, and Gui-Song Xia. Redet: A rotation-equivarian detector for aerial object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2786-2795, 2021b.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2961-2969, 2017.
Liping Hou, Ke Lu, Jian Xue, and Yuqiu Li. Shape-adaptive selection and measurement for oriented object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
Liping Hou, Ke Lu, Xue Yang, Yuqiu Li, and Jian Xue. G-rep: Gaussian representation for arbitrary- oriented object detection. Remote Sensing, 15(3):757, 2023.
Cheng-Chun Hsu, Kuang-Jui Hsu, Chung-Chi Tsai, Yen-Yu Lin, and Yung-Yu Chuang- Weakly supervised instance segmentation using the bounding box tightness prior. Advances in Neural Information Processing Systems, 32, 2019.
Shi-Min Hu, Dun Liang, Guo-Ye Yang, Guo-Wei Yang, and Wen- Wen-Yang Zhou. Jittor: novel deep learning framework with meta-operators and unified graph execution. Science China Information Sciences, 63:1-21, 2020.
Anna Khoreva, Rodrigo Benenson, Jan Hosang, Matthias Hein, and Bernt Schiele. Simple does it:
Weakly supervised instance and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 876-885, 2017.
Ke Li, Gang Wan, Gong Cheng, Liqiu Meng, and Junwei Han. Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 159:296-307, 2020.
Wentong Li, Yijie Chen, Kaixuan Hu, and Jianke Zhu. Oriented reppoints for aerial object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1829- 1838, 2022a.
Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Xiansheng Hua, and Lei Zhang. Box- supervised instance segmentation with level set evolution. In European Conference on Computer Vision, 2022b.
10

---

Published as a conference paper at ICLR 2023
Minghui Liao, Baoguang Shi, and Xiang Bai. Textboxes++: A single-shot oriented scene text detector. IEEE Transactions on Image Processing, 27(8):667- 2018.
Tsung-Yi Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 2117-2125, 2017a.
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE International Conference Computer Vision, pp- 1980-2988, 2017b.
Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A survey. International journal of computer vision, 128(2)2661-118, 2020.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Confer- ence on Learning Representations, 2018.
Jianqi Ma, Weiyuan Shao, Hao Ye, Li Wang, Hong Wang, Yingbin Zheng, Xiangyang Xue. orbinray scene text detection via rotation proposals. IEEE Transactions Multimedia, 101111:11133122, 2018.
Xingjia Pan, Yuqiang Ren, Kekai Sheng, Weiming Dong, Haolei Yuan, Xiaowei Guo, Chongyang Ma, and Changsheng Xu. Dynamic refinement network for oriented and densely packed object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11207-11216, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In Advances in neural information processing systems, 2019.
Jordi Pont-Tuset, Pablo Arbelacz, Jonathan T Barron, Ferran Marques, and Jitendra Malik. Mul- tiscale combinatorial grouping for image segmentation and object proposal generation. IEEE transactions on pattern analysis and machine intelligence, 39(1): 2016.
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 9627- 9636, 2019.
Zhi Tian, Chunhua Shen, and Hao Chen. Conditional convolutions for instance segmentation. In European conference computer vision, pp. 982-298 Springer, 2020.
Zhi Tian, Chunhua Shen, Xinlong Wang, and Hao Chen. Boxinst: High-performance instance segmentation with box annotations. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5443-5452, 2021.
Xuchui Wang, Kai Zhao, Ruixin Zhang, Shouhong Ding, Yan Wang, and Wei Shen. Contrastmask: Contrastive learning to segment every thing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11164-116111 2022.
Haoran Wei, Yue Zhang, Bing Wang, Yang Yang, Hao Li, and Hongqi Wang. X-linenet: Detecting aireraft in remote sensing images by pair of intersecting line segments. IEEE Transactions Geoscience and Remote Sensing, 59(2):1645-1633 2020.
Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang. Dota: A large-scale dataset for object detection in aerial images. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3974- 3983, 2018.
Yongchao Xu, Mingtao Fu, Qimeng Wang, Yukang Wang, Kai Chen, Gui-Song Xia, and Xiang Bai. Gliding vertex on the horizontal bounding box for multi-oriented object detection. JEEE Transactions Pattern Analysis and Machine Intelligence, 13(4):1452-1455) 2020.

---

Published as a conference paper at ICLR 2023
Xue Yang and Junchi Yan. Arbirray--orientd object detection with circular smooth label. In Euro- pean Conference on Computer Vision. pp. 677-694, 2020.
Xue Yang and Junchi Yan. On the arbitrary-oriented object detection: Classification based ap- proaches revisited. International Journal of Computer Vision, 13065):1340-1355 2022.
Xue Yang, Hao Sun, Kun Fu, Jinui Yang, Xian Sun, Menglong Yan, and Zhi Guo. Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sensing, 10(1):132, 2018.
Xue Yang, Jirui Yang. Junchi Yan, Yue Zhang, Tengfei Zhang, Zhi Guo, Xian Sun, and Kun Fu. Scrdet: Towards more robust detection for small, cluttered and rotated objects. In Proceedings of the IEEE International Conference on Computer Vision, pp. 8232-82411 2019a.
Xue Yang, Liping Hou, Yue Zhou, Wentao Wang, and Junchi Yan. Dense label encoding for bound- ary discontinuity free rotation detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1pp.558121 15819-15829, 2021a.
Xue Yang, Junchi Yan, Ziming Feng, and Tao He. R3det: Refined single-stage detector with feature refinement for rotating object. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 3163-3171 2021b.
Xue Yang, Junchi Yan, Qi Ming, Wentao Wang, Xiaopeng Zhang, and Qi Tian. Rethinking rotated object detection with gaussian wasserstein distance loss. In International Conference on Machine Learming, pp. 11830-11841, PMLR, 2021c
Xue Yang, Xiaojiang Yang. Jirui Yang, Qi Ming. Wentao Wang, Qi Tian, and Junchi Yan. Learning Sigh-precision bounding box for rotated object detection via kullback-leible divergence. Ad- vances Neural Information Processing Systems, 34:18381-18394 2021d.
Xue Yang, Gefan Zhang, Xiaojiang Yang, Yue Zhou, Wentao Wang, Jin Tang, Tao He, and Junchi Yan. Detecting rotated objects gaussian distributions and its 3-d generalization. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2022.
Xue Yang. Junchi Yan, Wenlong Liao, Xiaokang Yang, Jin Tang, and Tao He. Scrdet++: Detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smooth- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2))2384-233)) 2023a.
Xue Yang, Yue Zhou, Gefan Zhang, Jirui Yang, Wentao Wang, Junchi Yan, Xiaopeng Zhang, and Qi Tian. The kfiou loss for rotated object detection. In International Conference on Learning Representations, 2023b.
Ze Yang. Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. Reppoints: Point set representation for object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 9657-9666, 2019b.
Jiahui Yu, Yuning Jiang, Zhangyang Wang. Zhimin Cao, and Thomas Huang. Unitbox: An ad- vanced object detection network. In Proceedings of the 24th ACM International Conference on Multimedia, pp. 516-520, 2016.
Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9759-9768. 2020.
Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. East: an efficient and accurate scene text detector. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5551-5560, 2017.
Yue Zhou, Xue Yang, Gefan Zhang, Jiabao Wang. Yanyi Liu, Liping Hou, Xue Jiang, Xingzhao Liu, Junchi Yan, Chengqi Lyu, Wenwei Zhang, and Kai Chen. Mmrotate: rotated object de- tection benchmark using pytorch. In Proceedings of the 30th ACM International Conference Multimedia, pp. 7331-7334, 2022.
12

---

Published as a conference paper at ICLR 2023
A FEASIBLE SOLUTIONS UNDER DIFFERENT CONSTRAINTS
Three different constraints, including horizontal circumscribed rectangle constraint (HCRC), scale constraint (SC) and angle constraint (AC), introduced in this paper guide the model to learn the correct result. Fig. 6(a) shows when there are only horizontal circumscribed rectangle constraint, the feasible solutions are still infinite. After adding scale constraint, only the symmetric case and the correct case are left, as shown in Fig. 6(b). The final angle constraint allows the correct solution be preserved, refer to Fig. 6(c).
(a) HCRC only
(b) HCRC SC
(c) HCRC + SC + AC
Figure 6: Visualization of feasible solutions under different constraints.
Table 7: Results of box the default AP50 (%) on the DOTA-v1.0. models are trained with ResNet50. 'Ix' and '3x' schedules indicate 12 epochs and 36 epochs for training. "indicates using NV V100 GPU with more memory. MS denotes multi-scale (Zhou et al., 2022) training and testing.
Method
Sched. MS
Size
Mem. (GB)
FPS
AP50
RBox supx-supee
RepPoints (Yang ct al., 2019b)
1x 1,024
3.44
24.5
64.18
RetinaNet (Lin et al., 2017b)
1x
1,024
3.61
25.4
67.83
RetinaNet (Lin al., 2017b)
1x
1,024
4.17
73.30
CSL (Yang & Yan, 2020)
1x
1,024
3.93
24.6
68.26
GWD (Yang al., 2021c)
1x
1,024
3.61
25.4
69.25
KLD (Yang et al., 2021d)
1,024
3.61
25.4
69.64
KFIoU (Yang ct , 2023b)
1x
1,024
3.61
25,4
70.05
SASM (Hou al., 2022)
1x
1,024
3.69
24.4
70.35
Det (Yang et al., 2021b)
1x
1,024
3.78
20.0
71.17
ATSS (Zhang et al., 2020)
1x
1,024
3.32
26.5
71.98
S A-Net (Han et al., 2021a)
1,024
3.37
23.3
74.13
FCOS (Tian et al., 2019)
1x
1,024
4.66
29.5
70.78
FCOS (Tian et al., 2019)
3x
1,024
4.66
29.5
72.22
FCOS (Tian et al., 2019)
Ix
1,024
6.23
75.31
HBox- BBxxsuuervsse:::
BoxInst-Boxx (Tian al., 2021)
1x
960
19,93
2.7
53.59
BoxLevelSet-RBoo (Li et al., 2022b)
1x
960
26.81
4.7
56.44
H2RBox (ATSS-based)
1x
1,024
5.50
25.7
67.24
H2RBox (FCOS-based)
1x
960
6.25
31.6
67.90
H2RBox IFCOS-based)
1x
1.024
7.02
29.1
67.82
H2RBox (FCOS-based)
3x
960
6.25
31.6
70.73
H2RBox (FCCOS-based)
3x
1.024
29.1
70.41
H2RBox IFCOS-baseed
1x
V
1,024
8.58
74.40
13

---

Published as a conference paper at ICLR 2023
Table 8:
Results of each category the DOTA-v1.. test set.
Method
BR
GTF
SY
LVV
IC
ST
SBF
RA
HIA
mAPoo
RepP-ints-ss00
$4.79
13.35
00.64
56.51
71.55
$2.21
73.40
90.64
76.25
K5.15
58.17
61.43
54.91
64.43
18.57
64.18
RetinaNet 3x (20176)
89.11
74.52
64.69
72.18
71.80
63.59
74.94
90.78
75.71
50.56
50.45
59.17
62.36
64.35
39,69
67.53
RetinaNet Ix ms (20175)
$3.10
34.43
50.54
73.65
72.94
90.39
86.45
57.24
65.02
05.55
07.09
70.72
58.48
73.30
(2020)
(2021
72:73
KFlo1--X (20236)
Wiloo sopersoddd
Boslast RBox-1 (2021)
68.43
40.75
33.0
32.29
46.91
55.43
56.55
79.49
66.81
K2.14
41.24
$2.83
$2.80
63.04
29.59
$3.59
BosLeveiSet R.Bo (20226)
63.48
71.27
39.34
61.06
41.89
41.83
45.83
90.57
74.12
72.13
47.59
62.99
50.00
56.42
28.63
56.44
H2RtBox KISS 1x
$7.82
74.73
43.22
09.57
72.67
53.95
70.91
59.33
85.58
K3.44
$4.77
03.77
47.15
66.25
41.29
67.24
H22Boxx FCOS Ix
$8.47
73.51
00.81
56.89
77.08
05.42
71.87
50.38
83.19
K3.27
55.27
62.90
32.41
63.83
43.26
67.82
H22tBox
88.24
79.30
02.
75.90
72.70
71.54
50:35
81.95
$4.38
55
61
70.83
51.51
70.41
H2RtBoo-F F'OOS-I--m Ix-ms
83.93
78.89
06.27
68.19
81.12
90.39
86.77
87.33
64.15
88
09.99
39.79
B VERIFICATION EXPERIMENTS ON DIFFERENT DETECTORS
As shown Tab. 7. we have conducted experiments on different basic detectors, including anchor based method (ATSS (Zhang al., 2020)) and anchor free method (FCOS (Tian al., 2019)). It can be seen that the method proposed in this paper has excellent portability. Tab. 8 lists the performance of each class of each method Tab. 7.
14


Reviews:
[{"review_id": "lAbigFrWTs1", "reviewer": "Reviewer_uvAp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a weakly supervised method for oriented object detection by leveraging only horizontal bounding boxes. The network is able to learn to predict the rotated boxes by building the transition functions between a WS branch on original images and a SS branch  on rotated images. By building upon the FCOS detector, the proposed method performs favorably against state-of-the-art weakly supervised approaches.", "summary_of_the_review": "This paper brings some new insights for weakly supervised oriented object detection. Currently, I'm glad to accept it.", "strength_and_weaknesses": "Strength\n\n-The main contribution that building the self-supervised learning strategy on a rotated image is interesting. \n\n-The experiments are sufficient to demonstrate the effectiveness of each component and the overall performance.\n\nWeaknesses\n\n-The authors mainly evaluate the proposed framework on FCOS, it's suggested to verify the model's generalization on other baseline methods for general object detection or oriented object detection. \n\n-The authors used several loss items with empirical trade-offs. It's necessary to explain the reasons and show their effects on final performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "eqGwfPzfBo", "reviewer": "Reviewer_o5T8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a method to detect oriented object with only horizontal bounding box annotations. In detail, it designed two losses. The first is the weakly supervised loss and the second is the self-supervised consistency loss. The proposed method outperformed the box supervised instance segmentation method such as BoxInst. It is also close to the oriented box supervised model.", "summary_of_the_review": "The problem studied in this paper is new. But the method is not very novel. But the performance is good. So I prefer to accept this paper.", "strength_and_weaknesses": "Strength\n1. The direction is interesting and new.\n2. The proposed method achieved very good performance (comapred to the previous methods and oriented box supervised models).\n\nWeakness\n1. The method is not very novel. For example using the horizontal circumscribed rectangles as supervision is similar to the techinical used in BoxInst[1]. The scale consistency loss is also widely used in previous methods, such as [2, 3].\n\n[1] https://arxiv.org/abs/2012.02310\n[2] https://arxiv.org/abs/1905.02249\n[3] https://arxiv.org/pdf/2004.04581.pdf", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "b3wdLSQJrwW", "reviewer": "Reviewer_jpAe", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed an oriented object detector that used horizontal box annotations. The proposed method applied weakly- and self-supervised learning for predicting the angle of the object. The experimental results on standard benchmarks show the effectiveness of the proposed method.", "summary_of_the_review": "Considering the strength and weaknesses of this paper, I tend to give a positve recommendation.", "strength_and_weaknesses": "## Strength:\n### (1) The proposed method is interesting since it can predict the angle of the object without the supervision of the angle.\n### (2) The experimental results show that it can achieve competing results compared to the fully-supervised methods.\n\n## Weaknesses:\n### (1) The proposed method need more GPU memory than previous methods, as listed in Table 1.\n### (2) The definition of the angles of the objects are quite fusing for me. For example, a horizontal box that exactly match the object can be either 0 degree or 90 degree. How to distinguish them?\n### (3) The contributions listed in the Introduction Section are weak since two items of them are the experimental results. It would be better to specify the technical contributions instead of the experimental results.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RwlcGULdsG", "reviewer": "Reviewer_upuU", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents HBox-supervised oriented object detector.\nH2RBox learns the rotation via self-supervised learning, whose loss measures the consistency of the predicted angles in two different views. \n", "summary_of_the_review": "This paper has significant contribution to the community, and the results are quite good (very close performance and speed compared to rotated box-supervised methods).\n It may reveal that we can use HBox annotations than RBox one to learn oriented object detectors.\n", "strength_and_weaknesses": "Compared to the alternative HBox-supervised instance segmentation methods, H2RBox achieves much higher detection accuracy especially for complex scenes, yet with lower memory and higher speed.\nCompared with fully RBox-supervised algorithms, our method still shows competitive, and sometimes even better performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster