Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence

1 Introduction . As a fundamental building block for visual analysis across aerial images , scene text etc. , rotated object detection has recently been developed rapidly [ 1 , 2 , 3 , 4 , 5 , 6 ] , which benefit themselves from the well-established horizontal detection approaches [ 7 , 8 , 9 , 10 , 11 ] . Specifically , many works [ 12 , 13 , 14 , 15 ] build themselves upon the previously established horizontal detection pipeline from an inductive perspective , as shown in Figure 1 ( a ) . However , these detectors are often unable to cope with challenging scenes well due to the limitations of current regression loss , such as large aspect ratio objects , dense scenes , etc. , resulting in obvious disadvantages in high-precision detection . ∗Part of the work was done during an internship at Huawei Inc. †Correspondence author is Junchi Yan . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . ( a ) Previous methods follow the induction paradigm from special horizontal to general rotated detection . ( b ) Our proposed method adopts a deduction methodology from general rotated to special horizontal detection . Figure 1 : Methodological road-map difference between horizontal detection ( special case ) and rotation detection ( general case ) in the previous methods [ 1 , 12 , 13 , 14 , 15 ] and the proposed method . In this paper , we take a step back , and aim to develop ( from a deductive perspective ) a unified regression framework for rotation detection and its special case : horizontal detection . In fact , our new framework enjoys a coherent property that it can be degenerated into the current commonly used regression loss ( e.g . ln-norm ) in special cases ( horizontal detection ) , as shown in Figure 1 ( b ) . For a devising a rotation regression loss for high-precision rotation detection , one important observation is that the importance of different parameters to different types of objects can vary . For example , the angle parameter ( θ ) and the center point parameter ( x , y ) are important for large aspect ratio objects and small objects , respectively . In another word , it is conjectured that regression loss should be self-modulated during the learning process and calls for more dynamic optimization strategy . Inspired by the above ideas , we first convert the rotated bounding box B ( x , y , h , w , θ ) into a 2-D Gaussian distribution N ( µ , Σ ) . As a standard distance metric , we then use the Kullback-Leibler Divergence ( KLD ) [ 16 ] to calculate the distribution distance between the predicted bounding box and ground truth as the regression loss . We compare KLD with Smooth L1 loss [ 7 ] and another distance metric , Gaussian Wasserstein Distance ( GWD ) [ 5 , 17 ] , and find that KLD has a more complete parameter optimization mechanism . In particular , by analyzing the gradient of the parameters during learning , we show that the optimization of one parameter will be affected by other parameters ( as the gradient weight ) . It means that the model will adaptively adjust the optimization strategy given a specific configuration of an object for detection , as shown can lead to excellent performance in high-precision detection . In addition , KLD is proven scale invariant , which is an important property that Smooth L1 loss and GWD do not possess . As the horizontal bounding box is a special case of the rotated bounding box , we show that KLD can also be degenerated into the ln-norm loss as commonly used in existing horizontal detection pipeline . The highlights of this paper are four-folds : 1 ) Differing from the dominant existing practices that build rotation detectors heavily upon the horizontal detectors , we develop new rotation detection loss from scratch and show that it is coherent with existing horizontal detection protocol in its degenerated case for horizontal detection . 2 ) To achieve a more principled measurement between the prediction and ground truth , instead of computing the difference for each physically-meaningful parameter related to the bounding box which are in different scales and units , we innovatively convert the regression loss of rotation detection into the KLD of two 2-D Gaussian distributions , leading to a clean and coherent regression loss . 3 ) Through the gradient analysis of each parameter in KLD , we further find that the self-modulated optimization mechanism of KLD greatly promotes the improvement of high-precision detection , which verify the advantage of our loss design . More importantly , we have theoretically shown ( in appendix ) that KLD is scale invariant for detection , which is crucial for the rotation cases . 4 ) Extensive experimental results on seven public datasets and two popular detectors show the effectiveness of our approach , which achieves new state-of-the-art performance for rotation detection . The source codes [ 18 ] are made public available . 2 Background . We first generally discuss the related works on both horizontal and rotated object detection . Then we summarize the current design paradigm of rotation regression loss from two kinds of methodologies , as shown in Figure 1 : one is inductive that tries to develop the general rotation detection from the special and classic horizontal detection pipeline . While the other is deductive that aims to devise a general rotation detection pipeline with horizontal detection as its special case . 2.1 Related Works . Horizontal object detection . Horizontal object detection which covers most existing detection literature , normally uses a horizontal bounding box to represent the object . The mainstream classical object detection algorithms can be roughly divided according to the following standards : Two [ 7 , 8 , 9 , 11 ] or Single-stage [ 10 , 19 , 20 ] object detection , Anchor-free [ 21 , 22 , 23 ] or Anchor-based [ 8 , 9 , 10 ] object detection and CNN [ 8 , 10 , 21 ] or Transformer-based [ 24 , 25 ] object detection . Although the pipelines may vary , the mainstream regression loss often uses the popular ln-norm loss ( such as smooth L1 loss ) or IoU-based loss ( such as GIoU [ 26 ] , and DIoU [ 27 ] ) . These abovementioned detectors have also been widely used in other scenarios and have achieved satisfactory performance . However , horizontal detectors do not provide accurate orientation and scale information . Rotated object detection . Recent advances in rotation detection [ 3 , 4 , 12 , 14 , 28 ] are mainly driven by adapting the horizontal object detectors with rotated bounding boxes to represent multi-oriented objects . To accurately predict the rotated bounding box , most rotation detection methods extend the ln-norm [ 12 , 15 , 29 , 30 , 31 ] used in horizontal detection , or construct a differentiable approximate IoU loss [ 3 , 5 , 32 ] . From scratch , we try to change the design of rotation regression loss from induction paradigm to deduction methodology , which in fact is a generalization to the horizontal case . In the following , we describe the existing works from the induction and deduction methodologies . 2.2 Inductive Thinking of Loss Design : from Special Horizon to General Rotation Detection . Regression loss is a vital part of most current object detection algorithms . For horizontal bounding box regression , the model [ 7 , 8 , 9 , 10 , 11 ] mainly outputs four items for location and size : tpx = xp − xa wa , tpy = yp − ya ha , tpw = ln ( wp wa ) , tph = ln ( hp ha ) ( 1 ) to match the four targets from the ground truth ttx = xt − xa wa , tty = yt − ya ha , ttw = ln ( wt wa ) , tth = ln ( ht ha ) ( 2 ) where x , y , h , w denote the center coordinates , height and width , respectively . Variables xt , xa , xp are for the ground-truth box , anchor box , and predicted box , respectively ( likewise for y , w , h ) . Extending the above horizontal case , existing rotation detection models [ 1 , 12 , 13 , 14 , 15 ] also use regression loss which simply involves an extra angle parameter θ : tpθ = f ( θp − θa ) , t t θ = f ( θt − θa ) ( 3 ) where f ( · ) is used to deal with angular periodicity , such as trigonometric functions , modulo , etc . The overall regression loss for rotation detection is : Lreg = ln-norm ( ∆tx , ∆ty , ∆tw , ∆th , ∆tθ ) ( 4 ) where ∆tx = tpx− ttx = ∆xwa , ∆ty = t p y− tty = ∆y ha , ∆tw = tpw− ttw = ln ( wp/wt ) , ∆th = t p h− tth = ln ( hp/ht ) , and ∆tθ = t p θ − ttθ = ∆θ . It can be seen that parameters are optimized independently , making the loss ( or detection accuracy ) sensitive to the under-fitting of any of the parameters . This mechanism is fatal to high-precision detection . Taking the left side of Figure 2 as an example , the detection result based on the Smooth L1 loss often shows the deviation of the center point or angle . Moreover , different types of objects have different sensitivity to these five parameters . For example , the angle parameter is very important for detecting objects with large aspect ratios . This requires to select an appropriate set of weights given a specific single object sample during the training , which is nontrivial or even unrealistic . 2.3 Deductive Thinking of Loss Design : from General Rotation to Special Horizon Detection . To break the original inductive design paradigm , we adopt deductive paradigm to construct more accurate rotation regression loss . Here we rephrase the main idea in the recent work [ 5 ] , which converts a arbitrary-oriented bounding box B ( x , y , h , w , θ ) into a 2-D Gaussian N ( µ , Σ ) , as illustracted in Figure 3 . Then the distance between two Gaussian is calculated as the final loss . Specifically , the conversion is : µ = ( x , y ) > Σ1/2 =RΛR > = ( cos θ − sin θ sin θ cos θ ) ( w 2 0 0 h 2 ) ( cos θ sin θ − sin θ cos θ ) = ( w 2 cos2 θ + h 2 sin2 θ w−h 2 cos θ sin θ w−h 2 cos θ sin θ w 2 sin2 θ + h 2 cos2 θ ) ( 5 ) where R represents the rotation matrix , and Λ represents the diagonal matrix of eigenvalues . The recent work [ 5 ] analyzes that the introduction of N ( µ , Σ ) can solve the inconsistency between metric and loss , boundary discontinuity and square-like problem . On this basis , we further study how to design high-precision detection regression loss through new parameter space . Our view is that the self-modulated mechanism is positively correlated with the final high-precision performance . Gaussian Wasserstein Distance . The Wasserstein distance [ 5 , 17 ] between two probability measures Xp ∼ Np ( µp , Σp ) and Xt ∼ Nt ( µt , Σt ) expressed as : Dw ( Np , Nt ) 2 = ‖µp − µt‖22︸ ︷︷ ︸ center distance + Tr ( Σp + Σt − 2 ( Σ1/2p ΣtΣ1/2p ) 1/2 ) ︸ ︷︷ ︸ coupling terms about hp , wp and θp ( 6 ) Eq . 6 shows that the Gaussian Wasserstein Distance ( GWD ) is mainly divided into two parts : the distance between the center points ( x , y ) and the coupling terms about h , w and θ . Accordingly , the regression loss based on GWD can be regarded as a semi-coupled loss . Although GWD can greatly improve the performance of high-precision rotation detection due to the coupling between part of the parameters , the independent optimization of the center point make the detection result slightly shifted ( see Figure 2 ) . Note that GWD is not scale invariant , which is not detection friendly . When all the boxes are horizontal ( θ = 0◦ ) , Eq . 6 can be further simplified : Dhw ( Np , Nt ) 2 =‖µp − µt‖22 + ‖Σ1/2p −Σ 1/2 t ‖ 2 F = ( xp − xt ) 2 + ( yp − yt ) 2 + ( ( wp − wt ) 2 + ( hp − ht ) 2 ) /4 =l2-norm ( ∆x , ∆y , ∆w/2 , ∆h/2 ) ( 7 ) where ‖ · ‖F is the Frobenius norm . Although Eq . 7 can still be used as the regression loss of horizontal detection , Eq . 4 and 7 are not completely consistent . Although GWD scheme has played a preliminary exploration of the deductive paradigm , it does not focus on achieving high-precision detection and scale invariance . In the following , we will propose our new approach based on the Kullback-Leibler divergence ( KLD ) [ 16 ] .

Summary: This paper presents a new design of the regression loss function for oriented object detection. When modeling the rotated bounding boxes as 2D Gaussian distributions, the authors introduced the Kullback-Leibler Divergence for oriented bounding boxes regression as an alternative to the commonly-used Smooth-l1 loss function. The experimental results show that the proposed loss function could obtain better performance on several benchmarks.