Title: Rank-DETR for High Quality Object Detection
Venue: NeurIPS 2023
Accepted: yes
Keywords: Object Detection
Abstract: Modern detection transformers (DETRs) use a set of object queries to predict a list of bounding boxes, sort them by their classification confidence scores, and select the top-ranked predictions as the final detection results for the given input image. A highly performant object detector requires accurate ranking for the bounding box predictions. For DETR-based detectors, the top-ranked bounding boxes suffer from less accurate localization quality due to the misalignment between classification scores and localization accuracy, thus impeding the construction of high-quality detectors. In this work, we introduce a simple and highly performant DETR-based object detector by proposing a series of rank-oriented designs, combinedly called Rank-DETR. Our key contributions include: (i) a rank-oriented architecture design that can prompt positive predictions and suppress the negative ones to ensure lower false positive rates, as well as (ii) a rank-oriented loss function and matching cost design that prioritizes predictions of more accurate localization accuracy during ranking to boost the AP under high IoU thresholds. We apply our method to improve the recent SOTA methods (e.g., H-DETR and DINO-DETR) and report strong COCO object detection results when using different backbones such as ResNet-$50$, Swin-T, and Swin-L, demonstrating the effectiveness of our approach. Code is available at \url{https://github.com/LeapLabTHU/Rank-DETR}.

Rank-DETR for High Quality Object Detection

Abstract

Modern detection transformers (DETRs) use a set of object queries to predict a list of bounding boxes, sort them by their classification confidence scores, and select the top-ranked predictions as the final detection results for the given input image. Ahighly performant object detector requires accurate ranking for the bounding box predictions. For DETR-based detectors, the top-ranked bounding boxes suffer from less accurate localization quality due to the misalignment between classification scores and localization accuracy, thus impeding the construction of high-quality detectors. In this work, we introduce a simple and highly performant DETR-based object detector by proposing a series of rank-oriented designs, combinedly called Rank-DETR. Our key contributions include: (i) a rank-oriented architecture design that can prompt positive predictions and suppress the negative ones to ensure lower false positive rates, as well as (ii) a rank-oriented loss function and matching cost design that prioritizes predictions of more accurate localization accuracy during ranking to boost the AP under high IoU thresholds. We apply our method to improve the recent SOTA methods (e.g., H-DETR and DINO-DETR) and report strong COCO object detection results when using different backbones such as ResNet50 , Swin-T, and Swin-L, demonstrating the effectiveness of our approach. Code is available at https://github.com/LeapLabTHU/Rank-DETR .

1 Introduction

The landscape of modern object detection systems has undergone significant transformation since the pioneering work DEtection TRansformer (DETR) [3]. Since DETR yielded impressive results in object detection, numerous subsequent research works such as Deformable-DETR [78], DINO [75], and H-DETR [30] have further advanced this field. Moreover, these DETR-based approaches have been successfully extended to address various core visual recognition tasks, including instance / panoptic segmentation [6, 38, 61, 69, 71, 35, 9, 8, 70], pose estimation [57, 36, 56], and multi-object tracking [5, 47, 58]. The notable progress in these areas can be credited to ongoing advancements in enhancing the DETR-based framework for improved object detection performance.

Considerable endeavors have been dedicated to advancing the performance of DETR-based methods from various perspectives. These efforts include refining transformer encoder and decoder architectures [48, 74, 78, 2, 12, 78, 11, 40, 43] as well as redesigning of query formulations [63, 44, 34, 75]. While substantial research has been dedicated to developing accurate ranking mechanisms for dense one-stage object detectors like FCOS [59] or ATSS [77], few studies have specifically investigated this aspect for modern object detectors based on DETR. However, ranking mechanisms are vital in enhancing the average precision performance, particularly under high IoU thresholds.

This study's primary focus revolves around constructing a high-quality object detector using DETR that exhibits strong performance at relatively high IoU thresholds. We acknowledge the criticality of establishing an accurate ranking order for bounding box predictions in constructing these detectors. To achieve this, we introduce two rank-oriented designs that effectively leverage the benefits of precise ranking information. First, we propose a rank-adaptive classification head and a query rank layer after each Transformer decoder layer. Rank-adaptive classification head adjusts the classification scores using rank-aware learnable logit bias vectors, while the query rank layer fuses additional ranking embeddings into the object queries. Second, we propose two rank-oriented optimization techniques: a loss function modification and a matching cost design. These functions facilitate the ranking procedure of the model and prioritize more accurate bounding box predictions with higher IoU scores when compared to the ground truth. In summary, our rank-oriented designs consistently enhance object detection performance, particularly the AP scores under high IoU thresholds.

To validate the efficacy of our approach, we conducted comprehensive experiments, showcasing consistent performance improvements across recent strong DETR-based methods such as H-DETR and DINO-DETR. For example, based on H-DETR, our method demonstrates a notable increase in AP 75 of + 2 . 1% ( 52 . 9% vs. 55 . 0% ) and + 2 . 7% ( 55 . 1% vs. 57 . 8% ) when utilizing ResNet50 and Swin-T backbones, respectively. It is worth highlighting that our approach achieves competitive performance, reaching an 50 . 2% AP in the 1 × training schedule on the COCO val dataset. These results serve as compelling evidence for the effectiveness and reliability of our proposed methodology.

2 Related Work

DETR for Object Detection. Since the groundbreaking introduction of transformers in 2D object detection by the pioneering work DETR [3], numerous subsequent studies[48, 11, 7, 63, 46] have developed diverse and advanced extensions based on DETR. This is primarily due to DETR's ability to eliminate the need for hand-designed components such as non-maximum suppression (NMS). One of the first foundational developments, Deformable-DETR [78], introduced a multi-scale deformable self/cross-attention scheme, which selectively attends to a small set of key sampling points in a reference bounding box. This approach yielded improved performance compared to DETR, particularly for small objects. Furthermore, DAB-DETR [44] and DN-DETR [34] demonstrated that a novel query formulation could also enhance performance. The subsequent work, DINO-DETR [75], achieved state-of-the-art results in object detection tasks, showcasing the advantages of DETR design by addressing the inefficiency caused by the one-to-one matching scheme. In contrast to these works, our focus lies in the design of the rank-oriented mechanism for DETR. We propose rank-oriented architecture designs and rank-oriented matching cost and loss function designs to construct a highly performant DETR-based object detector with competitive AP 75 results.

Ranking for Object Detection. There exists a lot of effort to study how to improve the ranking for object detection tasks. For example, IoU-Net [31] constructed an additional IoU predictor and an IoU-guided NMS scheme that considers both classification scores and localization scores during inference. Generalized focal loss [37] proposed a quality focal loss to act as a joint representation of the IoU score and classification score. VarifocalNet [76] introduced an IoU-aware classification score to achieve a more accurate ranking of candidate detection results. TOOD [10] defined a high-order combination of the classification score and the IoU score as the anchor alignment metric to encourage the object detector to focus on high-quality anchors dynamically. In addition, ranking-based loss functions [53, 4, 49, 50, 32] are designed to encourage the detector to rank the predicted bounding boxes according to their quality and penalizes incorrect rankings. The very recent con-current work Stable-DINO [45] and Align-DETR [1] also applied the idea of IoU-aware classification score to improve the loss and matching design for DINO-DETR [75]. In contrast to the aforementioned endeavors, we further introduce a query rank scheme aimed to reduce false positive rates.

Dynamic Neural Networks. In contrast to static models, which have fixed computational graphs and parameters at the inference stage, dynamic neural networks [15, 65] can adapt their structures or parameters to different inputs, leading to notable advantages in terms of performance, adaptiveness [68, 13], computational efficiency [72, 73, 67, 60], and representational power [52]. Dynamic networks are typically categorized into three types: sample-wise [27, 66, 18, 14, 51], spatial-wise [62, 28, 19, 17, 16], and temporal-wise [20, 64]. In this work, we introduce a novel query-wise dynamic approach, which dynamically integrates ranking information into the object queries based on their box quality ranking, endowing object queries with better representation ability.

3 Approach

Above all, we revisit the overall pipeline of the modern DETR-based methods [30, 75] in Section 3.1. The detailed design of the proposed method, including the rank-oriented architecture design and optimization design, is subsequently illustrated in Section 3.2 and Section 3.3. Eventually, we discuss the connections and differences between our approach and related works in Section 3.4.

3.1 Preliminary

Pipeline. Detection Transformers (DETRs) process an input image I by first passing it through a backbone network and a Transformer encoder to obtain a sequence of enhanced pixel embeddings X = { x 1 , x 2 , · · · , x N } . The enhanced pixel embeddings, along with a default set of object query embeddings Q 0 = { q 0 1 , q 0 2 , · · · , q 0 n } , are then fed into the Transformer decoder. After each Transformer decoder layer, task-specific prediction heads are applied to the updated object query embeddings to generate a set of classification predictions P l = { p l 1 , p l 2 , · · · , p l n } and bounding box predictions B l = { b l 1 , b l 2 , · · · , b l n } , respectively, where l ∈ { 1 , 2 , · · · , L } denotes the layer index of the Transformer decoder. Finally, DETR performs one-to-one bipartite matching between the predictions and the ground-truth bounding boxes and labels G = { g 1 , g 2 , · · · , g m } by associating each ground truth with the prediction that has the minimal matching cost and applying the corresponding supervision.

Object Query. To update the object query Q 0 after each Transformer decoder layer, typically, DETRs form a total of L subsets, i.e., {Q 1 , Q 2 , · · · , Q L } , for L Transformer decoder layers. For both the initial object query Q 0 and the updated ones after each layer, each Q l is formed by adding two parts: content queries Q l c = { q l c, 1 , q l c, 2 , · · · , q l c,n } and position queries Q l p = { q l p, 1 , q l p, 2 , · · · , q l p,n } . The content queries capture semantic category information, while the position queries encode prior positional information such as the distribution of bounding box centers and sizes.

Ranking in DETR. Rank-oriented design plays a crucial role in modern object detectors, particularly in achieving superior average precision (AP) scores under high Intersection over Union (IoU) thresholds. The success of state-of-the-art detectors, such as H-DETR and DINO-DETR, relies on using simple rank-oriented designs, specifically a two-stage scheme and mixed query selection. These detectors generate the initial positional query Q 0 p by ranking the dense coarse bounding box predictions output by the Transformer encoder feature maps and selecting the top ∼ 300 confident ones. During evaluation, they gather n × K bounding box predictions based on the object query embedding Q L produced by the final Transformer decoder layer (each query within Q L generates K predictions associated with each category), sort them by their classification confidence scores in descending order, and only return the top ∼ 100 most confident predictions.

In this work, we focus on further extracting the benefits brought by the ranking-oriented designs and introduces a set of improved designs to push the envelope of high-quality object detection performance. The subsequent discussion provides further elaboration on these details.

3.2 Rank-oriented Architecture Design: ensure lower FP and FN

While the original rank-oriented design only incorporates rank information into the initial positional query Q 0 p , we propose an enhanced approach that leverages the benefits of sorting throughout the entire Transformer decoder process. Specifically, we introduce a rank-adaptive classification head after each Transformer decoder layer, and a query rank layer before each of the last L -1 Transformer decoder layers. This novel design is intended to boost the detection of true positives while suppressing false positives and correcting false negatives, leading to lower false positive rates and false negative rates. Figure 1 illustrates the detailed pipeline of our rank-oriented architecture designs.

Rank-adaptive Classification Head. We modify the original classification head by adding a set of learnable logit bias vectors S l = { s l 1 , s l 2 , · · · , s l n } to the classification scores T l = { t l 1 , t l 2 , · · · , t l n } (before Sigmoid( · ) function) associated with each object query independently. The classification predictions of the l -th decoder layer P l = { p l 1 , p l 2 , · · · , p l n } can be formulated as:

where Q l = { q l 1 , q l 2 , · · · , q l n } represents the output embedding after the l -th Transformer decoder layer. The hidden dimensions of both t l i and s l i are the number of categories, i.e., K . The overall

pipeline is shown in Figure 1(b). It is noteworthy that we can directly incorporate a set of learnable embedding, denoted as S l , into the classification scores T l . This is practicable because the associated Q l has already been sorted in the query rank layer, as explained below.

Query Rank Layer. We further introduce a query rank layer before each of the last L -1 Transformer decoder layers to regenerate the sorted positional query and content query accordingly.

First, we explain how to construct the rank-aware content query:

where we first sort the output of ( l -1) -th Transformer decoder layer Q l -1 c in descending order of ˆ P l -1 = MLPcls ( Q l -1 c ) . Since each element in ˆ P l -1 is a K -dimensional vector, we use the maximum value over K categories (classification confidence) as the ranking basis. The operation symbol Sort( A ; B ) sorts elements within A based on the decreasing order of the elements in B . Then, we concatenate ( ∥ ) the sorted object content queries ˆ Q l -1 c with a set of randomly initialized content query C l in the feature dimension, where l ∈ { 2 , · · · , L } . This set of content query C l is optimized in an end-to-end manner. Subsequently, we fuse them back to the original dimension using a fully connected layer ( MLP fuse). In other words, for each Transformer decoder layer, we maintain a set of rank-aware static content embeddings shared across different samples. These embeddings effectively model and utilize the distribution of the most frequent semantic information [26].

Next, we present the mathematical formulations for computing the rank-aware positional query. To align the order of the positional queries with the ranked content query, we either sort or recreate the positional queries, depending on the initialization method of positional queries for different DETR-based detectors. For H-DETR, which inherits Deformable DETR and uses the same positional query for all L Transformer decoder layers, we simply sort the positional query of the previous layer:

For DINO-DETR, which generates new positional queries from the bounding box predictions in each Transformer decoder layer, we sort the bounding box predictions of each object query and recreate the positional query embedding from the sorted boxes:

where B l -1 and ˆ P l -1 represent the bounding box predictions and classification predictions based on the output of ( l -1) -th Transformer decoder layer, i.e., Q l -1 . PE( · ) contains a sine position encoding function and a small multilayer perceptron to recreate the positional query embedding Q l p . In other words, each element of Q l p is estimated by q l p,i = PE( b l -1 i ) . In Figure 1(c), we illustrate the positional query update process for H-DETR (Equation (3)) and omit that process for DINO-DETR (Equation (4)), because we primarily conducted experiments on H-DETR.

Finally, we transmit the regenerated rank-aware positional query embedding and content query embedding to the subsequent Transformer decoder layer.

(a) The original Matching Cost and Loss

(b) Rank-oriented Matching Cost and Loss

Analysis. The key motivations behind these two rank-oriented architecture designs are adjusting the classification scores of the object queries according to their ranking order information. Within each Transformer decoder layer, we incorporate two sets of learnable representations: the logit bias vectors S l and the content query vectors C l . By leveraging these two rank-aware representations, we have empirically demonstrated the capability of our approach to effectively address false positives (oLRPFP: 24 . 5% → 24 . 1% ) and mitigate false negatives (oLRPFN: 39 . 5% → 38 . 6% ). For a more comprehensive understanding of our findings, please refer to the experiment section.

3.3 Rank-oriented Matching Cost and Loss: boost the AP under high IoU thresholds

The conventional DETR and its derivatives specify the Hungarian matching cost function L Hungarian and the training loss function L using the identical manner, as shown below:

where we use ˆ b , ˆ p (or b , c ) to represent the predicted (or ground-truth) bounding box and classification score respectively. c corresponds to the ground-truth semantic category of b . In addition, we use the notation FL( · ) to represent the semantic classification focal loss [41]. We propose to enhance the rank-oriented design by introducing the GIoU-aware classification head (supervised by the proposed GIoU-aware classification loss) and the high-order matching cost function scheme as follows.

GIoU-aware Classification Loss. Instead of applying the binary target to supervise the classification head, we propose to use the normalized GIoU scores to supervise the classification prediction:

where we let t = (GIoU( ˆ b , b )+1) / 2 , and we denote the GIoU-aware classification loss as FL GIoU ( · ) . To incorporate both semantic classification and localization accuracy, we modify the original classification head to a GIoU-aware classification head supervised by the above loss function. When t = 1 , FL GIoU ( · ) simplifies to the original focal loss, denoted as FL( · ) . Furthermore, we compare our results with the varifocal loss [76], and we provide its mathematical formulation below:

High-order Matching Cost. In contrast to utilizing the Hungarian algorithm with a matching cost defined as the weighted sum of classification cost, ℓ 1 loss, and GIoU loss, we propose employing a high-order matching cost function that captures a more intricate combination of the classification score and the IoU scores:

where IoU represents the intersection over union scores between the predicted bounding box and the ground truth one. We adopt a larger value of α (e.g., > 2 ) to prioritize the importance of localization accuracy, thereby promoting more accurate bounding box predictions and downgrading the inaccurate ones. It is worth noting that we use a high-order matching cost from the middle training stage, as most predictions exhibit poor localization quality during the early training stage.

Analysis. The rank-oriented loss function and matching cost are designed to enhance object detection performance at high IoU thresholds. The GIoU-aware classification loss facilitates the ranking

procedure by endowing the classification score with GIoU-awareness, resulting in more accurate ranking in the query ranking layer. Meanwhile, the high-order matching cost selects the queries with both high classification confidence and superior IoU scores as positive samples, effectively suppressing challenging negative predictions with high classification scores but low localization IoU scores. This is achieved by magnifying the advantage of a more accurate localization score using γ α , where γ is the ratio of accurate localization score to less accurate score. Empirical results show significant boosts in AP 75 with GIoU-aware classification loss ( 52 . 9% → 54 . 1% ) or high-order matching cost design ( 52 . 9% → 54 . 0% ). Detailed comparisons are provided in the experiment section.

3.4 Discussion

The concept of GIoU-aware classification loss has been explored in several prior works [31, 37, 76, 10] predating the era of DETR. These works aimed to address the discrepancy between classification scores and localization accuracy. In line with recent concurrent research [1, 45], our method shares the same insight in designing rank-oriented matching cost and loss functions. However, our approach distinguishes itself by emphasizing the ranking aspect and introducing an additional rank-oriented architecture design, which includes the rank-adaptive classification head and query rank layer. Furthermore, our empirical results demonstrate the complementarity between the rank-oriented architecture design and the rank-oriented matching cost and loss design.

4 Experiment

4.1 Experiment Setting

We perform object detection experiments using the COCO object detection benchmark [42] with detrex [54] toolbox. Our model is trained on the train set and evaluated on the val set. We adhere to the same experimental setup as the original papers for H-DETR [30] and DINO-DETR [75].

4.2 Main Results

Comparison with competing methods. Table 1 compares Rank-DETR with other high-performing DETR-based methods on the COCO object detection val dataset. The evaluation demonstrates that Rank-DETR achieves remarkable results, attaining an AP score of 50 . 2% with only 12 training epochs. This performance surpasses H-DETR [30] by + 1 . 5% and outperforms the recent state-of-theart method DINO-DETR [75] by + 1 . 2% AP. Notably, we observe significant improvements in AP 75 , highlighting the advantage of our approach at higher IoU thresholds.

Improving H-DETR [30]. Table 2 presents a detailed comparison of our proposed approach with the highly competitive H-DETR [30]. The experimental evaluation demonstrates consistent enhancements in object detection performance across various backbone networks and training schedules. For example, under the 12 epochs training schedule, our method achieves superior AP scores of 50 . 2% , 52 . 7% , and 57 . 3% with ResNet-50, Swin-Tiny, and Swin-Large backbone networks, respectively. These results surpass the baseline methods by + 1 . 5% , + 2 . 1% , and + 1 . 4% , respectively. Extending the training schedule to 36 epochs consistently improves the AP scores, resulting in + 1 . 2% for ResNet-50, + 1 . 5% for Swin-Tiny, and + 1 . 1% for Swin-Large. The AP improvement in performance is more significant under high IoU thresholds, which outperform the baseline by + 2 . 1% , + 2 . 7% , and + 1 . 9% in AP 75 with ResNet-50, Swin-Tiny, and Swin-Large, respectively. These findings validate our proposed mechanism's consistent and substantial performance improvements across diverse settings and with different backbone networks, especially under high IoU thresholds. We also validate our performance gain by providing the PR curves under different IoU thresholds in Figure 3a.

Method

DINO-DETR

Ours

DINO-DETR

Ours

AP

S

32

.

33

41

41

0

.

.

.

6

1

6

AP

M

52

.

3

53

.

60

.

61

.

8

6

4

AP

L

63

.

64

0

.

73

.

73

.

2

5

8

Backbone

50

R

50

R

Swin-L

Swin-L

#epochs

12

12

12

12

AP

49

.

0

50

56

57

.

4

.

8

.

6

AP

50

66

.

6

67

75

76

.

9

.

4

.

0

AP

75

53

.

5

55

.

62

.

63

.

2

3

4

Improving DINO-DETR [75]. Table 3 shows the results of applying our approach to improve the DINO-DETR [75]. Notably, our method demonstrates an increase of + 1 . 4% with the ResNet50 backbone and + 0 . 8% with the Swin-Large backbone. Under the higher IoU setting, our method further obtains + 1 . 8% AP 75 improvement with ResNet50 and + 1 . 1% with Swin-Large. These results provide evidence for the generalization ability of our approach across different DETR-based models.

4.3 Ablation Study and Analysis

We conduct a systematic analysis to assess each proposed component's influence within our method. We followed a step-by-step approach, progressively adding modules on top of the baseline (Table 4a), incorporating each module into the baseline (Table 4b), and subsequently removing each module from our method (Table 4c). This procedure allowed us to understand the contribution of each individual component to the final performance. Furthermore, we conducted statistical and qualitative analyses to comprehensively assess the functionality of each component. We mark the best-performing numbers with ■ colored regions in each table along each column.

Effect of Adding Each Component Progressively. We choose H-DETR with ResNet-50 backbone as our baseline method. By progressively adding the proposed mechanisms on top of the baseline model, it is observed that the performance is steadily increasing, and the best performance is achieved by using all the proposed components (Table 4a). It is also observed that the lowest false negative rate (oLRPFN) is achieved when only using the rank-oriented architecture designs.

Rank-adaptive Classification Head (RCH). Tables 4b and 4c demonstrate that rank-adaptive classification head can slightly improve the AP ( +0 . 2% when adding RCH to the H-DETR baseline, comparing row1 and row2 in Table 4b; +0 . 4% when adding RCH to complete our methods, comparing row1 and row2 in Table 4c). Furthermore, RCH improves AP 75 more than AP.

Query Ranking Layer (QRL). The proposed QRL mechanism effectively integrates ranking information into the DETR architecture, compensating for the absence of sequential handling of queries in attention layers. The detector's performance is also consistently improved by utilizing QRL ( +0 . 3% when adding QRL to the H-DETR baseline; +0 . 7% to complete our method). We further compute the cumulative probability distribution of classification scores of positive and negative queries. QRL yields enhanced classification confidence for matched positive queries (Figure 3b),

RCH

QRL

GCL

HMC

AP

AP

50

AP

75

AP

S

AP

M

AP

L

AR

1

AR

10

AR

100

AR

S

AR

M

AR

L

oLRP

oLRP

oLRP

FP

oLRP

FN

✗

✗

✗

✗

48

.

7

66

.

4

52

.

9

31

.

2

51

.

5

63

.

5

37

.

2

63

.

4

68

.

4

49

.

7

72

.

5

85

.

9

61

.

0

13

.

3

24

.

5

39

.

5

✓

✗

✗

✗

48

.

9

66

.

9

53

.

3

31

.

2

52

.

4

63

.

7

37

.

5

64

.

4

71

.

2

53

.

5

75

.

5

87

.

1

61

.

2

13

.

3

24

.

0

39

.

2

✓

✓

✗

✗

49

.

3

67

.

3

53

.

7

32

.

4

52

.

2

63

.

4

37

.

8

65

.

0

71

.

7

54

.

6

75

.

7

88

.

4

60

.

8

13

.

3

24

.

1

38

.

6

✓

✓

✓

✗

49

.

8

67

.

5

54

.

3

33

.

3

53

.

4

63

.

7

37

.

9

65

.

1

71

.

8

54

.

8

76

.

1

87

.

8

60

.

7

13

.

0

23

.

5

39

.

3

✓

✓

✓

✓

50

.

2

67

.

7

55

.

0

34

.

1

53

.

6

64

.

0

38

.

1

64

.

9

71

.

6

56

.

5

75

.

8

86

.

4

60

.

4

12

.

9

22

.

4

39

.

3

(a) Effect of gradually adding modules on the baseline.

RCH

QRL

GCL

HMC

AP

AP

50

AP

75

AP

S

AP

M

AP

L

AR

1

AR

10

AR

100

AR

S

AR

M

AR

L

oLRP

oLRP

oLRP

FP

oLRP

FN

✗

✗

✗

✗

48

.

7

66

.

4

52

.

9

31

.

2

51

.

5

63

.

5

37

.

2

63

.

4

68

.

4

49

.

7

72

.

5

85

.

9

61

.

0

13

.

3

24

.

5

39

.

5

✓

✗

✗

✗

48

.

9

66

.

9

53

.

3

31

.

2

52

.

4

63

.

7

37

.

5

64

.

4

71

.

2

53

.

5

75

.

5

87

.

1

61

.

2

13

.

3

24

.

0

39

.

2

✗

✓

✗

✗

49

.

0

67

.

2

53

.

2

32

.

3

51

.

9

63

.

5

37

.

9

64

.

8

71

.

5

55

.

1

75

.

6

87

.

5

61

.

1

13

.

4

23

.

8

39

.

2

✗

✗

✓

✗

49

.

4

67

.

0

54

.

1

32

.

0

52

.

8

64

.

0

37

.

9

64

.

9

71

.

7

55

.

4

75

.

7

88

.

5

60

.

7

12

.

9

22

.

5

39

.

8

✗

✗

✗

✓

49

.

3

67

.

3

54

.

0

31

.

8

52

.

4

63

.

4

37

.

7

64

.

4

71

.

1

53

.

7

74

.

9

85

.

8

61

.

1

13

.

2

23

.

8

39

.

2

(b) Effect of incorporating each module on the baseline.

RCH

QRL

GCL

HMC

AP

AP

50

AP

75

AP

S

AP

M

AP

L

AR

1

AR

10

AR

100

AR

S

AR

M

AR

L

oLRP

oLRP

oLRP

FP

oLRP

FN

✓

✓

✓

✓

50

.

2

67

.

7

55

.

0

34

.

1

53

.

6

64

.

0

38

.

1

64

.

9

71

.

6

56

.

5

75

.

8

86

.

4

60

.

4

12

.

9

22

.

4

39

.

3

✗

✓

✓

✓

49

.

8

67

.

3

54

.

5

33

.

5

53

.

4

63

.

6

38

.

1

64

.

9

71

.

5

55

.

9

75

.

3

86

.

6

60

.

5

12

.

9

22

.

9

39

.

6

✓

✗

✓

✓

49

.

5

67

.

4

54

.

2

33

.

1

52

.

8

63

.

5

38

.

0

64

.

7

71

.

6

55

.

8

75

.

4

85

.

9

60

.

9

12

.

9

24

.

6

39

.

3

✓

✓

✗

✓

49

.

5

67

.

6

54

.

1

32

.

4

52

.

6

64

.

3

37

.

9

64

.

3

71

.

0

54

.

4

75

.

1

85

.

4

60

.

8

13

.

4

23

.

7

38

.

7

✓

✓

✓

✗

49

.

8

67

.

5

54

.

3

33

.

3

53

.

4

63

.

7

37

.

9

65

.

1

71

.

8

54

.

8

76

.

1

87

.

8

60

.

7

13

.

0

23

.

5

39

.

3

(c) Effect of removing each module in our method.

(a) Effect of the GIoU-aware classification loss target formulation.

(b) Effect of the matching cost formulation.

while the classification confidence for unmatched queries is effectively suppressed (Figure 3c), thereby ranking true predictions higher than potential false predictions. This phenomenon is further apparent from the oLRP FP results showcased in Table 4b, which is reduced from 24 . 5% to 23 . 8% by using QRL, reducing the false positive rates. These results are in accordance with our design intent.

GIoU-aware Classification Loss (GCL). Table 4 also shows the effectiveness of the proposed GIoU-aware classification loss, which gains 0 . 7% mAP over the vanilla baseline by comparing row1 and row4 in Table 4b and 0 . 7% mAP increase by comparing row1 and row4 in Table 4c. We

(a)

0.0

0.2

0.4

0.6

0.8

1.0

Classification Score

0.000

0.005

0.010

0.015

Frequency

H-DETR

H-DETR + QRL

(b)

0.00

0.05

0.10

0.15

0.20

Classification Score

0.00

0.01

0.02

0.03

0.04

Frequency

H-DETR

H-DETR + QRL

(c)

0.0

0.2

0.4

0.6

0.8

1.0

IoU

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative Frequency

H-DETR

H-DETR + HMC

(a) Layer #1

0.0

0.2

0.4

0.6

0.8

1.0

IoU

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative Frequency

H-DETR

H-DETR + HMC

(b) Layer #2

0.0

0.2

0.4

0.6

0.8

1.0

IoU

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative Frequency

H-DETR

H-DETR + HMC

(c) Layer #3

0.0

0.2

0.4

0.6

0.8

1.0

IoU

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative Frequency

H-DETR

H-DETR + HMC

(d) Layer #4

0.0

0.2

0.4

0.6

0.8

1.0

IoU

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative Frequency

H-DETR

H-DETR + HMC

(e) Layer #5

0.0

0.2

0.4

0.6

0.8

1.0

IoU

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative Frequency

H-DETR

H-DETR + HMC

(f) Layer #6

also ablate the formulation of the learning target t in Eq. (6) in Table 5a. The results show that the performance of adopting IoU (and its exponent) as the optimization target is inferior to use ( GIoU +1) / 2 because GIoU can better model the relationship of two non-overlapped boxes. We use ( GIoU +1) / 2 rather than GIoU because -1 < GIoU ≤ 1 and 0 < ( GIoU +1) / 2 ≤ 1 .

High-order Matching Cost (HMC). We also show how the high-order matching cost affects the overall performance in Table 4. The HMC can also significantly improve the overall performance of the object detector ( +0 . 6% when comparing row1 and row5 in Table 4b, +0 . 4% when comparing row1 and row5 in Table 4c). We further ablate the formulation of the matching cost. As illustrated in Table 5b, a high-order exponent IoU can consistently improve the performance and achieve a peak when the power is 4 . Using a high-order exponent can suppress the importance of the predicted boxes with low IoU. We can also observe the same trend from Table 5b when replacing IoU with ( GIoU+1 ) / 2 , but the latter practice has a slightly inferior performance.

HMCSuppresses the Overlap between Negative Query and Ground Truth . Figure 4 illustrates the cumulative probability distribution of the IoU of unmatched queries. The IoU of each unmatched query is defined as the largest IoU between it and all ground truth boxes. As shown in Figure 4, the adoption of HMC can decrease the IoU between unmatched queries and all the ground truth bounding boxes, effectively pushing the negative queries away from the ground truth boxes. Furthermore, this phenomenon is increasingly remarkable in the latter Transformer decoder layers.

Comparison with Varifocal loss. In order to assess the effectiveness of the proposed GIoU-aware classification loss (GCL, Eq. (6)), we compare it with varifocal loss (VFL) [76] due to their similar mathematical formulations. Following VFL [76], we utilize the training target t = IoU in Eq. (7). To simplify the comparison and focus on the impact of GCL, we conduct the evaluation without HMC ( row3 in Table 4a). Our method, leveraging the GCL, achieves a mAP of 49 . 8% ( row4 in Table 4a), whereas VFL achieves only 49 . 5% mAP. The primary distinctions between GCL and VFL lie in the optimization target. By employing the normalized GIoU as the training target, our approach better models the distance between two non-overlapping boxes, leading to improved performance. In addition, VFL removes scaling factors on positive samples, as they are rare compared to negatives in CNN-based detectors. However, for DETR-based detectors, where positive examples are relatively more abundant, we empirically show that retaining a scaling factor can enhance performance.

Layer #1

score: 0.34

score: 0.48

Layer #2

score: 0.43

score: 0.42

Layer #3

score: 0.44

score: 0.33

Layer #4

score: 0.42

score: 0.31

Layer #5

score: 0.43

score: 0.35

Layer #6

score: 0.41

score: 0.36

score: 0.31

score: 0.13

score: 0.35

score: 0.05

score: 0.47

score: 0.12

score: 0.49

score: 0.06

score: 0.46

score: 0.02

score: 0.44

score: 0.02

Computational Efficiency. In Table 6, we provide comprehensive data on the number of parameters, computational complexity measured in FLOPs, training time per epoch, inference frames per second (FPS), and Average Precision performance, for both the H-DETR baseline and our approach. These assessments were conducted on RTX 3090 GPUs, allowing us to evaluate testing and training efficiency. The results unequivocally highlight a substantial enhancement in detection performance achieved by our proposed method, with only a slight increase in FLOPs and inference latency. Considering the effectiveness and efficiency, our method has the potential to be adapted into 3D object detection [55, 29], semantic segmentation [39, 33] tasks, or other applications [23, 24, 25, 21, 22].

Qualitative Analysis. Figure 5 visualizes the predicted bounding boxes and their classification confidence scores of both the matched positive queries and the unmatched hard negative queries, respectively. We find that, compared to the baseline method ( row1 ), the proposed approach ( row2 ) effectively promotes the classification scores of positive samples, while that of hard negative queries is rapidly suppressed, progressing layer by layer. These qualitative results further illustrate how the proposed approach achieves high performance by decreasing the false positive rate.

5 Conclusion

This paper presents a series of simple yet effective rank-oriented designs to boost the performance of modern object detectors and result in the development of a high-quality object detector named RankDETR. The core insight behind the effectiveness lies in establishing a more precise ranking order of predictions, thereby ensuring robust performance under high IoU thresholds. By incorporating accurate ranking order information into the network architecture and optimization procedure, our approach demonstrates improved performance under high IoU thresholds within the DETR framework. While there remains ample scope to explore leveraging rank-oriented designs, we hope that our initial work serves as an inspiration for future efforts in building high-quality object detectors.

References

Zhi Cai, Songtao Liu, Guodong Wang, Zheng Ge, Xiangyu Zhang, and Di Huang. Align-DETR: Improving DETR with simple IoU-aware BCE loss. arXiv:2304.07527 , 2023.

Xipeng Cao, Peng Yuan, Bailan Feng, Kun Niu, and Yao Zhao. CF-DETR: Coarse-to-fine transformers for end-to-end object detection. In AAAI , 2022.

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV , 2020.

Kean Chen, Weiyao Lin, Jianguo Li, John See, Ji Wang, and Junni Zou. AP-loss for accurate one-stage object detection. IEEE TPAMI , 2020.

Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, and Huchuan Lu. Transformer tracking. In CVPR , 2021.

Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In CVPR , 2022.

Xiyang Dai, Yinpeng Chen, Jianwei Yang, Pengchuan Zhang, Lu Yuan, and Lei Zhang. Dynamic DETR: End-to-end object detection with dynamic attention. In ICCV , 2021.

Bin Dong, Fangao Zeng, Tiancai Wang, Xiangyu Zhang, and Yichen Wei. SOLQ: Segmenting objects by learning queries. In NeurIPS , 2021.

Yuxin Fang, Shusheng Yang, Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. Instances as queries. In ICCV , 2021.

Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R Scott, and Weilin Huang. TOOD: Taskaligned one-stage object detection. In ICCV , 2021.

Peng Gao, Minghang Zheng, Xiaogang Wang, Jifeng Dai, and Hongsheng Li. Fast convergence of detr with spatially modulated co-attention. In ICCV , 2021.

Ziteng Gao, Limin Wang, Bing Han, and Sheng Guo. Adamixer: A fast-converging query-based object detector. In CVPR , 2022.

Jiayi Guo, Chaofei Wang, You Wu, Eric Zhang, Kai Wang, Xingqian Xu, Humphrey Shi, Gao Huang, and Shiji Song. Zero-shot generative model adaptation via image-specific prompt learning. In CVPR , 2023.

Yizeng Han, Dongchen Han, Zeyu Liu, Yulin Wang, Xuran Pan, Yifan Pu, Chao Deng, Junlan Feng, Shiji Song, and Gao Huang. Dynamic perceiver for efficient visual recognition. In ICCV , 2023.

Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang. Dynamic neural networks: A survey. TPAMI , 2021.

Yizeng Han, Gao Huang, Shiji Song, Le Yang, Yitian Zhang, and Haojun Jiang. Spatially adaptive feature refinement for efficient inference. IEEE TIP , 2021.

Yizeng Han, Zeyu Liu, Zhihang Yuan, Yifan Pu, Chaofei Wang, Shiji Song, and Gao Huang. Latency-aware unified dynamic networks for efficient image recognition. arXiv:2308.15949 , 2023.

Yizeng Han, Yifan Pu, Zihang Lai, Chaofei Wang, Shiji Song, Junfen Cao, Wenhui Huang, Chao Deng, and Gao Huang. Learning to weight samples for dynamic early-exiting networks. In ECCV , 2022.

Yizeng Han, Zhihang Yuan, Yifan Pu, Chenhao Xue, Shiji Song, Guangyu Sun, and Gao Huang. Latency-aware spatial-wise dynamic networks. In NeurIPS , 2022.

Christian Hansen, Casper Hansen, Stephen Alstrup, Jakob Grue Simonsen, and Christina Lioma. Neural speed reading with structural-jump-lstm. In ICLR , 2019.

Chunming He, Kai Li, Guoxia Xu, Jiangpeng Yan, Longxiang Tang, Yulun Zhang, Xiu Li, and Yaowei Wang. Hqg-net: Unpaired medical image enhancement with high-quality guidance. arXiv:2307.07829 , 2023.

Chunming He, Kai Li, Guoxia Xu, Yulun Zhang, Runze Hu, Zhenhua Guo, and Xiu Li. Degradation-resistant unfolding network for heterogeneous image fusion. In ICCV , 2023.

Chunming He, Kai Li, Yachao Zhang, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Camouflaged object detection with feature decomposition and edge reconstruction. In CVPR , 2023.

Chunming He, Kai Li, Yachao Zhang, Guoxia Xu, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Weakly-supervised concealed object segmentation with sam-based pseudo labeling and multi-scale feature grouping. arXiv:2305.11003 , 2023.

Chunming He, Kai Li, Yachao Zhang, Yulun Zhang, Zhenhua Guo, Xiu Li, Martin Danelljan, and Fisher Yu. Strategic preys make acute predators: Enhancing camouflaged object detectors by generating camouflaged objects. arXiv:2308.03166 , 2023.

Haodi He, Yuhui Yuan, Xiangyu Yue, and Han Hu. RankSeg: Adaptive pixel classification with image category ranking for segmentation. In ECCV , 2022.

Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens Van Der Maaten, and Kilian Q Weinberger. Multi-scale dense networks for resource efficient image classification. In ICLR , 2018.

Gao Huang, Yulin Wang, Kangchen Lv, Haojun Jiang, Wenhui Huang, Pengfei Qi, and Shiji Song. Glance and focus networks for dynamic visual recognition. IEEE TPAMI , 2022.

Rui Huang, Xuran Pan, Henry Zheng, Haojun Jiang, Zhifeng Xie, Shiji Song, and Gao Huang. Joint representation learning for text and 3d point cloud. arXiv:2301.07584 , 2023.

Ding Jia, Yuhui Yuan, Haodi He, Xiaopei Wu, Haojun Yu, Weihong Lin, Lei Sun, Chao Zhang, and Han Hu. DETRs with hybrid matching. In CVPR , 2023.

Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, and Yuning Jiang. Acquisition of localization confidence for accurate object detection. In ECCV , 2018.

Fehmi Kahraman, Kemal Oksuz, Sinan Kalkan, and Emre Akbas. Correlation loss: Enforcing correlation between classification and localization. In AAAI , 2023.

Xin Lai, Yuhui Yuan, Ruihang Chu, Yukang Chen, Han Hu, and Jiaya Jia. Mask-attention-free transformer for 3d instance segmentation. In ICCV , 2023.

Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. DN-DETR: Accelerate detr training by introducing query denoising. In CVPR , 2022.

Feng Li, Hao Zhang, Shilong Liu, Lei Zhang, Lionel M Ni, Heung-Yeung Shum, et al. Mask DINO: Towards a unified transformer-based framework for object detection and segmentation. In CVPR , 2023.

Ke Li, Shijie Wang, Xiang Zhang, Yifan Xu, Weijian Xu, and Zhuowen Tu. Pose recognition with cascade transformers. In CVPR , 2021.

Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. In NeurIPS , 2020.

Zhiqi Li, Wenhai Wang, Enze Xie, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, Ping Luo, and Tong Lu. Panoptic segformer: Delving deeper into panoptic segmentation with transformers. In CVPR , 2022.

Zhanhao Liang and Yuhui Yuan. Mask frozen-detr: High quality instance segmentation with one gpu. arXiv preprint arXiv:2308.03747 , 2023.

Matthieu Lin, Chuming Li, Xingyuan Bu, Ming Sun, Chen Lin, Junjie Yan, Wanli Ouyang, and Zhidong Deng. DETR for crowd pedestrian detection. arXiv:2012.06785 , 2020.

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV , 2017.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In ECCV , 2014.

Yutong Lin, Yuhui Yuan, Zheng Zhang, Chen Li, Nanning Zheng, and Han Hu. Detr does not need multi-scale or locality design. In ICCV , 2023.

Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. DAB-DETR: Dynamic anchor boxes are better queries for DETR. In ICLR , 2022.

Shilong Liu, Tianhe Ren, Jiayu Chen, Zhaoyang Zeng, Hao Zhang, Feng Li, Hongyang Li, Jun Huang, Hang Su, Jun Zhu, et al. Detection transformer with stable matching. arXiv:2304.04742 , 2023.

Yan Ma, Weicong Liang, Yiduo Hao, Bohan Chen, Xiangyu Yue, Chao Zhang, and Yuhui Yuan. Revisiting detr pre-training for object detection. arXiv:2308.01300 , 2023.

Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer. Trackformer: Multi-object tracking with transformers. In CVPR , 2022.

Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, and Jingdong Wang. Conditional DETR for fast training convergence. In ICCV , 2021.

Kemal Oksuz, Baris Can Cam, Emre Akbas, and Sinan Kalkan. A ranking-based, balanced loss function unifying classification and localisation in object detection. In NeurIPS , 2020.

Kemal Oksuz, Baris Can Cam, Emre Akbas, and Sinan Kalkan. Rank & sort loss for object detection and instance segmentation. In ICCV , 2021.

Yifan Pu, Yizeng Han, Yulin Wang, Junlan Feng, Chao Deng, and Gao Huang. Fine-grained recognition with learnable semantic data augmentation. arXiv:2309.00399 , 2023.

Yifan Pu, Yiru Wang, Zhuofan Xia, Yizeng Han, Yulin Wang, Weihao Gan, Zidong Wang, Shiji Song, and Gao Huang. Adaptive rotated convolution for rotated object detection. In ICCV , 2023.

Qi Qian, Lei Chen, Hao Li, and Rong Jin. DR loss: Improving object detection by distributional ranking. In CVPR , 2020.

Tianhe Ren, Shilong Liu, Feng Li, Hao Zhang, Ailing Zeng, Jie Yang, Xingyu Liao, Ding Jia, Hongyang Li, He Cao, et al. detrex: Benchmarking detection transformers. arXiv:2306.07265 , 2023.

Yichao Shen, Zigang Geng, Yuhui Yuan, Yutong Lin, Ze Liu, Chunyu Wang, Han Hu, Nanning Zheng, and Baining Guo. V-detr: Detr with vertex relative position encoding for 3d object detection. arXiv:2308.04409 , 2023.

Dahu Shi, Xing Wei, Liangqi Li, Ye Ren, and Wenming Tan. End-to-end multi-person pose estimation with transformers. In CVPR , 2022.

Lucas Stoffl, Maxime Vidal, and Alexander Mathis. End-to-end trainable multi-instance pose estimation with transformers. arXiv:2103.12115 , 2021.

Peize Sun, Jinkun Cao, Yi Jiang, Rufeng Zhang, Enze Xie, Zehuan Yuan, Changhu Wang, and Ping Luo. Transtrack: Multiple object tracking with transformer. arXiv:2012.15460 , 2020.

Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object detection. In ICCV , 2019.

Chaofei Wang, Qisen Yang, Rui Huang, Shiji Song, and Gao Huang. Efficient knowledge distillation from model checkpoints. In NeurIPS , 2022.

Huiyu Wang, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. Max-deeplab: End-to-end panoptic segmentation with mask transformers. In CVPR , 2021.

Shenzhi Wang, Liwei Wu, Lei Cui, and Yujun Shen. Glancing at the patch: Anomaly localization with global and local feature comparison. In CVPR , 2021.

Yingming Wang, Xiangyu Zhang, Tong Yang, and Jian Sun. Anchor DETR: Query design for transformer-based detector. In AAAI , 2022.

Yulin Wang, Zhaoxi Chen, Haojun Jiang, Shiji Song, Yizeng Han, and Gao Huang. Adaptive focus for efficient video recognition. In ICCV , 2021.

Yulin Wang, Yizeng Han, Chaofei Wang, Shiji Song, Qi Tian, and Gao Huang. Computationefficient deep learning for computer vision: A survey. arXiv:2308.13998 , 2023.

Yulin Wang, Rui Huang, Shiji Song, Zeyi Huang, and Gao Huang. Not all images are worth 16x16 words: Dynamic transformers for efficient image recognition. In NeurIPS , 2021.

Qisen Yang, Shenzhi Wang, Matthieu Gaetan Lin, Shiji Song, and Gao Huang. Boosting offline reinforcement learning with action preference query. In ICML , 2023.

Qisen Yang, Shenzhi Wang, Qihang Zhang, Gao Huang, and Shiji Song. Hundreds guide millions: Adaptive offline reinforcement learning with expert guidance. IEEE TNNLS , 2023.

Qihang Yu, Huiyu Wang, Dahun Kim, Siyuan Qiao, Maxwell Collins, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. CMT-DeepLab: Clustering mask transformers for panoptic segmentation. In CVPR , 2022.

Xiaodong Yu, Dahu Shi, Xing Wei, Ye Ren, Tingqun Ye, and Wenming Tan. SOIT: Segmenting objects with instance-aware transformers. In AAAI , 2022.

Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object-contextual representations for semantic segmentation. In ECCV , 2020.

Yang Yue, Bingyi Kang, Xiao Ma, Gao Huang, Shiji Song, and Shuicheng Yan. Offline prioritized experience replay. arXiv:2306.05412 , 2023.

Yang Yue, Bingyi Kang, Zhongwen Xu, Gao Huang, and Shuicheng Yan. Value-consistent representation learning for data-efficient reinforcement learning. In AAAI , 2023.

Gongjie Zhang, Zhipeng Luo, Yingchen Yu, Kaiwen Cui, and Shijian Lu. Accelerating DETR convergence via semantic-aligned matching. In CVPR , 2022.

Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and HeungYeung Shum. DINO: DETR with improved denoising anchor boxes for end-to-end object detection. In ICLR , 2023.

Haoyang Zhang, Ying Wang, Feras Dayoub, and Niko Sunderhauf. VarifocalNet: An iou-aware dense object detector. In CVPR , 2021.

Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In CVPR , 2020.

Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: Deformable transformers for end-to-end object detection. In ICLR , 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the ranking problem in object detection. Inspired by the misalignment between class and location scores in DTER models, the paper proposes to redesign the model architecture and the loss to modulate the rank information. Experiments show that , the proposed method can build upon the current strong DETR detectors such as DINO and H-DETR to further boost the performance by a notable margin (>1 mAP). The experiments also suggest the complementarity between the ranking model architecture and the ranking loss design.\n\nSTRENGTHS:\n+ The paper tackles an important problem in DETRs for object detection: the misalignment of the class scores and the localization scores.\n+ The paper is well written and easy to follow. The paper has provided informative statistical plots. For example, the density distribution for QRL and HWC, and the illustration of box scores with and without the proposed ranking mechanism.\n+ The experiments are solid and sound. The paper uses detailed ablation studies to justify the design choices of the proposed components. The method seems effective according to the experimental results, i.e., around +1% AP metric for DINO and H-DETR.\n\nWEAKNESSES:\n- The whole process seems a little engineering. E.g., the addition of learnable bias.\n- The main boost originates from the GIoU-aware loss, which has been investigated in Varifocal loss, Align-DETR and Stable-DETR (I am aware that the latter two are not formally published). This somehow hurts the novelty of the paper. A thorough discussion/comparison is expected.\n- The method seems complicated and includes many steps. For example the rank-driven model design seems not to invite as much performance gains as expected.\n\nQUESTIONS:\n- The title is too broad and has little meaning. I recommend to change a name including the technical contribution.\n- The addition of the RCH module even hurts the performance, especially for AP_L, it degrades 0.6 points. Please explain this phenomenon.\n- The learnable logit bias vectors S and the randomly initialized content query C are confusing to me. Are they tricks or can the author give an explanation (or illustrations with figures) for their practical functions? I think further ablations on the two learnable bias are required.\n- The result for DINO, R50, 12epochs in Table 1 is different from that in Table 3. Please check.\n\nLIMITATIONS:\nThe current solution is somewhat straightforward, and the topic can be deep investigated.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

comments: SUMMARY:
In this paper, the authors study the problem of object detection. To be specific, they introduce rank-awareness into transformer-based detectors both at the architecture-level and the loss/cost-level. 

After the rebuttal:

The authors have addressed my concerns about the comparison with other ranking-based solutions. Therefore, I recommend the paper to be accepted.

STRENGTHS:
1. Ranking is an important problem in object detection.

2. The paper introduces and incorporates many interesting ways of integrating ranking into transformer-based detectors.

3. Strong results compared to existing methods.

WEAKNESSES:
1. The paper is missing citations and comparison to significant ranking-based object detectors. To name a few, AP Loss, aLRP Loss, Rank & Sort Loss, Correlation Loss.

2. "Rank-adaptive Classification Head" => What does it have to do with ranking? This module just adds a learnable bias to classification scores, which do not depend on negatives or positives or localization qualities.

3. Some aspects require clarifications:

3.1. Eq 2: How does updating the positional encodings based on the sorted boxes disrupt positional information of the boxes?

3.2. Eq 2 & 3: Do you not need Sort() to be differentiable to pass gradients through?

3.3. "High-order Matching Cost" => Why high-order? What is high and what is order in this context?

4. Improvement over Varifocal loss (0.3% mAP) is insignificant. It is not clear what was missing in existing ranking-based loss functions.

5. Presumably there was not sufficient time to compare against [1, 21]. If you have performed a comparison in the meantime, can you please share the results?

6. It is a pity that there are no results on the LVIS dataset.

QUESTIONS:
Please see above.

LIMITATIONS:
None.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a DETR training method named Rank DETR that integrates multiple (four) rank-oriented designs, i.e., rank-adaptive classification head (RCH), query ranking layer (QRL), GIoU-aware classification loss (GCL), and high-order matching cost (HMC). Among these four components, the former two (RCH and QRL) are relatively novel, while the latter two (GCL and HMC) are close to already-existing IoU-based loss function and matching criterion (e.g., as in Stable DINO). Experimental results show that the proposed Rank DETR improves multiple DETR baselines. However, some recent methods (e.g., Stable DINO) with fewer components achieve comparable or even higher results.\n\nSTRENGTHS:\n- Two (out of four) major components, i.e., rank-adaptive classification head (RCH), query ranking layer (QRL) are novel.\n- Ablation experiments show that most components  bring considerable improvement (the improvement from RCH is relatively trivial) and integrating them brings further improvement.\n\nWEAKNESSES:\n- Two (out of four) major components, i.e.,  GIoU-aware classification loss (GCL), and high-order matching cost (HMC), share close insight, motivation and mechanism with recent methods, e.g., Stable DINO and Aligned DETR (though the detailed implementation is different). Moreover, the improvement brought by these two components is smaller than the similar ones in the competing methods. \n- More importantly, though the proposed Rank DETR adds two more components (RCH and QRL) based on IoU-related loss and matching (GCL and HMC), the overall results of Rank DETR is still lower than the recent methods that only has IoU-related loss (and matching), i.e., Stable DINO. For example, with DINO baseline, the proposed Rank DETR achieves 49.6 AP (12 epochs), while Stable DINO achieves 50.4 AP (12 epochs).\n- The definition of ranking in DETR is not clear enough. What criterion is the ranking based on? The IoU or the predicted confidence. This should be clearly pointed out at the very beginning. \n- The relation between the proposed method and IoU-related methods is not clear enough. Section 2 should explain their differences against IoU-related methods, as well as connections, in more details.\n\nQUESTIONS:\n- Eqn. 2 is confusing. What does Sort(A, B) perform? Sorting B and duplicate the sorting results onto A?\n- In Eqn. 2, positional embedding is irrelevant to the rank of a predicted results, but determined by the coordinates. Since the ranking (sorting) does not change the coordinates, how does the ranking impact the positional embedding? If there is no impact, why do you enforce ranking in Eqn. 2?\n\nLIMITATIONS:\nThe authors have  discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Rank-DETR for image object detection. The key contributions of Rank-DETR include (1) a rank-oriented architecture design, which comprises a rank-adaptive classification head and query rank layer to ensure lower FP and FN in predictions; (2) a rank-oriented loss and matching design, which introduces GIoU-aware classification loss and high-order matching cost to boost the AP under high IoU thresholds. Experiments on COCO demonstrate each component can contribute to the overall performance and align the design expedition, and RANK-DETR outperforms the current SOTA detector DINO.\n\nSTRENGTHS:\n1. The paper is well written, neatly written, clean, and easy to follow.\n2. The motivation of the paper is clear, 1. Rank-oriented Architecture Design: ensure lower FP and FN 2. Rank-oriented Matching Cost and Loss: boost the AP under high IoU thresholds. Experiments demonstrate the proposed component can align well with the design intent.\n3. The experiments are very solid, it compares two SOTA query-based detectors H-DETR and DINO, and outperform them. The ablation study thoroughly analyzes each component's effect and performance contribution to demonstrate its effectiveness.\n\nWEAKNESSES:\n1. Some module designs are not very novel. (1) Rank-adaptive Classification Head learns a class-aware and input-independent logits vector to model the class distribution and calibrate each query's score predictions. Such technique is well used in other fields like long-tailed classification/detection, and few-shot classification/detection. (2) GIoU-aware Classification Loss is very similar to Veri-Focal-Loss, except the modulating factor of positive classification loss is $t - \\hat{p}[l]$ instead of $t$, and such a minor modification leads to a slight performance boost, $~0.3mAP$. \n\n2. The Query Rank Layer design decouples content and positional query. The content query is constructed from MLP mapping of the fusing of a static query $C_l$ and output of the last decoder layer $O_l$. The positional query is constructed from PE encoding of ranked predicted boxes. This is a new design of query initialization, it will be interesting to compare it with other query initialization methods in DETR, Deformable-DETR, and DINO, but I can't find such comparisons.\n\n3. There is no analysis or comparison of parameter FLOPs and computation cost (training & testing speed).\n\nQUESTIONS:\nPlease refer to the weakness section.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}