SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhengdong Hu $^{1,2*}$ , Yifan Sun $^{2}$ , Jingdong Wang $^{2}$ , Yi Yang $^{3}$
<sup>1</sup>ReLER, AAII, University of Technology Sydney
2 Baidu Inc.
$^{3}$  CCAI, College of Computer Science and Technology, Zhejiang University
huzhengdongcs@gmail.com,sunyf15@tsinghua.org.cn,
wangjingdong@baidu.com, yangyics@zju.edu.cn
This paper reveals a characteristic of DEtection Transformer (DETR) that negatively impacts its training efficacy, i.e., the cross-attention and self-attention layers in DETR decoder have opposing impacts on the object queries (though both impacts are important). Specifically, we observe the cross-attention tends to gather multiple queries around the same object, while the self-attention disperses these queries far away. To improve the training efficacy, we propose a Divide-And-Conquer DETR (DAC-DETR) that separates out the cross-attention to avoid these competing objectives. During training, DAC-DETR employs an auxiliary decoder that focuses on learning the cross-attention layers. The auxiliary decoder, while sharing all the other parameters, has NO self-attention layers and employs one-to-many label assignment to improve the gathering effect. Experiments show that DAC-DETR brings remarkable improvement over popular DETRs. For example, under the 12 epochs training scheme on MS-COCO, DAC-DETR improves Deformable DETR (ResNet50) by  $+3.4\mathrm{AP}$  and achieves 50.9 (ResNet-50) / 58.1 AP (Swin-Large) based on some popular methods (i.e., DINO and an IoU-related loss). Our code will be made available at https://github.com/huzhengdongcs/DAC-DETR.
Though DEetection Transformers (DETRs) have already achieved great progress, their low training efficacy problem remains a critical challenge. The research community has been paying great efforts to locate the causes of this problem and to improve the training efficacy. For example, DN-DETR [15] and DINO [42] find the one-to-one matching is prone to inconsistent optimization goals. In response, they propose to accelerate DETR training with auxiliary denoising queries. Hybrid DETR [12] and Group DETR [5] attribute the reason to the rare positive training queries and correspondingly increase positive queries through additional one-to-many matching. Some other works [20, 1] notice the misalignment between the predicted probability and position. They suppress such misalignment through IoU-related loss functions and achieve improvement.
This paper, from another perspective, investigates this problem by rethinking the cooperation between cross-attention and self-attention layers in the DETR decoder. We find during their cooperation, these two attention types actually have some opposing impacts on the object queries. These two impacts are both critical for DETR (as explained later), but their opposition impairs the training efficacy. Specifically, we observe the cross-attention layers in DETR decoder tends to gather multiple queries around a single object, while the self-attention disperses these queries far away from each other. The above "gather  $\leftrightarrow$  disperse" phenomenon concerns not only the positional relationship (1st row in Fig. 1), but also the feature distance (2nd row in Fig. 1 visualized by t-SNE [36]):
*Zhengdong Hu makes his part of work during internship in Baidu Inc.
(a) before decoder
(b) decoder w/o self-attention
(c) decoder w/ self-attention
Figure 1: Comparison between (a) initial queries before the DETR decoder, (b) queries output from a decoder variant without self-attention layers and (c) queries output from the normal DETR decoder. We visualize their position (1st row) and feature distribution (2nd row). For clarity, we randomly choose only 4 queries with relatively large IoU. From (a)  $\rightarrow$  (b), multiple queries are pulled towards a same object, showing the gathering effect of cross-attention layers. From (b)  $\rightarrow$  (c), adding the self-attention layers disperses these queries except a single one ( $q_{1}$ ), which approaches even closer towards their center. In spite of the conflicting objectives, we find that these two effects are both important for DETR.
- Cross-attention layers tend to gather multiple queries around the same object. Given an already-trained Deformable DETR [45], we remove all the self-attention layers in its decoder. This removal compromises DETR to duplicate detections. As illustrated in Fig. 1 (b), multiple queries locate a same object ("bear") with relatively large Intersection-over-Union (IoU). Compared with their initial states (before the decoder) in Fig. 1 (a), they become closer towards each other, regarding both their position (1st row) and feature distance (2nd row).
- Self-attention layers disperse these queries from each other. In Fig. 1 (c), we restore the original deformable DETR. Correspondingly, the queries are dispersed from each other regarding both position (1st row) and feature distance (2nd row). Due to this dispersion, most queries become farther to their original center point, except that a single query (with the highest classification score) approaches even closer. This phenomenon explains how DETR makes non-duplicate detection, and is consistent with the consensus [23, 3, 41, 25], i.e., self-attention layers play a critical role in removing duplicates.
In spite of their opposition, these two effects are both critical for DETR. The gathering effect allows the queries to aggregate feature from the nearby object. The dispersing effect allows the self-attention layers to remove duplicate detection. However, learning the stacked self-attention and cross-attention layers within such "gather  $\leftrightarrow$  disperse" opposition is difficult. To improve the training efficacy, we propose a Divide-and-Conquer DETR (DAC-DETR) that divides the cross-attention out from this opposition for better conquering. We explain the key points of DAC-DETR in more details as below:
1) "Divide": DAC-DETR employs an auxiliary decoder that focuses on learning the cross-attention layers. The auxiliary decoder has NO self-attention layers and shares all the other parameters (i.e., the cross-attention layers and the feed-forward networks) with the original decoder (O-Decoder). We name the auxiliary decoder as C-Decoder to highlight its focus on the cross-attention.
2) "Conquer": During training, DAC-DETR feeds all the queries into two decoders in parallel. O-Decoder uses the one-to-one matching to learn non-duplicate detection. In contrast, C-Decoder uses the one-to-many matching that assigns each object with multiple positive queries. C-Decoder has no self-attention layers and is prone to duplicate detection results. Therefore, using one-to-many label assignment is a natural choice and benefits learning the gathering effect with more positive queries. During inference, DAC-DETR discards C-Decoder and maintains the baseline efficiency.
Through the above "divide-and-conquer", DAC-DETR defuses the opposition when learning the "gathering" and "dispersing" effects which are both important for DETR. Mechanism analysis in Section 3.3 shows that due to the enhanced feature aggregation ability, DAC-DETR improves both the quantity and quality of intermediate queries that are gathered to each ground-truth object. Based on these improved queries, DETR increases the detection accuracy (after removing duplicate predictions).
We conduct extensive experiments to validate the effectiveness of DAC-DETR and empirically show remarkable improvement over various DETRs. For example, based on a popular baseline, i.e., ResNet-50 Deformable DETR [45], DAC-DETR brings  $+3.4$  AP improvement and achieves 47.1 AP on MS-COCO within  $12(1\times)$  training epochs. On some more recent state-of-the-art methods (that usually integrate a battery of good practices), DAC-DETR still gains consistent and complementary benefits. For example, under the  $1\times$  learning scheme on MS-COCO, DAC-DETR improves ResNet-50 DINO to 50.0  $(+1.0)$  AP. Combining an IoU-related Loss in Align DETR [1], our method further achieves 50.9 AP (ResNet-50) / 58.1 AP (Swin-Large), surpassing Align DETR itself by  $+0.7$  AP. Moreover, the achieved results (50.9 and 58.1 AP) are also higher than Stable DETR [20] (50.4 and 57.7 AP), which adopts even more good practices (i.e., stable matching and memory fusion).
Our main contributions are summarized as follows: First, we reveal the "gather  $\leftrightarrow$  disperse" opposition between cross-attention and self-attention layers in DETR decoder as a reason for its low training efficacy. Second, we propose to divide the cross-attention layers out from this opposition for better conquering the DETR training, yielding the so-called Divide-and-Conquer DETR (DAC-DETR). Third, we empirically show that DAC-DETR brings remarkable improvement over popular DETR baselines and performs favorably against most recent state-of-the-art methods.
DETR-like object detectors. The original Detection Transformer (DETR) [37, 3] is featured for its end-to-end detection without any hand-crafted modules (e.g., Non-maximum Suppression). Since DETR [3] suffers from low training convergence, many efforts [45, 33, 23, 42, 38, 6, 8, 34, 43, 39, 32, 16] have been paid to improve the training efficacy. Deformable DETR [45] proposes to replace traditional attention modules with deformable attention and improves the training convergence. Conditional DETR [23] performs the conditional spatial query for decoder multi-head cross-attention. DAB-DETR [19] utilizes box coordinates as priors and dynamically updates anchor boxes layer-by-layer. DN-DETR [15] introduces denoising part to stabilize the bipartial matching, and DINO [42] further adds contrastive learning to enhance the performance. Some works [20, 1] propose IoU-related loss functions to align the predicted probability score against position. This paper, from another perspective, finds a opposing impact between the cross-attention and self-attention layers and proposes to divide the attention layers during training.
Attention layers in DETR decoder. The cross-attention and self-attention layers are both essential for DETRs. The former allows the object queries to aggregate information from the image features, while the latter facilitates query-to-query interaction to remove duplicate detection [23, 45, 3, 2, 9, 40, 31]. Our observation in this paper is consistent with the above consensus and goes a step forward: the cross-attention layers tend to gather multiple queries towards each object, while the self-attention disperses them far away. By separating the cross-attention layers out from this opposition, the proposed DAC-DETR improves DETR training efficacy.
One-to-many label assignment. One-to-many label assignment is widely utilized in CNN-based detectors [29, 27, 28, 11, 4, 35, 13]. these methods produce duplicate predictions on each object and uses hand-crafted non-maximum suppression (NMS) [24] for post-processing. As the DETRs prevail, some recent literature [5, 12, 26] explore one-to-many label assignment for DETRs and demonstrate benefit. Group DETR [5] employs multiple parallel object queries, which jointly contribute multiple positive queries for each object. Hybrid DETR [12] adds an auxiliary query branch that performs one-to-many label assignment.
DAC-DETR uses the one-to-many label assignment for the auxiliary C-Decoder, because C-Decoder has no self-attention layers that help removing duplicate predictions. Therefore, DAC-DETR also gains the benefit from one-to-many assignment. That being said, the benefit from one-to-many assignment is not our key contribution but a natural gain. While a co-current work Hybrid DETR [12] already shows that adding a one-to-many decoder improves DETR, our method shows that removing
Figure 2: Overview of DAC-DETR. Left: for the backbone and encoder, DAC-DETR follows the standard DETR pipeline: it uses a backbone (e.g., ResNet-50) to extract feature, splits the feature into patch tokens and then feeds the tokens into the transformer encoder. The tokens output from the encoder are fed into the decoders for object detection. Right: the main difference between DAC-DETR and previous DETRs is that DAC-DETR has two parallel decoders, i.e., an O-Decoder (standard decoder) and an auxiliary C-Decoder (NO self-attention layers). These two decoders share most parameters and the same object queries, except that C-Decoder has no self-attention layers. O-Decoder is supervised with one-to-one matching to learn non-duplicate detection. In contrast, Decoder uses one-to-many matching to assign each ground-truth object with multiple positive queries. During inference, DAC-DETR only uses the O-Decoder and maintains the baseline efficiency.
the self-attention in the one-to-many decoder is better (e.g., +0.6 AP on MS-COCO in 12 training epochs, as detailed in Table 6).
3.1 Overview
Preliminaries on DETRs. A DETR-style detector typically consists of a backbone (e.g., ResNet [10], Swin Transformer [21]), a transformer encoder and a transformer decoder. Given an input image, the DETR-style detector first feeds it into the backbone to extract image features, splits the features into patch tokens and then enhances the patch tokens by the transformer encoder. The enhanced tokens from the encoder are  $\mathbf{Z} = \{z_1,\dots,z_m\}$ . The transformer decoder takes multiple object queries  $\mathbf{Q} = \{q_1,\dots,q_n\}$  as its input. The decoder has  $L$  stacked decoder blocks. In each decoder block, the object queries  $\mathbf{Q}$  sequentially undergo a self-attention layer, a cross-attention layer, and a feed-forward network (FFN). Specifically, the cross-attention layer makes  $\mathbf{Q}$  interact with the patch tokens  $\mathbf{Z}$ , so as to aggregate features for locating and identifying the objects. Afterwards, the FFN transforms  $\mathbf{Q}$  into the output embeddings, which make final prediction through a classification and regression head.
DAC-DETR follows the standard DETR pipeline for the backbone and transformer encoder and is featured for the "Divide-and-Conquer" decoders. As illustrated in Fig. 2, DAC-DETR has two decoders, i.e., 1) an O-Decoder which is the same as the baseline DETR decoder, and 2) a C-Decoder that divides the cross-attention layers from the "gather  $\leftrightarrow$  disperse" for better learning. C-Decoder removes all the self-attention layers and shares all the other decoder parameters (cross-attention, FFN), object queries and the prediction head with the O-Decoder. Section 3.2 elaborates on these two decoders regarding their architecture, training and inference. Section 3.3 looks into the mechanism,
i.e., how DAC-DETR impacts the learning of the DETR decoder. It shows that DAC-DETR improves both the quantity (number) and the quality (larger IoU) of queries that attend to each object.
3.2 Divide-and-Conquer Decoders
DAC-DETR constructs two parallel decoders, i.e., a canonical DETR decoder (O-Decoder) and a decoder without self-attention layers (C-Decoder), as shown in Fig. 2. The initial object queries  $\mathbf{Q}$  are fed into the first block of O-Decoder and C-Decoder in parallel. Since the object queries undergo different transformations in these two decoders, they are denoted as  $\mathbf{Q}_{\mathrm{o}}^{l}$  (queries in O-Decoder) and  $\mathbf{Q}_{\mathrm{c}}^{l}$  (queries in C-Decoder), where the superscript  $l(l = 0,1,\dots ,5)$  indicates the index of the decoder block. We have  $\mathbf{Q}_{\mathrm{o}}^{0} = \mathbf{Q}_{\mathrm{c}}^{0} = \mathbf{Q}$ .
- O-Decoder: a canonical DETR-decoder revisit. O-Decoder is a canonical DETR decoder with each block consisting of a self-attention layer, a cross-attention layer and a FFN. Specifically, the  $l$ -th decoder block first updates all the queries through self-attention, which is formulated as:
$$
\widetilde {\mathbf {Q}} _ {0} ^ {l} = \mathbf {Q} _ {0} ^ {l} + \operatorname {A t t} _ {\text {s e l f}} \left(\operatorname {Q u e} \left(\mathbf {Q} _ {0} ^ {l}\right), \operatorname {K e y} \left(\mathbf {Q} _ {0} ^ {l}\right), \operatorname {V a l} \left(\mathbf {Q} _ {0} ^ {l}\right)\right), \tag {1}
$$
where  $\text{Que}$ ,  $\text{Key}$ ,  $\text{Val}$  are the projections to derive the query,  $\text{key}$  and  $\text{value}$  embedding from tokens  $\mathbf{Q}_0^l$ , respectively. We omit the description of "Layer norm" and "Dropout" in all equations for better clarity.
Given the queries  $\mathbf{Q}_{\mathrm{o}}^{l}$  output from the preceding self-attention layer, the cross-attention layer further updates them through interaction with the feature embedding  $\mathbf{Z}$ , which is formulated as:
$$
\mathbf {Q} _ {\mathrm {O}} ^ {l + 1} = \operatorname {F F N} \left(\widetilde {\mathbf {Q}} _ {\mathrm {O}} ^ {l} + \operatorname {A t t} _ {\text {c r o s s}} \left(\operatorname {Q u e} \left(\widetilde {\mathbf {Q}} _ {\mathrm {O}} ^ {l}\right), \operatorname {K e y} (\mathbf {Z}), \operatorname {V a l} (\mathbf {Z})\right)\right), \tag {2}
$$
where  $\text{Que}$ ,  $\text{Key}$ ,  $\text{Val}$  are the projections and are parameterized differently from those in the self-attention layer (Eqn. 1), though sharing the same symbols.
Finally, the output embeddings  $\mathbf{Q}_0^{l + 1}$  are fed into a predictor head to make label and position predictions. The predictions are supervised with the one-to-one matching and  $\mathcal{L}_{\mathrm{O - decoder}}$  , which is the standard Hungarian loss [3] as in the original DETR.
- C-decoder removes all the self-attention layer in each decoder block and shares all the other parameters with O-decoder. Therefore, in C-Decoder, the queries undergo only cross-attention with the feature embeddings  $\mathbf{Z}$  and a following FFN transformation, which is formulated as:
$$
\mathbf {Q} _ {\mathrm {C}} ^ {l + 1} = \operatorname {F F N} \left(\mathbf {Q} _ {\mathrm {C}} ^ {l} + \operatorname {A t t} _ {\text {c r o s s}} (\operatorname {Q u e} \left(\mathbf {Q} _ {\mathrm {C}} ^ {l}\right), \operatorname {K e y} (\mathbf {Z}), \operatorname {V a l} (\mathbf {Z}))\right), \tag {3}
$$
where all the model parameters are shared with the O-Decoder (Eqn. 2).
In parallel to the prediction from O-Decoder, the output embeddings from C-Decoder  $\mathbf{Q}_{\mathrm{C}}^{l + 1}$  are fed into the shared predictor head. The corresponding label and position prediction from a query  $q\in \mathbf{Q}$  are  $[p_{(q)},b_{(q)}]$ .
Different from the bipartite matching strategy in O-Decoder, C-Decoder employs one-to-many label assignment for supervision [26, 44], i.e., assigning each ground-truth object with multiple queries. Specifically, given an object with ground-truth annotation  $y = [\hat{c},\hat{b}]$  ( $\hat{c}$  and  $\hat{b}$  are the class and bounding box, respectively), we measure its matching scores with the prediction from all the queries by:
$$
\mathbf {M} = \left\{p _ {(q)} (\hat {c}) + \mathrm {I} \circ \mathrm {U} <   b _ {(q)}, \hat {b} > \right\} \quad \forall q \in \mathbf {Q}, \tag {4}
$$
where  $p_{(q)}(\hat{c})$  denotes the predicted label score on class  $\hat{c}, <, >$  denotes the IoU operation between predicted box and ground truth  $\hat{b}$ .
Given the matching scores  $\mathbf{M}$  for the ground-truth object, we select multiple positive predictions for it by two criteria: 1) the matching score should be larger than a threshold  $t$  and 2) the matching score should be among the top-k scores in  $\mathbf{M}$ . There two criteria are required simultaneously and the second one is to suppress the label imbalance regarding different objects. All the other predictions are assigned with negative labels for this object. With the assigned labels, we supervise the predictions from C-Decoder with  $\mathcal{L}_{\mathrm{C - decoder}}$ , which is a standard detection loss function consisting of classification [18] and regression loss [30].
Figure 3: The averaged number of queries that each ground-truth object gathers on the validation set of MS-COCO. Compared with the Deforamble-DETR baseline, DAC-DETR 1) has more queries for each object, and 2) improves the quality of the closest queries. Results for more layers are provided in the supplementary. y-axis denotes "avg number of queries / object".
During training,  $\mathcal{L}_{\mathrm{O - decoder}}$  and  $\mathcal{L}_{\mathrm{C - decoder}}$  are summed up. During inference, DAC-DETR discards C-Decoder and maintains the same pipeline and efficiency as the baseline.
Discussion. Using the one-to-many label assignment for supervising C-Decoder is a natural choice. It is because C-Decoder has no self-attention layers (which are critical for removing duplicate detections) and is thus prone to multiple detections for each single object. If C-Decoder employs the one-to-one label assignment, DAC-DETR will bring no more improvement but actually compromises the baseline accuracy (Section 4.3).
3.3 Mechanism Analysis
We investigate how the proposed divide-and-conquer method benefits DETR with focus on the gathering effects of the cross-attention layers. Specifically, we are interested in: how many queries can each ground-truth object gather in DAC-DETR and in the baseline? To this end, we count the averaged number of queries that have large affinity (according to the matching score in Eqn. 4) with each object. The queries with high matching scores are important: they aggregate abundant information from a corresponding object and thus serve as the proposals which are potential to predict the object. Based on these highly-matched queries (proposals), the self-attention layer removes redundant queries and gives the final non-duplicate prediction.
The comparison between DAC-DETR and the Deformable-DETR baseline is in Fig. 3. We list the results for the last decoder layer and leave the results for more layers to the supplementary. From Fig. 3, we draw two important observations as below:
- Remark-1: DAC-DETR increases the number of queries gathered for each object. Comparing DAC-DETR and its baseline under various matching score threshold (i.e.,  $t = 0.6, 0.7, 0.8$ ), we observe that DAC-DETR consistently gathers more queries for each single object. Please note that the total number of queries is the same for DAC-DETR and the baseline. We thus infer that DAC-DETR makes higher utilization of the object queries.
- Remark-2: DAC-DETR improves the quality of the best queries for each object. Focusing on the comparison under large threshold (e.g.,  $t = 0.8$ ), we find the baseline has rare high-quality queries, while DAC-DETR has about 1 high-quality query for each object. We recall that the matching score is composed of IoU and classification score. This suggests that some queries in DAC-DETR either have good IoU or high confidence to the ground-truth object.
Combining the above two remarks, we conclude that DAC-DETR improves both the quantity and quality of the queries for each object. This improvement is the direct reason how DAC-DETR increases the detection accuracy (Section 4).
Dataset and backbone. We evaluate the proposed DAC-DETR on COCO 2017 [17] detection dataset. Following the common practices, we evaluate the performance on validation dataset(5k images)