  The recently proposed Detection Transformer (DETR) model successfully applies
Transformer to objects detection and achieves comparable performance with
two-stage object detection frameworks, such as Faster-RCNN. However, DETR
suffers from its slow convergence. Training DETR \cite{carion2020end} from
scratch needs 500 epochs to achieve a high accuracy. To accelerate its
convergence, we propose a simple yet effective scheme for improving the DETR
framework, namely Spatially Modulated Co-Attention (SMCA) mechanism. The core
idea of SMCA is to conduct regression-aware co-attention in DETR by
constraining co-attention responses to be high near initially estimated
bounding box locations. Our proposed SMCA increases DETR's convergence speed by
replacing the original co-attention mechanism in the decoder while keeping
other operations in DETR unchanged. Furthermore, by integrating multi-head and
scale-selection attention designs into SMCA, our fully-fledged SMCA can achieve
better performance compared to DETR with a dilated convolution-based backbone
(45.6 mAP at 108 epochs vs. 43.3 mAP at 500 epochs). We perform extensive
ablation studies on COCO dataset to validate the effectiveness of the proposed
SMCA.
