Recently, plenty of work has tried to introduce transformers into computer vision tasks, with good results. Unlike classic convolution networks, which extract features within a local receptive field, transformers can adaptively aggregate similar features from a global view using self-attention mechanism. For object detection, Feature Pyramid Network (FPN) proposes feature interaction across layers and proves its extremely importance . However, its interaction is still in a local manner, which leaves a lot of room for improvement. Since transformer was originally designed for NLP tasks, adapting processing subject directly from text to image will cause unaffordable computation and space overhead. In this paper, we utilize a linearized attention function to overcome above problems and build a novel architecture , named Content-Augmented Feature Pyramid Network (CA-FPN) , which proposes a global content extraction module and deeply combines with FPN through light linear transformers. What's more, light transformers can further make the application of multi-head attention mechanism easier. Most importantly, our CA-FPN can be readily plugged into existing FPN-based models. Extensive experiments on the challenging COCO object detection dataset demonstrated that our CA-FPN significantly outperforms competitive baselines without bells and whistles. Code will be made publicly available .
As one of the prevalent components, Feature Pyramid Network (FPN) is widely used in the current object detection models to improve the performance of multi-scale detection . However, its interaction is still in a local and lossy manner, thus limiting the representation power. In this paper, to simulate a global view of human vision in object detection and address the inherent defects of interaction mode in FPN, we construct a novel architecture termed Content-Augmented Feature Pyramid Network (CA-FPN) . Unlike the vanilla FPN, which fuses features within a local receptive field, CA-FPN can adaptively aggregate similar features from a global view. It is equipped with a global content extraction module and light linear spatial transformers. The former allows to extract multi-scale context information and the latter can deeply combine the global content extraction module with the vanilla FPN using the linearized attention function, which is designed to reduce model complexity. Furthermore, CA-FPN can be readily plugged into existing FPN-based models. Extensive experiments on the challenging COCO and PASCAL VOC object detection datasets demonstrated that our CA-FPN significantly outperforms competitive FPN-based detectors without bells and whistles. When plugging CA-FPN into Cascade R-CNN framework built upon a standard ResNet-50 backbone, our method can achieve 44.8 AP on COCO mini-val. Its performance surpasses the previous state-of-the-art by 1.5 AP, demonstrating the potentiality of application .
R
Recently, plenty of work has tried to introduce transformers into computer vision tasks, with good results. Unlike classic convolution networks, which extract features within a local receptive field, transformers can adaptively aggregate similar features from a global view using self-attention mechanism. For object detection,
As one of the prevalent components,
R
proposes feature interaction across layers and proves its extremely importance
is widely used in the current object detection models to improve the performance of multi-scale detection
R
manner, which leaves a lot of room for improvement. Since transformer was originally designed for NLP tasks, adapting processing subject directly from text to image will cause unaffordable computation and space overhead.
and lossy manner, thus limiting the representation power.
R
we utilize a linearized attention function to overcome above problems and build
to simulate a global view of human vision in object detection and address the inherent defects of interaction mode in FPN, we construct
R
, named
termed
R
, which proposes a global
. Unlike the vanilla FPN, which fuses features within a local receptive field, CA-FPN can adaptively aggregate similar features from a global view. It is equipped with a global
R
deeply combines with FPN through light linear transformers. What's more, light transformers can further make the application of multi-head attention mechanism easier. Most importantly, our
light linear spatial transformers. The former allows to extract multi-scale context information and the latter can deeply combine the global content extraction module with the vanilla FPN using the linearized attention function, which is designed to reduce model complexity. Furthermore,
R
object detection dataset
and PASCAL VOC object detection datasets
R
baselines
FPN-based detectors
R
Code will be made publicly available
When plugging CA-FPN into Cascade R-CNN framework built upon a standard ResNet-50 backbone, our method can achieve 44.8 AP on COCO mini-val. Its performance surpasses the previous state-of-the-art by 1.5 AP, demonstrating the potentiality of application