  Although convolutional networks (ConvNets) have enjoyed great success in
computer vision (CV), it suffers from capturing global information crucial to
dense prediction tasks such as object detection and segmentation. In this work,
we innovatively propose ConTNet (ConvolutionTransformer Network), combining
transformer with ConvNet architectures to provide large receptive fields.
Unlike the recently-proposed transformer-based models (e.g., ViT, DeiT) that
are sensitive to hyper-parameters and extremely dependent on a pile of data
augmentations when trained from scratch on a midsize dataset (e.g.,
ImageNet1k), ConTNet can be optimized like normal ConvNets (e.g., ResNet) and
preserve an outstanding robustness. It is also worth pointing that, given
identical strong data augmentations, the performance improvement of ConTNet is
more remarkable than that of ResNet. We present its superiority and
effectiveness on image classification and downstream tasks. For example, our
ConTNet achieves 81.8% top-1 accuracy on ImageNet which is the same as DeiT-B
with less than 40% computational complexity. ConTNet-M also outperforms
ResNet50 as the backbone of both Faster-RCNN (by 2.6%) and Mask-RCNN (by 3.2%)
on COCO2017 dataset. We hope that ConTNet could serve as a useful backbone for
CV tasks and bring new ideas for model design
