projects/deformable_detr/README.md

Deformable DETR: Deformable Transformers for End-to-End Object Detection

Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai

[arXiv] [BibTeX]

Pretrained Weights

Here we provide the pretrained Deformable-DETR weights based on detrex.

Name	Backbone	Pretrain	Epochs	box AP	download
Deformable-DETR-R50 + Box-Refinement	R-50	IN1k	50	47.0	model
Deformable-DETR-R50 + Box-Refinement + Two-Stage	R-50	IN1k	50	48.2	model

All the models are trained using 8 GPUs with total batch size equals to 16. We've observed that the result of deformable-two-stage model trained using 8 GPUs may be slightly lower than 16 GPUs with 32 total batch size.

Notable facts and caveats: The training settings are different from the original repo. Most of the training settings are following DINO. As we set lr=1e-5 for backbone and 1e-4 for the other modules. The original implementation sets lr to 2e-5 for backbone, sampling_offsets and reference_points, and 2e-4 for other modules. And we used top-300 confidence boxes for testing, which may get a slightly better results on COCO evaluation. And we only freeze the stem layer in ResNet backbone by setting freeze_at=1 in config.

Converted Weights

Name	Backbone	Pretrain	Epochs	box AP	download
Deformable-DETR-R50	R-50	IN1k	50	44.59	model
Deformable-DETR-R50 + Box-Refinement	R-50	IN1k	50	46.28	model
Deformable-DETR-R50 + Box-Refinement + Two-Stage	R-50	IN1k	50	47.09	model

Note: Here we borrowed the pretrained weight from Deformable-DETR official repo. And all the pretrained weights are tested using top-300 confidence boxes (top-100 in original repo) which may brings about 0.2 AP gain on COCO evaluation.

Training

All configs can be trained with:

cd detrex
python tools/train_net.py --config-file projects/deformable_detr/configs/path/to/config.py --num-gpus 8

By default, we use 8 GPUs with total batch size as 16 for training.

Evaluation

Model evaluation can be done as follows:

cd detrex
python tools/train_net.py --config-file projects/deformable_detr/configs/path/to/config.py --eval-only train.init_checkpoint=/path/to/model_checkpoint

Citing Deformable-DETR

@article{zhu2020deformable,
  title={Deformable DETR: Deformable Transformers for End-to-End Object Detection},
  author={Zhu, Xizhou and Su, Weijie and Lu, Lewei and Li, Bin and Wang, Xiaogang and Dai, Jifeng},
  journal={arXiv preprint arXiv:2010.04159},
  year={2020}
}