rusrakhimov/DensePoseFnL

Making DensePose fast and light

Checkpoints for Making DensePose fast and light (WACV 2021). Code, configs and training instructions: zetyquickly/DensePoseFnL.

📄 Paper | 💻 Code ]

Checkpoints

All three use the Mobile-Parsing-RCNN-B architecture (timm spnasnet_100 backbone, BiFPN, DeepLab DensePose head) and the s3x schedule. Each file is a detectron2 state dict ({"model": ...}) for the config of the same name in the code repository.

file	config	box AP	segm AP	dp. AP GPS	dp. AP GPSm
mobile_parsing_rcnn_b_s3x.pth	configs/mobile_parsing_rcnn_b_s3x.yaml	57.1	59.0	50.4	54.4
mobile_parsing_rcnn_b_wc2m_s3x.pth	configs/mobile_parsing_rcnn_b_wc2m_s3x.yaml	59.4	63.7	57.3	60.3
mobile_parsing_rcnn_b_cse_s3x.pth	configs/mobile_parsing_rcnn_b_cse_s3x.yaml	60.2	64.3	59.0	61.2

Numbers are on DensePose-COCO minival 2014 (1508 images), as listed in the code repository's model zoo. Re-evaluating these exact files with the repository's test command (Detectron2 bb96d0b, torch 1.10.1, timm 0.4.12) gives the same table to one decimal: 57.1 / 59.0 / 50.4 / 54.4, 59.4 / 63.7 / 57.3 / 60.3 and 60.2 / 64.3 / 59.0 / 61.2.

• WC2M adds confidence estimation for the regressed UV coordinates and for the coarse and fine segmentation (Sanakoyeu et al., 2020).
• CSE is a continuous surface embeddings model for humans (Neverova et al., 2020).
• The Mobile-Parsing-RCNN-B (s3x) weights are not the ones reported in the paper but perform similarly.

Usage

Install Detectron2 and DensePose at the commit pinned in the code repository's README, then:

git clone https://github.com/zetyquickly/DensePoseFnL && cd DensePoseFnL
hf download rusrakhimov/DensePoseFnL config.yaml mobile_parsing_rcnn_b_cse_s3x.pth --local-dir models
python train_net.py --config-file configs/mobile_parsing_rcnn_b_cse_s3x.yaml --eval-only \
    MODEL.WEIGHTS models/mobile_parsing_rcnn_b_cse_s3x.pth
python apply_net.py show configs/mobile_parsing_rcnn_b_cse_s3x.yaml models/mobile_parsing_rcnn_b_cse_s3x.pth \
    image.jpg dp_vertex,bbox --output out.png

SHA256SUMS lists the checksum of each file; config.yaml maps each file to its config and scores.

Upstream terms

The weights are released under Apache-2.0. They were trained on DensePose-COCO, which was released with the original DensePose repository under CC BY-NC 4.0, on COCO images. The CSE checkpoint also stores the 27554 x 256 vertex basis of the SMPL mesh (roi_heads.embedder.embedder_smpl_27554.features, from DensePose's phi_smpl_27554_256.pkl); SMPL has its own license. Check those terms for your use.

Citation

@inproceedings{rakhimov2021making,
  title={Making DensePose fast and light},
  author={Rakhimov, Ruslan and Bogomolov, Emil and Notchenko, Alexandr and Mao, Fung and Artemov, Alexey and Zorin, Denis and Burnaev, Evgeny},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={1869--1877},
  year={2021}
}