executorch-community/D-FINE-Small-COCO-ExecuTorch-WebGPU

D-FINE small object detection for ExecuTorch WebGPU

This repository contains an ExecuTorch conversion of ustc-community/dfine-small-coco for the ExecuTorch WebGPU backend, so the model runs entirely in a web browser on the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on executorch.ai).

• Base model: ustc-community/dfine-small-coco
• Format: ExecuTorch .pte programs delegated to the WebGPU backend
• Quantization: fp32 (as exported)
• Total size: 40.3 MB

Note: The weights are the Hugging Face transformers port of the authors' checkpoint; every tensor was checked to be bit-identical to the official dfine_s_coco.pth published by the first author (see Provenance and export/verify_official.py).

About the original model

The small COCO model of D-FINE, a real-time DETR-style object detector that refines boxes as probability distributions.

• Developed by: Yansong Peng et al. (USTC)
• Paper: D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement
• 10.3M parameters
• Detects the 80 COCO object classes
• No non-maximum suppression needed

For training data, evaluation results and full details, see the original model card: ustc-community/dfine-small-coco.

Intended use and limitations

Detecting everyday objects in photos, video and camera frames.

Only knows the 80 COCO classes. Small, unusual or heavily occluded objects are often missed, and confidence scores are not calibrated probabilities.

What changed in this conversion

• Graph rewritten with output-preserving substitutions so it runs as a single WebGPU delegate (checked against upstream on a real image: boxes within 0.001 px)
• Fixed 640x640 input; the 300 query outputs are filtered by score outside the program

Inputs and outputs

• Input [1, 3, 640, 640] float32 RGB in [0, 1] (x / 255, no mean/std); letterboxed or plain-resized
• Output [1, 84, 300] float32: rows 0-3 are box center x, center y, width, height in input pixels; rows 4-83 are sigmoid scores for the 80 COCO classes

Files

File	Size	SHA-256
dfine_s_coco_webgpu.pte	40.3 MB	96cc65f615dc2c79…

Full checksums are in manifest.json. The playground verifies them after download.

Running it

These programs were exported with the ExecuTorch WebGPU backend (backends/webgpu) and validated with the WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on executorch.ai). That runtime build is not published yet; until it is, the playground is the supported way to run them. It downloads these files once, caches them in the browser, and runs everything locally.

Requirements: a browser with WebGPU enabled.

Provenance

• Weights: ustc-community/dfine-small-coco @ f79e65b5fbb33ceb9d3ebba042955d7410c608f8, file model.safetensors, sha256 a144baff3a27bfd538fe37b11fa8d4fca5744b8c054b5301e7c699a1379cd18f (from a community mirror of the official release)
• Checked against the official checkpoint: Peterande/D-FINE @ 19214e91f5ce4c1e609022c38b1fc953c47404e7, file dfine_s_coco.pth (sha256 48a6c8cc43eb57186843f752e2e8461ddd3326e0d3c575e71e6e960844683e89): all 764 tensors bit-identical (verify_official.py)
• Exported with: executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git 3b6068392324)
• Reproduce: python recipes/detector-dfine-small.py --out out/detector-dfine-small --image photo.jpg (recipe in export/)
• Re-running the recipe produces byte-identical files.

Citation

If you use this model, please cite the original work:

@misc{peng2024dfine,
  title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},
  year={2024},
  eprint={2410.13842},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

License

This is a derivative of the upstream model and is distributed under the same terms (apache-2.0); see LICENSE. Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc.