openEuler/grounding_dino_swint_seq8_1280x720

Model Card for Grounding DINO SwinT (IB-Robot)

Grounding DINO with Swin-Tiny backbone and OGC (Open-vocabulary Grounding with Conditional) checkpoint, compiled as 12 Ascend 310P OM modules for text-conditioned zero-shot object detection. Packaged for the IB-Robot framework.

Repository Structure

• inference_manifest.json — deployment routing (schema v3, 12-role pipeline with 24 device links)
• assets/adapter.json — deployment adapter config
• assets/encoder_tgt.npy — pre-computed encoder target query
• assets/bert-base-uncased/vocab.txt — BERT vocabulary for text encoder
• artifacts/ascend_310p/ — 12 OM modules:
  • gdino_text.om — BERT text encoder
  • gdino_vision.om — SwinT vision backbone
  • gdino_vision_flatten.om — multi-scale feature flatten
  • gdino_encoder_layer0..5.om — 6 cross-modal encoder layers
  • gdino_proposal_cube.om — reference point proposal
  • gdino_decoder_cube.om — transformer decoder
  • gdino_head_origin.om — classification + bbox regression head

Deployment

Target	Backend	Runtime	Hardware
ascend_310p	ascend	ACL	Ascend 310P1

Execution pipeline (12 stages with device-pointer links between stages):

text + vision -> flatten -> encoder_0 -> encoder_1 -> ... -> encoder_5 -> proposal + decoder -> head

Inputs:

• image float32 [1,3,720,1280] NCHW
• input_ids int64 [1,8] (BERT tokenized text prompt)
• token_type_ids, position_ids, text_self_attention_masks int64 [1,8] / [1,8,8]
• text_token_mask int64 [1,8]
• encoder_tgt float32 [1,900,256] (pre-computed)

Outputs:

• pred_logits float32 [1,900,256] (class confidence logits)
• pred_boxes float32 [1,900,4] (cxcywh format, normalized)

Source Model

This bundle's OM artifacts were compiled from the upstream Grounding DINO checkpoint:

• Source checkpoint: groundingdino_swint_ogc.pth
• Source repository: github.com/IDEA-Research/GroundingDINO (v0.1.0-alpha)
• Text encoder: BERT base uncased

The OM artifacts were compiled from the torch checkpoint via ATC for Ascend 310P1 (fp16). See scripts/download_perception_models.sh for the download procedure.

Citation

@inproceedings{grounding_dino,
  title     = {Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection},
  author    = {Liu, Shilong and Zeng, Zhaoyang and Ren, Tianhe and Li, Feng and Zhang, Hongyang and Yang, Jian and Li, Chunyuan and Yang, Jianwei and Su, Hang and Zhu, Jun and Zhang, Lei},
  booktitle = {ECCV},
  year      = {2024}
}
@software{ib_robot,
  title   = {IB-Robot: Intelligence Boom Robot},
  url     = {https://gitcode.com/openeuler/IB_Robot},
  license = {Apache-2.0}
}