DPT-Hybrid-MiDaS: Vision-transformer hybrid for monocular relative depth estimation

DPT-Hybrid-MiDaS is the MiDaS v3.0 dense prediction transformer released by Intel. A BiT-R50 convolutional stem feeds a ViT-Base encoder, and a DPT convolutional decoder progressively fuses the transformer's intermediate tokens back into a full-resolution depth map. It was trained on a mix of six depth datasets (MIX-6) for zero-shot transfer, so it predicts relative inverse depth up to an unknown global scale and shift rather than metric distance. This recipe wraps the HuggingFace checkpoint (Intel/dpt-hybrid-midas, 122M parameters) at its native 384x384 input resolution.

This is based on the implementation of DPT-Hybrid-MiDaS found here. This is a standalone recipe compatible with the Qualcomm® AI Hub Models CLI — it can be compiled and evaluated on real Snapdragon devices via Qualcomm® AI Hub Workbench.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Setup

1. Install the package

Install the base package, fetch this recipe from Hugging Face, then use the qai-hub-models CLI to install the recipe's dependencies:

# NOTE: 3.10 <= PYTHON_VERSION < 3.14 is supported.
pip install qai-hub-models
qai-hub-models register ashwmurt/dpt-hybrid-midas
qai-hub-models install dpt_hybrid_midas

register downloads the recipe and names it dpt_hybrid_midas, which is how every command below refers to it.

2. Configure Qualcomm® AI Hub Workbench

Sign-in to Qualcomm® AI Hub Workbench with your Qualcomm® ID. Once signed in navigate to Account -> Settings -> API Token.

With this API token, you can configure your client to run models on the cloud hosted devices.

qai-hub configure --api_token API_TOKEN

Navigate to docs for more information.

Run CLI Demo

Run the following simple CLI demo to verify the model is working end to end:

qai-hub-models demo dpt_hybrid_midas

More details on the CLI tool can be found with the --help option. See demo.py for sample usage of the model including pre/post processing scripts.

By default, the demo will run locally in PyTorch. Pass --eval-mode on-device to run the model on a cloud-hosted target device.

Export for on-device deployment

To run the model on Qualcomm® devices, you must export the model for use with an edge runtime such as TensorFlow Lite, ONNX Runtime, or Qualcomm AI Engine Direct. Use the following command to export the model:

qai-hub-models export dpt_hybrid_midas

Additional options are documented with the --help option.

License

• The license for the original implementation of DPT-Hybrid-MiDaS can be found here.

References

• Vision Transformers for Dense Prediction
• Source Model Implementation

Community

• Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
• For questions or feedback please reach out to us.