Package: qai-hub-models
Qualcomm® AI Hub ModelsTheQualcomm® AI Hub Modelsare a collection of
state-of-the-art machine learning models optimized for performance (latency,
memory etc.) and ready to deploy on Qualcomm® devices.Explore models optimized for on-device deployment of vision, speech, text, and genenrative AI.View open-source recipes to quantize, optimize, and deploy these models on-device.Browse throughperformance metricscaptured for these models on several devices.Access the models throughHugging Face.Sign upto run these models on hosted Qualcomm® devices.Supported runtimesTensorFlow LiteQualcomm AI Engine DirectSupported operating systems:Android 11+Supported compute unitsCPU, GPU, NPU (includesHexagon DSP,HTP)Supported precisionFloating Points: FP16Integer: INT8 (8-bit weight and activation on select models), INT4 (4-bit weight, 16-bit activation on select models)Supported chipsetsSnapdragon 845,Snapdragon 855/855+,Snapdragon 865/865+,Snapdragon 888/888+Snapdragon 8 Gen 1,Snapdragon 8 Gen 2,Snapdragon 8 Gen 3Select supported devicesSamsung Galaxy S21 Series, Galaxy S22 Series, Galaxy S23 Series, Galaxy S24 SeriesXiaomi 12, 13Google Pixel 3, 4, 5and many more.InstallationWe currently supportPython >=3.8 and <= 3.10.We recommend using a Python
virtual environment
(minicondaorvirtualenv).You can setup a virtualenv using:python -m venv qai_hub_models_env && source qai_hub_models_env/bin/activateOnce the environment is setup, you can install the base package using:pipinstallqai_hub_modelsSome models (e.g.YOLOv7) require
additional dependencies. You can install those dependencies automatically
using:pipinstall"qai_hub_models[yolov7]"Getting StartedEach model comes with the following set of CLI demos:Locally runnable PyTorch based CLI demo to validate the model off device.On-device CLI demo that produces a model ready for on-device deployment and runs the model on a hosted Qualcomm® device (needssign up).All the models produced by these demos are freely available onHugging
Faceor through ourwebsite. See the individual model readme
files (e.g.YOLOv7) for more
details.Local CLI Demo with PyTorchAll modelscontain CLI demos that run the model inPyTorchlocally with sample input. Demos are optimized for code clarity
rather than latency, and run exclusively in PyTorch. Optimal model latency can
be achieved with model export viaQualcomm® AI
Hub.python-mqai_hub_models.models.yolov7.demoFor additional details on how to use the demo CLI, use the--helpoptionpython-mqai_hub_models.models.yolov7.demo--helpSee themodel directorybelow to explore all other models.Note that most ML use cases require some pre and post-processing that are not
part of the model itself. A python reference implementation of this is provided
for each model inapp.py. Apps load & pre-process model input, run model
inference, and post-process model output before returning it to you.Here is an example of how the PyTorch CLI works forYOLOv7:fromPILimportImagefromqai_hub_models.models.yolov7importModelasYOLOv7Modelfromqai_hub_models.models.yolov7importAppasYOLOv7Appfromqai_hub_models.utils.asset_loadersimportload_imagefromqai_hub_models.models.yolov7.demoimportIMAGE_ADDRESS# Load pre-trained modeltorch_model=YOLOv7Model.from_pretrained()# Load a simple PyTorch based applicationapp=YOLOv7App(torch_model)image=load_image(IMAGE_ADDRESS,"yolov7")# Perform prediction on a sample imagepred_image=app.predict(image)[0]Image.fromarray(pred_image).show()CLI demo to run on hosted Qualcomm® devicesSome modelscontain CLI demos that run the model on a hosted
Qualcomm® device usingQualcomm® AI Hub.To run the model on a hosted device,sign up for access to Qualcomm® AI
Hub. Sign-in to Qualcomm® AI Hub with your
Qualcomm® ID. Once signed in navigate to Account -> Settings -> API Token.With this API token, you can configure your client to run models on the cloud
hosted devices.qai-hubconfigure--api_tokenAPI_TOKENNavigate todocsfor more information.The on-device CLI demo performs the following:Exports the model for on-device execution.Profiles the model on-device on a cloud hosted Qualcomm® device.Runs the model on-device on a cloud hosted Qualcomm® device and compares accuracy between a local CPU based PyTorch run and the on-device run.Downloads models (and other required assets) that can be deployed on-device in an Android application.python-mqai_hub_models.models.yolov7.exportMany models may have initialization parameters that allow loading custom
weights and checkpoints. See--helpfor more detailspython-mqai_hub_models.models.yolov7.export--helpHow does this export script work?As described above, the script above compiles, optimizes, and runs the model on
a cloud hosted Qualcomm® device. The demo usesQualcomm® AI Hub's Python
APIs.Here is a simplified example of code that can be used to run the entire model
on a cloud hosted device:fromtypingimportTupleimporttorchimportqai_hubashubfromqai_hub_models.models.yolov7importModelasYOLOv7Model# Load YOLOv7 in PyTorchtorch_model=YOLOv7Model.from_pretrained()torch_model.eval()# Trace the PyTorch model using one data point of provided sample inputs to# torch tensor to trace the model.example_input=[torch.tensor(data[0])forname,dataintorch_model.sample_inputs().items()]pt_model=torch.jit.trace(torch_model,example_input)# Select a devicedevice=hub.Device("Samsung Galaxy S23")# Compile model for a specific devicecompile_job=hub.submit_compile_job(model=pt_model,device=device,input_specs=torch_model.get_input_spec(),)# Get target model to run on a cloud hosted devicetarget_model=compile_job.get_target_model()# Profile the previously compiled model on a cloud hosted deviceprofile_job=hub.submit_profile_job(model=target_model,device=device,)# Perform on-device inference on a cloud hosted deviceinput_data=torch_model.sample_inputs()inference_job=hub.submit_inference_job(model=target_model,device=device,inputs=input_data,)# Returns the output as dict{name: numpy}on_device_output=inference_job.download_output_data()Working with source codeYou can clone the repository using:gitclonehttps://github.com/quic/ai-hub-models/blob/maincdmain
pipinstall-e.Install additional dependencies to prepare a model before using the following:cdmain
pipinstall-e".[yolov7]"All models have accuracy and end-to-end tests when applicable. These tests as
designed to be run locally and verify that the PyTorch code produces correct
results.  To run the tests for a model:python-mpytest--pyargsqai_hub_models.models.yolov7.testFor any issues, please contact us atai-hub-support@qti.qualcomm.com.Model DirectoryComputer VisionModelREADMETorch AppDevice ExportCLI DemoImage ClassificationMobileNet-v2-Quantizedqai_hub_models.models.mobilenet_v2_quantized✔️✔️✔️GoogLeNetqai_hub_models.models.googlenet✔️✔️✔️ResNet50qai_hub_models.models.resnet50✔️✔️✔️Swin-Smallqai_hub_models.models.swin_small✔️✔️✔️Inception-v3Quantizedqai_hub_models.models.inception_v3_quantized✔️✔️✔️MobileNet-v3-Smallqai_hub_models.models.mobilenet_v3_small✔️✔️✔️GoogLeNetQuantizedqai_hub_models.models.googlenet_quantized✔️✔️✔️RegNetqai_hub_models.models.regnet✔️✔️✔️ResNeXt50qai_hub_models.models.resnext50✔️✔️✔️VITqai_hub_models.models.vit✔️✔️✔️ResNet18Quantizedqai_hub_models.models.resnet18_quantized✔️✔️✔️ResNet101qai_hub_models.models.resnet101✔️✔️✔️ResNeXt101qai_hub_models.models.resnext101✔️✔️✔️MobileNet-v2qai_hub_models.models.mobilenet_v2✔️✔️✔️SqueezeNet-1_1qai_hub_models.models.squeezenet1_1✔️✔️✔️SqueezeNet-1_1Quantizedqai_hub_models.models.squeezenet1_1_quantized✔️✔️✔️Inception-v3qai_hub_models.models.inception_v3✔️✔️✔️WideResNet50qai_hub_models.models.wideresnet50✔️✔️✔️ResNet101Quantizedqai_hub_models.models.resnet101_quantized✔️✔️✔️MNASNet05qai_hub_models.models.mnasnet05✔️✔️✔️Swin-Baseqai_hub_models.models.swin_base✔️✔️✔️DenseNet-121qai_hub_models.models.densenet121✔️✔️✔️Shufflenet-v2Quantizedqai_hub_models.models.shufflenet_v2_quantized✔️✔️✔️Shufflenet-v2qai_hub_models.models.shufflenet_v2✔️✔️✔️ResNeXt101Quantizedqai_hub_models.models.resnext101_quantized✔️✔️✔️ResNet18qai_hub_models.models.resnet18✔️✔️✔️EfficientNet-B0qai_hub_models.models.efficientnet_b0✔️✔️✔️MobileNet-v3-Largeqai_hub_models.models.mobilenet_v3_large✔️✔️✔️WideResNet50-Quantizedqai_hub_models.models.wideresnet50_quantized✔️✔️✔️ConvNext-Tinyqai_hub_models.models.convnext_tiny✔️✔️✔️Swin-Tinyqai_hub_models.models.swin_tiny✔️✔️✔️Image EditingLaMa-Dilatedqai_hub_models.models.lama_dilated✔️✔️✔️Image GenerationStyleGAN2qai_hub_models.models.stylegan2✔️✔️✔️Super ResolutionQuickSRNetLargeqai_hub_models.models.quicksrnetlarge✔️✔️✔️ESRGANqai_hub_models.models.esrgan✔️✔️✔️Real-ESRGAN-x4plusqai_hub_models.models.real_esrgan_x4plus✔️✔️✔️XLSR-Quantizedqai_hub_models.models.xlsr_quantized✔️✔️✔️QuickSRNetMediumqai_hub_models.models.quicksrnetmedium✔️✔️✔️Real-ESRGAN-General-x4v3qai_hub_models.models.real_esrgan_general_x4v3✔️✔️✔️SESR-M5-Quantizedqai_hub_models.models.sesr_m5_quantized✔️✔️✔️QuickSRNetSmallqai_hub_models.models.quicksrnetsmall✔️✔️✔️SESR-M5qai_hub_models.models.sesr_m5✔️✔️✔️XLSRqai_hub_models.models.xlsr✔️✔️✔️Semantic SegmentationYolo-v8-Segmentationqai_hub_models.models.yolov8_seg✔️✔️✔️SINetqai_hub_models.models.sinet✔️✔️✔️Unet-Segmentationqai_hub_models.models.unet_segmentation✔️✔️✔️FCN_ResNet50qai_hub_models.models.fcn_resnet50✔️✔️✔️DDRNet23-Slimqai_hub_models.models.ddrnet23_slim✔️✔️✔️FastSam-Sqai_hub_models.models.fastsam_s✔️✔️✔️FFNet-122NS-LowResqai_hub_models.models.ffnet_122ns_lowres✔️✔️✔️FFNet-78S-Quantizedqai_hub_models.models.ffnet_78s_quantized✔️✔️✔️FFNet-40S-Quantizedqai_hub_models.models.ffnet_40s_quantized✔️✔️✔️MediaPipe-Selfie-Segmentationqai_hub_models.models.mediapipe_selfie✔️✔️✔️DeepLabV3-ResNet50qai_hub_models.models.deeplabv3_resnet50✔️✔️✔️FastSam-Xqai_hub_models.models.fastsam_x✔️✔️✔️FFNet-54Sqai_hub_models.models.ffnet_54s✔️✔️✔️FFNet-78S-LowResqai_hub_models.models.ffnet_78s_lowres✔️✔️✔️Segment-Anything-Modelqai_hub_models.models.sam✔️✔️✔️FFNet-78Sqai_hub_models.models.ffnet_78s✔️✔️✔️FFNet-40Sqai_hub_models.models.ffnet_40s✔️✔️✔️FFNet-54S-Quantizedqai_hub_models.models.ffnet_54s_quantized✔️✔️✔️Object DetectionMediaPipe-Hand-Detectionqai_hub_models.models.mediapipe_hand✔️✔️✔️DETR-ResNet50-DC5qai_hub_models.models.detr_resnet50_dc5✔️✔️✔️DETR-ResNet101-DC5qai_hub_models.models.detr_resnet101_dc5✔️✔️✔️Yolo-v8-Detectionqai_hub_models.models.yolov8_det✔️✔️✔️DETR-ResNet101qai_hub_models.models.detr_resnet101✔️✔️✔️DETR-ResNet50qai_hub_models.models.detr_resnet50✔️✔️✔️Yolo-v7qai_hub_models.models.yolov7✔️✔️✔️Yolo-v6qai_hub_models.models.yolov6✔️✔️✔️MediaPipe-Face-Detectionqai_hub_models.models.mediapipe_face✔️✔️✔️Pose EstimationHRNetPoseQuantizedqai_hub_models.models.hrnet_pose_quantized✔️✔️✔️MediaPipe-Pose-Estimationqai_hub_models.models.mediapipe_pose✔️✔️✔️LiteHRNetqai_hub_models.models.litehrnet✔️✔️✔️HRNetPoseqai_hub_models.models.hrnet_pose✔️✔️✔️OpenPoseqai_hub_models.models.openpose✔️✔️✔️AudioModelREADMETorch AppDevice ExportCLI DemoSpeech RecognitionWhisper-Baseqai_hub_models.models.whisper_asr✔️✔️✔️HuggingFace-WavLM-Base-Plusqai_hub_models.models.huggingface_wavlm_base_plus✔️✔️✔️Audio EnhancementFacebook-Denoiserqai_hub_models.models.facebook_denoiser✔️✔️✔️MultimodalModelREADMETorch AppDevice ExportCLI DemoTrOCRqai_hub_models.models.trocr✔️✔️✔️OpenAI-Clipqai_hub_models.models.openai_clip✔️✔️✔️Generative AiModelREADMETorch AppDevice ExportCLI DemoImage GenerationStable-Diffusionqai_hub_models.models.stable_diffusion_quantized✔️✔️✔️ControlNetqai_hub_models.models.controlnet_quantized✔️✔️✔️Text GenerationLlama-v2-7B-Chatqai_hub_models.models.llama_v2_7b_chat_quantized✔️✔️✔️Baichuan-7Bqai_hub_models.models.baichuan_7b_quantized✔️✔️✔️