Package: openvino-model-api
OpenVINO Model APIModel API is a set of wrapper classes for particular tasks and model architectures, simplifying data preprocess and postprocess as well as routine procedures (model loading, asynchronous execution, etc.). It is aimed at simplifying end-to-end model inference for different deployment scenarious, including local execution and serving. The Model API is based on the OpenVINO inference API.How it worksModel API searches for additional information required for model inference, data, pre/postprocessing, label names, etc. directly in OpenVINO Intermediate Representation. This information is used to prepare the inference data, process and ouput the inference results in a human-readable format.FeaturesPython and C++ APIAutomatic prefetch of public models fromOpenVINO Model Zoo(Python only)Syncronous and asynchronous inferenceLocal inference and servring through the rest API (Python only)Model preprocessing embedding for faster inferenceInstallationPythonClone this repositoryNavigate tomodel_api/pythonfolderRunpip install .C++Install dependencies. For installation on Ubuntu, you can use the following script:chmod+xmodel_api/cpp/install_dependencies.sh
sudomodel_api/cpp/install_dependencies.shBuild library:Createbuildfolder and navigate into it:mkdir build && cd buildRun cmake:cmake ../model_api/cpp -DOpenCV_DIR=<OpenCV cmake dir> -DOpenVINO_DIR=<OpenVINO cmake dir>Build:cmake --build . -jTo build a.tar.gzpackage with the library, run:cmake --build . --target package -jUsagePythonfromopenvino.model_api.modelsimportDetectionModel# Create a model (downloaded and cached automatically for OpenVINO Model Zoo models)# Use URL to work with served model, e.g. "localhost:9000/models/ssd300"ssd=DetectionModel.create_model("ssd300")# Run synchronous inference locallydetections=ssd(image)# image is numpy.ndarray# Print the list of Detection objects with box coordinates, confidence and label stringprint(f"Detection results:{detections}")C++#include<models/detection_model.h>#include<models/results.h>// Load the model fetched using Python APIautomodel=DetectionModel::create_model("~/.cache/omz/public/ssd300/FP16/ssd300.xml");// Run synchronous inference locallyautoresult=model->infer(image);// image is cv::Mat// Iterate over the vector of DetectedObject with box coordinates, confidence and label stringfor(auto&obj:result->objects){std::cout<<obj.label<<" | "<<obj.confidence<<" | "<<int(obj.x)<<" | "<<int(obj.y)<<" | "<<int(obj.x+obj.width)<<" | "<<int(obj.y+obj.height)<<std::endl;}Model's static methodcreate_model()has two overloads. One constructs the model from a string (a path or a model name) (shown above) and the other takes an already constructedInferenceAdapter.Prepare a model forInferenceAdapterThere are usecases when it is not possible to modify an internalov::Modeland it is hidden behindInferenceAdapter. For example the model can be served usingOVMS.create_model()can construct a model from a givenInferenceAdapter. That approach assumes that the model inInferenceAdapterwas already configured bycreate_model()called with a string (a path or a model name). It is possible to prepare such model using C++ or Python:
C++automodel=DetectionModel::create_model("~/.cache/omz/public/ssd300/FP16/ssd300.xml");conststd::shared_ptr<ov::Model>&ov_model=model->getModel();ov::serialize(ov_model,"serialized.xml");Pythonmodel=DetectionModel.create_model("~/.cache/omz/public/ssd300/FP16/ssd300.xml")model.save("serialized.xml")After that the model can be constructed fromInferenceAdapter:ov::Corecore;std::shared_ptr<ov::Model>ov_model=core.read_model("serialized.xml");std::shared_ptr<InferenceAdapter>adapter=std::make_shared<OpenVINOInferenceAdapter>();adapter->loadModel(ov_model,core);automodel=DetectionModel::create_model(adapter);For more details please refer to theexamplesof this project.Supported modelsPython:Image Classification:OpenVINO Model Zoo modelsObject Detection:OpenVINO Model Zoo models:SSD-based models (e.g. "ssd300", "ssdlite_mobilenet_v2", etc.)YOLO-based models (e.g. "yolov3", "yolov4", etc.)CTPN: "ctpn"DETR: "detr-resnet50"CenterNet: "ctdet_coco_dlav0_512"FaceBoxes: "faceboxes-pytorch"RetinaFace: "retinaface-resnet50-pytorch"Ultra Lightweight Face Detection: "ultra-lightweight-face-detection-rfb-320" and "ultra-lightweight-face-detection-slim-320"NanoDet with ShuffleNetV2: "nanodet-m-1.5x-416"NanoDet Plus with ShuffleNetV2: "nanodet-plus-m-1.5x-416"Semantic Segmentation:OpenVINO Model Zoo modelsInstance Segmentation:OpenVINO Model Zoo modelsC++:Image Classification:OpenVINO Model Zoo modelsObject Detection:SSD-based models (e.g. "ssd300", "ssdlite_mobilenet_v2", etc.)YOLO-based models (e.g. "yolov3", "yolov4", etc.)CenterNet: "ctdet_coco_dlav0_512"FaceBoxes: "faceboxes-pytorch"RetinaFace: "retinaface-resnet50-pytorch"Semantic Segmentation:OpenVINO Model Zoo modelsModel configurationdiscusses possible configurations.