Package: onnx4acumos
onnx4acumosonnx4acumosis a client library that allows modelers to on-board their onnx models
on an Acumos platform and also to test and run their onnx models.For more informations on Acumos see :Acumos AI Linux Fondation project, hisAcumos AI Wikiand hisDocumentation.Based on theacumospython client, we builtonnx4acumosclient able to create the onnx model bundle with all the
required files needed by Acumos platform.
When you usedonnx4acumos, you can choose to on-board your onnx model directly in Acumos with or whithout micro-service
creation (CLI on-boarding). Or you can choose to save your Acumos model bundle locally for later manual on-boarding (Web-onboarding).
It that caseonnx4acumoswill create a ModelName Directory in which you will find the Acumos model bundle and all the
necessary files to test and run the Acumos onnx model bundle locally.Micro-service generation in Acumos will transform your onnx model as a serving model, based on docker, ready to be deployed.InstallationThe main requirements to installonnx4acumosis to install first the following dependancies :onnx, zipp, acumos, acumos-model-runner, numpy, requests, protobuf, dill, appdirs, filelock, typing-inspect, grpcio, onnxruntimeOnce it is done, you can installonnx4acumoswith pip:pipinstallonnx4acumosremark : if you used Acumos CLIO you must used python3.6 with acumos 0.8.0 and acumos_model_runner 0.2.3onnx4acumos TutorialThis tutorial explains how to on-board an onnx model in an Acumos platform with microservice creation.
It’s meant to be followed linearly, and some code snippets depend on earlier imports and objects.
Full onnx python client examples are available in the/acumos-onnx-client/acumos-package/onnx4acumosdirectory of theAcumos onnx client repository.We assume that you have already installedonnx4acumospackage.On-boarding Onnx Model on Acumos PlatformHow to test & run your ONNX modelMore ExamplesOn-boarding Onnx Model on Acumos PlatformClone the acumos-onnx-client from gerritgitclone"ssh://your_gerrit_login@gerrit.acumos.org:29418/acumos-onnx-client"&&scp-p-P29418your_gerrit_login@gerrit.acumos.org:hooks/commit-msg"acumos-onnx-client/.git/hooks/"orgitclone"ssh://your_gerrit_login@gerrit.acumos.org:29418/acumos-onnx-client"or fromGithubYou will need the two following files for this tutorial :The model located at/acumos-onnx-client/acumos-package/onnx4acumos/OnnxModels/super_resolution_zoo.onnxA configuration file located at/acumos-onnx-client/acumos-package/onnx4acumos/Templates/onnx4acumos.iniThis configuration file is mandatory if you want to push your model in Acumos by CLI (CLI on-boarding). You must copy
this file locally if you want to onboard your own models (sometimes onnx4acumos folder name can be confused with onnx4acumos command).onnx4acumos.ini looks like :[certificates]CURL_CA_BUNDLE:/etc/ssl/certs/ca-certificates.crt[proxy]https_proxy:socks5h://127.0.0.1:8886/http_proxy:socks5h://127.0.0.1:8886/[session]push_api:https://acumos/onboarding-app/v2/modelscertificates :location of acumos certificates generated during the installation,
you can also let this parameter empty (CURL_CA_BUNDLE:), in that case you will just
receive a warning.proxy: The proxy you used to reach your acumos platform. If you don’t use proxy, you can also let this
parameter empty (https:proxy:).session: The on-boarding model push API URL, available in Acumos GUI in the ON-BOARDING MODEL page.To on-board, by CLI, the super_resolution_zoo model in Acumos platform with micro-service activation, use the following
command line :onnx4acumossuper_resolution_zoo.onnxonnx4acumos.ini-push-ms-li"path to your license file"-deployIn this command line the -push parameter is used to on-board the onnx model directly
in Acumos (CLI on-boarding). You will be prompted to enter your on-boarding token
: onboarding token = “your Acumos login”:”authentication token” (example : acumos_user:a2a6a9e8f4gbg3c147eq9g3h).
The “authentication token” can be retrieved in the ACUMOS GUI in your personal settings.The -ms parameter is used to launch the micro-service creation in Acumos right after the on-boarding.
If -ms is omitted, the model will be on-boarded whithout micro-service generation.
(don’t worry, you can create the micro-service later in Acumos)).The -li parameter is used to onboard a license file alongside your model in Acumos in order to protect the model’s copyright.
This parameter is optional. Please refers to the licence management project in the Acumos wiki. You can find a license
template in the doc folder of the acumos4onnx repo in github.The -deploy parameter is used to deploy the model automatically after the microservice generation (based in Jenkins server
configuration set up in Acumos/SITE ADMIN/model deployment automation), by default deploy=False, so if deploy is not mentionned
in the command line the model will not be deployed. If -deploy is added in the comand line and -ms has been ommitted,
the microservice will be created and deployed.To on-board by web the super_resolution_zoo model in Acumos platform, follow the next step :First you have to dump the super_resolution_zoo model locally :onnx4acumossuper_resolution_zoo.onnxonnx4acumos.ini-dump-finput/cat.jpgThe onnx4acumos.ini configuration file is optionnal when you dump your model bundle localy for WEB on-boarding purpose, however
it can be provided, in the command line, in order to copy it in “ModelName” directory for later use (push using ModelName/ModelName_OnnxModelOnBoarding.py).Thanks to the command line above a “ModelName” directory (“super_resolution_zoo” directory in our case)
is created and it contains all the files needed to test the onnx model locally, the -f parameter is optional and
is used to add an input data file in the ModelName_OnnxClient folder.An Acumos model bundle is also created locally and ready to be on-boarded in Acumos manually (Web onboarding).
The default parameter -dump (can be omitted) allows the bundle to be saved locally.You can find the “ModelName” directory contents description below :In this directory, you cand find :ModelName_OnnxModelOnboarding.py : Python file used to onboard a model in Acumos by CLI and/or to dump the model bundle locally.Dumped Model directory(model bundle) : Directory that contains all the required files nedded by an Acumos platform.Zipped model bundle(ModelName.zip) : zip file (built from Dumped Model directory) ready to be onboarded in Acumos.ModelName_OnnxClient directory : Directory that contains all the necessary files to create a client/server able to test & run your model.Then The last thing to do is to drag and drop the Zipped model bundle in the “ON-BOARDING BY WEB” page of Acumos or use the browse function to on-board your
model.How to test & run your ONNX modelThis on-boarding client can also be used to test and run your onnx model, regardless of whether you want to on-board it or not in Acumos.
You have to follow the two main steps, first Launch the model runner server and then fill the skeleton client file to create the onnx client.We assume that:You have installedacumos_model_runnerpackage.You have dumped the model bundle locally as explained above.We use a client-server architecture to test and run onnx models, first you have to launch your model runner locally to create the server,
then you have to use a python sript as an onnx client to interact with the server.Launch model runner serverThe local server part can be started quite simply as follows :acumos_model_runnersuper_resolution_zoo/dumpedModel/super_resolution_zooThe acumos model runner will also create a swagger interface available at localhost:3330.Fill skeleton client file to create the ONNX clientYou can find the python client skeleton file desciptions below :This python client skeleton file is available in the following foldersuper_resolution_zoo/super_resolution_zoo_OnnxClientAll steps, in order to fill this python client skeleton, are described below. You must filled the part between two lines of “*******”
You just have to copy/paste the following code snipsets below in the right place of your skeleton file.First import your own needed libraries:# Import your own needed library below"**************************************"fromnumpyimportclipimportPIL# torch importsimporttorchvision.transformsastransforms"**************************************"Second, define your own needed methods:# Define your own needed method below"**************************************"defto_numpy(tensor):returntensor.detach().cpu().numpy()iftensor.requires_gradelsetensor.cpu().numpy()"**************************************"Third, define Preprocessing method:# Import the management of the Onnx data preprocessing below.# The "preProcessingOutput" variable must contain the preprocessing result with type found in run_xx_OnnxModel method signature below"*************************************************************************************************"globalimg_cb,img_crimg=PIL.Image.open(preProcessingInput)resize=transforms.Resize([224,224])img=resize(img)img.show()img_ycbcr=img.convert('YCbCr')img_y,img_cb,img_cr=img_ycbcr.split()to_tensor=transforms.ToTensor()img_y=to_tensor(img_y)img_y.unsqueeze_(0)preprocessingResult=to_numpy(img_y)"**************************************************************************************************"# "PreProcessingOutput" variable affectation with the preprocessing resultFourth, define Postprocessing method:# Import the management of the Onnx data postprocessing below.# The "postProcessingInput" variable must contain the data of the Onnx model result with type found in method signature below"*************************************************************************************************"globalimg_cb,img_crimg_out_y=output[0]img_out_y=np.array((img_out_y[0]*255.0))img_out_y=clip(img_out_y,0,255)img_out_y=PIL.Image.fromarray(np.uint8(img_out_y),mode='L')final_img=PIL.Image.merge("YCbCr",[img_out_y,img_cb.resize(img_out_y.size,PIL.Image.BICUBIC),img_cr.resize(img_out_y.size,PIL.Image.BICUBIC),]).convert("RGB")f=io.BytesIO()final_img.save(f,format='jpeg')imageOutputData=f.getvalue()final_img.show()postProcessingResult=imageOutputData"*************************************************************************************************"And finally :Redefine the REST URL if necessary (by default, localhost on port 3330):restURL="http://localhost:3330/model/methods/run_super_resolution_zoo_OnnxModel"The final name of the filled skeleton ModelName_OnnxClientSkeleton.py could be  ModelName_OnnxClient.py
(the same name without Skeleton, super_resolution_zoo_OnnxClient.py for our example).The filled python client skeleton file can be retrieved in the acumos-onnx-client folder :
acumos-onnx-client/acumos-package/onnx4acumos/FilledClientSkeletonsExamples/super_resolution_zoo_OnnxClient.py.Remark : To test super_resolution_zoo you must have a server X running on your local system.Command linesYou can find all command lines to test and run onnx model super_resolution_zoo below :onnx4acumossuper_resolution_zoo.onnxonnx4acumos.ini-fInputData/cat.jpgacumos_model_runnersuper_resolution_zoo/dumpedModel/super_resolution_zoo/## Launch the model runner serverpythonsuper_resolution_zoo_OnnxClient.py-finput/cat.jpg## Launch client and send input datasuper_resolution_zoo_Model exampleMore ExamplesBelow are some additional examples.
Post and Pre-processing methods are available in the Github folder :onnx/modelsGoogLeNetYou can find all command lines for GoogleNetexample below :onnx4acumosOnnxModels/GoogleNet.onnxonnx4acumos.ini-fInputData/car4.jpgacumos_model_runnerGoogLeNet/dumpedModel/GoogleNet/## Lanch the model runner servercdGoogLeNet/GoogLeNet_OnnxClientpythonGoogLeNet_OnnxClient.py-finput/car4.jpg## Launch client and send input dataIn our example above :pythonGoogLeNet_OnnxClient.py-finput/car4.jpgpythonGoogLeNet_OnnxClient.py-finput/BM4.jpegpythonGoogLeNet_OnnxClient.py-finput/espresso.jpegpythonGoogLeNet_OnnxClient.py-finput/cat.jpgpythonGoogLeNet_OnnxClient.py-finput/pesan3.jpgEmotion Ferplus Model examplepythonemotion_ferplus_model_OnnxClient.py-finput/angryMan.pngpythonemotion_ferplus_model_OnnxClient.py-finput/sadness.pngpythonemotion_ferplus_model_OnnxClient.py-finput/happy.jpgpythonemotion_ferplus_model_OnnxClient.py-finput/joker.jpgThat’s all  :-)Acumos ONNX Client Release Notesv1.0.5 30 June 2021Minor documentation changeACUMOS-4337v1.0.4 08 June 2021Fix failed with onnx new version 1_9_0ACUMOS-4337v1.0.3, 27 April 2021Adding deploy managementACUMOS-4308Adding licence file managementACUMOS-4319Avoid the use of configuration file when model bundle is dumpedACUMOS-4317fix typo “Exemples” in folderACUMOS-4318fix typo “and/” in index fileACUMOS-4320v1.0.0, 22 January 2021Creation of onnx4acumos ‘ACUMOS-3101 <https://jira.acumos.org/browse/ACUMOS-3101>’_Acumos onnx Client Developer GuideTestingWe use a combination oftox,pytest, andflake8to testacumos_onnx_client. Code which is not PEP8 compliant (aside from E501) will be
considered a failing test. You can use tools likeautopep8to
“clean” your code as follows:$pipinstallautopep8$cdacumos-onnx-client$autopep8-r--in-place--ignoreE501acumos_onnx_client/testing/examples/Run tox directly:$cdacumos-onnx-client$exportWORKSPACE=$(pwd)# env var normally provided by Jenkins$toxYou can also specify certain tox environments to test:$tox-epy36# only test against Python 3.6$tox-eflake8# only lint codeA set of integration test is also available inacumos-package/testing/integration_tests.
To run those, useacumos-package/testing/tox-integration.inias tox config (-c flag),
onboarding tests will be ran with python 3.6 to 3.9.
You will need to set your user credentials and platform configuration intox-integration.ini.$tox-cacumos-package/testing/integration_testsPackagingThe RST files in the docs/ directory are used to publish HTML pages to
ReadTheDocs.io and to build the package long description in setup.py.
The symlink from the subdirectory acumos-package to the docs/ directory
is required for the Python packaging tools.  Those tools build a source
distribution from files in the package root, the directory acumos-package.
The MANIFEST.in file directs the tools to pull files from directory docs/,
and the symlink makes it possible because the tools only look within the
package root.