Package: onnxscript
ONNX ScriptONNX Script enables developers to naturally author ONNX functions and
models using a subset of Python. ONNX Script is:Expressive:enables the authoring of all ONNX functions.Simple and concise:function code is natural and simple.Debuggable:allows for eager-mode evaluation that provides for a
more delightful ONNX model debugging experience.Note however that ONNX Script doesnotintend to support the entirety
of the Python language.Design OverviewONNX Script provides a few major capabilities for authoring and debugging
ONNX models and functions:A converter which translates a Python ONNX Script function into an
ONNX graph, accomplished by traversing thePython Abstract Syntax Treeto build an ONNX graph equivalent of the function.A converter that operates inversely, translating ONNX models and
functions into ONNX Script. This capability can be used to fully round-trip
ONNX Script ↔ ONNX graph.A runtime shim that allows such functions to be evaluated
(in an "eager mode"). This functionality currently relies onONNX Runtimefor executing everyONNX Operator,
and there is a Python-only reference runtime for ONNX underway that
will also be supported.Note that the runtime is intended to help understand and debug function definitions. Performance is not a goal here.Installing ONNX Scriptpipinstall--upgradeonnxscriptInstall for Developmentpipinstallonnxonnxruntimepytest
gitclonehttps://github.com/microsoft/onnxscriptcdonnxscript
pipinstall-e.Run Unit TestspytestonnxscriptExampleimportonnx# We use ONNX opset 15 to define the function below.fromonnxscriptimportFLOAT,scriptfromonnxscriptimportopset15asop# We use the script decorator to indicate that# this is meant to be translated to ONNX.@script()defonnx_hardmax(X,axis:int):"""Hardmax is similar to ArgMax, with the result being encoded OneHot style."""# The type annotation on X indicates that it is a float tensor of# unknown rank. The type annotation on axis indicates that it will# be treated as an int attribute in ONNX.## Invoke ONNX opset 15 op ArgMax.# Use unnamed arguments for ONNX input parameters, and named# arguments for ONNX attribute parameters.argmax=op.ArgMax(X,axis=axis,keepdims=False)xshape=op.Shape(X,start=axis)# use the Constant operator to create constant tensorszero=op.Constant(value_ints=[0])depth=op.GatherElements(xshape,zero)empty_shape=op.Constant(value_ints=[0])depth=op.Reshape(depth,empty_shape)values=op.Constant(value_ints=[0,1])cast_values=op.CastLike(values,X)returnop.OneHot(argmax,depth,cast_values,axis=axis)# We use the script decorator to indicate that# this is meant to be translated to ONNX.@script()defsample_model(X:FLOAT[64,128],Wt:FLOAT[128,10],Bias:FLOAT[10])->FLOAT[64,10]:matmul=op.MatMul(X,Wt)+Biasreturnonnx_hardmax(matmul,axis=1)# onnx_model is an in-memory ModelProtoonnx_model=sample_model.to_model_proto()# Save the ONNX model at a given pathonnx.save(onnx_model,"sample_model.onnx")# Check the modeltry:onnx.checker.check_model(onnx_model)exceptonnx.checker.ValidationErrorase:print(f"The model is invalid:{e}")else:print("The model is valid!")The decorator parses the code of the function, converting it into an
intermediate representation. If it fails, it produces an error message
indicating the line where the error was detected. If it succeeds, the
intermediate representation can be converted into an ONNX graph
structure of typeFunctionProto:Hardmax.to_function_proto()returns aFunctionProtoEager Mode EvaluationEager mode is mostly used to debug and validate that intermediate results
are as expected. The function defined above can be called as below,
executing in an eager-evaluation mode:importnumpyasnpv=np.array([[0,1],[2,3]],dtype=np.float32)result=Hardmax(v)More examples can be found in thedocs/examplesdirectory.Development GuidelinesEvery change impacting the converter or the eager evaluation must be
unit tested with classOnnxScriptTestCaseto ensure both systems do
return the same results with the same inputs.Coding StyleWe useruff,black,isort, andmypyetc. to check code formatting and uselintrunnerto run all linters.
You can install the dependencies and initialize withpipinstalllintrunnerlintrunner-adapters
lintrunnerinitThis will install lintrunner on your system and download all the necessary dependencies to run linters locally.
If you want to see what lintrunner init will install, runlintrunner init --dry-run.To lint local changes:lintrunnerTo format files:lintrunnerfTo lint all files:lintrunner--all-filesUse--output onelineto produce a compact list of lint errors, useful when
there are many errors to fix.See all available options withlintrunner -h.To read more about lintrunner, seewiki.
To update an existing linting rule or create a new one, modify.lintrunner.tomlor create a
new adapter following examples inhttps://github.com/justinchuby/lintrunner-adapters.ContributingWe're always looking for your help to improve the product (bug fixes, new features, documentation, etc). Currently ONNX Script is under early and heavy development, so we encourage proposing any major changes byfiling an issueto discuss your idea with the team first.Report a Security IssuePlease do not report security vulnerabilities through public GitHub issues.Please refer to our guidance on filingSecurity Issues.Licensing GuidelinesThis project welcomes contributions and suggestions. Most contributions require you to
agree to a Contributor License Agreement (CLA) declaring that you have the right to,
and actually do, grant us the rights to use your contribution. For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether you need
to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the
instructions provided by the bot. You will only need to do this once across all repositories using our CLA.Code of ConductThis project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor contactopencode@microsoft.comwith any additional questions or comments.TrademarksThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must followMicrosoft's Trademark & Brand Guidelines.
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos is subject to those third-party's policies.