haz un resumen punto por punto de esta informacion: "--- Page 1 ---
Sentis overview | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Sentis overview Sentis is a neural network inference library for Unity. The package is in a "release" state and available to all Unity users via the package manager. You can use Sentis to import trained neural network models into Unity, and then run them in real-time. Sentis utilizes the end-users device compute (GPU or CPU) and can run any supported Unity runtime platform. To use Sentis, it helps if you have some experience in using machine learning models, for example in a framework like TensorFlow or PyTorch . Section Description Get started Install Sentis, find and use sample projects, and understand the Sentis workflow. Create a model Create a runtime model by importing an ONNX model file or using the Sentis model API. Run a model Create input data for a model, create an engine to run the model, and get output. Use Tensors Get, set and modify input and output data. Profile a model Use Unity tools to profile the speed and performance of a model. Supported platforms Sentis works on all Unity runtime platforms . Performance may vary and is based upon: Model operators and complexity. End-user hardware and software platform constraints. Type of engine used. Refer to Models and Create an engine for more information. Supported model types Sentis supports most models in Open Neural Network Exchange (ONNX) format with an opset version between 7 and 15. Refer to Supported models and Supported ONNX operators for more information. Places to find pre-trained models There are various sources to find pre-trained models, which may either be available in the ONNX format or in a format that you can convert. Examples include: Hugging Face Kaggle Models (Formerly TensorFlow Hub) PyTorch Hub Model Zoo XetData Meta Research If you want to train your own models, refer to the following links: Google Colab Kaggle Additional resources Sample scripts Unity Discussions group Understand the Sentis workflow Understand models in Sentis Tensor fundamentals in Sentis Collaborate with us In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 2 ---
Create an engine to run a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Create an engine to run a model To run a model, you need to create a worker. A worker is the engine that breaks the model down into executable tasks and schedules the tasks to run on a backend, usually the GPU or CPU. Create a Worker Use new Worker(...) to create a worker. You must specify a backend type, which tells Sentis where to run the worker and a runtime model . For example, the following code creates a worker that runs on the GPU using Sentis compute shaders. using UnityEngine;
using Unity.Sentis;

public class CreateWorker : MonoBehaviour
{
    ModelAsset modelAsset;
    Model runtimeModel;
    Worker worker;

    void Start()
    {
        runtimeModel = ModelLoader.Load(modelAsset);
        worker = new Worker(runtimeModel, BackendType.GPUCompute);
    }
} Backend types Sentis provides CPU and GPU backend types. To understand how Sentis executes operations using the different backends, refer to How Sentis runs a model . If a backend type doesn't support a Sentis layer in a model, the worker will assert. Refer to Supported ONNX operators for more information. Among the backend types, BackendType.GPUCompute and BackendType.CPU are the fastest. Only use BackendType.GPUPixel if the platform does not support compute shaders. To check if your runtime platform supports compute shaders, use SystemInfo.supportsComputeShaders . If you use BackendType.CPU with WebGL, Burst compiles to WebAssembly code which might be slow. Refer to Getting started with WebGL development for more information. The speed of model execution depends on the platform's support for multithreading in Burst or its full support for compute shaders. You can profile a model to understand the performance of a model. Additional resources Create a runtime model How Sentis runs a model Supported ONNX operators Run a model In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 3 ---
Profile a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Profile a model The performance of a model depends on the following factors: The complexity of the model. Whether the model uses performance-heavy operators such as Conv or MatMul . The features of the platform you run the model on, for example CPU memory, GPU memory, and number of cores. Whether Sentis downloads data to CPU memory when you access a tensor. Refer to Get output from a model for more information. Profile a model in the Profiler window To get performance information when you run a model, you can use the following: The Profiler window . RenderDoc , a third-party graphics debugger. The Profiler window displays each Sentis layer as a dropdown item in the Module Details panel. Open a layer to get a detailed timeline of the execution of the layer. When a layer executes methods that include Download or Upload , Sentis transfers data to or from the CPU or the GPU. This might slow down the model. If your model runs slower than you expect, refer to: Understand models in Sentis for information about how the complexity of a model might affect performance. Create an engine to run a model for information about different types of worker. Additional resources Inspect a model How Sentis optimizes a model Supported ONNX operators In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 4 ---
Supported models | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Supported models You can import open-source models into your Sentis project. Explore the following sections to understand the models Sentis supports, and find an appropriate model for your project. Pre-trained models There are various sources to find pre-trained models, which may either be available in the ONNX format or in a format that you can convert. Examples include: Hugging Face Kaggle Models (Formerly TensorFlow Hub) PyTorch Hub Model Zoo XetData Meta Research If you want to train your own models, refer to the following links: Google Colab Kaggle Models from Hugging Face You can access validated AI models for use with Sentis from Hugging Face . Models available from Hugging Face are already in the .sentis format to remove the need for ONNX conversion. To find and models from Hugging Face, you can either: Navigate to the Unity Hugging Face space and select a model under the Models section. View models that are validated for Sentis, identified by the unity-sentis tag on the Models page. To import and use the model in a Unity project, follow the instructions in the How to Use section on the model page . ONNX models You can download ONNX model files from the ONNX model zoo on GitHub. Sentis supports most ONNX model files with an opset version between 7 and 15. While versions below 7 or above 15 may still be imported into Sentis, the results obtained may be unexpected. Unsupported models Sentis doesn't support the following: Models that use tensors with more than eight dimensions Sparse input tensors or constants String tensors Complex number tensors Sentis also converts some tensor data types like bools to floats or ints. This might increase the memory your model uses. Additional resources Unity Hugging Face ONNX model zoo Understand models in Sentis Import a model file Supported ONNX operators In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 5 ---
How Sentis runs a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents How Sentis runs a model Sentis performs optimized tensor operations by scheduling the work across multiple threaded jobs on the central processing unit (CPU) or scheduling the task in parallel on the graphics processing unit (GPU) using pixel and compute shaders. When the worker schedules a model, it steps through each layer of the model. It schedules the layer operation on the input tensors to calculate one or more output tensors. The BackendType you choose determines how and when the worker performs each operation. The following tables defines the types of backend available: BackendType Runs on Description CPU CPU, using Burst Sentis creates, sets up, and schedules a Burst job for the operation. If the input tensors are output from other jobs, the worker creates a job dependency to ensure correct inference without blocking. GPUCompute GPU, using Sentis compute shaders with command buffers Sentis creates, sets up, and adds a compute shader the command buffer. Sentis executes the command buffer to perform the operations. GPUPixel GPU, using Sentis pixel shaders Sentis creates, sets up, and executes a pixel shader by blitting. Tensor outputs The tensor values might not be fully calculated when the tensor object is returned, as there might still be scheduled work pending. This allows you to schedule further tensor operations without waiting for or interrupting scheduled operations. To complete the execution of the work on the backend, move the tensor data to the CPU. Call ReadbackAndClone to get a CPU copy of the tensor. This is a blocking call that waits synchronously for both the execution to complete and the data to be read back from the backend. Note that this process can be slow, especially when reading back from the GPU. To avoid blocking calls on the main thread, use one of the following: ReadbackAndCloneAsync for an Awaitable version of this method. ReadbackRequest to trigger an async download. When IsReadbackRequestDone return true, ReadbackAndClone is immediate. To move the tensor data to the CPU with a non-blocking, non-destructive download, use one of the following: ReadbackRequest on your tensor. ReadbackAndCloneAsync on your tensor. DownloadToNativeArray or DownloadToArray . Download on the dataOnBackend of your tensor. CPU fallback Sentis doesn't support every operator on every backend type. For more information, refer to Supported ONNX operators . If Sentis supports an operator on the CPU but not the GPU, Sentis might automatically fall back to running on the CPU. This requires Sentis to sync with the GPU and read back the input tensors to the CPU. If a GPU operation uses the output tensor, Sentis completes the operation and uploads the tensor to the GPU. If a model has many layers that use CPU fallback, Sentis might spend significant time to upload and read back from the CPU. This can impact the performance of your model. To reduce CPU fallback, build the model so that Sentis runs effectively on your chosen backend type or use the CPU backend. For some operations, Sentis needs to read the data contents of a tensor in the main thread so it can schedule the operation. For example, the shape input tensor for an Expand or the axes input for a Reduce . These tensors might be the result of other operations. Sentis optimizes the model during input to determine which tensors need to be calculated on the CPU, irrespective of the model's backend type. Additional resources Use output data Read output from a model asynchronously In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 6 ---
Understand the Sentis workflow | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Understand the Sentis workflow To use Sentis to run a neural network in Unity, follow these steps: Use the Unity.Sentis namespace. Load a neural network model file. Create input for the model. Create an inference engine (a worker). Run the model with the input to compute a result (inference). Get the result. Tip Use the Workflow example to understand the workflow applied to a simple example. Use the Unity.Sentis namespace To use the Unity.Sentis namespace, add the following to the top of your script: using Unity.Sentis; Load a model Sentis can import model files in Open Neural Network Exchange (ONNX) format. To load a model, follow these steps: Export a model to ONNX format from a machine learning framework or download an ONNX model from the Internet. Add the model file to the Assets folder of the Project window. Create a runtime model in your script: ModelAsset modelAsset = Resources.Load("model-file-in-assets-folder") as ModelAsset;
var runtimeModel = ModelLoader.Load(modelAsset); You can also add a public ModelAsset modelAsset as a public variable in GameObjects. In this case specify the model manually. Refer to Import a model file for more information. Create input for the model Use the Tensor API to create a tensor with data for the model. You can convert an array or a texture to a tensor. For example: // Convert a texture to a tensor
Texture2D inputTexture = Resources.Load("image-file") as Texture2D;
Tensor<float> inputTensor = TextureConverter.ToTensor(inputTexture);
// Convert an array to a tensor
int[] array = new int[] {1,2,3,4};
Tensor<int> inputTensor = new Tensor<int>(new TensorShape(4), array); Refer to Create input for a model for more information. Create an inference engine (a worker) In Sentis, a worker is the inference engine. You create a worker to break down the model into executable tasks, run the tasks on the GPU or CPU, and retrieve the result. For example, the following creates a worker that runs on the GPU using Sentis compute shaders: Worker worker = new Worker(runtimeModel, BackendType.GPUCompute); Refer to Create an engine for more information. Schedule the model To run the model, use the Schedule method of the worker object with the input tensor. worker.Schedule(inputTensor); Sentis schedules the model layers on the given backend. Execution is asynchronous, so after this is called, tensor operations may still be pending. Refer to Run a model for more information. Get the output You can use methods such as PeekOutput to get the output data from the model. For example: Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>; Refer to Get output from a model for more information. Additional resources Workflow example Samples Unity Discussions group for the Sentis beta Understand models in Sentis Tensor fundamentals in Sentis In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 7 ---
Run a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Run a model After you create a worker , call Schedule to run the model. worker.Schedule(inputTensor); The first scheduling of a model within the Unity Editor may be slow as Sentis needs to compile code and shaders as well as allocating internal memory. Subsequent runs will be faster due to caching.
Including a dummy run when starting the application is a good idea. For an example, refer to the Run a model sample in the sample scripts . Additional resources Run a model a layer at a time Understand models in Sentis Create an engine to run a model Profile a model In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 8 ---
Understand models in Sentis | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Understand models in Sentis Sentis can import and run trained machine learning model files in Open Neural Network Exchange (ONNX) format. To get a model that's compatible with Sentis, you can do one of the following: Train a model using a framework like TensorFlow, PyTorch, or Keras, and subsequently export it in ONNX format . Download a trained model file and convert to ONNX format . Refer to the ONNXXMLTools Python package for more information. Download a trained model that's already in ONNX format, such as those available in the ONNX Model Zoo . Refer to supported models for more resources. How Sentis optimizes a model When you import a model, each ONNX operator in the model graph becomes a Sentis layer. Open the Model Asset Import Settings to check the list of layers in the imported model, in the order Sentis runs them. Refer to Supported ONNX operators for more information. Sentis optimizes models to make them smaller and more efficient. For example, Sentis might do the following to an imported model: Remove a layer or subgraph and turn it into a constant. Replace a layer or subgraph with a simpler layer or subgraph that works the same way. Set a layer to run on the CPU, if the data must be read at inference time. The optimization doesn't affect what the model inputs or outputs. Model inputs You can get the shape of your model inputs in one of two ways: Inspect a model to use the inputs property of the runtime model. Select your model from the Project window to open the Model Asset Import Settings and view the Inputs section. The shape of a model input consists of multiple dimensions, defined as a DynamicTensorShape . The dimensions of a model input are either fixed or dynamic: An int denotes a fixed dimension. The strings ? and d0 , d1 etc. denote dynamic dimensions. Fixed dimensions The value of the int defines the specific size of the input the model accepts. For example, if the Inputs section displays (1, 1, 28, 28) , the model only accepts a tensor of size 1 x 1 x 28 x 28 . Dynamic dimensions When the shape of a model input contains a dynamic dimension, a ? or string , for example batch_size or height , the input dimension can be any size. For example, if the input is (d0, 1, 28, 28) , the first dimension of the input can be any size. When you define the input tensor for this input shape, the following input tensor shapes are valid: [1, 1, 28, 28]
[2, 1, 28, 28]
[3, 1, 28, 28] ... If you change the size of another dimension, however, the tensor input is not valid. For example: [1, 3, 28, 28] Note If a model uses inputs with dynamic shapes, Sentis might not be able to optimize the model as efficiently as a model that uses fixed input dimensions. This might slow down the model. Additional resources Import a model Supported models Export an ONNX file from a machine learning framework Supported ONNX operators In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 9 ---
Samples | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Samples Samples are provided with the Sentis package to illustrate how to use the API and facilitate the learning process of using Sentis. The available samples are: Sample projects from the Sentis GitHub Sample scripts from the Package Manager Validated models are also available to use in your project. To understand and download available models, refer to Supported models . Sample projects Full sample projects are available to demonstrate various Sentis use cases. To access complete sample projects, visit the Sentis samples GitHub repository. This repository will be updated over time with more samples. Some samples contain a helpful video overview linked in the readme file. Sample scripts Use the sample scripts to implement specific features in your own project. To find the sample scripts: Go to Window > Package Manager , and select Sentis from the package list. Select Samples . To import a sample folder into your project, select Import . Unity creates a Samples folder in your project, and imports the sample folder you selected. The following table describes the available samples: Sample folder Description Convert tensors to textures Examples of converting tensors to textures. Refer to Use output data for more information. Convert textures to tensors Examples of converting textures to tensors. Refer to Create input for a model for more information. Copy a texture tensor to the screen An example of using TextureConverter.RenderToScreen to copy a texture tensor to the screen. Refer to Use output data for more information. Encrypt a model Example of serializing an encrypted model to disk using a custom editor window and loading that encrypted model at runtime. Refer to Encrypt a model for more information. Quantize a model Example of serializing a quantized model to disk using a custom editor window and loading that quantized model at runtime. Refer to Quantize a model for more information. Read output asynchronously Examples of reading the output from a model asynchronously using compute shaders. Refer to Read output from a model asynchronously for more information. Run a model a layer at a time An example of using ScheduleIterable to run a model a layer a time. Refer to Run a model for more information. Run a model Examples of running models with different numbers of inputs and outputs. Refer to Run a model for more information. Use the functional API with an existing model An example of using the functional API to extend an existing model. Refer to Edit a model for more information. Use a compute buffer An example of using a compute shader to write data to a tensor on the GPU. Use Burst to write data An example of using Burst to write data to a tensor in the Job system. Use tensor indexing methods Examples of using tensor indexing methods to get and set tensor values. Additional resources Unity Discussions group for the Sentis beta Understand the Sentis workflow Understand models in Sentis Tensor fundamentals in Sentis Supported models In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 10 ---
Encrypt a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Encrypt a model Encrypt a model so that only a user with the correct key can read the model description and weights from disk. You can encrypt a Sentis model to disk using the ModelWriter and ModelLoader APIs. Encrypt a model and save to disk To encrypt a model and save it to disk, follow these steps, typically in the Unity Editor before building and distributing your project: Get a Sentis model by importing an ONNX file or using the Sentis model API. Create a Stream object for the encrypted model using a cryptography API with your key. Encrypt and write the serialized model to the stream using the ModelWriter.Save method. Certain types of streams such as MemoryStream may not be compatible with large models over 2GB. Decrypt a model from disk To decrypt a model from disk, follow these steps, usually before running the model: Create a Stream object for the encrypted model using a cryptography API with your key. Decrypt and deserialize the model using the ModelLoader.Load method. Example using AES Encryption The following code samples demonstrate how to serialize a runtime model to disk, encrypt it using the Advanced Encryption Standard (AES), and how to decrypt the encrypted model for inference. This code sample uses AES encryption in C# to encrypt a model. using System.IO;
using System.Security.Cryptography;
using Unity.Sentis;

void SaveModelAesEncrypted(Model model, string path, byte[] key)
{
    // Create a `FileStream` with the path of the encrypted asset
    using FileStream fileStream = new FileStream(path, FileMode.OpenOrCreate);
    using Aes aes = Aes.Create();
    aes.Key = key;

    byte[] iv = aes.IV;

    // Write the initialization vector to the file
    fileStream.Write(iv, 0, iv.Length);

    // Create a `CryptoStream` that writes to the `FileStream`
    using CryptoStream cryptoStream = new CryptoStream(fileStream, aes.CreateEncryptor(), CryptoStreamMode.Write);

    // Serialize the model to the `CryptoStream`
    ModelWriter.Save(cryptoStream, model);
} This code sample uses AES decryption in C# to decrypt a model. using System.IO;
using System.Security.Cryptography;
using Unity.Sentis;

Model LoadModelAesEncrypted(string path, byte[] key)
{
    // Create a `FileStream` with the path of the encrypted asset
    using var fileStream = new FileStream(path, FileMode.Open);
    using Aes aes = Aes.Create();

    // Read the initialization vector from the file
    byte[] iv = new byte[aes.IV.Length];
    int numBytesToRead = aes.IV.Length;
    int numBytesRead = 0;
    while (numBytesToRead > 0)
    {
        int n = fileStream.Read(iv, numBytesRead, numBytesToRead);
        if (n == 0) break;

        numBytesRead += n;
        numBytesToRead -= n;
    }

    // Create a `CryptoStream` that reads from the `FileStream`
    using CryptoStream cryptoStream = new CryptoStream(fileStream, aes.CreateDecryptor(key, iv), CryptoStreamMode.Read);

    // Deserialize the model from the `CryptoStream`
    return ModelLoader.Load(cryptoStream);
} Refer to the Encrypt a model example in the sample scripts for an example. In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 11 ---
Create input for a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Create input for a model A model requires input tensors with certain shapes and data types. Use this information to find model inputs and create input tensors for your model. Understand the required input Before you can create input tensors for a model, you must first understand the shape and data types of the model inputs. For more information, refer to Model inputs . The TensorShape of the Tensor you create must be compatible with the DynamicTensorShape , which defines the shape of the model input. Convert an array to a tensor To create a CPU tensor from a one-dimensional data array, follow these steps: Create a TensorShape object that has the length of each axis. Create a Tensor<T> object with the shape and the data array. For example, the following code creates a tensor for a model that takes an input tensor of shape 3 × 1 × 3. using UnityEngine;
using Unity.Sentis;

public class ConvertArrayToTensor : MonoBehaviour
{
    void Start()
    {
        // Create a data array with 9 values
        float[] data = new float[] { 1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f };

        // Create a 3D tensor shape with size 3 × 1 × 3
        TensorShape shape = new TensorShape(3, 1, 3);

        // Create a new tensor from the array
        Tensor<float> tensor = new Tensor<float>(shape, data);
    }
} Create an empty tensor You can create a CPU tensor with its memory 0-initialized as follows: var tensor = new Tensor<int>(new TensorShape(1), clearOnInit: true); The parameter clearOnInit determines whether the resulting tensor memory should be 0-initialized. If it is unimportant what the initial data of the tensor is, set clearOnInit to false to avoid this additional operation. Pass inputs to a worker If a model needs multiple input tensors, do one of the following: Call SetInput on the worker for each input, and then call Schedule on the worker with no arguments. Call Schedule on the worker with an array of the desired inputs. worker.SetInput("x", xTensor);
worker.SetInput("y", yTensor);
worker.Schedule(); worker.Schedule(xTensor, yTensor); To avoid garbage collection due to the params array creation, you can pre-allocate the input array. var inputs = new [] { xTensor, yTensor };
//...
worker.Schedule(inputs); Edit a model Use the functional API to add operations to your model inputs. For more information, refer to Edit a model . Additional resources Tensor fundamentals Edit a model Convert a texture to a tensor In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 12 ---
Supported ONNX operators | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Supported ONNX operators When you import a model, each Open Neural Network Exchange (ONNX) operator in the model graph becomes a Sentis layer. A Sentis layer has the same name as the ONNX operator, unless the table shows the operator maps to a different layer. Refer to How Sentis optimizes a model for more information. Supported ONNX operators The table below shows which ONNX operators Sentis supports, and which data types Sentis supports for each backend type . Name Supported data types with BackendType.CPU Supported data types with BackendType.GPUCompute Supported data types with BackendType.GPUPixel Notes Abs float, int float, int float, int Acos float float float Acosh float float float Add float, int float, int float, int And int int int ArgMax float, int float, int float, int ArgMin float, int float, int float, int Asin float float float Asinh float float float Atan float float float Atanh float float float AveragePool float float (1D and 2D only) float (1D and 2D only) The ceil_mode and count_include_pad parameters aren't supported. BatchNormalization float float float The momentum , spatial and training_mode parameters aren't supported. Bernoulli float, int float, int float, int Cast float, int, short float, int, short float, int, short CastLike float, int, short float, int, short float, int, short Ceil float float float Celu float float float Clip float, int float, int float, int Compress float, int Not supported Not supported Concat float, int float, int float, int Constant - - - The sparse_value parameter isn't supported. ConstantOfShape float, int float, int float, int Conv float float (1D, 2D and 3D) float (1D, 2D and 3D) ConvTranspose float float (1D, 2D and 3D) float (1D, 2D and 3D) The dilations , group and output_shape parameters aren't supported. Cos float float float Cosh float float float CumSum float, int float, int float, int DepthToSpace float float float Div float, int float, int float, int Dropout - - - The operator maps to the Sentis layer Identity Einsum float float (1 or 2 inputs only) Not supported Elu float float float Equal float, int float, int float, int Erf float float float Exp float float float Expand float, int float, int float, int Flatten float, int float, int float, int Floor float float float Gather float, int float, int float, int GatherElements float, int float, int float, int GatherND float, int float, int float, int Gemm float float float GlobalAveragePool float float float GlobalMaxPool float float float Greater float, int float, int float, int GreaterOrEqual float, int float, int float, int GridSample float float float Hardmax float float float HardSigmoid float float float HardSwish float float float Identity float, int float, int float, int InstanceNormalization float float float IsInf float float float (Infs not supported) IsNaN float float float (NaNs not supported) LayerNormalization float float float LeakyRelu float float float Less float, int float, int float, int LessOrEqual float, int float, int float, int Log float float float LogSoftmax float float float LRN float Not supported Not supported LSTM float float Not supported MatMul float float float Max float, int float, int float, int MaxPool float float (1D and 2D only) float (1D and 2D only) The ceil_mode , dilations and storage_order parameters aren't supported. Mean float float float The operator maps to the Sentis layers Add and ScalarMad . Min float, int float, int float, int Mod float, int float, int float, int Mul float, int float, int float, int Multinomial float Not supported Not supported Neg float, int float, int float, int NonMaxSuppression float float Not supported NonZero float, int Not supported Not supported Not int int int OneHot float, int float, int float, int Or int int int Pad float, int float, int float, int Pow float, int float, int float, int PRelu float float float RandomNormal float float float RandomNormalLike float float float RandomUniform float float float RandomUniformLike float float float Range float, int float, int float, int Reciprocal float float float ReduceL1 float, int float, int float, int ReduceL2 float float float ReduceLogSum float float float ReduceLogSumExp float float float ReduceMax float, int float, int float, int ReduceMean float float float ReduceMin float, int float, int float, int ReduceProd float, int float, int float, int ReduceSum float, int float, int float, int ReduceSumSquare float, int float, int float, int Relu float float float Reshape float, int float, int float, int Resize float float float The cubic_coeff_a , exclude_outside , extrapolation_value and roi parameters aren't supported. RoiAlign float float float The coordinate_transformation_mode parameter isn't supported. Round float float float Scatter (deprecated) float, int float, int float, int The operator maps to the Sentis layer ScatterElements . ScatterElements float, int float, int (no ScatterReductionMode) float, int ScatterND float, int float, int float, int Selu float float float Shape - - - The operator returns a CPU tensor without downloading the input tensor. Shrink float float float Sigmoid float float float Sign float, int float, int float, int Sin float float float Sinh float float float Size - - - The operator returns a CPU tensor without downloading the input tensor. Slice float, int float, int float, int Softmax float float float Softplus float float float Softsign float float float SpaceToDepth float float float Split float, int float, int float, int Sqrt float float float Squeeze float, int float, int float, int Sub float, int float, int float, int Sum float, int float, int float, int The operator maps to the Sentis layer Add . Tan float float float Tanh float float float ThresholdedRelu float float float Tile float, int float, int float, int TopK float float Not supported Transpose float, int float, int float, int Trilu float, int float, int float, int Unsqueeze float, int float, int float, int Upsample (deprecated) float float float The operator maps to the Sentis layer Resize . Where float, int float, int float, int Xor int int int Sentis-only layers Sentis might create the following layers when it optimizes the model . Name Supported data types with BackendType.CPU Supported data types with BackendType.GPUCompute Supported data types with BackendType.GPUPixel Dense float float float DequantizeUint8 byte byte byte Gelu float float float GeluFast float float float MatMul2D float float float MoveDim float, int float, int float, int Narrow float, int float, int float, int RandomChoice float, int float, int float, int Relu6 float float float ScalarMad float, int float, int float, int Select float, int float, int float, int SliceSet float, int float, int float, int Square float, int float, int float, int Swish float float float ScaleBias float float float Unsupported operators The following ONNX operators aren't supported in the current version of Sentis. BitShift ConcatFromSequence ConvInteger DequantizeLinear Det DynamicQuantizeLinear EyeLike If GRU Loop LpPool MatMulInteger MaxUnpool MeanVarianceNormalization NegativeLogLikelihoodLoss Optional OptionalGetElement OptionalHasElement QLinearConv QLinearMatMul QuantizeLinear ReverseSequence RNN Scan SequenceAt SequenceConstruct SequenceEmpty SequenceErase SequenceInsert SequenceLength SoftmaxCrossEntropyLoss SplitToSequence StringNormalizer TfIdfVectorizer Unique Additional resources ONNX operator schemas Export an ONNX file from a machine learning framework Profile a model Supported functional methods In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 13 ---
Tensor fundamentals in Sentis | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Tensor fundamentals in Sentis In Sentis, data is input and output using multi-dimensional arrays called tensors. These tensors function similarly to those in TensorFlow, PyTorch, and other machine learning frameworks. Tensors can have up to eight dimensions. If a tensor has zero dimensions, it contains a single value called a scalar. You can create the following types of tensor: Tensor<float> , which stores the data as floats. Tensor<int> , which stores the data as ints. For more information, refer to Create and modify tensors . Memory layout Sentis stores tensors in memory in row-major order, so the values of the last dimension are stored adjacently. Here's an example of a 2 × 2 × 3 tensor with the values 0 to 11 and how Sentis stores the tensor in memory. Format A model usually needs an input tensor in a certain format. For example, a model that processes images might need a 3-channel 240 × 240 image in one of the following formats: 1 × 240 × 240 × 3, where the order of the dimensions is batch size, height, width, channels (NHWC) 1 × 3 × 240 × 240, where the order of the dimensions is batch size, channels, height, width (NCHW) If your tensor doesn't match the format the model needs, you might get unexpected results. You can use the Sentis functional API to convert a tensor to a different format. For more information, refer to Edit a model . To convert a texture to a tensor in a specific format, refer to Create input for a model . Memory location Sentis stores tensor data in either graphics processing unit (GPU) memory or central processing unit (CPU) memory, depending on the backend type you select. For example, if you use the BackendType.GPUCompute backend type, tensors are typically stored in GPU memory. Directly reading from and writing to tensor elements is only possible when the tensor is on the CPU, which can be slow. For better performance, it's more efficient to modify your model using the functional API. To prevent Sentis from performing a blocking readback and upload, use a compute shader, Burst, or a native array. This allows you to read from and write to the tensor data directly in memory. For more information, refer to Access tensor data directly . When you need to read an output tensor, perform an asynchronous readback. This prevents Sentis from blocking the main code thread while waiting for the model to finish and download the entire tensor. For more information, refer to Read output from a model asynchronously . Additional resources Understand the Sentis workflow Understand models in Sentis Create and modify tensors In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 14 ---
Quantize a Model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Quantize a Model Sentis imports model constants and weights as 32-bit values. If you wish to reduce the storage space required by your model on disk and in memory, you may consider model quantization. Quantization works by representing the weight values in a lower precision format, and then casting back to a high precision format before executing the operation. Quantization types Sentis currently supports the following quantization types. Quantization type Bits per value Description None 32 bit The values are stored in full precision. Float16 16 bit The values are cast to a 16 bit floating point value, model accuracy is often close to the original model. Uint8 8 bit The values are quantized linearly as fixed point values between the highest and lowest value in the data set, model accuracy may degrade severely Reducing the number of bits per value decreases the disk and memory usage of your model, while the inference speed remains almost unchanged. Sentis only quantizes float weights used as inputs to specific operations (Dense, MatMul, Conv, etc.), leaving integer constants unchanged. The impact on model accuracy varies depending on the model type. The most effective way to assess whether model quantization is appropriate for your needs is through testing. Quantizing a loaded model Use the ModelQuantizer API to quantize a model and the ModelWriter API to save the quantized model to disk. using Unity.Sentis;

void QuantizeAndSerializeModel(Model model, string path)
{
    // Sentis destructively edits the source model in memory when quantizing.
    ModelQuantizer.QuantizeWeights(QuantizationType.Float16, ref model);

    // Serialize the quantized model to a file.
    ModelWriter.Save(path, model);
} In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 15 ---
Use output data | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Use output data After you get the output from a model as a tensor, you can post-process the data to use it in your project. Download to a CPU tensor Use ReadbackAndClone or ReadbackAndCloneAsync to move a tensor on the graphics processing unit (GPU) to the central processing unit (CPU) to read it. Refer to read the tensor data asynchronously for best practices on how to do this efficiently. For example: var outputTensor = worker.Schedule(inputTensor).PeekOutput() as Tensor<float>;
var cpuTensor = outputTensor.ReadbackAndClone(); The returned tensor will be a cpu read-writable copy of the output tensor. Refer to tensor fundamentals and access tensor data directly for details on how to properly index and access the tensor. Convert to a render texture To convert a tensor to a render texture, use one of the following methods: TextureConverter.ToTexture to output tensor data to a render texture. Sentis creates a new render texture to do this. TextureConverter.RenderToTexture to write tensor data to an existing render texture you provide. When you use TextureConverter.ToTexture , Sentis uses the tensor shape to determine the size and channels of the render texture. If the tensor doesn't match the render texture, Sentis makes the following adjustments: Samples the tensor linearly if the dimensions don't match. Removes channels from the end if the render texture has fewer channels than the tensor. Sets values in RGB channels to 0 and values in the alpha channel to 1 if the render texture has more channels than the tensor. For working examples, refer to the Convert tensors to textures example in the sample scripts . ToTexture example // Define an empty render texture
public RenderTexture rt;

void Start()
{
    ...

    // Get the output of the model as a tensor
    Tensor<float> outputTensor = worker.Schedule(inputTensor).PeekOutput() as Tensor<float>;

    // Convert the tensor to a texture and store it in the uninstantiated render texture
    rt = TextureConverter.ToTexture(outputTensor);
} You can use the parameters in TextureConverter.ToTexture to override the width, height, and number of channels of a texture. For example: // Set a property to -1 to use the default value
    rt = TextureConverter.ToTexture(outputTensor, width: 4, height: 12, channels: -1); RenderToTexture example // Instantiate the render texture
public RenderTexture rt = new RenderTexture(24, 32, 0, RenderTextureFormat.ARGB32);
Worker worker;
Tensor inputTensor;

void Start()
{
    // Get the output of the model as a tensor
    Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;

    // Convert the tensor to a texture and store it in the render texture
    TextureConverter.RenderToTexture(outputTensor, rt);
} Note Avoid unnecessary resource allocation, and don't re-allocate Tensors or Textures every frame.
The TextureConverter provides versions of all methods that to do in-place copies between pre-allocated Textures and Tensors. Copy to the screen To copy an output tensor to the screen, follow these steps: Set the Camera.targetTexture property of Camera.main to null. Create a script and attach it to the Camera. In the script, use TextureConverter.RenderToScreen in an event function such as OnRenderImage . If the image is too bright, the output tensor might be using values from 0 to 255 instead of 0 to 1 . You can use Edit a model to remap the values in the output tensor before calling RenderToScreen . The following script uses a model to change a texture, then copies the result to the screen. Set modelAsset to one of the style transfer models from ONNX and inputImage to a texture. Check the Texture import settings to make sure the texture matches the shape and layout the model needs. using UnityEngine;
using Unity.Sentis;

public class StyleTransfer : MonoBehaviour
{
    public ModelAsset modelAsset;
    public Model runtimeModel;
    public Texture2D inputImage;
    public RenderTexture outputTexture;

    Worker worker;
    Tensor<float> inputTensor;

    void Start()
    {
        var sourceModel = ModelLoader.Load(modelAsset);
        var graph = new FunctionalGraph();
        var input = graph.AddInput(sourceModel, 0);
        var output = Functional.Forward(sourceModel, input)[0];
        // rescale output of source model
        output /= 255f;
        var runtimeModel = graph.Compile(output);

        worker = new Worker(runtimeModel, BackendType.GPUCompute);
        inputTensor = new Tensor<float>(new TensorShape(1, 3, 256, 256));
    }

    void OnRenderImage(RenderTexture source, RenderTexture destination)
    {
        // Create the input tensor from the texture
        TextureConverter.ToTensor(inputImage, inputTensor, new TextureTransform());

        // Run the model and get the output as a tensor
        worker.Schedule(inputTensor);
        Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;

        // Copy the rescaled tensor to the screen as a texture
        TextureConverter.RenderToScreen(outputTensor);
    }

    void OnDisable()
    {
        worker.Dispose();
    }
} When using Universal Render Pipeline (URP) or the High-Definition Render Pipeline (HDRP), call RenderToScreen in the RenderPipelineManager.endFrameRendering or RenderPipelineManager.endContextRendering callbacks. For more information, refer to Rendering.RenderPipelineManager . For an example, refer to the Copy a texture tensor to the screen example in the sample scripts . Additional resources Get output from a model Create and modify tensors In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 16 ---
Export and convert a file to ONNX | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Export and convert a file to ONNX Sentis currently only supports importing files in the Open Neural Network Exchange (ONNX) format, or the Sentis serialized format. If your model is not in one of these formats, you must convert it. The following sections describe how to export a file in ONNX format and convert files from a different format to ONNX. Export an ONNX file from a machine learning framework You can export a model from most machine learning frameworks in ONNX format. Refer to the following documentation to understand how to export files in ONNX format from common machine learning frameworks: Exporting a model from PyTorch to ONNX on the PyTorch website. Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX on the ONNX GitHub repository. To make sure the exported model is compatible with Sentis, set the ONNX opset version to 15. Refer to import a model file for more information about ONNX compatibility. Convert TensorFlow files to ONNX Exporting files from TensorFlow involves two key file types: SavedModel and Checkpoints. Refer to the following sections to understand each file type and how to convert them to the ONNX format. Model files TensorFlow saves models in SavedModel files, which contain a complete TensorFlow program, including trained parameters and computation. SavedModels have the .pb file extension. Refer to Using the SavedModel format (TensorFlow documentation) for more information on SavedModels. To generate an ONNX file from a SavedModel, you can use the tf2onnx tool. This is a command line tool that works best when you provide the full path names. Checkpoints Checkpoints (TensorFlow documentation) contain only the parameters of the model. Checkpoints in TensorFlow can consist up of two file formats: A file to store the graph, with the extension .ckpt.meta. A file to store the weights, with the extension .ckpt. If you have both the graph and weight file types, you can use the tf2onnx tool to create an ONNX file. If you only have the .ckpt file, you will need to find the Python code that constructs the model and loads in the weights. After that, you can proceed to export the model to ONNX . Convert PyTorch files to ONNX Refer to the following sections to understand how to convert PyTorch files to the ONNX format. Model files PyTorch model files usually have the .pt file extension. To export a model file to ONNX, refer to the links in the following instructions: Load the model in Python. Export the model as an ONNX file. It is recommended to use Opset 15 or higher when exporting your model. If your .pt file doesn't contain the model graph, you must find the Python code that constructs the model and loads in the weights. After that, you can proceed to export the model to ONNX . Checkpoints You can create Checkpoints in PyTorch to save the state of your model at any instance of time. Checkpoint files are usually denoted with the .tar or .pth extension. To convert a checkpoint file to ONNX, you must find the Python code which constructs the model and loads in the weights. After that, you can proceed to export the model to ONNX . Additional resources Open Neural Network Exchange Supported ONNX operators Profile a model Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 17 ---
Read output from a model asynchronously | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Read output from a model asynchronously After you schedule a model and access an output tensor from PeekOutput , the following are true: Sentis might not have finished calculating the final tensor data, so there's pending scheduled work. If you use a graphics processing unit (GPU) backend, the calculated tensor data might be on the GPU. This requires a read back to copy the data to the central processing unit (CPU) in a readable format. If either of these conditions is true, ReadbackAndClone or CompleteAllPendingOperations methods block the main thread until the operations are complete. To avoid this, follow these two methods to use asynchronous readback: Use the awaitable ReadbackAndCloneAsync method. Sentis returns a CPU copy of the input tensor in a non blocking way. using Unity.Sentis;
using UnityEngine;

public class AsyncReadbackCompute : MonoBehaviour
{
    [SerializeField]
    ModelAsset modelAsset;

    Tensor<float> m_Input;
    Worker m_Worker;

    async void OnEnable()
    {
        var model = ModelLoader.Load(modelAsset);
        m_Input = new Tensor<float>(new TensorShape(1, 1), new[] { 43.0f });
        m_Worker = new Worker(model, BackendType.GPUCompute);
        m_Worker.Schedule(m_Input);

        // Peek the value from Sentis, without taking ownership of the tensor
        var outputTensor = m_Worker.PeekOutput() as Tensor<float>;
        var cpuCopyTensor = await outputTensor.ReadbackAndCloneAsync();

        Debug.Assert(cpuCopyTensor[0] == 42);
        Debug.Log($"Output tensor value {cpuCopyTensor[0]}");
        cpuCopyTensor.Dispose();
    }

    void OnDisable()
    {
        m_Input.Dispose();
        m_Worker.Dispose();
    }
} Use a polling mechanism with the ReadbackRequest and Tensor.IsReadbackRequestDone methods. bool inferencePending = false;
Tensor<float> outputTensor;

void OnUpdate()
{
    if (!inferencePending)
    {
        m_Worker.Schedule(m_Input);
        outputTensor = m_Worker.PeekOutput() as Tensor<float>;

        // Trigger a non-blocking readback request
        outputTensor.ReadbackRequest();
        inferencePending = true;
    }
    else if (inferencePending && outputTensor.IsReadbackRequestDone())
    {
        // m_Output is now downloaded to the cpu. Using ReadbackAndClone or ToReadOnlyArray will not be blocking
        var array = outputTensor.DownloadToArray();
        inferencePending = false;
    }
} Use an awaitable with a callback. bool inferencePending = false;

void Update()
{
    if (!inferencePending)
    {
        m_Worker.Schedule(m_Input);
        var outputTensor = m_Worker.PeekOutput() as Tensor<float>;
        inferencePending = true;

        var awaiter = outputTensor.ReadbackAndCloneAsync().GetAwaiter();
        awaiter.OnCompleted(() =>
        {
            var tensorOut = awaiter.GetResult();
            inferencePending = false;
            tensorOut.Dispose();
        });
    }
} Note To avoid a Tensor data mutation to a CPU tensor from calling ReadbackAndClone , call tensor.dataOnBackend.Download to get the data directly. This keeps the tensor.dataOnBackend on the given backend while providing a CPU copy. Be cautious with synchronization issues: if you re-schedule a worker, make a new download request. For an example, refer to the Read output asynchronously example in the sample scripts . Additional resources Tensor fundamentals Use output data Get output from any layer In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 18 ---
Create a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Create a model Create a runtime model by importing an ONNX model file or using the Sentis functional API. Page Description Understand models in Sentis Understand how Sentis optimizes models, and information about fixed and dynamic input dimensions. Export and convert a file to ONNX Export an ONNX file from a machine learning framework and convert other file types to the ONNX format. Import an ONNX file Import an ONNX file and create a runtime model. Supported models Understand the models Sentis supports and find a model for your project. Serialize a model Create a serialized model (.sentis file) in the StreamingAssets folder. Inspect a model Check the inputs, outputs and layers of a model. Create a new model Create a new runtime model with the Sentis functional API. Edit a model Make changes to an existing runtime model. Encrypt a model Encrypt and decrypt a Sentis model. Quantize a model Quantize the weights of a Sentis model. Supported ONNX operators Understand which ONNX operators Sentis supports. Supported functional methods Understand which functional methods Sentis supports. Model Asset Import Settings reference Understand the settings and properties of an imported model in the Unity Inspector. In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 19 ---
Edit a model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Edit a model Use the Sentis Functional API to edit a model after you create or load it. Preprocess inputs or postprocess outputs Sometimes your model expects inputs or returns outputs in a format that doesn't match your tensor data. Use the Functional API to prepend operations to inputs, append operations to outputs, or to easily add or remove inputs or outputs. In the following example, the mnist-8 model is adjusted to return the softmax of the output. using UnityEngine;
using Unity.Sentis;

public class AddOutput : MonoBehaviour
{
    ModelAsset modelAsset;

    void Start()
    {
        // Load the source model from the model asset
        Model model = ModelLoader.Load(modelAsset);

        // Define the functional graph of the model.
        var graph = new FunctionalGraph();

        // Set the inputs of the graph from the original model inputs and return an array of functional tensors
        var inputs = graph.AddInputs(model);

        // Apply the model forward function to the inputs to get the source model functional outputs.
        // Sentis will destructively change the loaded source model. To avoid this at the expense of
        // higher memory usage and compile time, use the Functional.ForwardWithCopy method.
        FunctionalTensor[] outputs = Functional.Forward(model, inputs);

        // Calculate the softmax of the first output with the functional API.
        FunctionalTensor softmaxOutput = Functional.Softmax(outputs[0]);

        // Build the model from the graph using the `Compile` method with the desired outputs.
        var modelWithSoftmax = graph.Compile(softmaxOutput);
    }
} Sentis runs model optimization on models you create using the Functional API. The Sentis operations used may, therefore, look different than expected. Note that Compile is a slow operation that uses a lot of memory. It is recommended to run this offline and to serialize the computed model.
See Serialize A Model for details. Additional resources Supported functional methods In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 20 ---
Get started | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Get started Use the following topics to get started with using Sentis. Tip If you are new to Sentis or just getting started with machine learning models, you can quickly begin by using a sample project . This will allow you to understand the full Sentis workflow and explore the key use cases of Sentis. Page Description Install Sentis Install the Sentis package in Unity. Understand the Sentis workflow Learn how to use Sentis. Workflow example Understand a simple example of the Sentis workflow. Samples Find and use the Sentis samples. In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 21 ---
Import a model file | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Import a model file To import an ONNX model file, drag the file from your computer into the Assets folder of the Project window. If your model has external weights files, put them in the same directory as the model file so that Sentis imports them automatically. To learn about the models Sentis supports, refer to Supported models . Create a runtime model To use an imported model, you must use ModelLoader.Load to create a runtime Model object. using UnityEngine;
using Unity.Sentis;

public class CreateRuntimeModel : MonoBehaviour
{
    public ModelAsset modelAsset;
    Model runtimeModel;

    void Start()
    {
        runtimeModel = ModelLoader.Load(modelAsset);
    }
} You can then create an engine to run a model . Additional resources How Sentis optimizes a model Export an ONNX file from a machine learning framework Model Asset Import Settings Supported models In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

--- Page 22 ---
Run an imported model | Sentis | 2.1.0 Toggle navigation docs.unity3d.com Search Results for Show / Hide Table of Contents Run an imported model Use Sentis to run an imported model with input data and get the output data. Page Description How Sentis runs a model Understand how Sentis runs a model. Create input for a model Create input data for a model from an array or a texture. Convert a texture to a tensor Convert a texture to a tensor, and override texture shape and layout. Create an engine to run a model Create a worker, which is the engine that breaks the model down into executable tasks. Run a model Run a model in a single frame. Split inference over multiple frames Run a model a layer at a time. Use a command buffer Use a command buffer to create a queue of Sentis commands, then run the commands on the GPU later. Get output from a model Get and log the output data from a model. Read output asynchronously Read the output data from a model asynchronously. Use output data Convert the output data to an array or a texture, copy the data to the screen. Manage memory Call Dispose on workers and tensors. In This Article Back to top Copyright © 2024 Unity Technologies — Trademarks and terms of use Legal Privacy Policy Cookie Policy Do Not Sell or Share My Personal Information Your Privacy Choices (Cookie Settings)

"