small fixes

v1.15.1/.gitattributes

@@ -1,4 +0,0 @@
-*.so* filter=lfs diff=lfs merge=lfs -text
-onnxruntime-linux-armhf/lib/libonnxruntime.so filter=lfs diff=lfs merge=lfs -text
-onnxruntime-linux-armhf/lib/libonnxruntime.so.1.15.1 filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text

v1.15.1/onnxruntime-linux-armhf/GIT_COMMIT_ID

@@ -1 +0,0 @@
-baeece44ba075009c6bfe95891a8c1b3d4571cb3

v1.15.1/onnxruntime-linux-armhf/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) Microsoft Corporation
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

v1.15.1/onnxruntime-linux-armhf/Privacy.md

@@ -1,21 +0,0 @@
-# Privacy
-
-## Data Collection
-The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
-
-***
-
-### Private Builds
-No data collection is performed when using your private builds built from source code.
-
-### Official Builds
-ONNX Runtime does not maintain any independent telemetry collection mechanisms outside of what is provided by the platforms it supports. However, where applicable, ONNX Runtime will take advantage of platform-supported telemetry systems to collect trace events with the goal of improving product quality.
-
-Currently telemetry is only implemented for Windows builds and is turned **ON** by default in the official builds distributed in their respective package management repositories ([see here](../README.md#binaries)). This may be expanded to cover other platforms in the future. Data collection is implemented via 'Platform Telemetry' per vendor platform providers (see [telemetry.h](../onnxruntime/core/platform/telemetry.h)).
-
-#### Technical Details
-The Windows provider uses the [TraceLogging](https://docs.microsoft.com/en-us/windows/win32/tracelogging/trace-logging-about) API for its implementation. This enables ONNX Runtime trace events to be collected by the operating system, and based on user consent, this data may be periodically sent to Microsoft servers following GDPR and privacy regulations for anonymity and data access controls. 
-
-Windows ML and onnxruntime C APIs allow Trace Logging to be turned on/off (see [API pages](../README.md#api-documentation) for details).
-For information on how to enable and disable telemetry, see [C API: Telemetry](./C_API.md#telemetry). 
-There are equivalent APIs in the C#, Python, and Java language bindings as well.

v1.15.1/onnxruntime-linux-armhf/README.md

@@ -1,57 +0,0 @@
-<p align="center"><img width="50%" src="docs/images/ONNX_Runtime_logo_dark.png" /></p>
-
-**ONNX Runtime is a cross-platform inference and training machine-learning accelerator**.
-
-**ONNX Runtime inference** can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. [Learn more &rarr;](https://www.onnxruntime.ai/docs/#onnx-runtime-for-inferencing)
-
-**ONNX Runtime training** can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts. [Learn more &rarr;](https://www.onnxruntime.ai/docs/#onnx-runtime-for-training)
-
-
-## Get Started & Resources
-
-* **General Information**: [onnxruntime.ai](https://onnxruntime.ai)
-
-* **Usage documention and tutorials**: [onnxruntime.ai/docs](https://onnxruntime.ai/docs)
-
-* **YouTube video tutorials**: [youtube.com/@ONNXRuntime](https://www.youtube.com/@ONNXRuntime)
-
-* [**Upcoming Release Roadmap**](https://github.com/microsoft/onnxruntime/wiki/Upcoming-Release-Roadmap)
-
-* **Companion sample repositories**: 
-  - ONNX Runtime Inferencing: [microsoft/onnxruntime-inference-examples](https://github.com/microsoft/onnxruntime-inference-examples)
-  - ONNX Runtime Training: [microsoft/onnxruntime-training-examples](https://github.com/microsoft/onnxruntime-training-examples)
-
-
-## Build Pipeline Status
-|System|Inference|Training|
-|---|---|---|
-|Windows|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Windows%20CPU%20CI%20Pipeline?label=Windows+CPU)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=9)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Windows%20GPU%20CI%20Pipeline?label=Windows+GPU)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=10)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Windows%20GPU%20TensorRT%20CI%20Pipeline?label=Windows+GPU+TensorRT)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=47)||
-|Linux|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Linux%20CPU%20CI%20Pipeline?label=Linux+CPU)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=11)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Linux%20CPU%20Minimal%20Build%20E2E%20CI%20Pipeline?label=Linux+CPU+Minimal+Build)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=64)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Linux%20GPU%20CI%20Pipeline?label=Linux+GPU)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=12)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Linux%20GPU%20TensorRT%20CI%20Pipeline?label=Linux+GPU+TensorRT)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=45)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Linux%20OpenVINO%20CI%20Pipeline?label=Linux+OpenVINO)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=55)|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/orttraining-linux-ci-pipeline?label=Linux+CPU+Training)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=86)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/orttraining-linux-gpu-ci-pipeline?label=Linux+GPU+Training)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=84)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/orttraining/orttraining-ortmodule-distributed?label=Training+Distributed)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=148)|
-|Mac|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/MacOS%20CI%20Pipeline?label=MacOS+CPU)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=13)||
-|Android|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/Android%20CI%20Pipeline?label=Android)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=53)||
-|iOS|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/iOS%20CI%20Pipeline?label=iOS)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=134)||
-|Web|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/ONNX%20Runtime%20Web%20CI%20Pipeline?label=Web)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=161)||
-|Other|[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/onnxruntime-binary-size-checks-ci-pipeline?repoName=microsoft%2Fonnxruntime&label=Binary+Size+Check)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=187&repoName=microsoft%2Fonnxruntime)<br>[![Build Status](https://dev.azure.com/onnxruntime/onnxruntime/_apis/build/status/onnxruntime-python-checks-ci-pipeline?label=Python+Checks)](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=164)||
-
-
-## Data/Telemetry
-
-Windows distributions of this project may collect usage data and send it to Microsoft to help improve our products and services. See the [privacy statement](docs/Privacy.md) for more details.
-
-## Contributions and Feedback
-
-We welcome contributions! Please see the [contribution guidelines](CONTRIBUTING.md).
-
-For feature requests or bug reports, please file a [GitHub Issue](https://github.com/Microsoft/onnxruntime/issues).
-
-For general discussion or questions, please use [GitHub Discussions](https://github.com/microsoft/onnxruntime/discussions).
-
-## Code of Conduct
-
-This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
-For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
-or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
-
-## License
-
-This project is licensed under the [MIT License](LICENSE).

v1.15.1/onnxruntime-linux-armhf/VERSION_NUMBER

@@ -1 +0,0 @@
-1.15.1

v1.15.1/onnxruntime-linux-armhf/include/cpu_provider_factory.h

@@ -1,19 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-#include "onnxruntime_c_api.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-/**
- * \param use_arena zero: false. non-zero: true.
- */
-ORT_EXPORT
-ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_CPU, _In_ OrtSessionOptions* options, int use_arena)
-ORT_ALL_ARGS_NONNULL;
-
-#ifdef __cplusplus
-}
-#endif

v1.15.1/onnxruntime-linux-armhf/include/onnxruntime_cxx_inline.h

@@ -1,2035 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-// Do not include this file directly. Please include "onnxruntime_cxx_api.h" instead.
-// If interested in trying out features of the new experimental C++ API, include "experimental_onnxruntime_cxx_api.h" instead.
-//
-// These are the inline implementations of the C++ header APIs. They're in this separate file as to not clutter
-// the main C++ file with implementation details.
-
-namespace Ort {
-
-namespace detail {
-inline void ThrowStatus(const Status& st) {
-  std::string error_message = st.GetErrorMessage();
-  OrtErrorCode error_code = st.GetErrorCode();
-  ORT_CXX_API_THROW(std::move(error_message), error_code);
-}
-}  // namespace detail
-
-inline void ThrowOnError(OrtStatus* ort_status) {
-  if (ort_status) {
-    Ort::Status st(ort_status);
-    detail::ThrowStatus(st);
-  }
-}
-
-inline void ThrowOnError(const Status& st) {
-  if (st) {
-    detail::ThrowStatus(st);
-  }
-}
-
-inline Status::Status(OrtStatus* status) noexcept : Base<OrtStatus>{status} {
-}
-
-inline Status::Status(const std::exception& e) noexcept {
-  p_ = GetApi().CreateStatus(ORT_FAIL, e.what());
-}
-
-inline Status::Status(const Exception& e) noexcept {
-  p_ = GetApi().CreateStatus(e.GetOrtErrorCode(), e.what());
-}
-
-inline Status::Status(const char* message, OrtErrorCode code) noexcept {
-  p_ = GetApi().CreateStatus(code, message);
-}
-
-inline std::string Status::GetErrorMessage() const {
-  std::string message(GetApi().GetErrorMessage(p_));
-  return message;
-}
-
-inline OrtErrorCode Status::GetErrorCode() const {
-  return GetApi().GetErrorCode(p_);
-}
-
-inline bool Status::IsOK() const noexcept {
-  return (p_ == nullptr);
-}
-
-// This template converts a C++ type into it's ONNXTensorElementDataType
-template <typename T>
-struct TypeToTensorType;
-template <>
-struct TypeToTensorType<float> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;
-};
-template <>
-struct TypeToTensorType<Float16_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16;
-};
-template <>
-struct TypeToTensorType<BFloat16_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16;
-};
-template <>
-struct TypeToTensorType<double> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE;
-};
-template <>
-struct TypeToTensorType<int8_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8;
-};
-template <>
-struct TypeToTensorType<int16_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16;
-};
-template <>
-struct TypeToTensorType<int32_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32;
-};
-template <>
-struct TypeToTensorType<int64_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64;
-};
-template <>
-struct TypeToTensorType<uint8_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8;
-};
-template <>
-struct TypeToTensorType<uint16_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16;
-};
-template <>
-struct TypeToTensorType<uint32_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32;
-};
-template <>
-struct TypeToTensorType<uint64_t> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64;
-};
-template <>
-struct TypeToTensorType<bool> {
-  static constexpr ONNXTensorElementDataType type = ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL;
-};
-
-inline MemoryAllocation::MemoryAllocation(OrtAllocator* allocator, void* p, size_t size)
-    : allocator_(allocator), p_(p), size_(size) {
-}
-
-inline MemoryAllocation::~MemoryAllocation() {
-  if (p_ != nullptr) {
-    // We do not throw out of destructor
-    auto ret = GetApi().AllocatorFree(allocator_, p_);
-    static_cast<void>(ret);
-  }
-}
-
-inline MemoryAllocation::MemoryAllocation(MemoryAllocation&& o) noexcept : allocator_(nullptr), p_(nullptr), size_(0) {
-  *this = std::move(o);
-}
-
-inline MemoryAllocation& MemoryAllocation::operator=(MemoryAllocation&& o) noexcept {
-  OrtAllocator* alloc = nullptr;
-  void* p = nullptr;
-  size_t sz = 0;
-
-  // Swap out this
-  std::swap(alloc, allocator_);
-  std::swap(p, p_);
-  std::swap(sz, size_);
-
-  // Swap with incoming
-  std::swap(allocator_, o.allocator_);
-  std::swap(p_, o.p_);
-  std::swap(size_, o.size_);
-
-  // Destroy this instance if needed
-  MemoryAllocation this_alloc(alloc, p, sz);
-  return *this;
-}
-
-namespace detail {
-
-template <typename T>
-inline void* AllocatorImpl<T>::Alloc(size_t size) {
-  void* out;
-  ThrowOnError(GetApi().AllocatorAlloc(this->p_, size, &out));
-  return out;
-}
-
-template <typename T>
-inline MemoryAllocation AllocatorImpl<T>::GetAllocation(size_t size) {
-  void* out;
-  ThrowOnError(GetApi().AllocatorAlloc(this->p_, size, &out));
-  MemoryAllocation result(this->p_, out, size);
-  return result;
-}
-
-template <typename T>
-inline void AllocatorImpl<T>::Free(void* p) {
-  ThrowOnError(GetApi().AllocatorFree(this->p_, p));
-}
-
-template <typename T>
-inline ConstMemoryInfo AllocatorImpl<T>::GetInfo() const {
-  const OrtMemoryInfo* out;
-  ThrowOnError(GetApi().AllocatorGetInfo(this->p_, &out));
-  return ConstMemoryInfo{out};
-}
-
-}  // namespace detail
-
-inline AllocatorWithDefaultOptions::AllocatorWithDefaultOptions() {
-  ThrowOnError(GetApi().GetAllocatorWithDefaultOptions(&this->p_));
-}
-
-inline Allocator::Allocator(const Session& sess, const OrtMemoryInfo* mem_info) {
-  ThrowOnError(GetApi().CreateAllocator(sess, mem_info, &this->p_));
-}
-
-namespace detail {
-
-template <typename T>
-inline std::string MemoryInfoImpl<T>::GetAllocatorName() const {
-  const char* name = nullptr;
-  ThrowOnError(GetApi().MemoryInfoGetName(this->p_, &name));
-  return std::string(name);
-}
-
-template <typename T>
-inline OrtAllocatorType MemoryInfoImpl<T>::GetAllocatorType() const {
-  OrtAllocatorType type;
-  ThrowOnError(GetApi().MemoryInfoGetType(this->p_, &type));
-  return type;
-}
-
-template <typename T>
-inline int MemoryInfoImpl<T>::GetDeviceId() const {
-  int id = 0;
-  ThrowOnError(GetApi().MemoryInfoGetId(this->p_, &id));
-  return id;
-}
-
-template <typename T>
-inline OrtMemoryInfoDeviceType MemoryInfoImpl<T>::GetDeviceType() const {
-  OrtMemoryInfoDeviceType type;
-  GetApi().MemoryInfoGetDeviceType(this->p_, &type);
-  return type;
-}
-
-template <typename T>
-inline OrtMemType MemoryInfoImpl<T>::GetMemoryType() const {
-  OrtMemType type;
-  ThrowOnError(GetApi().MemoryInfoGetMemType(this->p_, &type));
-  return type;
-}
-
-template <typename T>
-template <typename U>
-inline bool MemoryInfoImpl<T>::operator==(const MemoryInfoImpl<U>& o) const {
-  int comp_result = 0;
-  ThrowOnError(Ort::GetApi().CompareMemoryInfo(this->p_, o, &comp_result));
-  return comp_result == 0;
-}
-
-}  // namespace detail
-
-inline MemoryInfo MemoryInfo::CreateCpu(OrtAllocatorType type, OrtMemType mem_type) {
-  OrtMemoryInfo* p;
-  ThrowOnError(GetApi().CreateCpuMemoryInfo(type, mem_type, &p));
-  return MemoryInfo(p);
-}
-
-inline MemoryInfo::MemoryInfo(const char* name, OrtAllocatorType type, int id, OrtMemType mem_type) {
-  ThrowOnError(GetApi().CreateMemoryInfo(name, type, id, mem_type, &this->p_));
-}
-
-namespace detail {
-template <typename T>
-inline std::vector<std::string> ConstIoBindingImpl<T>::GetOutputNames() const {
-  AllocatorWithDefaultOptions allocator;
-  return binding_utils::GetOutputNamesHelper(this->p_, allocator);
-}
-
-template <typename T>
-inline std::vector<std::string> ConstIoBindingImpl<T>::GetOutputNames(OrtAllocator* allocator) const {
-  return binding_utils::GetOutputNamesHelper(this->p_, allocator);
-}
-
-template <typename T>
-inline std::vector<Value> ConstIoBindingImpl<T>::GetOutputValues() const {
-  AllocatorWithDefaultOptions allocator;
-  return binding_utils::GetOutputValuesHelper(this->p_, allocator);
-}
-
-template <typename T>
-inline std::vector<Value> ConstIoBindingImpl<T>::GetOutputValues(OrtAllocator* allocator) const {
-  return binding_utils::GetOutputValuesHelper(this->p_, allocator);
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::BindInput(const char* name, const Value& value) {
-  ThrowOnError(GetApi().BindInput(this->p_, name, value));
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::BindOutput(const char* name, const Value& value) {
-  ThrowOnError(GetApi().BindOutput(this->p_, name, value));
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::BindOutput(const char* name, const OrtMemoryInfo* mem_info) {
-  ThrowOnError(GetApi().BindOutputToDevice(this->p_, name, mem_info));
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::ClearBoundInputs() {
-  GetApi().ClearBoundInputs(this->p_);
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::ClearBoundOutputs() {
-  GetApi().ClearBoundOutputs(this->p_);
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::SynchronizeInputs() {
-  ThrowOnError(GetApi().SynchronizeBoundInputs(this->p_));
-}
-
-template <typename T>
-inline void IoBindingImpl<T>::SynchronizeOutputs() {
-  ThrowOnError(GetApi().SynchronizeBoundOutputs(this->p_));
-}
-
-namespace binding_utils {
-inline std::vector<std::string> GetOutputNamesHelper(const OrtIoBinding* binding, OrtAllocator* allocator) {
-  std::vector<std::string> result;
-  auto free_fn = detail::AllocatedFree(allocator);
-  using Ptr = std::unique_ptr<void, decltype(free_fn)>;
-
-  char* buffer = nullptr;
-  size_t* lengths = nullptr;
-  size_t count = 0;
-  ThrowOnError(GetApi().GetBoundOutputNames(binding, allocator, &buffer, &lengths, &count));
-
-  if (count == 0) {
-    return result;
-  }
-
-  Ptr buffer_g(buffer, free_fn);
-  Ptr lengths_g(lengths, free_fn);
-
-  result.reserve(count);
-  for (size_t i = 0; i < count; ++i) {
-    auto sz = *lengths;
-    result.emplace_back(buffer, sz);
-    buffer += sz;
-    ++lengths;
-  }
-  return result;
-}
-
-inline std::vector<Value> GetOutputValuesHelper(const OrtIoBinding* binding, OrtAllocator* allocator) {
-  std::vector<Value> result;
-  size_t owned = 0;
-  size_t output_count = 0;
-  // Lambda to release the buffer when no longer needed and
-  // make sure that we destroy all instances on exception
-  auto free_fn = [&owned, &output_count, allocator](OrtValue** buffer) {
-    if (buffer) {
-      while (owned < output_count) {
-        auto* p = buffer + owned++;
-        GetApi().ReleaseValue(*p);
-      }
-      allocator->Free(allocator, buffer);
-    }
-  };
-  using Ptr = std::unique_ptr<OrtValue*, decltype(free_fn)>;
-
-  OrtValue** output_buffer = nullptr;
-  ThrowOnError(GetApi().GetBoundOutputValues(binding, allocator, &output_buffer, &output_count));
-  if (output_count == 0) {
-    return result;
-  }
-
-  Ptr buffer_g(output_buffer, free_fn);
-
-  result.reserve(output_count);
-  for (size_t i = 0; i < output_count; ++i) {
-    result.emplace_back(output_buffer[i]);
-    ++owned;
-  }
-  return result;
-}
-
-}  // namespace binding_utils
-}  // namespace detail
-
-inline IoBinding::IoBinding(Session& session) {
-  ThrowOnError(GetApi().CreateIoBinding(session, &this->p_));
-}
-
-inline ArenaCfg::ArenaCfg(size_t max_mem, int arena_extend_strategy, int initial_chunk_size_bytes, int max_dead_bytes_per_chunk) {
-  ThrowOnError(GetApi().CreateArenaCfg(max_mem, arena_extend_strategy, initial_chunk_size_bytes, max_dead_bytes_per_chunk, &p_));
-}
-
-inline ThreadingOptions::ThreadingOptions() {
-  ThrowOnError(GetApi().CreateThreadingOptions(&p_));
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalIntraOpNumThreads(int intra_op_num_threads) {
-  ThrowOnError(GetApi().SetGlobalIntraOpNumThreads(p_, intra_op_num_threads));
-  return *this;
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalInterOpNumThreads(int inter_op_num_threads) {
-  ThrowOnError(GetApi().SetGlobalInterOpNumThreads(p_, inter_op_num_threads));
-  return *this;
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalSpinControl(int allow_spinning) {
-  ThrowOnError(GetApi().SetGlobalSpinControl(p_, allow_spinning));
-  return *this;
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalDenormalAsZero() {
-  ThrowOnError(GetApi().SetGlobalDenormalAsZero(p_));
-  return *this;
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalCustomCreateThreadFn(OrtCustomCreateThreadFn ort_custom_create_thread_fn) {
-  ThrowOnError(GetApi().SetGlobalCustomCreateThreadFn(p_, ort_custom_create_thread_fn));
-  return *this;
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalCustomThreadCreationOptions(void* ort_custom_thread_creation_options) {
-  ThrowOnError(GetApi().SetGlobalCustomThreadCreationOptions(p_, ort_custom_thread_creation_options));
-  return *this;
-}
-
-inline ThreadingOptions& ThreadingOptions::SetGlobalCustomJoinThreadFn(OrtCustomJoinThreadFn ort_custom_join_thread_fn) {
-  ThrowOnError(GetApi().SetGlobalCustomJoinThreadFn(p_, ort_custom_join_thread_fn));
-  return *this;
-}
-
-inline Env::Env(OrtLoggingLevel logging_level, _In_ const char* logid) {
-  ThrowOnError(GetApi().CreateEnv(logging_level, logid, &p_));
-  if (strcmp(logid, "onnxruntime-node") == 0) {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_NODEJS));
-  } else {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_CPLUSPLUS));
-  }
-}
-
-inline Env::Env(OrtLoggingLevel logging_level, const char* logid, OrtLoggingFunction logging_function, void* logger_param) {
-  ThrowOnError(GetApi().CreateEnvWithCustomLogger(logging_function, logger_param, logging_level, logid, &p_));
-  if (strcmp(logid, "onnxruntime-node") == 0) {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_NODEJS));
-  } else {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_CPLUSPLUS));
-  }
-}
-
-inline Env::Env(const OrtThreadingOptions* tp_options, OrtLoggingLevel logging_level, _In_ const char* logid) {
-  ThrowOnError(GetApi().CreateEnvWithGlobalThreadPools(logging_level, logid, tp_options, &p_));
-  if (strcmp(logid, "onnxruntime-node") == 0) {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_NODEJS));
-  } else {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_CPLUSPLUS));
-  }
-}
-
-inline Env::Env(const OrtThreadingOptions* tp_options, OrtLoggingFunction logging_function, void* logger_param,
-                OrtLoggingLevel logging_level, _In_ const char* logid) {
-  ThrowOnError(GetApi().CreateEnvWithCustomLoggerAndGlobalThreadPools(logging_function, logger_param, logging_level, logid, tp_options, &p_));
-  if (strcmp(logid, "onnxruntime-node") == 0) {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_NODEJS));
-  } else {
-    ThrowOnError(GetApi().SetLanguageProjection(p_, OrtLanguageProjection::ORT_PROJECTION_CPLUSPLUS));
-  }
-}
-
-inline Env& Env::EnableTelemetryEvents() {
-  ThrowOnError(GetApi().EnableTelemetryEvents(p_));
-  return *this;
-}
-
-inline Env& Env::DisableTelemetryEvents() {
-  ThrowOnError(GetApi().DisableTelemetryEvents(p_));
-  return *this;
-}
-
-inline Env& Env::UpdateEnvWithCustomLogLevel(OrtLoggingLevel log_severity_level) {
-  ThrowOnError(GetApi().UpdateEnvWithCustomLogLevel(p_, log_severity_level));
-  return *this;
-}
-
-inline Env& Env::CreateAndRegisterAllocator(const OrtMemoryInfo* mem_info, const OrtArenaCfg* arena_cfg) {
-  ThrowOnError(GetApi().CreateAndRegisterAllocator(p_, mem_info, arena_cfg));
-  return *this;
-}
-
-inline CustomOpDomain::CustomOpDomain(const char* domain) {
-  ThrowOnError(GetApi().CreateCustomOpDomain(domain, &p_));
-}
-
-inline void CustomOpDomain::Add(const OrtCustomOp* op) {
-  ThrowOnError(GetApi().CustomOpDomain_Add(p_, op));
-}
-
-inline RunOptions::RunOptions() {
-  ThrowOnError(GetApi().CreateRunOptions(&p_));
-}
-
-inline RunOptions& RunOptions::SetRunLogVerbosityLevel(int level) {
-  ThrowOnError(GetApi().RunOptionsSetRunLogVerbosityLevel(p_, level));
-  return *this;
-}
-
-inline RunOptions& RunOptions::SetRunLogSeverityLevel(int level) {
-  ThrowOnError(GetApi().RunOptionsSetRunLogSeverityLevel(p_, level));
-  return *this;
-}
-
-inline int RunOptions::GetRunLogVerbosityLevel() const {
-  int out;
-  ThrowOnError(GetApi().RunOptionsGetRunLogVerbosityLevel(p_, &out));
-  return out;
-}
-
-inline int RunOptions::GetRunLogSeverityLevel() const {
-  int out;
-  ThrowOnError(GetApi().RunOptionsGetRunLogSeverityLevel(p_, &out));
-  return out;
-}
-
-inline RunOptions& RunOptions::SetRunTag(const char* run_tag) {
-  ThrowOnError(GetApi().RunOptionsSetRunTag(p_, run_tag));
-  return *this;
-}
-
-inline const char* RunOptions::GetRunTag() const {
-  const char* out;
-  ThrowOnError(GetApi().RunOptionsGetRunTag(p_, &out));
-  return out;
-}
-
-inline RunOptions& RunOptions::AddConfigEntry(const char* config_key, const char* config_value) {
-  ThrowOnError(GetApi().AddRunConfigEntry(p_, config_key, config_value));
-  return *this;
-}
-
-inline RunOptions& RunOptions::SetTerminate() {
-  ThrowOnError(GetApi().RunOptionsSetTerminate(p_));
-  return *this;
-}
-
-inline RunOptions& RunOptions::UnsetTerminate() {
-  ThrowOnError(GetApi().RunOptionsUnsetTerminate(p_));
-  return *this;
-}
-
-namespace detail {
-
-template <typename T>
-inline Ort::SessionOptions ConstSessionOptionsImpl<T>::Clone() const {
-  OrtSessionOptions* out;
-  ThrowOnError(GetApi().CloneSessionOptions(this->p_, &out));
-  return SessionOptions{out};
-}
-
-template <typename T>
-inline std::string ConstSessionOptionsImpl<T>::GetConfigEntry(const char* config_key) const {
-  size_t size = 0;
-  // Feed nullptr for the data buffer to query the true size of the string value
-  Ort::ThrowOnError(GetApi().GetSessionConfigEntry(this->p_, config_key, nullptr, &size));
-
-  std::string out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().GetSessionConfigEntry(this->p_, config_key, &out[0], &size));
-  out.resize(size - 1);  // remove the terminating character '\0'
-
-  return out;
-}
-
-template <typename T>
-inline bool ConstSessionOptionsImpl<T>::HasConfigEntry(const char* config_key) const {
-  int out = 0;
-  Ort::ThrowOnError(GetApi().HasSessionConfigEntry(this->p_, config_key, &out));
-  return static_cast<bool>(out);
-}
-
-template <typename T>
-inline std::string ConstSessionOptionsImpl<T>::GetConfigEntryOrDefault(const char* config_key, const std::string& def) {
-  if (!this->HasConfigEntry(config_key)) {
-    return def;
-  }
-
-  return this->GetConfigEntry(config_key);
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetIntraOpNumThreads(int intra_op_num_threads) {
-  ThrowOnError(GetApi().SetIntraOpNumThreads(this->p_, intra_op_num_threads));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetInterOpNumThreads(int inter_op_num_threads) {
-  ThrowOnError(GetApi().SetInterOpNumThreads(this->p_, inter_op_num_threads));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetGraphOptimizationLevel(GraphOptimizationLevel graph_optimization_level) {
-  ThrowOnError(GetApi().SetSessionGraphOptimizationLevel(this->p_, graph_optimization_level));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetOptimizedModelFilePath(const ORTCHAR_T* optimized_model_filepath) {
-  ThrowOnError(GetApi().SetOptimizedModelFilePath(this->p_, optimized_model_filepath));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::EnableProfiling(const ORTCHAR_T* profile_file_prefix) {
-  ThrowOnError(GetApi().EnableProfiling(this->p_, profile_file_prefix));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::DisableProfiling() {
-  ThrowOnError(GetApi().DisableProfiling(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::EnableOrtCustomOps() {
-  ThrowOnError(GetApi().EnableOrtCustomOps(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::EnableMemPattern() {
-  ThrowOnError(GetApi().EnableMemPattern(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::DisableMemPattern() {
-  ThrowOnError(GetApi().DisableMemPattern(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::EnableCpuMemArena() {
-  ThrowOnError(GetApi().EnableCpuMemArena(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::DisableCpuMemArena() {
-  ThrowOnError(GetApi().DisableCpuMemArena(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetExecutionMode(ExecutionMode execution_mode) {
-  ThrowOnError(GetApi().SetSessionExecutionMode(this->p_, execution_mode));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetLogId(const char* logid) {
-  ThrowOnError(GetApi().SetSessionLogId(this->p_, logid));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetLogSeverityLevel(int level) {
-  ThrowOnError(GetApi().SetSessionLogSeverityLevel(this->p_, level));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::Add(OrtCustomOpDomain* custom_op_domain) {
-  ThrowOnError(GetApi().AddCustomOpDomain(this->p_, custom_op_domain));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AddConfigEntry(const char* config_key, const char* config_value) {
-  ThrowOnError(GetApi().AddSessionConfigEntry(this->p_, config_key, config_value));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AddInitializer(const char* name, const OrtValue* ort_val) {
-  ThrowOnError(GetApi().AddInitializer(this->p_, name, ort_val));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::DisablePerSessionThreads() {
-  ThrowOnError(GetApi().DisablePerSessionThreads(this->p_));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AddExternalInitializers(const std::vector<std::string>& names,
-                                                                             const std::vector<Value>& ort_values) {
-  const size_t inputs_num = names.size();
-  if (inputs_num != ort_values.size()) {
-    ORT_CXX_API_THROW("Expecting names and ort_values to have the same length", ORT_INVALID_ARGUMENT);
-  }
-  std::vector<const char*> names_ptr;
-  std::vector<const OrtValue*> ort_values_ptrs;
-  names_ptr.reserve(inputs_num);
-  ort_values_ptrs.reserve(inputs_num);
-  for (size_t i = 0; i < inputs_num; ++i) {
-    names_ptr.push_back(names[i].c_str());
-    ort_values_ptrs.push_back(ort_values[i]);
-  }
-  ThrowOnError(GetApi().AddExternalInitializers(this->p_, names_ptr.data(), ort_values_ptrs.data(), inputs_num));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_CUDA(const OrtCUDAProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_CUDA(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_CUDA_V2(const OrtCUDAProviderOptionsV2& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_CUDA_V2(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_ROCM(const OrtROCMProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_ROCM(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_TensorRT(const OrtTensorRTProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_TensorRT(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_TensorRT_V2(const OrtTensorRTProviderOptionsV2& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_TensorRT_V2(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_MIGraphX(const OrtMIGraphXProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_MIGraphX(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_CANN(const OrtCANNProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_CANN(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_Dnnl(const OrtDnnlProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_Dnnl(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider(
-    const std::string& provider_name,
-    const std::unordered_map<std::string, std::string>& provider_options) {
-  auto num_entries = provider_options.size();
-  std::vector<const char*> keys, values;
-  if (num_entries > 0) {
-    keys.reserve(num_entries);
-    values.reserve(num_entries);
-
-    for (const auto& entry : provider_options) {
-      keys.push_back(entry.first.c_str());
-      values.push_back(entry.second.c_str());
-    }
-  }
-
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider(this->p_, provider_name.c_str(),
-                                                              keys.data(), values.data(), num_entries));
-
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetCustomCreateThreadFn(OrtCustomCreateThreadFn ort_custom_create_thread_fn) {
-  ThrowOnError(GetApi().SessionOptionsSetCustomCreateThreadFn(this->p_, ort_custom_create_thread_fn));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetCustomThreadCreationOptions(void* ort_custom_thread_creation_options) {
-  ThrowOnError(GetApi().SessionOptionsSetCustomThreadCreationOptions(this->p_, ort_custom_thread_creation_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetCustomJoinThreadFn(OrtCustomJoinThreadFn ort_custom_join_thread_fn) {
-  ThrowOnError(GetApi().SessionOptionsSetCustomJoinThreadFn(this->p_, ort_custom_join_thread_fn));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_OpenVINO(const OrtOpenVINOProviderOptions& provider_options) {
-  ThrowOnError(GetApi().SessionOptionsAppendExecutionProvider_OpenVINO(this->p_, &provider_options));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::RegisterCustomOpsLibrary(const ORTCHAR_T* library_name,
-                                                                              const CustomOpConfigs& custom_op_configs) {
-  // Add custom op config entries before registering the custom op library. Otherwise, the config entries _may_ be ignored by
-  // the custom op library.
-  for (const auto& config_iter : custom_op_configs.GetFlattenedConfigs()) {
-    AddConfigEntry(config_iter.first.c_str(), config_iter.second.c_str());
-  }
-
-  ThrowOnError(GetApi().RegisterCustomOpsLibrary_V2(this->p_, library_name));
-  return *this;
-}
-
-template <typename T>
-inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::RegisterCustomOpsUsingFunction(const char* registration_function_name) {
-  ThrowOnError(GetApi().RegisterCustomOpsUsingFunction(this->p_, registration_function_name));
-  return *this;
-}
-
-/// Session
-template <typename T>
-inline size_t ConstSessionImpl<T>::GetInputCount() const {
-  size_t out;
-  ThrowOnError(GetApi().SessionGetInputCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline size_t ConstSessionImpl<T>::GetOutputCount() const {
-  size_t out;
-  ThrowOnError(GetApi().SessionGetOutputCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline size_t ConstSessionImpl<T>::GetOverridableInitializerCount() const {
-  size_t out;
-  ThrowOnError(GetApi().SessionGetOverridableInitializerCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline AllocatedStringPtr ConstSessionImpl<T>::GetInputNameAllocated(size_t index, OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().SessionGetInputName(this->p_, index, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-template <typename T>
-inline AllocatedStringPtr ConstSessionImpl<T>::GetOutputNameAllocated(size_t index, OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().SessionGetOutputName(this->p_, index, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-template <typename T>
-inline AllocatedStringPtr ConstSessionImpl<T>::GetOverridableInitializerNameAllocated(size_t index, OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().SessionGetOverridableInitializerName(this->p_, index, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-template <typename T>
-inline uint64_t ConstSessionImpl<T>::GetProfilingStartTimeNs() const {
-  uint64_t out;
-  ThrowOnError(GetApi().SessionGetProfilingStartTimeNs(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline ModelMetadata ConstSessionImpl<T>::GetModelMetadata() const {
-  OrtModelMetadata* out;
-  ThrowOnError(GetApi().SessionGetModelMetadata(this->p_, &out));
-  return ModelMetadata{out};
-}
-
-template <typename T>
-inline TypeInfo ConstSessionImpl<T>::GetInputTypeInfo(size_t index) const {
-  OrtTypeInfo* out;
-  ThrowOnError(GetApi().SessionGetInputTypeInfo(this->p_, index, &out));
-  return TypeInfo{out};
-}
-
-template <typename T>
-inline TypeInfo ConstSessionImpl<T>::GetOutputTypeInfo(size_t index) const {
-  OrtTypeInfo* out;
-  ThrowOnError(GetApi().SessionGetOutputTypeInfo(this->p_, index, &out));
-  return TypeInfo{out};
-}
-
-template <typename T>
-inline TypeInfo ConstSessionImpl<T>::GetOverridableInitializerTypeInfo(size_t index) const {
-  OrtTypeInfo* out;
-  ThrowOnError(GetApi().SessionGetOverridableInitializerTypeInfo(this->p_, index, &out));
-  return TypeInfo{out};
-}
-
-template <typename T>
-inline std::vector<Value> SessionImpl<T>::Run(const RunOptions& run_options, const char* const* input_names, const Value* input_values, size_t input_count,
-                                              const char* const* output_names, size_t output_count) {
-  std::vector<Value> output_values;
-  output_values.reserve(output_count);
-  for (size_t i = 0; i < output_count; i++)
-    output_values.emplace_back(nullptr);
-  Run(run_options, input_names, input_values, input_count, output_names, output_values.data(), output_count);
-  return output_values;
-}
-
-template <typename T>
-inline void SessionImpl<T>::Run(const RunOptions& run_options, const char* const* input_names, const Value* input_values, size_t input_count,
-                                const char* const* output_names, Value* output_values, size_t output_count) {
-  static_assert(sizeof(Value) == sizeof(OrtValue*), "Value is really just an array of OrtValue* in memory, so we can reinterpret_cast safely");
-  auto ort_input_values = reinterpret_cast<const OrtValue* const*>(input_values);
-  auto ort_output_values = reinterpret_cast<OrtValue**>(output_values);
-  ThrowOnError(GetApi().Run(this->p_, run_options, input_names, ort_input_values, input_count, output_names, output_count, ort_output_values));
-}
-
-template <typename T>
-inline void SessionImpl<T>::Run(const RunOptions& run_options, const IoBinding& io_binding) {
-  ThrowOnError(GetApi().RunWithBinding(this->p_, run_options, io_binding));
-}
-
-template <typename T>
-inline AllocatedStringPtr SessionImpl<T>::EndProfilingAllocated(OrtAllocator* allocator) {
-  char* out = nullptr;
-  ThrowOnError(GetApi().SessionEndProfiling(this->p_, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-}  // namespace detail
-
-inline SessionOptions::SessionOptions() {
-  ThrowOnError(GetApi().CreateSessionOptions(&this->p_));
-}
-
-/// CustomOpConfigs
-inline std::string detail::MakeCustomOpConfigEntryKey(const char* custom_op_name, const char* config) {
-  std::string config_key = "custom_op.";
-
-  config_key += custom_op_name;
-  config_key += ".";
-  config_key += config;
-
-  return config_key;
-}
-
-inline CustomOpConfigs& CustomOpConfigs::AddConfig(const char* custom_op_name, const char* config_key, const char* config_value) {
-  const std::string full_flat_key = detail::MakeCustomOpConfigEntryKey(custom_op_name, config_key);
-  flat_configs_[full_flat_key] = config_value;
-  return *this;
-}
-
-inline const std::unordered_map<std::string, std::string>& CustomOpConfigs::GetFlattenedConfigs() const {
-  return flat_configs_;
-}
-
-inline Session::Session(const Env& env, const ORTCHAR_T* model_path, const SessionOptions& options) {
-  ThrowOnError(GetApi().CreateSession(env, model_path, options, &this->p_));
-}
-
-inline Session::Session(const Env& env, const ORTCHAR_T* model_path, const SessionOptions& options,
-                        OrtPrepackedWeightsContainer* prepacked_weights_container) {
-  ThrowOnError(GetApi().CreateSessionWithPrepackedWeightsContainer(env, model_path, options, prepacked_weights_container, &this->p_));
-}
-
-inline Session::Session(const Env& env, const void* model_data, size_t model_data_length, const SessionOptions& options) {
-  ThrowOnError(GetApi().CreateSessionFromArray(env, model_data, model_data_length, options, &this->p_));
-}
-
-inline Session::Session(const Env& env, const void* model_data, size_t model_data_length,
-                        const SessionOptions& options, OrtPrepackedWeightsContainer* prepacked_weights_container) {
-  ThrowOnError(GetApi().CreateSessionFromArrayWithPrepackedWeightsContainer(env, model_data, model_data_length, options,
-                                                                            prepacked_weights_container, &this->p_));
-}
-
-inline AllocatedStringPtr ModelMetadata::GetProducerNameAllocated(OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().ModelMetadataGetProducerName(p_, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-inline AllocatedStringPtr ModelMetadata::GetGraphNameAllocated(OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().ModelMetadataGetGraphName(p_, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-inline AllocatedStringPtr ModelMetadata::GetDomainAllocated(OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().ModelMetadataGetDomain(p_, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-inline AllocatedStringPtr Ort::ModelMetadata::GetDescriptionAllocated(OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().ModelMetadataGetDescription(p_, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-inline AllocatedStringPtr ModelMetadata::GetGraphDescriptionAllocated(OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().ModelMetadataGetGraphDescription(p_, allocator, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-inline AllocatedStringPtr ModelMetadata::LookupCustomMetadataMapAllocated(const char* key, OrtAllocator* allocator) const {
-  char* out;
-  ThrowOnError(GetApi().ModelMetadataLookupCustomMetadataMap(p_, allocator, key, &out));
-  return AllocatedStringPtr(out, detail::AllocatedFree(allocator));
-}
-
-inline std::vector<AllocatedStringPtr> ModelMetadata::GetCustomMetadataMapKeysAllocated(OrtAllocator* allocator) const {
-  auto deletor = detail::AllocatedFree(allocator);
-  std::vector<AllocatedStringPtr> result;
-
-  char** out = nullptr;
-  int64_t num_keys = 0;
-  ThrowOnError(GetApi().ModelMetadataGetCustomMetadataMapKeys(p_, allocator, &out, &num_keys));
-  if (num_keys <= 0) {
-    return result;
-  }
-
-  // array of pointers will be freed
-  std::unique_ptr<void, decltype(deletor)> array_guard(out, deletor);
-  // reserve may throw
-  auto strings_deletor = [&deletor, num_keys](char** out) { for(int64_t i = 0; i < num_keys; ++i) deletor(out[i]); };
-  std::unique_ptr<char*, decltype(strings_deletor)> strings_guard(out, strings_deletor);
-  result.reserve(static_cast<size_t>(num_keys));
-  strings_guard.release();
-  for (int64_t i = 0; i < num_keys; ++i) {
-    result.push_back(AllocatedStringPtr(out[i], deletor));
-  }
-
-  return result;
-}
-
-inline int64_t ModelMetadata::GetVersion() const {
-  int64_t out;
-  ThrowOnError(GetApi().ModelMetadataGetVersion(p_, &out));
-  return out;
-}
-
-namespace detail {
-
-template <typename T>
-inline ONNXTensorElementDataType TensorTypeAndShapeInfoImpl<T>::GetElementType() const {
-  ONNXTensorElementDataType out;
-  ThrowOnError(GetApi().GetTensorElementType(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline size_t TensorTypeAndShapeInfoImpl<T>::GetElementCount() const {
-  size_t out;
-  ThrowOnError(GetApi().GetTensorShapeElementCount(this->p_, &out));
-  return static_cast<size_t>(out);
-}
-
-template <typename T>
-inline size_t TensorTypeAndShapeInfoImpl<T>::GetDimensionsCount() const {
-  size_t out;
-  ThrowOnError(GetApi().GetDimensionsCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline void TensorTypeAndShapeInfoImpl<T>::GetDimensions(int64_t* values, size_t values_count) const {
-  ThrowOnError(GetApi().GetDimensions(this->p_, values, values_count));
-}
-
-template <typename T>
-inline void TensorTypeAndShapeInfoImpl<T>::GetSymbolicDimensions(const char** values, size_t values_count) const {
-  ThrowOnError(GetApi().GetSymbolicDimensions(this->p_, values, values_count));
-}
-
-template <typename T>
-inline std::vector<int64_t> TensorTypeAndShapeInfoImpl<T>::GetShape() const {
-  std::vector<int64_t> out(GetDimensionsCount(), 0);
-  ThrowOnError(GetApi().GetDimensions(this->p_, out.data(), out.size()));
-  return out;
-}
-
-template <typename T>
-inline ConstTensorTypeAndShapeInfo TypeInfoImpl<T>::GetTensorTypeAndShapeInfo() const {
-  const OrtTensorTypeAndShapeInfo* out;
-  ThrowOnError(GetApi().CastTypeInfoToTensorInfo(this->p_, &out));
-  return ConstTensorTypeAndShapeInfo{out};
-}
-
-template <typename T>
-inline ConstSequenceTypeInfo TypeInfoImpl<T>::GetSequenceTypeInfo() const {
-  const OrtSequenceTypeInfo* out;
-  ThrowOnError(GetApi().CastTypeInfoToSequenceTypeInfo(this->p_, &out));
-  return ConstSequenceTypeInfo{out};
-}
-
-template <typename T>
-inline ConstMapTypeInfo TypeInfoImpl<T>::GetMapTypeInfo() const {
-  const OrtMapTypeInfo* out;
-  ThrowOnError(GetApi().CastTypeInfoToMapTypeInfo(this->p_, &out));
-  return ConstMapTypeInfo{out};
-}
-
-template <typename T>
-inline ONNXType TypeInfoImpl<T>::GetONNXType() const {
-  ONNXType out;
-  ThrowOnError(GetApi().GetOnnxTypeFromTypeInfo(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline TypeInfo SequenceTypeInfoImpl<T>::GetSequenceElementType() const {
-  OrtTypeInfo* output;
-  ThrowOnError(GetApi().GetSequenceElementType(this->p_, &output));
-  return TypeInfo{output};
-}
-
-template <typename T>
-inline TypeInfo OptionalTypeInfoImpl<T>::GetOptionalElementType() const {
-  OrtTypeInfo* info;
-  ThrowOnError(GetApi().GetOptionalContainedTypeInfo(this->p_, &info));
-  return TypeInfo{info};
-}
-
-template <typename T>
-inline ONNXTensorElementDataType MapTypeInfoImpl<T>::GetMapKeyType() const {
-  ONNXTensorElementDataType out;
-  ThrowOnError(GetApi().GetMapKeyType(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline TypeInfo MapTypeInfoImpl<T>::GetMapValueType() const {
-  OrtTypeInfo* output;
-  ThrowOnError(GetApi().GetMapValueType(this->p_, &output));
-  return TypeInfo{output};
-}
-
-template <typename T>
-inline ConstOptionalTypeInfo TypeInfoImpl<T>::GetOptionalTypeInfo() const {
-  const OrtOptionalTypeInfo* info;
-  ThrowOnError(GetApi().CastTypeInfoToOptionalTypeInfo(this->p_, &info));
-  return ConstOptionalTypeInfo{info};
-}
-
-}  // namespace detail
-
-namespace detail {
-
-template <typename T>
-template <typename R>
-inline void ConstValueImpl<T>::GetOpaqueData(const char* domain, const char* type_name, R& out) const {
-  ThrowOnError(GetApi().GetOpaqueValue(domain, type_name, this->p_, &out, sizeof(R)));
-}
-
-template <typename T>
-inline bool ConstValueImpl<T>::IsTensor() const {
-  int out;
-  ThrowOnError(GetApi().IsTensor(this->p_, &out));
-  return out != 0;
-}
-
-template <typename T>
-inline bool ConstValueImpl<T>::HasValue() const {
-  int out;
-  ThrowOnError(GetApi().HasValue(this->p_, &out));
-  return out != 0;
-}
-
-template <typename T>
-inline size_t ConstValueImpl<T>::GetCount() const {
-  size_t out;
-  ThrowOnError(GetApi().GetValueCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline Value ConstValueImpl<T>::GetValue(int index, OrtAllocator* allocator) const {
-  OrtValue* out;
-  ThrowOnError(GetApi().GetValue(this->p_, index, allocator, &out));
-  return Value{out};
-}
-
-template <typename T>
-inline size_t ConstValueImpl<T>::GetStringTensorDataLength() const {
-  size_t out;
-  ThrowOnError(GetApi().GetStringTensorDataLength(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline size_t ConstValueImpl<T>::GetStringTensorElementLength(size_t element_index) const {
-  size_t out;
-  ThrowOnError(GetApi().GetStringTensorElementLength(this->p_, element_index, &out));
-  return out;
-}
-
-template <typename T>
-template <typename R>
-inline const R* ConstValueImpl<T>::GetTensorData() const {
-  R* out;
-  ThrowOnError(GetApi().GetTensorMutableData(const_cast<OrtValue*>(this->p_), (void**)&out));
-  return out;
-}
-
-template <typename T>
-inline const void* ConstValueImpl<T>::GetTensorRawData() const {
-  void* out;
-  ThrowOnError(GetApi().GetTensorMutableData(const_cast<OrtValue*>(this->p_), &out));
-  return out;
-}
-
-template <typename T>
-inline TypeInfo ConstValueImpl<T>::GetTypeInfo() const {
-  OrtTypeInfo* output;
-  ThrowOnError(GetApi().GetTypeInfo(this->p_, &output));
-  return TypeInfo{output};
-}
-
-template <typename T>
-inline TensorTypeAndShapeInfo ConstValueImpl<T>::GetTensorTypeAndShapeInfo() const {
-  OrtTensorTypeAndShapeInfo* output;
-  ThrowOnError(GetApi().GetTensorTypeAndShape(this->p_, &output));
-  return TensorTypeAndShapeInfo{output};
-}
-
-template <typename T>
-inline ConstMemoryInfo ConstValueImpl<T>::GetTensorMemoryInfo() const {
-  const OrtMemoryInfo* mem_info;
-  ThrowOnError(GetApi().GetTensorMemoryInfo(this->p_, &mem_info));
-  return ConstMemoryInfo(mem_info);
-}
-
-template <typename T>
-inline void ConstValueImpl<T>::GetStringTensorElement(size_t buffer_length, size_t element_index, void* buffer) const {
-  ThrowOnError(GetApi().GetStringTensorElement(this->p_, buffer_length, element_index, buffer));
-}
-
-template <typename T>
-inline std::string ConstValueImpl<T>::GetStringTensorElement(size_t element_index) const {
-  size_t buffer_length;
-  ThrowOnError(GetApi().GetStringTensorElementLength(this->p_, element_index, &buffer_length));
-
-  std::string s;
-  s.resize(buffer_length);
-  ThrowOnError(GetApi().GetStringTensorElement(this->p_, buffer_length, element_index, &s[0]));
-  return s;
-}
-
-template <typename T>
-inline void ConstValueImpl<T>::GetStringTensorContent(void* buffer, size_t buffer_length, size_t* offsets, size_t offsets_count) const {
-  ThrowOnError(GetApi().GetStringTensorContent(this->p_, buffer, buffer_length, offsets, offsets_count));
-}
-
-#if !defined(DISABLE_SPARSE_TENSORS)
-template <typename T>
-inline OrtSparseFormat ConstValueImpl<T>::GetSparseFormat() const {
-  OrtSparseFormat format;
-  ThrowOnError(GetApi().GetSparseTensorFormat(this->p_, &format));
-  return format;
-}
-
-template <typename T>
-inline TensorTypeAndShapeInfo ConstValueImpl<T>::GetSparseTensorValuesTypeAndShapeInfo() const {
-  OrtTensorTypeAndShapeInfo* output;
-  ThrowOnError(GetApi().GetSparseTensorValuesTypeAndShape(this->p_, &output));
-  return TensorTypeAndShapeInfo{output};
-}
-
-template <typename T>
-inline TensorTypeAndShapeInfo ConstValueImpl<T>::GetSparseTensorIndicesTypeShapeInfo(OrtSparseIndicesFormat indices_format) const {
-  OrtTensorTypeAndShapeInfo* output;
-  ThrowOnError(GetApi().GetSparseTensorIndicesTypeShape(this->p_, indices_format, &output));
-  return TensorTypeAndShapeInfo{output};
-}
-
-template <typename T>
-template <typename R>
-inline const R* ConstValueImpl<T>::GetSparseTensorIndicesData(OrtSparseIndicesFormat indices_format, size_t& num_indices) const {
-  const void* out;
-  ThrowOnError(GetApi().GetSparseTensorIndices(this->p_, indices_format, &num_indices, &out));
-  return reinterpret_cast<const R*>(out);
-}
-
-template <typename T>
-inline bool ConstValueImpl<T>::IsSparseTensor() const {
-  int out;
-  ThrowOnError(GetApi().IsSparseTensor(this->p_, &out));
-  return out != 0;
-}
-
-template <typename T>
-template <typename R>
-inline const R* ConstValueImpl<T>::GetSparseTensorValues() const {
-  const void* out;
-  ThrowOnError(GetApi().GetSparseTensorValues(this->p_, &out));
-  return reinterpret_cast<const R*>(out);
-}
-
-#endif
-
-template <typename T>
-void ValueImpl<T>::FillStringTensor(const char* const* s, size_t s_len) {
-  ThrowOnError(GetApi().FillStringTensor(this->p_, s, s_len));
-}
-
-template <typename T>
-void ValueImpl<T>::FillStringTensorElement(const char* s, size_t index) {
-  ThrowOnError(GetApi().FillStringTensorElement(this->p_, s, index));
-}
-
-template <typename T>
-inline char* ValueImpl<T>::GetResizedStringTensorElementBuffer(size_t index, size_t buffer_length) {
-  char* result;
-  ThrowOnError(GetApi().GetResizedStringTensorElementBuffer(this->p_, index, buffer_length, &result));
-  return result;
-}
-
-template <typename T>
-void* ValueImpl<T>::GetTensorMutableRawData() {
-  void* out;
-  ThrowOnError(GetApi().GetTensorMutableData(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-template <typename R>
-R* ValueImpl<T>::GetTensorMutableData() {
-  R* out;
-  ThrowOnError(GetApi().GetTensorMutableData(this->p_, (void**)&out));
-  return out;
-}
-
-template <typename T>
-template <typename R>
-R& ValueImpl<T>::At(const std::vector<int64_t>& location) {
-  static_assert(!std::is_same<T, std::string>::value, "this api does not support std::string");
-  R* out;
-  ThrowOnError(GetApi().TensorAt(this->p_, location.data(), location.size(), (void**)&out));
-  return *out;
-}
-
-#if !defined(DISABLE_SPARSE_TENSORS)
-template <typename T>
-void ValueImpl<T>::UseCooIndices(int64_t* indices_data, size_t indices_num) {
-  ThrowOnError(GetApi().UseCooIndices(this->p_, indices_data, indices_num));
-}
-
-template <typename T>
-void ValueImpl<T>::UseCsrIndices(int64_t* inner_data, size_t inner_num, int64_t* outer_data, size_t outer_num) {
-  ThrowOnError(GetApi().UseCsrIndices(this->p_, inner_data, inner_num, outer_data, outer_num));
-}
-
-template <typename T>
-void ValueImpl<T>::UseBlockSparseIndices(const Shape& indices_shape, int32_t* indices_data) {
-  ThrowOnError(GetApi().UseBlockSparseIndices(this->p_, indices_shape.shape, indices_shape.shape_len, indices_data));
-}
-
-template <typename T>
-void ValueImpl<T>::FillSparseTensorCoo(const OrtMemoryInfo* mem_info, const OrtSparseValuesParam& values_param,
-                                       const int64_t* indices_data, size_t indices_num) {
-  ThrowOnError(GetApi().FillSparseTensorCoo(this->p_, mem_info, values_param.values_shape,
-                                            values_param.values_shape_len, values_param.data.p_data,
-                                            indices_data, indices_num));
-}
-
-template <typename T>
-void ValueImpl<T>::FillSparseTensorCsr(const OrtMemoryInfo* data_mem_info,
-                                       const OrtSparseValuesParam& values,
-                                       const int64_t* inner_indices_data, size_t inner_indices_num,
-                                       const int64_t* outer_indices_data, size_t outer_indices_num) {
-  ThrowOnError(GetApi().FillSparseTensorCsr(this->p_, data_mem_info, values.values_shape, values.values_shape_len, values.data.p_data,
-                                            inner_indices_data, inner_indices_num,
-                                            outer_indices_data, outer_indices_num));
-}
-
-template <typename T>
-void ValueImpl<T>::FillSparseTensorBlockSparse(const OrtMemoryInfo* data_mem_info,
-                                               const OrtSparseValuesParam& values,
-                                               const Shape& indices_shape,
-                                               const int32_t* indices_data) {
-  ThrowOnError(GetApi().FillSparseTensorBlockSparse(this->p_, data_mem_info, values.values_shape, values.values_shape_len, values.data.p_data,
-                                                    indices_shape.shape, indices_shape.shape_len,
-                                                    indices_data));
-}
-
-#endif  // !defined(DISABLE_SPARSE_TENSORS)
-
-}  // namespace detail
-
-template <typename T>
-inline Value Value::CreateTensor(const OrtMemoryInfo* info, T* p_data, size_t p_data_element_count, const int64_t* shape, size_t shape_len) {
-  return CreateTensor(info, p_data, p_data_element_count * sizeof(T), shape, shape_len, TypeToTensorType<T>::type);
-}
-
-inline Value Value::CreateTensor(const OrtMemoryInfo* info, void* p_data, size_t p_data_byte_count, const int64_t* shape, size_t shape_len,
-                                 ONNXTensorElementDataType type) {
-  OrtValue* out;
-  ThrowOnError(GetApi().CreateTensorWithDataAsOrtValue(info, p_data, p_data_byte_count, shape, shape_len, type, &out));
-  return Value{out};
-}
-
-template <typename T>
-inline Value Value::CreateTensor(OrtAllocator* allocator, const int64_t* shape, size_t shape_len) {
-  return CreateTensor(allocator, shape, shape_len, TypeToTensorType<T>::type);
-}
-
-inline Value Value::CreateTensor(OrtAllocator* allocator, const int64_t* shape, size_t shape_len, ONNXTensorElementDataType type) {
-  OrtValue* out;
-  ThrowOnError(GetApi().CreateTensorAsOrtValue(allocator, shape, shape_len, type, &out));
-  return Value{out};
-}
-
-#if !defined(DISABLE_SPARSE_TENSORS)
-
-template <typename T>
-inline Value Value::CreateSparseTensor(const OrtMemoryInfo* info, T* p_data, const Shape& dense_shape,
-                                       const Shape& values_shape) {
-  return CreateSparseTensor(info, p_data, dense_shape, values_shape, TypeToTensorType<T>::type);
-}
-
-inline Value Value::CreateSparseTensor(const OrtMemoryInfo* info, void* p_data, const Shape& dense_shape,
-                                       const Shape& values_shape, ONNXTensorElementDataType type) {
-  OrtValue* out;
-  ThrowOnError(GetApi().CreateSparseTensorWithValuesAsOrtValue(info, p_data, dense_shape.shape, dense_shape.shape_len,
-                                                               values_shape.shape, values_shape.shape_len, type, &out));
-  return Value{out};
-}
-
-template <typename T>
-inline Value Value::CreateSparseTensor(OrtAllocator* allocator, const Shape& dense_shape) {
-  return CreateSparseTensor(allocator, dense_shape, TypeToTensorType<T>::type);
-}
-
-inline Value Value::CreateSparseTensor(OrtAllocator* allocator, const Shape& dense_shape,
-                                       ONNXTensorElementDataType type) {
-  OrtValue* out;
-  ThrowOnError(GetApi().CreateSparseTensorAsOrtValue(allocator, dense_shape.shape, dense_shape.shape_len, type, &out));
-  return Value{out};
-}
-#endif  // !defined(DISABLE_SPARSE_TENSORS)
-
-inline Value Value::CreateMap(Value& keys, Value& values) {
-  OrtValue* out;
-  OrtValue* inputs[2] = {keys, values};
-  ThrowOnError(GetApi().CreateValue(inputs, 2, ONNX_TYPE_MAP, &out));
-  return Value{out};
-}
-
-inline Value Value::CreateSequence(std::vector<Value>& values) {
-  OrtValue* out;
-  std::vector<OrtValue*> values_ort{values.data(), values.data() + values.size()};
-  ThrowOnError(GetApi().CreateValue(values_ort.data(), values_ort.size(), ONNX_TYPE_SEQUENCE, &out));
-  return Value{out};
-}
-
-template <typename T>
-inline Value Value::CreateOpaque(const char* domain, const char* type_name, const T& data_container) {
-  OrtValue* out;
-  ThrowOnError(GetApi().CreateOpaqueValue(domain, type_name, &data_container, sizeof(T), &out));
-  return Value{out};
-}
-
-//
-// Custom OP Inlines
-//
-inline Logger::Logger(const OrtLogger* logger) : logger_(logger) {
-  Ort::ThrowOnError(GetApi().Logger_GetLoggingSeverityLevel(this->logger_, &this->cached_severity_level_));
-}
-
-inline OrtLoggingLevel Logger::GetLoggingSeverityLevel() const noexcept {
-  return cached_severity_level_;
-}
-
-inline Status Logger::LogMessage(OrtLoggingLevel log_severity_level, const ORTCHAR_T* file_path, int line_number,
-                                 const char* func_name, const char* message) const noexcept {
-  OrtStatus* status = GetApi().Logger_LogMessage(logger_, log_severity_level, message, file_path, line_number,
-                                                 func_name);
-  return Status{status};
-}
-
-// Disable warnings about the format string not being a literal (-Wformat-nonliteral and -Wformat-security)
-// for gcc and clang. The alternative is to use actual C-style variadic parameters and apply
-// __attribute__(format(printf...)), which does not work with variadic templates.
-#if defined(__GNUC__)
-#pragma GCC diagnostic push
-#pragma GCC diagnostic ignored "-Wformat-nonliteral"
-#pragma GCC diagnostic ignored "-Wformat-security"
-#elif defined(__clang__)
-#pragma clang diagnostic push
-#pragma clang diagnostic ignored "-Wformat-nonliteral"
-#pragma clang diagnostic ignored "-Wformat-security"
-#endif
-template <typename... Args>
-inline Status Logger::LogFormattedMessage(OrtLoggingLevel log_severity_level, const ORTCHAR_T* file_path,
-                                          int line_number, const char* func_name, const char* format,
-                                          Args&&... args) const noexcept {
-  int msg_len = std::snprintf(nullptr, 0U, format, std::forward<Args>(args)...);
-
-  if (msg_len < 0) {  // Formatting error
-    return Status("Failed to log message due to formatting error", OrtErrorCode::ORT_FAIL);
-  }
-
-  OrtStatus* status = nullptr;
-  const size_t buffer_size = static_cast<size_t>(msg_len) + 1U;
-
-  constexpr size_t kStackBufferSize = 1024;
-
-  if (buffer_size < kStackBufferSize) {
-    char buffer[kStackBufferSize];
-    snprintf(buffer, kStackBufferSize, format, std::forward<Args>(args)...);
-    status = GetApi().Logger_LogMessage(logger_, log_severity_level, buffer, file_path, line_number, func_name);
-  } else {
-    // std::make_unique is only supported starting at C++14.
-#if (__cplusplus >= 201402L) || (_MSC_VER >= 1900)
-    auto buffer = std::make_unique<char[]>(buffer_size);
-#else
-    std::unique_ptr<char[]> buffer(new char[buffer_size]);
-#endif
-    std::snprintf(buffer.get(), buffer_size, format, std::forward<Args>(args)...);
-    status = GetApi().Logger_LogMessage(logger_, log_severity_level, buffer.get(), file_path, line_number, func_name);
-  }
-
-  return Status{status};
-}
-// Re-enable -Wformat-nonliteral and -Wformat-security
-#if defined(__GNUC__)
-#pragma GCC diagnostic pop
-#elif defined(__clang__)
-#pragma clang diagnostic pop
-#endif
-
-inline KernelContext::KernelContext(OrtKernelContext* context) : ctx_(context) {
-}
-
-inline size_t KernelContext::GetInputCount() const {
-  size_t out = 0;
-  Ort::ThrowOnError(GetApi().KernelContext_GetInputCount(ctx_, &out));
-  return out;
-}
-
-inline size_t KernelContext::GetOutputCount() const {
-  size_t out = 0;
-  Ort::ThrowOnError(GetApi().KernelContext_GetOutputCount(ctx_, &out));
-  return out;
-}
-
-inline ConstValue KernelContext::GetInput(size_t index) const {
-  const OrtValue* out = nullptr;
-  Ort::ThrowOnError(GetApi().KernelContext_GetInput(ctx_, index, &out));
-  return ConstValue{out};
-}
-
-inline UnownedValue KernelContext::GetOutput(size_t index, const int64_t* dim_values, size_t dim_count) const {
-  OrtValue* out = nullptr;
-  Ort::ThrowOnError(GetApi().KernelContext_GetOutput(ctx_, index, dim_values, dim_count, &out));
-  return UnownedValue(out);
-}
-
-inline UnownedValue KernelContext::GetOutput(size_t index, const std::vector<int64_t>& dims) const {
-  OrtValue* out = nullptr;
-  Ort::ThrowOnError(GetApi().KernelContext_GetOutput(ctx_, index, dims.data(), dims.size(), &out));
-  return UnownedValue(out);
-}
-
-inline void* KernelContext::GetGPUComputeStream() const {
-  void* out = nullptr;
-  Ort::ThrowOnError(GetApi().KernelContext_GetGPUComputeStream(ctx_, &out));
-  return out;
-}
-
-inline OrtAllocator* KernelContext::GetAllocator(const OrtMemoryInfo& memory_info) const {
-  OrtAllocator* out = nullptr;
-  Ort::ThrowOnError(GetApi().KernelContext_GetAllocator(ctx_, &memory_info, &out));
-  return out;
-}
-
-inline Logger KernelContext::GetLogger() const {
-  const OrtLogger* out = nullptr;
-  ThrowOnError(GetApi().KernelContext_GetLogger(this->ctx_, &out));
-  return Logger{out};
-}
-
-inline OpAttr::OpAttr(const char* name, const void* data, int len, OrtOpAttrType type) {
-  Ort::ThrowOnError(GetApi().CreateOpAttr(name, data, len, type, &p_));
-}
-
-namespace detail {
-template <typename T>
-inline KernelInfo KernelInfoImpl<T>::Copy() const {
-  OrtKernelInfo* info_copy = nullptr;
-  Ort::ThrowOnError(GetApi().CopyKernelInfo(this->p_, &info_copy));
-  return KernelInfo{info_copy};
-}
-
-template <typename T>
-inline size_t KernelInfoImpl<T>::GetInputCount() const {
-  size_t out = 0;
-  ThrowOnError(GetApi().KernelInfo_GetInputCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline size_t KernelInfoImpl<T>::GetOutputCount() const {
-  size_t out = 0;
-  ThrowOnError(GetApi().KernelInfo_GetOutputCount(this->p_, &out));
-  return out;
-}
-
-template <typename T>
-inline std::string KernelInfoImpl<T>::GetInputName(size_t index) const {
-  size_t size = 0;
-
-  // Feed nullptr for the data buffer to query the true size of the string value
-  Ort::ThrowOnError(GetApi().KernelInfo_GetInputName(this->p_, index, nullptr, &size));
-
-  std::string out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().KernelInfo_GetInputName(this->p_, index, &out[0], &size));
-  out.resize(size - 1);  // remove the terminating character '\0'
-
-  return out;
-}
-
-template <typename T>
-inline std::string KernelInfoImpl<T>::GetOutputName(size_t index) const {
-  size_t size = 0;
-
-  // Feed nullptr for the data buffer to query the true size of the string value
-  Ort::ThrowOnError(GetApi().KernelInfo_GetOutputName(this->p_, index, nullptr, &size));
-
-  std::string out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().KernelInfo_GetOutputName(this->p_, index, &out[0], &size));
-  out.resize(size - 1);  // remove the terminating character '\0'
-
-  return out;
-}
-
-template <typename T>
-inline TypeInfo KernelInfoImpl<T>::GetInputTypeInfo(size_t index) const {
-  OrtTypeInfo* out = nullptr;
-  ThrowOnError(GetApi().KernelInfo_GetInputTypeInfo(this->p_, index, &out));
-  return TypeInfo{out};
-}
-
-template <typename T>
-inline TypeInfo KernelInfoImpl<T>::GetOutputTypeInfo(size_t index) const {
-  OrtTypeInfo* out = nullptr;
-  ThrowOnError(GetApi().KernelInfo_GetOutputTypeInfo(this->p_, index, &out));
-  return TypeInfo{out};
-}
-
-template <typename T>
-inline Value KernelInfoImpl<T>::GetTensorAttribute(const char* name, OrtAllocator* allocator) const {
-  OrtValue* out = nullptr;
-  ThrowOnError(GetApi().KernelInfoGetAttribute_tensor(this->p_, name, allocator, &out));
-  return Value{out};
-}
-
-template <typename T>
-inline ConstValue KernelInfoImpl<T>::GetTensorConstantInput(size_t index, int* is_constant) const {
-  const OrtValue* out = nullptr;
-  ThrowOnError(GetApi().KernelInfoGetConstantInput_tensor(this->p_, index, is_constant, &out));
-  return ConstValue{out};
-}
-
-template <typename T>
-inline std::string KernelInfoImpl<T>::GetNodeName() const {
-  size_t size = 0;
-
-  // Feed nullptr for the data buffer to query the true size of the string value
-  Ort::ThrowOnError(GetApi().KernelInfo_GetNodeName(this->p_, nullptr, &size));
-
-  std::string out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().KernelInfo_GetNodeName(this->p_, &out[0], &size));
-  out.resize(size - 1);  // remove the terminating character '\0'
-
-  return out;
-}
-
-template <typename T>
-inline Logger KernelInfoImpl<T>::GetLogger() const {
-  const OrtLogger* out = nullptr;
-  ThrowOnError(GetApi().KernelInfo_GetLogger(this->p_, &out));
-  return Logger{out};
-}
-
-inline void attr_utils::GetAttr(const OrtKernelInfo* p, const char* name, float& out) {
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttribute_float(p, name, &out));
-}
-
-inline void attr_utils::GetAttr(const OrtKernelInfo* p, const char* name, int64_t& out) {
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttribute_int64(p, name, &out));
-}
-
-inline void attr_utils::GetAttr(const OrtKernelInfo* p, const char* name, std::string& result) {
-  size_t size = 0;
-  // Feed nullptr for the data buffer to query the true size of the string attribute
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttribute_string(p, name, nullptr, &size));
-
-  std::string out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttribute_string(p, name, &out[0], &size));
-  out.resize(size - 1);  // remove the terminating character '\0'
-  out.swap(result);
-}
-
-inline void attr_utils::GetAttrs(const OrtKernelInfo* p, const char* name, std::vector<float>& result) {
-  size_t size = 0;
-  // Feed nullptr for the data buffer to query the true size of the attribute
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttributeArray_float(p, name, nullptr, &size));
-
-  std::vector<float> out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttributeArray_float(p, name, out.data(), &size));
-  out.swap(result);
-}
-
-inline void attr_utils::GetAttrs(const OrtKernelInfo* p, const char* name, std::vector<int64_t>& result) {
-  size_t size = 0;
-
-  // Feed nullptr for the data buffer to query the true size of the attribute
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttributeArray_int64(p, name, nullptr, &size));
-
-  std::vector<int64_t> out;
-  out.resize(size);
-  Ort::ThrowOnError(GetApi().KernelInfoGetAttributeArray_int64(p, name, out.data(), &size));
-  out.swap(result);
-}
-}  // namespace detail
-
-inline KernelInfo::KernelInfo(OrtKernelInfo* info) : detail::KernelInfoImpl<OrtKernelInfo>{info} {}
-
-inline Op::Op(OrtOp* p) : Base<OrtOp>(p) {}
-
-inline Op Op::Create(const OrtKernelInfo* info, const char* op_name, const char* domain, int version,
-                     const char** type_constraint_names,
-                     const ONNXTensorElementDataType* type_constraint_values,
-                     size_t type_constraint_count,
-                     const OpAttr* attr_values, size_t attr_count,
-                     size_t input_count, size_t output_count) {
-  static_assert(sizeof(OpAttr) == sizeof(OrtOpAttr*),
-                "OpAttr's is expected to be just an array of OrtOpAttr in memory so we can reinterpret safely");
-  auto attr_input_values = reinterpret_cast<const OrtOpAttr* const*>(attr_values);
-  OrtOp* op;
-  Ort::ThrowOnError(GetApi().CreateOp(info, op_name, domain, version, type_constraint_names, type_constraint_values,
-                                      static_cast<int>(type_constraint_count),
-                                      attr_input_values,
-                                      static_cast<int>(attr_count),
-                                      static_cast<int>(input_count),
-                                      static_cast<int>(output_count), &op));
-  return Op{op};
-}
-
-inline void Op::Invoke(const OrtKernelContext* context,
-                       const Value* input_values,
-                       size_t input_count,
-                       Value* output_values,
-                       size_t output_count) {
-  static_assert(sizeof(Value) == sizeof(OrtValue*),
-                "Value is really just an array of OrtValue* in memory, so we can reinterpret_cast safely");
-  auto ort_input_values = reinterpret_cast<const OrtValue* const*>(input_values);
-  auto ort_output_values = reinterpret_cast<OrtValue**>(output_values);
-  Ort::ThrowOnError(GetApi().InvokeOp(context, p_, ort_input_values, static_cast<int>(input_count),
-                                      ort_output_values, static_cast<int>(output_count)));
-}
-
-inline void Op::Invoke(const OrtKernelContext* context,
-                       const OrtValue* const* input_values,
-                       size_t input_count,
-                       OrtValue* const* output_values,
-                       size_t output_count) {
-  Ort::ThrowOnError(GetApi().InvokeOp(context, p_, input_values, static_cast<int>(input_count),
-                                      output_values, static_cast<int>(output_count)));
-}
-
-inline void CustomOpApi::ThrowOnError(OrtStatus* status) {
-  Ort::ThrowOnError(status);
-}
-
-template <>
-inline float CustomOpApi::KernelInfoGetAttribute<float>(_In_ const OrtKernelInfo* info, _In_ const char* name) {
-  float out;
-  Ort::ThrowOnError(api_.KernelInfoGetAttribute_float(info, name, &out));
-  return out;
-}
-
-template <>
-inline int64_t CustomOpApi::KernelInfoGetAttribute<int64_t>(_In_ const OrtKernelInfo* info, _In_ const char* name) {
-  int64_t out;
-  Ort::ThrowOnError(api_.KernelInfoGetAttribute_int64(info, name, &out));
-  return out;
-}
-
-template <>
-inline std::string CustomOpApi::KernelInfoGetAttribute<std::string>(_In_ const OrtKernelInfo* info, _In_ const char* name) {
-  size_t size = 0;
-  std::string out;
-
-  // Feed nullptr for the data buffer to query the true size of the string attribute
-  OrtStatus* status = api_.KernelInfoGetAttribute_string(info, name, nullptr, &size);
-
-  if (status == nullptr) {
-    out.resize(size);
-    Ort::ThrowOnError(api_.KernelInfoGetAttribute_string(info, name, &out[0], &size));
-    out.resize(size - 1);  // remove the terminating character '\0'
-  } else {
-    Ort::ThrowOnError(status);
-  }
-  return out;
-}
-
-template <>
-inline std::vector<float> CustomOpApi::KernelInfoGetAttribute(_In_ const OrtKernelInfo* info, _In_ const char* name) {
-  size_t size = 0;
-  std::vector<float> out;
-
-  // Feed nullptr for the data buffer to query the true size of the attribute
-  OrtStatus* status = api_.KernelInfoGetAttributeArray_float(info, name, nullptr, &size);
-
-  if (status == nullptr) {
-    out.resize(size);
-    Ort::ThrowOnError(api_.KernelInfoGetAttributeArray_float(info, name, out.data(), &size));
-  } else {
-    Ort::ThrowOnError(status);
-  }
-  return out;
-}
-
-template <>
-inline std::vector<int64_t> CustomOpApi::KernelInfoGetAttribute(_In_ const OrtKernelInfo* info, _In_ const char* name) {
-  size_t size = 0;
-  std::vector<int64_t> out;
-
-  // Feed nullptr for the data buffer to query the true size of the attribute
-  OrtStatus* status = api_.KernelInfoGetAttributeArray_int64(info, name, nullptr, &size);
-
-  if (status == nullptr) {
-    out.resize(size);
-    Ort::ThrowOnError(api_.KernelInfoGetAttributeArray_int64(info, name, out.data(), &size));
-  } else {
-    Ort::ThrowOnError(status);
-  }
-  return out;
-}
-inline OrtTensorTypeAndShapeInfo* CustomOpApi::GetTensorTypeAndShape(_In_ const OrtValue* value) {
-  OrtTensorTypeAndShapeInfo* out;
-  Ort::ThrowOnError(api_.GetTensorTypeAndShape(value, &out));
-  return out;
-}
-
-inline size_t CustomOpApi::GetTensorShapeElementCount(_In_ const OrtTensorTypeAndShapeInfo* info) {
-  size_t out;
-  Ort::ThrowOnError(api_.GetTensorShapeElementCount(info, &out));
-  return out;
-}
-
-inline ONNXTensorElementDataType CustomOpApi::GetTensorElementType(const OrtTensorTypeAndShapeInfo* info) {
-  ONNXTensorElementDataType out;
-  Ort::ThrowOnError(api_.GetTensorElementType(info, &out));
-  return out;
-}
-
-inline size_t CustomOpApi::GetDimensionsCount(_In_ const OrtTensorTypeAndShapeInfo* info) {
-  size_t out;
-  Ort::ThrowOnError(api_.GetDimensionsCount(info, &out));
-  return out;
-}
-
-inline void CustomOpApi::GetDimensions(_In_ const OrtTensorTypeAndShapeInfo* info, _Out_ int64_t* dim_values, size_t dim_values_length) {
-  Ort::ThrowOnError(api_.GetDimensions(info, dim_values, dim_values_length));
-}
-
-inline void CustomOpApi::SetDimensions(OrtTensorTypeAndShapeInfo* info, _In_ const int64_t* dim_values, size_t dim_count) {
-  Ort::ThrowOnError(api_.SetDimensions(info, dim_values, dim_count));
-}
-
-template <typename T>
-inline T* CustomOpApi::GetTensorMutableData(_Inout_ OrtValue* value) {
-  T* data;
-  Ort::ThrowOnError(api_.GetTensorMutableData(value, reinterpret_cast<void**>(&data)));
-  return data;
-}
-
-inline const OrtMemoryInfo* CustomOpApi::GetTensorMemoryInfo(_In_ const OrtValue* value) {
-  const OrtMemoryInfo* mem_info;
-  Ort::ThrowOnError(api_.GetTensorMemoryInfo(value, &mem_info));
-  return mem_info;
-}
-
-template <typename T>
-inline const T* CustomOpApi::GetTensorData(_Inout_ const OrtValue* value) {
-  T* data = nullptr;
-  Ort::ThrowOnError(api_.GetTensorMutableData(const_cast<OrtValue*>(value), reinterpret_cast<void**>(&data)));
-  return data;
-}
-
-inline std::vector<int64_t> CustomOpApi::GetTensorShape(const OrtTensorTypeAndShapeInfo* info) {
-  size_t out;
-  Ort::ThrowOnError(api_.GetDimensionsCount(info, &out));
-  std::vector<int64_t> output(out);
-  Ort::ThrowOnError(api_.GetDimensions(info, output.data(), out));
-  return output;
-}
-
-inline void CustomOpApi::ReleaseTensorTypeAndShapeInfo(OrtTensorTypeAndShapeInfo* input) {
-  api_.ReleaseTensorTypeAndShapeInfo(input);
-}
-
-inline size_t CustomOpApi::KernelContext_GetInputCount(const OrtKernelContext* context) {
-  size_t out;
-  Ort::ThrowOnError(api_.KernelContext_GetInputCount(context, &out));
-  return out;
-}
-
-inline const OrtValue* CustomOpApi::KernelContext_GetInput(const OrtKernelContext* context, _In_ size_t index) {
-  const OrtValue* out;
-  Ort::ThrowOnError(api_.KernelContext_GetInput(context, index, &out));
-  return out;
-}
-
-inline size_t CustomOpApi::KernelContext_GetOutputCount(const OrtKernelContext* context) {
-  size_t out;
-  Ort::ThrowOnError(api_.KernelContext_GetOutputCount(context, &out));
-  return out;
-}
-
-inline OrtValue* CustomOpApi::KernelContext_GetOutput(OrtKernelContext* context, _In_ size_t index,
-                                                      _In_ const int64_t* dim_values, size_t dim_count) {
-  OrtValue* out;
-  Ort::ThrowOnError(api_.KernelContext_GetOutput(context, index, dim_values, dim_count, &out));
-  return out;
-}
-
-inline void* CustomOpApi::KernelContext_GetGPUComputeStream(const OrtKernelContext* context) {
-  void* out;
-  Ort::ThrowOnError(api_.KernelContext_GetGPUComputeStream(context, &out));
-  return out;
-}
-
-inline OrtOpAttr* CustomOpApi::CreateOpAttr(_In_ const char* name,
-                                            _In_ const void* data,
-                                            _In_ int len,
-                                            _In_ OrtOpAttrType type) {
-  OrtOpAttr* op_attr{};
-  Ort::ThrowOnError(api_.CreateOpAttr(name, data, len, type, &op_attr));
-  return op_attr;
-}
-
-inline void CustomOpApi::ReleaseOpAttr(_Frees_ptr_opt_ OrtOpAttr* op_attr) {
-  api_.ReleaseOpAttr(op_attr);
-}
-
-inline OrtOp* CustomOpApi::CreateOp(_In_ const OrtKernelInfo* info,
-                                    _In_z_ const char* op_name,
-                                    _In_z_ const char* domain,
-                                    int version,
-                                    _In_reads_(type_constraint_count) const char** type_constraint_names,
-                                    _In_reads_(type_constraint_count) const ONNXTensorElementDataType* type_constraint_values,
-                                    int type_constraint_count,
-                                    _In_reads_(attr_count) const OrtOpAttr* const* attr_values,
-                                    int attr_count,
-                                    int input_count,
-                                    int output_count) {
-  OrtOp* ort_op{};
-  Ort::ThrowOnError(api_.CreateOp(info, op_name, domain, version, type_constraint_names, type_constraint_values,
-                                  type_constraint_count, attr_values, attr_count, input_count, output_count, &ort_op));
-  return ort_op;
-}
-
-inline void CustomOpApi::InvokeOp(_In_ const OrtKernelContext* context,
-                                  _In_ const OrtOp* ort_op,
-                                  _In_ const OrtValue* const* input_values,
-                                  _In_ int input_count,
-                                  _Inout_ OrtValue* const* output_values,
-                                  _In_ int output_count) {
-  Ort::ThrowOnError(api_.InvokeOp(context, ort_op, input_values, input_count, output_values, output_count));
-}
-
-inline void CustomOpApi::ReleaseOp(_Frees_ptr_opt_ OrtOp* ort_op) {
-  api_.ReleaseOp(ort_op);
-}
-
-inline OrtKernelInfo* CustomOpApi::CopyKernelInfo(_In_ const OrtKernelInfo* info) {
-  OrtKernelInfo* info_copy{};
-  Ort::ThrowOnError(api_.CopyKernelInfo(info, &info_copy));
-  return info_copy;
-}
-
-inline void CustomOpApi::ReleaseKernelInfo(_Frees_ptr_opt_ OrtKernelInfo* info_copy) {
-  api_.ReleaseKernelInfo(info_copy);
-}
-
-inline std::string GetVersionString() {
-  return OrtGetApiBase()->GetVersionString();
-}
-
-inline std::string GetBuildInfoString() {
-  return GetApi().GetBuildInfoString();
-}
-
-inline std::vector<std::string> GetAvailableProviders() {
-  char** providers;
-  int len;
-
-  auto release_fn = [&len](char** providers) {
-    // This should always return nullptr.
-    ThrowOnError(GetApi().ReleaseAvailableProviders(providers, len));
-  };
-
-  ThrowOnError(GetApi().GetAvailableProviders(&providers, &len));
-  std::unique_ptr<char*, decltype(release_fn)> guard(providers, release_fn);
-  std::vector<std::string> available_providers;
-  available_providers.reserve(static_cast<size_t>(len));
-  for (int i = 0; i < len; ++i) {
-    available_providers.emplace_back(providers[i]);
-  }
-  return available_providers;
-}
-
-template <typename TOp, typename TKernel>
-void CustomOpBase<TOp, TKernel>::GetSessionConfigs(std::unordered_map<std::string, std::string>& out,
-                                                   ConstSessionOptions options) const {
-  const TOp* derived = static_cast<const TOp*>(this);
-  std::vector<std::string> keys = derived->GetSessionConfigKeys();
-
-  out.reserve(keys.size());
-
-  std::string config_entry_key = detail::MakeCustomOpConfigEntryKey(derived->GetName(), "");
-  const size_t prefix_size = config_entry_key.length();
-
-  for (const auto& key : keys) {
-    config_entry_key.resize(prefix_size);
-    config_entry_key.append(key);
-    out[key] = options.GetConfigEntryOrDefault(config_entry_key.c_str(), "");
-  }
-}
-
-}  // namespace Ort

v1.15.1/onnxruntime-linux-armhf/include/onnxruntime_run_options_config_keys.h

@@ -1,32 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-#pragma once
-
-/*
- * This file defines RunOptions Config Keys and format of the Config Values.
- *
- * The Naming Convention for a RunOptions Config Key,
- * "[Area][.[SubArea1].[SubArea2]...].[Keyname]"
- * Such as "ep.cuda.use_arena"
- * The Config Key cannot be empty
- * The maximum length of the Config Key is 128
- *
- * The string format of a RunOptions Config Value is defined individually for each Config.
- * The maximum length of the Config Value is 1024
- */
-
-// Key for enabling shrinkages of user listed device memory arenas.
-// Expects a list of semi-colon separated key value pairs separated by colon in the following format:
-// "device_0:device_id_0;device_1:device_id_1"
-// No white-spaces allowed in the provided list string.
-// Currently, the only supported devices are : "cpu", "gpu" (case sensitive).
-// If "cpu" is included in the list, DisableCpuMemArena() API must not be called (i.e.) arena for cpu should be enabled.
-// Example usage: "cpu:0;gpu:0" (or) "gpu:0"
-// By default, the value for this key is empty (i.e.) no memory arenas are shrunk
-static const char* const kOrtRunOptionsConfigEnableMemoryArenaShrinkage = "memory.enable_memory_arena_shrinkage";
-
-// Set to '1' to not synchronize execution providers with CPU at the end of session run.
-// Per default it will be set to '0'
-// Taking CUDA EP as an example, it omit triggering cudaStreamSynchronize on the compute stream.
-static const char* const kOrtRunOptionsConfigDisableSynchronizeExecutionProviders = "disable_synchronize_execution_providers";

v1.15.1/onnxruntime-linux-armhf/include/onnxruntime_session_options_config_keys.h

@@ -1,199 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-#pragma once
-
-/*
- * This file defines SessionOptions Config Keys and format of the Config Values.
- *
- * The Naming Convention for a SessionOptions Config Key,
- * "[Area][.[SubArea1].[SubArea2]...].[Keyname]"
- * Such as "ep.cuda.use_arena"
- * The Config Key cannot be empty
- * The maximum length of the Config Key is 128
- *
- * The string format of a SessionOptions Config Value is defined individually for each Config.
- * The maximum length of the Config Value is 1024
- */
-
-// Key for disable PrePacking,
-// If the config value is set to "1" then the prepacking is disabled, otherwise prepacking is enabled (default value)
-static const char* const kOrtSessionOptionsConfigDisablePrepacking = "session.disable_prepacking";
-
-// A value of "1" means allocators registered in the env will be used. "0" means the allocators created in the session
-// will be used. Use this to override the usage of env allocators on a per session level.
-static const char* const kOrtSessionOptionsConfigUseEnvAllocators = "session.use_env_allocators";
-
-// Set to 'ORT' (case sensitive) to load an ORT format model.
-// If unset, model type will default to ONNX unless inferred from filename ('.ort' == ORT format) or bytes to be ORT
-static const char* const kOrtSessionOptionsConfigLoadModelFormat = "session.load_model_format";
-
-// Set to 'ORT' (case sensitive) to save optimized model in ORT format when SessionOptions.optimized_model_path is set.
-// If unset, format will default to ONNX unless optimized_model_filepath ends in '.ort'.
-static const char* const kOrtSessionOptionsConfigSaveModelFormat = "session.save_model_format";
-
-// If a value is "1", flush-to-zero and denormal-as-zero are applied. The default is "0".
-// When multiple sessions are created, a main thread doesn't override changes from succeeding session options,
-// but threads in session thread pools follow option changes.
-// When ORT runs with OpenMP, the same rule is applied, i.e. the first session option to flush-to-zero and
-// denormal-as-zero is only applied to global OpenMP thread pool, which doesn't support per-session thread pool.
-// Note that an alternative way not using this option at runtime is to train and export a model without denormals
-// and that's recommended because turning this option on may hurt model accuracy.
-static const char* const kOrtSessionOptionsConfigSetDenormalAsZero = "session.set_denormal_as_zero";
-
-// It controls to run quantization model in QDQ (QuantizelinearDeQuantizelinear) format or not.
-// "0": enable. ORT does fusion logic for QDQ format.
-// "1": disable. ORT doesn't do fusion logic for QDQ format.
-// Its default value is "0" unless the DirectML execution provider is registered, in which case it defaults to "1".
-static const char* const kOrtSessionOptionsDisableQuantQDQ = "session.disable_quant_qdq";
-
-// It controls whether to enable Double QDQ remover and Identical Children Consolidation
-// "0": not to disable. ORT does remove the middle 2 Nodes from a Q->(QD->Q)->QD pairs
-// "1": disable. ORT doesn't remove the middle 2 Nodes from a Q->(QD->Q)->QD pairs
-// Its default value is "0"
-static const char* const kOrtSessionOptionsDisableDoubleQDQRemover = "session.disable_double_qdq_remover";
-
-// If set to "1", enables the removal of QuantizeLinear/DequantizeLinear node pairs once all QDQ handling has been
-// completed. e.g. If after all QDQ handling has completed and we have -> FloatOp -> Q -> DQ -> FloatOp -> the
-// Q -> DQ could potentially be removed. This will provide a performance benefit by avoiding going from float to
-// 8-bit and back to float, but could impact accuracy. The impact on accuracy will be model specific and depend on
-// other factors like whether the model was created using Quantization Aware Training or Post Training Quantization.
-// As such, it's best to test to determine if enabling this works well for your scenario.
-// The default value is "0"
-// Available since version 1.11.
-static const char* const kOrtSessionOptionsEnableQuantQDQCleanup = "session.enable_quant_qdq_cleanup";
-
-// Enable or disable gelu approximation in graph optimization. "0": disable; "1": enable. The default is "0".
-// GeluApproximation has side effects which may change the inference results. It is disabled by default due to this.
-static const char* const kOrtSessionOptionsEnableGeluApproximation = "optimization.enable_gelu_approximation";
-
-#ifdef ENABLE_TRAINING
-// Specifies a list of op types for memory footprint reduction.
-// The value should be a ","-delimited list of pair of
-// <subgraph string : optimization strategy : number of subgraph to apply>.
-// For example, "Gelu+Cast+:1:0,Dropout+:1:1".
-//   A valid "subgraph string" should be one subgraph representation output by ORT graph transformations.
-//   "optimization strategy" currently has valid values: 0 - disabled, 1 - recompute.
-//   "number of subgraph to apply" is used to control how many subgraphs to apply optimization, to avoid "oversaving"
-//   the memory.
-static const char* const kOrtSessionOptionsMemoryOptimizerEnabler = "optimization.enable_memory_optimizer";
-
-// Specifies the level for detecting subgraphs for memory footprint reduction.
-// The value should be an integer. The default value is 0.
-static const char* const kOrtSessionOptionsMemoryOptimizerProbeLevel = "optimization.enable_memory_probe_recompute_level";
-#endif
-
-// Enable or disable using device allocator for allocating initialized tensor memory. "1": enable; "0": disable. The default is "0".
-// Using device allocators means the memory allocation is made using malloc/new.
-static const char* const kOrtSessionOptionsUseDeviceAllocatorForInitializers = "session.use_device_allocator_for_initializers";
-
-// Configure whether to allow the inter_op/intra_op threads spinning a number of times before blocking
-// "0": thread will block if found no job to run
-// "1": default, thread will spin a number of times before blocking
-static const char* const kOrtSessionOptionsConfigAllowInterOpSpinning = "session.inter_op.allow_spinning";
-static const char* const kOrtSessionOptionsConfigAllowIntraOpSpinning = "session.intra_op.allow_spinning";
-
-// Key for using model bytes directly for ORT format
-// If a session is created using an input byte array contains the ORT format model data,
-// By default we will copy the model bytes at the time of session creation to ensure the model bytes
-// buffer is valid.
-// Setting this option to "1" will disable copy the model bytes, and use the model bytes directly. The caller
-// has to guarantee that the model bytes are valid until the ORT session using the model bytes is destroyed.
-static const char* const kOrtSessionOptionsConfigUseORTModelBytesDirectly = "session.use_ort_model_bytes_directly";
-
-/// <summary>
-/// Key for using the ORT format model flatbuffer bytes directly for initializers.
-/// This avoids copying the bytes and reduces peak memory usage during model loading and initialization.
-/// Requires `session.use_ort_model_bytes_directly` to be true.
-/// If set, the flatbuffer bytes provided when creating the InferenceSession MUST remain valid for the entire
-/// duration of the InferenceSession.
-/// </summary>
-static const char* const kOrtSessionOptionsConfigUseORTModelBytesForInitializers =
-    "session.use_ort_model_bytes_for_initializers";
-
-// This should only be specified when exporting an ORT format model for use on a different platform.
-// If the ORT format model will be used on ARM platforms set to "1". For other platforms set to "0"
-// Available since version 1.11.
-static const char* const kOrtSessionOptionsQDQIsInt8Allowed = "session.qdqisint8allowed";
-
-// x64 SSE4.1/AVX2/AVX512(with no VNNI) has overflow problem with quantizied matrix multiplication with U8S8.
-// To avoid this we need to use slower U8U8 matrix multiplication instead. This option, if
-// turned on, use slower U8U8 matrix multiplications. Only effective with AVX2 or AVX512
-// platforms.
-static const char* const kOrtSessionOptionsAvx2PrecisionMode = "session.x64quantprecision";
-
-// Specifies how minimal build graph optimizations are handled in a full build.
-// These optimizations are at the extended level or higher.
-// Possible values and their effects are:
-// "save": Save runtime optimizations when saving an ORT format model.
-// "apply": Only apply optimizations available in a minimal build.
-// ""/<unspecified>: Apply optimizations available in a full build.
-// Available since version 1.11.
-static const char* const kOrtSessionOptionsConfigMinimalBuildOptimizations =
-    "optimization.minimal_build_optimizations";
-
-// Note: The options specific to an EP should be specified prior to appending that EP to the session options object in
-// order for them to take effect.
-
-// Specifies a list of stop op types. Nodes of a type in the stop op types and nodes downstream from them will not be
-// run by the NNAPI EP.
-// The value should be a ","-delimited list of op types. For example, "Add,Sub".
-// If not specified, the default set of stop ops is used. To specify an empty stop ops types list and disable stop op
-// exclusion, set the value to "".
-static const char* const kOrtSessionOptionsConfigNnapiEpPartitioningStopOps = "ep.nnapi.partitioning_stop_ops";
-
-// Enabling dynamic block-sizing for multithreading.
-// With a positive value, thread pool will split a task of N iterations to blocks of size starting from:
-// N / (num_of_threads * dynamic_block_base)
-// As execution progresses, the size will decrease according to the diminishing residual of N,
-// meaning the task will be distributed in smaller granularity for better parallelism.
-// For some models, it helps to reduce the variance of E2E inference latency and boost performance.
-// The feature will not function by default, specify any positive integer, e.g. "4", to enable it.
-// Available since version 1.11.
-static const char* const kOrtSessionOptionsConfigDynamicBlockBase = "session.dynamic_block_base";
-
-// This option allows to decrease CPU usage between infrequent
-// requests and forces any TP threads spinning stop immediately when the last of
-// concurrent Run() call returns.
-// Spinning is restarted on the next Run() call.
-// Applies only to internal thread-pools
-static const char* const kOrtSessionOptionsConfigForceSpinningStop = "session.force_spinning_stop";
-
-// "1": all inconsistencies encountered during shape and type inference
-// will result in failures.
-// "0": in some cases warnings will be logged but processing will continue. The default.
-// May be useful to expose bugs in models.
-static const char* const kOrtSessionOptionsConfigStrictShapeTypeInference = "session.strict_shape_type_inference";
-
-// The file saves configuration for partitioning node among logic streams
-static const char* const kNodePartitionConfigFile = "session.node_partition_config_file";
-
-// This Option allows setting affinities for intra op threads.
-// Affinity string follows format:
-// logical_processor_id,logical_processor_id;logical_processor_id,logical_processor_id
-// Semicolon isolates configurations among threads, while comma split processors where ith thread expected to attach to.
-// e.g.1,2,3;4,5
-// specifies affinities for two threads, with the 1st thread attach to the 1st, 2nd, and 3rd processor, and 2nd thread to the 4th and 5th.
-// To ease the configuration, an "interval" is also allowed:
-// e.g. 1-8;8-16;17-24
-// orders that the 1st thread runs on first eight processors, 2nd thread runs on next eight processors, and so forth.
-// Note:
-// 1. Once set, the number of thread affinities must equal to intra_op_num_threads - 1, since ort does not set affinity on the main thread which
-//    is started and managed by the calling app;
-// 2. For windows, ort will infer the group id from a logical processor id, for example, assuming there are two groups with each has 64 logical processors,
-//    an id of 64 will be inferred as the last processor of the 1st group, while 65 will be interpreted as the 1st processor of the second group.
-//    Hence 64-65 is an invalid configuration, because a windows thread cannot be attached to processors across group boundary.
-static const char* const kOrtSessionOptionsConfigIntraOpThreadAffinities = "session.intra_op_thread_affinities";
-
-// This option will dump out the model to assist debugging any issues with layout transformation,
-// and is primarily intended for developer usage. It is only relevant if an execution provider that requests
-// NHWC layout is enabled such as NNAPI, XNNPACK or QNN.
-//
-// Default is off. Set to "1" to enable.
-//
-// If modified by layout transformation the model will be dumped after these steps:
-//   1) insertion of the layout transformation Transpose nodes
-//   2) after those are optimized using the transpose optimizer,
-//   3) after the L1 transformers are applied to the updated graph.
-// The model will be saved to filename post_layout_transform_step_<step_number>.onnx.
-static const char* const kDebugLayoutTransformation = "session.debug_layout_transformation";

v1.15.1/onnxruntime-linux-armhf/include/onnxruntime_training_c_api.h

@@ -1,630 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-// This file contains the training c apis.
-
-#pragma once
-#include <stdbool.h>
-#include "onnxruntime_c_api.h"
-
-/** \page training_c_cpp_api Training C & C++ APIs
- *
- * Training C and C++ APIs are an extension of the \ref c_cpp_api "onnxruntime core C and C++ APIs" and should be used in conjunction with them.
- *
- * In order to train a model with onnxruntime, the following training artifacts must be generated:
- * - The training onnx model
- * - The checkpoint directory
- * - The optimizer onnx model
- * - The eval onnx model model (optional)
- *
- * These training artifacts can be generated as part of an offline step using the python [utilities](https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/python/training/onnxblock/README.md) made available in the `onnxruntime-training` python package.
- *
- * After these artifacts have been generated, the C and C++ utilities listed in this documentation can be leveraged to perform training.
- *
- * If any problem is encountered, please create an [issue](https://github.com/microsoft/onnxruntime/issues/new) with your scenario and requirements, and we will be sure to respond and follow up on the request.
- *
- * <h1>Training C API</h1>
- *
- * ::OrtTrainingApi - Training C API functions.
- *
- * This C structure contains functions that enable users to perform training with onnxruntime.
- *
- * _Sample Code_:
- *
- * ```c
- * #include <onnxruntime_training_api.h>
- *
- * OrtApi* g_ort_api = OrtGetApiBase()->GetApi(ORT_API_VERSION);
- * OrtTrainingApi* g_ort_training_api = g_ort_api->GetTrainingApi(ORT_API_VERSION);
- *
- * OrtEnv* env = NULL;
- * g_ort_api->CreateEnv(logging_level, logid, &env);
- * OrtSessionOptions* session_options = NULL;
- * g_ort_api->CreateSessionOptions(&session_options);
- *
- * OrtCheckpointState* state = NULL;
- * g_ort_training_api->LoadCheckpoint(path_to_checkpoint, &state);
- *
- * OrtTrainingSession* training_session = NULL;
- * g_ort_training_api->CreateTrainingSession(env, session_options, training_model_path,
- *                                           state, eval_model_path, optimizer_model_path,
- *                                           &training_session);
- * // Training loop
- * {
- *     g_ort_training_api->TrainStep(...);
- *     g_ort_training_api->OptimizerStep(...);
- *     g_ort_training_api->LazyResetGrad(...);
- * }
- *
- * g_ort_training_api->ExportModelForInferencing(training_session, inference_model_path, ...);
- * g_ort_training_api->SaveCheckpoint(state, path_to_checkpoint, false);
- *
- * g_ort_training_api->ReleaseTrainingSession(training_session);
- * g_ort_training_api->ReleaseCheckpointState(state);
- * ```
- *
- * > **Note**
- * > The ::OrtCheckpointState contains the entire training state that the ::OrtTrainingSession uses. As a result, the training session must always have access to the state. That is to say, the ::OrtCheckpointState instance must outlive the lifetime of the ::OrtTrainingSession instance.
- *
- * <h1>Training C++ API</h1>
- *
- * @ref TrainingCpp - Training C++ API classes and functions.
- *
- * These C++ classes and functions enable users to perform training with onnxruntime.
- *
- * _Sample Code_:
- *
- * ```cc
- * #include <onnxruntime_training_cxx_api.h>
- *
- * Ort::Env env;
- * Ort::SessionOptions session_options;
- *
- * auto state = Ort::CheckpointState::LoadCheckpoint(path_to_checkpoint);
- * auto training_session = Ort::TrainingSession(env, session_options, state, training_model_path,
- *                                              eval_model_path, optimizer_model_path);
- *
- * // Training Loop
- * {
- *     training_session.TrainStep(...);
- *     training_session.OptimizerStep(...);
- *     training_session.LazyResetGrad(...);
- * }
- *
- * training_session->ExportModelForInferencing(inference_model_path, ...);
- * Ort::CheckpointState::SaveCheckpoint(state, path_to_checkpoint, false);
- * ```
- * > **Note**
- * > The ::Ort::CheckpointState contains the entire training state that the ::Ort::TrainingSession uses. As a result, the training session must always have access to the state. That is to say, the ::Ort::CheckpointState instance must outlive the lifetime of the ::Ort::TrainingSession instance.
- */
-
-/** @defgroup TrainingC Ort Training C API
- * @{
- */
-ORT_RUNTIME_CLASS(TrainingSession);  // Type that enables performing training for the given user models.
-ORT_RUNTIME_CLASS(CheckpointState);  // Type that holds the training states for the training session.
-
-/** \brief Type of property to be added to or returned from the ::OrtCheckpointState.
- */
-typedef enum OrtPropertyType {
-  OrtIntProperty = 0,
-  OrtFloatProperty = 1,
-  OrtStringProperty = 2,
-} OrtPropertyType;
-
-/** \brief The Training C API that holds onnxruntime training function pointers
- *
- * All the Training C API functions are defined inside this structure as pointers to functions.
- * Call OrtApi::GetTrainingApi to get a pointer to this struct.
- *
- * \nosubgrouping
- */
-struct OrtTrainingApi {
-  /// \name Accessing The Training Session State
-  /// @{
-
-  /** \brief Load a checkpoint state from directory on disk into checkpoint_state.
-   *
-   * This function will parse a checkpoint directory, pull relevant files and load the training
-   * state into the checkpoint_state. This checkpoint state can then be used to create the
-   * training session by invoking OrtTrainingApi::CreateTrainingSession. By doing so, the training
-   * session will resume training from the given checkpoint state.
-   * \note Note that the training session created with a checkpoint state uses this state to store the entire
-   * training state (including model parameters, its gradients, the optimizer states and the properties).
-   * As a result, it is required that the checkpoint state outlive the lifetime of the training session.
-   *
-   * \param[in] checkpoint_path Path to the checkpoint directory
-   * \param[out] checkpoint_state Checkpoint state that contains the states of the training session.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(LoadCheckpoint, _In_ const ORTCHAR_T* checkpoint_path,
-                  _Outptr_ OrtCheckpointState** checkpoint_state);
-
-  /** \brief Save the given state to a checkpoint directory on disk.
-   *
-   * This function serializes the provided checkpoint state to a directory on disk.
-   * This checkpoint can later be loaded by invoking OrtTrainingApi::LoadCheckpoint to resume
-   * training from this snapshot of the state.
-   *
-   * \param[in] checkpoint_state The checkpoint state to save.
-   * \param[in] checkpoint_path Path to the checkpoint directory.
-   * \param[in] include_optimizer_state Flag to indicate whether to save the optimizer state or not.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(SaveCheckpoint, _In_ OrtCheckpointState* checkpoint_state, _In_ const ORTCHAR_T* checkpoint_path,
-                  const bool include_optimizer_state);
-
-  /// @}
-
-  /// \name Implementing The Training Loop
-  /// @{
-  /** \brief Create a training session that can be used to begin or resume training.
-   *
-   * This function creates a training session based on the env and session options provided that can
-   * begin or resume training from a given checkpoint state for the given onnx models.
-   * The checkpoint state represents the parameters of the training session which will be moved
-   * to the device specified by the user through the session options (if necessary).
-   * The training session requires four training artifacts
-   * - The training onnx model
-   * - The evaluation onnx model (optional)
-   * - The optimizer onnx model
-   * - The checkpoint directory
-   *
-   * These artifacts can be generated using the `onnxruntime-training` python [utility](https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/python/training/onnxblock/README.md).
-   *
-   * \param[in] env Environment to be used for the training session.
-   * \param[in] options Session options that the user can customize for this training session.
-   * \param[in] checkpoint_state Training states that the training session uses as a starting point for training.
-   * \param[in] train_model_path Model to be used to perform training.
-   * \param[in] eval_model_path Model to be used to perform evaluation.
-   * \param[in] optimizer_model_path Model to be used to perform gradient descent.
-   * \param[out] out Created training session.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(CreateTrainingSession, _In_ const OrtEnv* env, _In_ const OrtSessionOptions* options,
-                  _Inout_ OrtCheckpointState* checkpoint_state, _In_ const ORTCHAR_T* train_model_path,
-                  _In_ const ORTCHAR_T* eval_model_path, _In_ const ORTCHAR_T* optimizer_model_path,
-                  _Outptr_ OrtTrainingSession** out);
-
-  /// @}
-
-  /// \name Model IO Information
-  /// @{
-
-  /** \brief Retrieves the number of user outputs in the training model.
-   *
-   * This function returns the number of outputs of the training model so that the user can
-   * allocate space for the number of outputs when OrtTrainingApi::TrainStep is invoked.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[out] out Number of user outputs in the training model.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetTrainingModelOutputCount, _In_ const OrtTrainingSession* sess, _Out_ size_t* out);
-
-  /** \brief Retrieves the number of user outputs in the eval model.
-   *
-   * This function returns the number of outputs of the eval model so that the user can
-   * allocate space for the number of outputs when OrtTrainingApi::EvalStep is invoked.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[out] out Number of user outputs in the eval model.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetEvalModelOutputCount, _In_ const OrtTrainingSession* sess, _Out_ size_t* out);
-
-  /** \brief Retrieves the names of user outputs in the training model.
-   *
-   * This function returns the names of outputs of the training model that can be associated with the OrtValue(s)
-   * returned by the OrtTrainingApi::TrainStep function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] index Index of the output name requested.
-   * \param[in] allocator Allocator to use to allocate the memory for the name.
-   * \param[out] output Name of the training model output at the given index.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetTrainingModelOutputName, _In_ const OrtTrainingSession* sess, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** output);
-
-  /** \brief Retrieves the names of user outputs in the eval model.
-   *
-   * This function returns the names of outputs of the eval model that can be associated with the OrtValue(s) returned
-   * by the OrtTrainingApi::EvalStep function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] index Index of the output name requested.
-   * \param[in] allocator Allocator to use to allocate the memory for the name.
-   * \param[out] output Name of the eval model output at the given index.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetEvalModelOutputName, _In_ const OrtTrainingSession* sess, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** output);
-
-  /// @}
-
-  /// \name Implementing The Training Loop
-  /// @{
-
-  /** \brief Reset the gradients of all trainable parameters to zero lazily.
-   *
-   * This function sets the internal state of the training session such that the gradients of the trainable
-   * parameters in the OrtCheckpointState will be scheduled to be reset just before the new gradients are
-   * computed on the next invocation of the next OrtTrainingApi::TrainStep.
-   *
-   * \param[in] session The `this` pointer to the training session.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(LazyResetGrad, _Inout_ OrtTrainingSession* session);
-
-  /** \brief Computes the outputs of the training model and the gradients of the trainable parameters for the given inputs
-   *
-   * This function performs a training step that computes the outputs of the training model and the gradients
-   * of the trainable parameters for the given inputs. The train step is performed based on the training model
-   * that was provided to the training session.
-   * The OrtTrainingApi::TrainStep is equivalent of running forward propagation and backward propagation in a single
-   * step.
-   * The gradients computed are stored inside the training session state so they can be later consumed
-   * by the OrtTrainingApi::OptimizerStep function.
-   * The gradients can be lazily reset by invoking the OrtTrainingApi::LazyResetGrad function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] run_options Run options for this training step.
-   * \param[in] inputs_len Number of user inputs to the training model.
-   * \param[in] inputs The user inputs to the training model.
-   * \param[in] outputs_len Number of user outputs expected from this training step.
-   * \param[out] outputs User outputs computed by train step.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainStep, _Inout_ OrtTrainingSession* sess, _In_opt_ const OrtRunOptions* run_options,
-                  _In_ size_t inputs_len, _In_reads_(inputs_len) const OrtValue* const* inputs,
-                  _In_ size_t outputs_len, _Inout_updates_all_(outputs_len) OrtValue** outputs);
-
-  /** \brief Computes the outputs for the eval model for the given inputs
-   *
-   * This function performs an eval step that computes the outputs of the eval model for the given inputs.
-   * The eval step is performed based on the eval model that was provided to the training session.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] run_options Run options for this eval step.
-   * \param[in] inputs_len Number of user inputs to the eval model.
-   * \param[in] inputs The user inputs to the eval model.
-   * \param[in] outputs_len Number of user outputs expected from this eval step.
-   * \param[out] outputs User outputs computed by eval step.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(EvalStep, _In_ const OrtTrainingSession* sess, _In_opt_ const OrtRunOptions* run_options,
-                  _In_ size_t inputs_len, _In_reads_(inputs_len) const OrtValue* const* inputs,
-                  _In_ size_t outputs_len, _Inout_updates_all_(outputs_len) OrtValue** outputs);
-
-  /** \brief Sets the learning rate for this training session.
-   *
-   * This function allows users to set the learning rate for the training session. The current
-   * learning rate is maintained by the training session and can be overwritten by invoking
-   * this function with the desired learning rate. This function should not be used when a valid
-   * learning rate scheduler is registered. It should be used either to set the learning rate
-   * derived from a custom learning rate scheduler or to set a constant learning rate to be used
-   * throughout the training session.
-   * \note Please note that this function does not set the initial learning rate that may be needed
-   * by the predefined learning rate schedulers. To set the initial learning rate for learning
-   * rate schedulers, please look at the function OrtTrainingApi::RegisterLinearLRScheduler.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] learning_rate Desired learning rate to be set.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(SetLearningRate, _Inout_ OrtTrainingSession* sess, _In_ float learning_rate);
-
-  /** \brief Gets the current learning rate for this training session.
-   *
-   * This function allows users to get the learning rate for the training session. The current
-   * learning rate is maintained by the training session, and users can query it for the purpose
-   * of implementing their own learning rate schedulers.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[out] learning_rate Learning rate currently in use by the training session.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(GetLearningRate, _Inout_ OrtTrainingSession* sess, _Out_ float* learning_rate);
-
-  /** \brief Performs the weight updates for the trainable parameters using the optimizer model.
-   *
-   * This function performs the weight update step that updates the trainable parameters such that they
-   * take a step in the direction of their gradients (gradient descent). The optimizer step is performed
-   * based on the optimizer model that was provided to the training session.
-   * The updated parameters are stored inside the training state so that they can be used by the next
-   * OrtTrainingApi::TrainStep function call.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] run_options Run options for this optimizer step.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(OptimizerStep, _Inout_ OrtTrainingSession* sess,
-                  _In_opt_ const OrtRunOptions* run_options);
-
-  /** \brief Registers a linear learning rate scheduler for the training session.
-   *
-   * Register a linear learning rate scheduler that decays the learning rate by linearly updated
-   * multiplicative factor from the initial learning rate set on the training session to 0. The decay
-   * is performed after the initial warm up phase where the learning rate is linearly incremented
-   * from 0 to the initial learning rate provided.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] warmup_step_count Warmup steps for LR warmup.
-   * \param[in] total_step_count Total step count.
-   * \param[in] initial_lr The initial learning rate to be used by the training session.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(RegisterLinearLRScheduler, _Inout_ OrtTrainingSession* sess, _In_ const int64_t warmup_step_count,
-                  _In_ const int64_t total_step_count, _In_ const float initial_lr);
-
-  /** \brief Update the learning rate based on the registered learing rate scheduler.
-   *
-   * Takes a scheduler step that updates the learning rate that is being used by the training session.
-   * This function should typically be called before invoking the optimizer step for each round,
-   * or as determined necessary to update the learning rate being used by the training session.
-   * \note Please note that a valid predefined learning rate scheduler must be first registered to invoke this
-   * function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(SchedulerStep, _Inout_ OrtTrainingSession* sess);
-
-  /// @}
-
-  /// \name Accessing The Training Session State
-  /// @{
-  /** \brief Retrieves the size of all the parameters.
-   *
-   * Calculates the total number of primitive (datatype of the parameters) elements of all the parameters in the
-   * training state.
-   * When trainable_only argument is true, the size is calculated for trainable params only.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[out] out Size of all parameter elements.
-   * \param[in] trainable_only Whether to skip non-trainable parameters
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(GetParametersSize, _Inout_ OrtTrainingSession* sess, _Out_ size_t* out, bool trainable_only);
-
-  /** \brief Copy all parameters to a contiguous buffer held by the argument parameters_buffer
-   *
-   * The parameters_buffer has to be of the size given by GetParametersSize api call,
-   * with matching setting for the argument trainable_only. All the target parameters must be of the same
-   * datatype. The OrtValue must be pre-allocated onto
-   * the desired device. This is a complementary function to OrtTrainingApi::CopyBufferToParameters.
-   * Parameter ordering is preserved.
-   * User is responsible for allocating and freeing the resources used by the parameters_buffer.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] trainable_only Whether to skip non-trainable parameters
-   * \param[out] parameters_buffer The pre-allocated OrtValue buffer to copy onto.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(CopyParametersToBuffer, _Inout_ OrtTrainingSession* sess,
-                  _Inout_ OrtValue* parameters_buffer, bool trainable_only);
-
-  /** \brief Copy parameter values from the given contiguous buffer held by parameters_buffer to the training state
-   *
-   * The parameters_buffer argument has to be of the size given by OrtTrainingApi::GetParametersSize api call,
-   * with matching setting for trainable_only argument. All the target parameters must be of the same
-   * datatype. This is a complementary function to OrtTrainingApi::CopyBufferToParameters
-   * and can be used to load updated buffer values onto the training state.
-   * Parameter ordering is preserved.
-   * User is responsible for allocating and freeing the resources used by the parameters_buffer.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] trainable_only Whether to skip non-trainable parameters
-   * \param[out] parameters_buffer The pre-allocated OrtValue buffer to copy from.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(CopyBufferToParameters, _Inout_ OrtTrainingSession* sess,
-                  _Inout_ OrtValue* parameters_buffer, bool trainable_only);
-
-  /// @}
-
-  /// \name Release Training Resources
-  /// @{
-
-  /** \brief Frees up the memory used up by the training session.
-   *
-   * This function frees up any memory that was allocated in the training session. The training
-   * session can no longer be used after this call.
-   *
-   */
-  ORT_CLASS_RELEASE(TrainingSession);
-
-  /** \brief Frees up the memory used up by the checkpoint state.
-   *
-   * This function frees up any memory that was allocated in the checkpoint state. The checkpoint
-   * state can no longer be used after this call.
-   * \note Note that the checkpoint state must be released only after the training session has been released.
-   *
-   */
-  ORT_CLASS_RELEASE(CheckpointState);
-
-  /// @}
-
-  /// \name Prepare For Inferencing
-  /// @{
-  /** \brief Export a model that can be used for inferencing.
-   *
-   * If the training session was provided with an eval model, the training session can generate
-   * an inference model if it knows the inference graph outputs. The input inference graph outputs
-   * are used to prune the eval model so that the inference model's outputs align with the provided outputs.
-   * The exported model is saved at the path provided and can be used for inferencing with InferenceSession.
-   * \note Note that the function re-loads the eval model from the path provided to OrtTrainingApi::CreateTrainingSession
-   * and expects that this path still be valid.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] inference_model_path Path where the inference model should be serialized to.
-   * \param[in] graph_outputs_len Size of the graph output names array.
-   * \param[in] graph_output_names Names of the outputs that are needed in the inference model.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(ExportModelForInferencing, _Inout_ OrtTrainingSession* sess,
-                  _In_ const ORTCHAR_T* inference_model_path, size_t graph_outputs_len,
-                  _In_reads_(graph_outputs_len) const char* const* graph_output_names);
-
-  /// @}
-
-  /// \name Training Utilities
-  /// @{
-  /** \brief Sets the seed used for random number generation in Onnxruntime.
-   *
-   * Use this function to generate reproducible results. It should be noted that completely reproducible
-   * results are not guaranteed.
-   *
-   * \param[in] seed The seed to be set.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(SetSeed, _In_ const int64_t seed);
-
-  /// @}
-
-  /// \name Model IO Information
-  /// @{
-  /** \brief Retrieves the number of user inputs in the training model.
-   *
-   * This function returns the number of inputs of the training model so that the user can accordingly
-   * allocate the OrtValue(s) provided to the OrtTrainingApi::TrainStep function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[out] out Number of user inputs in the training model.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetTrainingModelInputCount, _In_ const OrtTrainingSession* sess, _Out_ size_t* out);
-
-  /** \brief Retrieves the number of user inputs in the eval model.
-   *
-   * This function returns the number of inputs of the eval model so that the user can accordingly
-   * allocate the OrtValue(s) provided to the OrtTrainingApi::EvalStep function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[out] out Number of user inputs in the eval model.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetEvalModelInputCount, _In_ const OrtTrainingSession* sess, _Out_ size_t* out);
-
-  /** \brief Retrieves the name of the user input at given index in the training model.
-   *
-   * This function returns the names of inputs of the training model that can be associated with the
-   * OrtValue(s) provided to the OrtTrainingApi::TrainStep function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] index The index of the training model input name requested.
-   * \param[in] allocator The allocator to use to allocate the memory for the requested name.
-   * \param[out] output Name of the user input for the training model at the given index.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetTrainingModelInputName, _In_ const OrtTrainingSession* sess, size_t index,
-                  _In_ OrtAllocator* allocator, _Outptr_ char** output);
-
-  /** \brief Retrieves the name of the user input at given index in the eval model.
-   *
-   * This function returns the names of inputs of the eval model that can be associated with the OrtValue(s) provided
-   * to the OrtTrainingApi::EvalStep function.
-   *
-   * \param[in] sess The `this` pointer to the training session.
-   * \param[in] index The index of the eval model input name requested.
-   * \param[in] allocator The allocator to use to allocate the memory for the requested name.
-   * \param[out] output Name of the user input for the eval model at the given index.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(TrainingSessionGetEvalModelInputName, _In_ const OrtTrainingSession* sess, size_t index,
-                  _In_ OrtAllocator* allocator, _Outptr_ char** output);
-
-  /// @}
-
-  /// \name Accessing The Training Session State
-  /// @{
-
-  /** \brief Adds the given property to the checkpoint state.
-   *
-   * Runtime properties such as epoch, training step, best score, and others can be added to the checkpoint
-   * state by the user if they desire by calling this function with the appropriate property name and
-   * value. The given property name must be unique to be able to successfully add the property.
-   *
-   * \param[in] checkpoint_state The checkpoint state which should hold the property.
-   * \param[in] property_name Unique name of the property being added.
-   * \param[in] property_type Type of the property associated with the given name.
-   * \param[in] property_value Property value associated with the given name.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(AddProperty, _Inout_ OrtCheckpointState* checkpoint_state,
-                  _In_ const char* property_name, _In_ enum OrtPropertyType property_type,
-                  _In_ void* property_value);
-
-  /** \brief Gets the property value associated with the given name from the checkpoint state.
-   *
-   * Gets the property value from an existing entry in the checkpoint state. The property must
-   * exist in the checkpoint state to be able to retrieve it successfully.
-   *
-   * \param[in] checkpoint_state The checkpoint state that is currently holding the property.
-   * \param[in] property_name Unique name of the property being retrieved.
-   * \param[in] allocator Allocator used to allocate the memory for the property_value.
-   * \param[out] property_type Type of the property associated with the given name.
-   * \param[out] property_value Property value associated with the given name.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  ORT_API2_STATUS(GetProperty, _In_ const OrtCheckpointState* checkpoint_state,
-                  _In_ const char* property_name, _Inout_ OrtAllocator* allocator,
-                  _Out_ enum OrtPropertyType* property_type, _Outptr_ void** property_value);
-
-  /// @}
-};
-
-typedef struct OrtTrainingApi OrtTrainingApi;
-
-/// @}

v1.15.1/onnxruntime-linux-armhf/include/onnxruntime_training_cxx_api.h

@@ -1,361 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-#pragma once
-#include "onnxruntime_training_c_api.h"
-#include <optional>
-#include <variant>
-
-namespace Ort::detail {
-
-#define ORT_DECLARE_TRAINING_RELEASE(NAME) \
-  void OrtRelease(Ort##NAME* ptr);
-
-// These release methods must be forward declared before including onnxruntime_cxx_api.h
-// otherwise class Base won't be aware of them
-ORT_DECLARE_TRAINING_RELEASE(CheckpointState);
-ORT_DECLARE_TRAINING_RELEASE(TrainingSession);
-
-}  // namespace Ort::detail
-
-#include "onnxruntime_cxx_api.h"
-
-namespace Ort {
-
-/// <summary>
-/// This function returns the C training api struct with the pointers to the ort training C functions.
-/// If using C++, please use the class instances instead of invoking the C functions directly.
-/// </summary>
-/// <returns>OrtTrainingApi struct with ort training C function pointers.</returns>
-inline const OrtTrainingApi& GetTrainingApi() { return *GetApi().GetTrainingApi(ORT_API_VERSION); }
-
-namespace detail {
-
-#define ORT_DEFINE_TRAINING_RELEASE(NAME) \
-  inline void OrtRelease(Ort##NAME* ptr) { GetTrainingApi().Release##NAME(ptr); }
-
-ORT_DEFINE_TRAINING_RELEASE(CheckpointState);
-ORT_DEFINE_TRAINING_RELEASE(TrainingSession);
-
-#undef ORT_DECLARE_TRAINING_RELEASE
-#undef ORT_DEFINE_TRAINING_RELEASE
-
-}  // namespace detail
-
-using Property = std::variant<int64_t, float, std::string>;
-
-/**
- * \defgroup TrainingCpp Ort Training C++ API
- * @{
- */
-
-/** \brief Holds the state of the training session.
- *
- * This class holds the entire training session state that includes model parameters, their gradients,
- * optimizer parameters, and user properties. The Ort::TrainingSession leverages the Ort::CheckpointState
- * by accessing and updating the contained training state.
- * \note Note that the training session created with a checkpoint state uses this state to store the entire
- * training state (including model parameters, its gradients, the optimizer states and the properties).
- * The Ort::TrainingSession does not hold a copy of the Ort::CheckpointState and as a result, it is required
- * that the checkpoint state outlive the lifetime of the training session.
- *
- */
-class CheckpointState : public detail::Base<OrtCheckpointState> {
- private:
-  CheckpointState(OrtCheckpointState* checkpoint_state) { p_ = checkpoint_state; }
-
- public:
-  // Construct the checkpoint state by loading the checkpoint by calling LoadCheckpoint
-  CheckpointState() = delete;
-
-  /// \name Accessing The Training Session State
-  /// @{
-
-  /** \brief Load a checkpoint state from directory on disk into checkpoint_state.
-   *
-   * This function will parse a checkpoint directory, pull relevant files and load the training
-   * state and return an instance of Ort::CheckpointState. This checkpoint state can then be used to create the
-   * training session by instantiating Ort::TrainingSession. By doing so, the training session will resume
-   * training from the given checkpoint state.
-   *
-   * \param[in] path_to_checkpoint Path to the checkpoint directory
-   * \return Ort::CheckpointState object which holds the state of the training session parameters.
-   *
-   */
-  static CheckpointState LoadCheckpoint(const std::basic_string<ORTCHAR_T>& path_to_checkpoint);
-
-  /** \brief Save the given state to a checkpoint directory on disk.
-   *
-   * This function serializes the provided checkpoint state to a directory on disk.
-   * This checkpoint can later be loaded by invoking Ort::CheckpointState::LoadCheckpoint to resume
-   * training from this snapshot of the state.
-   *
-   * \param[in] checkpoint_state The checkpoint state to save.
-   * \param[in] path_to_checkpoint Path to the checkpoint directory.
-   * \param[in] include_optimizer_state Flag to indicate whether to save the optimizer state or not.
-   *
-   */
-  static void SaveCheckpoint(const CheckpointState& checkpoint_state,
-                             const std::basic_string<ORTCHAR_T>& path_to_checkpoint,
-                             const bool include_optimizer_state = false);
-
-  /** \brief Adds the given property to the checkpoint state.
-   *
-   * Runtime properties such as epoch, training step, best score, and others can be added to the checkpoint
-   * state by the user if they desire by calling this function with the appropriate property name and
-   * value. The given property name must be unique to be able to successfully add the property.
-   *
-   * \param[in] property_name Unique name of the property being added.
-   * \param[in] property_value Property value associated with the given name.
-   *
-   */
-  void AddProperty(const std::string& property_name, const Property& property_value);
-
-  /** \brief Gets the property value associated with the given name from the checkpoint state.
-   *
-   * Gets the property value from an existing entry in the checkpoint state. The property must
-   * exist in the checkpoint state to be able to retrieve it successfully.
-   *
-   * \param[in] property_name Unique name of the property being retrieved.
-   * \return Property value associated with the given property name.
-   *
-   */
-  Property GetProperty(const std::string& property_name);
-
-  /// @}
-};
-
-/** \brief Trainer class that provides training, evaluation and optimizer methods for training an ONNX models.
- *
- * The training session requires four training artifacts
- * - The training onnx model
- * - The evaluation onnx model (optional)
- * - The optimizer onnx model
- * - The checkpoint directory
- *
- * These artifacts can be generated using the `onnxruntime-training` python [utility](https://github.com/microsoft/onnxruntime/blob/main/orttraining/orttraining/python/training/onnxblock/README.md).
- *
- */
-class TrainingSession : public detail::Base<OrtTrainingSession> {
- private:
-  size_t training_model_output_count_, eval_model_output_count_;
-
- public:
-  /// \name Constructing the Training Session
-  /// @{
-  /** \brief Create a training session that can be used to begin or resume training.
-   *
-   * This constructor instantiates the training session based on the env and session options provided that can
-   * begin or resume training from a given checkpoint state for the given onnx models.
-   * The checkpoint state represents the parameters of the training session which will be moved
-   * to the device specified by the user through the session options (if necessary).
-   *
-   * \param[in] env Env to be used for the training session.
-   * \param[in] session_options SessionOptions that the user can customize for this training session.
-   * \param[in] checkpoint_state Training states that the training session uses as a starting point for training.
-   * \param[in] train_model_path Model to be used to perform training.
-   * \param[in] eval_model_path Model to be used to perform evaluation.
-   * \param[in] optimizer_model_path Model to be used to perform gradient descent.
-   *
-   */
-  TrainingSession(const Env& env, const SessionOptions& session_options, CheckpointState& checkpoint_state,
-                  const std::basic_string<ORTCHAR_T>& train_model_path,
-                  const std::optional<std::basic_string<ORTCHAR_T>>& eval_model_path = std::nullopt,
-                  const std::optional<std::basic_string<ORTCHAR_T>>& optimizer_model_path = std::nullopt);
-
-  /// @}
-
-  /// \name Implementing The Training Loop
-  /// @{
-  /** \brief Computes the outputs of the training model and the gradients of the trainable parameters for the given inputs
-   *
-   * This function performs a training step that computes the outputs of the training model and the gradients
-   * of the trainable parameters for the given inputs. The train step is performed based on the training model
-   * that was provided to the training session.
-   * The Ort::TrainingSession::TrainStep is equivalent of running forward propagation and backward propagation in a single
-   * step.
-   * The gradients computed are stored inside the training session state so they can be later consumed
-   * by the Ort::TrainingSession::OptimizerStep function.
-   * The gradients can be lazily reset by invoking the Ort::TrainingSession::LazyResetGrad function.
-   *
-   * \param[in] input_values The user inputs to the training model.
-   * \return A std::vector of Ort::Value objects that represents the output of the forward pass of the training model.
-   *
-   * \snippet{doc} snippets.dox OrtStatus Return Value
-   *
-   */
-  std::vector<Value> TrainStep(const std::vector<Value>& input_values);
-
-  /** \brief Reset the gradients of all trainable parameters to zero lazily.
-   *
-   * This function sets the internal state of the training session such that the gradients of the trainable
-   * parameters in the OrtCheckpointState will be scheduled to be reset just before the new gradients are
-   * computed on the next invocation of the next Ort::TrainingSession::TrainStep.
-   *
-   */
-  void LazyResetGrad();
-
-  /** \brief Computes the outputs for the eval model for the given inputs
-   *
-   * This function performs an eval step that computes the outputs of the eval model for the given inputs.
-   * The eval step is performed based on the eval model that was provided to the training session.
-   *
-   * \param[in] input_values The user inputs to the eval model.
-   * \return A std::vector of Ort::Value objects that represents the output of the eval pass.
-   *
-   */
-  std::vector<Value> EvalStep(const std::vector<Value>& input_values);
-
-  /** \brief Sets the learning rate for this training session.
-   *
-   * This function allows users to set the learning rate for the training session. The current
-   * learning rate is maintained by the training session and can be overwritten by invoking
-   * this function with the desired learning rate. This function should not be used when a valid
-   * learning rate scheduler is registered. It should be used either to set the learning rate
-   * derived from a custom learning rate scheduler or to set a constant learning rate to be used
-   * throughout the training session.
-   * \note Please note that this function does not set the initial learning rate that may be needed
-   * by the predefined learning rate schedulers. To set the initial learning rate for learning
-   * rate schedulers, please look at the function Ort::TrainingSession::RegisterLinearLRScheduler.
-   *
-   * \param[in] learning_rate Desired learning rate to be set.
-   *
-   */
-  void SetLearningRate(float learning_rate);
-
-  /** \brief Gets the current learning rate for this training session.
-   *
-   * This function allows users to get the learning rate for the training session. The current
-   * learning rate is maintained by the training session, and users can query it for the purpose
-   * of implementing their own learning rate schedulers.
-   *
-   * \return float representing the current learning rate.
-   *
-   */
-  float GetLearningRate() const;
-
-  /** \brief Registers a linear learning rate scheduler for the training session.
-   *
-   * Register a linear learning rate scheduler that decays the learning rate by linearly updated
-   * multiplicative factor from the initial learning rate set on the training session to 0. The decay
-   * is performed after the initial warm up phase where the learning rate is linearly incremented
-   * from 0 to the initial learning rate provided.
-   *
-   * \param[in] warmup_step_count Warmup steps for LR warmup.
-   * \param[in] total_step_count Total step count.
-   * \param[in] initial_lr The initial learning rate to be used by the training session.
-   *
-   */
-  void RegisterLinearLRScheduler(int64_t warmup_step_count, int64_t total_step_count,
-                                 float initial_lr);
-
-  /** \brief Update the learning rate based on the registered learing rate scheduler.
-   *
-   * Takes a scheduler step that updates the learning rate that is being used by the training session.
-   * This function should typically be called before invoking the optimizer step for each round,
-   * or as determined necessary to update the learning rate being used by the training session.
-   * \note Please note that a valid predefined learning rate scheduler must be first registered to invoke this
-   * function.
-   *
-   */
-  void SchedulerStep();
-
-  /** \brief Performs the weight updates for the trainable parameters using the optimizer model.
-   *
-   * This function performs the weight update step that updates the trainable parameters such that they
-   * take a step in the direction of their gradients (gradient descent). The optimizer step is performed
-   * based on the optimizer model that was provided to the training session.
-   * The updated parameters are stored inside the training state so that they can be used by the next
-   * Ort::TrainingSession::TrainStep function call.
-   *
-   */
-  void OptimizerStep();
-
-  /// @}
-
-  /// \name Prepare For Inferencing
-  /// @{
-
-  /** \brief Export a model that can be used for inferencing.
-   *
-   * If the training session was provided with an eval model, the training session can generate
-   * an inference model if it knows the inference graph outputs. The input inference graph outputs
-   * are used to prune the eval model so that the inference model's outputs align with the provided outputs.
-   * The exported model is saved at the path provided and can be used for inferencing with Ort::Session.
-   * \note Note that the function re-loads the eval model from the path provided to Ort::TrainingSession
-   * and expects that this path still be valid.
-   *
-   * \param[in] inference_model_path Path where the inference model should be serialized to.
-   * \param[in] graph_output_names Names of the outputs that are needed in the inference model.
-   *
-   */
-  void ExportModelForInferencing(const std::basic_string<ORTCHAR_T>& inference_model_path,
-                                 const std::vector<std::string>& graph_output_names);
-
-  /// @}
-
-  /// \name Model IO Information
-  /// @{
-  /** \brief Retrieves the names of the user inputs for the training and eval models.
-   *
-   * This function returns the names of inputs of the training or eval model that can be associated
-   * with the Ort::Value(s) provided to the Ort::TrainingSession::TrainStep or Ort::TrainingSession::EvalStep
-   * function.
-   *
-   * \param[in] training Whether the training model input names are requested or eval model input names.
-   * \return Graph input names for either the training model or the eval model.
-   *
-   */
-  std::vector<std::string> InputNames(const bool training);
-
-  /** \brief Retrieves the names of the user outputs for the training and eval models.
-   *
-   * This function returns the names of outputs of the training or eval model that can be associated
-   * with the Ort::Value(s) returned by the Ort::TrainingSession::TrainStep or Ort::TrainingSession::EvalStep
-   * function.
-   *
-   * \param[in] training Whether the training model output names are requested or eval model output names.
-   * \return Graph output names for either the training model or the eval model.
-   *
-   */
-  std::vector<std::string> OutputNames(const bool training);
-
-  /// @}
-
-  /// \name Accessing The Training Session State
-  /// @{
-
-  /** \brief Returns a contiguous buffer that holds a copy of all training state parameters
-   *
-   * \param[in] only_trainable Whether to only copy trainable parameters or to copy all parameters.
-   * \return Contiguous buffer to the model parameters.
-   *
-   */
-  Value ToBuffer(const bool only_trainable);
-
-  /** \brief Loads the training session model parameters from a contiguous buffer
-   *
-   * \param[in] buffer Contiguous buffer to load the parameters from.
-   */
-  void FromBuffer(Value& buffer);
-
-  /// @}
-};
-
-/// \name Training Utilities
-/// @{
-/** \brief This function sets the seed for generating random numbers.
- *
- * Use this function to generate reproducible results. It should be noted that completely
- * reproducible results are not guaranteed.
- *
- * \param[in] seed Manual seed to use for random number generation.
- */
-void SetSeed(const int64_t seed);
-/// @}
-
-/// @}
-
-}  // namespace Ort
-
-#include "onnxruntime_training_cxx_inline.h"

v1.15.1/onnxruntime-linux-armhf/include/onnxruntime_training_cxx_inline.h

@@ -1,256 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-#pragma once
-#include "onnxruntime_training_c_api.h"
-#include "onnxruntime_cxx_api.h"
-
-namespace Ort {
-
-inline TrainingSession::TrainingSession(const Env& env, const SessionOptions& session_options,
-                                        CheckpointState& checkpoint_state,
-                                        const std::basic_string<ORTCHAR_T>& train_model_path,
-                                        const std::optional<std::basic_string<ORTCHAR_T>>& eval_model_path,
-                                        const std::optional<std::basic_string<ORTCHAR_T>>& optimizer_model_path) {
-  ThrowOnError(GetTrainingApi().CreateTrainingSession(
-      env, session_options, checkpoint_state,
-      train_model_path.c_str(),
-      eval_model_path.has_value() ? eval_model_path.value().c_str() : nullptr,
-      optimizer_model_path.has_value() ? optimizer_model_path.value().c_str() : nullptr,
-      &p_));
-
-  ThrowOnError(GetTrainingApi().TrainingSessionGetTrainingModelOutputCount(p_, &training_model_output_count_));
-
-  ThrowOnError(GetTrainingApi().TrainingSessionGetEvalModelOutputCount(p_, &eval_model_output_count_));
-}
-
-inline std::vector<Value> TrainingSession::TrainStep(const std::vector<Value>& input_values) {
-  std::vector<Value> output_values;
-  output_values.reserve(training_model_output_count_);
-  for (size_t i = 0; i < training_model_output_count_; i++) output_values.emplace_back(nullptr);
-  auto ort_input_values = reinterpret_cast<const OrtValue* const*>(input_values.data());
-  auto ort_output_values = reinterpret_cast<OrtValue**>(output_values.data());
-  RunOptions run_options;
-  ThrowOnError(GetTrainingApi().TrainStep(
-      p_, run_options, input_values.size(), ort_input_values,
-      training_model_output_count_, ort_output_values));
-
-  return output_values;
-}
-
-inline void TrainingSession::LazyResetGrad() {
-  ThrowOnError(GetTrainingApi().LazyResetGrad(p_));
-}
-
-inline std::vector<Value> TrainingSession::EvalStep(const std::vector<Value>& input_values) {
-  std::vector<Value> output_values;
-  output_values.reserve(eval_model_output_count_);
-  for (size_t i = 0; i < eval_model_output_count_; i++) output_values.emplace_back(nullptr);
-  auto ort_input_values = reinterpret_cast<const OrtValue* const*>(input_values.data());
-  auto ort_output_values = reinterpret_cast<OrtValue**>(output_values.data());
-  RunOptions run_options;
-  ThrowOnError(GetTrainingApi().EvalStep(
-      p_, run_options, input_values.size(), ort_input_values,
-      training_model_output_count_, ort_output_values));
-
-  return output_values;
-}
-
-inline void TrainingSession::SetLearningRate(float learning_rate) {
-  ThrowOnError(GetTrainingApi().SetLearningRate(p_, learning_rate));
-}
-
-inline float TrainingSession::GetLearningRate() const {
-  float learning_rate = 0;
-  ThrowOnError(GetTrainingApi().GetLearningRate(p_, &learning_rate));
-  return learning_rate;
-}
-
-inline void TrainingSession::RegisterLinearLRScheduler(int64_t warmup_step_count, int64_t total_step_count,
-                                                       float initial_lr) {
-  ThrowOnError(GetTrainingApi().RegisterLinearLRScheduler(p_, warmup_step_count, total_step_count,
-                                                          initial_lr));
-}
-
-inline void TrainingSession::SchedulerStep() {
-  ThrowOnError(GetTrainingApi().SchedulerStep(p_));
-}
-
-inline void TrainingSession::OptimizerStep() {
-  RunOptions run_options;
-  ThrowOnError(GetTrainingApi().OptimizerStep(p_, run_options));
-}
-
-inline std::vector<std::string> TrainingSession::InputNames(const bool training) {
-  auto& input_count_function = training ? GetTrainingApi().TrainingSessionGetTrainingModelInputCount
-                                        : GetTrainingApi().TrainingSessionGetEvalModelInputCount;
-  auto& input_name_function = training ? GetTrainingApi().TrainingSessionGetTrainingModelInputName
-                                       : GetTrainingApi().TrainingSessionGetEvalModelInputName;
-
-  size_t input_count = 0;
-  ThrowOnError(input_count_function(p_, &input_count));
-  std::vector<std::string> input_names(input_count);
-  AllocatorWithDefaultOptions allocator;
-  for (size_t index = 0; index < input_count; ++index) {
-    char* input_name;
-    ThrowOnError(input_name_function(p_, index, allocator, &input_name));
-    input_names[index] = std::string(input_name);
-    allocator.Free(input_name);
-  }
-
-  return input_names;
-}
-
-inline std::vector<std::string> TrainingSession::OutputNames(const bool training) {
-  auto& output_count_function = training ? GetTrainingApi().TrainingSessionGetTrainingModelOutputCount
-                                         : GetTrainingApi().TrainingSessionGetEvalModelOutputCount;
-  auto& output_name_function = training ? GetTrainingApi().TrainingSessionGetTrainingModelOutputName
-                                        : GetTrainingApi().TrainingSessionGetEvalModelOutputName;
-
-  size_t output_count = 0;
-  ThrowOnError(output_count_function(p_, &output_count));
-  std::vector<std::string> output_names(output_count);
-  AllocatorWithDefaultOptions allocator;
-  for (size_t index = 0; index < output_count; ++index) {
-    char* output_name;
-    ThrowOnError(output_name_function(p_, index, allocator, &output_name));
-    output_names[index] = std::string(output_name);
-    allocator.Free(output_name);
-  }
-
-  return output_names;
-}
-
-inline Value TrainingSession::ToBuffer(const bool only_trainable) {
-  size_t buffer_size = 0U;
-  ThrowOnError(GetTrainingApi().GetParametersSize(p_, &buffer_size, only_trainable));
-
-  std::array<int64_t, 1> buffer_shape{static_cast<int64_t>(buffer_size)};
-
-  AllocatorWithDefaultOptions allocator;
-  Value buffer = Value::CreateTensor(allocator, buffer_shape.data(), 1U,
-                                     ONNXTensorElementDataType::ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT);
-
-  ThrowOnError(GetTrainingApi().CopyParametersToBuffer(p_, buffer, only_trainable));
-
-  return buffer;
-}
-
-inline void TrainingSession::FromBuffer(Value& buffer) {
-  if (!buffer.IsTensor()) {
-    ThrowStatus(Status("Incorrect buffer received. Expected a tensor buffer.", OrtErrorCode::ORT_INVALID_ARGUMENT));
-  }
-
-  auto tensor_info = buffer.GetTensorTypeAndShapeInfo();
-  auto buffer_shape = tensor_info.GetShape();
-
-  if (buffer_shape.size() != 1U) {
-    ThrowStatus(Status("Incorrect buffer received. Expected a contiguous tensor buffer.",
-                       OrtErrorCode::ORT_INVALID_ARGUMENT));
-  }
-
-  auto buffer_size = buffer_shape.front();
-
-  size_t session_buffer_size_trainable_only = 0U;
-  ThrowOnError(GetTrainingApi().GetParametersSize(p_, &session_buffer_size_trainable_only, true));
-
-  if (buffer_size == static_cast<int64_t>(session_buffer_size_trainable_only)) {
-    ThrowOnError(GetTrainingApi().CopyBufferToParameters(p_, buffer, true));
-    return;
-  }
-
-  size_t session_buffer_size = 0U;
-  ThrowOnError(GetTrainingApi().GetParametersSize(p_, &session_buffer_size, false));
-
-  if (buffer_size != static_cast<int64_t>(session_buffer_size)) {
-    ThrowStatus(Status("Incorrect buffer size received.", OrtErrorCode::ORT_INVALID_ARGUMENT));
-  }
-
-  ThrowOnError(GetTrainingApi().CopyBufferToParameters(p_, buffer, false));
-}
-
-inline CheckpointState CheckpointState::LoadCheckpoint(const std::basic_string<ORTCHAR_T>& path_to_checkpoint) {
-  OrtCheckpointState* checkpoint_state;
-  ThrowOnError(GetTrainingApi().LoadCheckpoint(path_to_checkpoint.c_str(), &checkpoint_state));
-  return CheckpointState(checkpoint_state);
-}
-
-inline void CheckpointState::SaveCheckpoint(const CheckpointState& checkpoint_states,
-                                            const std::basic_string<ORTCHAR_T>& path_to_checkpoint,
-                                            const bool include_optimizer_state) {
-  ThrowOnError(GetTrainingApi().SaveCheckpoint(checkpoint_states, path_to_checkpoint.c_str(),
-                                               include_optimizer_state));
-}
-
-inline void TrainingSession::ExportModelForInferencing(const std::basic_string<ORTCHAR_T>& inference_model_path,
-                                                       const std::vector<std::string>& graph_output_names) {
-  std::vector<const char*> output_names;
-  output_names.reserve(graph_output_names.size());
-  for (const auto& output_name : graph_output_names) {
-    output_names.push_back(output_name.c_str());
-  }
-  ThrowOnError(GetTrainingApi().ExportModelForInferencing(
-      p_, inference_model_path.c_str(), graph_output_names.size(), output_names.data()));
-}
-
-inline void SetSeed(const int64_t seed) {
-  ThrowOnError(GetTrainingApi().SetSeed(seed));
-}
-
-inline void CheckpointState::AddProperty(const std::string& property_name, const Property& property_value) {
-  if (std::holds_alternative<int64_t>(property_value)) {
-    int64_t value = std::get<int64_t>(property_value);
-    void* value_p = &value;
-    ThrowOnError(GetTrainingApi().AddProperty(p_, property_name.c_str(), OrtPropertyType::OrtIntProperty, value_p));
-  } else if (std::holds_alternative<float>(property_value)) {
-    float value = std::get<float>(property_value);
-    void* value_p = &value;
-    ThrowOnError(GetTrainingApi().AddProperty(p_, property_name.c_str(), OrtPropertyType::OrtFloatProperty, value_p));
-  } else if (std::holds_alternative<std::string>(property_value)) {
-    std::string value = std::get<std::string>(property_value);
-    auto buffer = std::make_unique<char[]>(value.length() + 1).release();
-    memcpy(buffer, value.c_str(), value.length());
-    ThrowOnError(GetTrainingApi().AddProperty(p_, property_name.c_str(), OrtPropertyType::OrtStringProperty, buffer));
-  } else {
-    ThrowStatus(Status("Unknown property type received.", OrtErrorCode::ORT_INVALID_ARGUMENT));
-  }
-}
-
-inline Property CheckpointState::GetProperty(const std::string& property_name) {
-  void* property_value = nullptr;
-  OrtPropertyType property_type;
-
-  AllocatorWithDefaultOptions allocator;
-  ThrowOnError(GetTrainingApi().GetProperty(p_, property_name.c_str(), allocator, &property_type, &property_value));
-
-  Property property;
-
-  switch (property_type) {
-    case OrtPropertyType::OrtIntProperty: {
-      auto value_p = reinterpret_cast<int64_t*>(property_value);
-      property = *value_p;
-      allocator.Free(property_value);
-      break;
-    }
-    case OrtPropertyType::OrtFloatProperty: {
-      auto value_p = reinterpret_cast<float*>(property_value);
-      property = *value_p;
-      allocator.Free(property_value);
-      break;
-    }
-    case OrtPropertyType::OrtStringProperty: {
-      auto value_p = reinterpret_cast<char*>(property_value);
-      property = std::string(value_p);
-      allocator.Free(property_value);
-      break;
-    }
-    default: {
-      ThrowStatus(Status("Unknown property type received.", OrtErrorCode::ORT_INVALID_ARGUMENT));
-      break;
-    }
-  }
-
-  return property;
-}
-
-}  // namespace Ort

v1.15.1/onnxruntime-linux-armhf/include/provider_options.h

@@ -1,18 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-#pragma once
-
-#include <string>
-#include <unordered_map>
-#include <vector>
-
-namespace onnxruntime {
-
-// data types for execution provider options
-
-using ProviderOptions = std::unordered_map<std::string, std::string>;
-using ProviderOptionsVector = std::vector<ProviderOptions>;
-using ProviderOptionsMap = std::unordered_map<std::string, ProviderOptions>;
-
-}  // namespace onnxruntime

v1.15.1/onnxruntime-linux-armhf/lib/libonnxruntime.so

@@ -1 +0,0 @@
-libonnxruntime.so.1.15.1

v1.15.1/onnxruntime-linux-armhf/lib/libonnxruntime.so.1.15.1

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:74a56eea1395b75adab9174534ab413ca77435e20fd83f64294932c22a081fde
-size 13943728