onnx2tf 2.0.19
Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras format (NHWC). The purpose of this tool is to solve the massive Transpose extrapolation problem in onnx-tensorflow (onnx-tf).
Requires Python: >=3.12
Keywords: onnx, tensorflow, tflite, keras, deep-learning, machine-learning

# onnx2tf
Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras format (NHWC).

You should use LiteRT Torch rather than onnx2tf. https://github.com/google-ai-edge/litert-torch

<p align="center">
  <img src="https://user-images.githubusercontent.com/33194443/193840307-fa69eace-05a9-4d93-9c5d-999cf88af28e.png" />
</p>

[![Downloads](https://static.pepy.tech/personalized-badge/onnx2tf?period=total&units=none&left_color=grey&right_color=brightgreen&left_text=Downloads)](https://pepy.tech/project/onnx2tf) ![GitHub](https://img.shields.io/github/license/PINTO0309/onnx2tf?color=2BAF2B) [![Python](https://img.shields.io/badge/Python-3.12-2BAF2B)](https://img.shields.io/badge/Python-3.8-2BAF2B) [![PyPI](https://img.shields.io/pypi/v/onnx2tf?color=2BAF2B)](https://pypi.org/project/onnx2tf/) [![CodeQL](https://github.com/PINTO0309/onnx2tf/workflows/CodeQL/badge.svg)](https://github.com/PINTO0309/onnx2tf/actions?query=workflow%3ACodeQL) ![Model Convert Test Status](https://github.com/PINTO0309/onnx2tf/workflows/Model%20Convert%20Test/badge.svg) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7230085.svg)](https://doi.org/10.5281/zenodo.7230085) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/PINTO0309/onnx2tf)

## Note

<details><summary>Click to Click to expand</summary>

- The torch.script-based `torch.onnx.export` has already been moved to maintenance mode, and we recommend moving to the FX graph-based `torch.onnx.dynamo_export` starting with PyTorch v2.2.0.
- The greatest advantage of ONNX generated by `torch.onnx.dynamo_export` would be that it directly references the PyTorch implementation, allowing for the conversion of any OP that was previously difficult to convert to ONNX.
- The maintainers of ONNX and PyTorch have assured us that they will not add new OPs after `opset=18` to the existing `torch.onnx.export`.
- https://pytorch.org/docs/stable/onnx_dynamo.html#torch.onnx.dynamo_export
- This can be converted directly into an ONNX graph using Pythonic code using `onnxscript`.

  ![image](https://github.com/PINTO0309/onnx2tf/assets/33194443/c17d4682-8a68-4a8f-9d74-18ca9dbc17dc)

- For future model versatility, it would be a good idea to consider moving to `torch.onnx.dynamo_export` at an early stage.
- [Google AI Edge Torch](https://github.com/google-ai-edge/ai-edge-torch) AI Edge Torch is a python library that supports converting PyTorch models into a .tflite format, which can then be run with TensorFlow Lite and MediaPipe. This enables applications for Android, iOS and IOT that can run models completely on-device. AI Edge Torch offers broad CPU coverage, with initial GPU and NPU support. AI Edge Torch seeks to closely integrate with PyTorch, building on top of torch.export() and providing good coverage of Core ATen operators.

  https://github.com/google-ai-edge/ai-edge-torch?tab=readme-ov-file#pytorch-converter
  ```python
  import torch
  import torchvision
  import ai_edge_torch

  # Use resnet18 with pre-trained weights.
  resnet18 = torchvision.models.resnet18(torchvision.models.ResNet18_Weights.IMAGENET1K_V1)
  sample_inputs = (torch.randn(1, 3, 224, 224),)

  # Convert and serialize PyTorch model to a tflite flatbuffer. Note that we
  # are setting the model to evaluation mode prior to conversion.
  edge_model = ai_edge_torch.convert(resnet18.eval(), sample_inputs)
  edge_model.export("resnet18.tflite")
  ```

- Google for Developers Blog MAY 14, 2024 - AI Edge Torch: High Performance Inference of PyTorch Models on Mobile Devices

  https://developers.googleblog.com/en/ai-edge-torch-high-performance-inference-of-pytorch-models-on-mobile-devices/

- Considering the compatibility of Pythonic code with TensorFlow/Keras/TFLite and the beauty of the conversion workflow, [nobuco](https://github.com/AlexanderLutsenko/nobuco) is the most optimal choice going forward.
- The role of `onnx2tf` will end within the next one to two years. I don't intend to stop the maintenance of `onnx2tf` itself anytime soon, but I will continue to maintain it little by little as long as there is demand for it from everyone. The end of `onnx2tf` will be when `TensorRT` and other runtimes support porting from FX Graph based models.

</details>

## Model Conversion Status
https://github.com/PINTO0309/onnx2tf/wiki/model_status

## Supported layers
- https://github.com/onnx/onnx/blob/main/docs/Operators.md
- :heavy_check_mark:: Supported　:white_check_mark:: Partial support　**Help wanted**: Pull Request are welcome

  <details><summary>See the list of supported layers</summary><div>

  |OP|Status|
  |:-|:-:|
  |Abs|:heavy_check_mark:|
  |Acosh|:heavy_check_mark:|
  |Acos|:heavy_check_mark:|
  |Add|:heavy_check_mark:|
  |AffineGrid|:heavy_check_mark:|
  |And|:heavy_check_mark:|
  |ArgMax|:heavy_check_mark:|
  |ArgMin|:heavy_check_mark:|
  |Asinh|:heavy_check_mark:|
  |Asin|:heavy_check_mark:|
  |Atanh|:heavy_check_mark:|
  |Atan|:heavy_check_mark:|
  |Attention|:heavy_check_mark:|
  |AveragePool|:heavy_check_mark:|
  |BatchNormalization|:heavy_check_mark:|
  |Bernoulli|:heavy_check_mark:|
  |BitShift|:heavy_check_mark:|
  |BitwiseAnd|:heavy_check_mark:|
  |BitwiseNot|:heavy_check_mark:|
  |BitwiseOr|:heavy_check_mark:|
  |BitwiseXor|:heavy_check_mark:|
  |BlackmanWindow|:heavy_check_mark:|
  |Cast|:heavy_check_mark:|
  |Ceil|:heavy_check_mark:|
  |Celu|:heavy_check_mark:|
  |CenterCropPad|:heavy_check_mark:|
  |Clip|:heavy_check_mark:|
  |Col2Im|:white_check_mark:|
  |Compress|:heavy_check_mark:|
  |ConcatFromSequence|:heavy_check_mark:|
  |Concat|:heavy_check_mark:|
  |ConstantOfShape|:heavy_check_mark:|
  |Constant|:heavy_check_mark:|
  |Conv|:heavy_check_mark:|
  |ConvInteger|:white_check_mark:|
  |ConvTranspose|:heavy_check_mark:|
  |Cosh|:heavy_check_mark:|
  |Cos|:heavy_check_mark:|
  |CumProd|:heavy_check_mark:|
  |CumSum|:heavy_check_mark:|
  |DeformConv|:white_check_mark:|
  |DepthToSpace|:heavy_check_mark:|
  |Det|:heavy_check_mark:|
  |DequantizeLinear|:heavy_check_mark:|
  |DFT|:white_check_mark:|
  |Div|:heavy_check_mark:|
  |Dropout|:heavy_check_mark:|
  |DynamicQuantizeLinear|:heavy_check_mark:|
  |Einsum|:heavy_check_mark:|
  |Elu|:heavy_check_mark:|
  |Equal|:heavy_check_mark:|
  |Erf|:heavy_check_mark:|
  |Expand|:heavy_check_mark:|
  |Exp|:heavy_check_mark:|
  |EyeLike|:heavy_check_mark:|
  |Flatten|:heavy_check_mark:|
  |Floor|:heavy_check_mark:|
  |FusedConv|:heavy_check_mark:|
  |GatherElements|:heavy_check_mark:|
  |GatherND|:heavy_check_mark:|
  |Gather|:heavy_check_mark:|
  |Gelu|:heavy_check_mark:|
  |Gemm|:heavy_check_mark:|
  |GlobalAveragePool|:heavy_check_mark:|
  |GlobalLpPool|:heavy_check_mark:|
  |GlobalMaxPool|:heavy_check_mark:|
  |GreaterOrEqual|:heavy_check_mark:|
  |Greater|:heavy_check_mark:|
  |GridSample|:white_check_mark:|
  |GroupNormalization|:heavy_check_mark:|
  |GRU|:heavy_check_mark:|
  |HammingWindow|:white_check_mark:|
  |HannWindow|:white_check_mark:|
  |Hardmax|:heavy_check_mark:|
  |HardSigmoid|:heavy_check_mark:|
  |HardSwish|:heavy_check_mark:|
  |Identity|:heavy_check_mark:|
  |If|:heavy_check_mark:|
  |ImageDecoder|:white_check_mark:|
  |Input|:heavy_check_mark:|
  |InstanceNormalization|:heavy_check_mark:|
  |Inverse|:heavy_check_mark:|
  |IsInf|:heavy_check_mark:|
  |IsNaN|:heavy_check_mark:|
  |LayerNormalization|:heavy_check_mark:|
  |LeakyRelu|:heavy_check_mark:|
  |LessOrEqual|:heavy_check_mark:|
  |Less|:heavy_check_mark:|
  |Log|:heavy_check_mark:|
  |LogSoftmax|:heavy_check_mark:|
  |Loop|:heavy_check_mark:|
  |LpNormalization|:heavy_check_mark:|
  |LpPool|:heavy_check_mark:|
  |LRN|:heavy_check_mark:|
  |LSTM|:heavy_check_mark:|
  |MatMul|:heavy_check_mark:|
  |MatMulInteger|:heavy_check_mark:|
  |MaxPool|:heavy_check_mark:|
  |Max|:heavy_check_mark:|
  |MaxRoiPool|:heavy_check_mark:|
  |MaxUnpool|:heavy_check_mark:|
  |Mean|:heavy_check_mark:|
  |MeanVarianceNormalization|:heavy_check_mark:|
  |MelWeightMatrix|:heavy_check_mark:|
  |Min|:heavy_check_mark:|
  |Mish|:heavy_check_mark:|
  |Mod|:heavy_check_mark:|
  |Mul|:heavy_check_mark:|
  |Multinomial|:heavy_check_mark:|
  |Neg|:heavy_check_mark:|
  |NegativeLogLikelihoodLoss|:heavy_check_mark:|
  |NonMaxSuppression|:heavy_check_mark:|
  |NonZero|:heavy_check_mark:|
  |Optional|:heavy_check_mark:|
  |OptionalGetElement|:heavy_check_mark:|
  |OptionalHasElement|:heavy_check_mark:|
  |Not|:heavy_check_mark:|
  |OneHot|:heavy_check_mark:|
  |Or|:heavy_check_mark:|
  |Pad|:heavy_check_mark:|
  |Pow|:heavy_check_mark:|
  |PRelu|:heavy_check_mark:|
  |QLinearAdd|:heavy_check_mark:|
  |QLinearAveragePool|:heavy_check_mark:|
  |QLinearConcat|:heavy_check_mark:|
  |QLinearConv|:heavy_check_mark:|
  |QGemm|:heavy_check_mark:|
  |QLinearGlobalAveragePool|:heavy_check_mark:|
  |QLinearLeakyRelu|:heavy_check_mark:|
  |QLinearMatMul|:heavy_check_mark:|
  |QLinearMul|:heavy_check_mark:|
  |QLinearSigmoid|:heavy_check_mark:|
  |QLinearSoftmax|:heavy_check_mark:|
  |QuantizeLinear|:heavy_check_mark:|
  |RandomNormalLike|:heavy_check_mark:|
  |RandomNormal|:heavy_check_mark:|
  |RandomUniformLike|:heavy_check_mark:|
  |RandomUniform|:heavy_check_mark:|
  |Range|:heavy_check_mark:|
  |Reciprocal|:heavy_check_mark:|
  |ReduceL1|:heavy_check_mark:|
  |ReduceL2|:heavy_check_mark:|
  |ReduceLogSum|:heavy_check_mark:|
  |ReduceLogSumExp|:heavy_check_mark:|
  |ReduceMax|:heavy_check_mark:|
  |ReduceMean|:heavy_check_mark:|
  |ReduceMin|:heavy_check_mark:|
  |ReduceProd|:heavy_check_mark:|
  |ReduceSum|:heavy_check_mark:|
  |ReduceSumSquare|:heavy_check_mark:|
  |RegexFullMatch|:heavy_check_mark:|
  |Relu|:heavy_check_mark:|
  |Reshape|:heavy_check_mark:|
  |Resize|:heavy_check_mark:|
  |ReverseSequence|:heavy_check_mark:|
  |RNN|:heavy_check_mark:|
  |RoiAlign|:heavy_check_mark:|
  |RotaryEmbedding|:heavy_check_mark:|
  |Round|:heavy_check_mark:|
  |ScaleAndTranslate|:heavy_check_mark:|
  |Scatter|:heavy_check_mark:|
  |ScatterElements|:heavy_check_mark:|
  |ScatterND|:heavy_check_mark:|
  |Scan|:heavy_check_mark:|
  |Selu|:heavy_check_mark:|
  |SequenceAt|:heavy_check_mark:|
  |SequenceConstruct|:heavy_check_mark:|
  |SequenceEmpty|:heavy_check_mark:|
  |SequenceErase|:heavy_check_mark:|
  |SequenceInsert|:heavy_check_mark:|
  |SequenceLength|:heavy_check_mark:|
  |Shape|:heavy_check_mark:|
  |Shrink|:heavy_check_mark:|
  |Sigmoid|:heavy_check_mark:|
  |Sign|:heavy_check_mark:|
  |Sinh|:heavy_check_mark:|
  |Sin|:heavy_check_mark:|
  |Size|:heavy_check_mark:|
  |Slice|:heavy_check_mark:|
  |Softmax|:heavy_check_mark:|
  |SoftmaxCrossEntropyLoss|:heavy_check_mark:|
  |Softplus|:heavy_check_mark:|
  |Softsign|:heavy_check_mark:|
  |SpaceToDepth|:heavy_check_mark:|
  |Split|:heavy_check_mark:|
  |SplitToSequence|:heavy_check_mark:|
  |Sqrt|:heavy_check_mark:|
  |Squeeze|:heavy_check_mark:|
  |STFT|:white_check_mark:|
  |StringConcat|:heavy_check_mark:|
  |StringNormalizer|:heavy_check_mark:|
  |StringSplit|:heavy_check_mark:|
  |Sub|:heavy_check_mark:|
  |Sum|:heavy_check_mark:|
  |Tan|:heavy_check_mark:|
  |Tanh|:heavy_check_mark:|
  |TensorScatter|:heavy_check_mark:|
  |TfIdfVectorizer|:white_check_mark:|
  |ThresholdedRelu|:heavy_check_mark:|
  |Tile|:heavy_check_mark:|
  |TopK|:heavy_check_mark:|
  |Transpose|:heavy_check_mark:|
  |Trilu|:heavy_check_mark:|
  |Unique|:heavy_check_mark:|
  |Unsqueeze|:heavy_check_mark:|
  |Upsample|:heavy_check_mark:|
  |Where|:heavy_check_mark:|
  |Xor|:heavy_check_mark:|

  </div></details>

> [!WARNING]
> `flatbuffer_direct` is an experimental backend. Behavior, supported patterns, and conversion quality may change between releases.
> For production use, keep `tf_converter` as baseline and validate `flatbuffer_direct` per model with `--report_op_coverage`.
>
> `flatbuffer_direct` now runs a layout-transpose chain optimizer during lowering.
> For NCW/NCHW/NCDHW <-> NWC/NHWC/NDHWC conversion paths, inverse `Transpose` pairs are removed automatically when safe.
> `Transpose -> (Quantize/Dequantize) -> inverse Transpose` and `Transpose -> Quantize -> Dequantize -> inverse Transpose` are also folded for per-tensor quantization.
> `Transpose -> (ADD/SUB/MUL/DIV) -> inverse Transpose` is folded when both binary inputs share the same pre-transpose permutation.
> For float outputs, terminal `QUANTIZE -> DEQUANTIZE` pairs are also removed when the pair is isolated and output-only.

### [WIP・experimental] `flatbuffer_direct` support status for ONNX ops in this list

The `flatbuffer_direct` conversion option exists to convert a QAT quantized ONNX model to an optimized quantized tflite (LiteRT) model. By the way, if you want to generate a highly optimized quantized tflite for your ONNX model, I recommend using this package. https://github.com/NXP/eiq-onnx2tflite

|INT8 ONNX|INT8 TFLite(LiteRT)|
|:-:|:-:|
|<img width="300" alt="Image" src="https://github.com/user-attachments/assets/c1411cb7-35aa-489d-ad87-291d64b766ec" />|<img width="300" alt="image" src="https://github.com/user-attachments/assets/7ffeaa53-4c83-4a9e-b5b4-17ea11c93b20" />|

<details><summary>Click to expand</summary>

- Scope: ONNX ops listed in the `Supported layers` table above.
- Source of truth: `onnx2tf/tflite_builder/op_registry.py` and `--report_op_coverage` output.
- Current summary:
  - Listed ONNX ops in this README section: `208`
  - Policy counts are generated in `*_op_coverage_report.json` (`schema_policy_counts`).
  - Check each conversion run with `--report_op_coverage` for the latest numbers.

Notes:
- `flatbuffer_direct` supports only a subset of ONNX ops as TFLite builtins.
- Some ops are conditionally supported (rank/attribute/constant-input constraints).
- For model-specific results, use `--report_op_coverage` and check `*_op_coverage_report.json`.

<details><summary>Builtin supported (ONNX -> TFLite) in flatbuffer_direct</summary><div>

|ONNX OP|TFLite OP|Key constraints (flatbuffer_direct)|
|:-|:-|:-|
|Abs|ABS|-|
|Acos|MUL + SUB + SQRT + ATAN2|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Acosh|SUB + ADD + SQRT + MUL + LOG|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Add|ADD|-|
|And|LOGICAL_AND|-|
|ArgMax|ARG_MAX (+ optional RESHAPE for keepdims)|`axis` must be in range, `keepdims` must be `0` or `1`, `select_last_index=0`, output dtype must be `INT32` or `INT64`|
|ArgMin|ARG_MIN (+ optional RESHAPE for keepdims)|`axis` must be in range, `keepdims` must be `0` or `1`, `select_last_index=0`, output dtype must be `INT32` or `INT64`|
|Asin|MUL + SUB + SQRT + ATAN2|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Asinh|MUL + ADD + SQRT + LOG|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Atan|ATAN2|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Atanh|ADD + SUB + DIV + LOG + MUL|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|AveragePool|AVERAGE_POOL_2D|2D only (rank=4), `ceil_mode=0`, zero pads or `auto_pad=SAME_*`|
|BatchNormalization|MUL + ADD|All parameter inputs (`scale`, `bias`, `mean`, `var`) must be constant|
|BitShift|RIGHT_SHIFT (RIGHT) or MUL-based (LEFT)|LHS/RHS must be integer tensors, `direction` must be `LEFT` or `RIGHT`; `LEFT` requires constant shift input|
|BitwiseAnd|LOGICAL_AND|BOOL tensors only|
|BitwiseNot|LOGICAL_NOT / SUB + CAST|Input dtype must be BOOL or integer|
|BitwiseOr|LOGICAL_OR|BOOL tensors only|
|BitwiseXor|BITWISE_XOR|Input dtypes must match and be BOOL/integer|
|Cast|CAST|-|
|Ceil|CEIL|-|
|Celu|MAXIMUM + MINIMUM + DIV + EXP + SUB + MUL + ADD|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Clip|RELU / RELU6 / MAXIMUM + MINIMUM|General constant clip ranges are supported via `MAXIMUM`/`MINIMUM` decomposition. ReLU fast-path: `min=0,max=+inf`; ReLU6 fast-path: `min=0,max=6`|
|Concat|CONCATENATION|-|
|ConstantOfShape|CAST + FILL|Shape input must be rank-1 integer tensor; `value` attribute must be scalar (or omitted for zero-fill)|
|Conv|CONV_2D / DEPTHWISE_CONV_2D|2D only (rank=4), weights must be constant, grouped conv only regular/depthwise, zero pads or `auto_pad=SAME_*`|
|ConvTranspose|TRANSPOSE_CONV (+ optional ADD bias)|2D only (input rank=4), weight must be constant rank=4, `group=1`, `dilations=[1,1]`, `output_padding=[0,0]`, and padding must be `auto_pad=SAME_*` or zero pads (`auto_pad` in `{NOTSET,VALID}`)|
|Cos|COS|-|
|Cosh|SUB + EXP + ADD + MUL|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|DequantizeLinear|DEQUANTIZE|`scale` must be constant, `zero_point` (if provided) must be constant, per-axis `axis` must be in range|
|Div|DIV or MUL (when divisor is constant reciprocal)|For non-floating outputs, lowered as `CAST -> MUL(reciprocal) -> CAST` to preserve output dtype without using unsupported integer DIV paths|
|DynamicQuantizeLinear|NEG + REDUCE_MAX + MINIMUM + MAXIMUM + SUB + DIV + ADD + CAST|Input dtype must be `FLOAT16/FLOAT32`, output dtypes must be `Y=UINT8`, `Y_Scale=FLOAT16/FLOAT32`, `Y_ZeroPoint=UINT8`; scale/zero-point outputs must be scalar|
|Einsum|FULLY_CONNECTED|Rank-2 matmul-style equation only (`ij,jk->ik`), rhs input must be constant weights|
|Elu|ELU|-|
|Equal|EQUAL|-|
|Exp|EXP|-|
|Expand|RESHAPE + MUL (broadcast via const ones)|Output shape must be statically known, non-negative, and broadcast-compatible with input shape (current direct lowering uses static `RESHAPE + MUL`)|
|EyeLike|RESHAPE (from const eye)|Output must be rank-2 with fully static positive shape|
|Flatten|RESHAPE|Input rank must be >= 1|
|Floor|FLOOR|-|
|FusedMatMul|BATCH_MATMUL (+ optional MUL for `alpha`)|Input rank >= 2, dtypes FLOAT16/FLOAT32 only, `transA/transB` must be 0 or 1, finite `alpha` required|
|Gather|GATHER|`batch_dims=0` only|
|GatherElements|CAST + RESHAPE + CONCATENATION + GATHER_ND|Data/indices ranks must match, output shape must equal indices shape, static positive output dims required, `axis` must be in range|
|GatherND|CAST + GATHER_ND|`batch_dims=0` only; indices must be integer type; indices last dim must be static positive and `<= params_rank`|
|Gelu|GELU|-|
|Gemm|FULLY_CONNECTED|Input rank=2, weight rank=2 + constant, `transA=0` only|
|Greater|GREATER|-|
|GRU|TRANSPOSE + SLICE + SQUEEZE + BATCH_MATMUL + ADD + MUL + SUB + LOGISTIC + TANH + RESHAPE + CONCATENATION + EXPAND_DIMS|`layout=0`; `direction` in `{forward, reverse, bidirectional}`; `sequence_lens` unsupported; `W/R` must be constant rank-3; `linear_before_reset` in `{0,1}`; activations `[Sigmoid,Tanh]`; `clip=0`|
|Hardmax|TRANSPOSE + ARG_MAX + ONE_HOT|`axis` must be in range; target axis size must be static positive|
|HardSigmoid|MUL + ADD + MAXIMUM + MINIMUM|Input/output dtype must be FLOAT16 or FLOAT32|
|Identity|RESHAPE|-|
|Less|LESS|-|
|LessOrEqual|LESS_EQUAL|-|
|LogSoftmax|SOFTMAX + LOG (+ transpose in/out for non-last axis)|`axis` must be in range (negative axis normalized)|
|LpNormalization|L2_NORMALIZATION|`p=2`, `axis=last` only|
|LRN|LOCAL_RESPONSE_NORMALIZATION (+ transpose in/out)|Input rank must be 4, `size` must be a positive odd integer|
|LSTM|BIDIRECTIONAL_SEQUENCE_LSTM + SPLIT + RESHAPE/EXPAND_DIMS + CONCATENATION|`direction=bidirectional`, `layout=0`, `input_forget=0`; `W/R` must be constant rank-3 with `num_directions=2`; optional `B` must be constant shape `[2, 8*hidden_size]`; `initial_h/initial_c` must be constant zero tensors of shape `[2, batch, hidden]`; `sequence_lens` and peephole input `P` unsupported; outputs `Y_h`/`Y_c` unsupported when consumed|
|MatMul|BATCH_MATMUL|Input rank >= 2. Dynamic rhs input is supported (no constant-weight requirement)|
|MatMulInteger|CAST + SUB + BATCH_MATMUL|A/B input rank must be >=2 (rank=1 placeholder allowed), A/B dtypes must be integer tensor types (`INT8/UINT8/INT16/UINT16/INT32`), output dtype must be `INT32/INT64`; optional zero-point inputs must be scalar/1D and shape-compatible|
|MaxPool|MAX_POOL_2D|2D only (rank=4), `ceil_mode=0`, zero pads or `auto_pad=SAME_*`|
|Mish|EXP + ADD + LOG + TANH + MUL|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Mod|FLOOR_MOD|`fmod=0` only|
|Mul|MUL|-|
|Neg|NEG|-|
|NonMaxSuppression|ARG_MAX + REDUCE_MAX + SQUEEZE + NON_MAX_SUPPRESSION_V4 + SLICE + GATHER + SUB + CAST + RESHAPE + CONCATENATION|Rank-3 boxes/scores only; `center_point_box=0`; currently `batch=1`; class dim `>1` requires `--output_nms_with_argmax`; optional thresholds/max_output must be scalar constants|
|NonZero|NOT_EQUAL + WHERE + TRANSPOSE + CAST|Input rank must be `>=1`; output rank must be `2`|
|Not|LOGICAL_NOT|-|
|OneHot|CAST + ADD + FLOOR_MOD + ONE_HOT|`depth` input must be constant scalar and `>0`; `values` input must be constant 2-element tensor `[off_value,on_value]`; normalized `axis` must be in range|
|Or|LOGICAL_OR|-|
|Pad|PAD|`mode=constant` only, `pads` must be constant, constant pad value (if provided) must be zero|
|Pow|POW|Output dtype must be `FLOAT16` or `FLOAT32`|
|PRelu|PRELU|`slope` must be constant (scalar or per-channel)|
|QGemm|FULLY_CONNECTED|Input rank=1 or 2, weight must be constant rank=2, bias must be constant, quantization params must be constant, `transA=0`, `transB` in `{0,1}`|
|QLinearAdd|ADD|All quantization params (`a/b/c scale`, `a/b/c zero_point`) must be constant|
|QLinearAveragePool|DEQUANTIZE + TRANSPOSE + AVERAGE_POOL_2D + TRANSPOSE + QUANTIZE|Input rank=4 only, all quantization params (`x scale/zero_point`, `y scale/zero_point`) must be constant, `kernel_shape/strides` must be 2D, `dilations=[1,1]`, `ceil_mode=0`, `count_include_pad=0`, and pads must satisfy flatbuffer_direct pool constraints (zero/symmetric or `auto_pad=SAME_*`)|
|QLinearConcat|DEQUANTIZE + CONCATENATION + QUANTIZE|`y scale/zero_point` and each input triplet (`x scale/zero_point`) must be constant, input ranks must match, `axis` must be in range|
|QLinearConv|CONV_2D / DEPTHWISE_CONV_2D|Input/output rank=4, weight must be constant rank=4, all quantization params constant, group conv only regular/depthwise (depthwise detection uses `group` and weight shape), optional bias must be constant|
|QLinearGlobalAveragePool|AVERAGE_POOL_2D (preferred) / DEQUANTIZE + MEAN + QUANTIZE (fallback)|All quantization params (`x scale/zero_point`, `y scale/zero_point`) must be constant, input rank >= 3, `channels_last` must be 0 or 1. Quantized `AVERAGE_POOL_2D` path is used for rank-4 with static spatial dims and per-tensor quantization|
|QLinearMatMul|FULLY_CONNECTED|Input rank=1 or 2, weight must be constant rank=2, all quantization params constant|
|QLinearMul|MUL|All quantization params (`a/b/c scale`, `a/b/c zero_point`) must be constant|
|QLinearSigmoid|DEQUANTIZE + LOGISTIC + QUANTIZE|All quantization params (`x scale/zero_point`, `y scale/zero_point`) must be constant|
|QuantizeLinear|QUANTIZE|`scale` must be constant, `zero_point` (if provided) must be constant, per-axis `axis` must be in range|
|Range|CAST + SQUEEZE + RANGE|Each of `start/limit/delta` must be scalar-like rank-1 length-1 tensor|
|Reciprocal|DIV|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|ReduceL1|ABS + SUM|Reduce axes must be constant when provided via input tensor|
|ReduceL2|MUL + SUM + SQRT + CAST|Reduce axes must be constant when provided via input tensor|
|ReduceMax|REDUCE_MAX|Reduce axes must be constant when provided via input tensor|
|ReduceMean|MEAN|Reduce axes must be constant when provided via input tensor|
|ReduceSum|SUM|Reduce axes must be constant when provided via input tensor|
|Relu|RELU|-|
|Reshape|RESHAPE|Shape input must be constant|
|Resize|RESIZE_NEAREST_NEIGHBOR / RESIZE_BILINEAR|Rank-4 only; supported modes: `nearest`/`linear` (limited attr combinations). Parameters must be either constant `scales/sizes` or dynamic rank-1 integer `sizes` (INT32/INT64)|
|RNN|UNIDIRECTIONAL_SEQUENCE_RNN + TRANSPOSE + EXPAND_DIMS + SLICE + RESHAPE|`direction=forward`, `layout=0`; `sequence_lens` unsupported; `W/R` must be constant rank-3 with `num_directions=1`; activations in `{tanh,relu,sigmoid}`; `clip=0`|
|Round|ROUND|-|
|Selu|MAXIMUM + MINIMUM + EXP + SUB + MUL + ADD|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Shape|SHAPE (+ SLICE for `start/end`)|Output dtype must be `INT32` or `INT64`; `start/end` slicing follows ONNX normalization|
|Sigmoid|LOGISTIC|-|
|Sign|SIGN|-|
|Sin|SIN|-|
|Sinh|SUB + EXP + MUL|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Softmax|SOFTMAX (+ transpose in/out for non-last axis)|`axis` must be in range (negative axis normalized)|
|Softplus|EXP + ADD + LOG|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Softsign|ABS + ADD + DIV|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|SpaceToDepth|SPACE_TO_DEPTH|`blocksize > 1`, rank=4 (NCHW)|
|Sqrt|SQRT|-|
|Squeeze|SQUEEZE|Axes must be constant when provided via input tensor|
|Sub|SUB|-|
|Tan|SIN + COS + DIV|Input/output dtype must be `FLOAT16` or `FLOAT32`|
|Tanh|TANH|-|
|Transpose|TRANSPOSE|Permutation input must be constant|
|Trilu|MUL / LOGICAL_AND|Input rank must be `>=2`; matrix dims must be static positive; optional `k` input must be constant|
|Unsqueeze|RESHAPE|Axes must be constant and in range|
|Where|CAST + SELECT|Condition input dtype must be BOOL or numeric|
|Xor|NOT_EQUAL|-|

</div></details>

<details><summary>Custom-op candidates in flatbuffer_direct (opt-in)</summary><div>

|ONNX OP|Default policy|When enabled|
|:-|:-|:-|
|DeformConv|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|GridSample|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|If|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|Loop|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|RoiAlign|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|Scan|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|ScatterElements|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|SequenceAt|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|SequenceConstruct|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|SequenceErase|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|SequenceInsert|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|SequenceLength|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|TopK|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|
|Unique|explicit_error (`custom_op_candidate_disabled`)|Lowered to TFLite `CUSTOM` when `--flatbuffer_direct_allow_custom_ops` is enabled and allowlist passes|

</div></details>

Notes:
- `Einsum` is now treated as `builtin_supported` when it matches builtin constraints; unsupported `Einsum` patterns may still fallback to `CUSTOM` if custom-op mode is enabled.
- `QLinearConv` is treated as `builtin_supported` for regular/depthwise patterns; unsupported grouped patterns may still fallback to `CUSTOM` when custom-op mode is enabled.
- `LogSoftmax` is now treated as `builtin_supported` when builtin constraints pass; unsupported patterns may still fallback to `CUSTOM` if custom-op mode is enabled.
- `LSTM` is now treated as `builtin_supported` for constrained bidirectional patterns; unsupported patterns may still fallback to `CUSTOM` if custom-op mode is enabled.
- `NonMaxSuppression` is now treated as `builtin_supported` when builtin constraints pass; unsupported patterns may still fallback to `CUSTOM` if custom-op mode is enabled.
- `DynamicQuantizeLinear` is now treated as `builtin_supported` for constrained float-input/uint8-output patterns; unsupported patterns may still fallback to `CUSTOM` if custom-op mode is enabled.
- `OneHot`, `MatMulInteger`, `Pow`, and `Reciprocal` are now treated as `builtin_supported` when builtin constraints pass.
- Newly added builtin-covered ops in this update include:
  `Abs`, `Acos`, `Acosh`, `And`, `ArgMin`, `Asin`, `Asinh`, `Atan`, `Atanh`, `BitShift`,
  `BitwiseAnd`, `BitwiseNot`, `BitwiseOr`, `BitwiseXor`, `Ceil`, `Celu`, `Cos`, `Cosh`,
  `Elu`, `Equal`, `EyeLike`, `Floor`, `GatherND`, `Gelu`, `Greater`, `GRU`, `Hardmax`,
  `Less`, `LessOrEqual`, `Mish`, `NonZero`, `Not`, `Or`, `Range`, `ReduceL1`, `ReduceL2`,
  `RNN`, `Round`, `Selu`, `Sign`, `Sin`, `Sinh`, `Softplus`, `Softsign`, `Tan`, `Trilu`,
  `Where`, and `Xor`.
- `Resize` builtin path now accepts dynamic rank-1 integer `sizes` input in addition to constant `scales/sizes`.

### tf_converter vs flatbuffer_direct (operational differences)

|Item|`tf_converter` (default)|`flatbuffer_direct`|
|:-|:-|:-|
|Final backend|TensorFlow Lite Converter|Direct FlatBuffer builder (`schema.fbs`)|
|Model optimization source|Large set of existing TF-path graph rewrites/heuristics|Dedicated direct preprocess pipeline + direct dispatch constraints|
|Failure behavior|Often absorbed by TF-side graph lowering|Explicit `reason_code`-based failure on unsupported patterns|
|Custom op handling|Typically avoided by TF-side replacement when possible|Opt-in only (`--flatbuffer_direct_allow_custom_ops`) with allowlist|
|Diagnostics|Standard conversion logs|`*_op_coverage_report.json` (`dispatch_mode`, `unsupported_reason_counts`, `custom_op_policy`, `preprocess_report`)|
|Fallback|N/A|`--flatbuffer_direct_fallback_to_tf_converter` available|

### flatbuffer_direct preprocess absorption scope

`flatbuffer_direct` runs staged preprocess rules before lowering. Current major coverage:
1. `pattern_fusion_wave2`
   - `Relu -> Clip(min=0,max=6)` chain normalization
   - GELU chain fusion (`Div -> Erf -> Add -> Mul -> Mul`)
   - `Reshape -> Transpose -> Reshape` to `SpaceToDepth`
2. `quant_chain_fusion_wave3`
   - `DequantizeLinear -> BatchNormalization -> PRelu -> QuantizeLinear` chain rewrite
   - BatchNormalization parameter folding into `Mul + Add`
3. `pseudo_ops_wave1`
   - `HardSwish`, `LeakyRelu`, `Gelu`, limited `Pow` rewrites to builtin-friendly forms
4. `constant_fold_a5`
   - Limited constant folding for shape/axes and arithmetic helper chains
   - Includes `DequantizeLinear` (axis/block-size aware) and downstream `Reshape` folding for constant-weight subgraphs
5. `normalize_attrs_a5`
   - Normalize `perm`/`axes`/negative-axis forms and softmax-axis bridge rewrites

Notes:
1. This reduces, but does not fully match, the TF-path replacement coverage.
2. To inspect what was applied, use `--report_op_coverage` and check `preprocess_report.applied_rules`.

### Known constraints and workaround options

|Symptom (`reason_code`)|Meaning|Recommended action|
|:-|:-|:-|
|`unsupported_onnx_op`|No direct builtin/custom path for the node|Use `--tflite_backend tf_converter`, or enable `--flatbuffer_direct_fallback_to_tf_converter`|
|`requires_constant_input`|Node requires compile-time constant input (e.g., axes/perm/shape)|Pre-fold ONNX graph (`onnxsim`) or rewrite model to constantize the input|
|`unsupported_attribute_value`|Attribute/rank/value not accepted by direct builtin constraints|Adjust ONNX export options or rewrite offending subgraph before conversion|
|`custom_op_candidate_disabled`|Op is in custom-candidate set but custom lowering is disabled|Enable `--flatbuffer_direct_allow_custom_ops` when runtime supports the custom op|
|`custom_op_not_in_allowlist`|Custom lowering enabled but op is not allowlisted|Add op to `--flatbuffer_direct_custom_op_allowlist` explicitly|

</details>

## Demo
Video speed is adjusted approximately 50 times slower than actual speed.
![render1665941718294](https://user-images.githubusercontent.com/33194443/196049928-57520fc2-842d-459c-9f28-7ee5f040c226.gif)

## Environment
- Linux / Windows
- onnx==1.20.1
- onnxruntime==1.24.1
- onnxsim-prebuilt==0.4.39.post2
- onnxoptimizer==0.4.2
- sne4onnx>=2.0.0
- sng4onnx>=2.0.0
- tensorflow==2.19.0
- tf-keras==2.19.0
- ai-edge-litert==2.1.2
- h5py==3.12.1
- psutil==5.9.5
- ml_dtypes==0.5.1
- flatbuffers-compiler (Optional, Only when using the `-coion` option. Executable file named `flatc`.)
- flatbuffers>=23.1.21
  ```bash
  # Custom flatc binary for Ubuntu 22.04+
  # https://github.com/PINTO0309/onnx2tf/issues/196

  # x86_64/amd64 v23.5.26
  wget https://github.com/PINTO0309/onnx2tf/releases/download/1.16.31/flatc.tar.gz \
  && tar -zxvf flatc.tar.gz \
  && sudo chmod +x flatc \
  && sudo mv flatc /usr/bin/

  # arm64 v23.1.21
  wget https://github.com/PINTO0309/onnx2tf/releases/download/1.26.6/flatc_arm64.tar.gz \
  && tar -zxvf flatc_arm64.tar.gz \
  && sudo chmod +x flatc \
  && sudo mv flatc /usr/bin/
  ```

## Sample Usage
### 1. Install
#### Note:
**1. If you are using TensorFlow v2.13.0 or earlier, use a version older than onnx2tf v1.17.5. onnx2tf v1.17.6 or later will not work properly due to changes in TensorFlow's API.**

**2. The latest onnx2tf implementation is based on Keras API 3 and will not work properly if you install TensorFlow v2.15.0 or earlier.**

**3. Starting with onnx2tf v2.0.0, due to onnxruntime issues, onnx2tf will no longer support environments older than Python 3.10. Accordingly, the Docker Image has been upgraded to Ubuntu 24.04. The dependency on onnx-graphsurgeon has also been completely removed. onnxruntime v1.24.1: https://github.com/microsoft/onnxruntime/releases/tag/v1.24.1**

- HostPC
  <details><summary>Click to expand</summary><div>

  - When using GHCR, see `Authenticating to the Container registry`

    https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-to-the-container-registry
  ```bash
  # PAT authentication is required to pull from GHCR.
  docker login ghcr.io

  Username (xxxx): {Enter}
  Password: {Personal Access Token}
  Login Succeeded

  # Start an interactive session on the terminal.
  docker run --rm -it \
  -v `pwd`:/workdir \
  -w /workdir \
  ghcr.io/pinto0309/onnx2tf:2.0.19

  or

  # Authentication is not required for pulls from Docker Hub.
  # Start an interactive session on the terminal.
  docker run --rm -it \
  -v `pwd`:/workdir \
  -w /workdir \
  docker.io/pinto0309/onnx2tf:2.0.19

  or

  # Direct execution in Docker
  # The model conversion is performed within Docker,
  # but the model is output to the host PC's storage.
  docker run --rm \
  --user $(id -u):$(id -g) \
  -v $(pwd):/work \
  docker.io/pinto0309/onnx2tf:2.0.19 \
  onnx2tf -i /work/densenet-12.onnx -o /work/saved_model

  or

  curl -LsSf https://astral.sh/uv/install.sh | sh
  uv python install 3.12.12
  uv venv -p 3.12.12 .venv
  source .venv/bin/activate
  uv pip install -U onnx2tf

  or

  curl -LsSf https://astral.sh/uv/install.sh | sh
  uv python install 3.12.12
  uv venv -p 3.12.12 .venv
  source .venv/bin/activate
  uv sync

  or

  pip install -e .

  or

  docker buildx build \
  --platform linux/amd64 \
  --build-arg BUILD_ARCH=linux/amd64 \
  --progress=plain \
  -t onnx2tf:amd64 \
  --load .

  or

  # It is possible to cross-compile an arm64 environment on an x64 environment.
  docker buildx build \
  --platform linux/arm64 \
  --build-arg BUILD_ARCH=linux/arm64 \
  --progress=plain \
  -t onnx2tf:arm64 \
  --load .
  ```

  </div></details>

### 2. Run test
Only patterns that are considered to be used particularly frequently are described. In addition, there are several other options, such as disabling Flex OP and additional options to improve inference performance. See: [CLI Parameter](#cli-parameter)
```bash
# Float32, Float16
# This is the fastest way to generate tflite.
# Improved to automatically generate `signature` without `-osd` starting from v1.25.3.
# Also, starting from v1.24.0, efficient TFLite can be generated
# without unrolling `GroupConvolution`. e.g. YOLOv9, YOLOvN
# Conversion to other frameworks. e.g. TensorFlow.js, CoreML, etc
# https://github.com/PINTO0309/onnx2tf#19-conversion-to-tensorflowjs
# https://github.com/PINTO0309/onnx2tf#20-conversion-to-coreml
wget https://github.com/PINTO0309/onnx2tf/releases/download/0.0.2/resnet18-v1-7.onnx
onnx2tf -i resnet18-v1-7.onnx

ls -lh saved_model/

assets
fingerprint.pb
resnet18-v1-7_float16.tflite
resnet18-v1-7_float32.tflite
saved_model.pb
variables

TF_CPP_MIN_LOG_LEVEL=3 \
saved_model_cli show \
--dir saved_model \
--signature_def serving_default \
--tag_set serve

The given SavedModel SignatureDef contains the following input(s):
  inputs['data'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 224, 224, 3)
      name: serving_default_data:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['output_0'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 1000)
      name: PartitionedCall:0
Method name is: tensorflow/serving/predict

# In the interest of efficiency for my development and debugging of onnx2tf,
# the default configuration shows a large amount of debug level logs.
# However, for most users, a large number of debug logs are unnecessary.
# If you want to reduce the amount of information displayed in the conversion log,
# you can change the amount of information in the log by specifying the
# `--verbosity` or `-v` option as follows.
# Possible values are "debug", "info", "warn", and "error".
wget https://github.com/PINTO0309/onnx2tf/releases/download/0.0.2/resnet18-v1-7.onnx
onnx2tf -i resnet18-v1-7.onnx -v info

# Override undefined batch size or other dimensions with static values.
# If the model has undefined dimensions, rewriting them to a static size will significantly
# improve the success rate of the conversion.
# The `-b` option overwrites the zero-dimensional batch size with the number specified
# without input OP name.
# Note that if there are multiple input OPs, the zero dimension of all input OPs is
# forced to be rewritten.
# The `-sh/--shape-hints` option provides shape hints for input tensors with undefined
# dimensions, significantly improving the conversion success rate for models with dynamic
# input shapes. Specifying this option in combination with the `-b` option will further
# improve the success rate of model conversion. The `-sh` option does not change ONNX
# input OPs to static shapes.
# The `-ois/--overwrite_input_shape` option allows undefined dimensions in all dimensions,
# including the zero dimensionality, to be overwritten to a static shape, but requires
# the input OP name to be specified.
# e.g. -ois data1:1,3,224,224 data2:1,255 data3:1,224,6
wget https://github.com/PINTO0309/onnx2tf/releases/download/0.0.2/resnet18-v1-7.onnx
onnx2tf -i resnet18-v1-7.onnx -b 1
or
onnx2tf -i resnet18-v1-7.onnx -sh data:1,3,224,224 -b 1
or
onnx2tf -i resnet18-v1-7.onnx -ois data:1,3,224,224

# Suppress automatic transposition of input OPs from NCW, NCHW, NCDHW to NWC, NHWC, NDHWC.
# onnx2tf is a specification that automatically transposes the input OP to [N,H,W,C] format
# before converting the model. However, since onnx2tf cannot determine from the structure of
# the model whether the input data is image, audio data, or something else, it unconditionally
# transposes the channels. Therefore, it is the models of STT/TTS models where the input is
# not NHWC that tend to have particular problems with the automatic transposition of the
# input OP.
# If you do not want input OPs to be automatically transposed, you can disable auto