Package: keras2trt
keras2trtKeras2TRT is a cli tool that is capable of converting keras saved_models to TensorRT engine. Currently supported conversions are:Keras to ONNXONNX to TensorRTKeras to TensorRTNOTE:The CLI is tested converting image segmentation, classification and detection models.RequirementsThe following packages need to be installed to use the cli.pipinstallnvidia-pyindex==1.0.9\&&pipinstallnvidia-tensorrt==8.4.2.4NOTE:nvidia-tensorrt==8.4.2.4 is compatible with nvcr.io/nvidia/tritonserver:22.08-py3 docker image.Installationpip install keras2trtUsageUsage: keras2trt [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.
  --help                          Show this message and exit.

Commands:
  keras2onnx
  keras2trt
  onnx2trt
  versionkeras2onnxUsage: keras2trt keras2onnx [OPTIONS]

  Convert Keras model to ONNX model.

  - if --save-path does not have a suffix, ".onnx" suffix will be added to the
  saved ONNX model.

Options:
  --opset INTEGER     ONNX model opset.  [default: 15]
  --keras-model PATH  Path to the Keras model.  [required]
  --save-path PATH    Path to save the TensorRT engine.  [required]
  --help              Show this message and exit.Examplekeras2trt keras2onnx --keras-model models/inceptionv3 --opset 13 --save-path models/tf2onnxModel path is a keras saved_model directory.models/inceptionv3
├── assets
├── keras_metadata.pb
├── saved_model.pb
└── variables
    ├── variables.data-00000-of-00001
    └── variables.indexkeras2trtUsage: keras2trt keras2trt [OPTIONS]

  Convert Keras model to tensorrt engine.

  - If --save-path does not have a suffix, ".engine" suffix will be added to
  the saved TensorRT engine.

  - All min_shape, opt_shape, and max_shape need to be set for dynamic batch
  size.

  - If none of the shape arguments is set, the batch size will be set as 1.

Options:
  --opset INTEGER     ONNX model opset.  [default: 15]
  --in-shape TEXT     Model input shape.
  --min-shape TEXT    Minimum input shape for dynamic batch.
  --opt-shape TEXT    Optimal input shape for dynamic batch.
  --max-shape TEXT    Maximum input shape for dynamic batch.
  --keras-model PATH  Path to the Keras model.  [required]
  --save-path PATH    Path to save the TensorRT engine.  [required]
  --help              Show this message and exit.Examplekeras2trt keras2trt --opset 17 --in-shape "(1,256,256,3)" --keras-model models/inceptionv3 --save-path models/keras2trt.trt

keras2trt keras2trt --opset 15 --min-shape "(5,256,256,3)" --opt-shape "(15,256,256,3)" --max-shape "(30,256,256,3)" --keras-model models/inceptionv3 --save-path models/keras2trtModel path is a keras saved_model directory.models/inceptionv3
├── assets
├── keras_metadata.pb
├── saved_model.pb
└── variables
    ├── variables.data-00000-of-00001
    └── variables.indexonnx2trtUsage: keras2trt onnx2trt [OPTIONS]

  Convert ONNX model to tensorrt engine.

  - If --save-path does not have a suffix, ".engine" suffix will be added to
  the saved TensorRT engine.

  - All min_shape, opt_shape, and max_shape need to be set for dynamic batch
  size.

  - If none of the shape arguments is set, the batch size will be set as 1.

Options:
  --in-shape TEXT    Model input shape.
  --min-shape TEXT   Minimum input shape for dynamic batch.
  --opt-shape TEXT   Optimal input shape for dynamic batch.
  --max-shape TEXT   Maximum input shape for dynamic batch.
  --onnx-model PATH  Path to the ONNX model.  [required]
  --save-path PATH   Path to save the TensorRT engine.  [required]
  --help             Show this message and exit.Examplekeras2trt onnx2trt --in-shape "(1,256,256,3)" --onnx-model models/tf2onnx.onnx --save-path models/onnx2trt

keras2trt onnx2trt --min-shape "(5,256,256,3)" --opt-shape "(15,256,256,3)" --max-shape "(30,256,256,3)" --onnx-model models/tf2onnx.onnx --save-path models/onnx2trt