How to use this onnx model
#1
by
mkj69
- opened
Ask how to use this onnx model, and especially how to configure this splitter based on these json files?
Hi
@mkj69
this model is not meant to be used. Please use the last up to date optimum version to export decoder models in a single ONNX file:
optimum-cli export onnx --model gpt2 gpt2_onnx
The prefill step needs to pass 0-length past key values (KV cache). You can inspect the exported model with Netron to understand what are the inputs/outputs.