Hugging Face's logo

Spaces:
onnx
/
export
Running on CPU Upgrade

App Files Community
7

Onnx for flan-t5

#4
by p-christ - opened
Discussion
p-christ
•
edited Mar 23, 2023

Hey,

Does anyone know how to get Onnx to work for flan-t5?

There's no class T5ForConditionalGeneration in optimum.onnxruntime like there is in transformers so it doesn't seem to work.

fxmarty
 onnx org

Hi @p-christ , thank you I'll have a look!

fxmarty
 onnx org

Hi @p-christ , flan-t5 reuses the architecture from t5. Thus you can use ORTModelForSeq2SeqLM: https://huggingface.co/docs/optimum/main/en/onnxruntime/package_reference/modeling_ort#optimum.onnxruntime.ORTModelForSeq2SeqLM

fxmarty changed discussion status to closed

Sign up or log in to comment