Xenova

/

tiny-random-falcon-7b

like 0

I've gone through the optimum documentation trying to export my falcon-7b model to onnx with no success.

Ive tried all the following methods explored here (https://huggingface.co/blog/convert-transformers-to-onnx) and got quite a number of errors with each of them (see below). Is there any more specific guidance you can give for how to export these models (I saw you also added support for Mistral as well)?

"low-level" error message:

batch_size, num_heads, kv_length, head_dim = past_key_value[0][0].shape
ValueError: not enough values to unpack (expected 4, got 0)

"mid-level" error message:

KeyError: "falcon is not supported yet. Only ['albert', 'bart', 'beit', 'bert', 'big-bird', 'bigbird-pegasus', 'blenderbot', 'blenderbot-small', 'bloom', 'camembert', 'clip', 'codegen', 'convbert', 'convnext', 'data2vec-text', 'data2vec-vision', 'deberta', 'deberta-v2', 'deit', 'detr', 'distilbert', 'electra', 'flaubert', 'gpt2', 'gptj', 'gpt-neo', 'groupvit', 'ibert', 'imagegpt', 'layoutlm', 'layoutlmv3', 'levit', 'longt5', 'longformer', 'marian', 'mbart', 'mobilebert', 'mobilenet-v1', 'mobilenet-v2', 'mobilevit', 'mt5', 'm2m-100', 'owlvit', 'perceiver', 'poolformer', 'rembert', 'resnet', 'roberta', 'roformer', 'segformer', 'squeezebert', 'swin', 't5', 'vision-encoder-decoder', 'vit', 'whisper', 'xlm', 'xlm-roberta', 'yolos'] are supported. If you want to support falcon please propose a PR or open up an issue."

"high-level" error message:

ValueError: Trying to export a falcon model, that is a custom or unsupported architecture for the task text-generation-with-past, but no custom onnx configuration was passed as `custom_onnx_configs`. Please refer to https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model#custom-export-of-transformers-models for an example on how to export custom models. For the task text-generation-with-past, the Optimum ONNX exporter supports natively the architectures: ['bart', 'blenderbot', 'blenderbot_small', 'bloom', 'codegen', 'gpt2', 'gpt_bigcode', 'gptj', 'gpt_neo', 'gpt_neox', 'marian', 'mbart', 'mpt', 'opt', 'llama', 'pegasus'].