Transformers documentation

Exporting 🤗 Transformers models to ONNX

You are viewing v4.17.0 version. A newer version v4.38.2 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Exporting 🤗 Transformers models to ONNX

🤗 Transformers provides a transformers.onnx package that enables you to convert model checkpoints to an ONNX graph by leveraging configuration objects.

See the guide on exporting 🤗 Transformers models for more details.

ONNX Configurations

We provide three abstract classes that you should inherit from, depending on the type of model architecture you wish to export:

OnnxConfig

class transformers.onnx.OnnxConfig

< >

( config: PretrainedConfig task: str = 'default' patching_specs: typing.List[transformers.onnx.config.PatchingSpec] = None )

Base class for ONNX exportable model describing metadata on how to export the model through the ONNX format.

flatten_output_collection_property

< >

( name: str field: typing.Iterable[typing.Any] ) (Dict[str, Any])

Returns

(Dict[str, Any])

Outputs with flattened structure and key mapping this new structure.

Flatten any potential nested structure expanding the name of the field with the index of the element within the structure.

from_model_config

< >

( config: PretrainedConfig task: str = 'default' )

Instantiate a OnnxConfig for a specific model

generate_dummy_inputs

< >

( tokenizer: PreTrainedTokenizer batch_size: int = -1 seq_length: int = -1 is_pair: bool = False framework: typing.Optional[transformers.file_utils.TensorType] = None )

Generate inputs to provide to the ONNX exporter for the specific framework

use_external_data_format

< >

( num_parameters: int )

Flag indicating if the model requires using external data format

OnnxConfigWithPast

class transformers.onnx.OnnxConfigWithPast

< >

( config: PretrainedConfig task: str = 'default' patching_specs: typing.List[transformers.onnx.config.PatchingSpec] = None use_past: bool = False )

fill_with_past_key_values_

< >

( inputs_or_outputs: typing.Mapping[str, typing.Mapping[int, str]] direction: str )

Fill the input_or_ouputs mapping with past_key_values dynamic axes considering.

with_past

< >

( config: PretrainedConfig task: str = 'default' )

Instantiate a OnnxConfig with use_past attribute set to True

OnnxSeq2SeqConfigWithPast

class transformers.onnx.OnnxSeq2SeqConfigWithPast

< >

( config: PretrainedConfig task: str = 'default' patching_specs: typing.List[transformers.onnx.config.PatchingSpec] = None use_past: bool = False )

ONNX Features

Each ONNX configuration is associated with a set of features that enable you to export models for different types of topologies or tasks.

FeaturesManager

class transformers.onnx.FeaturesManager

< >

( )

check_supported_model_or_raise

< >

( model: typing.Union[transformers.modeling_utils.PreTrainedModel, transformers.modeling_tf_utils.TFPreTrainedModel] feature: str = 'default' )

Check whether or not the model has the requested features.

get_model_class_for_feature

< >

( feature: str )

Attempt to retrieve an AutoModel class from a feature name.

get_model_from_feature

< >

( feature: str model: str )

Attempt to retrieve a model from a model’s name and the feature to be enabled.

get_supported_features_for_model_type

< >

( model_type: str model_name: typing.Optional[str] = None )

Try to retrieve the feature -> OnnxConfig constructor map from the model type.