Optimization

🤗 Optimum provides an optimum.onnxruntime package that enables you to apply graph optimization on many model hosted on the 🤗 hub using the ONNX Runtime model optimization tool.

ORTOptimizer

class optimum.onnxruntime.ORTOptimizer

< source >

( tokenizer: PreTrainedTokenizer model: PreTrainedModel feature: str = 'default' opset: typing.Optional[int] = None )

Handles the ONNX Runtime optimization process for models shared on huggingface.co/models.

export

< source >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_output_path: typing.Union[str, os.PathLike] optimization_config: OptimizationConfig use_external_data_format: bool = False )

Parameters

onnx_model_path (Union[str, os.PathLike]) — The path used to save the model exported to an ONNX Intermediate Representation (IR).
onnx_optimized_model_output_path (Union[str, os.PathLike]) — The path used to save the optimized model exported to an ONNX Intermediate Representation (IR).
optimization_config (OptimizationConfig) — The configuration containing the parameters related to optimization.
use_external_data_format (bool, defaults to False) — Whether uto se external data format to store model which size is >= 2Gb.

Optimize a model given the optimization specifications defined in optimization_config.

from_pretrained

< source >

( model_name_or_path: typing.Union[str, os.PathLike] feature: str opset: typing.Optional[int] = None )

Parameters

model_name_or_path (Union[str, os.PathLike]) — Repository name in the Hugging Face Hub or path to a local directory hosting the model.
feature (str) — Feature to use when exporting the model.
opset (int, optional) — ONNX opset version to export the model with.

Instantiate a ORTOptimizer from a pretrained pytorch model and tokenizer.

get_fused_operators

< source >

( onnx_model_path: typing.Union[str, os.PathLike] )

Parameters

onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

Compute the dictionary mapping the name of the fused operators to their number of apparition in the model.

get_nodes_number_difference

< source >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the difference in the number of nodes between the original and the optimized model.

get_operators_difference

< source >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the dictionary mapping the operators name to the difference in the number of corresponding nodes between the original and the optimized model.