Optimum documentation

Optimization

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Optimization

🤗 Optimum provides an optimum.onnxruntime package that enables you to apply graph optimization on many model hosted on the 🤗 hub using the ONNX Runtime model optimization tool.

ORTOptimizer

class optimum.onnxruntime.ORTOptimizer

< >

( tokenizer: PreTrainedTokenizer model: PreTrainedModel feature: str = 'default' opset: typing.Optional[int] = None )

Handles the ONNX Runtime optimization process for models shared on huggingface.co/models.

export

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_output_path: typing.Union[str, os.PathLike] optimization_config: OptimizationConfig use_external_data_format: bool = False )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — The path used to save the model exported to an ONNX Intermediate Representation (IR).
  • onnx_optimized_model_output_path (Union[str, os.PathLike]) — The path used to save the optimized model exported to an ONNX Intermediate Representation (IR).
  • optimization_config (OptimizationConfig) — The configuration containing the parameters related to optimization.
  • use_external_data_format (bool, defaults to False) — Whether uto se external data format to store model which size is >= 2Gb.

Optimize a model given the optimization specifications defined in optimization_config.

from_pretrained

< >

( model_name_or_path: typing.Union[str, os.PathLike] feature: str opset: typing.Optional[int] = None )

Parameters

  • model_name_or_path (Union[str, os.PathLike]) — Repository name in the Hugging Face Hub or path to a local directory hosting the model.
  • feature (str) — Feature to use when exporting the model.
  • opset (int, optional) — ONNX opset version to export the model with.

Instantiate a ORTOptimizer from a pretrained pytorch model and tokenizer.

get_fused_operators

< >

( onnx_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

Compute the dictionary mapping the name of the fused operators to their number of apparition in the model.

get_nodes_number_difference

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
  • onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the difference in the number of nodes between the original and the optimized model.

get_operators_difference

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
  • onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the dictionary mapping the operators name to the difference in the number of corresponding nodes between the original and the optimized model.