Hugging Face's logo

Optimum documentation

Optimization

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v1.19.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Optimization

ORTOptimizer

class optimum.onnxruntime.ORTOptimizer

< >

( onnx_model_path: typing.List[os.PathLike] config: PretrainedConfig from_ortmodel: bool = False )

Handles the ONNX Runtime optimization process for models shared on huggingface.co/models.

from_pretrained

< >

( model_or_path: typing.Union[str, os.PathLike, optimum.onnxruntime.modeling_ort.ORTModel] file_names: typing.Optional[typing.List[str]] = None )

Parameters

  • model_or_path (Union[str, os.PathLike, ORTModel]) — The path to a local directory hosting the model to optimize or an instance of an ORTModel to quantize. Can be either:
    • A path to a local directory containing the model to optimize.
    • An instance of ORTModel.
  • file_names(Optional[List[str]], defaults to None) — The list of file names of the models to optimize.

get_fused_operators

< >

( onnx_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

Computes the dictionary mapping the name of the fused operators to their number of apparition in the model.

get_nodes_number_difference

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
  • onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the difference in the number of nodes between the original and the optimized model.

get_operators_difference

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
  • onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the dictionary mapping the operators name to the difference in the number of corresponding nodes between the original and the optimized model.

optimize

< >

( optimization_config: OptimizationConfig save_dir: typing.Union[str, os.PathLike] file_suffix: typing.Optional[str] = 'optimized' use_external_data_format: typing.Optional[bool] = None one_external_file: bool = True )

Parameters

  • optimization_config (OptimizationConfig) — The configuration containing the parameters related to optimization.
  • save_dir (Union[str, os.PathLike]) — The path used to save the optimized model.
  • file_suffix (str, defaults to "optimized") — The file suffix used to save the optimized model.
  • use_external_data_format (Optional[bool], defaults to None) — Whether to use external data format to store model of size >= 2Gb. This argument is deprecated.
  • one_external_file (bool, defaults to True) — When use_external_data_format=True, whether to save all tensors to one external file. If False, save each tensor to a file named with the tensor name.

Optimizes a model given the optimization specifications defined in optimization_config.

←Configuration Quantization→