Hugging Face's logo

Optimum documentation

Optimization

You are viewing v1.21.4 version. A newer version v1.23.3 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Optimization

ORTOptimizer

class optimum.onnxruntime.ORTOptimizer

< >

( onnx_model_path: typing.List[os.PathLike] config: PretrainedConfig from_ortmodel: bool = False )

Handles the ONNX Runtime optimization process for models shared on huggingface.co/models.

from_pretrained

< >

( model_or_path: typing.Union[str, os.PathLike, optimum.onnxruntime.modeling_ort.ORTModel] file_names: typing.Optional[typing.List[str]] = None )

Parameters

  • model_or_path (Union[str, os.PathLike, ORTModel]) — The path to a local directory hosting the model to optimize or an instance of an ORTModel to quantize. Can be either:
    • A path to a local directory containing the model to optimize.
    • An instance of ORTModel.
  • file_names(Optional[List[str]], defaults to None) — The list of file names of the models to optimize.

get_fused_operators

< >

( onnx_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.

Computes the dictionary mapping the name of the fused operators to their number of apparition in the model.

get_nodes_number_difference

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
  • onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the difference in the number of nodes between the original and the optimized model.

get_operators_difference

< >

( onnx_model_path: typing.Union[str, os.PathLike] onnx_optimized_model_path: typing.Union[str, os.PathLike] )

Parameters

  • onnx_model_path (Union[str, os.PathLike]) — Path of the ONNX model.
  • onnx_optimized_model_path (Union[str, os.PathLike]) — Path of the optimized ONNX model.

Compute the dictionary mapping the operators name to the difference in the number of corresponding nodes between the original and the optimized model.

optimize

< >

( optimization_config: OptimizationConfig save_dir: typing.Union[str, os.PathLike] file_suffix: typing.Optional[str] = 'optimized' use_external_data_format: typing.Optional[bool] = None one_external_file: bool = True )

Parameters

  • optimization_config (OptimizationConfig) — The configuration containing the parameters related to optimization.
  • save_dir (Union[str, os.PathLike]) — The path used to save the optimized model.
  • file_suffix (str, defaults to "optimized") — The file suffix used to save the optimized model.
  • use_external_data_format (Optional[bool], defaults to None) — Whether to use external data format to store model of size >= 2Gb. This argument is deprecated.
  • one_external_file (bool, defaults to True) — When use_external_data_format=True, whether to save all tensors to one external file. If False, save each tensor to a file named with the tensor name.

Optimizes a model given the optimization specifications defined in optimization_config.

< > Update on GitHub

←Configuration Quantization→