Pruning

🤗 Optimum provides an optimum.intel.neural_compressor package that enables you to apply magnitude pruning on many model hosted on the 🤗 hub using the Intel Neural Compressor pruning API.

IncPruner

class optimum.intel.IncPruner

< source >

( model: typing.Union[transformers.modeling_utils.PreTrainedModel, torch.nn.modules.module.Module] config_path_or_obj: typing.Union[str, optimum.intel.neural_compressor.configuration.IncPruningConfig] tokenizer: typing.Optional[transformers.tokenization_utils_base.PreTrainedTokenizerBase] = None eval_func: typing.Optional[typing.Callable] = None train_func: typing.Optional[typing.Callable] = None )

from_config

< source >

( model_name_or_path: str inc_config: typing.Union[optimum.intel.neural_compressor.configuration.IncPruningConfig, str, NoneType] = None config_name: str = None **kwargs ) → pruner

Parameters

model_name_or_path (str) — Repository name in the Hugging Face Hub or path to a local directory hosting the model.
inc_config (Union[IncPruningConfig, str], optional) — Configuration file containing all the information related to the pruning strategy. Can be either:
- an instance of the class IncPruningConfig,
- a string valid as input to IncPruningConfig.from_pretrained.
config_name (str, optional) — Name of the configuration file.
cache_dir (str, optional) — Path to a directory in which a downloaded configuration should be cached if the standard cache should not be used.
force_download (bool, optional, defaults to False) — Whether or not to force to (re-)download the configuration files and override the cached versions if they exist.
resume_download (bool, optional, defaults to False) — Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists.
revision(str, optional) — The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.
eval_func (Callable, optional) — Evaluation function to evaluate the tuning objective.
train_func (Callable, optional) — Training function which will be combined with pruning.

Returns

pruner

IncPruner object.

Instantiate an IncPruner object from a configuration file which can either be hosted on huggingface.co or from a local directory path.