Optimum documentation

πŸ€— Optimum

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

πŸ€— Optimum

πŸ€— Optimum is an extension of πŸ€— Transformers, providing a set of performance optimization tools enabling maximum efficiency to train and run models on targeted hardware.

The AI ecosystem evolves quickly and more and more specialized hardware along with their own optimizations are emerging every day. As such, Optimum enables users to efficiently use any of these platforms with the same ease inherent to transformers.

Integration with Hardware Partners

πŸ€— Optimum aims at providing more diversity towards the kind of hardware users can target to train and finetune their models.

To achieve this, we are collaborating with the following hardware manufacturers in order to provide the best transformers integration:

Optimizing models towards inference

Along with supporting dedicated AI hardware for training, Optimum also provides inference optimizations towards various frameworks and platforms.

We currently support ONNX runtime along with Intel Neural Compressor (INC).

Features ONNX Runtime Intel Neural Compressor
Post-training Dynamic Quantization βœ… βœ…
Post-training Static Quantization βœ… βœ…
Quantization Aware Training (QAT) Stay tuned! ⭐ βœ…
Pruning N/A βœ…

Installation

πŸ€— Optimum can be installed using pip as follows:

python -m pip install optimum

If you’d like to use the accelerator-specific features of πŸ€— Optimum, you can install the required dependencies according to the table below:

Accelerator Installation
ONNX runtime python -m pip install optimum[onnxruntime]
Intel Neural Compressor (INC) python -m pip install optimum[intel]
Graphcore IPU python -m pip install optimum[graphcore]
Habana Gaudi Processor (HPU) python -m pip install optimum[habana]

If you’d like to play with the examples or need the bleeding edge of the code and can’t wait for a new release, you can install the base library from source as follows:

python -m pip install git+https://github.com/huggingface/optimum.git

For the acclerator-specific features, you can install them by appending #egg=optimum[accelerator_type] to the pip command, e.g.

python -m pip install git+https://github.com/huggingface/optimum.git#egg=optimum[onnxruntime]