🤗 Optimum is an extension of 🤗 Transformers, providing a set of performance optimization tools enabling maximum efficiency to train and run models on targeted hardware.
The AI ecosystem evolves quickly and more and more specialized hardware along with their own optimizations are emerging every day. As such, Optimum enables users to efficiently use any of these platforms with the same ease inherent to transformers.
🤗 Optimum aims at providing more diversity towards the kind of hardware users can target to train and finetune their models.
To achieve this, we are collaborating with the following hardware manufacturers in order to provide the best transformers integration:
- Graphcore IPUs - IPUs are a completely new kind of massively parallel processor to accelerate machine intelligence. More information here.
- Habana Gaudi Processor (HPU) - HPUs are designed to maximize training throughput and efficiency. More information here.
- Intel - Enabling the usage of Intel tools to accelerate end-to-end pipelines on Intel architectures. More information here.
- More to come soon! :star:
Along with supporting dedicated AI hardware for training, Optimum also provides inference optimizations towards various frameworks and platforms.
|Features||ONNX Runtime||Intel Neural Compressor|
|Post-training Dynamic Quantization||✅||✅|
|Post-training Static Quantization||✅||✅|
|Quantization Aware Training (QAT)||Stay tuned! ⭐||✅|
🤗 Optimum can be installed using
pip as follows:
python -m pip install optimum
If you’d like to use the accelerator-specific features of 🤗 Optimum, you can install the required dependencies according to the table below:
|Intel Neural Compressor (INC)|
|Habana Gaudi Processor (HPU)|
If you’d like to play with the examples or need the bleeding edge of the code and can’t wait for a new release, you can install the base library from source as follows:
python -m pip install git+https://github.com/huggingface/optimum.git
For the acclerator-specific features, you can install them by appending
#egg=optimum[accelerator_type] to the
pip command, e.g.
python -m pip install git+https://github.com/huggingface/optimum.git#egg=optimum[onnxruntime]