Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

๐Ÿค— Optimum Habana

๐Ÿค— Optimum Habana is the interface between the ๐Ÿค— Transformers library and Habanaโ€™s Gaudi processor (HPU). It provides a set of tools enabling easy model loading and training on single- and multi-HPU settings for different downstream tasks. The list of officially validated models and tasks is available here. Users can try other models and tasks with only few changes.

What is a Habana Processing Unit (HPU)?

Quote from the Hugging Face blog post:

Habana Gaudi training solutions, which power Amazonโ€™s EC2 DL1 instances and Supermicroโ€™s X12 Gaudi AI Training Server, deliver price/performance up to 40% lower than comparable training solutions and enable customers to train more while spending less. The integration of ten 100 Gigabit Ethernet ports onto every Gaudi processor enables system scaling from 1 to thousands of Gaudis with ease and cost-efficiency. Habanaโ€™s SynapseAIยฎ is optimizedโ€”at inceptionโ€”to enable Gaudi performance and usability, supports TensorFlow and PyTorch frameworks, with a focus on computer vision and natural language processing applications.

Install

To install the latest release of this package:
pip install optimum[habana]

To use DeepSpeed on HPUs, you also need to run the following command:

pip install git+https://github.com/HabanaAI/DeepSpeed.git@1.6.0

Optimum Habana is a fast-moving project, and you may want to install it from source:

pip install git+https://github.com/huggingface/optimum-habana.git

Alternatively, you can install the package without pip as follows:

git clone https://github.com/huggingface/optimum-habana.git
cd optimum-habana
python setup.py install

Last but not least, donโ€™t forget to install requirements for every example:

cd <example-folder>
pip install -r requirements.txt

Validated Models

The following model architectures, tasks and device distributions have been validated for ๐Ÿค— Optimum Habana:

Text Classification Question Answering Language Modeling Summarization Translation Image Classification Single Card Multi Card DeepSpeed
BERT โœ… โœ… โœ… โŒ โŒ โŒ โœ… โœ… โœ…
RoBERTa โŒ โœ… โœ… โŒ โŒ โŒ โœ… โœ… โœ…
ALBERT โŒ โœ… โœ… โŒ โŒ โŒ โœ… โœ… โœ…
DistilBERT โŒ โœ… โœ… โŒ โŒ โŒ โœ… โœ… โœ…
GPT2 โŒ โŒ โœ… โŒ โŒ โŒ โœ… โœ… โœ…
T5 โŒ โŒ โŒ โœ… โœ… โŒ โœ… โœ… โœ…
ViT โŒ โŒ โŒ โŒ โŒ โœ… โœ… โœ… โœ…
Swin โŒ โŒ โŒ โŒ โŒ โœ… โœ… โœ… โœ…

Other models and tasks supported by the ๐Ÿค— Transformers library may also work. You can refer to this section for using them with ๐Ÿค— Optimum Habana. Besides, this page explains how to modify any example from the ๐Ÿค— Transformers library to make it work with ๐Ÿค— Optimum Habana.

If you find any issue while using those, please open an issue or a pull request in the Github repository of the project.

Gaudi Setup

Please refer to Habana Gaudiโ€™s official installation guide.

Tests should be run in a Docker container based on Habana Docker images.