🤗 Optimum Habana

🤗 Optimum Habana is the interface between the 🤗 Transformers and 🤗 Diffusers libraries and Habana’s Gaudi processor (HPU). It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below.

HPUs offer fast model training and inference as well as a great price-performance ratio. Check out this blog post about BERT pre-training and this article benchmarking Habana Gaudi2 versus Nvidia A100 GPUs for concrete examples. If you are not familiar with HPUs, we recommend you take a look at our conceptual guide.

The table below shows which model architectures, tasks and device distributions are currently supported for 🤗 Optimum Habana:

Architecture	Single Card	Multi Card	DeepSpeed	Tasks
BERT	✅	✅	✅	text classification question answering language modeling
RoBERTa	✅	✅	✅	question answering language modeling
ALBERT	✅	✅	✅	question answering language modeling
DistilBERT	✅	✅	✅	question answering language modeling
GPT2	✅	✅	✅	language modeling text generation
T5	✅	✅	✅	summarization translation question answering
ViT	✅	✅	✅	image classification
Swin	✅	✅	✅	image classification
Wav2Vec2	✅	✅	✅	audio classification speech recognition
Stable Diffusion	✅	❌	❌	text-to-image generation
CLIP	✅	✅	✅	contrastive image-text training
BLOOM(Z)	❌	❌	✅	text generation
StarCoder	✅	❌	✅	text generation
ESMFold	✅	❌	❌	protein folding
GPT-J	✅	❌	✅	text generation
GPT-NeoX	✅	❌	✅	text generation
OPT	✅	❌	✅	text generation
BridgeTower	✅	✅	✅	contrastive image-text training
Llama	✅	❌	❌	text generation

Other models and tasks supported by the 🤗 Transformers library may also work. You can refer to this page for examples of how to use them with 🤗 Optimum Habana. Besides, the Quickstart explains how to modify any example from the 🤗 Transformers library to make it work with 🤗 Optimum Habana.

Tutorials

Learn the basics and become familiar with training transformers on HPUs with 🤗 Optimum. Start here if you are using 🤗 Optimum Habana for the first time!

How-to guides

Practical guides to help you achieve a specific goal. Take a look at these guides to learn how to use 🤗 Optimum Habana to solve real-world problems.

Conceptual guides

High-level explanations for building a better understanding of important topics such as HPUs.

Reference

Technical descriptions of how the Habana classes and methods of 🤗 Optimum Habana work.