Optimum documentation

πŸ€— Optimum Habana

You are viewing v1.10.1 version. A newer version v1.22.0 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

πŸ€— Optimum Habana

πŸ€— Optimum Habana is the interface between the πŸ€— Transformers and πŸ€— Diffusers libraries and Habana’s Gaudi processor (HPU). It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below.

HPUs offer fast model training and inference as well as a great price-performance ratio. Check out this blog post about BERT pre-training and this article benchmarking Habana Gaudi2 versus Nvidia A100 GPUs for concrete examples. If you are not familiar with HPUs, we recommend you take a look at our conceptual guide.

The table below shows which model architectures, tasks and device distributions are currently supported for πŸ€— Optimum Habana:

Architecture Single Card Multi Card DeepSpeed
Tasks
BERT βœ… βœ… βœ…
  • text classification
  • question answering
  • language modeling
  • RoBERTa βœ… βœ… βœ…
  • question answering
  • language modeling
  • ALBERT βœ… βœ… βœ…
  • question answering
  • language modeling
  • DistilBERT βœ… βœ… βœ…
  • question answering
  • language modeling
  • GPT2 βœ… βœ… βœ…
  • language modeling
  • text generation
  • T5 βœ… βœ… βœ…
  • summarization
  • translation
  • question answering
  • ViT βœ… βœ… βœ…
  • image classification
  • Swin βœ… βœ… βœ…
  • image classification
  • Wav2Vec2 βœ… βœ… βœ…
  • audio classification
  • speech recognition
  • Stable Diffusion βœ… ❌ ❌
  • text-to-image generation
  • CLIP βœ… βœ… βœ…
  • contrastive image-text training
  • BLOOM(Z) ❌ ❌ βœ…
  • text generation
  • StarCoder βœ… ❌ βœ…
  • text generation
  • ESMFold βœ… ❌ ❌
  • protein folding
  • GPT-J βœ… ❌ βœ…
  • text generation
  • GPT-NeoX βœ… ❌ βœ…
  • text generation
  • OPT βœ… ❌ βœ…
  • text generation
  • BridgeTower βœ… βœ… βœ…
  • contrastive image-text training
  • Llama βœ… ❌ ❌
  • text generation
  • Other models and tasks supported by the πŸ€— Transformers library may also work. You can refer to this page for examples of how to use them with πŸ€— Optimum Habana. Besides, the Quickstart explains how to modify any example from the πŸ€— Transformers library to make it work with πŸ€— Optimum Habana.