Repository: huggingface/transformers
Pull request: Update HuBERT model card according to template

# What does this PR do?
This PR updates the HuBERT model card to comply with the format introduced in #36979.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@stevhliu 

## Notes
- I did not include the **AttentionMaskVisualizer** section since I'm unfamiliar. Please advise if that should be added.
- I preserved the **Flash Attention 2** section from the original model card, since it appears to be relevant and informative. Let me know if you'd prefer it removed for template consistency.

Changed files:
filename: docs/source/en/model_doc/hubert.md
patch: @@ -14,88 +14,115 @@ rendered properly in your Markdown viewer.
 
 -->
 
-# Hubert
-
-<div class="flex flex-wrap space-x-1">
-<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
-<img alt="TensorFlow" src="https://img.shields.io/badge/TensorFlow-FF6F00?style=flat&logo=tensorflow&logoColor=white">
-<img alt="FlashAttention" src="https://img.shields.io/badge/%E2%9A%A1%EF%B8%8E%20FlashAttention-eae0c8?style=flat">
-<img alt="SDPA" src="https://img.shields.io/badge/SDPA-DE3412?style=flat&logo=pytorch&logoColor=white">
+<div style="float: right;">
+    <div class="flex flex-wrap space-x-1">
+        <img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
+        <img alt="TensorFlow" src="https://img.shields.io/badge/TensorFlow-FF6F00?style=flat&logo=tensorflow&logoColor=white">
+        <img alt="FlashAttention" src="https://img.shields.io/badge/%E2%9A%A1%EF%B8%8E%20FlashAttention-eae0c8?style=flat">
+        <img alt="SDPA" src="https://img.shields.io/badge/SDPA-DE3412?style=flat&logo=pytorch&logoColor=white">
+    </div>
 </div>
 
-## Overview
+# HuBERT
+
+[HuBERT](https://huggingface.co/papers/2106.07447) is a self-supervised speech model to cluster aligned target labels for BERT-like prediction loss and applying the prediction loss only over masked regions to force the model to learn both acoustic and language modeling over continuous inputs. It addresses the challenges of multiple sound units per utterance, no lexicon during pre-training, and variable-length sound units without explicit segmentation.
+
+You can find all the original HuBERT checkpoints under the [HuBERT](https://huggingface.co/collections/facebook/hubert-651fca95d57549832161e6b6) collection.
 
-Hubert was proposed in [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://huggingface.co/papers/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan
-Salakhutdinov, Abdelrahman Mohamed.
+> [!TIP]
+> This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten).
+>
+> Click on the HuBERT models in the right sidebar for more examples of how to apply HuBERT to different audio tasks.
 
-The abstract from the paper is the following:
+The example below demonstrates how to automatically transcribe speech into text with [`Pipeline`] or the [`AutoModel`] class.
 
-*Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are
-multiple sound units in each input utterance, (2) there is no lexicon of input sound units during the pre-training
-phase, and (3) sound units have variable lengths with no explicit segmentation. To deal with these three problems, we
-propose the Hidden-Unit BERT (HuBERT) approach for self-supervised speech representation learning, which utilizes an
-offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our
-approach is applying the prediction loss over the masked regions only, which forces the model to learn a combined
-acoustic and language model over the continuous inputs. HuBERT relies primarily on the consistency of the unsupervised
-clustering step rather than the intrinsic quality of the assigned cluster labels. Starting with a simple k-means
-teacher of 100 clusters, and using two iterations of clustering, the HuBERT model either matches or improves upon the
-state-of-the-art wav2vec 2.0 performance on the Librispeech (960h) and Libri-light (60,000h) benchmarks with 10min, 1h,
-10h, 100h, and 960h fine-tuning subsets. Using a 1B parameter model, HuBERT shows up to 19% and 13% relative WER
-reduction on the more challenging dev-other and test-other evaluation subsets.*
+<hfoptions id="usage">
+<hfoption id="Pipeline">
 
-This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten).
+```python
+import torch
+from transformers import pipeline
 
-# Usage tips
+pipeline = pipeline(
+    task="automatic-speech-recognition",
+    model="facebook/hubert-large-ls960-ft",
+    torch_dtype=torch.float16,
+    device=0
+)
 
-- Hubert is a speech model that accepts a float array corresponding to the raw waveform of the speech signal.
-- Hubert model was fine-tuned using connectionist temporal classification (CTC) so the model output has to be decoded
-  using [`Wav2Vec2CTCTokenizer`].
-- The `head_mask` argument is ignored when using all attention implementation other than "eager". If you have a `head_mask` and want it to have effect, load the model with `XXXModel.from_pretrained(model_id, attn_implementation="eager")`  
+pipeline("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac")
+```
 
-## Using Flash Attention 2
+</hfoption>
+<hfoption id="AutoModel">
 
-Flash Attention 2 is an faster, optimized version of the model.
+```python
+import torch
+from transformers import AutoProcessor, AutoModelForCTC
+from datasets import load_dataset
 
-### Installation 
+dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation").sort("id")
+sampling_rate = dataset.features["audio"].sampling_rate
 
-First, check whether your hardware is compatible with Flash Attention 2. The latest list of compatible hardware can be found in the [official documentation](https://github.com/Dao-AILab/flash-attention#installation-and-features). If your hardware is not compatible with Flash Attention 2, you can still benefit from attention kernel optimisations through Better Transformer support covered [above](https://huggingface.co/docs/transformers/main/en/model_doc/bark#using-better-transformer).
+processor = AutoProcessor.from_pretrained("facebook/hubert-base-ls960")
+model = AutoModelForCTC.from_pretrained("facebook/hubert-base-ls960", torch_dtype=torch.float16, device_map="auto", attn_implementation="sdpa")
 
-Next, [install](https://github.com/Dao-AILab/flash-attention#installation-and-features) the latest version of Flash Attention 2:
+inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
+with torch.no_grad():
+    logits = model(**inputs).logits
+predicted_ids = torch.argmax(logits, dim=-1)
 
-```bash
-pip install -U flash-attn --no-build-isolation
+transcription = processor.batch_decode(predicted_ids)
+print(transcription[0])
 ```
 
-### Usage
+</hfoption>
+</hfoptions>
 
-Below is an expected speedup diagram comparing the pure inference time between the native implementation in transformers of `facebook/hubert-large-ls960-ft`, the flash-attention-2 and the sdpa (scale-dot-product-attention) version. We show the average speedup obtained on the `librispeech_asr` `clean` validation split: 
+## Quantization
 
-```python
->>> from transformers import HubertModel
->>> import torch
+Quantization reduces the memory burden of large models by representing the weights in a lower precision.
+Refer to the [Quantization](../quantization/overview) overview for more available quantization backends.
 
->>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft", torch_dtype=torch.float16, attn_implementation="flash_attention_2").to("cuda")
-...
-```
+The example below uses [bitsandbytes](../quantization/bitsandbytes) to quantize the weights to 4-bits.
+
+```python
+import torch
+from transformers import AutoProcessor, AutoModelForCTC, BitsAndBytesConfig
+from datasets import load_dataset
 
-### Expected speedups
+bnb_config = BitsAndBytesConfig(
+    load_in_8bit=True,
+    llm_int8_threshold=6.0
+)
 
-Below is an expected speedup diagram comparing the pure inference time between the native implementation in transformers of the `facebook/hubert-large-ls960-ft` model and the flash-attention-2 and sdpa (scale-dot-product-attention) versions. . We show the average speedup obtained on the `librispeech_asr` `clean` validation split: 
+dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation").sort("id")
+sampling_rate = dataset.features["audio"].sampling_rate
 
+processor = AutoProcessor.from_pretrained("facebook/hubert-base-ls960")
+model = AutoModelForCTC.from_pretrained("facebook/hubert-base-ls960", quantization_config=bnb_config, torch_dtype=torch.float16, device_map="auto", attn_implementation="sdpa")
 
-<div style="text-align: center">
-<img src="https://huggingface.co/datasets/kamilakesbi/transformers_image_doc/resolve/main/data/Hubert_speedup.png">
-</div>
+inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
+with torch.no_grad():
+    logits = model(**inputs).logits
+predicted_ids = torch.argmax(logits, dim=-1)
 
+transcription = processor.batch_decode(predicted_ids)
+print(transcription[0])
+```
 
-## Resources
+## Notes
 
-- [Audio classification task guide](../tasks/audio_classification)
-- [Automatic speech recognition task guide](../tasks/asr)
+- HuBERT models expect raw audio input as a 1D float array sampled at 16kHz.
+- If you want to use a `head_mask`, use the model with `attn_implementation="eager"`.
+  ```python
+  model = HubertModel.from_pretrained("facebook/hubert-base-ls960", attn_implementation="eager")
+  ```
 
 ## HubertConfig
 
 [[autodoc]] HubertConfig
+    - all
 
 <frameworkcontent>
 <pt>