---
title: Accelerate Examples
emoji: 🌖
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 4.36.0
app_file: src/app.py
pinned: false
---

## Accelerate Integration Examples

This is an interactive utility to show users how to incorporate parts of 🤗 Accelerate into their code. 
To use it select a feature to add and a github-like `diff` will be rendered showing what code to remove
and add based on the initial template code.

These are more simplified versions of examples that exist in the [accelerate](https://github.com/huggingface/accelerate) and [transformers](https://github.com/huggingface/transforms) repositories. 

## How each example is made

In the `code_examples` folder are basic text-like files which contain a much-simplified version of some integration. For example:

```
##
<pre>
+from accelerate import Accelerator
+accelerator = Accelerator()
+dataloader, model, optimizer scheduler = accelerator.prepare(
+        dataloader, model, optimizer, scheduler
+)

for batch in dataloader:
    optimizer.zero_grad()
    inputs, targets = batch
-    inputs = inputs.to(device)
-    targets = targets.to(device)
    outputs = model(inputs)
    loss = loss_function(outputs, targets)
-    loss.backward()
+    accelerator.backward(loss)
    optimizer.step()
    scheduler.step()</pre>
##
Everything around `accelerate` ...

##
To learn more checkout the related documentation:
```

There are three overall "sections" separated by two "##" and a newline:

1. The code diff that should be rendered with the new code
2. An explanation of what the diff represents and what's new. Should be up to a paragraph as a TL;DR
3. Link to the documentation users can go to quickly to learn more.

## Creating a `diff`

To create a diff, a similar `pre` tag should be wrapped around the code, and a single `+` or `-` (showing an addition or subtraction) should be added to the code with no extra spacing or formatting. The tool will automatically know how to render these properly. 

For example:

```
<pre>
+from accelerate import Accelerator
+accelerator = Accelerator()
+dataloader, model, optimizer scheduler = accelerator.prepare(
+        dataloader, model, optimizer, scheduler
+)

for batch in dataloader:
    optimizer.zero_grad()
    inputs, targets = batch
-    inputs = inputs.to(device)
-    targets = targets.to(device)
    outputs = model(inputs)
    loss = loss_function(outputs, targets)
-    loss.backward()
+    accelerator.backward(loss)
    optimizer.step()
    scheduler.step()</pre>
```

Also note that the initial `pre` is on a newline, and the latter `</pre>` is not. 

## Rendering the diff

After a diff and starter (if needed) has been made, if a new template was created add it to `TEMPLATES` in `src/template.py`. Otherwise in `app.py` modify the `change` function to properly point to the new integration example to show on a particular selection:

```
def change(inp):
    if inp == "Basic":
        return (templates["initial"], highlight(inp), "## Accelerate Code (Base Integration)")
    elif inp == "Calculating Metrics":
        return (templates["initial_with_metrics"], highlight(inp), f"## Accelerate Code ({inp})")
    else:
        return (templates["accelerate"], highlight(inp), f"## Accelerate Code ({inp})")
```