amtutorial 0.0.9
Add your description here
Requires Python: >=3.9

# Getting started


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

This website serves as a living companion to the [tutorial
manuscript](https://arxiv.org/abs/2507.06211) and to the [tutorial
presentation at ICML 2025](https://icml.cc/virtual/2025/40004). It
dreams of being a one-stop shop for learning all things about
Associative Memory. It’s definitely not there yet.

### Website structure

#### Notebook demos

The website is a (growing) collection of notebook demos on Associative
Memory. Each notebook is primarily a blog post on this site, but it is
also fully runnable on colab and as a raw `.ipynb` file using the `uv`
environment setup [below](#sec-installation).

- [**Dense binary storage**](./tutorial/00_dense_storage.qmd), also
  distributed as [colab
  notebook](https://colab.research.google.com/github/bhoov/amtutorial/blob/main/tutorial_ipynbs/00_dense_storage.ipynb)
  and [raw
  .ipynb](https://github.com/bhoov/amtutorial/blob/main/tutorial_ipynbs/00_dense_storage.ipynb).
- [**Energy Transformer**](./tutorial/01_energy_transformer.qmd), also
  distributed as [colab
  notebook](https://colab.research.google.com/github/bhoov/amtutorial/blob/main/tutorial_ipynbs/01_energy_transformer.ipynb)
  and [raw
  .ipynb](https://github.com/bhoov/amtutorial/blob/main/tutorial_ipynbs/01_energy_transformer.ipynb).
  <!-- - **Energy Transformer: Rederiving the Transformer as an energy-based Associative Memory**. Distributed as [blog](./tutorial/01_energy_transformer.qmd), [colab notebook](https://colab.research.google.com/github/bhoov/amtutorial/blob/main/tutorial_ipynbs/01_energy_transformer.ipynb), and [raw .ipynb](https://github.com/bhoov/amtutorial/blob/main/tutorial_ipynbs/01_energy_transformer.ipynb). -->
- [**Diffusion as Memory**](./tutorial/02_diffusion_as_memory.qmd), also
  distributed as [colab
  notebook](https://colab.research.google.com/github/bhoov/amtutorial/blob/main/tutorial_ipynbs/02_diffusion_as_memory.ipynb)
  and [raw
  .ipynb](https://github.com/bhoov/amtutorial/blob/main/tutorial_ipynbs/02_diffusion_as_memory.ipynb).
- [**Distributed Associative
  Memory**](./tutorial/03_distributed_memory.qmd), also distributed as
  [colab
  notebook](https://colab.research.google.com/github/bhoov/amtutorial/blob/main/tutorial_ipynbs/03_distributed_memory.ipynb)
  and [raw
  .ipynb](https://github.com/bhoov/amtutorial/blob/main/tutorial_ipynbs/03_distributed_memory.ipynb).

See the overview in [tutorials](./tutorial/index.ipynb) for a bit more
detail.

To add new examples, edit the source tutorial notebooks (as either
`.ipynb` or plain text `.qmd` files) saved in `nbs/tutorial/`.

> [!WARNING]
>
> ### Slow first run
>
> The first time you run the notebooks will be slow. We cache some of
> the long-running code after the first time, but the cache will not
> persist across Colab sessions.

#### Utitity library

`pip install amtutorial`

We aim for simplicity and clarity in the notebooks. Thus, we migrate
some helper functions (particularly around loading and processing data)
to a pypi package called
[`amtutorial`](https://www.piwheels.org/project/amtutorial/) to avoid
cluttering the notebooks. An added benefit of this is that all
dependencies needed to run these notebooks can be installed using
`pip install amtutorial`.

> [!NOTE]
>
> ### How this website is built
>
> The [website](https://tutorial.amemory.net) is built using an in-house
> fork of [`nbdev`](https://github.com/bhoov/nbdev/tree/qmd_support)
> that develops everything in this tutorial from source `.ipynb` or
> `.qmd` files saved in `nbs/`. The website, pypi package, and package
> documentation all come for free with `nbdev`. The in-house fork
> enables working with plain text `.qmd` files instead of `.ipynb`
> files. With the [right
> extensions](https://quarto.org/docs/tools/vscode/index.html) and
> [hotkeys](https://github.com/bhoov/nbdev/blob/qmd_support/nbs/tutorials/develop_in_plain_text.qmd),
> `.qmd` files are pleasant to develop inside VSCode and interop
> seamlessly with both git and AI tooling.

## Installation

### I just want to run the notebooks locally

``` sh
pip install amtutorial

## Install torch to run the `diffusion as memory` notebook. CPU or CUDA versions work
# pip install torch --index-url https://download.pytorch.org/whl/cpu

## OPTIONAL: For rendering videos in notebooks, use ffmpeg. Can use conda to install as
#conda install conda-forge::ffmpeg conda-forge::openh264 
```

Then open up the .ipynb notebooks in `tutorial_ipynbs/` in your favorite
notebook editor, using the same env where you installed `amtutorial`.

### I want to develop the website

**Pre-requisites**

- Install
  [`uv`](https://docs.astral.sh/uv/getting-started/installation/) using
  `curl -LsSf https://astral.sh/uv/install.sh | sh`
- Install [`quarto`](https://quarto.org/docs/download/)
- We use
  [`conda`](https://www.anaconda.com/docs/getting-started/miniconda/install)
  (or better yet, [`mamba`](https://github.com/conda-forge/miniforge))
  for managing the `ffmpeg` dependency, which only matters if `ffmpeg`
  is not already installed on your system.

**Setting up the environment**

From the root of the repo:

``` sh
uv sync
source .venv/bin/activate

# Expose venv to ipython
uv run ipython kernel install --user --env VIRTUAL_ENV $(pwd)/.venv --name=amtutorial 

## Install torch to run the `diffusion as memory` notebook. CPU or CUDA versions work
# pip install torch --index-url https://download.pytorch.org/whl/cpu

# OPTIONAL: For rendering videos in notebooks
conda install conda-forge::ffmpeg conda-forge::openh264 
```

**Development pipelines**

View a local version of the website with:

    uv run nbdev_preview

Pushes to `main` deploy the website. The site will be live after a few
minutes on github.

``` sh
git checkout main

# Update the website. Takes a moment even with cached training runs
make deploy && git add . && git commit -m "Update site" && git push
```

Make a minor-patch update to the pypi package (preferably, only if
`amtutorials/src` was updated):

``` sh
make pypi && uv run nbdev_pypi
```

**Useful scripts** (for reference only)

``` sh
uv run nbdev_preview                         # Preview website locally
bash scripts/prep_website_deploy.sh          # Sync dependencies, export qmd notebooks to ipynb for colab, and build website
bash scripts/export_qmd_as_ipynb.sh          # Export qmd notebooks to ipynb for colab
uv run python scripts/sync_dependencies.py   # Sync nbdev and pyproject.toml dependencies
uv run python scripts/prep_pypi.py           # Bump patch version and sync dependencies
uv run nbdev_pypi                            # Push to pypi
```