---
title: Attention Rollout
emoji: 🐢
colorFrom: blue
colorTo: pink
sdk: gradio
sdk_version: 2.9.1
app_file: app.py
pinned: false
license: apache-2.0
---

Attention Rollout was proposed by [Abnar et al.](https://arxiv.org/abs/2005.00928) to quantify the information
that flows through self-attention layers. In the original ViT paper ([Dosovitskiy et al.](https://arxiv.org/abs/2010.11929)),
the authors use it to investigate the representations learned by ViTs. The model used in the backend is a ViT B-16 model. For more
details about it, refer to [this notebook](https://github.com/sayakpaul/probing-vits/blob/main/notebooks/load-jax-weights-vitb16.ipynb).