Spaces:
Runtime error
Runtime error
| title: Attention Rollout | |
| emoji: 🐢 | |
| colorFrom: blue | |
| colorTo: pink | |
| sdk: gradio | |
| sdk_version: 2.9.1 | |
| app_file: app.py | |
| pinned: false | |
| license: apache-2.0 | |
| Attention Rollout was proposed by [Abnar et al.](https://arxiv.org/abs/2005.00928) to quantify the information | |
| that flows through self-attention layers. In the original ViT paper ([Dosovitskiy et al.](https://arxiv.org/abs/2010.11929)), | |
| the authors use it to investigate the representations learned by ViTs. The model used in the backend is `deit_tiny_patch16_224`. For more details about it, refer [here](https://tfhub.dev/sayakpaul/collections/deit/1). DeiT was proposed by [Touvron et al.](https://arxiv.org/abs/2012.12877)" | |