Appendix to Building a ML Transformer in a Spreadsheet
Channel: Kevin Lubick
Date: 2022-03-13

This covers multi-headed attention, scaling/mask, residual connections + Add & Normalization. These topics did not fit in the main video.

Original Video: Building an ML Transformer in a Spreadsheet
 - https://www.youtube.com/watch?v=S9eKuRVigjY

Some figures are from "Attention Is All You Need"
 - https://arxiv.org/pdf/1706.03762.pdf

Filled out Machine Learning Transformer in a Spreadsheet: https://bit.ly/TransformerSheet2

More resources about Attention and Transformer Models
 - https://nlp.seas.harvard.edu/2018/04/03/attention.html
 - https://jalammar.github.io/illustrated-transformer/
 - https://towardsdatascience.com/illustrated-guide-to-transformers-step-by-step-explanation-f74876522bc0
 - The Narrated Transformer Language Model https://youtu.be/-QH8fRhqFHM
 - Attention Model Intuition https://youtu.be/SysgYptB198
 - Attention Mechanisms https://youtu.be/3qdQAtfq1jU
 - LSTM is dead. Long Live Transformers! https://youtu.be/S27pHKBEp30
 - Answer to "What exactly are keys, queries, and values?" https://stats.stackexchange.com/a/424127

Narration by James K. Script and visualizations by Kaylee L.

0:00 Introduction
0:15 Multi-headed Attention
2:23 Masking and Scaling
3:33 Residual Connections; Add&Norm