09L – Differentiable associative memories, attention, and transformers
Channel: Alfredo Canziani
Date: 2021-08-12

Course website: http://bit.ly/DLSP21-web
Playlist: http://bit.ly/DLSP21-YouTube
Speaker: Yann LeCun

Chapters
00:00:00 – Motivation for reasoning & planning
00:09:11 – Inference through energy minimization
00:18:08 – Disclaimer
00:19:02 – Planning through energy minimization
00:32:59 – Q&A Optimal control diagram
00:39:23 – Differentiable associative memory and attention
01:01:03 – Transformers
01:08:14 – Q&A Other differentiable attention architectures
01:10:32 – Transformer architecture
01:27:54 – Transformer applications: 1. Multilingual transformer Architecture XML-R
01:30:16 – 2. Supervised symbol manipulation
01:32:14 – 3. NL understanding & generation
01:36:51 – 4. DETR
01:46:47 – Planing through optimal control
01:55:37 – Conclusion