LSTM is dead. Long Live Transformers!
Channel: Seattle Applied Deep Learning
Date: 2019-12-03

Leo Dirac (@leopd) talks about how LSTM models for Natural Language Processing (NLP) have been practically replaced by transformer-based models.  Basic background on NLP, and a brief history of supervised learning techniques on documents, from bag of words, through vanilla RNNs and LSTM.  Then there's a technical deep dive into how Transformers work with multi-headed self-attention, and positional encoding.  Includes sample code for applying these ideas to real-world projects.