Google BERT Architecture Explained 3/3 -(Masked Language Model, Attention visualizations etc)
Channel: Mixture
Date: 2019-03-12

Google BERT (Bidirectional Encoder Representations from Transformers) Machine Learning model for NLP has been a breakthrough. In this video series I am going to explain the architecture and help reducing time to understand the complex architecture.
Paper reference: Attention is all you need

All References:
https://arxiv.org/pdf/1706.03762.pdf
https://github.com/huggingface/pytorch-pretrained-BERT
http://mlexplained.com/2017/12/29/attention-is-all-you-need-explained/
https://towardsdatascience.com/deconstructing-bert-distilling-6-patterns-from-100-million-parameters-b49113672f77
https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270
https://ai.google/research/teams/language/
https://rajpurkar.github.io/SQuAD-explorer/
https://google.github.io/seq2seq/
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
https://stats.stackexchange.com/questions/321054/what-are-residual-connections-in-rnns