Building a Recurrent Neural Network in a spreadsheet
Channel: Kevin Lubick
Date: 2021-10-25

We continue the theme of building neural networks in a spreadsheet, as a way to do a worked example of them.
In this video, we build a Recurrent Neural Network (RNN) that can predict the next letter given what the user has typed so far.

Building and Training a Traditional Neural Net in a spreadsheet:
https://youtu.be/fjfZZ6S1ad4
https://youtu.be/1zwnPt73pow

Building a Convolutional Neural Net in a spreadsheet:
https://youtu.be/8QfFtUcuYCU
https://youtu.be/pH0s0e62GT8

Sheets Template:
https://bit.ly/RNNSheet1

Overviews of Recurrent Neural Networks:
 - MIT Open Courseware https://youtu.be/qjrad0V0uJE
 - https://karpathy.github.io/2015/05/21/rnn-effectiveness/
 - https://towardsdatascience.com/a-battle-against-amnesia-a-brief-history-and-introduction-of-recurrent-neural-networks-50496aae6740
 
Applications of RNNs:
 - https://quicktotheratcave.tumblr.com/post/187432425523/shall-we-play-a-game
 - https://futurism.com/grad-student-neural-network-write-papers
 - http://proceedings.mlr.press/v32/pinheiro14.pdf
 
More advanced RNNs - LSTM Networks
 - codebasics - https://youtu.be/LfnrRPFhkuY
 - http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Narration by James K. Script and visualizations by Kaylee L.

0:00 Intro 
0:34 Applications of RNNs
1:01 How traditional networks struggle with word prediction
2:36 RNN Structure
4:01 How do numbers remember anything?
5:22 Visualizing the second layer weights as regions
7:09 Visualizing the first layer weights as vectors
11:00 Implementing the second layer weights in the spreadsheet
13:35 Computing the network using built-in spreadsheet functions
17:39 Experimenting to find the first layer weights
19:34 Completed spreadsheet
19:51 How are real networks trained?
21:07 Final recap

Finished Sheet:
https://bit.ly/RNNSheet2

Image credits:
 - https://commons.wikimedia.org/wiki/File:School_in_Laos_-_Reading_time.jpg
 
Popcorn Reading:
 - https://classroom.synonym.com/popcorn-reading-7349968.html

Full playlist of various Neural Networks in Spreadsheets: https://www.youtube.com/watch?v=fjfZZ6S1ad4&list=PLDs5gxU1J5CIt5kIiQLC3RBOAKnUAo412