# The 27 papers

List of 27 papers (*supposedly*) given to John Carmack by Ilya Sutskever: "If you really learn all of these, you’ll know 90% of what matters today."   
The list has been floating around for a few weeks on [Twitter](https://x.com/keshavchan/status/1787861946173186062)/[LinkedIn](https://www.linkedin.com/pulse/27-papers-read-know-90-ai-ilia-sutskever-gave-john-carmack-bullock-rqyte). I figure some might have missed it so here you go.  
Regardless of the veracity of the tale I am still finding it valuable.  
  
[https://punkx.org/jackdoe/30.html](https://punkx.org/jackdoe/30.html)

1.  [The Annotated Transformer](https://nlp.seas.harvard.edu/annotated-transformer/) (nlp.seas.harvard.edu)
2.  [The First Law of Complexodynamics](https://scottaaronson.blog/?p=762) (scottaaronson.blog)
3.  [The Unreasonable Effectiveness of RNNs](https://karpathy.github.io/2015/05/21/rnn-effectiveness/) (karpathy.github.io)
4.  [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/) (colah.github.io)
5.  [Recurrent Neural Network Regularization](https://arxiv.org/pdf/1409.2329.pdf) (arxiv.org)
6.  [Keeping Neural Networks Simple by Minimizing the Description Length of the Weights](https://www.cs.toronto.edu/~hinton/absps/colt93.pdf) (cs.toronto.edu)
7.  [Pointer Networks](https://arxiv.org/pdf/1506.03134.pdf) (arxiv.org)
8.  [ImageNet Classification with Deep CNNs](https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf) (proceedings.neurips.cc)
9.  [Order Matters: Sequence to sequence for sets](https://arxiv.org/pdf/1511.06391.pdf) (arxiv.org)
10.  [GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism](https://arxiv.org/pdf/1811.06965.pdf) (arxiv.org)
11.  [Deep Residual Learning for Image Recognition](https://arxiv.org/pdf/1512.03385.pdf) (arxiv.org)
12.  [Multi-Scale Context Aggregation by Dilated Convolutions](https://arxiv.org/pdf/1511.07122.pdf) (arxiv.org)
13.  [Neural Quantum Chemistry](https://arxiv.org/pdf/1704.01212.pdf) (arxiv.org)
14.  [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) (arxiv.org)
15.  [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/pdf/1409.0473.pdf) (arxiv.org)
16.  [Identity Mappings in Deep Residual Networks](https://arxiv.org/pdf/1603.05027.pdf) (arxiv.org)
17.  [A Simple NN Module for Relational Reasoning](https://arxiv.org/pdf/1706.01427.pdf) (arxiv.org)
18.  [Variational Lossy Autoencoder](https://arxiv.org/pdf/1611.02731.pdf) (arxiv.org)
19.  [Relational RNNs](https://arxiv.org/pdf/1806.01822.pdf) (arxiv.org)
20.  [Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton](https://arxiv.org/pdf/1405.6903.pdf) (arxiv.org)
21.  [Neural Turing Machines](https://arxiv.org/pdf/1410.5401.pdf) (arxiv.org)
22.  [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://arxiv.org/pdf/1512.02595.pdf) (arxiv.org)
23.  [Scaling Laws for Neural LMs](https://arxiv.org/pdf/2001.08361.pdf) (arxiv.org)
24.  [A Tutorial Introduction to the Minimum Description Length Principle](https://arxiv.org/pdf/math/0406077.pdf) (arxiv.org)
25.  [Machine Super Intelligence Dissertation](https://www.vetta.org/documents/Machine_Super_Intelligence.pdf) (vetta.org)
26.  [PAGE 434 onwards: Komogrov Complexity](https://www.lirmm.fr/~ashen/kolmbook-eng-scan.pdf) (lirmm.fr)
27.  [CS231n Convolutional Neural Networks for Visual Recognition](https://cs231n.github.io/) (cs231n.github.io)