機械学習・ニューラルネットワークなどの年表

趣味でまとめている年表です．参考になれば幸いです．気が向いたときに元論文へのリンクなども追加しています．項目の分け方は著者の趣味です．

大規模言語モデルは別の表にしています．

## 機械学習，ニューラルネットワークなどの年表

| 年      | 機械学習                                | ニューラルネットワーク                              | その他                                      |
|:--------|:----------------------------------------|:----------------------------------------------------|:--------------------------------------------|
| 1700s   | ベイズ定理(Bayes)                       |                                                     |                                             |
| 1795    | 最小二乗法 (Gauss)                      |                                                     |                                             |
| 1865    |                                         |                                                     | Clausius' entropy (Clausius)                |
| 1870s   |                                         |                                                     | Boltzmann's entropy (Boltzmann)             |
| 1873    |                                         | ゴルジ染色 (Golgi)                                  |                                             |
| 1888    |                                         | Neuron doctrine (Cajal)                             |                                             |
| 1901    | PCA (Pearson)                           |                                                     |                                             |
| 1905    | [Random walk (Pearson)][random_walk]    |                                                     | ブラウン運動 (Einstein)                     |
| 1906    |                                         | Receptive field (Sherrington)                       |                                             |
| 1912-22 | 最尤推定 (Fisher)                       |                                                     |                                             |
| 1920    |                                         |                                                     | Ising model (Lenz)                          |
| 1938    |                                         | [Receptive field (Hartline)][receptive_field]       |                                             |
|         |                                         |                                                     | Turing's BombeがEnigma解読                  |
| 1943    |                                         | Neural network (Neuron) model (McCulloch and Pitts) |                                             |
| 1946    |                                         |                                                     | ENIAC完成・使用開始                         |
| 1948    | Shannon's entropy (Shannon)             | [Jeffress model (Jeffress)][jeffress]               |                                             |
| 1949    |                                         | Hebbian Learning (Hebb)                             |                                             |
| 1950s   |                                         |                                                     | Checker (Samuel)                            |
| 1950    |                                         |                                                     | [Turing test][turingtest]                   |
| 1951    |                                         | SNARC迷路を解く (Minsky and Edmons)                 |                                             |
| 1952    |                                         | Hodgkin Huxley model (Hodgkin and Huxley)           |                                             |
| 1956    |                                         |                                                     | Artificial Intelligence (Dartmouth meeting) |
| 1957    | Bellman equation (Bellman)              | Perceptron (Rosenblatt)                             |                                             |
| 1957    | k-means published in 1980 (Lloyd)       |                                                     |                                             |
| 1959    | Machine learning (Samuel)               | V1 (Hubel and Wiesel)                               |                                             |
| 1960    |                                         | Delta rule (Widrow and Hoff)                        |                                             |
| 1966    | Hidden Markov Model (Baum and Petrie)   |                                                     | Chatbot Eliza (Weizenbaum)                  |
| 1967    | (Online) k-means (MacQueen)             |                                                     |                                             |
| 1969    |                                         | Perceptrons (Minski and Papert)                     |                                             |
|         |                                         | Rectified linear (Fukushima)                        |                                             |
| 1971    |                                         | a deep GMDH network with 8 layers (Ivakhnenko)      |                                             |
| 1973    |                                         | Self organizing map (SOM) (von der Malsberg)        |                                             |
| 1977    | EM Algorithm (Dempster et al.)          |                                                     |                                             |
| 1980    |                                         | Neocognitron (Fukushima)                            | PacMan                                      |
| 1980    |                                         | SOM (Amari)                                         | MS-DOS                                      |
| 1982    |                                         | Hopfield Network (Hopfield)                         |                                             |
|         |                                         | Vision (Marr)                                       |                                             |
| 1983    |                                         | SOM (Kohonen)                                       |                                             |
| 1986    |                                         | Backpropagation (Rumelhart, Hinton, Williams)       |                                             |
|         |                                         | Boltzmann Machine (Ackley, Hinton, Seinowski)       |                                             |
| 1988    |                                         | Autoencoder (Baldi and Hornik)                      |                                             |
|         |                                         | Autoencoder (Bourland and Lecun)                    |                                             |
| 1989    |                                         | LeNet (LeCun)                                       |                                             |
| 1989    |                                         | Neurogammon (Tesauro)                               |                                             |
| 1994    | ICA (Pierre)                            |                                                     |                                             |
| 1995    | Positive Matrix Factorization (Paatero) | TD-gammon (Tesauro)                                 |                                             |
|         | SVM (Vapnik)                            |                                                     |                                             |
| 1996    |                                         |                                                     |                                             |
|         |                                         | [Sparse Coding (Olshausen)][sparse_coding]          |                                             |
| 1997    |                                         | [LSTM (Hochreiter and Schmidhuber)][lstm]           | Deep Blue beat Kasparov.                    |
|         |                                         |                                                     | Logistello beat Murakami.                   |
| 1998    | Kernel PCA (Scholkopf et al.)           |                                                     |                                             |
|         | [Quantum annealing][quantum_annealing]  |                                                     |                                             |
| 1999    | Non-Negative Factorization (Lee)        |                                                     | Aibo (Sony)                                 |
| 2001    | Topographic ICA (Hyvarinen et al.)      | [Echo state network (Jaeger)][echo_state]           |                                             |
| 2002    |                                         | [Liquid state machine (Maass)][liquid_state]        | Roomba (iRobot)                             |
| 2003    |                                         | [Izhikevich model (Izhikevich)][izhikevich]         |                                             |
| 2006    | IVA (Kim et al.)                        | Deep Belief Network (Hinton and Salakhutdinov)      |                                             |
|         | Monte Calro Tree Search (Coulom)        |                                                     |                                             |
| 2007    |                                         | Stacked Autoencoder (Bengio et al.)                 |                                             |
| 2010    |                                         | Rectufied linear (Nair et al.)                      |                                             |
| 2012    |                                         | AlexNet (Krizhevsky, Sutskever, Hinton)             |                                             |
|         |                                         | [Dropout (Hinton et al.)][dropout]                  |                                             |
| 2013    |                                         | [Variational Autoencoder (Kingma and Welling)][VAE] |                                             |
| 2014    |                                         | [GoogLeNet (Szegedy et al.)][googlenet]             |                                             |
|         |                                         | [VGG (Simonyan and Zisserman)][vgg]                 |                                             |
| 2015    |                                         | ResNet (He et al.)                                  |                                             |
|         |                                         | Deep Q network (Mnih et al.)                        |                                             |
| 2016    |                                         | Xception (Chollet)                                  |                                             |
|         |                                         | [AlphaGo (Silver et al.)][alphago]                  |                                             |
| 2017    |                                         | Squeeze-and-Excitation Networks (Hu et al.)         |                                             |
| 2017    |                                         | [Transformer (Vaswani et al.)][transformer]         |                                             |
|         |                                         | [AlphaGo Zero (Silver et al.)][alphagozero]         |                                             |
| 2018    |                                         | [AlphaZero (Silver et al.)][alphazero]              |                                             |
|         |                                         | R2D2                                                |                                             |
|         |                                         | [BERT][bert]                                        |                                             |
|         |                                         | AlphaFold 1                                         |                                             |
| 2019    |                                         | EfficientNet (Tan and Le)                           |                                             |
|         |                                         | AlphaStar                                           |                                             |
|         |                                         | [MuZero (Schrittwieser et al.)][muzero]             |                                             |
| 2020    |                                         | [Vision Transformer][vit]                           |                                             |
|         |                                         | [Diffusion model (Ho et al.)][dif]                  |                                             |
| 2021    |                                         | [EfficientNetV2 (Tan and Le)][efficientnet2]        |                                             |
|         |                                         | [MLP-Mixer][mlp-mixer]                              | NovelAI (Anlatan)                           |
|         |                                         | [Latent Diffusion][latent]                          |                                             |
| 2022    |                                         | [Gato][gato]                                        | DALL-E 2 (OpenAI)                           |
|         |                                         |                                                     | Midjourney (Midjourney)                     |
|         |                                         |                                                     | Stable Diffusion                            |
|         |                                         |                                                     | ChatGPT (OpenAI)                            |
| 2023    |                                         | RT2      (Google)                                   |                                             |
|         |                                         |                                                     | Gemini   (Google)                           |
| 2024    |                                         |                                                     | Copilotキー搭載を発表 (Microsoft)           |
|         |                                         |                                                     | Apple Intelligenceを発表 (Apple)            |
|         |                                         |                                                     |                                             |


## 大規模言語モデル

| 年月    | LLM                        |
|:--------|:---------------------------|
| 2018/6  | GPT (OpenAI)               |
| 2018/10 | BERT (Google)              |
| 2019/2  | GPT-2 (OpenAI)             |
| 2020/5  | GPT-3 (OpenAI)             |
| 2021/12 | Claude (Anthropic)         |
| 2022/1  | LaMDA (Google)             |
| 2022/4  | PaLM (Google)              |
| 2022/3  | GPT-3.5 (OpenAI)           |
| 2023/2  | LLaMA   (Meta)             |
| 2023/3  | GPT-4   (OpenAI)           |
| 2023/5  | PaLM (Google)              |
| 2023/7  | LLama2   (Meta)            |
|         | Claude2  (Anthropic)       |
| 2023/11 | Claude 2.1 (Anthropic)     |
|         | Grok-1 (x.AI)              |
| 2023/12 | Gemini 1.0 (Google)        |
|         | Mixtral 8x7B (Mistral AI)  |
|         | Phi-2 (Microsof)           |
| 2024/2  | Gemini 1.5 (Google)        |
|         | Gemma (Google)             |
| 2024/3  | Claude3 (Anthropic)        |
| 2024/4  | LLama3 (Meta)              |
|         | Mixtral 8x22B (Mistral AI) |
|         | Phi3 (Microsof)            |
| 2024/5  | Deepseek V2 (DeepSeek)     |
| 2024/6  | Qwen2 (Alibaba)            |
|         | Nemotron-4 (NVIDIA)        |
| 2024/7  | LLama3.1 (Meta)            |
|         | Gemma2 (Google)            |
|2025/1   |DeepSeek R1 (DeepSeek)      |


[jeffress]:https://psycnet.apa.org/doiLanding?doi=10.1037%2Fh0061495
[turingtest]:https://academic.oup.com/mind/article/LIX/236/433/986238
[latent]:https://arxiv.org/abs/2105.01601
[mlp-mixer]:https://arxiv.org/abs/2105.01601
[vit]:https://arxiv.org/abs/2010.11929
[dropout]:https://arxiv.org/abs/1207.0580
[receptive_field]:https://doi.org/10.1152/ajplegacy.1938.121.2.400
[sparse_coding]:https://doi.org/10.1038/381607a0
[alphago]:https://doi.org/10.1038/nature16961
[alphagozero]:https://doi.org/10.1038/nature24270
[alphazero]:https://doi.org/10.1126/science.aar6404
[random_walk]:https://doi.org/10.1038/072294b0
[lstm]:https://doi.org/10.1162/neco.1997.9.8.1735
[echo_state]:https://www.ai.rug.nl/minds/uploads/EchoStatesTechRep.pdf
[liquid_state]:https://doi.org/10.1016/j.jcss.2004.04.001
[quantum_annealing]:https://doi.org/10.1103/PhysRevE.58.5355
[izhikevich]:https://doi.org/10.1109/TNN.2003.820440
[googlenet]:https://arxiv.org/abs/1409.4842
[vgg]:https://arxiv.org/abs/1409.1556
[fep]:https://www.fil.ion.ucl.ac.uk/~karl/A%20free%20energy%20principle%20for%20the%20brain.pdf
[muzero]:https://arxiv.org/abs/1911.08265
[efficientnet2]:https://arxiv.org/abs/2104.00298
[gan]:https://arxiv.org/abs/1406.2661
[transformer]:https://arxiv.org/abs/1706.03762
[VAE]:https://arxiv.org/abs/1312.6114
[dif]:https://arxiv.org/abs/2006.11239
[gato]:https://openreview.net/forum?id=1ikK0kHjvj
[bert]:https://arxiv.org/abs/1810.04805