深度学习
参考资料

↑  Deng, L.; Yu, D.  (PDF). Foundations and Trends in Signal Processing. 2014, 7: 3–4  [2015-10-23]. （原始内容存档 (PDF)于2016-03-14）.
1 2  Bengio, Yoshua.  (PDF). Foundations and Trends in Machine Learning. 2009, 2 (1): 1–127. （原始内容 (PDF)存档于2016-03-04）.
1 2 3 4  Bengio, Y.; Courville, A.; Vincent, P. . IEEE Transactions on Pattern Analysis and Machine Intelligence. 2013, 35 (8): 1798–1828. arXiv:1206.5538 .
↑  Schmidhuber, J. . Neural Networks. 2015, 61: 85–117. arXiv:1404.7828 . doi:10.1016/j.neunet.2014.09.003.
↑  Bengio, Yoshua; LeCun, Yann; Hinton, Geoffrey. . Nature. 2015, 521: 436–444.
↑  Glauner, P.  (学位论文). Imperial College London, Department of Computing. 2015. arXiv:1508.06535 .
↑  Song, H.A.; Lee, S. Y. . . Lectures Notes in Computer Sciences 8226. Springer Berlin Heidelberg. 2013: 466–473. ISBN 978-3-642-42053-5. doi:10.1007/978-3-642-42054-2_58.
↑  Olshausen, B. A. . Nature. 1996, 381 (6583): 607–609.
↑  Collobert, R. . VideoLectures.net. April 2011.  事件发生在 7min 45s  [2015-10-31]. （原始内容存档于2020-10-19）.
↑  Gomes, L. . IEEE Spectrum. 20 October 2014  [2015-10-31]. （原始内容存档于2019-12-12）.
↑  K. Fukushima., "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position," Biol. Cybern., 36, 193–202, 1980
↑  P. Werbos., "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences," PhD thesis, Harvard University, 1974.
↑  LeCun et al., "Backpropagation Applied to Handwritten Zip Code Recognition," Neural Computation, 1, pp. 541–551, 1989.
↑  S. Hochreiter., "Untersuchungen zu dynamischen neuronalen Netzen," Diploma thesis. Institut f. Informatik, Technische Univ. Munich. Advisor: J. Schmidhuber, 1991.
↑  S. Hochreiter et al., "Gradient flow in recurrent nets: the difficulty of learning long-term dependencies," In S. C. Kremer and J. F. Kolen, editors, A Field Guide to Dynamical Recurrent Neural Networks. IEEE Press, 2001.
↑  J. Weng, N. Ahuja and T. S. Huang, "Cresceptron: a self-organizing neural network which grows adaptively （页面存档备份，存于）," Proc. International Joint Conference on Neural Networks, Baltimore, Maryland, vol I, pp. 576-581, June, 1992.
↑  J. Weng, N. Ahuja and T. S. Huang, "Learning recognition and segmentation of 3-D objects from 2-D images （页面存档备份，存于）," Proc. 4th International Conf. Computer Vision, Berlin, Germany, pp. 121-128, May, 1993.
↑  J. Weng, N. Ahuja and T. S. Huang, "Learning recognition and segmentation using the Cresceptron （页面存档备份，存于）," International Journal of Computer Vision, vol. 25, no. 2, pp. 105-139, Nov. 1997.
↑  G. E. Hinton., "Learning multiple layers of representation," Trends in Cognitive Sciences, 11, pp. 428–434, 2007.
1 2  J. Schmidhuber., "Learning complex, extended sequences using the principle of history compression," Neural Computation, 4, pp. 234–242, 1992.
↑  J. Schmidhuber., "My First Deep Learning System of 1991 + Deep Learning Timeline 1962–2013."
1 2  D. C. Ciresan et al., "Deep Big Simple Neural Nets for Handwritten Digit Recognition," Neural Computation, 22, pp. 3207–3220, 2010.
↑  R. Raina, A. Madhavan, A. Ng., "Large-scale Deep Unsupervised Learning using Graphics Processors," Proc. 26th Int. Conf. on Machine Learning, 2009.
↑  J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur and E. Thelen,  "Autonomous Mental Development by Robots and Animals （页面存档备份，存于）," Science, vol. 291, no. 5504, pp. 599 - 600, Jan. 26, 2001.
↑  J. Weng,  "Brains as Naturally Emerging Turing Machines （页面存档备份，存于）," in Proc. International Joint Conference on Neural Networks, Killarney, Ireland, 8 pages, July 12-17. 2015.
↑  M Riesenhuber, T Poggio. Hierarchical models of object recognition in cortex. Nature neuroscience, 1999(11) 1019–1025.
↑  Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackel. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1(4):541–551, 1989.
↑  S. Hochreiter. Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis, Institut f. Informatik, Technische Univ. Munich, 1991. Advisor: J. Schmidhuber
↑  S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In S. C. Kremer and J. F. Kolen, editors, A Field Guide to Dynamical Recurrent Neural Networks. IEEE Press, 2001.
↑  Hochreiter, Sepp; and Schmidhuber, Jürgen; Long Short-Term Memory, Neural Computation, 9(8):1735–1780, 1997
↑  Graves, Alex; and Schmidhuber, Jürgen; Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris K. I.; and Culotta, Aron (eds.), Advances in Neural Information Processing Systems 22 (NIPS'22), December 7th–10th, 2009, Vancouver, BC, Neural Information Processing Systems (NIPS) Foundation, 2009, pp. 545–552
↑  A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, J. Schmidhuber. A Novel Connectionist System for Improved Unconstrained Handwriting Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence,  vol. 31, no. 5, 2009.
↑  Sven Behnke.  (PDF). Lecture Notes in Computer Science 2766. Springer. 2003  [2014-09-16]. （原始内容存档 (PDF)于2021-03-08）.
↑  
Smolensky, P.  1. 1986: 194–281.  |journal=被忽略 (帮助)
↑  Hinton, G. E.; Osindero, S.; Teh, Y.  (PDF). Neural Computation. 2006, 18 (7): 1527–1554  [2014-09-16]. PMID 16764513. doi:10.1162/neco.2006.18.7.1527. （原始内容存档 (PDF)于2015-12-23）.
↑  Geoffrey Hinton. . Scholarpedia. 2009-05-31, 4 (5)  [2018-04-02]. ISSN 1941-6016. doi:10.4249/scholarpedia.5947. （原始内容存档于2015-12-04） （英语）.
↑  John Markoff. . New York Times. 25 June 2012  [2014-09-16]. （原始内容存档于2018-07-11）.
↑  Ng, Andrew; Dean, Jeff.  (PDF). 2012  [2014-09-16]. （原始内容存档 (PDF)于2017-10-12）.
1 2  D. C. Ciresan, U. Meier, J. Masci, L. M. Gambardella, J. Schmidhuber. Flexible, High Performance Convolutional Neural Networks for Image Classification. International Joint Conference on Artificial Intelligence (IJCAI-2011, Barcelona), 2011.
↑  Martines, H., Bengio, Y., & Yannakakis, G. N. (2013). Learning Deep Physiological Models of Affect. I EEE Computational Intelligence, 8(2), 20.
↑  D. C. Ciresan, U. Meier, J. Masci, J. Schmidhuber. Multi-Column Deep Neural Network for Traffic Sign Classification. Neural Networks, 2012.
↑  D. C. Ciresan, U. Meier, J. Schmidhuber. Multi-column Deep Neural Networks for Image Classification. IEEE Conf. on Computer Vision and Pattern Recognition CVPR 2012.
↑  T. Mikolov et al., "Recurrent neural network based language model," Interspeech, 2010.
↑  Y. LeCun et al., "Gradient-based learning applied to document recognition," Proceedings of the IEEE, 86 (11), pp. 2278–2324.
↑  T. Sainath et al., "Convolutional neural networks for LVCSR," ICASSP, 2013.
↑  G. E. Hinton et al., "Deep Neural Networks for Acoustic Modeling in Speech Recognition: The shared views of four research groups," IEEE Signal Processing Magazine, pp. 82–97, November 2012.
↑  Y. Bengio et al., "Advances in optimizing recurrent networks," ICASSP', 2013.
↑  G. Dahl et al., "Improving DNNs for LVCSR using rectified linear units and dropout," ICASSP', 2013.
1 2 3  G. E. Hinton., "A Practical Guide to Training Restricted Boltzmann Machines," Tech. Rep. UTML TR 2010-003, Dept. CS., Univ. of Toronto, 2010.
↑  G.E. Hinton., "Deep belief networks," Scholarpedia, 4(5):5947.
↑  H. Larochelle et al., "An empirical evaluation of deep architectures on problems with many factors of variation," in Proc. 24th Int. Conf. Machine Learning, pp. 473–480, 2007.
↑  .   [2014-09-16]. （原始内容存档于2020-10-29）.
↑  Honglak Lee; Roger Grosse; Rajesh Ranganath; Andrew Y. Ng. . ICML '09. 2009: 609–616.
↑  TIMIT Acoustic-Phonetic Continuous Speech Corpus
Linguistic Data Consortium, Philadelphia.
↑  http://yann.lecun.com/exdb/mnist/ （页面存档备份，存于）.
↑  D. Ciresan, U. Meier, J. Schmidhuber., "Multi-column Deep Neural Networks for Image Classification," Technical Report No. IDSIA-04-12', 2012.
↑  P. E. Utgoff and D. J. Stracuzzi., "Many-layered learning," Neural Computation, 14, pp. 2497–2529, 2002.
↑  J. Elman, et al., "Rethinking Innateness," 1996.
↑  J. Shrager, MH Johnson., "Dynamic plasticity influences the emergence of function in a simple cortical array," Neural Networks, 9 (7), pp. 1119–1129, 1996
↑  SR Quartz and TJ Sejnowski., "The neural basis of cognitive development: A constructivist manifesto," Behavioral and Brain Sciences, 20 (4), pp. 537–556, 1997.
↑  S. Blakeslee., "In brain's early growth, timetable may be critical," The New York Times, Science Section, pp. B5–B6, 1995.
↑  {BUFILL} E. Bufill, J. Agusti, R. Blesa., "Human neoteny revisited: The case of synaptic plasticity," American Journal of Human Biology, 23 (6), pp. 729–739, 2011.
↑  J. Shrager and M. H. Johnson., "Timing in the development of cortical function: A computational approach," In B. Julesz and I. Kovacs (Eds.), Maturational windows and adult cortical plasticity, 1995.
↑  D. Hernandez., "The Man Behind the Google Brain: Andrew Ng and the Quest for the New AI （页面存档备份，存于）," Wired, 10 May 2013.
↑  C. Metz., "Facebook's 'Deep Learning' Guru Reveals the Future of AI （页面存档备份，存于）," Wired, 12 December 2013.
↑  . CSDN. 2013-03-13  [2014-07-20]. （原始内容存档于2020-03-24）.
↑  G. Marcus., "Is "Deep Learning" a Revolution in Artificial Intelligence?" The New Yorker, 25 November 2012.
↑  .   [2017-07-21]. （原始内容存档于2020-12-08）.
↑  Caffe （页面存档备份，存于）
↑  .   [2018-05-02]. （原始内容存档于2020-11-28）.
↑  .   [2017-03-17]. （原始内容存档于2017-08-18）.
一个包含完全连接可见层和隐层的受限玻尔兹曼机（RBM）。注意到可见层单元和隐层单元内部彼此不相连。