M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems, Jan. 2015. URL https://www.tensorflow.org/. Software available from tensorflow.org. 29
N. Ahmed and M. Wahed. The de-democratization of AI: Deep learning and the compute divide in artificial intelligence research, 2020. URL http://arxiv.org/abs/2010.15581
W. B. Arthur. Increasing Returns and Path Dependence in the Economy. University of Michigan Press, 1994. ISBN 978-0-472-09496-7. doi: 10.3998/mpub.10029. URL https://www.jstor.org/stable/10.3998/mpub.10029. 10
W. B. Arthur. The Nature of Technology: What It Is and How It Evolves. Free Press, reprint edition, 2011. ISBN 978-1-4165-4406-7. [10]
P. Barham and M. Isard. Machine learning systems are stuck in a rut. In Proceedings of the Workshop on Hot Topics in Operating Systems, HotOS '19, page 177-183, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450367271. doi: 10.1145/3317550.3321441. URL https://doi.org/10.1145/3317550.3321441. [1] [9] [14]
P. Barham, A. Chowdhery, J. Dean, S. Ghemawat, S. Hand, D. Hurt, M. Isard, H. Lim, R. Pang, S. Roy, B. Saeta, P. Schuh, R. Sepassi, L. E. Shafey, C. A. Thekkath, and Y. Wu. Pathways: Asynchronous distributed dataflow for ml, 2022. [14]
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.29
T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. CoRR, abs/1512.01274, 2015. URL http://arxiv.org/abs/1512.01274.9
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-Ari, P. Yin, T. Duke, A. Levskaya, S. Ghemawat, S. Dev, H. Michalewski, X. Garcia, V. Misra, K. Robinson, L. Fedus, D. Zhou, D. Ippolito, D. Luan, H. Lim, B. Zoph, A. Spiridonov, R. Sepassi, D. Dohan, S. Agrawal, M. Omernick, A. M. Dai, T. S. Pillai, M. Pellat, A. Lewkowycz, E. Moreira, R. Child, O. Polozov, K. Lee, Z. Zhou, X. Wang, B. Saeta, M. Diaz, O. Firat, M. Catasta, J. Wei, K. Meier-Hellstern, D. Eck, J. Dean, S. Petrov, and N. Fiedel. Palm: Scaling language modeling with pathways, 2022. URL https://arxiv.org/abs/2204.02311
P. David, D. Foray, and P. Aghion. Science, technology and innovation for economic growth: Linking policy research and practice in "stig systems". Research Policy, 38:681-693, 05 2009. doi: 10.2139/ssrn.1285612. 9 10
P. A. David. Clio and the economics of QWERTY. The American Economic Review, 75(2):332-337, 1985. ISSN 0002-8282. URL https://www.jstor.org/stable/1805621. Publisher: American Economic Association. [10]
K. M. Eisenhardt and B. N. Tabrizi. Accelerating adaptive processes: Product innovation in the global computer industry. Administrative Science Quarterly, 40(1):84-110, 1995. ISSN 00018392. URL http://www.jstor.org/stable/2393701. [14]
J. L. Hennessy and D. A. Patterson. A new golden age for computer architecture. Commun. ACM, 62(2):48-60, jan 2019a. ISSN 0001-0782. doi: 10.1145/3282307. URL https://doi.org/10.1145/3282307. 2