Kearns, M. (1998). Efficient noise-tolerant learning from statistical queries. Journal of the ACM (JACM), 45(6):983-1006.
Kol, G., Raz, R., and Tal, A. (2017). Time-space hardness of learning sparse parities. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pages 1067-1080.
Liu, Z., Michaud, E. J., and Tegmark, M. (2022). Omnigrok: Groking beyond algorithmic data. arXiv preprint arXiv:2210.01117.
Lutz, P., Arnould, L., Boyer, C., and Scornet, E. (2022). Sparse tree-based initialization for neural networks. arXiv preprint arXiv:2209.15283.
Malach, E., Kamath, P., Abbe, E., and Srebro, N. (2021). Quantifying the benefit of using differentiable learning over tangent kernels. In International Conference on Machine Learning, pages 7379-7389. PMLR.
Merrill, W., Tsilivis, N., and Shukla, A. (2023). A tale of two circuits: Grokking as competition of sparse and dense subnetworks. arXiv preprint arXiv:2303.11873.
Michaud, E. J., Liu, Z., Girit, U., and Tegmark, M. (2023). The quantization model of neural scaling. arXiv preprint arXiv:2303.13506.
Minsky, M. and Papert, S. (1969). Perceptrons: an introduction to computational geometry. MIT Press.
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., and Sutskever, I. (2021). Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021(12):124003.
O'Donnell, R. (2014). Analysis of Boolean functions. Cambridge University Press.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830.
Power, A., Burda, Y., Edwards, H., Babuschkin, I., and Misra, V. (2022). Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprint arXiv:2201.02177.
Rosen-Zvi, M., Klein, E., Kanter, I., and Kinzel, W. (2002). Mutual learning in a tree parity machine and its application to cryptography. Physical Review E, 66(6):066135.
Shalev-Shwartz, S. and Ben-David, S. (2014). Understanding machine learning: From theory to algorithms. Cambridge university press.
Shalev-Shwartz, S., Shamir, O., and Shammah, S. (2017). Failures of gradient-based deep learning. In International Conference on Machine Learning, pages 3067-3075. PMLR.
Shamir, O. and Zhang, T. (2013). Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes. In International conference on machine learning, pages 71-79. PMLR.
Shi, Z., Wei, J., and Liang, Y. (2021). A theoretical analysis on feature learning in neural networks: Emergence from inputs and advantage over fixed features. In International Conference on Learning Representations.
Telgarsky, M. (2022). Feature selection with gradient descent on two-layer networks in low-rotation regimes. arXiv preprint arXiv:2208.02789.