SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Another line of work focuses on the generalization of neural network that are trained with Stochastic Gradient Descent (SGD) or its noisy variant Stochastic Gradient Langevin Descent (SGLD) (Russo and Zou, 2016; Xu and Raginsky, 2017; Russo and Zou, 2019; Steinke and Zakynthinou, 2020; Raginsky et al., 2017; Haghifam et al., 2020; Neu et al., 2021). Zhao et al. (2020) analyze the memory properties of recurrent networks and how well they can remember the input sequence. Tu et al. (2020) study the generalization of RNN by analyzing the Fisher-Rao norm of weights, which they obtain from the gradients of the network. They offer generalization bounds that can potentially become polynomial in  $T$ . Allen-Zhu and Li (2019) analyze the change in output through the dynamics of training RNNs and prove generalization bounds for recurrent networks that are again polynomial in  $T$ .
Zeyuan Allen-Zhu and Yanzhi Li. Can sgd learn recurrent neural networks with provable generalization? Advances in Neural Information Processing Systems, 32, 2019.
Martin Anthony, Peter L Bartlett, Peter L Bartlett, et al. Neural network learning: Theoretical foundations, volume 9. Cambridge university press Cambridge, 1999.
Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. Stronger generalization bounds for deep nets via a compression approach. In International Conference on Machine Learning, pages 254-263. PMLR, 2018.
Sanjeev Arora, Nadav Cohen, Wei Hu, and Yuping Luo. Implicit regularization in deep matrix factorization. Advances in Neural Information Processing Systems, 32, 2019.
Peter Bartlett. For valid generalization the size of the weights is more important than the size of the network. Advances in neural information processing systems, 9, 1996.
Peter Bartlett, Vitaly Maiorov, and Ron Meir. Almost linear vc dimension bounds for piecewise polynomial networks. Advances in neural information processing systems, 11, 1998.