SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The authors would like to thank Murat Erdogdu, Jalaj Upadhyay, and Mohammad Yaghini for helpful discussions. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute www.vectorinstitute.ai/partners.
[ABH17] N. Agarwal, B. Bullins, and E. Hazan. "Second-order stochastic optimization for machine learning in linear time". The Journal of Machine Learning Research 18.1 (2017), pp. 4148-4187.
[ABL21] M. Avella-Medina, C. Bradshaw, and P.-L. Loh. "Differentially private inference via noisy optimization". arXiv preprint arXiv:2103.11003 (2021).
[ACGM+16] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, et al. “Deep learning with differential privacy”. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016, pp. 308-318.
[AMN20] J. Arbel, O. Marchal, and H. D. Nguyen. “On strict sub-Gaussianity, optimal proxy variance and symmetry for bounded random variables”. *ESAIM: Probability and Statistics* 24 (2020), pp. 39–55.
[Bac10] F. Bach. "Self-concordant analysis for logistic regression". Electronic Journal of Statistics 4 (2010), pp. 384-414.
[Bac14] F. Bach. "Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression". The Journal of Machine Learning Research 15.1 (2014), pp. 595-627.
[BD99] J. A. Blackard and D. J. Dean. "Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables". Computers and electronics in agriculture 24.3 (1999), pp. 131-151.
[BFTG19] R. Bassily, V. Feldman, K. Talwar, and A. Guha Thakurta. "Private stochastic convex optimization with optimal rates". Advances in neural information processing systems 32 (2019).
[BS16] M. Bun and T. Steinke. "Concentrated differential privacy: Simplifications, extensions, and lower bounds". In: Theory of Cryptography Conference. Springer. 2016, pp. 635-658.
[BST14] R. Bassily, A. Smith, and A. Thakurta. "Private empirical risk minimization: Efficient algorithms and tight error bounds". In: 2014 IEEE 55th annual symposium on foundations of computer science. IEEE. 2014, pp. 464-473.
[ BV04] S. P. Boyd and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.
[CJB04] R. Caruana, T. Joachims, and L. Backstrom. "KDD-Cup 2004: results and analysis". ACM SIGKDD Explorations Newsletter 6.2 (2004), pp. 95-108.
[CMS11] K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. "Differentially Private Empirical Risk Minimization". Journal of Machine Learning Research 12.29 (2011), pp. 1069-1109.
[DG17] D. Dua and C. Graff. UCI Machine Learning Repository. 2017.
[DMNS06] C. Dwork, F. McSherry, K. Nissim, and A. Smith. "Calibrating noise to sensitivity in private data analysis". In: Theory of cryptography conference. Springer. 2006, pp. 265-284.
[DR16] C. Dwork and G. N. Rothblum. “Concentrated differential privacy”. arXiv preprint arXiv:1603.01887 (2016).
[EM15] M. A. Erdogdu and A. Montanari. "Convergence rates of sub-sampled Newton methods". Advances in Neural Information Processing Systems 28 (2015).
[NJLS09] A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. "Robust Stochastic Approximation Approach to Stochastic Programming". SIAM Journal on Optimization 19.4 (2009), pp. 1574-1609. eprint: https://doi.org/10.1137/070704277.
[NP06] Y. Nesterov and B. T. Polyak. "Cubic regularization of Newton method and its global performance". Mathematical Programming 108.1 (2006), pp. 177-205.
[NW99] J. Nocedal and S. J. Wright. Numerical optimization. Springer, 1999.
[PHKX+23] N. Ponomareva, H. Hazimeh, A. Kurakin, Z. Xu, et al. "How to dp-fy ml: A practical guide to machine learning with differential privacy". arXiv preprint arXiv:2303.00654 (2023).
[PS22] N. Papernot and T. Steinke. “Hyperparameter Tuning with Renyi Differential Privacy”. In: International Conference on Learning Representations. 2022.
[RM16] F. Roosta-Khorasani and M. W. Mahoney. "Sub-sampled newton methods ii: Local convergence rates". arXiv preprint arXiv:1601.04738 (2016).
[RM19] F. Roosta-Khorasani and M. W. Mahoney. "Sub-sampled Newton methods". Mathematical Programming 174 (2019), pp. 293-326.
[SCS13] S. Song, K. Chaudhuri, and A. D. Sarwate. "Stochastic gradient descent with differentially private updates". In: 2013 IEEE global conference on signal and information processing. IEEE. 2013, pp. 245-248.
[SSTT21] S. Song, T. Steinke, O. Thakkar, and A. Thakurta. "Evading the curse of dimensionality in unconstrained private glms". In: International Conference on Artificial Intelligence and Statistics. PMLR. 2021, pp. 2638-2646.
[Ste98] G. W. Stewart. Matrix algorithms: volume 1: basic decompositions. SIAM, 1998.
[STT20] S. Song, O. Thakkar, and A. Thakurta. "Characterizing Private Clipped Gradient Descent on Convex Generalized Linear Problems". arXiv preprint arXiv:2006.06783 (2020).
[STU17] A. Smith, A. Thakurta, and J. Upadhyay. "Is interaction necessary for distributed private learning?" In: 2017 IEEE Symposium on Security and Privacy (SP). IEEE. 2017, pp. 58-77.
[VZ22] S. Vadhan and W. Zhang. "Concurrent Composition Theorems for Differential Privacy". arXiv preprint arXiv:2207.08335 (2022).
[WLKC+17] X. Wu, F. Li, A. Kumar, K. Chaudhuri, et al. “Bolt-on Differential Privacy for Scalable Stochastic Gradient Descent-based Analytics”. In: Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD. Ed. by S. Salihoglu, W. Zhou, R. Chirkova, J. Yang, and D. Suciu. 2017.
[XRV17] H. Xiao, K. Rasul, and R. Vollgraf. "Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms". arXiv preprint arXiv:1708.07747 (2017).
[XYRRM16] P. Xu, J. Yang, F. Roosta, C. Ré, and M. W. Mahoney. "Sub-sampled Newton methods with non-uniform sampling". Advances in Neural Information Processing Systems 29 (2016).
[ZLNM+19] G. Zhang, L. Li, Z. Nado, J. Martens, et al. "Which algorithmic choices matter at which batch sizes? insights from a noisy quadratic model". Advances in neural information processing systems 32 (2019).
[ZZMW17] J. Zhang, K. Zheng, W. Mou, and L. Wang. "Efficient private ERM for smooth objectives". In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, pp. 3922-3928.