[18] Jagielski, M., Ullman, J., and Oprea, A. Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems (NeurIPS), 33: 22205-22216, 2020. 14
[19] Kairouz, P., Oh, S., and Viswanath, P. The composition theorem for differential privacy. In Proceedings of the 32nd International Conference on International Conference on Machine Learning (ICML), pp. 1376-1385, 2015. 4
[20] Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Dennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1-2):1-210, 2021. 2, 9, 15
[21] Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009. 8
[22] Lehmann, E. L., Romano, J. P., and Casella, G. Testing statistical hypotheses, volume 3. Springer, 2005. 24
[23] Long, Y., Bindschaedler, V., Wang, L., Bu, D., Wang, X., Tang, H., Gunter, C. A., and Chen, K. Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04889, 2018. 3, 14
[24] Maddock, S., Sablayrolles, A., and Stock, P. Canife: Crafting canaries for empirical privacy measurement in federated learning. In International Conference on Learning Representations (ICLR), 2023. 14
[25] Nasr, M., Shokri, R., and Houmansadr, A. Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 634-646, 2018. 8
[26] Nasr, M., Songi, S., Thakurta, A., Papernot, N., and Carlin, N. Adversary instantiation: Lower bounds for differentially private machine learning. In 2021 IEEE Symposium on security and privacy (SP), pp. 866-882. IEEE, 2021. 1, 2, 14
[27] Nasr, M., Hayes, J., Steinke, T., Balle, B., Tramér, F., Jagielski, M., Carlini, N., and Terzis, A. Tight auditing of differentially private machine learning. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 1631-1648. USENIX Association, 2023. 1, 2, 3, 14
[28] Neyman, J. and Pearson, E. S. Ix. on the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231(694-706):289-337, 1933. 6
[29] Pawelczyk, M., Lakkaraju, H., and Neel, S. On the Privacy Risks of Algorithmic Recourse. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. 1, 3, 9, 14, 15, 17
[30] Sablayrolles, A., Douze, M., Schmid, C., Ollivier, Y., and Jegou, H. White-box vs black-box: Bayes optimal strategies for membership inference. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019. 3, 14
[31] Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017. 1, 2, 3, 4, 14
[32] Shokri, R., Strobel, M., and Zick, Y. On the privacy risks of model explanations. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES), pp. 231-241, 2021. 1, 3, 8, 14, 15
[33] Song, S., Chaudhuri, K., and Sarwate, A. D. Stochastic gradient descent with differentially private updates. In 2013 IEEE global conference on signal and information processing, pp. 245-248. IEEE, 2013. 7, 16
[34] Steinke, T., Nasr, M., and Jagielski, M. Privacy auditing with one (1) training run. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 14