[23] Giannou, A., Vlatakis-Gkaragkounis, E. V., and Mertikopoulos, P. The convergence rate of regularized learning in games: From bandits and uncertainty to optimism and beyond. In NeurIPS '21: Proceedings of the 35th International Conference on Neural Information Processing Systems, 2021.
[24] Gidel, G., Berard, H., Vignoud, G., Vincent, P., and Lacoste-Julien, S. A variational inequality perspective on generative adversarial networks. In *ICLR '19: Proceedings of the 2019 International Conference on Learning Representations*, 2019.
[25] Hall, P. and Heyde, C. C. Martingale Limit Theory and Its Application. Probability and Mathematical Statistics. Academic Press, New York, 1980.
[26] Hannan, J. Approximation to Bayes risk in repeated play. In Dresher, M., Tucker, A. W., and Wolfe, P. (eds.), Contributions to the Theory of Games, Volume III, volume 39 of Annals of Mathematics Studies, pp. 97-139. Princeton University Press, Princeton, NJ, 1957.
[27] Hofbauer, J. and Sigmund, K. Evolutionary game dynamics. Bulletin of the American Mathematical Society, 40(4):479-519, July 2003.
[28] Hsieh, Y.-G., Iutzeler, F., Malick, J., and Mertikopoulos, P. On the convergence of single-call stochastic extra-gradient methods. In NeurIPS '19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 6936-6946, 2019.
[29] Juditsky, A., Nemirovski, A. S., and Tauvel, C. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1):17-58, 2011.
[30] Kohlberg, E. and Mertens, J.-F. On the strategic stability of equilibria. *Econometrica*, 54(5):1003-1037, September 1986.
[31] Lattimore, T. and Szepesvári, C. Bandit Algorithms. Cambridge University Press, Cambridge, UK, 2020.
[32] Littlestone, N. and Warmuth, M. K. The weighted majority algorithm. Information and Computation, 108 (2):212-261, 1994.
[33] Mertikopoulos, P. and Sandholm, W. H. Learning in games via reinforcement and regularization. Mathematics of Operations Research, 41(4):1297-1324, November 2016.
[34] Mertikopoulos, P. and Zhou, Z. Learning in games with continuous action sets and unknown payoff functions. Mathematical Programming, 173(1-2):465-507, January 2019.
[35] Mertikopoulos, P., Papadimitriou, C. H., and Piliouras, G. Cycles in adversarial regularized learning. In SODA '18: Proceedings of the 29th annual ACM-SIAM Symposium on Discrete Algorithms, 2018.
[36] Mertikopoulos, P., Lecouat, B., Zenati, H., Foo, C.-S., Chandrasekhar, V., and Piliouras, G. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. In ICLR '19: Proceedings of the 2019 International Conference on Learning Representations, 2019.
[37] Nemirovski, A. S. Prox-method with rate of convergence  $O(1 / t)$  for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15(1):229-251, 2004.
[38] Palaiopanos, G., Panageas, I., and Piliouras, G. Multiplicative weights update with constant step-size in congestion games: Convergence, limit cycles and chaos. In NIPS '17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
[39] Piliouras, G., Sim, R., and Skoulakis, S. Optimal no-regret learning in general games: Bounded regret with unbounded step-sizes via clairvoyant mwu. https://arxiv.org/abs/2111.14737, 2021.
[40] Popov, L. D. A modification of the Arrow-Hurwicz method for search of saddle points. Mathematical Notes of the Academy of Sciences of the USSR, 28(5):845-848, 1980.
[41] Rakhlin, A. and Sridharan, K. Online learning with predictable sequences. In COLT '13: Proceedings of the 26th Annual Conference on Learning Theory, 2013.
[42] Rakhlin, A. and Sridharan, K. Optimization, learning, and games with predictable sequences. In NIPS '13: Proceedings of the 27th International Conference on Neural Information Processing Systems, 2013.
[43] Ritzberger, K. and Weibull, J. W. Evolutionary selection in normal-form games. *Econometrica*, 63(6): 1371–99, November 1995.
[44] Rockafellar, R. T. and Wets, R. J. B. Variational Analysis, volume 317 of A Series of Comprehensive Studies in Mathematics. Springer-Verlag, Berlin, 1998.
[45] Shalev-Shwartz, S. Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2):107-194, 2011.
[46] Shalev-Shwartz, S. and Singer, Y. Convex repeated games and Fenchel duality. In NIPS' 06: Proceedings of the 19th Annual Conference on Neural Information Processing Systems, pp. 1265-1272. MIT Press, 2006.
[47] Syrgkanis, V., Agarwal, A., Luo, H., and Schapire, R. E. Fast convergence of regularized learning in games. In NIPS '15: Proceedings of the 29th International Conference on Neural Information Processing Systems, pp. 2989-2997, 2015.