  In game-theoretic learning, several agents are simultaneously following their
individual interests, so the environment is non-stationary from each player's
perspective. In this context, the performance of a learning algorithm is often
measured by its regret. However, no-regret algorithms are not created equal in
terms of game-theoretic guarantees: depending on how they are tuned, some of
them may drive the system to an equilibrium, while others could produce cyclic,
chaotic, or otherwise divergent trajectories. To account for this, we propose a
range of no-regret policies based on optimistic mirror descent, with the
following desirable properties: i) they do not require any prior tuning or
knowledge of the game; ii) they all achieve O(\sqrt{T}) regret against
arbitrary, adversarial opponents; and iii) they converge to the best response
against convergent opponents. Also, if employed by all players, then iv) they
guarantee O(1) social regret; while v) the induced sequence of play converges
to Nash equilibrium with O(1) individual regret in all variationally stable
games (a class of games that includes all monotone and convex-concave zero-sum
games).
