SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Example 3.1. A standard family of kernelized regularizers is given by  $\theta(z) = z^{\rho} / [\rho(\rho - 1)]$  for  $\rho \in (0, 1) \cup (1, 2]$  and  $\theta(z) = z \log z$  for  $\rho = 1$  [10, 31, 33, 53]. This family includes:
- For  $\rho = 2$ , the quadratic regularizer  $\theta(z) = z^2 / 2$ , which yields the Euclidean projection map
$$
Q _ {i} \left(y _ {i}\right) = \Pi_ {\mathcal {X} _ {i}} \left(y _ {i}\right) \equiv \arg \min  _ {x _ {i} \in \mathcal {X} _ {i}} \left\| y _ {i} - x _ {i} \right\| _ {2}. \tag {10}
$$
- For  $\rho = 1$ , the entropic regularizer  $\theta(z) = z \log z$ , which yields the logit choice map (7).
- For  $\rho = 1/2$ , the fractional power regularizer  $\theta(z) = -4\sqrt{z}$  that underlies the TSALLIS-INF algorithm of [1, 53] (see also Section 3.5 below).
3.5. Specific algorithms. We now proceed to discuss some archetypal examples of (RL).
Algorithm 1 (Follow the regularized leader). The standard "follow-the-regularized-leader" (FTRL) method of Shalev-Shwartz & Singer [46] is obtained when players observe their full payoff vectors, that is,  $\hat{v}_{i,t} = v_{i}(X_{t})$ . In this case, (RL) boils down to the deterministic update rule
$$
Y _ {i, t + 1} = Y _ {i, t} + \gamma_ {t} v _ {i} \left(X _ {t}\right) \quad X _ {i, t + 1} = Q _ {i} \left(Y _ {i, t + 1}\right)
$$
or, more explicitly
$$
X _ {i, t + 1} = \arg \max  _ {x _ {i} \in \mathcal {X} _ {i}} \left\{\sum_ {s = 1} ^ {t} \gamma_ {s} u _ {i} \left(x _ {i}; X _ {- i, s}\right) - h _ {i} \left(x _ {i}\right) \right\} \tag {FTRL}
$$
For a detailed discussion of (FTRL), see [10, 31, 45]. We only note here that, as a special case, when (FTRL) is run with the logit choice setup of Eq. (7), a standard calculation yields the exponential/multiplicative weights (HEDGE) [4, 32, 45, 50].
Algorithm 2 (Optimistic FTRL). A notable variant of FTRL – originally due to Popov [40] and subsequently popularized by Rakhlin & Sridharan [41, 42] – is the so-called optimistic FTRL method. This scheme employs an “optimistic” correction intended to anticipate future steps, and it updates as
$$
Y _ {i, t + 1} = Y _ {i, t} + \gamma_ {t} \left[ 2 v _ {i} \left(X _ {t}\right) - v _ {i} \left(X _ {t - 1}\right) \right] \tag {Opt-FTRL}
$$
with  $X_{i,t} = Q_i(Y_{i,t})$ . As a special case, if (Opt-FTRL) is run with the logit choice map (7), we obtain the familiar update rule known as optimistic multiplicative weights (OMW) [15, 41, 42, 47].
Compared to (FTRL), the gain vector  $\hat{v}_t = 2v(X_t) - v(X_{t-1})$  of (Opt-FTRL) has offset  $b_t = v(X_t) - v(X_{t-1})$  relative to  $v(X_t)$ . Thus, even though (Opt-FTRL) assumes full access to the players' mixed payoff vectors, it uses this information differently than (FTRL): in particular, the offset of (Opt-FTRL) is non-zero by design, not because of some systematic error in the payoff measurement process.
Now, up to this point, we have not detailed how players might observe their full, mixed payoff vectors. This assumption simplifies the analysis immensely, but it is not realistic in applications to e.g., online advertising and network science, where players may only be able to observe their realized payoffs, and have no information about the strategies of other players or actions they did not play. On that account, we describe below a range of payoff-based policies where players estimate their counterfactual, "what-if" payoffs indirectly.
The most common way to achieve this is via the importance-weighted estimator
$$
\mathrm {I W E} _ {i \alpha_ {i}} (x) = \frac {\mathbb {1} \left\{\hat {\alpha} _ {i} = \alpha_ {i} \right\}}{x _ {i \alpha_ {i}}} u _ {i} (\hat {\alpha}) \quad \text {f o r a l l} \alpha_ {i} \in \mathcal {A} _ {i}, i \in \mathcal {N}, \tag {IWE}
$$
where  $x \in \mathcal{X}$  is the players' strategy profile, and  $\hat{\alpha} \in \mathcal{A}$  is drawn according to  $x$ . This estimator is at the heart of the online learning literature [10, 11, 31, 45] and it leads to the following methods:
Algorithm 3 (Bandit FTRL). Plugging (IWE) directly into (RL) yields the bandit FTRL policy
$$
Y _ {i, t + 1} = Y _ {i, t} + \gamma_ {t} \operatorname {I W E} _ {i} (\hat {X} _ {t}) \quad X _ {i, t + 1} = Q _ {i} \left(Y _ {i, t + 1}\right) \tag {B-FTRL}
$$
where (IWE) is sampled at the mixed strategy profile
$$
\hat {X} _ {i, t} = \left(1 - \delta_ {t}\right) X _ {i, t} + \delta_ {t} \operatorname {u n i f} _ {\mathcal {A} _ {i}} \tag {11}
$$
for some "explicit exploration" parameter  $\delta_t \propto 1 / t^{\ell_{\delta}}$ ,  $\ell_{\delta} > 0$ , which specifies the mix between  $X_{i,t}$  and the uniform distribution  $\mathrm{unif}_{\mathcal{A}_i}$  on  $\mathcal{A}_i$ . As we discuss in the sequel, this combination of (IWE) with the explicit exploration mechanism (11) means that the surrogate payoff vector  $\hat{v}_t = \mathrm{IWE}(\hat{X}_t)$  used to update (B-FTRL) has offset and noise bounded respectively as  $b_t = \mathcal{O}(\delta_t)$  and  $U_t = \mathcal{O}(1 / \delta_t)$ .
Two special cases of (B-FTRL) that have attracted significant attention in the literature are:
1. The exponential weights algorithm for exploration and exploitation (EXP3) [6, 11, 31], obtained by running (B-FTRL) with the logit choice map (7).
2. The Tsallis implicitly normalized forecaster (TSALLIS-INF) [1, 3, 52, 53] that was proposed as a more efficient alternative to EXP3, and which updates as
$$
X _ {i, t} = \arg \max  _ {x _ {i} \in \mathcal {X} _ {i}} \left\{\langle Y _ {i, t}, x _ {i} \rangle + 4 \sum_ {\alpha_ {i} \in \mathcal {A} _ {i}} \sqrt {x _ {i} \alpha_ {i}} \right\} \tag {TSALLIS-INF}
$$
i.e., as (B-FTRL) with the fractional power regularizer  $\theta(z) = -4\sqrt{z}$  of Example 3.1.
For illustration purposes, we provide some more examples of (RL) in Appendix B.
4 First results: resilience to strategic deviations
We are now in a position to begin our analysis of the rationality properties of the players' long-run behavior under (RL). To that end, we should first note that no-regret play may still lead to counterintuitive and highly non-rationalizable outcomes, e.g., with all players selecting dominated strategies for all time. The example below is adapted from Viossat & Zapechelyuk [49].
Example 4.1. Consider the  $4 \times 4$  symmetric 2-player game with payoff bimatrix
A	B	C	D
A	(1,1)	(1,2/3)	(0,0)	(0,-1/3)
B	(2/3,1)	(2/3,2/3)	(-1/3,0)	(-1/3,-1/3)
C	(0,0)	(0,-1/3)	(1,1)	(1,2/3)
D	(-1/3,0)	(-1/3,-1/3)	(2/3,1)	(2/3,2/3)
In this game,  $B$  and  $D$  are strictly dominated for both players by their stronger "twins" ( $A$  and  $C$  respectively). However, it is easy to check that if both players choose between  $(B, B)$  and  $(D, D)$  with probability  $1/2$  each, the resulting distribution of play  $z \in \Delta(\mathcal{A})$  satisfies  $u_i(\alpha_i; z_{-i}) - u_i(z) \leq -1/6$  for all  $\alpha_i \in \{A, B, C, D\}$ ,  $i = 1, 2$ . As a result, the players' regret under  $z_t \equiv z$  is negative, even though both players play strictly dominated strategies at all times.
The example above shows unequivocally that
No-regret play does not suffice to exclude non-rationalizable outcomes.
In addition, Example 4.1 also shows that predictions based on correlated play are not always appropriate for describing the players' behavior under (RL): the end-state of any regularized learning algorithm will be a closed connected set of mixed strategies, so it is not possible to play only  $(B, B)$  or  $(D, D)$  in the long run. We are thus led to the following natural questions: What are the rationality properties of long-run play under (RL)? Is the players' behavior robust to strategic deviations?
To study these questions formally, we will focus on the limit set  $\mathcal{L}(X)$  of  $X_{t}$  under (RL), viz.
$$
\mathcal {L} (X) := \bigcap_ {t} \operatorname {c l} \left\{X _ {s}: s \geq t \right\} \equiv \left\{\hat {x} \in \mathcal {X}: X _ {t _ {k}} \rightarrow \hat {x} \text {f o r s o m e s u b s e q u e n c e} X _ {t _ {k}} \text {o f} X _ {t} \right\}. \tag {12}
$$
In words,  $\mathcal{L}(X)$  is the set of limit points of  $X_{t}$  or, equivalently, the smallest subset of  $\mathcal{X}$  to which  $X_{t}$  converges. Clearly, the simplest instance of a limit set is when  $\mathcal{L}(X)$  is a singleton, i.e., when  $X_{t}$  converges to a point. This case has attracted significant interest in the literature: for example, if  $\mathcal{L}(X) = \{x^{*}\}$  then, for certain special cases of (RL), it is known that  $x^{*}$  is a Nash equilibrium of  $\Gamma$  [34]. However, beyond this relatively simple regime, the structure of the limit sets of (RL) could be arbitrarily complicated and their rationality properties are not well-understood.
With this in mind, as a first attempt to study whether the long-run behavior of (RL) is "robust to strategic deviations", we will consider the following notion of resilience to strategic deviations:
Definition 1. A closed subset  $S$  of  $\mathcal{X}$  is resilient to strategic deviations – or simply resilient – if, for every deviation  $x_{i} \in \mathcal{X}_{i}$  of every player  $i \in \mathcal{N}$ , we have
$$
u _ {i} \left(x ^ {*}\right) \geq u _ {i} \left(x _ {i}, x _ {- i} ^ {*}\right) \quad \text {f o r s o m e} x ^ {*} \in \mathcal {S}. \tag {13}
$$
Informally,  $S$  is resilient if every unilateral deviation from  $S$  is deterred by some (possibly different) element thereof. In particular, if  $S$  is a singleton, we immediately recover the definition of a Nash equilibrium; beyond this case however, other examples include the set of undominated strategies of a game, the support face of the equilibria of two-player zero-sum games, etc. Importantly, as we show below, the limit sets of (RL) are resilient in all games:
Theorem 1. Let  $X_{t}$ ,  $t = 1,2,\ldots$ , be the sequence of play generated by (RL) with step-size/gain parameters  $\ell_{\gamma} > 2\ell_{\sigma}$  and  $\ell_b > 0$ . Then, with probability 1, the limit set  $\mathcal{L}(X)$  of  $X_{t}$  is resilient.
Proof sketch. The proof of Theorem 1 boils down to two interleaved arguments that we detail in Appendix C. The first hinges on showing that, if  $\mathbb{P}(\mathcal{L}(X) = S) > 0$  for some non-random  $S \subseteq \mathcal{X}$ ,  $S$  must be resilient. This is argued by contradiction: if  $p_i \in \mathcal{X}_i$  is a unilateral deviation violating Definition 1, we must also have  $\lim \inf_{t \to \infty} [u_i(p_i; X_{-i,t}) - u_i(X_t)] > 0$  with positive probability. However, the existence of a strategy that consistently outperforms  $X_t$  runs contrary to the fact that strategies that (RL) selects against underperforming strategies. We make this intuition precise via an energy argument that leverages a series of results from martingale limit theory (which is where the requirements for  $\gamma_t$ ,  $b_t$  and  $U_t$  come in). Then, to get the stronger statement that the random set  $\mathcal{L}(X)$  is resilient w.p.1, we show that the above remains true if  $p_i$  is replaced by a deviation  $q_i$  which is close enough to  $p_i$  and has rational entries. Since there is a countable number of such profiles, we can use a union bound on an enumeration of the rationals to isolate a deviation witnessing the negation of Definition 1; our claim then follows by applying our argument for non-random sets.
Theorem 1 is our first universal guarantee for (RL), so some remarks are in order. First, we should point out that the requirements  $\ell_b > 0$  and  $2\ell_{\sigma} < \ell_{\gamma}$  are a priori implicit because they depend on the offset and magnitude statistics of the feedback sequence  $\hat{v}_t$ . However, in most learning algorithms, these quantities are under the explicit control of the players: for example, as we show in Appendix B, Algorithm 2 has  $\ell_b = \ell_{\gamma}$  while, for Algorithm 3, we have  $\ell_b = \ell_{\sigma} = \ell_{\delta}$ . In this way, when instantiated to Algorithms 1-3 (and special cases thereof), Theorem 1 yields the following corollary:
Corollary 1. Suppose that Algorithms 1-3 are run with  $\ell_{\gamma} \in (0,1]$  and, for Algorithm 3,  $\ell_{\delta} \in (0,\ell_{\gamma}/2)$ . Then, with probability 1, the limit set  $\mathcal{L}(X)$  of  $X_{t}$  is resilient.
Now, since Theorem 1 applies to all games, it would seem to provide a universally positive answer to whether (RL) is robust to strategic deviations. However, this is not so: a direct calculation shows that the face of  $\mathcal{X}$  that is spanned by the dominated strategies  $(B,B)$  and  $(D,D)$  of Example 4.1 is resilient, so Theorem 1 cannot exclude convergence to a set where dominated strategies survive. Thus, just like no-regret play, the notion of resilience does not suffice by itself to capture the idea of rational behavior. This is because, albeit natural, resilience is too lax to provide a meaningful link between robustness to unilateral deviations - a game-theoretic requirement - and stability under regularized learning - a dynamic requirement. We address this question in detail in the next section.
5 A characterization of strategic stability under regularized learning
Similar to the set of pure strategies that arise from no-regret play, the main limitation of resilience is that a payoff-improving deviation may be countered by an action profile where the deviator also switched to a different strategy; in other words, resilience is not a self-enforcing barrier to deviations. In view of this, we will focus below on a much more stringent criterion of strategic stability, namely that any deviation from the set in question incurs a cost to the deviating agent.
Club sets. To make all this precise, define the better-reply correspondence of player  $i \in \mathcal{N}$  as
$$
\operatorname {b t r} _ {i} (x) = \left\{x _ {i} ^ {\prime} \in \mathcal {X} _ {i}: u _ {i} \left(x _ {i} ^ {\prime}; x _ {- i}\right) \geq u _ {i} (x) \right\} \tag {14}
$$
and write  $\mathsf{btr} = \prod_{i}\mathsf{btr}_{i}$  for the product correspondence  $\mathsf{btr}(x) = \mathsf{btr}_1(x)\times \dots \times \mathsf{btr}_N(x)$ . [In words,  $\mathsf{btr}_i$  assigns to each  $x\in \mathcal{X}$  those strategies of player  $i$  that are (weakly) better against  $x$  than  $x_{i}$ .] In addition, given a product of pure strategies  $\mathcal{C} = \prod_{i\in \mathcal{N}}\mathcal{C}_i$  with  $\mathcal{C}_i\subseteq \mathcal{A}_i$  for all  $i\in \mathcal{N}$ , let  $S = \Delta (\mathcal{C})$  denote the span of  $\mathcal{C}$ , and let  $\mathcal{P}(\mathcal{X})$  denote the collection of all such sets. We then say that  $S\in \mathcal{P}(\mathcal{X})$  is closed under better replies - a club set for short - if it is closed under btr, i.e.,  $\mathsf{btr}(S)\subseteq S$ ; finally,  $S$  is said to be minimally club (m-club) if it does not admit a proper club subset.
Of course, the entire strategy space  $\mathcal{X}$  is closed under better replies so, a priori, club sets could also contain dominated strategies and/or other non-rationalizable outcomes. By contrast, minimal club sets are much more rigid in their relation to rational behavior because any unilateral deviation from an m-club set is costly, and m-club sets are minimal in this regard. On that account, m-club sets can be seen as the closest setwise analogue to strict Nash equilibria.
This analogy is accentuated further by the following properties of m-club sets, all due to Ritzberger & Weibull [43], who introduced the concept:
1. Every game admits an m-club set; and if this set is a singleton, then it is a strict Nash equilibrium.
2. Any m-club set  $S$  is fixed under better replies, that is,  $\mathtt{btr}(S) = S$  (implying in turn that  $S$  cannot contain any dominated strategies, including iteratively dominated ones).
3. Any m-club set  $S$  contains an essential equilibrium component, i.e., a component of Nash equilibria such that every small perturbation of the game admits a nearby equilibrium; in addition, this component has full support on  $S$ , i.e., it employs all pure strategy profiles that lie in  $S$ .<sup>4</sup>
Going back to our online learning setting, the above leads to the following natural set of questions:
Are club sets (minimal or not) stable under the dynamics of regularized learning?
Are they attracting? And, if so, are they the only such sets?
Any answer to these questions - positive or negative - would be an important step in delineating the relation between strategic stability (in the above sense) and dynamic stability under (RL). To that end, we start by formalizing some notions of dynamic stability that will be central in the sequel:
Definition 2. Fix some subset  $S$  of  $\mathcal{X}$  and a tolerance level  $\epsilon >0$ . We then say that  $S$  is:
1. Stochastically stable if, for every neighborhood  $\mathcal{U}$  of  $\mathcal{S}$  in  $\mathcal{X}$ , there exists a neighborhood  $\mathcal{U}_1$  of  $\mathcal{S}$  such that
$$
\mathbb {P} \left(X _ {t} \in \mathcal {U} \text {f o r a l l} t = 1, 2, \dots\right) \geq 1 - \epsilon \quad \text {w h e n e v e r X} _ {1} \in \mathcal {U} _ {1}. \tag {15}
$$
2. Stochastically attracting if there exists a neighborhood  $\mathcal{U}_1$  of  $S$  such that
$$
\mathbb {P} \left(\lim  _ {t \rightarrow \infty} \operatorname {d i s t} \left(X _ {t}, S\right) = 0\right) \geq 1 - \epsilon \quad \text {w h e n e v e r} X _ {1} \in \mathcal {U} _ {1}. \tag {16}
$$
3. Stochastically asymptotically stable if it is stochastically stable and attracting.
4. Irreducibly stable if  $S$  is stochastically asymptotically stable and it does not admit a strictly smaller stochastically asymptotically subset  $S'$  with  $\operatorname{supp}(S') \subsetneq \operatorname{supp}(S)$ .
With all this in hand, our main result below provides a sharp characterization of strategic stability in the context of regularized learning:
Theorem 2. Fix some set  $S \in \mathcal{P}(\mathcal{X})$  and suppose that (RL) is run with a steep regularizer and step-size/gain parameters  $\ell_{\gamma} \in [0,1]$ ,  $\ell_b > 0$ , and  $\ell_{\sigma} < 1/2$ . Then:
1.  $S$  is stochastically asymptotically stable under (RL) if and only if it is a club set.
2.  $S$  is irreducibly stable under (RL) if and only if it is an m-club set.
In addition, we also get the following convergence rate estimates for club sets:
Theorem 3. Let  $S \in \mathcal{P}(\mathcal{X})$  be a club set, and let  $X_{t}, t = 1,2,\ldots$ , be the sequence of play generated by (RL) with parameters  $\ell_{\gamma} \in [0,1]$ ,  $\ell_{b} > 0$ , and  $\ell_{\sigma} < 1/2$ . Then, for all  $\epsilon > 0$ , there exists an (open, unbounded) initialization domain  $\mathcal{D} \subseteq \mathcal{Y}$  such that, with probability at least  $1 - \epsilon$ , we have
$$
\operatorname {d i s t} \left(X _ {t}, \mathcal {S}\right) \leq C \varphi \left(c _ {1} - c _ {2} \sum_ {s = 1} ^ {t} \gamma_ {s}\right) \quad \text {w h e n e v e r} Y _ {1} \in \mathcal {D} \tag {17}
$$
where  $C, c_1, c_2$  are constants ( $C, c_2 > 0$ ), and the rate function  $\varphi$  is given by  $\varphi(z) = (\theta')^{-1}(z)$  if  $z > \lim_{z \to 0^+} \theta'(z)$ , and  $\varphi(z) = 0$  otherwise.
Specifically, if we instantiate Theorem 3 to Algorithms 1-3, we get the explicit estimates:
Corollary 2. Suppose that Algorithms 1-3 are run with  $\ell_{\gamma} \in [0,1]$  and, for Algorithm 3,  $\ell_{\delta} \in (0,1/2)$ . Then, with notation as in Theorem 3,  $X_{t}$  converges to  $\mathcal{S}$  at a rate of
$$
\operatorname {d i s t} \left(X _ {t}, \mathcal {S}\right) \leq C \cdot \left\{ \begin{array}{l l} {\left[ 1 - c \sum_ {s = 1} ^ {t} \gamma_ {s} \right] _ {+}} & {\text {i f} \theta (z) = z ^ {2} / 2 \quad \# q u a d r a t i c r e g u l a r i z a t i o n} \\ {\exp \left(- c \sum_ {s = 1} ^ {t} \gamma_ {s}\right)} & {\text {i f} \theta (z) = z \log z \quad \# e n t r o p i c r e g u l a r i z a t i o n} \\ {1 / \left(c + \sum_ {s = 1} ^ {t} \gamma_ {s}\right) ^ {2}} & {\text {i f} \theta (z) = - 4 \sqrt {z} \quad \# f r a c t i o n a l r e g u l a r i z a t i o n} \end{array} \right. \tag {18}
$$
for positive constants  $C, c > 0$ . In particular, the projection-based variants of Algorithms 1-3 converge to  $m$ -club sets in a finite number of steps.
Formally, a component  $\mathcal{X}^*$  of Nash equilibria of  $\Gamma$  is essential if, for all  $\varepsilon > 0$ , there exists  $\delta > 0$  such that any perturbation of the payoffs of  $\Gamma$  by at most  $\delta$  produces a Nash equilibrium that is  $\varepsilon$ -close to  $\mathcal{X}^*$  [48]. This property - known as "essentiality" - has a long history as one of the strictest setwise solution refinements in game theory; in particular, it satisfies all the seminal strategic stability requirements of Kohlberg & Mertens [30], including robustness to strategic payoff perturbations. For an in-depth discussion, see van Damme [48].
Figure 1: The long-run behavior of EXP3 (Algorithm 3) in four representative  $2 \times 2 \times 2$  games. In all cases, the dynamics converge to m-club sets, either strict equilibria themselves, or spanning an essential component of Nash equilibria. The details of the numerics and the games being played are provided in the appendix.
Proof sketch. The proof of Theorems 2 and 3 is quite involved so we defer it to Appendix D. At a high level, it hinges on constructing a family of "primal-dual" energy functions, one per pure deviation from the set  $S$  under study. If unilateral deviations from  $S$  incur a cost to the deviator (that is, if  $S$  is club), these energy functions can be "bundled together" to produce a suitable Lyapunov-like function for  $S$ . In more detail, the minimization of each individual energy function implies that the score variable  $Y_{t}$  of (RL) diverges along an "astral direction" in the payoff space  $\mathcal{V}$  - i.e., it escapes to infinity along the interior of a certain convex cone of  $\mathcal{V}$  [18]. Because this minimization occurs at infinity, the aggregation of offsets and random errors in (RL) affords some extra "wiggle room" in our martingale analysis, so we are able to show that  $X_{t} = Q(Y_{t})$  remains close to  $S$  under a much wider range of parameters compared to Theorem 1. Then, a series of convex analysis arguments in the spirit of [33] coupled with the definition of  $Q$  allows us to show that the escape of  $Y_{t}$  along the intersection of all these cones implies convergence to  $S$  at the specified rate.
On the converse side, if an asymptotically stable set is not club, we can find a non-costly (and possibly profitable) deviation  $z$  from  $S$  which is selected against by (RL). However, this extinction runs contrary to the reinforcement of better replies under (RL), an argument which can be made precise by applying the martingale law of large numbers to  $\langle Y_t, z \rangle$  [25]. The irreducible stability of m-club sets then follows by invoking this criterion reductively for any potentially stable subset  $S'$  of  $S$ .
Discussion and remarks. Theorems 2 and 3 are our main results linking dynamic and strategic stability, so we conclude with a series of remarks. First, we should note that Theorem 2 can be summed up as follows: a product of pure strategies is (minimally) closed under better replies if and only if its span is (irreducibly) stable under regularized learning. Importantly, this equivalence is based solely on the game's payoff data: it does not depend on the specific choices underlying (RL), including the choice map employed by each player, whether some players are using an optimistic adjustment or not, if they have access to their full payoff vectors, etc. As such, this equivalence provides a crisp operational criterion for identifying which pure strategy combinations ultimately persist under regularized learning – and, via Theorem 3, how fast this identification takes place.