SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Kernel Function Class. Now we consider  $\mathcal{F}$  as the kernel function class, namely, we suppose  $f_{i} \in (\mathcal{H})^{p}$ , where  $\mathcal{H}$  is an RKHS induced by a Mercer kernel  $K: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$  and a user-specified probability measure  $\rho_{\mathcal{X}}$ . By the reproducing property of  $\mathcal{H}$ ,  $f_{i}$  can be represented as  $f_{i}(x) = \langle f_{i}, \phi_{x} \rangle_{\mathcal{H}}$ , where  $\phi: \mathcal{X} \to \mathcal{H}$  is the feature map, i.e.  $\phi_{x} := K(\cdot, x) \in \mathcal{H}$  for any  $x \in \mathcal{X}$ . Therefore, (7) becomes the kernel regression  $\arg \min_{f \in \mathcal{F}^{p}} \mathbb{E}_{(u_{i}, y_{i}) \sim \rho_{i}} \|y_{i} - \langle f, \phi_{u_{i}} \rangle_{\mathcal{H}}\|^{2}$ . However, as  $\mathcal{H}$  is generally an infinite-dimensional space, the aforementioned regression problem might lead to ill-posed solutions. Consequently, we consider the regularized kernel ridge regression  $\arg \min_{f \in \mathcal{F}^{p}} \mathbb{E}_{(u_{i}, y_{i}) \sim \rho_{i}} \|y_{i} - \langle f, \phi_{u_{i}} \rangle_{\mathcal{H}}\|^{2}/2 + \lambda_{t} \|f\|_{\mathcal{H}}^{2}$ . In each iteration  $t$ , we suppose that  $f_{i}^{t-1}$  is the estimation of  $f_{i}$  from the previous iteration, the OPGD algorithm samples  $u_{i}^{t} \sim \rho_{\mathcal{X}}, y_{i}^{t} \sim \mathcal{D}_{i}(u_{i}^{t})$  and takes gradient steps on the kernel ridge objective  $\|y_{i}^{t} - \langle f, \phi_{u_{i}^{t}} \rangle_{\mathcal{H}}\|^{2}/2 + \lambda_{t} \|f\|_{\mathcal{H}}^{2}$ , i.e. it takes the online kernel ridge update (Tarres and Yao, 2014; Dieuleveut and Bach, 2016):
$$
f _ {i} ^ {t} = f _ {i} ^ {t - 1} - \nu_ {t} \left[ \left(f _ {i} ^ {t - 1} \left(u _ {i} ^ {t}\right) - y _ {i} ^ {t}\right) \phi_ {u _ {i} ^ {t}} + \lambda_ {t} f _ {i} ^ {t - 1} \right]. \tag {10}
$$
We suppose that the kernel  $K$  is 2-differentiable, i.e.,  $K \in C^{2}(\mathcal{X},\mathcal{X})$ . Define  $\partial_i\phi : \mathcal{X} \to \mathcal{H}$  as the partial derivative of the feature map  $\phi$  with respect to  $x_{i}$ , namely,  $\partial_i\phi_x = \partial_iK(x,\cdot) = \partial K(x,\cdot) / \partial x_i$ . Steinwart and Christmann (2008, Lemma 4.34) shows that  $\partial_i\phi_x$  exists, continuous and  $\partial_i\phi_x \in \mathcal{H}$ . By the reproducing property  $\partial f_{i}(x) / \partial x_{i} = \partial \langle f_{i},\phi_{x}\rangle_{\mathcal{H}} / \partial x_{i} = \langle f_{i},\partial_{i}\phi_{x}\rangle_{\mathcal{H}}$ , the individual gradient  $\nabla_{i}\mathcal{L}_{i}(x)$  has the form  $\nabla_{i}\mathcal{L}_{i}(x) = \mathbb{E}_{z_{i} \sim \mathcal{D}_{i}(x)}[\nabla_{i}\ell_{i}(x,z_{i}) + (\langle f_{i},\partial_{i}\phi_{x}\rangle_{\mathcal{H}})^{\top}\nabla_{z_{i}}\ell_{i}(x,z_{i})]$ . After step (i), OPGD draws a sample  $z_{i}^{t} \sim \mathcal{D}_{i}(x^{t})$  and performs the projected gradient step:
$$
\left. \left. x _ {i} ^ {t + 1} \leftarrow \operatorname {p r o j} _ {\mathcal {X} _ {i}} \left(x _ {i} ^ {t} - \eta_ {t} \left(\nabla_ {i} \ell_ {i} \left(x ^ {t}, z _ {i} ^ {t}\right) + \left(\left\langle f _ {i} ^ {t}, \partial_ {i} \phi_ {x ^ {t}} \right\rangle_ {\mathcal {H}}\right) ^ {\top} \nabla_ {z _ {i}} \ell_ {i} \left(x ^ {t}, z _ {i} ^ {t}\right)\right)\right). \right. \right. \tag {11}
$$
We remark that the gradient steps  $\eta_t, \nu_t$  and regularization terms  $\lambda_t$  should be chosen carefully to ensure convergence (see Theorem 2). Specifically, the regularization terms  $\lambda_t$  must shift to 0 gradually. If  $\lambda_t$  is a constant,  $f_i^t$  in (10) converges to the solution of a regularized kernel ridge regression, which is a biased estimator of  $f_i$ . Thus (11) fails to converge because the gradient estimation has a constant bias. We present the pseudocode of OPGD for the linear setting as Algorithm 1 and for the RKHS setting as Algorithm 2 in Appendix A.
Comparison with Narang et al. (2022). We clarify the difference between  $\mathsf{OPGD}$  and the Adaptive Gradient Method (AGM) proposed in Narang et al. (2022). To elaborate, AGM samples  $z_{i}^{t} \sim \mathcal{D}_{i}(x^{t})$  at current the action and let agents play again with an injected noise  $u^{t}$  to obtain  $q_{i}^{t} \sim \mathcal{D}_{i}(x^{t} + u^{t})$ . The algorithm is based on the fact that  $\mathbb{E}[q_{i}^{t} - z_{i}^{t}|u^{t}, x^{t}] = A_{i}u^{t}$ , which is not related to  $x^{t}$ . Thus,  $A_{i}$  can be estimated by online least squares. We remark that  $\mathbb{E}[q_{i}^{t} - z_{i}^{t}|u^{t}, x^{t}]$  depends on agents' actions in the non-linear (RKHS) cases, because  $\mathbb{E}[q_{i}^{t} - z_{i}^{t}|u^{t}, x^{t}] = f_{i}(x^{t} + u^{t}) - f_{i}(x^{t}) = \langle f_{i}, \phi_{x^{t} + u^{t}} - \phi_{x^{t}} \rangle_{\mathcal{H}}$ . Thus, the change of action will bring additional error that makes the estimation fail to converge. In contrast,  $\mathsf{OPGD}$  lets agents play  $u_{i}^{t} \sim \rho_{X}$  to explore the action space and learn the strategic behavior of other agents.  $\mathsf{OPGD}$  estimates the parametric function by solving the ERM version of (7) using online stochastic approximation (8) and (10). This learning framework can be applied to RKHS and potentially beyond that, such as overparameterized neural networks using the technique of neural tangent kernel (Allen-Zhu et al., 2019).
We provide theoretical guarantees for OPGD in both linear and RKHS settings. We first impose some mild assumptions. Similar assumptions are adopted in Mendler-Dünner et al. (2020); Izzo et al. (2021); Narang et al. (2022); Cutler et al. (2022).
Assumption 2. ( $\tau$ -strongly monotone). The game (1) is  $\tau$ -strongly monotone.
Assumption 3. (Smoothness).  $H(x)$  is  $L$ -Lipschitz continuous:
$$
H \left(x _ {1}\right) - H \left(x _ {2}\right) \leq L \| x _ {1} - x _ {2} \|, \quad \forall x _ {1}, x _ {2} \in \mathcal {X}.
$$
Assumption 4. (Lipschitz continuity in  $z$ ). Define  $\mathcal{D} = \mathcal{D}_1 \times \mathcal{D}_2 \times \dots \times \mathcal{D}_n : \mathcal{X} \to \mathbb{P}(\mathcal{Z})$ , where  $\mathcal{Z}$  is the sample space  $\mathcal{Z} = \mathcal{Z}_1 \times \mathcal{Z}_2 \times \dots \times \mathcal{Z}_n$ . For all  $i \in [n], x \in \mathcal{X}$ , there exists a constant  $\delta > 0$
$$
\operatorname * {E} _ {z \sim \mathcal {D} (x)} \sqrt {\sum_ {i = 1} ^ {n} \| \nabla_ {z _ {i}} \ell_ {i} (x , z _ {i}) \| ^ {2}} \leq \delta .
$$
Assumption 5. (Finite variance). There exists a constant  $\zeta >0$
$$
\underset {z _ {i} \sim \mathcal {D} _ {i} (x)} {\mathbb {E}} \| \nabla_ {i, z _ {i}} \ell_ {i} (x, z _ {i}) - \underset {z _ {i} \sim \mathcal {D} _ {i} (x)} {\mathbb {E}} \nabla_ {i, z _ {i}} \ell_ {i} (x, z _ {i}) \| ^ {2} \leq \zeta^ {2}, \quad \forall i \in [ n ], \forall x \in \mathcal {X},
$$
where  $\nabla_{i,z_i}\ell_i$  denotes the gradient of  $\ell_i(x,z_i)$  with respect to  $x_{i}$  and  $z_{i}$ .
We remark that Assumption 3 is the standard smoothness assumption for the utility functions  $\mathcal{L}_i(x)$  (Boyd et al., 2004; Nesterov et al., 2018). Since  $\mathcal{X}$  is a compact set within  $\mathbb{R}^d$ , Assumption 4 holds if  $\ell_i(x,z_i)$  is Lipschitz continuous in  $z_{i}$  and the gradient  $\nabla_{z_i}\ell_i(x,z_i)$  is continuous in  $x$ , and Assumption 5 holds if  $\ell_i(x,z_i)$  is Lipschitz in  $x$  and  $z_{i}$  (thus  $\nabla_{i,z_i}\ell_i(x,z_i)$  has a bounded norm). Assumption 5 implies that the variances of  $\nabla_{i}\ell_{i}(x,z_{i})$  and  $\nabla_{z_i}\ell_i(x,z_i)$  are both bounded by  $\zeta^2$  for any  $x\in \mathcal{X}$  and  $z_{i}\sim \mathcal{D}_{i}(x)$ . We provide sufficient conditions for Assumption 2 in Appendix B.1.
4.1 Convergence Rate in the Linear Setting
We introduce two assumptions necessary to derive theoretical guarantees for the linear function class.
Assumption 6. (Linear assumption). Suppose that the parametric assumption holds (Assumption 1) and  $f_{i}(x) = A_{i}x$  for  $i\in [n]$ , where  $A_{i}\in \mathbb{R}^{p\times d}$  are unknown matrices.
Assumption 7. (Sufficiently isotropic). There exists constants  $l_{1}, l_{2}, R > 0$  such that
$$
l _ {1} I \preceq \mathbb {E} _ {u \sim \rho_ {\mathcal {X}}} u u ^ {\top}, \quad \mathbb {E} _ {u \sim \rho_ {\mathcal {X}}} \| u \| ^ {2} \leq l _ {2}, \quad \mathbb {E} _ {u \sim \rho_ {\mathcal {X}}} \left[ \| u \| ^ {2} u u ^ {\top} \right] \preceq R \mathbb {E} _ {u \sim \rho_ {\mathcal {X}}} u u ^ {\top}.
$$
Assumption 7 has been studied in the literature on online least squares regression (Dieuleveut et al., 2017; Narang et al., 2022). Essentially, this requires the distribution  $\rho_{\mathcal{X}}$  to be sufficiently isotropic and non-singular, and it ensures the random variable  $u_i^t \sim \rho_{\mathcal{X}}$  in the online estimation update step (8) can explore all the "directions" of  $\mathbb{R}^p$ . A simple example that satisfies Assumption 7 is the uniform distribution  $\rho_{\mathcal{X}} = \mathcal{U}[0,1]$ , in which case  $l_1 = l_2 = 1/3$ ,  $R = 3/5$ .
The next theorem provides the convergence rate of 0PGD under the linear setting.
Theorem 1. (Convergence in the linear setting). Suppose that Assumptions 2, 3, 4, 5, 6, and 7 hold. Set  $\eta_t = 2 / (\tau(t + t_0))$ ,  $\nu_t = 2 / (l_1(t + t_0))$ , where  $t_0$  is a constant that satisfies  $t_0 \geq 2l_2R / l_1^2$ . For
all iterations  $t \geq 1$ , the  $x^t$  generated by the OPGD algorithm in Section 3 for linear function class satisfies
$$
\mathbb {E} \| x ^ {t} - x ^ {*} \| ^ {2} \leq \frac {\left(4 D _ {1} + 2 D _ {2} \left(t _ {0} + 1\right) \tau\right) \left(t _ {0} + 2\right) ^ {2} / \left(t _ {0} + 1\right) ^ {2}}{\tau^ {2} \left(t + t _ {0}\right)} + \frac {\left(t _ {0} + 1\right) ^ {2} \| x ^ {1} - x ^ {*} \| ^ {2}}{\left(t + t _ {0}\right) ^ {2}}, \tag {12}
$$
where  $D_{1}$  and  $D_{2}$  are constants that
$$
D _ {1} := 4 \zeta^ {2} (1 + 2 (M / (t _ {0} + 1) + \sup _ {i \in [ n ]} \| A _ {i} \| _ {F} ^ {2})), D _ {2} := 2 \delta^ {2} M, M := \frac {2 t _ {0} ^ {4} \sum_ {i = 1} ^ {n} \| A _ {i} ^ {0} - A _ {i} \| _ {F} ^ {2}}{(t _ {0} + 1) ^ {3}} + \frac {8 n l _ {2} \sigma^ {2} (t _ {0} + 2) ^ {2}}{l _ {1} ^ {2} (t _ {0} + 1) ^ {2}}.
$$
We refer the reader to Appendix C.1 for complete proof. Next, we illustrate the parameters involved in Theorem 1:  $\tau$  is the strongly monotone parameter of the game (1),  $l_{1}, l_{2}, R$  are intrinsic parameters describing the isotropy of the distribution  $\rho_{\mathcal{X}}$  (Assumption 7),  $\sigma^2$  is the variance of the noise term  $\epsilon_i$  defined in Assumption 1,  $\zeta$  and  $\delta$  describe the continuity of  $\ell_i$  (Assumption 4, 5),  $t_0$  is a sufficiently large value,  $A_i^0$  is the initial estimation of  $A_i$ ,  $x^1$  is the initial input. Theorem 1 is a combination of Lemma 2 and Lemma 3, where Lemma 2 is the statistical error of the online approximation step (8) and Lemma 3 is the one-step optimization error of the projected gradient step (9). Theorem 1 implies the convergence rate of OPGD in the linear setting is  $\mathcal{O}(t^{-1})$ , which matches the optimal rate of stochastic gradient descent in the strongly-convex setting.
4.2 Convergence Rate in the RKHS Setting
Suppose that  $K:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$  is a continuous Mercer kernel, by Mercer's theorem, it has the spectral representation  $K = \sum_{i = 1}^{\infty}\mu_i e_i\otimes e_i$  , where  $\{\mu_i\}_{i = 1}^{\infty}$  are eigenvalues,  $\{e_i\}_{i = 1}^{\infty}$  are eigenfunctions, and  $\otimes$  denotes the tensor product. Moreover,  $\{e_i\}_{i = 1}^{\infty}$  is an orthogonal basis of  $\mathcal{L}_{\rho \chi}^{2}$  and  $\{\mu_i^{1 / 2}e_i\}_{i = 1}^{\infty}$  is the orthogonal basis of  $\mathcal{H}$ , which induces the representation  $\mathcal{H} = \{\sum_{i = 1}^{\infty}a_i\mu_i^{1 / 2}e_i:\{a_i\}_{i = 1}^{\infty}\in \ell^2\}$ .
Definition 4. (Power space). For a constant  $\alpha \geq 0$ , the  $\alpha$ -power space of an RKHS  $\mathcal{H}$  is defined by
$$
\mathcal {H} ^ {\alpha} = \left\{\sum_ {i = 1} ^ {\infty} a _ {i} \mu_ {i} ^ {\alpha / 2} e _ {i}: \{a _ {i} \} _ {i = 1} ^ {\infty} \in \ell^ {2} \right\},
$$
equipped with the  $\alpha$ -power norm  $\|\cdot\|_{\alpha}$  and inner product  $\langle \cdot, \cdot \rangle_{\alpha}$ , where  $\|\sum_{i=1}^{\infty} a_i \mu_i^{\alpha/2} e_i\|_{\alpha} \coloneqq \left( \sum_{i=1}^{\infty} a_i^2 \right)^{1/2}$  and  $\langle \sum_{i=1}^{\infty} a_i \mu_i^{\alpha/2} e_i, \sum_{i=1}^{\infty} b_i \mu_i^{\alpha/2} e_i \rangle_{\alpha} = \sum_{i=1}^{\infty} a_i b_i$ .
We remark that: (i)  $\mathcal{H}^1 = \mathcal{H}$  and  $\mathcal{H}^\alpha \subset \mathcal{H}^\beta$  for any  $\alpha >\beta$ , (ii)  $\| \cdot \| _1 = \| \cdot \|_{\mathcal{H}}$  and  $\| \cdot \| _0 = \| \cdot \|_{\rho_\mathcal{X}}$ , and (iii)  $\mathcal{H}^\alpha$  is an RKHS on  $\mathcal{X}$  with kernel  $K^{\alpha}\coloneqq \sum_{i = 1}^{\infty}\mu_i^\alpha e_i\otimes e_i$  and measure  $\rho_{\mathcal{X}}$ . We review more properties of RKHS and power spaces in Appendix Sections B.3 and B.4.
We present assumptions on the kernel function class, similar assumptions can be found in the literature on kernel regression and stochastic approximation (Caponnetto and De Vito, 2007; Steinwart et al., 2009; Dicker et al., 2017; Pillaud-Vivien et al., 2018; Fischer and Steinwart, 2020).
Assumption 8. (Source condition). Suppose Assumption 1 holds and there exists an RKHS,  $\mathcal{H}$ , with a bounded differentiable Mercer kernel,  $K$ , and constants  $\beta, \kappa > 0$  such that  $\sup_{x \in \mathcal{X}} K(x, x) \leq \kappa^2$  and  $f_i \in (\mathcal{H}^\beta)^p$  for all  $i \in [n]$ .
Assumption 9. (Embedding property). There exist constants  $\alpha \in (0,1]$ ,  $A > 0$  such that  $K^{\alpha}(x,x) = \sum_{i=1}^{\infty} \mu_i^\alpha e_i^2(x) \leq A^2$ , for all  $x \in \mathcal{X}$ .
Assumption 10. (Lipschitz kernel). Suppose Assumption 9 holds and there exists  $\xi > 0$  such that  $\| \partial_i \phi_x^\alpha \|_\alpha \leq \xi$  for any  $i \in [n]$  and  $x \in \mathcal{X}$ , where  $\phi_x^\alpha : \mathcal{X} \to \mathcal{H}^\alpha$  is the feature map of the kernel  $K^\alpha$ .
Assumption 8 holds when  $K$  is bounded, differentiable, and each coordinate of parametric functions  $f_{i}$  lies in the power space  $\mathcal{H}^{\beta}$ . When  $\beta < 1$ , Assumption 8 includes the challenging scenario, namely,  $f_{i} \notin (\mathcal{H})^{p}$ . Assumption 9 holds if there exists a power space  $\mathcal{H}^{\alpha}$  such that the kernel  $K^{\alpha}$  is bounded. Thus, Assumption 9 holds with  $\alpha = 1$  for any bounded kernel  $K$ . We further propose Proposition 1 as sufficient conditions for the embedding property following Mendelson and Neeman (2010). Recalling the definition of partial derivative  $\partial_i \phi^\alpha : \mathcal{X} \to \mathcal{H}^\alpha$  (Section 3), Assumption 10 holds if  $\partial_i \partial_{i+d} K^\alpha(x, x) = \| \partial_i \phi_x^\alpha \|_\alpha^2 \leq \xi^2$  for any  $x \in \mathcal{X}$ , i.e., it holds for any Lipschitz kernel  $K^\alpha$ .
Proposition 1. (Sufficient conditions for Assumption 9) Suppose there exist constants  $C, D, p > 0$  and  $q \in (0,1)$  such that
$$
\sup  _ {i \in \mathbb {N}} \mu_ {i} ^ {p} \| e _ {i} \| _ {\infty} \leq C \quad a n d \quad \mu_ {i} \leq D i ^ {- 1 / q},
$$
where  $\| \cdot \|_{\infty}$  denotes the  $L^{\infty}$  norm. Then Assumption 9 holds for any  $\alpha >2p + q$
Proposition 1 follows from the inequality:  $\sup_{x\in \mathcal{X}}K^{\alpha}(x,x) = \sup_{x\in \mathcal{X}}\sum_{i = 1}^{\infty}(\mu_i^p e_i(x))^2\mu_i^{\alpha -2p}\leq$ $C^2 D^{\alpha -2p}\sum_{i = 1}^{\infty}i^{-(\alpha -2p) / q} <   \infty .$  We refer readers to Appendix B.4 for examples that satisfy these assumptions. Next, we provide the convergence of the proposed algorithm under the RKHS setting.
Theorem 2. (Convergence in the RKHS setting). Suppose that Assumptions 2, 3, 4, 5 hold, Assumption 8 holds for some  $\beta \in (0,2]$ , and Assumptions 9, 10 hold for some  $\alpha \in (0,1]$  and  $\alpha < \beta$ . For all iterations  $t \geq 1$  and positive constant  $a$ , define  $\bar{t} = t + t_0$ , where  $t_0$  is a constant that satisfies  $t_0 \geq (a\kappa^2 + 1)^2$ . Set the gradient steps and regularization terms as
$$
\eta_ {t} = (\tau \bar {t}) ^ {- 1}, \quad \nu_ {t} = a \cdot \bar {t} ^ {- \frac {\beta - \alpha + 1}{\beta - \alpha + 2}}, \quad \lambda_ {t} = a ^ {- 1} \cdot \bar {t} ^ {- \frac {1}{\beta - \alpha + 2}}.
$$
If  $a < \sqrt{(\beta - \alpha + 2) / (\beta - \alpha)} (t_0 + 1) / (t_0 + 2)\kappa^{\alpha -2}A^{-1}$ , the  $x^{t}$  generated by the OPGD algorithm in Section 3 using kernel  $K$  for online estimation steps (10) and projected gradient steps (11) satisfies
$$
\mathbb {E} \| x ^ {t} - x ^ {*} \| ^ {2} \lesssim \mathcal {O} \left(t ^ {- \frac {\beta - \alpha}{\beta - \alpha + 2}}\right). \tag {13}
$$
See Appendix C.2 for complete proof. We remark that bounding the estimation error  $\mathbb{E}\| f_i^t -f_i\|_\gamma^2$  under the  $\gamma$ -power norm for some  $\gamma \in [\alpha ,\beta)$  and  $\gamma \leq 1$  (Lemma 4) plays a key role in the proof of Theorem 2, and we regard this lemma as the most challenging part of our theory. Lemma 4 extends the classical theory of online stochastic approximation under the RKHS norm  $\| \cdot \|_{\mathcal{H}}$  into a continuous scale. For  $\beta >1$  and  $\gamma = 1$ , this rate would be  $\mathcal{O}(t^{-(\beta -1) / (\beta +1)})$  and matches the optimal rate under the RKHS norm Ying and Pontil (2008); Tarres and Yao (2014). If the embedding property (Assumption 9) holds for some  $\alpha < 1$ , we choose  $\gamma = \alpha$  to achieve a faster rate  $\mathcal{O}(t^{-(\beta -\alpha) / (\beta -\alpha +2)})$  (which further leads to Theorem 2). To prove Lemma 4, we derive novel proof steps, where we decouple the power norm by considering semi-population iteration and recursive decomposition, we refer the reader to Appendix E.1 for more explanations.
We demonstrate the parameters involved in Theorem 2. Parameters  $\alpha, \beta, \kappa, \tau, A$  are intrinsic:  $\beta, \kappa, A$  are determined by source condition (Assumption 8),  $\alpha$  is determined by embedding property (Assumption 9), and  $\tau$  is the strongly monotone parameter. Parameters  $a, t_0$  are user-specified:  $t_0$  is a sufficiently large value,  $a$  is characterized by the inequality  $a < \sqrt{(\beta - \alpha + 2) / (\beta - \alpha)} (t_0 + 1) / (t_0 + 2)\kappa^{\alpha - 2}A^{-1}$  when  $t_0$  is determined, a smaller  $a$  leads to a larger constant term in (13). Theorem 2 implies that OPGD leverages the embedding property (Assumption 9) to obtain better convergence rates. For any bounded kernel, Assumption 9 holds for  $\alpha = 1$ , thus Theorem 2 guarantees the rate  $\mathcal{O}(t^{-\frac{\beta - 1}{\beta + 1}})$ . Moreover, suppose that the kernel satisfies some good embedding property, that is,  $\alpha < 1$ , since larger  $\beta - \alpha$  leads to faster convergence rates. In that case, we obtain a better rate  $\mathcal{O}(t^{-\frac{\beta - \alpha}{\beta - \alpha + 2}})$  by setting the gradient steps and regularization terms  $\nu_t, \lambda_t$  corresponding to  $\alpha, \beta$ . Besides, OPGD can handle the challenging scenario ( $f_i \notin (\mathcal{H})^p$  if  $\beta < 1$ ) when the embedding property of kernel holds for  $\alpha < \beta$ .
In this section, we conduct experiments on decision-dependent games in both the linear and the RKHS settings to verify our theory. All experiments are conducted with Python on a laptop using 14 threads of a 12th Gen Intel(R) Core(TM) i7-12700H CPU.
Basic Setup. We consider two-agent decision-dependent games with 1-dimensional actions. Namely, for all  $i \in [2]$ , define the game
$$
\min  _ {x \in \mathcal {X}} \mathcal {L} _ {i} (x), \quad \text {w h e r e} \quad \mathcal {L} _ {i} (x) := \underset {z _ {i} \sim \mathcal {D} _ {i} (x)} {\mathbb {E}} \ell_ {i} (x, z _ {i}), \tag {14}
$$
where  $\mathcal{X} = [0,1]\times [0,1],x\in \mathcal{X},z_i\in \mathbb{R}$ , and  $\ell_{i}(x,z_{i})$  is the loss function to be determined. Let the distribution map be  $\mathcal{D}_i(x)\sim \mathcal{N}(f_i(x),0.2)$ , where  $f_{i}$  is the parametric function determined by the specific function class. Then the game (14) follows the parametric assumption (Assumption 1) with  $z_{i} = f_{i}(x) + \epsilon_{i}$ , where  $\epsilon_{i}\sim \mathcal{N}(0,0.2)$  the independent Gaussian noise term.
(b)
Figure 1: (a) Linear setting: The X-axis represents the iteration from 1 to 10,000, while the Y-axis represents the norm-squared error of  $x^{t}$  to the Nash equilibrium  $x^{*} = (1 / 2,1)$ , averaged over 20 random seeds. The blue solid line represents the output of OPGD and the orange dashed line represents the theoretical rate  $\mathcal{O}(t^{-1})$ . (b) RKHS setting: The X-axis represents the iteration from 1 to 10,000, while the Y-axis represents the norm-squared error to the Nash equilibrium  $x^{*} = (1 / 2,1 / 2)$ , averaged over 400 random seeds. The blue solid line represents the output of OPGD and the orange dashed line represents the theoretical rate  $\mathcal{O}(t^{-1 / 2})$ .
Linear function class. Let the loss function be  $\ell_i(x,z_i) = -z_i + x_i^2$  and set the linear parametric function as  $f_{1}(x) = x_{1}$  and  $f_{2}(x) = 2x_{2}$ , namely, the parametric model is  $z_{i} = A_{i}x + \epsilon_{i}$  where  $A_{1} = [10]$  and  $A_{2} = [02]$ . The game (14) has the gradient  $H(x) = (2x_{1} - 1,2x_{2} - 2)$ , therefore, the game (14) is convex,  $C^1$ -smooth, 1-strongly monotone and the Nash equilibrium is  $x^{*} = (1 / 2,1)$ . We set the sampling distribution as  $\rho_{\mathcal{X}} = \mathcal{U}[0,1]\times \mathcal{U}[0,1]$ , the initial point as  $x^0\sim \rho_\mathcal{X}$ , and the initial estimation as zero. Moreover, letting  $t_0 = 10$ , we set the gradient step sizes as  $\eta_t = 6 / (t + t_0),\nu_t = 6 / (t + t_0)$ .
Kernel function class. Let  $\mathcal{X} = [0,1] \times [0,1]$ ,  $\rho_{\mathcal{X}} = \mathcal{U}[0,1] \times \mathcal{U}[0,1]$ , and define the kernel  $Q((x_1,x_2),(y_1,y_2)) = K(x_1,y_1) \cdot K(x_2,y_2)$  as the product kernel of  $K(x,y) = 40B_4(\{x - y\})$ . Suppose that  $\mathcal{H}$  is the RKHS on  $\mathcal{X}$  induced by the kernel  $Q$  and the distribution  $\rho_{\mathcal{X}}$ . Set the parametric function as the product of two 3-order Bernoulli polynomials, namely,  $f(x_1,x_2) = B_3(x_1) \cdot B_3(x_2) = (x_1^3 - 3x_1^2 / 2 + x_1 / 2) \cdot (x_2^3 - 3x_2^2 / 2 + x_2 / 2)$ . Set  $\ell_i(x,z_i) = -z_i + \cos(2\pi x_1) \cos(2\pi x_2) - x_i + x_i^2$  and let  $f_i(x) = \cos(2\pi x_1) \cos(2\pi x_2)$  for  $i \in [2]$ . Then the gradient of this game is  $H(x) = (2x_1 - 1, 2x_2 - 1)$ , thus, this game is convex,  $C^1$ -smooth, 1-strongly monotone and the Nash equilibrium is  $x^* = (0.5, 0.5)$ . Following Example 1, Assumption 8, 9, 10 hold for any  $\beta > 1$  and any  $\alpha > 1/4$ . Set  $t_0 = 10$ ,  $a = 7$ ,  $\eta_t = 6/(t+t_0)$ ,  $\nu_t = a/(t+t_0)^{3/4}$ , and  $\lambda_t = 1/(a(t+t_0)^{1/4})$ . Following Theorem 2, the convergence rate is  $\mathcal{O}(t^{-1/2})$ .
Results. We perform experiments for both parametric settings to verify the convergence rates and compare the theoretical and simulated rates, as shown in Figure 1, where both X and Y axes take the log scale. Figure 1(a) shows the converge rate of the linear setting within 10,000 iterations, the simulated rate matches our prediction, i.e. it is close to  $\mathcal{O}(t^{-1})$ . Figure 1(b) shows the convergence rate of the RKHS setting, it implies that the simulated rate is close to the theoretical rate  $\mathcal{O}(t^{-1/2})$  when the iteration  $t$  is larger than 1,000. These results validate Theorems 1 and 2.