SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Finally, although the focus of this paper is theoretical, in this section we provide some illustrative experimental examples. In particular, Appendix B.1 contains experiments on time-varying potential games, while Appendix B.2 focuses on time-varying (two-player) zero-sum games. For simplicity, we will be assuming that each game is represented in normal form.
B.1 Time-varying potential games
Here we consider time-varying 2-player identical-interest games. We point out that such games are potential games (recall Definition A.23), and as such, they are indeed amenable to our theory in Section 3.3.
In our first experiment, we first sampled two matrices  $\mathbf{A}, \mathbf{P} \in \mathbb{R}^{d_x \times d_y}$ , where  $d_x = d_y = 1000$ . Then, we defined each payoff matrix as  $\mathbf{A}^{(t)} \coloneqq \mathbf{A}^{(t-1)} + \mathbf{P} t^{-\alpha}$  for  $t \geq 1$ , where  $\mathbf{A}^{(0)} \coloneqq \mathbf{A}$ . Here,  $\alpha > 0$  is a parameter that controls the variation of the payoff matrices. In this time-varying setup, we let each player employ (online) GD with learning rate  $\eta \coloneqq 0.1$ . The results obtained under different random initializations of matrices  $\mathbf{A}$  and  $\mathbf{P}$  are illustrated in Figure 1.
Next, we operate in the same time-varying setup but each player is now employing multiplicative weights update (MWU), instead of gradient descent, with  $\eta \coloneqq 0.1$ . As shown in Figure 2, while the cumulative equilibrium gap is much larger compared to using GD (Figure 1), the dynamics still appear to be approaching equilibria, although our theory does not cover MWU. We suspect that theoretical results such as Theorem 3.7 should hold for MWU as well, but that has been left for future work.
In our third experiment for identical-interest games, we again first sampled two matrices  $\mathbf{A},\mathbf{P}\in$ $\mathbb{R}^{d_x\times d_y}$ , where  $d_{x} = d_{y} = 1000$ . Then, we defined  $\mathbf{A}^{(t)}\coloneqq \mathbf{A}^{(t - 1)} + \epsilon \mathbf{P}$  for  $t\geq 1$ , where  $\mathbf{A}^{(0)}\coloneqq \mathbf{A}$ . Here,  $\epsilon >0$  is the parameter intended to capture the variation of the payoff matrices. The results obtained under different random initializations of  $\mathbf{A}$  and  $\mathbf{P}$  are illustrated in Figure 3. As an aside, it is worth pointing out that this particular setting can be thought of as a game in which the variation in the payoff matrices is controlled by another learning agent. In particular, our theoretical results could be helpful for characterizing the convergence properties of two-timescale learning algorithms, in which the deviation of the game is controlled by a player constrained to be updating its strategies with a much smaller learning rate.
B.2 Time-varying zero-sum games
We next conduct experiments on time-varying bilinear saddle-point problems when players are employing OG. Such problems were studied extensively earlier in Section 3.1 from a theoretical standpoint.
First, we sampled two matrices  $\mathbf{A},\mathbf{P}\in \mathbb{R}^{d_x\times d_y}$ , where  $d_{x} = d_{y} = 10$ ; here we consider lower-dimensional payoff matrices compared to the experiments in Appendix B.1 for convenience in the graphical illustrations. Then, we defined each payoff matrix as  $\mathbf{A}^{(t)}\coloneqq \mathbf{A}^{(t - 1)} + \mathbf{P}t^{-\alpha}$  for  $t\geq 1$  where  $\mathbf{A}^{(1)}\coloneqq \mathbf{A}$ . The results obtained under different random initializations are illustrated in Figure 4.
Figure 1: The equilibrium gap and the players' regrets in 2-player time-varying identical-interest games when both players are employing (online) GD with learning rate  $\eta \coloneqq 0.1$  for  $T \coloneqq 200$  iterations. Each row corresponds to a different random initialization of the matrices  $\mathbf{A}$ ,  $\mathbf{P} \in \mathbb{R}^{d_x \times d_y}$ , which in turn induces a different time-varying game. Further, each figure contains trajectories corresponding to three different values of  $\alpha \in \{0.1, 0.2, 0.5\}$ , but under the same initialization of  $\mathbf{A}$  and  $\mathbf{P}$ . As expected, smaller values of  $\alpha$  generally increase the equilibrium gap since the variation of the games is more significant. Nevertheless, for all games we observe that the players are gradually approaching equilibria.
Figure 2: The equilibrium gap and the players' regrets in 2-player time-varying identical-interest games when both players are employing (online) GD with learning rate  $\eta := 0.1$  for  $T := 200$  iterations. Each row corresponds to a different random initialization of the matrices  $\mathbf{A}, \mathbf{P} \in \mathbb{R}^{d_x \times d_y}$ , which in turn induces a different time-varying game. Further, each figure contains trajectories corresponding to three different values of  $\alpha \in \{0.1, 0.2, 0.5\}$ , but under the same initialization of  $\mathbf{A}$  and  $\mathbf{P}$ . The MwU dynamics still appear to be approaching equilibria, although the cumulative gap is much larger compared to GD (Figure 1).
Figure 3: The equilibrium gap and the players' regrets in 2-player time-varying identical-interest games when both players are employing (online) GD with learning rate  $\eta := 0.1$  for  $T := 500$  iterations. Each row corresponds to a different random initialization of the matrices  $\mathbf{A}$ ,  $\mathbf{P} \in \mathbb{R}^{d_x \times d_y}$ , which in turn induces a different time-varying game. Further, each figure contains trajectories from three different values of  $\epsilon \in \{0.1, 0.01, 0.001\}$ , but under the same initialization of  $\mathbf{A}$  and  $\mathbf{P}$ . As expected, larger values of  $\epsilon$  generally increase the equilibrium gap since the variation of the games is more significant. Yet, even for the larger value  $\epsilon = 0.1$ , the dynamics are still appear to be approaching Nash equilibria.
Figure 4: The equilibrium gap and the players' regrets in 2-player time-varying zero-sum games when both players are employing OGD with learning rate  $\eta := 0.01$  and  $T := 1000$  iterations. Each row corresponds to a different random initialization of the matrices  $\mathbf{A}$ ,  $\mathbf{P} \in \mathbb{R}^{d_x \times d_y}$ , which in turn induces a different time-varying game. Further, each figure contains trajectories from three different values of  $\alpha \in \{0.7, 1, 2\}$ , but under the same initialization of  $\mathbf{A}$  and  $\mathbf{P}$ . The OGD dynamics appear to be approaching equilibria, albeit with a much slower rate compared to the ones observed earlier for potential games (Figure 1).