SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 3: Duality gap as a function of the number of full (or equivalent) gradient computations for when restarting is applied to ECyclicPDA with different block construction strategies. We take the best duality gap seen so far so that the plot is monotonic.
Figure 4: Duality gap as a function of the number of full (or equivalent) gradient computations for when restarting is applied to ECyclicPDA, MP,  $\mathrm{CFR}^{+}$ ,  $\mathrm{PCFR}^{+}$ . We take the best duality gap seen so far so that the plot is monotonic.
Table 1: The average wall clock time per iteration in milliseconds of ECyclicPDA instantiated with different block construction strategies, MP,  $\mathrm{CFR}^+$ , and  $\mathrm{PCFR}^+$ . The duality gap is computed every 100 iterations.
Name	Goofspiel (4 ranks)	Liar's Dice	Leduc (13 ranks)	Battleship
ECyclicPDA single block	2.470	11.370	6.770	46.020
ECyclicPDA children	4.270	15.020	7.450	47.230
ECyclicPDA infosets	8.850	16.240	7.370	53.490
ECyclicPDA postorder	4.340	14.350	7.430	49.020
MP	3.720	19.530	10.360	83.450
CFR+	1.330	1.360	0.370	7.980
Predictive CFR+	1.750	1.920	0.430	9.410
Discussion. We develop the first cyclic block-coordinate-like method for two-player zero-sum EFGs. Our algorithm relies on the recursive nature of the prox updates for dilated regularizers, cycling through blocks that respect the partial order induced on decision points by the treeplex, and extrapolation to conduct pseudo-block updates, produce feasible iterates, and achieve  $O\left(\frac{1}{T}\right)$  ergodic convergence. Furthermore, the runtime of our algorithm has no dependence on the number of blocks. We present empirical evidence that our algorithm generally outperforms MP, and is the first FOM to compete with  $\mathrm{CFR}^{+}$  and  $\mathrm{PCFR}^{+}$  on non-trivial EFGs. Finally, we introduce a restarting heuristic for EFG solving, and demonstrate often huge gains in convergence rate. An open question raised by our work is understanding what makes restarting work for methods used with regularizers besides the  $\ell_{2}$  regularizer (the only setting for which there exist linear convergence guarantees). This may be challenging because existing proofs require upper bounding the corresponding Bregman divergence (for a given non-  $\ell_{2}$  regularizer) between iterates by the distance to optimality. This is difficult for entropy or any dilated regularizer because the initial iterate used by the algorithm after restarting may have entries arbitrarily close to zero even if they are guaranteed to not exactly be zero (as is the case for entropy). Relatedly, both our block-coordinate method and restarting have a much bigger advantage in some numerical instances (Battleship, Liar's Dice) than others (Leduc and Goofspiel); a crucial question is to understand what type of game structure drives this behavior.
Darshan Chakrabarti was supported by National Science Foundation Graduate Research Fellowship Program under award number DGE-2036197. Jelena Diakonikolas was supported by the Office of Naval Research under award number N00014-22-1-2348. Christian Kroer was supported by the Office of Naval Research awards N00014-22-1-2530 and N00014-23-1-2374, and the National Science Foundation awards IIS-2147361 and IIS-2238960.
[1] Aviad Aberdam and Amir Beck. An accelerated coordinate gradient descent algorithm for non-separable composite optimization. Journal of Optimization Theory and Applications, 193 (1-3):219-246, 2021.
[2] Ahmet Alacaoglu, Quoc Tran Dinh, Olivier Fercoq, and Volkan Cevher. Smooth primal-dual coordinate descent algorithms for nonsmooth convex optimization. In Advances in Neural Information Processing Systems, 2017.
[3] Zeyuan Allen-Zhu, Zheng Qu, Peter Richtárik, and Yang Yuan. Even faster accelerated coordinate descent using non-uniform sampling. In Proceedings of International Conference on Machine Learning, 2016.
[4] Ioannis Anagnostides, Gabriele Farina, and Tuomas Sandholm. Near-optimal  $\phi$ -regret learning in extensive-form games. arXiv preprint arXiv:2208.09747, 2022.
[5] David Applegate, Oliver Hinder, Haihao Lu, and Miles Lubin. Faster first-order primal-dual methods for linear programming using restarts and sharpness. arXiv preprint arXiv:2105.12715, 2022.
[6] Yu Bai, Chi Jin, Song Mei, Ziang Song, and Tiancheng Yu. Efficient phi-regret minimization in extensive-form games via online mirror descent. In Advances in Neural Information Processing Systems, 2022.
[7] Amir Beck and Luba Tetruashvili. On the convergence of block coordinate descent type methods. SIAM Journal on Optimization, 23(4):2037-2060, 2013.
[8] Noam Brown and Tuomas Sandholm. Solving imperfect-information games via discounted regret minimization. In Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
[9] Yair Carmon, Yujia Jin, Aaron Sidford, and Kevin Tian. Coordinate methods for matrix games. arXiv preprint arXiv:2009.08447, 2020.
[10] Flavia Chorobura and Ion Necoara. Random coordinate descent methods for nonseparable composite optimization. arxiv preprint arXiv:2203.14368, 2022.
[11] Jelena Diakonikolas and Lorenzo Orecchia. Alternating randomized block coordinate descent. In Proceedings of International Conference on Machine Learning, 2018.
[12] Gabriele Farina, Christian Kroer, Noam Brown, and Tuomas Sandholm. Stable-predictive optimistic counterfactual regret minimization. In Proceedings of the International Conference on Machine Learning, 2019.
[13] Gabriele Farina, Christian Kroer, and Tuomas Sandholm. Optimistic regret minimization for extensive-form games via dilated distance-generating functions. In Advances in Neural Information Processing Systems, 2019.
[14] Gabriele Farina, Christian Kroer, and Tuomas Sandholm. Better regularization for sequential decision spaces: Fast convergence rates for Nash, correlated, and team equilibria. In Proceedings of the ACM Conference on Economics and Computation, 2021.
[15] Gabriele Farina, Christian Kroer, and Tuomas Sandholm. Faster game solving via predictive blackwell approachability: Connecting regret matching and mirror descent. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[33] Arkadi Nemirovski. Prox-method with rate of convergence o (1/t) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15(1):229-251, 2004.
[34] Yu. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2):341-362, 2012.
[35] Yurii Nesterov. Dual extrapolation and its applications to solving variational inequalities and related problems. Mathematical Programming, 109(2-3):319-344, 2007.
[36] Sheldon M. Ross. Goofspiel — the game of pure strategy. Journal of Applied Probability, 8(3): 621-625, 1971.
[37] Ankan Saha and Ambuj Tewari. On the nonasymptotic convergence of cyclic coordinate descent methods. SIAM Journal on Optimization, 23(1):576-601, 2013.
[38] Hao-Jun Michael Shi, Shenyinying Tu, Yangyang Xu, and Wotao Yin. A primer on coordinate descent algorithms. arXiv preprint arXiv:1610.00040, 2016.
[39] Chaobing Song and Jelena Diakonikolas. Fast cyclic coordinate dual averaging with extrapolation for generalized variational inequalities. arXiv preprint arXiv:2102.13244, 2021.
[40] Finnegan Southey, Michael P. Bowling, Bryce Larson, Carmelo Piccione, Neil Burch, Darse Billings, and Chris Rayner. Bayes' bluff: Opponent modelling in poker. arXiv preprint arXiv:1207.1411, 2012.
[41] Oskari Tammelin. Solving large imperfect information games using CFR+. arXiv preprint arXiv:1407.5042, 2014.
[42] Oskari Tammelin, Neil Burch, Michael Johanson, and Michael Bowling. Solving heads-up limit Texas hold'em. In Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.
[43] Paul Tseng. On linear convergence of iterative methods for the variational inequality problem. Journal of Computational and Applied Mathematics, 60(1-2):237-252, 1995.
[44] Bernhard von Stengel. Efficient computation of behavior strategies. Games and Economic Behavior, 14(2):220-246, 1996.
[45] Chen-Yu Wei, Chung-Wei Lee, Mengxiao Zhang, and Haipeng Luo. Linear last-iterate convergence in constrained saddle-point optimization. In Proceedings of International Conference on Learning Representations, 2021.
[46] Stephen J. Wright. Coordinate descent algorithms. Mathematical Programming, 151(1):3-34, 2015.
[47] Tong Tong Wu and Kenneth Lange. Coordinate descent algorithms for lasso penalized regression. The Annals of Applied Statistics, 2(1):224-244, 2008.
[48] Yuchen Zhang and Xiao Lin. Stochastic primal-dual coordinate method for regularized empirical risk minimization. In Proceedings of International Conference on Machine Learning, 2015.
[49] Martin Zinkevich, Michael Johanson, Michael Bowling, and Carmelo Piccione. Regret minimization in games with incomplete information. In Advances in Neural Information Processing Systems, 2007.
There has been significant work on FOMs for two-player zero-sum EFG solving. Because this is a BSPP, off-the-shelf FOMs for BSPPs can be applied, with the caveat that proximal oracles are required. The standard Euclidean distance has been used in some cases [21], but it requires solving a projection problem that takes  $O(n \log^2 n)$  time, where  $n$  is the dimension of a player's decision space [16, 21]. While this is "nearly" linear time, such projections have not been used much in practice. Proximal oracles have instead been based on dilated regularizers [23], which lead to a proximal update that can be performed with a single pass over the decision space. With the dilated entropy regularizer, this can be performed in linear time, and this regularizer leads to the strongest bounds on game constants that impact the convergence rate of proximal-oracle-based FOMs [14, 25]. More recently, it has been shown that a specialized kernelization can be used to achieve linear-time proximal updates and stronger convergence rates specifically for the dilated entropy with optimistic online mirror descent through a correspondence with optimistic multiplicative weights on the exponentially-many vertices of the decision polytope [6, 17]. Yet this approach was shown to have somewhat disappointing numerical performance in Farina et al. [17], and thus is less important practically despite its theoretical significance.
A completely different approach for first-order-based updates is the CFR framework, which decomposes regret minimization on the EFG decision sets into local simplex-based regret minimization [49]. In theory, CFR-based results have mostly led to an inferior  $T^{-1/2}$  rate of convergence, but in practice the CFR framework instantiated with regret matching $^{+}$  ( $\mathrm{RM^{+}}$ ) [41] or predictive  $\mathrm{RM^{+}}$  ( $\mathrm{PRM^{+}}$ ) [15] is the fastest approach for essentially every EFG setting.  $\mathrm{RM^{+}}$  is often fastest for "poker-like" EFGs, while  $\mathrm{PRM^{+}}$  is often fastest for other classes of games [15]. Improved rates on the order of  $T^{-3/4}$  [12] and  $\log T/T$  [4] have been achieved within the CFR framework, but only while using regret minimizers that lead to significantly worse practical performance (in particular, numerically these perform worse than the best  $1/T$  FOMs such as mirror prox with appropriate stepsize tuning).
In the last few years there has been a growing literature on last-iterate convergence in EFGs. There, the goal is to show that one can converge to an equilibrium without averaging the iterates generated by a FOM or CFR-based method. It has long been known that with the Euclidean regularizer, it is possible to converge at a linear rate in last iterate with e.g., the extragradient method (a.k.a. mirror prox with the Euclidean regularizer) on BSPPs with polyhedral decision sets, as they are in EFGs [21, 43, 45]. More recently, it has been shown that a linear rate can be achieved with certain dilated regularizers [27], with the kernelization approach of Farina et al. [17], and in a regularized CFR setup [31]. At this stage, however, these last-iterate results are of greater theoretical significance than practical significance, because the linear rate often does not occur until after quite many iterations, and typically the methods do not match the performance of ergodic methods at reasonable time scales. For this reason, we do not compare to last-iterate algorithms in our experiments.
B Treeplex Example
As an example, consider the treeplex of Kuhn poker [26] adapted from [13] shown in Figure 5. Kuhn poker is a game played with a three card deck: jack, queen, and king. In this case, for example, we have  $\mathcal{J}_{\mathcal{X}} = \{0,1,2,3,4,5,6\}$ ,  $p_0 = \emptyset$ ,  $p_1 = p_2 = p_3 = (0,\mathrm{start})$ ,  $p_4 = (1,\mathrm{check})$ ,  $p_5 = (2,\mathrm{check})$ ,  $p_6 = (3,\mathrm{check})$ ,  $A_0 = \{\mathrm{start}\}$ ,  $A_1 = A_2 = A_3 = \{\mathrm{check},\mathrm{raise}\}$ ,  $A_4 = A_5 = A_6 = \{\mathrm{fold},\mathrm{call}\}$ ,  $\mathcal{C}_{(0,\mathrm{start})} = \{1,2,3\}$ ,  $\mathcal{C}_{(1,\mathrm{raise})} = \mathcal{C}_{(1,\mathrm{raise})} = \mathcal{C}_{(2,\mathrm{raise})} = \mathcal{C}_{(3,\mathrm{raise})} = \mathcal{C}_{(4,\mathrm{fold})} = \mathcal{C}_{(5,\mathrm{fold})} = \mathcal{C}_{(6,\mathrm{fold})} = \mathcal{C}_{(4,\mathrm{call})} = \mathcal{C}_{(5,\mathrm{call})} = \mathcal{C}_{(6,\mathrm{call})} = \emptyset$ ,  $\downarrow 0 = \mathcal{J}_{\mathcal{X}}, \downarrow 1 = \{1,4\}$ ,  $\downarrow 2 = \{2,5\}$ ,  $\downarrow 3 = \{3,6\}$ ,  $\downarrow 4 = \{4\}$ ,  $\downarrow 5 = \{5\}$ ,  $\downarrow 6 = \{6\}$ . In this case,  $\emptyset$  represents the empty sequence.
C.1 Proof of Proposition 3.1
Proof. The claim that  $\mathbf{x}_k\in \mathcal{X}$ ,  $\mathbf{y}_k\in \mathcal{Y}$  is immediate from the algorithm description, as both are solutions to constrained optimization problems with these same constraints.
Figure 5: The sequential decision problem for the first player in Kuhn poker.  $\bullet$  represents the end of the decision process and  $\otimes$  represents an observation (which may lead to multiple decision points). Adapted from [13].
For the remaining claim, observe first that
$$
\eta_ {k} \left\langle \mathbf {M} \mathbf {x} _ {k}, \mathbf {v} \right\rangle = \eta_ {k} \left\langle \mathbf {g} _ {k}, \mathbf {v} \right\rangle - D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k - 1}) + D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k - 1}) + \eta_ {k} \left\langle \mathbf {M} \mathbf {x} _ {k} - \mathbf {g} _ {k}, \mathbf {v} \right\rangle .
$$
Recall from Algorithm 1 that
$$
\mathbf {y} _ {k} = \underset {\mathbf {v} \in \mathcal {Y}} {\operatorname {a r g m a x}} \left\{\eta_ {k} \left\langle \mathbf {g} _ {k}, \mathbf {v} \right\rangle - D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k - 1}) \right\}.
$$
Define the function under the max defining  $\mathbf{y}_k$  by  $\Psi_k$ . Then as  $\Psi_k(\cdot)$  is the sum of  $-\psi(\cdot)$  and linear terms, we have  $D_{\Psi_k}(\cdot, \mathbf{y}) = -D_{\psi}(\cdot, \mathbf{y})$ , for any  $\mathbf{y}$ . Further, as  $\Psi_k$  is maximized by  $\mathbf{y}_k$ , we have  $\Psi_k(\mathbf{v}) \leq \Psi_k(\mathbf{y}_k) - D_{\psi}(\mathbf{v}, \mathbf{y}_k)$ . Thus, it follows that
$$
\begin{array}{l} \eta_ {k} \left\langle \mathbf {M} \mathbf {x} _ {k}, \mathbf {v} \right\rangle \leq \eta_ {k} \left\langle \mathbf {g} _ {k}, \mathbf {y} _ {k} \right\rangle - D _ {\psi} \left(\mathbf {y} _ {k}, \mathbf {y} _ {k - 1}\right) + \left\langle \mathbf {M} \mathbf {x} _ {k} - \mathbf {g} _ {k}, \mathbf {v} \right\rangle \tag {C.1} \\ - D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k}) + D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k - 1}). \\ \end{array}
$$
Using the same ideas for the primal side, we have
$$
\begin{array}{l} \eta_ {k} \left\langle \mathbf {M} \mathbf {u}, \mathbf {y} _ {k} \right\rangle \geq \eta_ {k} \left\langle \mathbf {x} _ {k}, \mathbf {h} _ {k} \right\rangle + D _ {\phi} \left(\mathbf {x} _ {k}, \mathbf {x} _ {k - 1}\right) + \eta_ {k} \left\langle \mathbf {u}, \mathbf {M} ^ {\top} \mathbf {y} _ {k} - \mathbf {h} _ {k} \right\rangle \tag {C.2} \\ + D _ {\phi} (\mathbf {u}, \mathbf {x} _ {k}) - D _ {\phi} (\mathbf {u}, \mathbf {x} _ {k - 1}) \\ \end{array}
$$
Combining (C.1) and (C.2),
$$
\begin{array}{l} \eta_ {k} \operatorname {G a p} ^ {\mathbf {u}, \mathbf {v}} \left(\mathbf {x} _ {k}, \mathbf {y} _ {k}\right) \leq \eta_ {k} \left\langle \mathbf {M} \mathbf {x} _ {k} - \mathbf {g} _ {k}, \mathbf {v} - \mathbf {y} _ {k} \right\rangle - \eta_ {k} \left\langle \mathbf {u} - \mathbf {x} _ {k}, \mathbf {M} ^ {\top} \mathbf {y} _ {k} - \mathbf {h} _ {k} \right\rangle \\ - D _ {\psi} (\mathbf {y} _ {k}, \mathbf {y} _ {k - 1}) - D _ {\phi} (\mathbf {x} _ {k}, \mathbf {x} _ {k - 1}) \\ - D _ {\phi} (\mathbf {u}, \mathbf {x} _ {k}) + D _ {\phi} (\mathbf {u}, \mathbf {x} _ {k - 1}) - D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k}) + D _ {\psi} (\mathbf {v}, \mathbf {y} _ {k - 1}). \\ \end{array}
$$
To complete the proof, it remains to combine the definition of  $\mathcal{E}_k$  from the proposition statement with the last inequality.
C.2 Proof of Theorem 3.2
For notational convenience, in this proof we define vectors  $\hat{\mathbf{x}}_k$  and  $\hat{\mathbf{y}}_k$  by  $\hat{\mathbf{x}}_k^j = \frac{\mathbf{x}_k^j}{x_k^{p_j}} x_{k - 1}^{p_j}$  for  $j\in \mathcal{J}_{\mathcal{X}}$  and  $\hat{\mathbf{y}}_k^j = \frac{\mathbf{y}_k^j}{y_k^{p_j}} y_{k - 1}^{p_j}$  for  $j\in \mathcal{J}_{\mathcal{Y}}$ , so that  $\widetilde{\mathbf{x}}_k = \mathbf{x}_k - \hat{\mathbf{x}}_k - \frac{\eta_{k - 1}}{\eta_k} (\mathbf{x}_{k - 1} - \hat{\mathbf{x}}_{k - 1})$  and  $\widetilde{\mathbf{y}}_k = \mathbf{y}_k - \hat{\mathbf{y}}_k - \frac{\eta_{k - 1}}{\eta_k} (\mathbf{y}_{k - 1} - \hat{\mathbf{y}}_{k - 1})$ .
To prove the theorem, we first prove the following auxiliary lemma which bounds the inner product terms appearing in the error terms  $\mathcal{E}_k$ .
Lemma C.1. In all iterations  $k$  of Algorithm 1, for any  $(\mathbf{u},\mathbf{v})\in \mathcal{X}\times \mathcal{Y}$  and any  $\alpha ,\beta >0$
$$
\begin{array}{l} \eta_ {k} \left\langle \mathbf {M} \mathbf {x} _ {k} - \mathbf {g} _ {k}, \mathbf {v} - \mathbf {y} _ {k} \right\rangle \leq \eta_ {k} \left\langle \mathbf {M} _ {\mathbf {y}} \left(\mathbf {x} _ {k} - \hat {\mathbf {x}} _ {k}\right), \mathbf {v} - \mathbf {y} _ {k} \right\rangle - \eta_ {k - 1} \left\langle \mathbf {M} _ {\mathbf {y}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right), \mathbf {v} - \mathbf {y} _ {k - 1} \right\rangle \\ + \eta_ {k} \left\langle \mathbf {M} _ {\mathbf {x}} \left(\mathbf {x} _ {k} - \mathbf {x} _ {k - 1}\right), \mathbf {v} - \mathbf {y} _ {k} \right\rangle - \eta_ {k - 1} \left\langle \mathbf {M} _ {\mathbf {x}} \left(\mathbf {x} _ {k - 1} - \mathbf {x} _ {k - 2}\right), \mathbf {v} - \mathbf {y} _ {k - 1} \right\rangle \\ + \eta_ {k - 1} \frac {\| \mathbf {M _ {x}} \| _ {*} + \| \mathbf {M _ {y}} \| _ {*}}{2} \left(\alpha \| \mathbf {x} _ {k - 1} - \mathbf {x} _ {k - 2} \| ^ {2} + \frac {1}{\alpha} \| \mathbf {y} _ {k} - \mathbf {y} _ {k - 1} \| ^ {2}\right). \\ \end{array}
$$
$$
\begin{array}{l} - \eta_ {k} \left\langle \mathbf {u} - \mathbf {x} _ {k}, \mathbf {M} ^ {\top} \mathbf {y} _ {k} - \mathbf {h} _ {k} \right\rangle \leq - \eta_ {k} \left\langle \mathbf {M} _ {\mathbf {x}} ^ {\top} (\mathbf {y} _ {k} - \hat {\mathbf {y}} _ {k}), \mathbf {u} - \mathbf {x} _ {k} \right\rangle + \eta_ {k - 1} \left\langle \mathbf {M} _ {\mathbf {x}} ^ {\top} (\mathbf {y} _ {k - 1} - \hat {\mathbf {y}} _ {k - 1}), \mathbf {u} - \mathbf {x} _ {k - 1} \right\rangle \\ - \eta_ {k} \left\langle \mathbf {M} _ {\mathbf {y}} ^ {\top} (\mathbf {y} _ {k} - \mathbf {y} _ {k - 1}), \mathbf {u} - \mathbf {x} _ {k} \right\rangle + \eta_ {k - 1} \left\langle \mathbf {M} _ {\mathbf {y}} ^ {\top} (\mathbf {y} _ {k - 1} - \mathbf {y} _ {k - 2}), \mathbf {u} - \mathbf {x} _ {k - 1} \right\rangle \\ + \eta_ {k - 1} \frac {\| \mathbf {M _ {x} ^ {\top}} + \mathbf {M _ {y} ^ {\top}} \| _ {*}}{2} \Big (\beta \| \mathbf {x} _ {k - 1} - \mathbf {x} _ {k} \| ^ {2} + \frac {1}{\beta} \| \mathbf {y} _ {k - 1} - \mathbf {y} _ {k - 2} \| ^ {2} \Big). \\ \end{array}
$$
Proof. Observe first that, by Algorithm 1,
$$
\begin{array}{l} \mathbf {M} ^ {(t, :)} \mathbf {x} _ {k} - \mathbf {g} _ {k} ^ {(t)} = \mathbf {M} ^ {(t, 1: t)} \left(\mathbf {x} _ {k} - \hat {\mathbf {x}} _ {k} - \frac {\eta_ {k - 1}}{\eta_ {k}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right)\right) ^ {(1: t)} \tag {C.3} \\ + \mathbf {M} ^ {(t, t + 1: s)} \left(\mathbf {x} _ {k} - \mathbf {x} _ {k - 1} - \frac {\eta_ {k - 1}}{\eta_ {k}} (\mathbf {x} _ {k - 1} - \mathbf {x} _ {k - 2})\right) ^ {(t + 1: s)}. \\ \end{array}
$$
Additionally, by definition (see Eq. (3.1)),  $\sum_{t=1}^{s} \mathbf{M}_{t,1:t} = \mathbf{M} - \mathbf{M}_{\mathbf{x}} = \mathbf{M}_{\mathbf{y}}$ . Hence,
$$
\begin{array}{l} \eta_ {k} \sum_ {t = 1} ^ {s} \left\langle \mathbf {M} ^ {(t, 1: t)} \left(\mathbf {x} _ {k} - \hat {\mathbf {x}} _ {k} - \frac {\eta_ {k - 1}}{\eta_ {k}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right)\right) ^ {(1: t)}, \mathbf {v} ^ {(t)} - \mathbf {y} _ {k} ^ {(t)} \right\rangle \\ = \eta_ {k} \sum_ {t = 1} ^ {s} \left\langle \mathbf {M} _ {t, 1: t} \left(\mathbf {x} _ {k} - \hat {\mathbf {x}} _ {k} - \frac {\eta_ {k - 1}}{\eta_ {k}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right)\right), \mathbf {v} - \mathbf {y} _ {k} \right\rangle \\ = \eta_ {k} \left\langle \mathbf {M} _ {\mathbf {y}} \left(\mathbf {x} _ {k} - \hat {\mathbf {x}} _ {k} - \frac {\eta_ {k - 1}}{\eta_ {k}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right)\right), \mathbf {v} - \mathbf {y} _ {k} \right\rangle \\ = \eta_ {k} \left\langle \mathbf {M} _ {\mathbf {y}} \left(\mathbf {x} _ {k} - \hat {\mathbf {x}} _ {k}\right), \mathbf {v} - \mathbf {y} _ {k} \right\rangle - \eta_ {k - 1} \left\langle \mathbf {M} _ {\mathbf {y}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right), \mathbf {v} - \mathbf {y} _ {k - 1} \right\rangle \\ - \eta_ {k - 1} \left\langle \mathbf {M} _ {\mathbf {y}} \left(\mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1}\right), \mathbf {y} _ {k - 1} - \mathbf {y} _ {k} \right\rangle . \tag {C.4} \\ \end{array}
$$
The first two terms in (C.4) telescope, so we focus on bounding  $-\eta_{k-1}\langle \mathbf{M}_{\mathbf{y}}(\mathbf{x}_{k-1} - \hat{\mathbf{x}}_{k-1}), \mathbf{y}_{k-1} - \mathbf{y}_k\rangle$ . By definition of  $\hat{\mathbf{x}}_k$ , for all  $j \in \mathcal{J}_{\mathcal{X}}$ ,
$$
\mathbf {x} _ {k - 1} ^ {j} - \hat {\mathbf {x}} _ {k - 1} ^ {j} = \frac {\mathbf {x} _ {k - 1} ^ {j}}{x _ {k - 1} ^ {p _ {j}}} \big (x _ {k - 1} ^ {p _ {j}} - x _ {k - 2} ^ {p _ {j}} \big).
$$
By the definition of a treeplex, each vector  $\frac{\mathbf{x}_{k - 1}^j}{x_{k - 1}^{p_j}}$  belongs to a probability simplex of the appropriate size. This further implies that
$$
\begin{array}{l} \| \mathbf {x} _ {k - 1} - \hat {\mathbf {x}} _ {k - 1} \| = \left\| \left[ \frac {\mathbf {x} _ {k - 1} ^ {j}}{x _ {k - 1} ^ {p _ {j}}} \left(x _ {k - 1} ^ {p _ {j}} - x _ {k - 2} ^ {p _ {j}}\right) \right] _ {j \in \mathcal {J} _ {\mathcal {X}}} \right\| \\ \leq \llbracket x _ {k - 1} ^ {p _ {j}} - x _ {k - 2} ^ {p _ {j}} \rrbracket_ {j \in \mathcal {I} _ {\mathcal {X}}} \llbracket \\ \leq \left\| \mathbf {x} _ {k - 1} - \mathbf {x} _ {k - 2} \right\|, \tag {C.5} \\ \end{array}
$$
where the notation  $[a^j]_{j \in \mathcal{J}_X}$  is used to denote the vector with entries  $a^j$ , for  $j \in \mathcal{J}_X$ . The first inequality in (C.5) holds for any  $\ell_p$  norm  $(p \geq 1)$ , by its definition and  $\left\langle \mathbf{1}, \mathbf{x}_{k-1}^j \right\rangle = 1, \forall j$ . Thus, applying the definitions of the norms from the preliminaries,