SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Bo Jiang
Ministry of Education Key Laboratory of NSLSCS
School of Mathematical Sciences, Nanjing Normal University
Nanjing 210023, China
jiangbo@njnu.edu.cn
Ya-Feng Liu *
State Key Laboratory of Scientific and Engineering Computing
Institute of Computational Mathematics and Scientific/Engineering Computing
Academy of Mathematics and Systems Science, Chinese Academy of Sciences
Beijing 100190, China
yafliu@1sec.cc.ac.cn
Projection robust Wasserstein (PRW) distance is recently proposed to efficiently mitigate the curse of dimensionality in the classical Wasserstein distance. In this paper, by equivalently reformulating the computation of the PRW distance as an optimization problem over the Cartesian product of the Stiefel manifold and the Euclidean space with additional nonlinear inequality constraints, we propose a Riemannian exponential augmented Lagrangian method (REALM) for solving this problem. Compared with the existing Riemannian exponential penalty-based approaches, REALM can potentially avoid too small penalty parameters and exhibit more stable numerical performance. To solve the subproblems in REALM efficiently, we design an inexact Riemannian Barzilai-Borwein method with Sinkhorn iteration (iRBBS), which selects the stepsizes adaptively rather than tuning the step sizes in efforts as done in the existing methods. We show that iRBBS can return an  $\epsilon$ -stationary point of the original PRW distance problem within  $\mathcal{O}(\epsilon^{-3})$  iterations, which matches the best known iteration complexity result. Extensive numerical results demonstrate that our proposed methods outperform the state-of-the-art solvers for computing the PRW distance.
The optimal transport (OT) problem has found wide applications in machine learning, representation learning, data sciences, and image sciences; see [21, 5, 42, 1, 14] and the references therein for more details. However, its direct application in machine learning may encounter the issue of the curse of dimensionality since the sample complexity of approximating the Wasserstein distance can grow exponentially in dimension [22, 49]. To resolve this issue, by making an important extension to the sliced Wasserstein distance [43], Paty and Cuturi [41], Deshpande et al. [16], and Niles-Weed and Rigollet [40] proposed to project the distributions to a low-dimensional subspace that maximizes the
Wasserstein distance between the projected distribution, which can reduce the sample complexity and overcome the issue of the curse of dimensionality [40, 16, 35].
In this paper, we focus on the discrete probability measure case. For  $\{x_1,\ldots ,x_n\} \subset \mathbb{R}^d$  and  $\{y_{1},\dots ,y_{n}\} \subset \mathbb{R}^{d}$ , define  $M_{ij} = (x_i - y_j)(x_i - y_j)^\top$  for each  $(i,j)\in [n]\times [n]$  with  $[n]\coloneqq \{1,\ldots n\}$ . Let  $\mathbf{1}\in \mathbb{R}^n$  be the all-one vector and  $\delta_{x}$  be the Dirac delta function at  $x$ . Given  $r = (r_1,\dots ,r_n)^\top \in \Delta^n\coloneqq \{z\in \mathbb{R}^n\mid \mathbf{1}^\top z = 1,z > 0\}$  and  $c = (c_{1},\dots ,c_{n})^{\top}\in \Delta^{n}$ , define two discrete probability measures  $\mu_{n} = \sum_{i = 1}^{n}r_{i}\delta_{x_{i}}$  and  $\nu_{n} = \sum_{i = 1}^{n}c_{i}\delta_{y_{i}}$ . For  $k\in [d]$ , the  $k$ -dimensional projection robust Wasserstein (PRW) distance between  $\mu_{n}$  and  $\nu_{n}$  is defined as [41]
$$
\mathcal {P} _ {k} ^ {2} \left(\mu_ {n}, \nu_ {n}\right) = \max  _ {U \in \mathcal {U}} \min  _ {\pi \in \Pi (r, c)} \langle \pi , C (U) \rangle , \tag {1}
$$
where  $\langle \cdot, \cdot \rangle$  is the standard inner product in  $\mathbb{R}^{n \times n}$ ,  $C(U) \in \mathbb{R}^{n \times n}$  with  $[C(U)]_{ij} = \langle M_{ij}, UU^{\top} \rangle$ ,  $\mathcal{U} = \{U \in \mathbb{R}^{d \times k} \mid U^{\top} U = I_k\}$  is known as the Stiefel manifold with  $I_k$  being the  $k$ -by- $k$  identity matrix, and  $\Pi(r, c) = \{\pi \in \mathbb{R}^{n \times n} \mid \pi \mathbf{1} = r, \pi^{\top} \mathbf{1} = c, \pi \geq 0\}$ . Problem (1) is a nonconcave-convex max-min problem over the Stiefel manifold, which makes it very challenging to solve.
Related works and motivations. To compute the PRW distance, Paty and Cuturi [41] proposed two algorithms for solving the subspace robust Wasserstein distance, which is a convex relaxation of problem (1) (without the theoretical guarantee on the relaxation gap). An OT or entropy-regularized OT subproblem with dimension  $n$  and a full or top  $k$  eigendecomposition of a  $d$ -by-  $d$  matrix needs to be solved exactly at each iteration. Very recently, Lin et al. [34] proposed a Riemannian (adaptive) gradient ascent with Sinkhorn (R(A)GAS) algorithm for solving the following entropy-regularized problem with a small regularization parameter  $\eta$ :
$$
\max  _ {U \in \mathcal {U}} p _ {\eta} (U), \tag {2}
$$
where  $p_{\eta}(U) = \min_{\pi \in \Pi(r, c)} \{ \langle \pi, C(U) \rangle - \eta H(\pi) \}$ , in which  $H(\pi) = -\sum_{ij} \pi_{ij} \log \pi_{ij}$  is the entropy function. They showed that R(A)GAS can return an  $\epsilon$ -stationary point of PRW problem (1) within  $\mathcal{O}(\epsilon^{-4})$  iterations if  $\eta = \mathcal{O}(\epsilon)$  in (2). However, at each iteration, R(A)GAS needs to solve a regularized OT problem in relatively high precision, which results in a high computational cost. To reduce the complexity of R(A)GAS, Huang et al. [28, 29] proposed a Riemannian (adaptive) block coordinate descent (R(A)BCD) algorithm for solving an equivalent "min" formulation of (2) as
$$
\min  _ {U \in \mathcal {U}, \alpha , \beta \in \mathbb {R} ^ {n}} \mathcal {L} _ {\eta} (\mathbf {x}, \mathbf {1 1} ^ {\top}), \tag {3}
$$
where  $\mathcal{L}_{\eta}(\cdot ,\cdot)$  is defined in (8) further ahead. By choosing  $\eta = \mathcal{O}(\epsilon)$  in (3), they showed that the whole iteration complexity of R(A)BCD to attain an  $\epsilon$  -stationary point of PRW problem (1) reduces to  $\mathcal{O}(\epsilon^{-3})$ , which significantly improves the complexity of R(A)GAS.
However, there are two main issues of R(A)BCD and R(A)GAS. First, to compute a solution of problem (1) with relatively high quality,  $\eta$  in problem (2) or (3) has to be chosen small, which makes the corresponding problem ill-conditioned and may cause numerical instability in solving it. Second, the performance of the above algorithms is sensitive to the step sizes in updating  $U$ . Hence, to achieve a better performance, one has to spend some efforts tuning the step sizes carefully. Resolving these two main issues demands some novel approaches from both theoretical and computational points of view, and this is the motivation and focus of our paper.
Contributions. In this paper, by reformulating (1) as an optimization problem defined over the Cartesian product of the Stiefel manifold and the Euclidean space with additional inequality constraints (see problem (6) further ahead), we can resolve the above-mentioned two issues. Our main contributions are summarized as follows. See also Figure 1 for a summary of the related works and the main results of this paper.
(i) We propose a Riemannian exponential augmented Lagrangian method (REALM) to efficiently and faithfully compute the PRW distance, in which a series of subproblems with dynamically decreasing penalty parameters and adaptively updated multiplier matrices are solved approximately. In theory, we establish the global convergence of REALM in the sense that any limit point of the sequence generated by the algorithm is a stationary point of the original problem; see Theorem 2.7. Numerically, REALM always outperforms the Riemannian exponential penalty approach since it could avoid too small penalty parameters in many cases.
Figure 1: A summary of the related works and the main results of this paper.
(ii) To efficiently solve the subproblem in REALM (i.e., problem (11) further ahead), we view it as a one-block optimization problem over the Stiefel manifold and propose a novel and practical algorithm, namely, the inexact Riemannian Barzilai-Borwein (BB) method with Sinkhorn iteration (iRBBS), wherein a flexible number of Sinkhorn iterations is performed to compute the inexact Riemannian gradient. Compared with R(A)BCD, our proposed iRBBS (applied to problem (11) with fixed  $\pi^k$  and  $\eta_k = O(\epsilon)$ ) can not only return a stronger  $\epsilon$ -stationary point of PRW problem (1) (compared with the definitions in [34, 28]; see Remark 2.2), within  $\mathcal{O}(\epsilon^{-3})$  iterations (see Theorem 3.2 with  $\epsilon_1 = \epsilon_2 = \epsilon$ ), but also has a better numerical performance, which mainly benefits from the adaptive Riemannian BB stepsize (based on the inexact Riemannian gradient information).
Notations. For  $x \in \mathbb{R}^n$ ,  $\mathrm{Diag}(x)$  is an  $n \times n$  diagonal matrix with  $x$  being its main diagonal. For a matrix  $A$ , denote  $A_{\min} = \min_{ij} A_{ij}$ ,  $A_{\max} = \max_{ij} A_{ij}$ ,  $\|A\|_1 = \sum_{ij} |A_{ij}|$ ,  $\|A\|_\infty = \max_{ij} |A_{ij}|$ ,  $\|A\|_{\mathsf{F}}^2 = \sum_{ij} A_{ij}^2$ , and  $\|A\|_{\mathrm{var}} = A_{\max} - A_{\min}$ . Denote by  $\log(\cdot)$  and  $\exp(\cdot)$  the element-wise logarithmic and exponential operators, respectively. We use  $\mathbb{R}_+^{n \times n}$  and  $\mathbb{R}_{++}^{n \times n}$  to denote the nonnegative and positive orthants of  $\mathbb{R}^{n \times n}$ , respectively. Throughout this paper, we define  $C \in \mathbb{R}^{n \times n}$  with  $C_{ij} = \|x_i - y_j\|^2$  and  $V_\pi = \sum_{ij} \pi_{ij} M_{ij}$ .
The tangent space at  $U \in \mathcal{U}$  is  $\mathrm{T}_U\mathcal{U} = \{\xi \in \mathbb{R}^{d \times k} \mid U^\top \xi + \xi^\top U = 0\}$ . Let  $\mathrm{T}\mathcal{U} = \{(U, \xi) \mid U \in \mathcal{U}$  and  $\xi \in \mathrm{T}_U\mathcal{U}\}$  be the tangent bundle of  $\mathcal{U}$ . A smooth map  $\operatorname{Retr}: \mathrm{T}\mathcal{U} \to \mathcal{U}: (U, \xi) \mapsto \operatorname{Retr}_U(\xi)$  is called a retraction if each curve  $\mathcal{R}(t) = \operatorname{Retr}_U(t\xi)$  satisfies  $\mathcal{R}(0) = U$  and  $\mathcal{R}'(0) = \xi$ ; see [9, Definition 3.47] or [2, Definition 4.1.1]. The Riemannian metric  $\langle \cdot, \cdot \rangle_U$  endowed on the Stiefel manifold  $\mathcal{U}$  is taken as the standard metric  $\langle \cdot, \cdot \rangle$  on  $\mathbb{R}^{d \times k}$ . The Riemannian gradient of a smooth function  $f: \mathbb{R}^{d \times k} \to \mathbb{R}$  at  $U \in \mathcal{U}$  is defined as  $\operatorname{grad} f(U)$ , which satisfies  $\langle \operatorname{grad} f(U), \xi \rangle_U = \langle \nabla f(U), \xi \rangle$  for all  $\xi \in \mathrm{T}_U\mathcal{U}$ , where  $\nabla f(U)$  denotes the Euclidean gradient of  $f$  at  $U$ . If  $U^\top\nabla f(U)$  is symmetric, we have  $\operatorname{grad} f(U) = \operatorname{Proj}_{\mathrm{T}_U\mathcal{U}}(\nabla f(U)) = (I_d - UU^\top)\nabla f(U)$ .
The rest of this paper is organized as follows. The proposed REALM is introduced in Section 2. A practical iRBBS for solving the subproblem in REALM is proposed in Section 3. Numerical results are presented in Section 4. Finally, we draw some concluding remarks in Section 5.
2 A Riemannian Exponential ALM for Computing the PRW Distance (1)
Given a fixed  $U\in \mathcal{U}$  , consider the OT problem
$$
\min  _ {\pi \in \mathbb {R} ^ {n \times n}} \langle \pi , C (U) \rangle \quad \text {s . t .} \quad \pi \mathbf {1} = r, \pi^ {\mathsf {T}} \mathbf {1} = c, \pi \geq 0. \tag {4}
$$
By adding a redundant constraint  $\| \pi \| _1 = 1$  [29, 36], we derive the dual of (4) as
$$
\max  _ {\alpha \in \mathbb {R} ^ {n}, \beta \in \mathbb {R} ^ {n}} - \left(r ^ {\mathsf {T}} \alpha + c ^ {\mathsf {T}} \beta + y\right) \quad \text {s . t .} \quad \varphi (\mathbf {x}) _ {i j} + y \geq 0, \quad \forall (i, j) \in [ n ] \times [ n ], \tag {5}
$$
where  $\mathbf{x} = (\alpha, \beta, U)$  and  $\varphi(\mathbf{x}) \in \mathbb{R}^{n \times n}$  with  $\varphi(\mathbf{x})_{ij} = \alpha_i + \beta_j + \langle M_{ij}, U U^\top \rangle$  for each  $(i,j) \in [n] \times [n]$ . Note that the matrix  $\pi$  in (4) can also be understood as the Lagrange multiplier corresponding to the inequalities in (5). Therefore, the value  $\mathcal{P}_k^2(\mu_n, \nu_n)$  defined in (1) is equal to the opposite of the optimal objective value of the following optimization problem:
$$
\min  _ {\mathbf {x} \in \mathcal {M}, y \in \mathbb {R}} r ^ {\top} \alpha + c ^ {\top} \beta + y \quad \text {s . t .} \quad \varphi (\mathbf {x}) _ {i j} + y \geq 0, \quad \forall (i, j) \in [ n ] \times [ n ], \tag {6}
$$
where  $\mathcal{M} = \mathbb{R}^n\times \mathbb{R}^n\times \mathcal{U}$ . Motivated by the first-order necessary condition of problem (6) (see Appendix A.1 for details), we define the  $(\epsilon_{1},\epsilon_{2})$ -stationary point of problem (1) as follows.
Definition 2.1. We call  $(\tilde{\mathbf{x}},\tilde{\pi})\in \mathcal{M}\times \Pi (r,c)$  an  $(\epsilon_{1},\epsilon_{2})$  -stationary point of PRW problem (1), if  $\| \mathrm{Proj}_{\mathrm{T}_{\tilde{U}}}\mathcal{U}(-2V_{\tilde{\pi}}\tilde{U})\|_{\mathsf{F}}\leq \epsilon_1$  and  $\langle \tilde{\pi},Z(\tilde{\mathbf{x}})\rangle \leq \epsilon_{2}$ , where  $Z(\tilde{\mathbf{x}})\in \mathbb{R}^{n\times n}$  with  $Z(\tilde{\mathbf{x}})_{ij} = \varphi (\mathbf{x})_{ij} - \varphi (\mathbf{x})_{\min}$ . If  $\epsilon_{1} = \epsilon_{2} = 0$ , we call such  $(\tilde{\mathbf{x}},\tilde{\pi})$  a stationary point of PRW problem (1).
Remark 2.2. Our Definition 2.1 is stronger than [28, Definition 3.1] and [34, Definition 2.7] in the sense that the  $(\epsilon_1,\epsilon_2)$ -stationary point satisfying the conditions here also satisfies all conditions therein. See Appendix A.2 for more details.
Given  $\pi \in \mathbb{R}_{++}^{n\times n}$  and  $\eta >0$ , define the function  $\zeta_{\eta}(\mathbf{x},\pi)\in \mathbb{R}^{n\times n}$  with
$$
\left[ \zeta_ {\eta} (\mathbf {x}, \pi) \right] _ {i j} = \pi_ {i j} \exp \left(- \frac {\varphi (\mathbf {x}) _ {i j}}{\eta}\right), \tag {7}
$$
$$
\widetilde {\mathcal {L}} _ {\eta} (\mathbf {x}, y, \pi) = r ^ {\mathsf {T}} \alpha + c ^ {\mathsf {T}} \beta + y + \eta \sum_ {i j} \pi_ {i j} \exp \left(- \frac {\varphi (\mathbf {x}) _ {i j} + y}{\eta}\right)
$$
$$
\mathcal {L} _ {\eta} (\mathbf {x}, \pi) := r ^ {\top} \alpha + c ^ {\top} \beta + \eta \log \left(\| \zeta_ {\eta} (\mathbf {x}, \pi) \| _ {1}\right). \tag {8}
$$
One natural approach for solving problem (6) is the Riemannian exponential penalty approach (where the manifold constraints are kept in the subproblem), which aims to solve the penalty subproblem
$$
\min  _ {\mathbf {x} \in \mathcal {M}, y \in \mathbb {R}} \widetilde {\mathcal {L}} _ {\eta} (\mathbf {x}, y, \mathbf {1 1} ^ {\top}). \tag {9}
$$
For any fixed  $x$ , letting  $\nabla_y\widetilde{\mathcal{L}}_\eta (\mathbf{x},y,\mathbf{11}^\top) = 0$ , we can obtain the optimal  $y$  as  $y = \eta \log (\| \zeta_{\eta}(\mathbf{x},\mathbf{11}^{T})\|_{1})$ . By eliminating the variable  $y$  in (9), we thus obtain the subproblem (3) of the approach in Huang et al. [28].
It is known that the exponential augmented Lagrangian method (ALM) is usually more stable than the exponential penalty approach; see [18, Tables 3.1-3.3] for a detailed example. More specifically, the penalty parameter in the exponential ALM can be chosen as any positive number in the convex case [47, 52] or can be bounded away from zero under some standard assumptions in the general nonlinear case [20], which is in sharp contrast to the exponential penalty approach. Based on the aforementioned knowledge, we thus extend the exponential ALM [8] to the manifold case to solve problem (6). Fix the current estimate of the Lagrange multiplier corresponding to the inequality constraints in (6) and the penalty parameter as  $\pi^k$  and  $\eta_k$ , respectively. Then the subproblem at the  $k$ -th iteration is given as
$$
\min  _ {\mathbf {x} \in \mathcal {M}, y \in \mathbb {R}} \widetilde {\mathcal {L}} _ {\eta_ {k}} (\mathbf {x}, y, \pi^ {k}). \tag {10}
$$
Similar to the way for eliminating  $y$  in (9), we obtain an equivalent formulation of (10):
$$
\min  _ {\mathbf {x} \in \mathcal {M}} \mathcal {L} _ {\eta_ {k}} (\mathbf {x}, \pi^ {k}). \tag {11}
$$
Define the matrix  $\phi_{\eta_k}(\mathbf{x},\pi^k)\in \mathbb{R}^{n\times n}$  with
$$
\left[ \phi_ {\eta_ {k}} \left(\mathbf {x}, \pi^ {k}\right) \right] _ {i j} = \left[ \zeta_ {\eta_ {k}} \left(\mathbf {x}, \pi^ {k}\right) \right] _ {i j} / \| \zeta_ {\eta_ {k}} \left(\mathbf {x}, \pi^ {k}\right) \| _ {1}. \tag {12}
$$
By the chain rule, we have  $\nabla_{\alpha}\mathcal{L}_{\eta_k}(\mathbf{x},\pi^k) = r - \phi_{\eta_k}(\mathbf{x},\pi^k)\mathbf{1}$ ,  $\nabla_{\beta}\mathcal{L}_{\eta_k}(\mathbf{x},\pi^k) = c - \phi_{\eta_k}(\mathbf{x},\pi^k)^{\top}\mathbf{1}$ ,  $\nabla_U\mathcal{L}_{\eta_k}(\mathbf{x},\pi^k) = -2V_{\phi_{\eta_k}(\mathbf{x},\pi^k)}U$ , and  $\mathrm{grad}_U\mathcal{L}_{\eta_k}(\mathbf{x},\pi^k) = \mathrm{Proj}_{\mathrm{T}_U\mathcal{U}}(-2V_{\phi_{\eta_k}(\mathbf{x},\pi^k)}U)$ . Let  $\mathsf{e}_{\eta_k}^1 (\mathbf{x},\pi^k) = \| \mathrm{grad}_U\mathcal{L}_{\eta_k}(\mathbf{x},\pi^k)\|_{\mathsf{F}}$  and  $\mathsf{e}_{\eta_k}^2 (\mathbf{x},\pi^k) = \| \nabla_\alpha \mathcal{L}_{\eta_k}(\mathbf{x},\pi^k)\| _1 + \| \nabla_\beta \mathcal{L}_{\eta_k}(\mathbf{x},\pi^k)\| _1$ . The  $(\epsilon_1,\epsilon_2)$ -stationary point of (11) and the connections of the approximate stationary points of problems (11) and (1) are given as follows.
Definition 2.3. We say  $\tilde{\mathbf{x}}\in \mathcal{M}$  an  $(\epsilon_{1},\epsilon_{2})$  -stationary point of problem (11) (with fixed  $\eta_{k}$  and  $\pi^k$ ) if  $\mathsf{e}_{\eta_k}^1 (\tilde{\mathbf{x}},\pi^k)\leq \epsilon_1$  and  $\mathsf{e}_{\eta_k}^2 (\tilde{\mathbf{x}},\pi^k)\leq \epsilon_2$
Theorem 2.4. Suppose  $\tilde{\mathbf{x}} = (\tilde{\alpha},\tilde{\beta},\tilde{U})\in \mathcal{M}$  with  $\| \zeta_{\eta_k}(\tilde{\mathbf{x}},\pi^k)\| _1 = 1$  is an  $(\epsilon_1,\epsilon_2)$ -stationary point of problem (11). Then, we have
$$
\left\| \operatorname {P r o j} _ {\mathrm {T} _ {\tilde {U}}} \mathcal {U} \left(- 2 V _ {\hat {\pi}} \tilde {U}\right) \right\| _ {\mathrm {F}} \leq \epsilon_ {1} + 2 \| C \| _ {\infty} \epsilon_ {2}, \tag {13a}
$$
$$
\langle \hat {\pi}, Z (\tilde {\mathbf {x}}) \rangle \leq (2 \log n + \| \log \pi^ {k} \| _ {\operatorname {v a r}}) \eta_ {k} + (\| \tilde {\alpha} \| _ {\operatorname {v a r}} + \| \tilde {\beta} \| _ {\operatorname {v a r}} + \| C \| _ {\infty}) \epsilon_ {2}, \tag {13b}
$$
where  $\hat{\pi} := \text{Round}(\phi_{\eta_k}(\tilde{\mathbf{x}}, \pi^k), \Pi(r, c))$  is a feasible matrix returned by running the rounding procedure "Round" given in [3, Algorithm 2] with input  $\phi_{\eta_k}(\tilde{\mathbf{x}}, \pi^k)$ .
Remark 2.5. By Theorem 2.4, we can see that, given any  $\epsilon >0$ , for fixed  $\pi^k$ , by choosing  $\eta_{k} = O(\epsilon)$  with  $\epsilon_1 = \epsilon_2 = \epsilon$ , an  $\epsilon$ -stationary point with bounded  $(\tilde{\alpha},\tilde{\beta})$  (which can be found efficiently by iRBBS proposed in Section 3) of problem (11) can recover an  $\epsilon$ -stationary point of PRW problem (1). This may also be of independent interest for the case with fixed  $U$ . In such case, one can return a feasible  $\epsilon$ -stationary point of the OT problem (4) (with fixed  $U$ ) evaluated by the primal-dual gap other than the primal gap typically used in the literature, such as [3, Theorem 1].
Let  $\mathbf{x}^0$  be an initial point satisfying  $r^{\mathsf{T}}\alpha^{0} = c^{\mathsf{T}}\beta^{0}$  and  $\| \zeta_{\eta_1}(\mathbf{x}^0,\pi^1)\| _1 = 1$  with  $\pi^1 = \mathbf{1}\mathbf{1}^\top$ . We require that  $\mathbf{x}^k = (\alpha^k,\beta^k,U^k)$ , the  $(\epsilon_{k,1},\epsilon_{k,2})$ -stationary point of the subproblem (11) with  $k\geq 1$ , satisfies the following conditions:
$$
r ^ {\mathsf {T}} \alpha^ {k} = c ^ {\mathsf {T}} \beta^ {k}, \| \zeta_ {\eta_ {k}} \left(\mathbf {x} ^ {k}, \pi^ {k}\right) \| _ {1} = 1, \mathcal {L} _ {\eta_ {k}} \left(\mathbf {x} ^ {k}, \pi^ {k}\right) \leq \min  \left\{\mathcal {L} _ {\eta_ {k}} \left(\mathbf {x} ^ {k - 1}, \pi^ {k}\right), \mathcal {L} _ {\eta_ {k}} \left(\mathbf {x} ^ {0}, \pi^ {k}\right) \right\}. \tag {14}
$$
These conditions are important to establish the convergence of REALM (as shown in Appendix A.4). The following key observation to (11) shows that  $\mathbf{x}^k$  satisfying the first two conditions in (14) can be easily obtained. We omit the detailed proof for brevity since they can be verified easily by noticing that  $\mathcal{L}_{\eta_k}(\mathbf{x},\pi^k) = \mathcal{L}_{\eta_k}(\mathbf{x}^s,\pi^k)$  where  $\mathbf{x}^s$  is defined in Proposition 2.6. The third condition in (14) can be easily satisfied by using any descent-like method starting from the better point of  $\mathbf{x}^{k - 1}$  and  $\mathbf{x}^0$  to solve problem (11).
Proposition 2.6. For any  $\mathbf{x} = (\alpha, \beta, U) \in \mathcal{M}$ , consider  $\mathbf{x}^s = (\alpha + v_1\mathbf{1}, \beta + v_2\mathbf{1}, U) \in \mathcal{M}$  with  $v_1 = (c^\top \beta - r^\top \alpha + \eta_k \log(\|\zeta_{\eta_k}(\mathbf{x}, \pi^k)\|_1)) / 2$  and  $v_2 = (r^\top \alpha - c^\top \beta + \eta_k \log(\|\zeta_{\eta_k}(\mathbf{x}, \pi^k)\|_1)) / 2$ . Then we have that  $r^\top(\alpha + v_1\mathbf{1}) = c^\top(\beta + v_2\mathbf{1})$  and  $\|\zeta_{\eta_k}(\mathbf{x}^s, \pi^k)\|_1 = 1$  and also that  $\mathcal{L}_{\eta_k}(\mathbf{x}, \pi^k) = \mathcal{L}_{\eta_k}(\mathbf{x}^s, \pi^k)$  and  $\nabla_{\mathbf{x}}\mathcal{L}_{\eta_k}(\mathbf{x}, \pi^k) = \nabla_{\mathbf{x}}\mathcal{L}_{\eta_k}(\mathbf{x}^s, \pi^k)$ .
With such  $\mathbf{x}^k$  in hand, we compute the candidate of the next estimate  $\pi^{k + 1}$  as
$$
\tilde {\pi} ^ {k + 1} = \phi_ {\eta_ {k}} \left(\mathbf {x} ^ {k}, \pi^ {k}\right). \tag {15}
$$
Denote  $W^{k} \in \mathbb{R}^{n \times n}$  with  $W_{ij}^{k} = \min \{\eta_{k}\tilde{\pi}_{ij}^{k + 1}, \varphi(\mathbf{x}^{k})_{ij}\}$ . The penalty parameter  $\eta_{k + 1}$  is updated according to the progress of the complementarity violation [4], denoted by  $\| W^{k}\|_{\mathsf{F}}$ . If  $\| W^{k}\|_{\mathsf{F}} \leq \gamma_{W}\| W^{k - 1}\|_{\mathsf{F}}$  with  $\gamma_{W} \in (0,1)$ , we keep  $\eta_{k + 1} = \eta_{k}$  and update  $\pi^{k + 1} = \tilde{\pi}^{k + 1}$ ; otherwise we keep  $\pi^{k + 1} = \pi^{k}$  and reduce  $\eta_{k + 1}$  via
$$
\eta_ {k + 1} = \min  \left\{\gamma_ {\eta} \eta_ {k}, \varrho_ {k} / \| \log \pi^ {k} \| _ {\infty} \right\}, \tag {16}
$$
where  $\gamma_{\eta}\in (0,1)$  is a constant and  $\varrho_{k}\to 0$  with  $\varrho_{k} > 0$
We summarize the above discussion as the complete algorithm in Algorithm 1, whose convergence is established as follows.
Theorem 2.7. Let  $\{(\mathbf{x}^k, \tilde{\pi}^k)\}$  be the sequence generated by Algorithm 1 with  $\epsilon_1 = \epsilon_2 = \epsilon_c = 0$  and  $(\mathbf{x}^\infty, \tilde{\pi}^\infty)$  be a limit point of  $\{(\mathbf{x}^k, \tilde{\pi}^k)\}$ . Then,  $(\mathbf{x}^\infty, \tilde{\pi}^\infty)$  is a stationary point of PRW problem (1).