SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemin Kong
1kong@se.cuhk.edu.hk
Jiajin Li
jiajinli@stanford.edu
Jianheng Tang
jtangbf@connect.ust.hk
Gromov-Wasserstein (GW) distance is a powerful tool for comparing and aligning probability distributions supported on different metric spaces. Recently, GW has become the main modeling technique for aligning heterogeneous data for a wide range of graph learning tasks. However, the GW distance is known to be highly sensitive to outliers, which can result in large inaccuracies if the outliers are given the same weight as other samples in the objective function. To mitigate this issue, we introduce a new and robust version of the GW distance called RGW. RGW features optimistically perturbed marginal constraints within a Kullback-Leibler divergence-based ambiguity set. To make the benefits of RGW more accessible in practice, we develop a computationally efficient and theoretically provable procedure using Bregman proximal alternating linearized minimization algorithm. Through extensive experimentation, we validate our theoretical results and demonstrate the effectiveness of RGW on real-world graph learning tasks, such as subgraph matching and partial shape correspondence.
Gromov-Wasserstein distance (GW) [26, 28] acts as a main model tool in data science to compare data distributions on unaligned metric spaces. Recently, it has received much attention across a host of applications in data analysis, e.g., shape correspondence [24, 31, 36, 27], graph alignment and partition [37, 38, 15, 18, 14, 44], graph embedding and classification [41, 43], unsupervised word embedding and translation [3, 19], generative modeling across incomparable spaces [8, 45].
In practice, the robustness of GW distance suffers heavily from its sensitivity to outliers. Here, outliers mean the samples with large noise, which usually are far away from the clean samples or have different structures from the clean samples. The hard constraints on the marginals in the Gromov-Wasserstein distance require all the mass in the source distribution to be entirely transported to the target distribution, making it highly sensitive to outliers. When the outliers are weighted similarly as other clean samples, even a small fraction of outliers corrupted can largely impact the GW distance value and the optimal coupling, which is unsatisfactory in real-world applications.
To overcome the above issue, some recent works are trying to relax the marginal constraints of GW distance. [33] introduces a  $L^1$  relaxation of mass conservation of the GW distance. However, this reformulation replaces the strict marginal constraint that the transport plan should be a joint distribution with marginals as specific distributions by the constraint that only requires the transport plan to be a joint distribution, which can easily lead to over-relaxation. On another front, [10] propose a so-called partial GW distance (PGW), which only transports a fraction of mass from source distribution to target distribution. The formulation of PGW is limited to facilitating mass
(a) Clean dataset
(b) Dataset with outliers
Figure 1: Visualization of Gromov-Wasserstein couplings between two shapes, with the source in blue and the target in orange. In (a), the GW coupling without outliers is shown. In (b), the coupling with  $10\%$  outliers added to the target distribution is depicted. The sensitivity of GW to outliers is evident from the plot. In (c), we present the coupling generated by our proposed RGW formulation, which effectively disregards outliers and closely approximates the true GW distance.
(c) Dataset with outliers
destruction, which restricts its ability to handle situations where outliers exist predominantly on one side. A formulation that allows both mass destruction and creation is proposed in [35] called unbalanced GW (UGW). The UGW relaxes the marginal constraint via adding the quadratic  $\varphi$ -divergence as the penalty function in the objective function and extends GW distance to compare metric spaces equipped with arbitrary positive measures. Additionally, [40] proved that UGW is robust to outliers and can effectively remove the mass of outliers with high transportation costs. However, UGW is sensitive to the penalty parameter as it balances the reduction of outlier impact and the control of marginal distortion in the transport plan. On the computational side, an alternate Sinkhorn minimization method is proposed to calculate the entropy-regularized UGW. Note that the algorithm does not exactly solve UGW but approximates the lower bound of the entropic regularized UGW instead. From a statistical viewpoint, these works do not establish a direct link between the reformulated GW distance and the GW distance in terms of uncontaminated levels.
In this work, we propose the robust Gromov-Wasserstein (RGW) to estimate the GW distance robustly when dealing with outliers. To achieve this, RGW simultaneously optimizes the transport plan and selects the best marginal distribution from a neighborhood of the given marginal distributions, avoiding contaminated distributions. Perturbed marginal distributions help to re-weight the samples and lower the weight assigned to the outliers. The introduction of relaxed distributions to handle outliers draws inspiration from robust OT techniques [4, 29, 23]. Unlike robust OT, which is convex, RGW is non-convex, posing algorithmic challenges. This idea is also closely related to optimistic modelings of distribution ambiguity in data-driven optimization, e.g., upper confidence bound in the multi-armed bandit problem and reinforcement learning [7, 30, 1], data-driven distributionally robust decision-making with outliers [21, 9], etc.
Moreover, inspired by UGW, RGW relaxes the marginal constraint via adding the Kullback-Leibler (KL) divergence between the marginals of the transport plan and the perturbed distributions as the penalty function in the objective function to lessen the impact of the outliers further. Instead of utilizing the quadratic KL divergence as employed in unbalanced GW, we opt for KL divergence due to its computational advantages. It allows for convex subproblems with closed-form solutions, as opposed to the linearization required for non-convex quadratic KL divergence, which could be challenging algorithmically. Furthermore, we leverage the convexity of KL divergence to establish the statistical properties of RGW, leading to an upper bound by the true GW distance with explicit control through marginal relaxation parameters and marginal penalty parameters. This statistical advantage is disrupted by the non-convex nature of quadratic KL divergence. Additionally, KL divergence aligns with our goal of outlier elimination and is less sensitive to outliers compared to quadratic KL divergence, which is more outlier-sensitive due to its quadratic nature. Overall, RGW combines the introduction of perturbed marginal distributions with the relaxation of hard marginal constraints to achieve greater flexibility, allowing control over marginal distortion through marginal penalty parameters and reduction of outlier impact using marginal relaxation parameters.
To realize the modeling benefits of RGW, we further propose an efficient algorithm based on the Bregman proximal alternating linearized minimization (BPALM) method. The updates in each
iteration of BPALM can be computed in a highly efficient manner. On the theoretical side, we prove that the BPALM algorithm converges to a critical point of the RGW problem. Empirically, we demonstrate the effectiveness of RGW and the proposed BPALM algorithm through extensive numerical experiments on subgraph alignment and partial shape correspondence tasks. The results reveal that RGW surpasses both the balanced GW-based method and the reformulations of GW, including PGW and UGW.
Our Contributions We summarize our main contributions as follows:
- We develop a new robust model called RGW to alleviate the impact of outliers on the GW distance. The key insight is to simultaneously optimize the transport plan and perturb marginal distributions in the most efficient way.
- On the statistical side, we demonstrate that the robust Gromov-Wasserstein is bounded above by the true GW distance under the Huber  $\epsilon$ -contamination model.
- On the computational side, we propose an efficient algorithm for solving RGW using the BPALM method and prove that the algorithm converges to a critical point of the RGW problem.
- Empirical results on subgraph alignment and partial shape correspondence tasks demonstrate the effectiveness of RGW. This is the first successful attempt to apply GW-based methods to partial shape correspondence, a challenging problem pointed out in [36].
In this section, we review the definition of Gromov-Wasserstein distance and formally formulate the robust Gromov-Wasserstein. Following that, we discuss the statistical properties of the proposed robust Gromov-Wasserstein model under the Huber  $\epsilon$ -contamination model.
For the rest of the paper, we will use the following notation. Let  $(X,d_X)$  be a complete separable metric space and denote the finite, positive Borel measure on  $X$  by  $\mathcal{M}_{+}(X)$ . Let  $\mathcal{P}(X)\subset \mathcal{M}_{+}(X)$  denotes the space of Borel probability measures on  $X$ . We use  $\Delta^n$  to denote the simplex in  $\mathbb{R}^n$ . We use  $\mathbf{1}_n$  and  $\mathbf{1}_{n\times m}$  to denote the  $n$ -dimensional all-one vector and  $n\times m$  all-one matrix. We use  $\mathcal{S}^n$  to denote the set of  $n\times n$  symmetric matrices. The indicator function of set  $C$  is denoted as  $\mathbb{I}_C(\cdot)$ .
2.1 Robust Gromov-Wasserstein
The Gromov-Wasserstein (GW) distance aims at matching distributions defined in different metric spaces. It is defined as follows:
Definition 2.1 (Gromov-Wasserstein). Suppose that we are given two unregistered complete separable metric spaces  $(X, d_X)$ ,  $(Y, d_Y)$  accompanied with Borel probability measures  $\mu, \nu$  respectively. The GW distance between  $\mu$  and  $\nu$  is defined as
$$
\inf _ {\pi \in \Pi (\mu , \nu)} \iint | d _ {X} (x, x ^ {\prime}) - d _ {Y} (y, y ^ {\prime}) | ^ {2} d \pi (x, y) d \pi (x ^ {\prime}, y ^ {\prime}),
$$
where  $\Pi (\mu ,\nu)$  is the set of all probability measures on  $X\times Y$  with  $\mu$  and  $\nu$  as marginals.
As shown in the definition, the sensitivity to outliers of Gromov-Wasserstein distance is due to its hard constraints on marginal distributions. This suggests relaxing the marginal constraints such that the weight assigned to the outliers by the transport plan can be small. To do it, we invoke the Kullback-Leibler divergence, defined as  $d_{\mathbf{KL}}(\alpha, \mu) = \int_X \alpha(x) \log \left( \frac{\alpha(x)}{\mu(x)} \right) dx$ , to soften the constraint on marginal distributions. We also introduce an optimistically distributionally robust mechanism to perturb the marginal distributions and reduce the weight assigned to outliers. Further details on this mechanism will be discussed later.
Definition 2.2 (Robust Gromov-Wasserstein). Suppose that we are given two unregistered complete separable metric spaces  $(X, d_X)$ ,  $(Y, d_Y)$  accompanied with Borel probability measures  $\mu, \nu$  respectively. The Robust GW between  $\mu$  and  $\nu$  is defined as
$$
\begin{array}{l} \mathbf {G W} _ {\rho_ {1}, \rho_ {2}} ^ {\text {r o b}} (\mu , \nu) := \min  _ {\alpha \in \mathcal {P} (X), \beta \in \mathcal {P} (Y)} F (\alpha , \beta) \tag {1} \\ \mathrm {s . t .} d _ {\mathbf {K L}} (\mu , \alpha) \leq \rho_ {1}, d _ {\mathbf {K L}} (\nu , \beta) \leq \rho_ {2}, \\ \end{array}
$$
where  $F(\alpha ,\beta) =$
$$
\inf _ {\pi \in \mathcal {M} ^ {+} (X \times Y)} \iint | d _ {X} (x, x ^ {\prime}) - d _ {Y} (y, y ^ {\prime})) | ^ {2} d \pi (x, y) d \pi (x ^ {\prime}, y ^ {\prime}) + \tau_ {1} d _ {\mathbf {K L}} (\pi_ {1}, \alpha) + \tau_ {2} d _ {\mathbf {K L}} (\pi_ {2}, \beta),
$$
and  $(\pi_1,\pi_2)$  are two marginals of the joint distribution  $\pi$ , defined by  $\pi_1(A) = \pi (A\times Y)$  for any Borel set  $A\subset X$  and  $\pi_2(B) = \pi (X\times B)$  for any Borel set  $B\subset Y$ .
The main idea of our formulation is to optimize the transport plan and perturbed distribution variables in the ambiguity set of the observed marginal distributions jointly. This formulation aims to find the perturbed distributions that approximate the clean distributions and compute the transport plan based on the perturbed distributions. However, incorporating the constraints of equal marginals between the transport plan  $\pi$  and the perturbed distributions  $\alpha$  and  $\beta$  directly poses challenges in developing an algorithm due to potential non-smoothness issues. Inspired by [35], we address this challenge by relaxing these marginal constraints and incorporating the KL divergence terms, denoted as  $d_{\mathbf{KL}}(\pi_1,\alpha)$  and  $d_{\mathbf{KL}}(\pi_2,\beta)$ , into the objective function as penalty functions. Different from [35], we use KL divergence instead of quadratic KL divergence due to its joint convexity, which is more amenable to algorithm development, as quadratic KL divergence is typically non-convex. Besides, transforming the hard marginal constraints into penalty functions can further lessen the impact of outliers on the transport plan.
Our new formulation extends the balanced GW distance and can recover it by choosing  $\rho_{1} = \rho_{2} = 0$  and letting  $\tau_{1}$  and  $\tau_{2}$  tend to infinity. When properly chosen,  $\rho_{1}$  and  $\rho_{2}$  can encompass the clean distributions within the ambiguity sets. In this scenario, the relaxed reformulation closely approximates the original GW distance in a certain manner. Building on this concept, we prove that RGW can serve as a robust approximation of the GW distance without outliers, given some mild assumptions on the outliers.
2.2 Robustness Guarantees
Robust Gromov-Wasserstein aims at mitigating the sensitivity of the GW distance to outliers, which can result in large inaccuracies if the outliers are given the same weight as other samples in the objective function. Specifically, RGW is designed to address the issue of the GW distance exploding as the distance between the clean samples and the outliers goes to infinity. In general, even a small number of outliers can cause the GW distance to change dramatically when added to the marginal distributions. To formalize this, consider the Huber  $\epsilon$ -contamination model popularized in robust statistics [22, 11, 12]. In that model, a base measure  $\mu_{c}$  is contaminated by an outlier distribution  $\mu_{a}$  to obtain a contaminated measure  $\mu$ ,
$$
\mu = (1 - \epsilon) \mu_ {c} + \epsilon \mu_ {a}. \tag {2}
$$
Under this model, data are drawn from  $\mu$  defined in (2).
Under the assumption of the Huber  $\epsilon$ -contamination model, it can be demonstrated that by selecting suitable values of  $\rho_{1}$  and  $\rho_{2}$ , the robust Gromov-Wasserstein distance ensures that outliers are unable to substantially inflate the transportation distance. For robust Gromov-Wasserstein, we have the following bound:
Theorem 2.3. Let  $\mu$  and  $\nu$  be two distributions corrupted by fractions  $\epsilon_{1}$  and  $\epsilon_{2}$  of outliers, respectively. Specifically,  $\mu$  is defined as  $(1 - \epsilon_{1})\mu_{c} + \epsilon_{1}\mu_{a}$ , and  $\nu$  is defined as  $(1 - \epsilon_{2})\nu_{c} + \epsilon_{2}\nu_{a}$ , where  $\mu_{c}$  and  $\nu_{c}$  represent the clean distributions, and  $\mu_{a}$  and  $\nu_{a}$  represent the outlier distributions. Then,
$$
\begin{array}{l} \mathrm {G W} _ {\rho_ {1}, \rho_ {2}} ^ {\mathrm {r o b}} (\mu , \nu) \leq \mathrm {G W} (\mu_ {c}, \nu_ {c}) + \max  \left(0, \epsilon_ {1} - \frac {\rho_ {1}}{d _ {\mathbf {K L}} (\mu_ {a} , \mu_ {c})}\right) \tau_ {1} d _ {\mathbf {K L}} (\mu_ {c}, \mu_ {a}) \\ + \max  \left(0, \epsilon_ {2} - \frac {\rho_ {2}}{d _ {\mathbf {K L}} (\nu_ {a} , \nu_ {c})}\right) \tau_ {2} d _ {\mathbf {K L}} (\nu_ {c}, \nu_ {a}). \\ \end{array}
$$
In Appendix B, we provide a proof that constructs a feasible transport plan and relaxed marginal distributions. By relaxing the strict marginal constraints, we can find a feasible transport plan that closely approximates the transport plan between the clean distributions and obtain relaxed marginal distributions that approximate the clean distributions.
The derived bound indicates that robust Gromov-Wasserstein provides a provably robust estimate under the Huber  $\epsilon$ -contamination model. If the fraction of outliers is known, the upper bound for the robust GW is determined by the true Gromov-Wasserstein distance, along with additional terms that account for the KL divergence between the clean distribution and the outlier distribution for both  $\mu$  and  $\nu$ . The impact of this factor is determined by the extent of relaxation in the marginal distributions  $\rho_{1}$  and  $\rho_{2}$ . By carefully choosing  $\rho_{1} = \epsilon_{1}d_{\mathbf{KL}}(\mu_{a},\mu_{c})$  and  $\rho_{2} = \epsilon_{2}d_{\mathbf{KL}}(\nu_{a},\nu_{c})$ , we can tighten the upper bound on the robust GW value, while still keeping it below the original GW distance (excluding outliers). Importantly, substituting these values of  $\rho_{1}$  and  $\rho_{2}$  yields  $\mathrm{GW}_{\rho_1,\rho_2}^{\mathrm{rob}}(\mu ,\nu)\leq \mathrm{GW}(\mu_c,\nu_c)$ , indicating that the robust GW between the contaminated distribution  $\mu$  and  $\nu$  is upper bounded by the original GW distance between the clean distribution  $\mu_{c}$  and  $\nu_{c}$ .
Remark 2.4. The following inequality for UGW under the Huber  $\epsilon$ -contamination model can be derived using the same techniques as in Theorem 2.3:
$$
\mathrm {U G W} (\mu , \nu) \leq \mathrm {G W} (\mu_ {c}, \nu_ {c}) + \tau_ {1} d _ {\mathbf {K L}} ^ {\otimes} (\mu_ {c}, \mu) + \tau_ {2} d _ {\mathbf {K L}} ^ {\otimes} (\nu_ {c}, \nu).
$$
We observe that the terms  $d_{\mathbf{KL}}^{\otimes}(\mu_c,\mu)$  and  $d_{\mathbf{KL}}^{\otimes}(\mu_c,\mu)$  cannot be canceled out unless  $\tau_{1}$  and  $\tau_{2}$  are set to zero, resulting in over-relaxation. However, RGW allows us to control the error terms  $d_{\mathbf{KL}}(\mu_c,\mu_a)$  and  $d_{\mathbf{KL}}(\nu_c,\nu_a)$  through the marginal relaxation parameters  $\rho_{1}$  and  $\rho_{2}$ .
To start with our algorithmic developments, we consider the discrete case for simplicity and practicality, where  $\mu$  and  $\nu$  are two empirical distributions, i.e.,  $\mu = \sum_{i=1}^{n} \mu_i \delta_{x_i}$  and  $\nu = \sum_{j=1}^{m} \nu_j \delta_{y_j}$ . Denote  $D \in S^n$ ,  $D_{ik} = d_X(x_i, x_k)$  and  $\bar{D} \in S^m$  and  $\bar{D}_{jl} = d_Y(y_j, y_l)$ . We construct a 4-way tensor as follows:
$$
\mathcal {L} (D, \bar {D}) := \left(\left| d _ {X} \left(x _ {i}, x _ {k}\right) - d _ {Y} \left(y _ {j}, y _ {l}\right) \right| ^ {2}\right) _ {i, j, k, l}.
$$
We define the tensor-matrix multiplication as
$$
\left(\mathcal {L} \otimes T\right) _ {i j} := \left(\sum_ {k, \ell} \mathcal {L} _ {i, j, k, \ell} T _ {k, \ell}\right) _ {i, j}.
$$
Then, the robust GW admits the following reformulation:
$$
\min  _ {\pi , \alpha , \beta} \langle \mathcal {L} (D, \bar {D}) \otimes \pi , \pi \rangle + \tau_ {1} d _ {\mathbf {K L}} (\pi_ {1}, \alpha) + \tau_ {2} d _ {\mathbf {K L}} (\pi_ {2}, \beta)
$$
$$
\text {s . t .} d _ {\mathbf {K L}} (\mu , \alpha) \leq \rho_ {1}, d _ {\mathbf {K L}} (\nu , \beta) \leq \rho_ {2}, \tag {3}
$$
$$
\alpha \in \Delta^ {n}, \beta \in \Delta^ {m}, \pi \geq 0.
$$
Here,  $\pi_1 = \pi \mathbf{1}_m$  and  $\pi_2 = \pi^T\mathbf{1}_n$
3.2 Bregman Proximal Alternating Linearized Minimization (BPALM) Method
As Problem (3) is non-convex and involves three variables, we employ BPALM [5, 2] to solve it. By choosing the KL divergence as Bregman distance, the updates of this algorithm are given by:
$$
\pi^ {k + 1} = \underset {\pi \geq 0} {\arg \min } \left\{\langle \mathcal {L} (D, \bar {D}) \otimes \pi^ {k}, \pi \rangle + \tau_ {1} d _ {\mathbf {K L}} \left(\pi_ {1}, \alpha^ {k}\right) + \tau_ {2} d _ {\mathbf {K L}} \left(\pi_ {2}, \beta^ {k}\right) + \frac {1}{t _ {k}} d _ {\mathbf {K L}} \left(\pi , \pi^ {k}\right) \right\}, \tag {4}
$$
$$
\alpha^ {k + 1} = \underset { \begin{array}{c} \alpha \in \Delta^ {n} \\ d _ {\mathbf {K L}} (\mu , \alpha) \leq \rho_ {1} \end{array} } {\arg \min } \left\{d _ {\mathbf {K L}} \left(\pi_ {1} ^ {k + 1}, \alpha\right) + \frac {1}{c _ {k}} d _ {\mathbf {K L}} \left(\alpha^ {k}, \alpha\right) \right\}, \tag {5}
$$
$$
\beta^ {k + 1} = \underset { \begin{array}{c} \beta \in \Delta^ {m} \\ d _ {\mathbf {K L}} (\nu , \beta) \leq \rho_ {2} \end{array} } {\arg \min } \left\{d _ {\mathbf {K L}} \left(\pi_ {2} ^ {k + 1}, \beta\right) + \frac {1}{r _ {k}} d _ {\mathbf {K L}} \left(\beta^ {k}, \beta\right) \right\}. \tag {6}
$$
Here,  $t_k$ ,  $c_k$ , and  $r_k$  are stepsizes in BPALM.
In our algorithm updates, we employ distinct proximal operators for  $\pi$  and  $\alpha$  (and  $\beta$ ). The use of  $d_{\mathbf{KL}}(\pi, \pi^k)$  in  $\pi$ -subproblem (4) allows for the application of the Sinkhorn algorithm, while the introduction of  $d_{\mathbf{KL}}(\alpha^k, \alpha)$  in  $\alpha$ -subproblem (5) facilitates a closed-form solution, which we will detail in the following part.
To solve the  $\pi$ -subproblem, we can utilize the Sinkhorn algorithm for the entropic regularized unbalanced optimal transport problem. This algorithm, which has been previously introduced in [13, 32], is well-suited for our needs. As for the  $\alpha$ -subproblem, we consider the case where  $\rho_{1}$  is strictly larger than 0. Otherwise, when  $\rho_{1} = 0$ ,  $\alpha$  should simply equal  $\mu$ , making the subproblem unnecessary. To solve the  $\alpha$ -subproblem, we attempt to find the optimal dual multiplier  $w^{*}$ . Specifically, consider the problem:
$$
\min  _ {\alpha \in \Delta^ {n}} d _ {\mathbf {K L}} \left(\pi_ {1} ^ {k + 1}, \alpha\right) + \frac {1}{c _ {k}} d _ {\mathbf {K L}} \left(\alpha^ {k}, \alpha\right) + w \left(d _ {\mathbf {K L}} (\mu , \alpha) - \rho_ {1}\right). \tag {7}
$$
Let  $\hat{\alpha}(w)$  represent the optimal solution to (7), and we define the function  $p: \mathbb{R}_+ \to \mathbb{R}$  by  $p(w) = d_{\mathbf{KL}}(\mu, \hat{\alpha}(w)) - \rho_1$ . We prove the convexity, differentiability, and monotonicity of  $p$ , which are crucial for developing an efficient algorithm for (5) later.
Proposition 3.1. Problem (7) has a closed-form solution
$$
\hat {\alpha} (w) = \frac {\pi^ {k + 1} \mathbf {1} _ {m} + \frac {1}{c _ {k}} \alpha^ {k} + w \mu}{\sum_ {i j} \pi_ {i j} ^ {k + 1} + \frac {1}{c _ {k}} + w}.
$$
If  $w$  satisfies (i)  $w = 0$  and  $p(w) \leq 0$ , or (ii)  $w > 0$ ,  $p(w) = 0$ , then  $\hat{\alpha}(w)$  is the optimal solution to the  $\alpha$ -subproblem (5). Moreover,  $p(\cdot)$  is convex, twice differentiable, and monotonically non-increasing on  $\mathbb{R}_+$ .
Given Proposition 3.1, we begin by verifying  $p(0) \leq 0$ . If this condition is not met, and given that  $p(0) > 0$  while  $\lim_{w \to +\infty} p(w) = -\rho_1 < 0$ , it implies that  $p$  possesses at least one root within  $\mathbb{R}_+$ . The following proposition provides the framework to seek the root of  $p$  by employing Newton's method, with the initialization set at 0. Hence, the  $\alpha$ -subproblem can be cast to search a root of  $p$  in one dimension, in which case it can be solved efficiently.
Proposition 3.2. Let  $p(\cdot): I \to \mathbb{R}$  be a convex, twice differentiable, and monotonically nonincreasing on the interval  $I \subset \mathbb{R}$ . Assume that there exists an  $\tilde{x}, \bar{x} \in I$  such that  $p(\tilde{x}) > 0$  and  $p(\bar{x}) < 0$ . Then  $p$  has a unique root on  $I$ , and the sequence obtained from Newton's method with initial point  $x_0 = \tilde{x}$  will converge to the root of  $p$ .
Since the  $\beta$ -subproblem shares the same structure as the  $\alpha$ -subproblem, we can apply this method to search for the optimal solution to the  $\beta$ -subproblem.
To illustrate the convergence result of BPALM, we consider the compact form for simplicity:
$$
\min _ {\alpha , \beta , \pi} F (\pi , \alpha , \beta) = f (\pi) + q (\pi) + g _ {1} (\pi , \alpha) + g _ {2} (\pi , \beta) + h _ {1} (\alpha) + h _ {2} (\beta),
$$