SlowGuess/NeurIPS_2023_OCR huggingface dataset:
DSPL vs. DSGD We give some further insights on the behavior of DSPL and DSGD for the composition function (2). Intuitively, as the algorithm converges, we have  $\lim_{k\to 0}\| x^k -x^{k + 1}\| = 0$  a.s. When  $x^{k}\approx x^{k - \tau_{k}}$ , DSPL enjoys an increasingly stable estimation of the proximal mapping (4), as the influence of delay is diminishing and the error is mainly driven by stochastic sampling. The same conclusion holds on smooth DSGD due to the Lipschitz continuity of  $\nabla f$ . On the other hand, when DSGD is applied for a nonsmooth problem, the master node receives an out-of-date subgradient  $f^{\prime}$  from the worker and solves (3). Since  $f(x,\xi)$  is nonsmooth, the subgradient  $f^{\prime}(x^{k - \tau_k},\xi)$  may differ significantly from  $f^{\prime}(x^{k},\xi)$  even when the sequence  $\{x^k\}$  converges. Hence, DSGD will constantly suffer from delay during all the updates (3).
DSPL with momentum We also remark that the momentum technique from DSGD can be extended to DSPL, which gives us the same  $\mathcal{O}(\tau_{\mathrm{max}} / \sqrt{K} +\tau_{\mathrm{max}}^2 /K)$  convergence rate as in [34]. We refer the interested readers to the Appendix F.
When  $f$  is smooth, the analysis of DSPL can be adapted to yield a comparable convergence rate for the proximal stochastic gradient method for minimizing  $\psi(\cdot)$ .
Theorem 3. Suppose all the assumptions in Lemma 1 and A4 hold, and that  $f$  is  $\lambda$ -smooth. Let  $\gamma_k \equiv \gamma = \rho + 6\lambda + \kappa + \sqrt{K} / \alpha$  for some  $\alpha > 0$ , then  $\mathbb{E}[\|\nabla \psi_{1/\rho}(x^{k^*})\|^2] = \mathcal{O}\left(\frac{1}{\sqrt{K}} + \frac{\lambda \Delta_2}{K}\right)$ .
So far we have spent two sections analyzing the two algorithms so that enough intuition can be established. All these serve for our ultimate goal: making DSGD and DSPL robust to arbitrary delays.
5 Weakly convex optimization robust to arbitrary delays
This section proposes robust variants of DSGD and DSPL, for which the explicit delays are eliminated from the convergence rate. What we will do is reduce the impact of delay to the number of agents in the network. Moving forward, we substitute A4 with D1.
D1: The distributed environment has  $m$  workers.
The previously established results have provided sufficient intuition on how delays impact our algorithms. In view of Theorem 1 and Theorem 2, we have isolated delay-dependent  $\mathcal{O}(\Delta_1 / \sqrt{K})$  and  $\mathcal{O}(\Delta_2 / K)$  in the proof. Although  $\Delta_1 / \sqrt{K}$  seems larger, it turns out that  $\Delta_2$  stands in our way. The following lemma shows  $\Delta_1 = \frac{1}{K}\sum_{k = 1}^{K}\tau_k$  is bounded by  $m$ .
Lemma 3. In a distributed environment of  $m$  workers  $\Delta_1 = \frac{1}{K} \sum_{k=1}^{K} \tau_k \leq m - 1 \leq m$ .
Given Lemma 3, we know  $\Delta_1 / \sqrt{K} = \mathcal{O}(m / \sqrt{K})$  and we can replace dependency on  $\Delta_{1}$  by  $m$ . Now we consider  $\Delta_{2} = \frac{1}{K}\sum_{k = 1}^{K}\tau_{k}^{2}$ . Even if we have a larger denominator  $K$  neutralizing the effect of  $\Delta_{2}$ , the following example shows that in the worst case,  $\Delta_{2}$  can be of  $\mathcal{O}(K)$  and result in  $\mathcal{O}(1)$  error.
Example 1 (Why  $\Delta_2$  hurts performance). Given sequence of delays  $\{\tau_k\}$  such that  $\tau_{k} = 0, k \leq K - 1, \tau_{K} = K$ . Then  $\Delta_2 = K$  and  $\Delta_2 / K = 1$ .
The example tells that delays of  $\mathcal{O}(K)$  ruin our convergence. In other words, to recover an overall  $\mathcal{O}(1 / \sqrt{K})$  convergence rate, we need  $\Delta_2 / K = \mathcal{O}(1 / \sqrt{K})\Rightarrow \sum_{k = 1}^{K}\tau_k^2 = \mathcal{O}(K^{3 / 2})$ . The next lemma provides a hint for our algorithm design.
Lemma 4. If a sequence of nonnegative integers  $\{\tau_k\}$  satisfies  $\sum_{k=1}^{K} \tau_k \leq mK$ , then given  $T \geq 0$ ,
1. if  $\tau_{k} \leq T$  for all  $k$ , then  $\sum_{k=1}^{K} \tau_{k}^{2} \leq mKT$ ;
2.  $\sum_{k = 1}^{K}\mathbb{I}\{\tau_k\leq T\} \geq K - mKT^{-1}.$
Lemma 4 tells us two facts about a nonnegative integer sequence of length  $K$  with sum bounded by  $\mathcal{O}(K)$ : 1). if we restrict the elements to be less than  $\mathcal{O}(T)$ , then  $\sum_{k=1}^{K} \tau_k^2 \leq \mathcal{O}(KT)$ . 2). there will be  $\Omega(K - mKT^{-1})$  elements bounded by  $T$ . Back to our context, this implies 1). To reduce  $\Delta_2 / K$  to  $\mathcal{O}(1 / \sqrt{K})$ , we can discard the iterations of delays greater than  $T = \mathcal{O}(\sqrt{K})$ . 2). We skip no more than  $\mathcal{O}(\sqrt{K})$  iterations and optimization works with  $\mathcal{O}(K - \sqrt{K}) = \mathcal{O}(K)$  iterations left.
Having established the foundational understanding, we outline the main steps in Algorithm 3. It is based on the aforementioned intuition and incorporates a "safeguarding" step to discard inaccurate information which could potentially hurt the convergence performance. As the above argument on the accumulated delay is independent of any specific algorithm, the safeguarding strategy can be applied to both DSGD and DSPL. To make our parameter setting more general, in our analysis we consider  $T = r^{-1}mK^{\beta}, r > 0, \beta \geq 0$ .
Algorithm 3: Safeguarded DSGD/DSPL
Input:  $x^{1},T = r^{-1}mK^{\beta}$
for  $k = 1,2,\dots$  do
if  $\tau_{k}\leq T$  then Update with
(3) for DSGD or (4) for DSPL
else  $x^{k + 1} = x^k$  end
Intuitively, under worst-case scenarios, Algorithm 3 will perform after  $K$  iterations in a manner similar to its non-safeguarded counterpart. However, the maximum delay will be capped at  $r^{-1}mK^{\beta}$ , and the iteration count will be  $K(1 - rK^{-\beta})$ .
Theorem 4 (Safeguarded DSGD). Under the same conditions as Lemma 1 as well as D1, taking  $\pmb{I}$ $\beta >0,K > r^{1 / \beta}$  or 2)  $\beta = 0,r <   1$  , then letting  $\gamma_{k}\equiv \gamma = \frac{\sqrt{K}}{\alpha\sqrt{\eta}} +\rho +\kappa +4\lambda ,\eta = 1 + \frac{r}{K^{\beta} - r}$  for
some  $\alpha > 0$  and  $k^*$  be uniformly chosen between iterations where  $\tau_k \leq T = r^{-1}mK^\beta$ ,
$$
\mathbb {E} [ \| \nabla \psi_ {1 / \rho} (x ^ {k ^ {*}}) \| ^ {2} ] \leq \frac {2 \rho}{\rho - 2 \lambda - \kappa} \Big [ \frac {\eta (\rho + 2 \lambda) D}{K} + \frac {\sqrt {\eta} D}{\sqrt {K} \alpha} + \frac {4 \eta^ {3 / 2} \rho L _ {f} (L _ {f} + L _ {\omega}) m \alpha}{\sqrt {K}} + \frac {8 \eta^ {2} \rho \lambda (L _ {f} + L _ {\omega}) ^ {2} m ^ {2} \alpha^ {2}}{r K ^ {1 - \beta}} \Big ],
$$
Theorem 5 (Safeguarded DSPL). Under the same conditions as Lemma 2 as well as D1, taking  $\mathbf{I}$ $\beta >0,K > r^{1 / \beta}$  or 2)  $\beta = 0,r < 1$ , then letting  $\gamma_{k}\equiv \gamma = \frac{\sqrt{K}}{\alpha\sqrt{\eta}} +\rho +\kappa +6\lambda ,\eta = 1 + \frac{r}{K^{\beta} - r}$  for some  $\alpha >0$  and  $k^{*}$  be uniformly chosen between iterations where  $\tau_k\leq T = r^{-1}mK^\beta$
$$
\mathbb {E} [ \| \nabla \psi_ {1 / \rho} (x ^ {k ^ {*}}) \| ^ {2} ] \leq \frac {2 \rho}{\rho - 2 \lambda - \kappa} \Big [ \frac {\eta (\rho + 4 \lambda) D}{K} + \frac {\sqrt {\eta} D}{\sqrt {K} \alpha} + \frac {2 \sqrt {\eta} \rho L _ {f} \alpha}{\sqrt {K}} + \frac {6 \eta^ {2} \rho \lambda (L _ {f} + L _ {\omega}) ^ {2} m ^ {2} \alpha^ {2}}{r K ^ {1 - \beta}} \Big ].
$$
Remark 5. Theorem 4 and 5 show that by employing a safeguarding step, both DSGD and DSPL can achieve delay-independent rates. It is also interesting to see how the choice of  $\beta$  and  $r$  affects performance. To recover a convergence rate of  $\mathcal{O}(K^{-1/2})$ , it is sufficient to have  $\beta \leq 1/2$ . If we set  $\beta > 0$ , then we skip  $rK^{-\beta}$  of all the iterations and incur a penalty of up to  $\eta^{3/2}$  in DSGD and  $\eta^{1/2}$  in DSPL. However, this loss becomes negligible for large  $K$ , as  $\eta = 1 + \frac{r}{K^{\beta-r}} \to 1$ . Alternatively, if we set  $\beta = 0$ , DSGD achieves  $\mathcal{O}(\sqrt{m}/\sqrt{K} + m/K)$  rate with  $\alpha = 1/\sqrt{m}$  while DSPL yields a rate of  $\mathcal{O}(1/\sqrt{K} + m^2/K)$  with  $\alpha = 1$ . This setting aligns with [7, 18] and achieves optimal rate on  $m$ . However, the penalty from  $\eta > 1$  is non-negligible and adversely affects the overall convergence rate by a constant factor. Therefore, we can strike a balance in practice by choosing  $\beta$  in  $(0,1/2]$  and allow delays of up to  $\mathcal{O}(K^{\beta})$ .
This section performs numerical experiments on the robust phase retrieval problem to demonstrate the efficiency of our methods. Given a measuring matrix  $A \in \mathbb{R}^{m \times n}$  and a set of observations  $b_{i} \approx |\langle a_{i}, \hat{x} \rangle|^{2}, 1 \leq i \leq m$  ( $m$  in this section represents the number of samples), robust phase retrieval aims to recover the true signal  $\hat{x}$  from
$$
\min _ {x \in \mathbb {R} ^ {n}} \frac {1}{m} \sum_ {i = 1} ^ {m} | \langle a _ {i}, x \rangle^ {2} - b _ {i} | + \delta_ {\{x: \| x \| \leq M \}},
$$
where  $\delta_S$  denotes the set indicator function and the  $\ell_1$  loss function improves robustness. Our experiment contains three parts. The first part profiles algorithms in an asynchronous environment simulated via MPI; our second experiment runs sequentially with simulated delays from common distributions; our last experiment also runs in the simulated environment and demonstrates the effectiveness of safeguarding step under adversarially chosen delays.
6.1 Experiment setup
Synthetic data. For the synthetic data, we take  $m = 300$ ,  $n = 100$  in the experiments of simulated delay and  $m = 1500$ ,  $n = 500$  in the asynchronous environment. Data generation follows the setup of [11], where, given some  $\kappa \geq 1$ , we compute  $A = QD$ ,  $Q \in \mathbb{R}^{m \times n}$ ,  $q_{ij} \sim \mathcal{N}(0,1)$  and  $D = \mathrm{diag}(d)$ ,  $d \in \mathbb{R}^n$ ,  $d_i \in [1 / \kappa, 1]$  for all  $i$ . Then we generate a true signal  $\hat{x} \sim \mathcal{N}(0,I)$  and obtain the measurements  $b$  using formula  $b_i = \langle a_i, \hat{x} \rangle^2$ . Last we randomly choose  $p_{\mathrm{fail}}$ -fraction of the measurements and add  $\mathcal{N}(0,25)$  to them to simulate data corruption.
Real-life data. The real-life data is generated from zipcode dataset, where we vectorize a  $16 \times 16$  hand-written digit from [16] and use it as the signal. The measuring matrix  $A$  comes from a normalized Hadamard matrix  $H \in \mathbb{R}^{256 \times 256}$ : we generate three diagonal matrices  $S_{j} = \mathrm{diag}(s_{j}), j = 1, 2, 3$ ; each element of  $s_{j} \in \mathbb{R}^{256}$  is taken from  $\{-1, 1\}$  randomly and we let  $A = H[S_{1}, S_{2}, S_{3}]^{\mathrm{T}} \in \mathbb{R}^{768 \times 256}$ . Finally  $p_{\mathrm{fail}}$ -fraction of the observations are set 0.
1) Dataset. In the asynchronous environment, we keep up with [34] setting  $\kappa = 1$ ,  $p_{\mathrm{fail}} = 0$  and in the simulated environment, we follow [11] setting  $\kappa \in \{1, 10\}$  and  $p_{\mathrm{fail}} \in \{0.2, 0.3\}$ .
2) Initial point and radius. Synthetic data: we generate  $x' \sim \mathcal{N}(0, I_n)$  and start from  $x^1 = \frac{x'}{\|x'\|}$ ; zipcode data: we generate  $x' \sim \mathcal{N}(\hat{x}, I_n)$  and take  $x^1 = 10x'$ .  $M = 1000 \| x^1 \|$ .
3) Stopping criterion. We run algorithms for 400 epochs ( $K = 400m$ ). In the asynchronous environment, algorithms run until reaching the maximum iteration. In the simulated environment, algorithms stop if  $f \leq 1.5f(\hat{x})$ . When  $f$  contains corrupted measurements,  $f(\hat{x}) > 0$ .
4) Stepsize. We set  $\gamma = \sqrt{K / \alpha}$ , where  $\alpha \in \{0.1, 0.5, 1.0\}$  in the asynchronous environment,  $\alpha \in [10^{-2}, 10^{1}]$  for synthetic data and  $\alpha \in [10^{1}, 10^{2}]$  for the zipcode dataset.
5) Simulated delay. In the simulated environment, we generate  $\tau_{k}$  from two common distributions from literature, which are geometric  $\mathcal{G}(p)$  and Poisson  $\mathcal{P}(\lambda)$  [37]. After the delay is generated, it is truncated by twice the mean of the distribution.
6) Adversarial delay. We let delay happen at the last iteration of each epoch and use the information of  $x^{1}$  to update. Safeguarding parameter is set to  $T = 0.1\sqrt{K}$ .
7) Trade-off between computation and communication. In the asynchronous environment, the numerical linear algebra on the worker uses a raw implementation (not importing package) to balance the cost of gradient computation and communication.
6.2 Asynchronous environment
Our first experiment runs in an asynchronous environment implemented by MPI Python interface and is profiled on an Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz machine with 10 cores and 20 threads. This experiment runs on a single machine to verify our theoretical finding rather than to test the algorithm's real performance on a specific distributed architecture.
Figure 2: First: speedup in time and the number of workers. Second: progress of  $\| x^k - \hat{x}\|$  in the first 40 epochs given  $\alpha = 0.1$ . Third:  $f(x^{k}) - f(\hat{x})$  in the first 40 epochs given  $\alpha = 0.5$ . For more details about the two figures on the right, please refer to Figure 12 in the appendix.
The first figure plots the wall-clock time (in seconds) for DSGD and DSPL to complete 400 epochs when the number of workers increases. It is observed that both algorithms exhibit speed-up with more workers and note that DSPL takes more time than DSGD due to the need to pass the function value to the master and slightly more expensive updates. But as the second and the third figure suggest, this extra cost is justified by the superior convergence: in the first several epochs DSPL reaches a high accuracy of  $10^{-6}$  in both function value and distance to the optimal solution, while DSGD stagnates at a relatively low-accuracy solution of  $10^{-2}$ . These observations suggest that DSPL offers better convergence behavior than the methods only based on subgradient. Finally, our experiments suggest both DSGD and DSPL are not sensitive to the increase in the number of workers when there are relatively few workers.
6.3 Simulated environment
The second part of our experiment compares the performance between DSGD and DSPL and is based on the simulated delay, where the algorithm runs sequentially but the gradient information is computed from the previous iterates.
Figure 3 plots the number of iterations for each algorithm to converge under different datasets and delay parameters. We see that in spite of delays, DSPL admits a wider range of stepsize parameters ensuring convergence than DSGD, and the performance slightly deteriorates as delay increases.
Figure 3: Left two: Geometric delay  $(\kappa, p_{\mathrm{fail}}) = (1,0.3), 2p^{-1} \in \{28,47\}$ ; right two: Poisson delay  $(\kappa, p_{\mathrm{fail}}) = (10,0.2), 2\lambda \in \{28,47\}$ . x-axis: parameter  $\alpha$ ; y-axis: number of iterations. Definition of delay is given in experiment setup 5).
6.4 Adversarial delay
The final experiment verifies the efficacy of our safeguarding step when delays are introduced in an adversarial setting. We evaluate the performance of DSGD, DSPL, both with and without the safeguarding step, under the delay patterns we have generated.
Figure 4: From left to right: zipcode data,  $p_{\mathrm{fail}} = 0.2$ ,  $\alpha \in \{10,100\}$ ,  $p_{\mathrm{fail}} = 0.3$ ,  $\alpha \in \{10,100\}$ . x-axis: iteration count; y-axis:  $f(x^{k})$ .
Figure 4 clearly illustrates the superior performance of our method incorporating the safeguarding step. As suggested by our theory, DSGD is notably sensitive to delays of  $\mathcal{O}(K)$ . However, upon discarding outdated information, DSGD exhibits much greater robustness. Interestingly, even under our adversarial setup, the performance of DSPL remains acceptable, albeit slightly inferior to its safeguarded version, which aligns well with our theoretical findings.
We offer a sharp analysis of delayed stochastic algorithms for weakly convex optimization, discussing the widely utilized DSGD method and introducing the novel DSPL method for problems with a composition structure. Through careful examination of delay factors, we propose a straightforward safeguarding approach to eliminate the effect of delays, and instead derive bounds depending on the number of agents in the distributed environment. This makes our algorithms resilient to arbitrary delays. A promising future direction is the application of these prox-linear methods in more diverse distributed settings, such as decentralized networks.
The authors thank the anonymous reviewers for their constructive suggestions. This research is partially supported by National Natural Science Foundation of China (NSFC-72150001, 11831002).
[1] Alekh Agarwal and John C Duchi. Distributed delayed stochastic optimization. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
A More on the related works 14
B Bregman context and auxiliary results 15
B.1 Context setup 15
B.2 Auxiliary lemmas 15
C Convergence Analysis of DSGD 16
C.1 Proof of Lemma 1 17
C.2 Proof of Theorem 1 18
D Convergence analysis of DSPL 19
D.1 Preliminaries and analysis of Bregman DSPL 19
D.2 Proof of Lemma 2 and Theorem 2 24
D.3 Proof of Theorem 3 24
E Delay-independent analysis 24
E.1 Proof of Lemma 3 24
E.2 Proof of Lemma 4 25
E.3 Proof of Theorem 4 25
E.4 Proof of Theorem 5 26
F DSPL with momentum 26
F.1 Preliminaries of DSEPL 26
F.2 Convergence analysis of DSEPL 28
G Kernel and subproblems 33
G.1 Choosing the divergence kernel 33
G.2 Bregman proximal subproblem 34
G.3 Euclidean subproblem with nonsmooth regularizer 35
H Additional experiments 36
H.1 Robust phase retrieval 36
H.2 Blind deconvolution problem 37
H.3 Proximal sub-problems 39
H.4 Separate figures of Section 6 40
Structure of the appendix In this paper, we strive to provide a comprehensive treatment of delayed stochastic algorithms for stochastic weakly convex optimization in both theoretical and practical aspects.
1) Theoretically, we choose to employ a more general Bregman divergence context in our proof. This broader context covers the primary results for the Euclidean setting discussed in the main section. Specifically, this context will be invoked in our analysis for DSPL method to accommodate real-life applications of prox-linear method and by relaxing the Lipschitz condition C1.
2) Practically, we provide a detailed discussion on how to implement the DSPL algorithm, both in Euclidean and Bregman context. We specifically address the selection of the Bregman divergence kernel and propose efficient subroutines to solve proximal subproblems. Additionally, in view of the wide usage of momentum techniques in stochastic optimization, we analyze DSEPL, the momentum variant of DSPL. While this is not the central contribution of our paper, we believe it represents an important step for making DSPL applicable to real-world problems.
Figure 5: Structure of the appendix
The appendix is organized as follows.
The first five sections address the theoretical aspects of delayed stochastic algorithms.
In Section A, we first conduct an extended review of other related works. In Section B, we introduce the Bregman context with some auxiliary results. In Section C, we perform convergence analysis for the DSGD method, and Section D presents a more general Bregman DSPL that relaxes the Lipschitzness assumption. Section E presents the safeguarded variant of our algorithms that are robust to arbitrary delays.
(Optional) The last three sections are devoted to more practical aspects.
In Section F, we present the convergence analysis of DSEPL, namely, the momentum variant of DSPL using variable extrapolation. Section G discusses how to construct the kernel and solve the Bregman proximal subproblems. Finally, Section H displays additional experiment results.
A More on the related works
First, we review the literature on stochastic weakly-convex optimization. The Prox-Linear (PL) method [19, 12, 15] has received more attention lately. The seminal work [8] conducts a novel complexity analysis using the Moreau envelope as the potential function. They show that SGD (and many other stochastic algorithms) achieves an  $\mathcal{O}\left(\frac{1}{\varepsilon^4}\right)$  complexity in terms of convergence to the proximity of approximate stationary points. [20] analyzes an incremental subgradient method for finite-sum problems. [36] extends the prox-linear algorithm to the finite-sum setting and provides analysis based on variance-reduction. SGD with momentum is studied in [24] and [11] reveals that SPL can be accelerated by minibatching even for nonsmooth objective functions. [11] also employs heavy-ball momentum to further improve the model-based optimization.
There is substantial literature on distributed and asynchronous optimization. We refer to the seminal work [4]. In addition to the aforementioned asynchronous issues, another important research direction concerns reducing the network communication cost by gradient compression. For example, see [17]. Besides the centralized setting, there is also growing interest in decentralized algorithms. Recently, [6] presents a decentralized SGD in the network where nodes exchange parameters and subgradients locally. However, their method requires all the nodes to update synchronously and only
proves asymptotic convergence of SGD. An extensive study of decentralized optimization is beyond our work. Therefore, we refer interested readers to [22, 23, 35] for more recent advances.
B Bregman context and auxiliary results
We initiate the theoretical discussion by introducing the Bregman context, which includes the concepts of Bregman divergence, relative Lipschitzian property, and Bregman Moreau envelope. Following the provision of requisite definitions, we present some auxiliary results that will be used frequently in our analysis.
B.1 Context setup
Bregman divergence and relative Lipschitzness Given a Legendre function  $d(\cdot)$  [7] that is proper, closed, strictly convex and essentially smooth, it induces the Bregman divergence defined by
$$
V _ {d} (x, y) := d (x) - d (y) - \langle \nabla d (y), x - y \rangle
$$
and  $d$  is called the kernel of  $V_{d}$ . Since  $d$  is strictly convex, we know  $V_{d}(x,y)\geq 0$  and the equality holds if and only if  $x = y$ . The notion of Bregman divergence greatly enhances the coverage of the algorithms and has recently been adopted in the context of weakly convex optimization [8, 3] to tackle a broader class of nonsmooth nonconvex optimization problems. In this paper, we mainly leverage the concept of relative Lipschitzian property to extend the analysis of delayed stochastic algorithms. First, we give its formal definition.
Definition 1 (Relative Lipschitzian property). [3] A function  $f$  satisfies  $L$ -relative Lipschitzian property to kernel  $d$  if for all  $x, y \in \operatorname{dom} d$ ,
$$
f (x) - f (y) \leq L \sqrt {2 V _ {d} (y , x)}. \tag {5}
$$
Bregman proximal mapping and Bregman envelope
As in conventional weakly convex optimization, we need an approximate measure of stationarity in the non-Euclidean setting. A naturally choice is the Bregman envelope [3]. Assume that  $d$  is 1-strongly convex, then for any  $\rho >0$ , the Bregman envelope of  $f$  with respect to kernel  $d$  is
$$
\psi_ {1 / \rho} ^ {d} (x) := \min  _ {y} \{f (y) + \rho V _ {d} (y, x) \}
$$
and the Bregman proximal mapping is represented using
$$
\operatorname{prox}_{\psi /\rho}^{d}(x):= \operatorname *{arg  min}_{y}\{f(y) + \rho V_{d}(y,x)\} .
$$
We sometimes refer to  $\mathrm{prox}_{\psi/\rho}^d(x)$  as  $\hat{x}$  and proximal distance  $\mathbb{E}[V_d(\hat{x}^k, x^k)]$  has been proven a proper measure of the convergence of stochastic algorithms [3] under specific choice of  $d$  and  $\rho$ .
Connection with Euclidean geometry One advantage of using Bregman context is that it subsumes the Euclidean case. If we take  $d(x) = \frac{1}{2} \| x\| ^2$ , then  $V_{d}(x,y) = \frac{1}{2}\| x - y\|^{2}$  and relative Lipschitzian property becomes standard Lipschitz condition. Particularly we have the following relation between  $V_{d}(\hat{x},x)$  and Moreau envelope.