SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Wenhan Xian
Computer Science University of Maryland College Parkwxian1@umd.edu
Computer Science  
University of Maryland College Park  
heng@umd.edu
In this paper we study the second-order optimality of decentralized stochastic algorithm that escapes saddle point efficiently for nonconvex optimization problems. We propose a new pure gradient-based decentralized stochastic algorithm PEDESTAL with a novel convergence analysis framework to address the technical challenges unique to the decentralized stochastic setting. Our method is the first decentralized stochastic algorithm to achieve second-order optimality with non-asymptotic analysis. We provide theoretical guarantees with the gradient complexity of  $\tilde{O} (\epsilon^{-3})$  to find  $O(\epsilon ,\sqrt{\epsilon})$ -second-order stationary point, which matches state-of-the-art results of centralized counterparts or decentralized methods to find first-order stationary point. We also conduct two decentralized tasks in our experiments, a matrix sensing task with synthetic data and a matrix factorization task with a real-world dataset to validate the performance of our method.
Decentralized optimization is a class of distributed optimization that trains models in parallel across multiple worker nodes over a decentralized communication network. Decentralized optimization has recently attracted increased attention in machine learning and emerged as a promising framework to solve large-scale tasks because of its capability to reduce communication costs. In the conventional centralized paradigm, all worker nodes need to communicate with the central node, which results in high communication cost on the central node when the number of nodes is large or the transmission between the center and some remote nodes suffers network latency. Conversely, decentralized optimization avoids these issues since each worker node only communicates with its neighbors.
Although decentralized optimization has shown advantageous performance in many previous works (Lian et al. [2017], Tang et al. [2018]), the study of second-order optimality for decentralized stochastic optimization algorithms is still limited. Escaping saddle point and finding local minima is a core problem in nonconvex optimization since saddle point is a category of first-order stationary point that can be reached by many gradient-based optimizers such as gradient descent but it is not the expected point to minimize the objective function.
Perturbed gradient descent (Jin et al. [2017]) and negative curvature descent (Xu et al. [2018], Allen-Zhu and Li [2018]) are two primary pure gradient-based methods (not involving second-order derivatives) to achieve second-order optimality. Typically, perturbed gradient descent method is composed of a descent phase and an escaping phase. If the norm of gradient is large, the algorithm will run the descent phase as normal. Otherwise it will run the escaping phase to discriminate whether the candidate first-order stationary point is a saddle point or local minimum. Negative curvature
descent method escapes saddle point by computing the direction of negative curvature at the candidate point. If it is categorized as a saddle point then the algorithm will update along the direction of negative curvature. Generally it involves a nested loop to perform the negative curvature subroutine.
Currently, the solution to the second-order optimality of decentralized problem in deterministic setting has been proposed. Perturbed Decentralized Gradient Tracking (PDGT) (Tziotis et al. [2020]) is a decentralized deterministic algorithm adopting the perturbed gradient descent strategy to achieve second-order stationary point. However, it is expensive to compute full gradients for large machine learning models. It is crucial to propose a stochastic algorithm to obtain second-order optimality for decentralized problems. Besides, there are some drawbacks of PDGT to make it less efficient and hard to be generalized to the stochastic setting. These drawbacks are also the key challenges to achieve second-order optimality for decentralized algorithms, which are listed as follows:
(1) PDGT runs fixed numbers of iterations in descent phase and escaping phase such that the phases of all nodes can be changed simultaneously. This strategy works because the descent is easy to be estimated in deterministic setting. Nonetheless, the exact descent of stochastic algorithm over a fixed number of iterations is hard to be bounded because of randomness and noises. If the fixed number is not large enough it is possible that the averaged model parameter is not a first-order stationary point. If the fixed number is as large as the expected number of iterations to achieve first-order stationary point, the algorithm will become less efficient as it is probably stuck at a saddle point for a long time before drawing the perturbation, especially in the second and later descent phase. Specifically, applying fixed number of iterations in each phase results in the complexity of at least  $\tilde{O} (\epsilon^{-4.5})$  (see Appendix D), which is higher than  $\tilde{O} (\epsilon^{-3})$  of our method. Therefore, we are motivated to propose an algorithm that can change phases adaptively (based on runtime gradient norm) and independently (not required to consider status on other nodes or notify other nodes).
(2) In PDGT the perturbations on all nodes are drawn from the same random seed. Besides, a coordinating protocol involving broadcast and aggregation is used to compute the averaged model parameter and the descent of overall loss function to discriminate the candidate point. These strategies together with the fixed number of iterations act as a hidden coordinator to make PDGT discriminate saddle point in the same way as centralized algorithms. However, when the number of worker nodes is large it is time-consuming to perform broadcast or aggregation over the whole decentralized network. Moreover, when generalized to stochastic setting the changing of phase is not guaranteed to be synchronized. Additionally, we will note in the Supplementary Material that the consensus error  $\frac{1}{n}\sum_{i=1}^{n}\|x_t^{(i)} - \bar{x}_t\|^2$  is another factor to impact the effectiveness of perturbed gradient descent, which is not present in centralized problems. All above issues are theoretical difficulties to study and ensure second-order optimality for decentralized stochastic algorithms.
(Vlaski and Sayed [2020]) proves the theoretical guarantee of second-order optimality for decentralized stochastic algorithm with perturbed gradient descent. However, it does not provide a non-asymptotic analysis to estimate the convergence rate or gradient complexity. The effectiveness of the result relies on a sufficiently small learning rate, and it does not present a specific algorithm. The analysis is based on the assumption that the iteration formula can be approximated by a centralized update scheme when the learning rate is small enough. Nevertheless, in practice it is difficult to maintain an ideally small learning rate, and the iterative update process can be more complex as previously mentioned. To our best knowledge, the second-order optimality issue of decentralized stochastic algorithm with non-asymptotic analysis is still not solved. Therefore, we are motivated to study this important and challenging issue and raise the following questions:
Can we design a decentralized stochastic optimization algorithm with non-asymptotic analysis to find local minima efficiently? Is the algorithm still effective to discriminate saddle point even if each node can change its phase adaptively and independently without any coordinating protocols?
The answer is affirmative. In this paper, we propose a novel gradient-based algorithm named PERTurbed DEcentralized STORM ALgorithm (PEDESTAL) which is the first decentralized stochastic algorithm to find second-order stationary point. We adopt perturbed gradient descent to ensure the second-order optimality and use STORM (Cutkosky and Orabona [2019]) estimator to accelerate the convergence. We provide completed convergence analysis to guarantee the second-order optimality theoretically. More details about the reason of choosing perturbed gradient descent and technical difficulties are discussed in Section 3.2. Next we will introduce the problem setup in this paper.
We focus on the following decentralized optimization problem:
$$
\min  _ {x} f (x) = \frac {1}{n} \sum_ {i = 1} ^ {n} f _ {i} (x), f _ {i} (x) = \mathbb {E} _ {\xi \sim D _ {i}} F _ {i} (x, \xi) \tag {1}
$$
where  $n$  is the number of worker nodes in the decentralized network and  $f_{i}$  is the local loss function on  $i$ -th worker node. Here  $f_{i}$  is supposed to take the form of stochastic expectation over local data distribution  $D_{i}$ , which covers a variety of optimization problems including finite-sum problem and online problem. Data distributions on different nodes are allowed to be heterogeneous. The objective function  $f$  is nonconvex such that saddle points probably exist.
The goal of our method is to find  $O(\epsilon, \epsilon_H)$ -second-order stationary point of problem 1, which is defined by the point  $x$  satisfying  $\| \nabla f(x) \| \leq \epsilon$  and  $\min e i g(\nabla^2 f(x)) \geq -\epsilon_H$ , where  $e i g(\cdot)$  represents the eigenvalues. The classic setting is  $\epsilon_H = \sqrt{\epsilon}$ .
We summarize the contributions of this paper as follows:
- We propose a novel algorithm PEDESTAL, which is the first decentralized stochastic gradient-based algorithm to achieve second-order optimality with non-asymptotic analysis.
- We provide a new analysis framework to support changing phases adaptively and independently on each node without any coordinating protocols involving broadcast or aggregation. We also address certain technical difficulties unique to decentralized optimization to justify the effectiveness of perturbed gradient descent in discriminating saddle point.
- We prove that our PEDESTAL achieves the gradient complexity of  $\tilde{O} (\epsilon^{-3} + \epsilon \epsilon_H^{-8} + \epsilon^4\epsilon_H^{-11})$  to find  $O(\epsilon ,\epsilon_{H})$ -second-order stationary point. Particularly, PEDESTAL achieves the gradient complexity of  $\tilde{O} (\epsilon^{-3})$  in the classic setting  $\epsilon_{H} = \sqrt{\epsilon}$ , which matches state-of-the-art results of centralized counterparts or decentralized methods to find first-order stationary point.
In this section we will introduce the background of related works. The comparison of important features is shown in Table 1. Here  $\tilde{O} (\cdot)$  refers to the big  $O$  notation that hides the logarithmic terms.
2.1 Decentralized Algorithms for First-Order Optimality
Decentralized optimization is an efficient framework to solve problem 1 collaboratively by multiple worker nodes. In each iteration a worker node only needs to communicate with its neighbors. One of the best-known decentralized stochastic algorithm is D-PSGD (Lian et al. [2017]), which integrates average consensus with local stochastic gradient descent steps and shows competitive result to centralized SGD. The ability to address Non-IID data is a limitation of D-PSGD and some variants of D-PSGD are studied to tackle the data heterogeneity issue, such as  $D^2$  (Tang et al. [2018]) by storing previous status and GT-DSGD (Xin et al. [2021b]) by using gradient tracking (Xu et al. [2015], Lorenzo and Scutari [2016]). D-GET (Sun et al. [2020]) and D-SPIDER-SFO (Pan et al. [2020]) improve the gradient complexity of D-PSGD from  $O(\epsilon^{-4})$  to  $O(\epsilon^{-3})$  by utilizing variance reduced gradient estimator SPIDER (Fang et al. [2018]). GT-HSGD also achieves gradient complexity of  $O(\epsilon^{-3})$  by combining gradient tracking and STORM gradient estimator (Cutkosky and Orabona [2019]). SPIDER requires a large batchsize of  $O(\epsilon^{-1})$  on average and a mega batchsize of  $O(\epsilon^{-2})$  periodically. In contrast, STORM only requires a large batch in the first iteration. After that the batchsize can be as small as  $O(1)$ , which makes STORM more efficient to be implemented in practice.
2.2 Centralized Algorithms for Second-Order Optimality
Perturbed gradient descent is a simple and effective method to escape saddle points and find local minima. PGD (Jin et al. [2017]) is the representative of this family of algorithms, which achieves second-order optimality in deterministic setting. It draws a perturbation when the gradient norm is small. If this point is a saddle point, the loss function value will decrease by a certain threshold within a specified number of iterations (i.e., breaking the escaping phase) with high probability. Otherwise, the candidate point is regarded as a second-order stationary point. In stochastic setting, Perturbed SGD perturbs every iteration and suffers a high gradient complexity of  $O(\epsilon^{-8})$  to achieve  $O(\epsilon, \sqrt{\epsilon})$ -second-order stationary point and the gradient complexity hides a polynomial factor of dimension  $d$ . CNC-SGD requires a Correlated Negative Curvature assumption and the gradient complexity of  $\tilde{O} (\epsilon^{-5})$  to achieve the classic second-order optimality. SSRGD (Li [2019]) adopts the
Name	Averaged Batchsize	Gradient Complexity	Classic Setting
D-PSGD [12]	O(1)	O(ε-4)	-
GT-DSGD [22]	O(1)	O(ε-4)	-
D-GET [16]	O(ε-1)	O(ε-3)	-
D-SPIDER-SFO [15]	O(ε-1)	O(ε-3)	-
GT-HSGD [21]	O(1)	O(ε-3)	-
SGD+Neon2 [1]	O(1)	O(ε-4+ε-2ε-3H+ε-5H)	O(ε-4)
SCSG+Neon2 [1]	O(ε-0.5)	O(ε-10/3+ε-2ε-3H+ε-5H)	O(ε-3.5)
Natasha2+Neon2 [1]	O(ε-2)	O(ε-3.25+ε-3ε-1H+ε-5H)	O(ε-3.5)
SPIDER-SFO+ [5]	O(ε-1)	O(ε-3+ε-2ε-2H+ε-5H)	O(ε-3)
Perturbed SGD [6]	O(1)	O(ε-4+ε-16H)	O(ε-8)
CNC-SGD [4]	O(1)	O(ε-4+ε-10H)	O(ε-5)
SSRGD [11]	O(ε-1)	O(ε-3+ε-2ε-3H+ε-1ε-4H)	O(ε-3.5)
Pullback [2]	O(ε-1)	O(ε-3+ε-6H)	O(ε-3)
PDGT [18]	Full	-	-
PEDESTAL-S (ours)	O(1)	O(ε-3), εH≥ε0.2	-
PEDESTAL (ours)	O(ε-3/4)	O(ε-3+εε-8H+ε4ε-11H)	O(ε-3)
Table 1: The comparison of important properties between related algorithms and our PEDESTAL. Column "Averaged Batchsize" is computed when  $\epsilon_{H} = \sqrt{\epsilon}$ . Column "Classic Setting" refers to the gradient complexity under the classic condition  $\epsilon_{H} = \sqrt{\epsilon}$ . The first group of algorithms are decentralized methods achieving first-order optimality. The second group of algorithms are centralized methods achieving second-order optimality. The last group of algorithms are decentralized methods achieving second-order optimality. PEDESTAL-S is a special case of PEDESTAL with  $O(1)$  batchsize. The complexity of PDGT is not shown because it is not stochastic.
same two-phase scheme as PGD but uses the moving distance as the criterion to discriminate saddle point in the escaping phase. It also takes advantage of variance reduction to improve the gradient complexity to  $\tilde{O} (\epsilon^{-3.5})$ . Pullback (Chen et al. [2022]) proposes a pullback step to further enhance the gradient complexity to  $\tilde{O} (\epsilon^{-3})$ , which matches the best result of reaching first-order stationary point.
2.3 Stochastic Gradient Descent
A branch of study of stochastic gradient descent argues that SGD can avoid saddle point under certain conditions. However, that is completely different from the problem we focus on. In this paper we propose a method that can find local minima effectively for a general problem 1, while escaping saddle point by stochastic gradient itself depends on some additional assumptions. For example, (Mertikopoulos et al. [2020]) requires the noise of gradient should be uniformly excited. According to our experimental result in Section 5, we can see in some cases stochastic gradient descent cannot escape saddle point effectively or efficiently. Besides, the gradient noise in variance reduced methods is reduced in order to accelerate the convergence. Our experimental results indicate that the gradient noise in variance reduced algorithms is not as good as SGD to serve as the perturbation to avoid saddle point. Therefore, it is necessary to study the second-order stationary point for variance reduced algorithms so as to enable both second-order optimality and fast convergence.
In this section, we will introduce our PEDESTAL algorithm, which is demonstrated in Algorithm 1. Suppose there are  $n$  worker nodes in the decentralized communication network connected by a weight matrix  $W$ . The initial value of model parameters on all nodes are identical and equal to  $x_0$ .  $x_{t}^{(i)}$ ,  $v_{t}^{(i)}$  and  $y_{t}^{(i)}$  are the model parameter, gradient estimator and gradient tracker on the  $i$ -th worker node in iteration  $t$ .  $z_{t}^{(i)}$  is the temporary model parameter that is awaiting communication.  $\bar{x}_t$ ,  $\bar{v}_{t}$  and  $\bar{y}_t$  are corresponding mean values over all nodes. Counter  $esc^{(i)}$  counts the number of iterations in the
Algorithm 1 Perturbed Decentralized STORM Algorithm (PEDESTAL)
Input: initial value  $x_0^{(i)} = x_0$ ,  $v_{-1}^{(i)} = 0$ ,  $y_{-1}^{(i)} = 0$ ,  $\text{esc}^{(i)} = -1$ .  
Parameter:  $b_0, b_1, \eta, \beta, r, C_v, C_d, C_T$ .  
1: On i-th node:  
2: for  $t = 0, 1, \ldots, T - 1$  do  
3: if  $t = 0$  then  
4: Compute  $v_0^{(i)} = \nabla F_i(x_0, \xi_0^{(i)})$  with  $|\xi_0^{(i)}| = b_0$ .  
5: else  
6: Compute  $v_t^{(i)} = \nabla F_i(x_t^{(i)}, \xi_t^{(i)}) + (1 - \beta)(v_{t-1}^{(i)} - \nabla F_i(x_{t-1}^{(i)}, \xi_t^{(i)}))$  with  $|\xi_t^{(i)}| = b_1$ .  
7: end if  
8: Communicate and update the gradient tracker:  $y_t^{(i)} = \sum_{j=1}^n w_{ij}(y_{t-1}^{(j)} + v_t^{(j)} - v_{t-1}^{(j)})$ .  
9: if  $\text{esc}^{(i)} = -1$  and  $\| y_t^{(i)}\| \leq C_v$  then  
10: Draw a perturbation  $\xi \sim B_0(r)$  and update  $z_t^{(i)} = x_t^{(i)} + \xi$ .  
11: Save  $x_t^{(i)}$  as  $\tilde{x}^{(i)}$  and set  $\text{esc}^{(i)} = 0$ .  
12: else  
13: Update  $z_t^{(i)} = x_t^{(i)} - \eta y_t^{(i)}$ .  
14: end if  
15: Communicate and update the model parameter:  $x_{t+1}^{(i)} = \sum_{j=1}^n w_{ij} z_t^{(j)}$ .  
16: if  $\text{esc}^{(i)} \geq 0$  then  
17: Reset  $\text{esc}^{(i)} = -1$  if  $\| x_{t+1}^{(i)} - \tilde{x}^{(i)} \| > C_d$  else update  $\text{esc}^{(i)} = \text{esc}^{(i)} + 1$ .  
18: end if  
19: end for  
Return:  $\bar{x}_{t-C_T}$  if there are at least  $\frac{n}{10}$  nodes satisfying  $\text{esc}^{(i)} \geq C_T$ .
current escaping phase on the  $i$ -th worker node, which is also the indicator of current phase. When it runs the descent phase on the  $i$ -th worker node  $esc^{(i)}$  is set to  $-1$ ; otherwise  $esc^{(i)} \geq 0$ .
In the first iteration, the gradient estimator is computed based on a large batch size with  $b_{0}$ . Beginning from the second iteration, the gradient estimator  $v_{t}^{(i)}$  is calculated by small mini-batch of samples according to the update rule of STORM, which can be formulated by line 6 in Algorithm 1 where  $\beta$  is a hyperparameter of STORM algorithm. Notation  $\nabla F_{i}(x_{t}^{(i)},\xi_{t}^{(i)})$  represents the stochastic gradient obtained from a batch of samples  $\xi_{t}^{(i)}$ , which can be written as  $\nabla F_{i}(x_{t}^{(i)},\xi_{t}^{(i)}) = (1 / |\xi_{t}^{(i)}|)\sum_{j\in \xi_{t}^{(i)}}F_{i}(x_{t}^{(i)},j)$ .
After calculating  $v_{t}^{(i)}$ , each worker node communicates with its neighbors and update the gradient tracker  $y_{t}^{(i)}$ . Inspired by the framework of Perturbed Gradient Descent, our PEDESTAL method also consists of two phases, the descent phase and the escaping phase. If worker node  $i$  is in the descent phase and the norm  $\| y_{t}^{(i)}\|$  is smaller than the given threshold  $C_v$ , then it will draw a perturbation  $\xi$  uniformly from  $B_0(r)$  and update  $z_{t}^{(i)} = x_{t}^{(i)} + \xi$ . The phase is switched to escaping phase and  $esc^{(i)}$  is set to 0. Anchor  $\tilde{x}^{(i)} = x_{t}^{(i)}$  is saved and will be used to discriminate whether the escaping phase is broken. After this iteration counter  $esc^{(i)}$  will be added by 1 in each following iteration until the moving distance from the anchor on  $i$ -th worker node (i.e.,  $\| x_{t}^{(i)} - \tilde{x}^{(i)}\|$ ) is larger than threshold  $C_d$  for some  $t$ , which breaks the escaping phase and turn back to descent phase. If the condition of drawing perturbation is not satisfied,  $z_{t}^{(i)}$  is updated by  $z_{t}^{(i)} = x_{t}^{(i)} - \eta y_{t}^{(i)}$  no matter which phase is running currently.
If the  $i$ -th worker node's counter  $esc^{(i)}$  is larger than the threshold  $C_T$ , it indicates that  $\bar{x}_{t - C_T}$  is a candidate second-order stationary point. When at least  $\frac{n}{10}$  nodes satisfy the condition  $esc^{(i)} \geq C_T$ , the algorithm is terminated. Notice that the fraction is set to  $\frac{1}{10}$  for convenience in the convergence analysis. Our algorithm also works for other constant fractions. From Algorithm 1 we can see the decision of changing phases on each node only depends on its own status, which is adaptive and independent. Coordinating protocol including broadcast or aggregation is not required.
3.2 Discussion
Here we will discuss the insight of the algorithm design and compare the differences between our method and related works. Some novel improvements are the key to the questions in Section 1.
3.2.1 Perturbed Gradient Descent or Negative Curvature Descent
Perturbed gradient descent and negative curvature descent are two of the most widely used pure first-order methods to find second-order stationary points. In PEDESTAL algorithm, we adopt the strategy of perturbed gradient descent rather than negative curvature descent because of the following reasons. First, negative curvature descent methods such as Neon (Xu et al. [2018]) and Neon2 (Allen-Zhu and Li [2018]) involves a nested loop to execute the negative curvature subroutine to recognize if a first-order stationary point is a local minimum. However, in decentralized setting, it is possible that the gradient norms on some nodes are smaller than the threshold while others are not. Therefore, some nodes will execute the negative curvature subroutine but its neighbors may not. In this case neighbor nodes need to wait for the nodes running negative curvature subroutines and there will be idle time on neighbor nodes. Besides, the analysis of negative curvature descent methods rely on the precision of the negative curvature direction. It is unknown if the theoretical results are still effective when only a fraction of nodes participate in the computation of negative curvature direction while the others use the gradient. In contrast, perturbed gradient descent only requires a simple operation of drawing perturbation, which is more suitable for decentralized algorithms.
3.2.2 Stepsize and Batchsize