Title: Finding Local Minima Efficiently in Decentralized Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: second-order optimality

decentralized optimization
Abstract: In this paper we study the second-order optimality of decentralized stochastic algorithm that escapes saddle point efficiently for nonconvex optimization problems. We propose a new pure gradient-based decentralized stochastic algorithm PEDESTAL with a novel convergence analysis framework to address the technical challenges unique to the decentralized stochastic setting. Our method is the first decentralized stochastic algorithm to achieve second-order optimality with non-asymptotic analysis. We provide theoretical guarantees with the gradient complexity of $\tilde{O} (\epsilon^{-3})$ to find $O(\epsilon, \sqrt{\epsilon})$-second-order stationary point, which matches state-of-the-art results of centralized counterparts or decentralized methods to find first-order stationary point. We also conduct two decentralized tasks in our experiments, a matrix sensing task with synthetic data and a matrix factorization task with a real-world dataset to validate the performance of our method.

Finding Local Minima Efficiently in Decentralized Optimization

Abstract

In this paper we study the second-order optimality of decentralized stochastic algorithm that escapes saddle point efficiently for nonconvex optimization problems. We propose a new pure gradient-based decentralized stochastic algorithm PEDESTAL with a novel convergence analysis framework to address the technical challenges unique to the decentralized stochastic setting. Our method is the first decentralized stochastic algorithm to achieve second-order optimality with non-asymptotic analysis. We provide theoretical guarantees with the gradient complexity of ˜ O ( ϵ -3 ) to find O ( ϵ, √ ϵ ) -second-order stationary point, which matches state-of-the-art results of centralized counterparts or decentralized methods to find first-order stationary point. We also conduct two decentralized tasks in our experiments, a matrix sensing task with synthetic data and a matrix factorization task with a real-world dataset to validate the performance of our method.

1 Introduction

Decentralized optimization is a class of distributed optimization that trains models in parallel across multiple worker nodes over a decentralized communication network. Decentralized optimization has recently attracted increased attention in machine learning and emerged as a promising framework to solve large-scale tasks because of its capability to reduce communication costs. In the conventional centralized paradigm, all worker nodes need to communicate with the central node, which results in high communication cost on the central node when the number of nodes is large or the transmission between the center and some remote nodes suffers network latency. Conversely, decentralized optimization avoids these issues since each worker node only communicates with its neighbors.

Although decentralized optimization has shown advantageous performance in many previous works (Lian et al. [2017], Tang et al. [2018]), the study of second-order optimality for decentralized stochastic optimization algorithms is still limited. Escaping saddle point and finding local minima is a core problem in nonconvex optimization since saddle point is a category of first-order stationary point that can be reached by many gradient-based optimizers such as gradient descent but it is not the expected point to minimize the objective function.

Perturbed gradient descent (Jin et al. [2017]) and negative curvature descent (Xu et al. [2018], Allen-Zhu and Li [2018]) are two primary pure gradient-based methods (not involving second-order derivatives) to achieve second-order optimality. Typically, perturbed gradient descent method is composed of a descent phase and an escaping phase. If the norm of gradient is large, the algorithm will run the descent phase as normal. Otherwise it will run the escaping phase to discriminate whether the candidate first-order stationary point is a saddle point or local minimum. Negative curvature

descent method escapes saddle point by computing the direction of negative curvature at the candidate point. If it is categorized as a saddle point then the algorithm will update along the direction of negative curvature. Generally it involves a nested loop to perform the negative curvature subroutine.

Currently, the solution to the second-order optimality of decentralized problem in deterministic setting has been proposed. Perturbed Decentralized Gradient Tracking (PDGT) (Tziotis et al. [2020]) is a decentralized deterministic algorithm adopting the perturbed gradient descent strategy to achieve second-order stationary point. However, it is expensive to compute full gradients for large machine learning models. It is crucial to propose a stochastic algorithm to obtain second-order optimality for decentralized problems. Besides, there are some drawbacks of PDGT to make it less efficient and hard to be generalized to the stochastic setting. These drawbacks are also the key challenges to achieve second-order optimality for decentralized algorithms, which are listed as follows:

(1) PDGT runs fixed numbers of iterations in descent phase and escaping phase such that the phases of all nodes can be changed simultaneously. This strategy works because the descent is easy to be estimated in deterministic setting. Nonetheless, the exact descent of stochastic algorithm over a fixed number of iterations is hard to be bounded because of randomness and noises. If the fixed number is not large enough it is possible that the averaged model parameter is not a first-order stationary point. If the fixed number is as large as the expected number of iterations to achieve first-order stationary point, the algorithm will become less efficient as it is probably stuck at a saddle point for a long time before drawing the perturbation, especially in the second and later descent phase. Specifically, applying fixed number of iterations in each phase results in the complexity of at least ˜ O ( ϵ -4 . 5 ) (see Appendix D), which is higher than ˜ O ( ϵ -3 ) of our method. Therefore, we are motivated to propose an algorithm that can change phases adaptively (based on runtime gradient norm) and independently (not required to consider status on other nodes or notify other nodes).

(2) In PDGT the perturbations on all nodes are drawn from the same random seed. Besides, a coordinating protocol involving broadcast and aggregation is used to compute the averaged model parameter and the descent of overall loss function to discriminate the candidate point. These strategies together with the fixed number of iterations act as a hidden coordinator to make PDGT discriminate saddle point in the same way as centralized algorithms. However, when the number of worker nodes is large it is time-consuming to perform broadcast or aggregation over the whole decentralized network. Moreover, when generalized to stochastic setting the changing of phase is not guaranteed to be synchronized. Additionally, we will note in the Supplementary Material that the consensus error 1 n ∑ n i =1 ∥ x ( i ) t -¯ x t ∥ 2 is another factor to impact the effectiveness of perturbed gradient descent, which is not present in centralized problems. All above issues are theoretical difficulties to study and ensure second-order optimality for decentralized stochastic algorithms.

(Vlaski and Sayed [2020]) proves the theoretical guarantee of second-order optimality for decentralized stochastic algorithm with perturbed gradient descent. However, it does not provide a non-asymptotic analysis to estimate the convergence rate or gradient complexity. The effectiveness of the result relies on a sufficiently small learning rate, and it does not present a specific algorithm. The analysis is based on the assumption that the iteration formula can be approximated by a centralized update scheme when the learning rate is small enough. Nevertheless, in practice it is difficult to maintain an ideally small learning rate, and the iterative update process can be more complex as previously mentioned. To our best knowledge, the second-order optimality issue of decentralized stochastic algorithm with non-asymptotic analysis is still not solved. Therefore, we are motivated to study this important and challenging issue and raise the following questions:

Can we design a decentralized stochastic optimization algorithm with non-asymptotic analysis to find local minima efficiently? Is the algorithm still effective to discriminate saddle point even if each node can change its phase adaptively and independently without any coordinating protocols?

The answer is affirmative. In this paper, we propose a novel gradient-based algorithm named PErturbed DEcentralized STORM ALgorithm (PEDESTAL) which is the first decentralized stochastic algorithm to find second-order stationary point. We adopt perturbed gradient descent to ensure the second-order optimality and use STORM (Cutkosky and Orabona [2019]) estimator to accelerate the convergence. We provide completed convergence analysis to guarantee the second-order optimality theoretically. More details about the reason of choosing perturbed gradient descent and technical difficulties are discussed in Section 3.2. Next we will introduce the problem setup in this paper.

We focus on the following decentralized optimization problem:

where n is the number of worker nodes in the decentralized network and f i is the local loss function on i -th worker node. Here f i is supposed to take the form of stochastic expectation over local data distribution D i , which covers a variety of optimization problems including finite-sum problem and online problem. Data distributions on different nodes are allowed to be heterogeneous. The objective function f is nonconvex such that saddle points probably exist.

The goal of our method is to find O ( ϵ, ϵ H ) -second-order stationary point of problem 1, which is defined by the point x satisfying ∥∇ f ( x ) ∥ ≤ ϵ and min eig ( ∇ 2 f ( x )) ≥ -ϵ H , where eig ( · ) represents the eigenvalues. The classic setting is ϵ H = √ ϵ .

We summarize the contributions of this paper as follows:

We propose a novel algorithm PEDESTAL, which is the first decentralized stochastic gradient-based algorithm to achieve second-order optimality with non-asymptotic analysis.

We provide a new analysis framework to support changing phases adaptively and independently on each node without any coordinating protocols involving broadcast or aggregation. We also address certain technical difficulties unique to decentralized optimization to justify the effectiveness of perturbed gradient descent in discriminating saddle point.

Weprove that our PEDESTAL achieves the gradient complexity of ˜ O ( ϵ -3 + ϵϵ -8 H + ϵ 4 ϵ -11 H ) to find O ( ϵ, ϵ H ) -second-order stationary point. Particularly, PEDESTAL achieves the gradient complexity of ˜ O ( ϵ -3 ) in the classic setting ϵ H = √ ϵ , which matches state-of-the-art results of centralized counterparts or decentralized methods to find first-order stationary point.

2 Related Work

In this section we will introduce the background of related works. The comparison of important features is shown in Table 1. Here ˜ O ( · ) refers to the big O notation that hides the logarithmic terms.

2.1 Decentralized Algorithms for First-Order Optimality

Decentralized optimization is an efficient framework to solve problem 1 collaboratively by multiple worker nodes. In each iteration a worker node only needs to communicate with its neighbors. One of the best-known decentralized stochastic algorithm is D-PSGD (Lian et al. [2017]), which integrates average consensus with local stochastic gradient descent steps and shows competitive result to centralized SGD. The ability to address Non-IID data is a limitation of D-PSGD and some variants of D-PSGD are studied to tackle the data heterogeneity issue, such as D 2 (Tang et al. [2018]) by storing previous status and GT-DSGD (Xin et al. [2021b]) by using gradient tracking (Xu et al. [2015], Lorenzo and Scutari [2016]). D-GET (Sun et al. [2020]) and D-SPIDER-SFO (Pan et al. [2020]) improve the gradient complexity of D-PSGD from O ( ϵ -4 ) to O ( ϵ -3 ) by utilizing variance reduced gradient estimator SPIDER (Fang et al. [2018]). GT-HSGD also achieves gradient complexity of O ( ϵ -3 ) by combining gradient tracking and STORM gradient estimator (Cutkosky and Orabona [2019]). SPIDER requires a large batchsize of O ( ϵ -1 ) on average and a mega batchsize of O ( ϵ -2 ) periodically. In contrast, STORM only requires a large batch in the first iteration. After that the batchsize can be as small as O (1) , which makes STORM more efficient to be implemented in practice.

2.2 Centralized Algorithms for Second-Order Optimality

Perturbed gradient descent is a simple and effective method to escape saddle points and find local minima. PGD (Jin et al. [2017]) is the representative of this family of algorithms, which achieves second-order optimality in deterministic setting. It draws a perturbation when the gradient norm is small. If this point is a saddle point, the loss function value will decrease by a certain threshold within a specified number of iterations ( i.e. , breaking the escaping phase) with high probability. Otherwise, the candidate point is regarded as a second-order stationary point. In stochastic setting, Perturbed SGD perturbs every iteration and suffers a high gradient complexity of O ( ϵ -8 ) to achieve O ( ϵ, √ ϵ ) -second-order stationary point and the gradient complexity hides a polynomial factor of dimension d . CNC-SGD requires a Correlated Negative Curvature assumption and the gradient complexity of ˜ O ( ϵ -5 ) to achieve the classic second-order optimality. SSRGD (Li [2019]) adopts the

same two-phase scheme as PGD but uses the moving distance as the criterion to discriminate saddle point in the escaping phase. It also takes advantage of variance reduction to improve the gradient complexity to ˜ O ( ϵ -3 . 5 ) . Pullback (Chen et al. [2022]) proposes a pullback step to further enhance the gradient complexity to ˜ O ( ϵ -3 ) , which matches the best result of reaching first-order stationary point.

2.3 Stochastic Gradient Descent

A branch of study of stochastic gradient descent argues that SGD can avoid saddle point under certain conditions. However, that is completely different from the problem we focus on. In this paper we propose a method that can find local minima effectively for a general problem 1, while escaping saddle point by stochastic gradient itself depends on some additional assumptions. For example, (Mertikopoulos et al. [2020]) requires the noise of gradient should be uniformly excited. According to our experimental result in Section 5, we can see in some cases stochastic gradient descent cannot escape saddle point effectively or efficiently. Besides, the gradient noise in variance reduced methods is reduced in order to accelerate the convergence. Our experimental results indicate that the gradient noise in variance reduced algorithms is not as good as SGD to serve as the perturbation to avoid saddle point. Therefore, it is necessary to study the second-order stationary point for variance reduced algorithms so as to enable both second-order optimality and fast convergence.

3 Method

3.1 Algorithm

In this section, we will introduce our PEDESTAL algorithm, which is demonstrated in Algorithm 1. Suppose there are n worker nodes in the decentralized communication network connected by a weight matrix W . The initial value of model parameters on all nodes are identical and equal to x 0 . x ( i ) t , v ( i ) t and y ( i ) t are the model parameter, gradient estimator and gradient tracker on the i -th worker node in iteration t . z ( i ) t is the temporary model parameter that is awaiting communication. ¯ x t , ¯ v t and ¯ y t are corresponding mean values over all nodes. Counter esc ( i ) counts the number of iterations in the

Algorithm 1 Perturbed Decentralized STORM Algorithm (PEDESTAL)

current escaping phase on the i -th worker node, which is also the indicator of current phase. When it runs the descent phase on the i -th worker node esc ( i ) is set to -1 ; otherwise esc ( i ) ≥ 0 .

In the first iteration, the gradient estimator is computed based on a large batch size with b 0 . Beginning from the second iteration, the gradient estimator v ( i ) t is calculated by small mini-batch of samples according to the update rule of STORM, which can be formulated by line 6 in Algorithm 1 where β is a hyperparameter of STORM algorithm. Notation ∇ F i ( x ( i ) t , ξ ( i ) t ) represents the stochastic gradient obtained from a batch of samples ξ ( i ) t , which can be written as ∇ F i ( x ( i ) t , ξ ( i ) t ) = (1 / | ξ ( i ) t | ) ∑ j ∈ ξ ( i ) t F i ( x ( i ) t , j ) .

After calculating v ( i ) t , each worker node communicates with its neighbors and update the gradient tracker y ( i ) t . Inspired by the framework of Perturbed Gradient Descent, our PEDESTAL method also consists of two phases, the descent phase and the escaping phase. If worker node i is in the descent phase and the norm ∥ y ( i ) t ∥ is smaller than the given threshold C v , then it will draw a perturbation ξ uniformly from B 0 ( r ) and update z ( i ) t = x ( i ) t + ξ . The phase is switched to escaping phase and esc ( i ) is set to 0 . Anchor ˜ x ( i ) = x ( i ) t is saved and will be used to discriminate whether the escaping phase is broken. After this iteration counter esc ( i ) will be added by 1 in each following iteration until the moving distance from the anchor on i -th worker node ( i.e. , ∥ x ( i ) t -˜ x ( i ) ∥ ) is larger than threshold C d for some t , which breaks the escaping phase and turn back to descent phase. If the condition of drawing perturbation is not satisfied, z ( i ) t is updated by z ( i ) t = x ( i ) t -ηy ( i ) t no matter which phase is running currently.

If the i -th worker node's counter esc ( i ) is larger than the threshold C T , it indicates that ¯ x t -C T is a candidate second-order stationary point. When at least n 10 nodes satisfy the condition esc ( i ) ≥ C T , the algorithm is terminated. Notice that the fraction is set to 1 10 for convenience in the convergence analysis. Our algorithm also works for other constant fractions. From Algorithm 1 we can see the decision of changing phases on each node only depends on its own status, which is adaptive and independent. Coordinating protocol including broadcast or aggregation is not required.

3.2 Discussion

Here we will discuss the insight of the algorithm design and compare the differences between our method and related works. Some novel improvements are the key to the questions in Section 1.

3.2.1 Perturbed Gradient Descent or Negative Curvature Descent

Perturbed gradient descent and negative curvature descent are two of the most widely used pure first-order methods to find second-order stationary points. In PEDESTAL algorithm, we adopt the strategy of perturbed gradient descent rather than negative curvature descent because of the following reasons. First, negative curvature descent methods such as Neon (Xu et al. [2018]) and Neon2 (Allen-Zhu and Li [2018]) involves a nested loop to execute the negative curvature subroutine to recognize if a first-order stationary point is a local minimum. However, in decentralized setting, it is possible that the gradient norms on some nodes are smaller than the threshold while others are not. Therefore, some nodes will execute the negative curvature subroutine but its neighbors may not. In this case neighbor nodes need to wait for the nodes running negative curvature subroutines and there will be idle time on neighbor nodes. Besides, the analysis of negative curvature descent methods rely on the precision of the negative curvature direction. It is unknown if the theoretical results are still effective when only a fraction of nodes participate in the computation of negative curvature direction while the others use the gradient. In contrast, perturbed gradient descent only requires a simple operation of drawing perturbation, which is more suitable for decentralized algorithms.

3.2.2 Stepsize and Batchsize

In Pullback, a dynamic stepsize η t = η/ ∥ v t ∥ in the descent phase where η = O ( ϵ ) and v t is the gradient estimator. This stepsize is originated from SPIDER (Fang et al. [2018]) which ensures its convergence by bounding the update distance ∥ x t +1 -x t ∥ . In the escaping phase, Pullback adopts a larger stepsize of O (1) in the escaping phase and a special pullback stepsize in the last iteration, which is the key to improve the gradient complexity. Different from Pullback, in Algorithm 1 we adopt a consistent stepsize such that it keeps invariant even if phase changes and all nodes always use the same stepsize. If there is no perturbation in iteration t , we have ¯ x t +1 = ¯ x t -η ¯ v t , which is important to the convergence analysis. We discard the strategy in Pullback for two reasons. First, the gradient normalization will probably cause divergence issues in decentralized optimization because in centralized algorithm the gradient direction is v t / ∥ v t ∥ , which is equivalent to v c = ∑ n i =1 v ( i ) t / ∥ ∑ n i =1 v ( i ) t ∥ . However, in decentralized algorithm the average of v ( i ) t is not available on local nodes. If the gradient normalization is done locally, we will get v d = ∑ n i =1 v ( i ) t / ∥ v ( i ) t ∥ , which is different to v c and the error is hard to be estimated. Actually, both D-GET and D-SPIDER-SFO adopt the constant stepsize in SpiderBoost (Wang et al. [2019]) to avoid performing the gradient normalization step. SPIDER needs the gradient normalization because ∥ x t +1 -x t ∥ is required to be small in the proof, while SpiderBoost improves the proof to bound ∥ x t +1 -x t ∥ by η ∥ v t ∥ which is canceled eventually. In our analysis we also adopt the strategy in SpiderBoost. Second, in our algorithm the changing of phase is occurred independently on each node. The phase-wise stepsize and pullback strategy will lead to different stepsizes among all nodes in one iteration, which will also cause potential convergence issues.

In (Chen et al. [2022]), two versions of Pullback are proposed, i.e., Pullback-SPIDER and PullbackSTORM using SPIDER and STORM as the gradient estimator respectively. As introduced previously, one of the advantages of STORM is avoiding large batchsize. Nonetheless, Pullback-STORM adopt a large batchsize of O ( ϵ -1 ) in each iteration, which violates the original intention of STORM. Besides, from Table 1 we can see all algorithms achieving second-order optimality with ˜ O ( ϵ -3 ) gradient complexity require a large batchsize of O ( ϵ -1 ) . Therefore, we propose a small batch version named PEDESTAL-S as a special case of PEDESTAL that only requires an averaged batchsize of O (1) .

3.2.3 Conditions of Termination

As a result of applying gradient tracking, we can bound 1 n ∑ n i =1 ∥ y ( i ) t -¯ y t ∥ 2 by O ( ϵ 2 ) . Even though we have such an estimation, it is still possible that the norm ∥ y ( i ) t ∥ is as large as O ( √ nϵ ) on some nodes when the entire decentralized network has already achieved optimality. Therefore, waiting for all nodes to reach second-order stationary point is not an efficient strategy. This is the reason why we terminate our algorithm when only a fraction of worker nodes satisfy esc ( i ) ≥ C T .

In SSRGD and Pullback, there is an upper bound of iteration numbers in the escaping phase. If the escaping phase is not broken in this number of iterations then the candidate point is regarded as a second-order stationary point. If the escaping phase is broken, then the averaged moving distance is larger than a threshold and the loss function will be reduced by O ( ϵ 2 ) on average. This strategy guarantees that the algorithm will terminate with a certain gradient complexity. However, in our algorithm worker nodes do not enter escaping phase simultaneously and thus we do not set such an upper bound. In this case the averaged moving distance cannot be lower bounded as C T has no upper bound. Fortunately, we can complete our analysis by a different novel framework (see the proof outline in the Appendix). An alternative solution is to stop the update on the node that has run certain number of iterations in the escaping phase while the algorithm will continue. But that solution is also challenging since the relation between the first-achieved local optimal solution and the final global optimal solution is unknown and the analysis is non-trivial.

One remaining issue of the current termination strategy is that it involves the global knowledge of how many worker nodes satisfying the termination condition. One solution is to run an additional process to track this global value. The cost of transmitting Boolean values is much less expensive than broadcasting the model. Another solution is to set a maximum iteration in practice. Generally we need to evaluate the model after certain epochs to see if the training process is running smoothly and we can save a checkpoint when we find a better evaluation result. The theoretical analysis ensures that an optimal solution can be visited if the number of iterations is as large as O ( ϵ -3 ) .

3.2.4 Small Stuck Region

The theoretical guarantee of second-order optimality in SSRGD and Pullback is mainly credit to the lemma of small stuck region, which states that if there are two decoupled sequences x t and x ′ t with identical stochastic samples, x s = x ′ s and x s +1 -x ′ s +1 = r 0 e 1 where e 1 is the eigenvector corresponding to the smallest eigenvalue, then it satisfies max {∥ x t -x s ∥ , ∥ x ′ t -x ′ s ∥} ≥ C d for some s ≤ t ≤ s + C T with high probability. In SSRGD and Pullback, the averaged moving distance 1 t -s ∑ t τ = s +1 ∥ x τ +1 -x τ ∥ 2 is used as the criterion to discriminate saddle point because the small stuck region lemma can be applied in this way. However, in decentralized algorithm some nodes enter the escaping phase before the candidate point ¯ x s is achieved. Suppose node i enters escaping phase in iteration s ′ , then the averaged moving distance starting from iteration s on node i cannot be well estimated because the condition of not breaking escaping phase on node i only guarantees the bound of averaged moving distance starting from s ′ . Therefore, in our method we use the total moving distance ∥ x ( i ) t -x ( i ) s ∥ as the criterion because we can obtain estimation ∥ x ( i ) t -x ( i ) s ∥ ≤ 2 C d given ∥ x ( i ) t -x ( i ) s ′ ∥ ≤ C d and ∥ x ( i ) s -x ( i ) s ′ ∥ ≤ C d . And we can further complete our analysis by the small stuck region lemma. Actually we do not require more memory because ˜ x is the point to return in SSRGD and Pullback (hence should be saved). In practice, we can also return ˜ x ( i ) for any node i drawing perturbation in iteration t -C T since ∥ x ( i ) t -¯ x t ∥ can be well bounded. Besides, we discover that the consensus error 1 n ∑ n i =1 ∥ x ( i ) t -¯ x t ∥ 2 results in an extra term when proving the small stuck region lemma, which becomes another challenge. If the consensus error is not under control, it can drive x away from x s or push x toward x s , no matter what ∇ f ( x ) is. In this manner, the stuck region cannot be estimated. In this work, we provide the corresponding proof to estimate this new term exclusively occurred in decentralized setting in our convergence analysis.

4 Convergence Analysis

4.1 Assumptions

In this section we will provide the main theorem of our convergence analysis. First we will introduce the assumptions used in this paper. All assumptions used in this paper are mild and commonly used in the analysis of related works.

Assumption 1. (Lower Bound) The objective f is lower bounded, i.e., inf x f ( x ) = f ∗ > -∞ .

Assumption 2. (Bounded Variance) The stochastic gradient of each local loss function is an unbiased estimator and has bounded variance, i.e., for any i ∈ { 1 , 2 , · · · , n } we have

Assumption 3. (Lipschitz Gradient) For all ξ and i ∈ { 1 , 2 , · · · , n } , F i ( x, ξ ) has Lipschitz gradient, i.e., for any x 1 and x 2 we have ∥∇ F i ( x 1 , ξ ) -∇ F i ( x 2 , ξ ) ∥ ≤ L ∥ x 1 -x 2 ∥ with a constant L .

Assumption 4. (Lipschitz Hessian) For all ξ and i ∈ { 1 , 2 , · · · , n } , F i ( x, ξ ) has Lipschitz hessian, i.e., for any x 1 and x 2 we have ∥∇ 2 F i ( x 1 , ξ ) -∇ 2 F i ( x 2 , ξ ) ∥ ≤ ρ ∥ x 1 -x 2 ∥ with a constant ρ .

Assumption 1, Assumption 2 and Assumption 3 are common assumptions used in the analysis of stochastic optimization algorithms. Assumption 4 is the standard assumption to find second-order optimality, which is used in all algorithms that achieves second-order stationary point in Table 1.

Assumption 5. (Spectral Gap) The decentralized network is connected by a doubly-stochastic weight matrix W ∈ R n × n satisfying W 1 n = W T 1 n = 1 n and λ := ∥ W -J ∥ ∈ [0 , 1) .

Here J is a n × n matrix with all elements equal to 1 n . W is the weight matrix of the decentralized network where w ij > 0 if node i and node j are connected, otherwise w ij = 0 . ∥ · ∥ denotes the spectral norm of matrix ( i.e. , largest singular value). Notice that λ is a connectivity measurement of the network graph and it is also the second largest singular value of W . We do not assume W to be symmetric and hence the communication network can be both directed graph and undirected graph. The spectral gap assumption is also used commonly in the analysis of decentralized algorithms.

4.2 Main Theorems

Let ϵ H = ϵ α . When α ≤ 0 . 5 , we have the following Theorem 1.

Theorem 1. Assume α ≤ 0 . 5 and Assumption 1 to 5 are satisfied. Let θ = min { 3 -5 α 2 , 1 } . We set η = Θ( ϵ θ L ) , β = Θ( ϵ 1+ θ ) , b 0 = Θ( ϵ -2 ) , b 1 = Θ(max { ϵ 2 -θ -5 α , 1 } ) , r = Θ( ϵ 1+ θ ) , C v = Θ( ϵ ) , C T = ˜ Θ( ϵ -θ -α ) and C d = ˜ Θ( ϵ 1 -α ) . Then our PEDESTAL algorithm will achieve O ( ϵ, ϵ H ) -secondorder stationary point with ˜ O ( ϵ -3 ) gradient complexity.

The specific constants hidden in Θ( · ) will be shown in Appendix B, where the proof outline and the completed proof of Theorem 1 can also be found. From Theorem 1 we can see our PEDESTAL-S with b 1 = O (1) can achieve O ( ϵ, ϵ H ) -second-order stationary point with ˜ O ( ϵ -3 ) gradient complexity for ϵ H ≥ ϵ 0 . 2 . In the classic setting, our PEDESTAL achieves second-order stationary point with ˜ O ( ϵ -3 ) gradient complexity. When α > 0 . 5 , i.e. , ϵ H < √ ϵ , we have the following Theorem 2. Since the parameter settings are different and the O (1) batchsize is only available in Theorem 1, we separate these two theorems. The proof of Theorem 2 can be found in Appendix D.

Theorem 2. When ϵ H < √ ϵ (i.e., α > 0 . 5 ), we set η = ˜ Θ( ϵ θ ) , β = Θ( ϵ 1+ θ ) , b 0 = Θ( ϵ -1 ) , b 1 = ˜ Θ( ϵ -max { 4 α -1 -θ,θ + α } ) , r = Θ( ϵ 1+ θ ) , C v = Θ( ϵ ) , C T = ˜ Θ( ϵ -θ -α ) and C d = ˜ Θ( ϵ α ) where θ = min { 3 α -1 2 , 3 α -2 } . Under Assumption 1 to 5, our PEDESTAL algorithm will achieve O ( ϵ, ϵ H ) -second-order stationary point with ˜ O ( ϵϵ -8 H + ϵ 4 ϵ -11 H ) gradient complexity.

5 Experiments

In this section we will demonstrate our experimental results to validate the performance of our method. We conduct two tasks in our experiment, a matrix sensing task on synthetic dataset and a matrix factorization task on real-world dataset. Both of these two tasks are non-spurious local minimum problems (Ge et al. [2017, 2016]), which means all local minima are global minima. Thus, we conclude an algorithm is stuck at saddle point if the loss function value does not achieve the global minimum. The source code is available in https://github.com/WH-XIAN/PEDESTAL .

5.1 Matrix Sensing

We follow the experimental setup of (Chen et al. [2022]) to solve a decentralized matrix sensing problem. The goal of this task is to recover a low-rank d × d symmetric matrix M ∗ = U ∗ ( U ∗ ) T where U ∗ ∈ R d × r for some small r . We set the number of worker nodes to n = 20 . We generate a synthetic dataset with N sensing matrices { A i } N i =1 and N corresponding observations b i = ⟨ A i , M ∗ ⟩ . Here the inner product ⟨ X,Y ⟩ of two matrices X and Y is defined by the trace tr ( X T Y ) . The decentralized optimization problem can be formulated by

(a)

d

= 50

, ring

(b)

d

= 50

, toroidal

(c)

d

= 50

, exponential

(d)

d

= 100

, ring

(e)

d

= 100

, toroidal

(f)

d

= 100

, exponential

Loss Function Value

D-GET

DPSGD

D-SPIDER-SFO

GTDSGD

PEDESTAL

0.7

D-GET

DPSGD

D-SPIDER-SFO

GTDSGD

PEDESTAL

.7

0.8

DPSGD

D-GET

GTDSGD

D-SPIDER-SFO

PEDESTAL

0.2

0.1

600

Number of Oracles

800100012001400

1800

400

Number of Oracles

10001200

400

600

800100012001400

Number of Oracles

D-GET

D-SPIDER-SFO

GTDSGD

DPSGD

D-GET

DPSGD

D-SPIDER-SFO

GTDSGD

D-GET

DPSGD

GTDSGD

D-SPIDER-SFO

s Function Value

0.4

PEDESTAL

Loss Function Value

PEDESTAL

PEDESTAL

0.3

SS0

200

300

Number of Oracles

500

600

700

800

900

1000

200

300

400

Number of Oracles

500

700

800

900

200

300

Number of Oracles

500

600

700

800

900

000

(a) ring, random

(b) toroidal, random

(c) exponential, random

(d) ring, Dirichlet

(e) toroidal, Dirichlet

(f) exponential, Dirichlet

D-GET

DPSGD

D-GET

DPSGD

D-GET

DPSGD

D-SPIDER-SFO

GTDSGD

GTDSGD

D-SPIDER-SFO

GTDSGD

D-SPIDER-SFO

Loss Function Value

PEDESTAL

Value

PEDESTAL

Loss Function Value

PEDESTAL

40

Number of Oracles

60

80

100

120

140

Number of Oracles

60

08

100

120

Number of Oracles

60

80

100

120

140

D-GET

D-GET

D-GET

DPSGD

DPSGD

DPSGD

D-SPIDER-SFO

D-SPIDER-SFO

D-SPIDER-SFO

GTDSGD

GTDSGD

GTDSGD

PEDESTAL

PEDESTAL

Loss Function Value

PEDESTAL

$

Loss Function

45

35

30

Number of Oracles

60

80

100

120

140

Number of Oracles

100

120

140

Number of Oracles

60

80

100

120

140

where N i is the amount of data assigned to worker node i .

The number of rows of matrix U is set to d = 50 and d = 100 respectively and the number of columns is set to r = 3 . The ground truth low-rank matrix M ∗ equals U ∗ ( U ∗ ) T where each entry of U ∗ is generated by Gaussian distribution N (0 , 1 /d ) independently. We randomly generate N = 20 × n × d samples of sensing matrices { A i } N i =1 , A i ∈ R d × d from standard Gaussian distribution and calculate the corresponding labels b i = ⟨ A i , M ∗ ⟩ . We consider two different types of data distribution, random distribution and Dirichlet distribution Dir 20 (0 . 3) to assign data to each worker node. We conduct experiments on three different types of network topology, i.e. , ring topology, toroidal

topology (2-dimensional ring) and undirected exponential graph. The initial value of U is set to [ u 0 , 0 , 0 ] where u 0 is yield from Gaussian distribution and multiplied by a scalar such that it satisfies ∥ u 0 ∥ ≤ max eig ( M ∗ ) . We compare our PEDESTAL algorithm to decentralized baselines including D-PSGD, GTDSGD, D-GET, D-SPIDER-SFO and GTHSGD. In this experiment, the learning rate is chosen from { 0 . 01 , 0 . 001 , 0 . 0001 } . The batchsize is set to 10 . For PEDESTAL and GTHSGD, parameter β is set to 0 . 01 . For D-GET and D-SPIDER-SFO, the period q is 100 . For PEDESTAL, threshold C v is set to 0 . 0001 . Perturbation radius r is set to 0 . 001 . The threshold of moving distance C d is set to 0 . 01 . The experimental results are shown in Figure 1. Due to the space limit, we only show the result of random data distribution in the main manuscript and leave the result of Dirichlet distribution to Appendix A.

From the experimental result we can see all baselines are stuck at the saddle point and cannot escape it effectively. In contrast, our PEDESTAL reaches and escapes saddle points and finally find the local minimum. We also calculate the smallest eigenvalue of hessian matrix for each algorithm at the converged optimal point, which is left to the Supplementary Material because of space limit. According to the eigenvalue result, we can see the smallest eigenvalue is much closer to 0 than all baselines. Therefore, our experiment verifies that our PEDESTAL achieves the best performance to escape saddle point and find local minimum.

5.2 Matrix Factorization

The second task in our experiment is matrix factorization, which aims to approximate a given matrix M ∈ R N × l by a low-rank matrix that can be decomposed to the product of two matrices U ∈ R N × r and V ∈ R l × r for some small r . The optimization problem can be formulated by

where ∥ · ∥ F denotes the Frobenius norm and subscript ij refers to the element at i -th row and j -th column. In our experiment we solve this problem on the MovieLens-100k dataset (Harper and Konstan [2015]). MovieLens-100k contains 100,000 ratings of 1682 movies provided by 943 users. Each rating is in the interval [0 , 5] and scaled to [0 , 1] in the experiment. This task can be regarded as an association task to predict users' potential ratings for unseen movies. In our experiment we set the number of worker node to n = 50 . Each node is assigned the data from different group of users. Similar to the matrix sensing task, here we also use random distribution and Dirichlet distribution respectively to distribute users to worker nodes. And we also use ring topology, toroidal topology and undirected exponential graph as the communication network. The baselines are also D-PSGD, GTDSGD, D-GET, D-SPIDER-SFO and GTHSGD. In this experiment, the number of worker nodes is 50 and the rank of the matrix M is set to 25 . The learning rate is chosen from { 0 . 01 , 0 . 001 , 0 . 0001 } . The batchsize is set to 100 . For PEDESTAL and GTHSGD, parameter β is set to 0 . 1 . For D-GET and D-SPIDER-SFO, the period q is 100 . For PEDESTAL, threshold C v is set to 0 . 002 . Perturbation radius r is set to 0 . 01 . The threshold of moving distance C d is set to 0 . 5 . The experimental results are shown in Figure 2.

From the experimental results we can see our PEDESTAL achieves the best performance to escape saddle point and find second-order stationary point. All baselines cannot escape saddle point effectively or efficiently. Particularly, variance reduced methods D-GET and D-SPIDER-SFO shows worse performance than SGD based algorithms D-PSGD and GTDSGD, which indicates that although reducing gradient noise can accelerate convergence, it also weakens the ability to escape saddle point. Therefore, our contribution is important since we make the fast convergence of variance reduction compatible with the capability to avoid saddle point.

6 Conclusion

In this paper we propose a novel algorithm PEDESTAL to find local minima in nonconvex decentralized optimization. PEDESTAL is the first decentralized stochastic algorithm to achieve second-order optimality with non-asymptotic analysis. We improve the drawbacks in previous deterministic counterpart to make phase changed independently on each node and avoid consensus protocols of broadcast or aggregation. We prove that PEDESTAL can achieve O ( ϵ, √ ϵ ) -second-order stationary point with the gradient complexity of ˜ O ( ϵ -3 ) , which matches state-of-the-art results of centralized counterpart or decentralized method to find first-order stationary point. We also conduct the matrix sensing and matrix factorization tasks in our experiments to validate the performance of PEDESTAL.

References

Zeyuan Allen-Zhu and Yuanzhi Li. Neon2: Finding local minima via first-order oracles. Advances in Neural Information Processing Systems , 31, 2018.

Zixiang Chen, Dongruo Zhou, and Quanquan Gu. Faster perturbed stochastic gradient methods for finding local minima. In Sanjoy Dasgupta and Nika Haghtalab, editors, Proceedings of The 33rd International Conference on Algorithmic Learning Theory , volume 167 of Proceedings of Machine Learning Research , pages 176-204. PMLR, 29 Mar-01 Apr 2022.

Ashok Cutkosky and Francesco Orabona. Momentum-based variance reduction in non-convex sgd. Neural Information Processing Systems (NeurIPS) , 2019.

Hadi Daneshmand, Jonas Kohler, Aurelien Lucchi, and Thomas Hofmann. Escaping saddles with stochastic gradients. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 1155-1164, 10-15 Jul 2018.

Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Rong Ge, Furong Huang, Chi Jin, and Yang Yuan. Escaping from saddle points - online stochastic gradient for tensor decomposition. In Peter Grünwald, Elad Hazan, and Satyen Kale, editors, Proceedings of The 28th Conference on Learning Theory , volume 40 of Proceedings of Machine Learning Research , pages 797-842, Paris, France, 03-06 Jul 2015. PMLR.

Rong Ge, Jason D Lee, and Tengyu Ma. Matrix completion has no spurious local minimum. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016.

Rong Ge, Chi Jin, and Yi Zheng. No spurious local minima in nonconvex low rank problems: A unified geometric analysis. In International Conference on Machine Learning , pages 1233-1242. PMLR, 2017.

F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) , 5(4):1-19, 2015.

Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M Kakade, and Michael I Jordan. How to escape saddle points efficiently. In International Conference on Machine Learning , pages 1724-1732. PMLR, 2017.

Zhize Li. Ssrgd: Simple stochastic recursive gradient descent for escaping saddle points. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017.

Paolo Di Lorenzo and Gesualdo Scutari. Next: In-network nonconvex optimization. IEEE Transactions on Signal and Information Processing over Networks , 2(2):120-136, 2016. doi: 10.1109/TSIPN.2016.2524588.

Panayotis Mertikopoulos, Nadav Hallak, Ali Kavis, and Volkan Cevher. On the almost sure convergence of stochastic gradient descent in non-convex problems. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 1117-1128. Curran Associates, Inc., 2020.

Taoxing Pan, Jun Liu, and Jie Wang. D-spider-sfo: A decentralized optimization algorithm with faster convergence rate for nonconvex problems. Proceedings of the AAAI Conference on Artificial Intelligence , 34(02):1619-1626, Apr. 2020. doi: 10.1609/aaai.v34i02.5523.

Haoran Sun, Songtao Lu, and Mingyi Hong. Improving the sample and communication complexity for decentralized non-convex optimization: Joint gradient estimation and tracking. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 9217-9228. PMLR, 13-18 Jul 2020.

Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu. d 2 : Decentralized training over decentralized data. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 4848-4856. PMLR, 10-15 Jul 2018.

Isidoros Tziotis, Constantine Caramanis, and Aryan Mokhtari. Second order optimality in decentralized non-convex optimization via perturbed gradient tracking. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 21162-21173. Curran Associates, Inc., 2020.

Stefan Vlaski and Ali H Sayed. Second-order guarantees in centralized, federated and decentralized nonconvex optimization. arXiv preprint arXiv:2003.14366 , 2020.

Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, and Vahid Tarokh. Spiderboost and momentum: Faster variance reduction algorithms. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

Ran Xin, Usman Khan, and Soummya Kar. A hybrid variance-reduced method for decentralized stochastic non-convex optimization. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 11459-11469. PMLR, 18-24 Jul 2021a.

Ran Xin, Usman A. Khan, and Soummya Kar. An improved convergence analysis for decentralized online stochastic non-convex optimization. IEEE Transactions on Signal Processing , 69:18421858, 2021b. doi: 10.1109/TSP.2021.3062553.

Jinming Xu, Shanying Zhu, Yeng Chai Soh, and Lihua Xie. Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes. In 2015 54th IEEE Conference on Decision and Control (CDC) , pages 2055-2060, 2015. doi: 10.1109/CDC.2015.7402509.

Yi Xu, Rong Jin, and Tianbao Yang. First-order stochastic algorithms for escaping from saddle points in almost linear time. Advances in neural information processing systems , 31, 2018.

A Additional Experimental Results

The experimental results of Dirichlet distribution of the matrix sensing task is shown in Figure 3. The smallest eigenvalue at the converged point for each algorithm is shown in Table 2 and Table 3.

(a)

d

= 50

, ring

(b)

d

= 50

, toroidal

(c)

d

= 50

, exponential

(d)

d

= 100

, ring

(e)

d

= 100

, toroidal

(f)

d

= 100

, exponential

D-GET

D-GET

D-GET

DPSGD

DPSGD

DPSGD

D-SPIDER-SFO

0.8

D-SPIDER-SFO

0.8

D-SPIDER-SFO

GTDSGD

GTDSGD

GTDSGD

Loss Function Value

Loss Function Value

0.7

0.7

PEDESTAL

0.6

PEDESTAL

PEDESTAL

0.3

0.3

0.3

0.2

400

600

Number of Oracles

10001200

1400

1600

200

600

Number of Oracles

800

10001200

1600

400

600

800100012001400

Number of Oracles

D-GET

D-GET

D-GET

DPSGD

DPSGD

DPSGD

D-SPIDER-SFO

D-SPIDER-SFO

D-SPIDER-SFO

GTDSGD

GTDSGD

GTDSGD

Loss Function Value

0.4

PEDESTAL

Loss Function Value

0.4

PEDESTAL

PEDESTAL

0.3

0.2

0.2

0.2

100

200

300

400

Number of Oracles

700

800

900

100

200

300

400

Number of Oracles

500

700

800

900

1000

200

300

400500600

Number of Oracles

700

800

900

1000

B Proof of Theorem 1

B.1 Notation

We define matrix X t = [ x (1) t , · · · , x ( n ) t ] ∈ R d × n where x ( i ) t is the model parameter on i -th worker node with dimension d and n is the number of worker nodes. Similarly we have Y t = [ y (1) t , · · · , y ( n ) t ] ,

Z t = [ z (1) t , · · · , z ( n ) t ] and V t = [ v (1) t , · · · , v ( n ) t ] . Let ω t = ∥ ¯ x t +1 -¯ x t ∥ 2 and Ω t = Z t -X t . Define p t = n t /n where n t is the number of worker nodes drawing perturbation in iteration t .

B.2 Outline

In this section we will provide the proof outline of Theorem 1. First, we prove some basic lemmas to estimate gradient noise and consensus error, which will be used frequently in later proof. The gradient noise is estimated by Lemma 1, the proof of which can be found in Section C.1. The consensus error is estimated by Lemma 2, the proof of which can be found in Section C.2.

Lemma 1. (Gradient Noise) Under Assumption 2 and Assumption 3 we have

Lemma 2. (Consensus Error) Let η ≤ (1 -λ ) 2 ϵ θ 600 log(4 /δ ) λ 2 L , β = C -1 1 ϵ 1+ θ and b 1 ≥ C 1 ϵ -1+ θ where C 1 ≥ 1 is a constant. Under Assumption 2, 3 and 5 we have

Next we will prove that PEDESTAL will terminate in certain number of iterations. Under Assumption 2, 3 and 5, we can prove the following Lemma 3. The proof is demonstrated in Section C.3.

Lemma 3. (Descent) Let η ≤ (1 -λ ) 2 ϵ θ 600 log(4 /δ ) λ 2 L , β = C -1 1 ϵ 1+ θ , b 1 ≥ C 1 ϵ -1+ θ and b 0 = C 1 ϵ -1 where C 1 ≥ 1 is a constant. Under Assumption 2, 3 and 5 we have

where

Here we call D t the descent of iteration t . We categorize all iterations into three types:

When at least n 5 nodes drawing perturbation in iteration t , then it is type-A. There are two cases where p t is small: most nodes in the descent phase or most nodes in the escaping phase. An iteration is type-B if p t < 1 5 and 1 n ∑ n i =1 ∥ y ( i ) t ∥ 2 ≥ 4 C 2 v 5 , which represents the case where most nodes are in the descent phase. And type-C iteration represents the case where most nodes are in the escaping phase. Next we will estimate type-A and type-C iteration with the following Lemma 4.

Lemma 4. Let η ≤ (1 -λ ) 2 ϵ θ 600 log(4 /δ ) λ 2 L , β = C -1 1 ϵ 1+ θ , b 1 ≥ C 1 ϵ -1+ θ , b 0 = C 1 ϵ -1 , C d = C 2 ηC T ϵ , C v = (1 -λ ) C 2 ϵ 200 and r ≤ ηC v / 4 where C 1 = 20000 σ (1 -λ ) 2 C 2 and C 2 is a constant. Under Assumption 2, 3 and 5, we can find disjoint intervals I = I 1 ∪··· ∪ I k such that the indexes of all type-A and type-C iterations except the last C T iterations are contained in I and the descent over I can be estimated by

where | · | denotes the total number of the set.

Besides, for all type-B iteration t , we have the following estimation

Lemma 5. Let parameter and assumption settings be the same as Lemma 4, then for all type-B iteration t we have

With Lemma 4, Lemma 5 and Assumption 1, we can conclude that PEDESTAL will terminate in ˜ O ( ϵ -2 -θ ) + C T iterations. As the last two negative terms in D t are canceled by 1 n ∑ n i =1 ∥ x ( i ) t +1 -x ( i ) t ∥ 2 and 1 n ∑ n i =1 ∥ y ( i ) t ∥ 2 respectively in Lemma 4 and Lemma 5, we have 1 η ∑ T -1 t =0 ω t ≤ O (1) . Hence by Lemma 2 we know the consensus error 1 n ∥ X t -¯ X t ∥ 2 F can be bounded by O ( ϵ 1+ θ ) on average. Besides, from the parameter setting we can see C v is Θ( ϵ ) , which ensures the first-order optimality of the decentralized algorithm.

Finally, we will prove PEDESTAL is able to achieve second-order stationary point. First, we will give the small stuck region lemma in decentralized setting. Recall that ϵ H = ϵ α is the tolerance of second-order stationary point. The proof is in Section C.6.

Lemma 6. (Small Stuck Region) Suppose n s worker nodes draw perturbation in iteration s and -γ = min eig ( ∇ 2 f (¯ x s )) ≤ -ϵ H . Let η ≤ (1 -λ ) 2 ϵ θ 1000 √ n log( C T ) log(4 /δ ) λ 2 L , β = C -1 1 ϵ 1+ θ , b 1 ≥ 1000 C 1 ϵ 2 -θ -5 α , C d = C 2 ηC T ϵ µ and C T = log(12 nC d /r 0 ) / ( ηγ ) where C 1 = 20000 (1 -λ ) 2 C 2 , C 2 ≤ 1 -λ 2000 log(4 /δ ) ρ log( C d ) and µ = max { 1 , 2 α } . Let X t and X ′ t be two coupled decentralized sequences by running PEDESTAL from X s with X s = X ′ s , x ( i ) s +1 = x ( i ) ′ s +1 if node i does not draw perturbation in iteration s and x ( i ) s +1 = x ( i ) ′ s +1 + r 0 e 1 otherwise. Here e 1 is the eigenvector with respect to the smallest eigenvalue γ . Define d i = max s ≤ t ≤ s + C T {∥ x ( i ) t -x ( i ) s ∥ , ∥ x ( i ) ′ t -x ( i ) s ∥} . Then there are at least 9 n 10 nodes such that d i ≥ 2 C d .

In decentralized small stuck region lemma, the consensus error will lead to a new term (see Eq. (34)) and make the proof more complicated. In our proof, we use the condition of ϵ H ≥ ϵ , i.e., α ≤ 1 . For smaller ϵ H the batchsize b 1 is required to set larger. With Lemma 6, we can prove that when PEDESTAL is terminated, it finds a local minimum with high probability.

Lemma 7. Let r 0 = δr/ d where d is the dimension of model parameter. Other parameters are the same as Lemma 6. Suppose PEDESTAL is terminated in iteration s + C T . Then ¯ x s is a second-order stationary point with probability at least 1 -δ .

√

Lemma 7 provides the guarantee of second-order optimality of PEDESTAL. When ϵ H ≥ √ ϵ , i.e., α ≤ 0 . 5 (including the classic setting ϵ H = √ ϵ ), the parameter settings of all lemmas are consistent and the main theorem is proven. The total gradient complexity is

When α = 0 . 5 , we have θ = 0 . 25 and b 1 = Θ( ϵ -0 . 75 ) . When α ≤ 0 . 2 , we can set θ = 1 and b 1 = O (1) , which is result of PEDESTAL-S. In Section D we will provide the analysis of the case α > 0 . 5 with a different parameter setting of θ and b 1 . We can achieve the gradient complexity of

over all cases of ϵ H .

C Proof of Lemmas

C.1 Proof of Lemma 1

Proof. According to the definition of v ( i ) t , we have

where | ξ ( i ) t +1 | = b 1 . The expectation of the right side of Eq. (6) over ξ ( i ) t +1 is 0 . Using Cauchy-Schwartz inequality, Assumption 2 and Assumption 3 we have

for each j ∈ ξ ( i ) t +1 . Thus, applying Azuma-Hoeffding inequality to Eq. (6) we can obtain

with probability 1 -δ . Here we use the fact that ∑ + ∞ s =0 (1 -β ) s = 1 β . Using Cauchy-Schwartz inequality to Eq. (8) we have

Sum Eq. (9), we obtain

which finishes the proof of (a) in Lemma 1. In the first inequality of Eq. (10) we apply AzumaHoeffding inequality to v ( i ) 0 -∇ f i ( x 0 ) . In the second inequality we apply Cauchy-Schwartz inequality and use the fact x ( i ) t +1 -x ( i ) t = ( x ( i ) t +1 -¯ x t +1 ) -( x ( i ) t -¯ x t ) + (¯ x t +1 -¯ x t ) . Mimic above steps and we can achieve the inequality (b) in Lemma 1. The term n in the denominator is derived by the fact that ξ ( i ) t 's on different nodes are independent.

C.2 Proof of Lemma 2

Proof. As Y t = W ( Y t -1 + V t -V t -1 ) , we have

where the first inequality is derived by Assumption 5, the second inequality is derived by Young's inequality and the last inequality is derived by Cauchy-Schwartz inequality and Assumption 3. When t = 0 , by Azuma-Hoeffding inequality we can get

with probability 1 -δ . As X t +1 = W ( X t +Ω t ) , by Assumption 5 and Young's inequality we have

where the second inequality is obtained by Cauchy-Schwartz inequality and the last inequality is because when node i draws perturbation it must satisfy ∥ y ( i ) t ∥ ≤ C v . Note that X 0 = ¯ X 0 . Sum Eq. (13), we have

where the last inequality comes from Eq. (11). When η ≤ (1 -λ ) 2 40 λ 2 L we have 288 L 2 η 2 λ 4 (1+ λ 2 ) (1 -λ 2 ) 4 ≤ 1 2 and

where the last inequality is achieved by Lemma 1. According to the parameter setting, we have

73728 log(4

/δ

(1

-

)

L

λ

2

2

)

η

4

2

b

λ

1

4

β

(1 +

λ

2

)

1

2

Therefore, we have

where we have used the condition λ ≤ 1 to simplify the inequality. Moreover, sum Eq. (11) and we can achieve

where the second inequality uses Lemma 1 and Eq. (12). The last inequality uses the sum of Eq. (13). As 37152 log(4 /δ ) L 2 η 2 λ 4 (1 -λ ) 3 min { b 1 β, 1 } ≤ 1 2 , we have

which finishes the proof.

C.3 Proof of Lemma 3

Proof. By Assumption 3 we have

where the first inequality is obtained by Young's inequality and the last inequality is obtained by Cauchy-Schwartz inequality, Assumption 3 and the fact that perturbation is only drawn when ∥ y ( i ) t ∥ ≤ C v and n t nodes draw perturbation in iteration t . Sum Eq. (19) and apply Lemma 1, we have

According to the update of gradient tracker, we have ¯ y t = ¯ v t . By Lemma 10 we have

Divide the term ∥ ¯ v ∥ in Eq. (20) into three portions and we get f (¯ x T

t 2 )

In the second inequality we use Eq. (22). In the third inequality we use Eq. (21) and Cauchy-Schwartz inequality. In the last inequality we use Lemma 2 and the coefficients are

According to the parameter setting, we have A 1 ≤ 1 16 Lη , A 2 ≤ 200 log(4 /δ ) (1 -λ ) 2 , A 3 ≤ 40 log(4 /δ ) (1 -λ ) 2 , A 4 ≤ 7 4 and A 5 ≤ 5 1 -λ . Therefore, we have

where

which reaches the conclusion.

C.4 Proof of Lemma 4

Proof. For convenience, the iteration that draws perturbation is considered to be included in the escaping phase. If an iteration belongs to type-A, i.e. , p t ≥ 1 5 , then at least n/ 5 worker nodes are in the escaping phase. If an iteration belongs to type-C, we have 1 n ∑ n i =1 ∥ y ( i ) t ∥ 2 ≤ 4 C 2 v 5 . Therefore, there are at least n 5 worker nodes satisfying ∥ y ( i ) t ∥ ≤ C v , which also indicates that at least n 5 worker nodes are in the escaping phase. Then if iteration t is either type-A or type-C, there must be n/ 5 worker nodes in the escaping phase. We denote the set of these n/ 5 worker nodes as E t . Furthermore, if this iteration t is not one of the last C T iterations before termination, then there must exist n/ 10 worker nodes out of E t such that they have not met the condition esc ( i ) ≥ C T and will break the escaping phase before meeting the condition because of the termination criterion in Algorithm 1. We use B t to denote these worker nodes.

For each i ∈ B t , we have an interval [ a ( i ) t , b ( i ) t ] such that t ∈ [ a ( i ) t , b ( i ) t ] and node i enters escaping phase in iteration a ( i ) t and breaks escaping phase in iteration b ( i ) t . Besides, we also have

Then by Cauchy-Schwartz inequality we have

Let a t = min i { a ( i ) t } and b t = max i { b ( i ) t } . It is easy to check that b t -a t ≤ 2 C T . Next, we will perform the refining step. If t < t ′ are two iterations that are either type-A or type-C and t ′ ∈ [ a t , b t ] , then we make a t ′ = a t and b t ′ = b t . Let I = ∪ t [ a t , b t ] for all type-A and type-C iterations t . Then I can be written as disjoint union of

because if a t ≤ a t ′ ≤ b t then [ a t , b t ] and [ a t ′ , b t ′ ] can be merged into one interval. Now we can see for each iteration t that is either type-A or type-C and t is not one of the last C T iterations, we have t ∈ I . Next we will estimate the descent over I . Without loss of generality, we consider an interval I j . I j can be expressed by union J 1 ∪ · · · ∪ J l where J m = [ a t m , b t m ] for some t m , m = 1 , · · · , l . Because of the refining step, we have each t m is only included in interval J m and the intersection of any three intervals in J 1 , · · · , J l is ∅ . According to Eq. (26) we have

since |B t | ≥ n 10 . Next, we will consider the intersection of J m and J m +1 . Notice that when estimating Eq. (28) we only add the terms ∥ x ( i ) t +1 -x ( i ) t ∥ 2 on nodes i ∈ B t m and in the intervals [ a ( i ) t m , b ( i ) t m ] . Therefore, for any node i / ∈ B t m ∩ B t m +1 , the terms used to estimate Eq. (28) will not be added repeatedly. If i ∈ B t m ∩ B t m +1 , we have [ a ( i ) t m , b ( i ) t m ] and [ a ( i ) t m +1 , b ( i ) t m +1 ] are disjoint because t m +1 ∈ [ a ( i ) t m +1 , b ( i ) t m +1 ] but t m +1 / ∈ [ a ( i ) t m , b ( i ) t m ] and a node cannot draw perturbation before breaking the escaping phase. Hence we can sum Eq. (28) over m and achieve

Since the length of each J m is not larger than 2 C T , we have

Combining Eq. (30) and Lemma 3, we can estimate the descent over I by

according to the parameter setting.

C.5 Proof of Lemma 5

Proof. According to Lemma 3 and the definition of type-B iteration, we have

for all type-B iteration t where we have used the parameter setting.

C.6 Proof of Lemma 6

Proof. Suppose the conclusion is not true and we will find the conflict. Thus, we have the assumption that there are at least n 10 worker nodes satisfying d i ≤ 2 C d . First, we define

and

Then we have

Here term ζ t is yield from consensus error and does not exist in centralized algorithms. Applying recursion to Eq. (33), we can obtain

Let q t = η ∑ t -1 τ = s +1 ( I -η H ) t -τ -1 ( ¯ ∆ τ w τ + ν τ + ζ τ ) . We will prove

which leads to

because ∥ ( I -η H ) t -s -1 w s +1 ∥ = (1+ ηγ ) t -s -1 p s r 0 according to the definition of w s +1 . We define ¯ d = max s ≤ t ≤ s + C T {∥ ¯ x t -¯ x s ∥ , ∥ ¯ x ′ t -¯ x s ∥} . Since at least n 10 nodes satisfy d i ≤ 2 C d , C d = ˜ O ( ϵ 1 -α ) and the averaged consensus error is bounded by O ( ϵ 2(1+ θ ) ) , we have

To achieve Eq. (35), it is sufficient to prove

which can be derived by induction. When t = s + 1 , the left side of Eq. (38) is 0 and thus the inequality is satisfied. Suppose Eq. (38) holds for t ≤ t 0 . When t = t 0 +1 , we have

where we use Assumption 4 and the case of t ≤ t 0 in the first two inequalities. We use the parameter setting of C d in the last inequality. Next, we will estimate the terms related to ν t . By Azuma-Hoeffding inequality we know Eq. (39) is satisfied when t = s +1 . We define

Then according to the definition of ν t we have

Define

Then we have

According to Assumption 3 and Assumption 4, we have

Applying Azuma-Hoeffding inequality to Eq. (42), with Eq. (45) we can obtain

since β is Θ( ϵ 1+ θ ) and C d is Θ( ϵ 1 -α ) . According to Eq. (34), we have

In the first inequality, the first term is derived by the definition of w s +1 . The second term is derived by the supposition that Eq. (38) holds for t ≤ t 0 and the fact that Eq. (38) implies

Combining Eq. (46) and Eq. (47), we have

The exponential term in Eq. (40) can be addressed by the following strategy. When t ≥ ˜ O ( 1 1 -λ ) , the term can be dominated by other terms such as 1 ηC T . When t < ˜ O ( 1 1 -λ ) , it can be bounded by

The term t -s in the numerator will be bounded by η in this case and hence it can be merged to the first term in Eq. (39). In the third inequality of Eq. (49), we split the last term into two parts: τ -s > Lη b 1 γ and τ -s ≤ Lη b 1 γ . Since ∫ + ∞ t dx x 2 = 1 t , we can merge the case τ -s > Lη b 1 γ into the second term and estimate the rest one where τ -s is small. According to the choice of θ , we have b 1 ≥ Θ( ϵ 2 -θ -5 α ) and η 2 C 2 d C 3 T b 1 ≤ O (1) and hence get the estimation in Eq. (49). We should notice that we use the relation η b 1 γ ≤ O (1) in our proof, which is automatically satisfied. By Eq. (49) we can reach the conclusion in Eq. (39). Furthermore, we have

The last step to prove Eq. (38) is to estimate the term corresponding to ζ t , which is a new term only occurred in decentralized algorithms. Recall the definitions in Eq. (43), we have

Then by Assumption 3, Assumption 4, Eq. (37), Lemma 10 and Cauchy-Schwartz inequality, we have

It is sufficient to prove

because of Eq. (53) and the parameter setting. Eq. (54) can also be proven by induction. When t = s +1 the condition is satisfied. Next we will estimate ∥ X t -¯ X t -( X ′ t -¯ X ′ t ) ∥ 2 F . By Assumption 5 and Young's inequality we have

where we apply recursion in the second inequality and use the definition of the decoupled sequences in the last equality. Similarly, by recursion we also have

Combining above two inequalities, we achieve

According to the update rule of v ( i ) t we have

Then mimic the estimation of ν t , we can obtain

Combining above inequalities and the parameter setting of β , we can obtain

Using Eq. (36), Eq. (47) and Eq. (54) we have

where

If t ≥ ˜ O ( 1 1 -λ ) , t 2 ( 1+ λ 2 2 ) t -s -1 is small and the first term of Eq. (61) can be merged to the second term. Otherwise if t < ˜ O ( 1 1 -λ ) , it can be merged to the last term according to the parameter setting of η and b 1 . When ϵ is small, we have (1+ λ 2 )(1+ ηγ ) 2 2 ≤ 3+ λ 2 4 . Hence the second term of Eq. (61) can be bounded by Lemma 8. The third term of Eq. (61) can be estimated by Lemma 9 (the case of t < ˜ O ( 1 1 -λ ) can be addressed by the parameter setting of η and b 1 ). Therefore, we can prove

because of the parameter setting. We should notice that here we also use the relation η b 1 γ ≤ O (1) , which is always satisfied according to the setting of b 1 . Based on Eq. (53) and Eq. (63), it is easy to check that ζ t satisfies Eq. (40). Moreover, we have

where the first inequality is derived by

Now combining Eq. (41), Eq. (51) and Eq. (64), we can reach the conclusion in Eq. (38) and finish the proof of the induction. Recall the assumption at the beginning, we have

since ∥ ¯ x t -¯ x ′ t ∥ ≤ ∥ ¯ x t -¯ x s ∥ + ∥ ¯ x ′ t -¯ x s ∥ . Eq. (66) implies that

which conflicts with the definition of C T . Therefore, the proof of Lemma 6 is finished.

C.7 Proof of Lemma 7

Proof. If node i enters the escaping phase in iteration s ′ before iteration s and does not break it in iteration s + C T , then for s ≤ t ≤ s + C T , we have ∥ x ( i ) t -x ( i ) s ∥ ≤ ∥ x ( i ) t -x ( i ) s ′ ∥ + ∥ x ( i ) s -x ( i ) s ′ ∥ ≤ 2 C d . Therefore, there are at least n 10 worker nodes satisfying max s ≤ t ≤ s + C T ∥ x ( i ) t -x ( i ) s ∥ ≤ 2 C d .

Suppose min eig ( ∇ 2 f (¯ x s )) ≤ -ϵ H and e 1 is the corresponding eigenvector. Let S i denote the region of the perturbation on node i that PEDESTAL will terminate in iteration s + C T , i.e., n 10 workers will not break the escaping phase. Then by Lemma 6 we can conclude that there must exist one worker node such that the projection of S i onto direction e 1 is smaller than r 0 . Since the perturbation ξ i is drawn from uniform distribution, the probability of ξ i ∈ S i can be bounded by

where V ( · ) denotes the volume and Ball ( d, r ) denotes the d -dimensional ball with radius r . The last inequality is achieved by the definition of r 0 . Therefore, we can prove that ¯ x s is a second-order stationary point with probability at least 1 -δ .

D Additional Theoretical Result

In this section we will provide some additional theoretical result of our PEDESTAL algorithm. First we will demonstrate the convergence analysis of the case ϵ H < √ ϵ , i.e., α > 0 . 5 . Next, we will discuss the strategy of using fixed number of iterations in each descent and escaping phase, which motivates the design of PEDESTAL.

D.1 Smaller Tolerance for Second-Order Optimality

When ϵ H < √ ϵ , the conclusions of previous Lemmas are still satisfied except Lemma 4. In this case, C d = C 2 ηC T ϵ µ where µ = 2 α > 1 . Parameter C d should be smaller than the original setting in Lemma 4, which results in more iterations to converge. Fortunately, the analysis of Lemma 4 can be adjusted and we can achieve Theorem 2. The proof is provided as follows.

Proof. The fourth term of D t in Lemma 3 is derived by ηβσ 2 b 1 and at this time we will set b 1 ≥ ϵ -(2 µ -1 -θ ) so that the ϵ term is replaced by ϵ µ . The last term of D t can be written as

where P is the set of all pairs of ( t, i ) such that node i draws perturbation in iteration t . We can divide P into two parts. P 1 contains all pairs of ( t, i ) such that node i breaks the escaping phase within M iterations, where M is an integer to be decided later. The rest part is denoted by P 2 .

For any ( t, i ) ∈ P 1 , suppose node i breaks escaping phase in iteration t + m , where m ≤ M . Then node i will never draw perturbation between iteration t and iteration t + M . Mimic the steps of Eq. (26), by Cauchy-Schwartz inequality we can obtain

Let M = ϵ -2 -2 θ +2 α . Then we have

and

by the parameter setting of C v . On the other hand, if ( t, i ) ∈ P 2 , then node i will not break the escaping phase in M steps and hence the perturbation step will not execute, either. Therefore, we have estimation

With Eq. (72), Eq. (73) and the new setting of b 1 , the descent in Lemma 3 can be improved to

When θ ≥ 3 α -2 , we have ϵ 2 M ≤ ϵ 2 µ and Lemma 4 still holds but the conclusion is changed to

In this case, PEDESTAL algorithm will terminate in ˜ O ( ϵ -θ -2 µ ) iterations. In Lemma 6 and Lemma 7 we need the relations

which implies b 1 ≥ ˜ O ( ϵ -θ -α ) . Therefore, we set b 1 = ˜ Θ( ϵ -max { 4 α -1 -θ,θ + α } ) with the condition θ ≥ 3 α -2 . When α ≤ 1 , we set θ = 3 α -1 2 , which satisfies θ ≥ 3 α -2 and

The gradient complexity in this case is

When α > 1 , we have θ = 3 α -2 and b 1 = ˜ Θ( ϵ -(4 α -2) ) . The gradient complexity is

which finishes the proof of Theorem 2.

Therefore, the gradient complexity over all cases of α can by written by

D.2 Phases with Fixed Number of Iterations

If a decentralized stochastic perturbed gradient descent method adopt the strategy of fixed number of iterations in each phase, the gradient complexity in the descent phase should be at least O ( ϵ -3 ) to ensure the first-order stationary point. But the total descent of a descent phase could be small because it is possible that it is stuck at a saddle point after only a few steps. Hence we need to consider the descent in the escaping phase. According to Lemma 3 and Lemma 4 we can see the descent of an escaping phase is O ( C 2 d ηC T ) . As the conditions ηC d C T ≤ O (1) and C T = ˜ O ( 1 ηϵ H ) are required in Lemma 6, we can obtain that the total descent of an escaping phase is no larger than ˜ O ( ϵ 3 H ) . In the classic setting of ϵ H = √ ϵ , the total descent of an escaping is upper bounded by ˜ O ( ϵ 1 . 5 ) . Consequently, the total gradient complexity to achieve ( ϵ, √ ϵ ) -second-order stationary point is at least ˜ O ( ϵ -4 . 5 ) , which is worse than the result of our PEDESTAL.

E Auxiliary Lemmas

Lemma 8. Let 0 < a < 1 . Then we have

Lemma 9. Let 0 < a < 1 . When t ≥ ˜ O ( 1 1 -a ) , we have

Proof. When τ ≤ t 2 , by Lemma 8 we have

When τ > t 2 , we have

Therefore, we can reach the conclusion when t ≥ ˜ O ( 1 1 -a ) .

Lemma 10. (Definition of Variance) For any random variable X, we have

Lemma 11. (Lemma D.1 in (Chen et al. [2022])) Let ϵ 1: k ∈ R d be a vector-valued martingale difference sequence with respect to F k , i.e., for each k ∈ [ K ] , E [ ϵ k |F k ] = 0 and ∥ ϵ k ∥ ≤ B k , then with probability 1 -δ we have

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn \"Finding Local Minima Efficiently in Decentralized Optimization\" the authors propose a perturbed variance reduction stochastic optimization scheme that finds second order stationary points over a mesh network. The authors establish convergence results of the order \\tilde(O)(\\epsilon^{-3})  and verify empirically that their theoretical findings hold. Overall I think this work is nice, I follow with some criticism in the following sections.\n\nSTRENGTHS:\n1. Originality: To the reviewer's knowledge this seems to be the first paper in the (stochastic) decentralized set-up to establish second order guarantees with a given rate. Consequently, in this regard the paper's novelty seems to be sufficient.\n2. Quality: The provided result seems to be technically sound. \n3. Clarity: The presentation of the paper and the use of language can be improved. In many situations, I feel the authors' comments do not provide an intuition on the scheme's dependence on relevant quantities but provide an explanation based on what can be controlled or not in the proof. I provide concrete feedback on this in \"Questions.\"\n4. Significance: I think this work is significant given that it is the first to provide this type of guarantees in the decentralized set-up.\n\nWEAKNESSES:\n1. It should be clear from the main text which polylog factors the notation \\tilde{O} is hiding. For this type of result, I would expect some scaling with the dimension (even if logarithmic) which should be highlighted at  some point. \n2. Experimental set-up: The results in this work hold under the assumptions that the gradients and hessian are Lipschitz. The problem upon which the simulations are run fulfills neither of these assumptions. I understand that the numerical simulations confirm that the scheme converges even if the assumptions are violated, but there is currently no way of understanding wether the order of number of gradient evaluations that it takes to achieve a certain precision is close to tight (given that the simulations work with a more general class of functions than that covered by the theory).\n3. Termination criteria: the scheme returns \\bar{x}_{t-C_T} if there are at least n/10 nodes satisfying esc^i \\geq  C_T. This seems to require global knowledge of the system. In other words, this does not seem implementable. As it is written, it seems to be the scheme does not terminate if this condition does not hold. The authors should clarify this. I see that in Section 3.2.3 the authors motivate the termination criteria via avoidance of the consensus error on the tracking variable (which is also not implementable), but this does not address how to implement the criteria.\n\nQUESTIONS:\nQuestions/Suggestions/Comments:\n1. There are paragraphs in which the language needs to be polished. See for example lines 45 to 48. I \n2. In 2.2. the authors state that Perturbed SGD requires a complexity of O(epsilon^{-8}) to obtain a second order stationary point, and that this further hides a polynomial scaling with the dimension. My understanding is that the complexity is of order O(epsilon^{-4}) with logarithmic dependence with the dimension, see, e.g. Theorem 16 in https://arxiv.org/abs/1902.04811 (or 17 for the mini-batch).\n3.  If my comment in 2 is correct, the way to justify the current work, as opposed to a decentralized extension of perturbed stochastic gradient descent (which would yield a much simpler scheme) is through the benefits of variance reduction. \n4. The termination criteria is problematic. This is covered in detail under Weaknesses.\n5. In my view the main technical contribution of this work should be to extend the \"small stuck region\" lemma to the decentralized case. In this sense, I would expect a similar intuition as that highlighted in https://arxiv.org/abs/1902.04811 with further complications due to the consensus error(s). Instead, the intuition provided in 3.2.4. is entirely based on the quantities that can or can not be controlled in the proof. I think having a geometric interpretation instead would make the paper more understandable.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel decentralized algorithm for escaping saddle points for non-convex problems. The authors focus primarily on the second-order optimality. Specifically, they propose a gradient-based stochastic optimization algorithm PEDESTAL with a novel convergence analysis framework. They provide the theoretical guarantees with the best available gradient complexity to achieve the second-order stationary point. To validate the proposed algorithm, the authors present two empirical results with the synthetic and real-world datasets.\n\nSTRENGTHS:\nI think the investigated topic is very interesting. The second-order optimality of decentralized stochastic optimization algorithm has not sufficiently been investigated. Most of existing works focused only on finding the first-order stationary point. It is exciting to see a fresh work that investigates the second-order optimality. The paper is easy to follow, though there are some typos that need to be corrected.\n\nWEAKNESSES:\nIn order to make the paper more technically solid and sound, the authors may have to pay attention to the following aspects.\n\n1. In the algorithm framework, the authors use $n/10$ to terminate the algorithm. Why? Why not $n/5$ or $n/2$? Any specific criterion to determine such a certain number of nodes? Otherwise, it would be odd to just have this condition.\n\n2. The authors mentioned that broadcasting or aggregation is not required. But what about step 8 or 15? Did I misunderstand something here?\n\n3. The experimental results are not promising. More complex datasets and models are required to validate the proposed algorithm. In the discussion, the authors also said that all baselines were stuck at the saddle point and couldn’t escape it efficiently. How to guarantee the point these baselines were stuck at is a saddle point? Not just a bad local minimum? I think the authors should be careful about their claim. It would be better to see a toy example problem that includes obvious saddle points and local minima. Then the authors show exactly how the proposed algorithm and baselines perform.\n\n4. Some confusion about the table. The authors showed that the PEDESTAL has the larger gradient complexity than GT-HSGD. However, in terms of empirical plots, the PEDESTAL had faster and better convergence rate. Why? Moreover, the classic setting for the PEDESTAL-S is missing.\n\nQUESTIONS:\n1. How to quantitatively decide the termination criterion for the PEDESTAL algorithm?\n2. How does the proposed algorithm perform in a non-IID data distribution setting?\n3. How to distinguish the bad local minima from the saddle points in terms of the results presented in the paper?\n\nLIMITATIONS:\nPlease include the discussion of the limitations for the proposed algorithms in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-01"}

comments: SUMMARY:
This paper proposes PEDESTAL, decentralized optimization algorithm that can find second-order stationary points efficiently, and provides extensive numerical experiments to support theoretical analysis. PEDESTAL outperforms other existing decentralized second-order algorithms as it requires a smaller batch size and less iterations to converge.

STRENGTHS:
1. The proposed algorithm is simple but has a stronger performance guarantee than existing algorithms. The proof requires only standard assumptions.
2. In addition to having superior convergence guarantees, the proposed algorithm outperforms other algorithms in numerical experiments.

WEAKNESSES:
1. The stopping criteria requires global synchronization. Even though this algorithm only requires to send an integer over the network, it is still required to sync every iteration to decided whether to stop.
2. Rates don't include the eigen gap. I suggest the authors to explicitly show this terms, which gives us a better idea of the algorithm's efficiency.
3. Notations are not clearly defined in the paper. I was not able to find the definition of $\tilde O(\cdot)$.
4. Some minor typesetting flaws. Page 7 line 276 - 279, should be $x_1$ instead of $x1$.

QUESTIONS:
I have a question about numerical experiments. Are all

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

comments: SUMMARY:
This paper proposed a gradient-based decentralized stochastic algorithm in decentralized optimization. The second-order optimality is proved with non-asymptotic analysis.

STRENGTHS:
The paper is well-written and easy to follow. The mathematical proof seems rigorous.

WEAKNESSES:
The numerical studies can be improved.

QUESTIONS:
I have some concerns on the numerical studies part. In the tasks studied, the performance proposed method seems to be too strong as compared to other baselines. All methods except the proposed one stuck at the saddle point. Maybe the author can consider some common machine learning jobs like image classification and do some comparsions.

LIMITATIONS:
See questions.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes PEDESTAL, the first decentralized algorithm for the stochastic non-convex optimization framework that escapes saddle points efficiently. Theoretical guarantees are provided indicating that PEDESTAL finds second order stationary points in $\\tilde{\\mathcal{O}} (\\epsilon^{-3} + \\epsilon_H^{-5})$. In the classical setting where $\\epsilon_H = \\sqrt{\\epsilon}$ this translates to $\\tilde{\\mathcal{O}$ which matches the state of the art results for finding a first order stationary point. To achieve this aforementioned result the authors utilize perturbed gradient decent (from the centralized regime) and STORM to accelerate convergence. Importantly, the ideas of perturbed gradient descent (i.e. the injection of noise followed by a phase that verifies whether a second order stationary point is reached) are utilized in a distributed manner thus improving from prior work and requiring no coordination between the nodes. Additionally, numerical results on matrix sensing and matrix factorization tasks are provided supporting the theoretical findings.\n\nSTRENGTHS:\n- The paper is $\\textbf{very well written}$ and easy to follow for the most part (more details on this in the weaknesses section.)\n\n- This paper studies an $\\textbf{interesting problem}$ in the field of non-convex stochastic optimization i.e. finding efficiently second order stationary points in a decentralized manner. \n\n- $\\textbf{Novel techniques}$. The paper combines ideas from prior works (perturbed gradient descent, perturbed gradient tracking, STORM estimator) in a non-trivial manner. The theoretical contribution is significant since the authors utilize ideas of perturbed gradient descent in a decentralized manner without the need for any coordination among the participating nodes. I believe that this is a novel and interesting technique that can be used by more decentralized methods in the future to escape efficiently from saddles.\n\n- $\\textbf{Impactful results}$. The resulting algorithm and analysis are impactful deriving the first decentralized algorithm for the stochastic non-convex optimization framework that escapes saddle points efficiently with convergence guarantee $\\tilde{\\mathcal{O}$; this is indeed a competitive rate matching the best known rates for finding first order stationary points.\n\n- $\\textbf{Numerical results}$ that support the theoretical findings are provided showcasing that PEDESTAL efficiently escapes saddle points.\n\nWEAKNESSES:\n- Although the paper has very nice structure and is well-written, I believe that the presentation of $\\textbf{section $3.2.4$ could be improved}$. Arguably one of the most interesting ideas of this work is the implementation of the descent and escape phases in a fully distributed fashion without the need for any coordination. I believe that conveying the intuition of how this is achieved in PEDESTAL is of crucial importance that needs to be emphasized. Although, the authors have attempted to do that in section $3.2.4$ I think that this part of the text should be revised explaining the underlining ideas more clearly and in more detail. Similarly, very little intuition or discussion has been for the proof of the important Lemma $6$ in the appendix.\n\n- I would like the authors to clarify how the algorithm terminates i.e. how the clients know that $n/10$ of them satisfy the $\\textbf{termination condition}$.\n\n- The bound provided in Theorem $1$ holds for a $\\textbf{specific range of}$   $\\boldsymbol{\\alpha \\leq 0.5}$. Although it is true that $\\alpha =0.5$ covers the most important (classic) setting, this limitation should be clearly stated in the main body of the paper (a footnote could suffice).\n\n- Typos: \nLine 170 \"REDESTAL\" change to \"PEDESTAL\".\nLine 171 change \"two phase\" to \"two phases\".\nLine 204 \"In Pullback...as SPIDER\" needs rephrasing.\nLine 214 \"If the...is used\" needs rephrasing.\n\nQUESTIONS:\nSee the weaknesses section.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}