A Stochastic Newton Algorithm for Distributed Convex Optimization

Abstract: We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same population objective, as well as stochastic Hessian-vector products (products of an independent unbiased estimator of the Hessian of the population objective with arbitrary vectors), with many such stochastic computations performed between rounds of communication.  We show that our method can reduce the number, and frequency, of required communication rounds, compared to existing methods without hurting performance, by proving convergence guarantees for quasi-self-concordant objectives (e.g., logistic regression), alongside empirical evidence.


Reviews and author responses:

 I would like to thank the authors for the detailed response. Overall, I find the natural scaling setting improves the readability and interpretability greatly. It would be better if the authors can provide more examples to help simplify the rate and make it clear about how the rates compare with each other. 

The authors have made it clear that the proposed methods are near-optimal in terms of some parameters like $K$, while suboptimal (and worse than other algorithms) in other parameters like $R$. This weakens the contribution since the algorithm is only good when the communication is limited.

Overall, based on the contribution and the writing, I believe it would be better to reject the paper for now and wait for a better version that incorporates all the issues in the comments with improvements in writing and contribution. 

The paper proposes a distributed stochastic Newton’s method which uses Local SGD as a quadratic solver to approximate the Newton steps at each iteration, leveraging recent results on the strong performance of Local SGD when optimizing quadratic objectives. An important feature of the algorithm is that it has access to two oracles: a noisy gradient, and a noisy Hessian-vector product. The authors provide convergence guarantees for the algorithm, as a function of the number of machines M, number of communication rounds R, and number of oracle queries per machine per round, denoted K. The results suggest that the second oracle can in fact lead to better convergence guarantees compared fiirst-order methods which have access only to the noisy gradients.  Thank you to the authors for the response. It was helpful, particularly in comparing quasi-self-concordance to standard self-concordance, as well as motivating the approximate Hessian-vector product. May I suggest that the authors also discuss in the paper how using quasi-self-concordance compares to using scaled self-concordance, which is considered in DISCO (Zhang and Xiao, 2015), since it seems that both of those can be used for logistic regression. Sounds like the authors agree that the paper needs some work in making the presentation of the results clearer and more precise, so my recommendation does not change.

-------------------------------
I found the paper to be extremely dense and difficult to get through, with very complicated notation to the point where it was difficult for me to put the results into perspective. Below are the main issues I ran into with the notation:
- First, it feels very strange that the main iterates of the algorithm are indexed by T, and yet T does not appear in the bound of the main result (Theorem 4).
- In fact, parameter T appears both in Algorithm 1 (the outer loop) and Algorithm 3 (the inner-most loop), which makes me think one of those parameters should be called something else.
- Parameters R and K, which seem to be the main quantities used to evaluate the convergence rate, do not appear anywhere in any of the algorithms, except in Appendix D, where they appear in the definitions of other parameters, like T.
- The definitions of all the parameters in Theorem 7 (the full version of the main result) seem to be entangled together. For instance, T is defined using N and N is defined using T. It is not clear to me how all those equations can be satisfied at once. 

Other major question that is not properly addressed in the paper is: How does the noisy Hessian-vector oracle arise practically, as a natural data access model? I didn’t see this addressed in the paper, and it is not clear to me how this was done in the experiments.

Finally, another definition that deserves clarification: What is the relationship between quasi-self-concordance and standard concordance? Is it a broader class, or a narrower class, or incomparable?

Overall, I find the paper very confusing and  poorly written, thus it is difficult for me to even judge the merit of the contributions. 
 Yes.

 **on writing:** Thanks for confirming the writing issues from your side, since I really struggled to properly understand the paper.

**on limitations:** To me, the sentence “FedSN has an additional binary search procedure necessary for finding the correct regularization parameter, which makes it practically less appealing than FedSN-Lite.” provided in your reply describes the limitations better then lines 263-265, which is more appropriate to "future work" section.

**on distributed setup:** Agree, distributed setup does *not always* imply full heterogeneous data distribution. But, lines 102-104 somehow define the type of distributed setup as heterogeneous case (“distributing the component functions of the finite-sum problem across machines”). Moreover, *it is not mentioned anywhere that you actually consider homogeneous setup*. This is another aspect of the writing that makes the paper confusing.

**on plots:** Indeed, I misinterpreted the plots. Thanks for the clarification.


**main result:** I did not find the author(s)' response to this part of my review.

> The main result of FedSN seems to be that in some regime it hits the lower bound obtained for first-order methods. The regime where this happens is highly unrealistic and impractical: all constants $H,B,\sigma, \alpha$ should be 1, the number of machines must be infinitely big, the number of local steps $K$ must be sufficiently large and noise rho must be sufficiently small (as $\rho < 1/\log K$ must hold).

--------------------

***I remain in my initial evaluation and strongly believe that the paper is not publishable in its current form due to the quality of the writing and the strength of the contributions.***

 > The quality and clarity of the writing is extremely low. In its current version, most part of the paper is not readable due to typos, grammar errors, odd word choices, sentence structure and lack of proper organization.

We are aware that some aspects of our writing were confusing to the reader. We have taken note of these issues, and will systematically address them in the renewed version. This includes changing how our algorithms are presented, for a better exposition of our technique, discussion of a *natural scaling* regime where the theoretical analysis can be better put into perspective w.r.t. the previous results, and a more detailed discussion about our assumptions and how we access the data.  

>The steps of the Algorithms are not explained properly, notations are introduced randomly without descriptions. For example, what are Q,C,λ¯,r∗(λ),ϵ(λ) in Theorem 2, are they defined before or inside of Theorem 2 ? Where the steps (and also notations) of Algorithm 2 are explained ?

We will update the manuscript to clarify our notational conventions.

> Where the limitations of your work are described ? It is mentioned “Yes” in the checklist. I failed to find it in the appendix too.

We believe the main limitation of our work is that it does not strictly dominate the first-order lower bound or FedAc. We discuss the regimes where it does beat these baselines in lines 210-230. We will be sure to elaborate on these points as limitations of our work. Another limitation, which we have in fact mentioned in our paper (lines 263-265), is the absence of a lower bound:
“Furthermore, just as it is known in the case of stochastic gradient oracle access for general convex objectives (Woodworth et al., 2021), a natural and complementary direction would be to understand the limitations of these oracles for distributed optimization. "

> The title, abstract and the intro motivate the topic by discussing distributed optimization. By distributed optimization it is typically assumed that data distributed among several machines (this is also mentioned in lines 102-104). However, the main contribution of this work (line 52, 82), FedSN method, is not analyzed under distributed setup since the key component of it, One-Shot-SGD (Algorithm 3), assumes the whole data is available to all machines (see 2nd of Algorithm 3). This is homogeneous data setup or alternatively dataset is shared across machines.

We would draw the reviewer’s attention to the fact that distributed optimization does not necessarily imply the data heterogeneous setting. For instance, the cross-silo federated learning as defined in Kairouz et. al. (2019), is in fact characterized by almost homogeneous data-distributions across different clients. For instance, if multiple hospitals pool their chest X-rays images for an image recognition task, then under usual conditions the images are likely to come from the same distribution. Thus, understanding the homogeneous setting is certainly useful, beyond pedantic reasons. Moreover the homogeneous setting has been extensively studied by other works including (but not limited to) Stich (2019) [1], Dieuleveult and Patel (2019) [2], Stich and Karimireddy (2019) [3], Khaled et. al. (2020) [4], Woodworth et al. (2020) [5], Yuan and Ma (2020) [6], Woodworth et. al. (2021) [7]. It serves as an important step towards understanding the heterogeneous regime as well. We will make sure to discuss this distinction in our updated version. 

>Why in the plots methods have so different sub-optimality at 5th round ? Are they initialized at the same point ?

We believe the reviewer has misunderstood the plots. Specifically, for each choice of the number of rounds of communication (which are denoted along the x axis), the y axis is the best achieved suboptimality for each algorithm during a fixed total number of steps T on each machine (T = 100 for Figure 1, and T = 1000 for Figure 2). The plots may then be interpreted as showing, for a fixed T, how each algorithm's *best* performance changes when increasing the number of rounds of communication.

>Why FedSN is not compared in the plots ? What are the practical limitations of it compared to FedSN-Lite ?

FedSN has an additional binary search procedure necessary for finding the correct regularization parameter, which makes it practically less appealing than FedSN-Lite. We will further clarify this distinction.

**References**

[1] Stich, Sebastian U. "Local SGD converges fast and communicates little." arXiv preprint arXiv:1805.09767 (2018).

[2] Dieuleveut, Aymeric, and Kumar Kshitij Patel. "Communication trade-offs for Local-SGD with large step size." Advances in Neural Information Processing Systems 32 (2019): 13601-13612.

[3] Stich, Sebastian U., and Sai Praneeth Karimireddy. "The error-feedback framework: Better rates for SGD with delayed gradients and compressed communication." arXiv preprint arXiv:1909.05350 (2019).

[4] Khaled, Ahmed, Konstantin Mishchenko, and Peter Richtárik. "Tighter theory for local SGD on identical and heterogeneous data." International Conference on Artificial Intelligence and Statistics. PMLR, 2020.

[5] Woodworth, Blake, et al. "Is local SGD better than minibatch SGD?." International Conference on Machine Learning. PMLR, 2020.

[6] Yuan, Honglin, and Tengyu Ma. "Federated accelerated stochastic gradient descent." arXiv preprint arXiv:2006.08950 (2020).

[7] Woodworth, Blake, et al. "The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication." arXiv preprint arXiv:2102.01583 (2021).


 We thank the reviewer for their comments, and address them in detail below:

> The paper is mainly well-written. However, the notations are confusing and wrong. The parameter T (number of rounds for local SGD) in algorithm 3 and Theorem 3 should be K in Theorem 4 and all the comparisons in this paper. It is really confusing to mix with the number of newton updates. For some key parameters like R,M,K and T, it would be much better if the authors can restate/emphasize their meaning before main theorems/algorithm comparisons

We have made note of the confusing usage around $T,K$, and will correct it in the revised version. We are furthermore aware that some other aspects of our writing were confusing. We have taken note of these issues, and will systematically address them in the renewed version. This includes changing how our algorithms are presented, for a better exposition of our technique, discussion of a *natural scaling* regime where the theoretical analysis can be better put into perspective w.r.t. the previous results, and a more detailed discussion about our assumptions and data-access.

>The authors should emphasize their contributions, on the theoretical analysis and the novelty of their algorithms.

We will do this. Specifically, we plan to compare our results with the existing results including the lower bound in a *natural scaling* regime. The *natural scaling* of the parameters that arises when the objective has the form $F(x) = E_{z}l(\langle x,z \rangle)$, where $\vert\vert l’\vert\vert$, $\vert\vert l''\vert\vert$, and $\vert\vert l'''\vert\vert$ are $\mathcal{O}(1)$. This holds, as an example, for logistic regression. In this case, upper bounds on the derivatives of $F$ will generally scale with the norm of the data, i.e., $\vert\vert z\vert\vert$. So if we assume that $\vert\vert z\vert\vert \leq D$ for some $D$, then the derivatives of $F$ would generally scale as $\vert\vert\nabla F(x)\vert\vert \lesssim D$,  $\vert\vert\nabla^2 F(x)\vert\vert_{op} \lesssim D^2$, and $\vert\vert\nabla^3 F(x)\vert\vert_{op} \lesssim D^3$, where $\vert\vert\cdot\vert\vert_{op}$ denotes the operator norm. It follows that, in this setting, we may take $H=D^2$, $\sigma = D$, $Q = D^3$, $\alpha = D$, and $\rho = D^2$. Though these parameters could of course have different relationships, we will provide an overview of this particular regime to help clarify the comparison.

As far as the novelty of our algorithms is concerned, we are not aware of any distributed Newton algorithm which uses one-shot SGD averaging as a subroutine. What is important is that the computational cost of our oracle is comparable to the stochastic gradient oracle, which most first-order algorithms use. This makes our algorithm appealing for federated learning.  
 

> Using quadratic programming to solve Newton updates is a very standard technique in recent 10 years optimization literature. From my perspective, the algorithm is just a combination of this technique and recently proposed local SGD in Woodworth et al. (2020). The theoretical analysis is also very straight-forward.

We agree with the reviewer in that solving QPs to get the Newton update is a standard technique. The novelty here is indeed in observing why such an approach makes sense in a distributed setting, even though it does not in a pure stochastic sequential setting. With this key insight, we do think that working out the details and evaluating the idea is worthwhile and significant.

>The Hessian-vector product is very important in their algorithm. I am very superised that the authors do not give any explicit form/algorithm to compute this product. They treat it as a black box with some outputs satisfying unbiasedness and bounded variance assumption. I think we need an explicit form for that (from some literature) which satisfies the assumptions proposed.

The Hessian-vector product can be efficiently computed for differentiable functions using two rounds of back-propagation (in the context of neural networks), see: Pearlmutter (1994) [1]. Thus, our oracle is easy to implement in practice. As a more specific example we can look at losses in the family of the *generalized linear models* of the form, $$F(x) = \frac{1}{N}\sum_{i=1}^{N}\phi(b_ix^\top a_i).$$ We have by a simple calculation that
\begin{align*}
\nabla_x F(x) = \frac{1}{N}\sum_{i=1}^{N}\phi'(b_ix^\top a_i)b_i\cdot a_i,\quad \textrm{and}\quad \nabla_x^2 F(x) := \frac{1}{N}\nabla_x(\nabla_x F(x)) = \sum_{i=1}^{N}\phi''(b_ix^\top a_i)b_i^2\cdot a_ia_i^\top.
\end{align*}
So for any $v \in \mathbb{R}^d$, 
\begin{equation*}
    \nabla_x^2 F(x)v = \frac{1}{N}\sum_{i=1}^{N}\phi''(b_ix^\top a_i)b_i^2\cdot a_i\cdot (a_i^\top v).
\end{equation*} 
Thus each term in the sum above can be calculated in $\mathcal{O}(d)$ time. When maximizing the log-likelihood in the logistic regression model with labels in {$-1,1$}, we need to minimize the following function:
\begin{equation*}
F(x) = \frac{1}{N}\sum_{i=1}^{N}\left(-b_ix^\top a_i + \ln\left(1+\exp\left(b_ix^\top a_i\right)\right)\right),
\end{equation*}
which is an instance of the generalized linear models as considered above. 

We will include this discussion in the renewed draft of the paper, and we will also discuss how the assumptions on the oracle translate to assumptions about the data.

**References**

[1] Pearlmutter, Barak A. "Fast exact multiplication by the Hessian." Neural computation 6.1 (1994): 147-160.

 We thank the reviewer for their comments. We would first address the comment about our notation. We have taken note of the issue around using $T$ in our guarantees and algorithms, where it would perhaps make more sense to use $R,K$, and we will make changes in a revised draft of the paper to alleviate this confusion. 

Besides that, we are aware that some aspects of our writing were confusing, and we will systematically address these issues in the renewed version. This includes changing how our algorithms are presented, for a better exposition of our technique, a discussion of a *natural scaling* regime where the theoretical analysis can be better put into perspective w.r.t. the previous results, and a more detailed discussion about our assumptions and data-access.

> The definitions of all the parameters in Theorem 7 (the full version of the main result) seem to be entangled together. For instance, T is defined using N and N is defined using T. It is not clear to me how all those equations can be satisfied at once.

This was certainly an issue in our original draft. We will make these separately defined for each parameter in the new version.

>Other major question that is not properly addressed in the paper is: How does the noisy Hessian-vector oracle arise practically, as a natural data access model? I didn’t see this addressed in the paper, and it is not clear to me how this was done in the experiments.

We will address this issue in the renewed version of the paper, through a running example of *generalized linear models*. To elaborate on this, we may consider the family of *generalized linear models* of the form $$F(x) = \frac{1}{N}\sum_{i=1}^{N}\phi(b_ix^\top a_i).$$ We have by a simple calculation that
\begin{align*}
\nabla_x F(x) = \frac{1}{N}\sum_{i=1}^{N}\phi'(b_ix^\top a_i)b_i\cdot a_i,\quad \textrm{and}\quad \nabla_x^2 F(x) := \frac{1}{N}\nabla_x(\nabla_x F(x)) = \sum_{i=1}^{N}\phi''(b_ix^\top a_i)b_i^2\cdot a_ia_i^\top.
\end{align*}
So for any $v \in \mathbb{R}^d$, 
\begin{equation*}
    \nabla_x^2 F(x)v = \frac{1}{N}\sum_{i=1}^{N}\phi''(b_ix^\top a_i)b_i^2\cdot a_i\cdot (a_i^\top v).
\end{equation*} 
Thus each term in the sum above can be calculated in $\mathcal{O}(d)$ time. Since we are computing only a single term in the above sum, which is chosen at random, we obtain an unbiased estimate of the true Hessian-vector product. This gives rise to the noisy Hessian-vector product oracle. This can be generalized to the case when we instead have a distribution $\mathcal{D}$ on our data instead of a finite sample. When maximizing the log-likelihood in the logistic regression model with labels in {$-1,1$}, we need to minimize the following function:
\begin{equation*}
F(x) = \frac{1}{N}\sum_{i=1}^{N}\left(-b_ix^\top a_i + \ln\left(1+\exp\left(b_ix^\top a_i\right)\right)\right),
\end{equation*}
which is an instance of the generalized linear models as considered above. 

Moreover the Hessian-vector products can be more generally computed for differentiable functions using two rounds of back-propagations (in the context of neural networks), e.g. Pearlmutter (1994) [1]. Thus, this oracle model is fairly general and almost as computationally cheap as the stochastic gradient oracle. We’d make sure to include this discussion in our revised draft. 

>Finally, another definition that deserves clarification: What is the relationship between quasi-self-concordance and standard concordance? Is it a broader class, or a narrower class, or incomparable?

We will add this clarification to the revised version of the manuscript. In general neither of these assumptions implies the other. Quasi-self-concordance recently came to light with the analysis of logistic regression in work by Bach (2010) [2]. Notably, for logistic regression, i.e., problems of the form
\begin{equation}
    \min_x F(x) = \frac{1}{N}\sum_{i=1}^{N}\left(-b_ix^\top a_i + \ln\left(1+\exp\left(b_ix^\top a_i\right)\right)\right),
\end{equation}
we observe that $\alpha$-quasi-self-concordance holds with $\alpha \leq \max_{i}${$||b_i a_i||$}. However, this function is *not* self-concordant. On the other hand the function $F(x)=-\ln(x)$ is $1$-self-concordant but not quasi-self-concordant.

**References**

[1] Pearlmutter, Barak A. "Fast exact multiplication by the Hessian." Neural computation 6.1 (1994): 147-160.

[2] Bach, Francis. "Self-concordant analysis for logistic regression." Electronic Journal of Statistics 4 (2010): 384-414.

 We’d like to thank the reviewer for their comments. We address each of their concerns below:

>The difference and connections between the three assumptions A1, A2 and A3 are not discussed sufficiently. It is unclear how the terms ρ, σ, Q among others are related, which makes it hard to determine which algorithm has a better rate.

We acknowledge the difficulty in comparing FedSN to previous first-order methods is determining the "typical" relative scale of these parameters. Therefore, we will include further discussion of a *natural scaling* of the parameters that arises when the objective has the form $F(x) = E_{z}l(\langle x,z \rangle)$, where $\vert\vert l’\vert\vert$, $\vert\vert l''\vert\vert$, and $\vert\vert l'''\vert\vert$ are $\mathcal{O}(1)$. This holds e.g. for logistic regression. In this case, upper bounds on the derivatives of $F$ will generally scale with the norm of the data, i.e., $\vert\vert z\vert\vert$. So if we assume that $\vert\vert z\vert\vert \leq D$ for some $D$, then the derivatives of $F$ would generally scale as $\vert\vert\nabla F(x)\vert\vert \lesssim D$,  $\vert\vert\nabla^2 F(x)\vert\vert_{op} \lesssim D^2$, and $\vert\vert\nabla^3 F(x)\vert\vert_{op} \lesssim D^3$, where $\vert\vert\cdot\vert\vert_{op}$ denotes the operator norm. It follows that, in this setting, we may take $H=D^2$, $\sigma = D$, $Q = D^3$, $\alpha = D$, and $\rho = D^2$. Though these parameters could of course have different relationships, we will provide an overview of this particular regime to help clarify the comparison.

>The rate for new algorithm has some counter-intuitive dependencies. For example, for all other first-order methods the algorithm will converge with all parameters fixed and the rounds of communication going to infinity. However, the upper bound in this paper suggests that the algorithm will not converge in this case. This is also inconsistent with the experiments in Figure 1 and 2, where we seem to have convergent result when the number of communication rounds gets larger.

We do agree with the reviewer that for fixed $T=KR$ in the regime with high $R$, and small $K$ our upper bound does not converge to zero. However, we underline that the primary focus of this work is the low $R$ regime, as communication cost is often the bottleneck in federated learning. Having said that, our rates do not contradict our experiments, because we only provide upper bounds for the convergence rate. We will make sure to remark about this limitation in the paper.  

>From the rate given in the paper, I think the contribution is not significant enough compared to the previous literature.

We would emphasize that in the *natural scaling* setting mentioned above we do in relevant regimes beat the upper bound of FedAc which was the best known first-order distributed method under higher-order smoothness assumptions. FedAc is an accelerated variant of the Local SGD method, which, as we recall, achieves a guarantee of
\begin{equation*}
    \mathbb{E} F(\hat{x}) - F^* \leq \tilde{O}\left(\frac{HB^2}{KR^2} + \frac{\sigma B}{\sqrt{MKR}} + \frac{H^{1/3}\sigma^{2/3}B^{4/3}}{M^{1/3}K^{1/3}R} + \frac{Q^{1/3}\sigma^{2/3}B^{5/3}}{K^{1/3}R^{4/3}}\right).
\end{equation*}

For the *natural setting* as outlined above, this bound reduces to
\begin{equation*}
\mathbb{E} F(\hat{x}) - F^* \lesssim \frac{D^2B^2}{KR^2} + \frac{D^{5/3}B^{5/3}}{K^{1/3}R^{4/3}}\ .
\end{equation*}
In the case where $DB$ is not too large ($DB \lesssim K^2R^2$), the dominant term for FedAc is $\frac{D^{5/3}B^{5/3}}{K^{1/3}R^{4/3}}$, and so we see that our algorithm improves upon FedAc as long as $R\lesssim \frac{\sqrt{K}}{DB}$, i.e., for sparse communication. In our experiments we also beat FedAc. This underlines an important contribution of our paper. 



>Since the algorithm uses an extra information of stochastic Hessian-vector products compared to other pure first-order methods, one shall expect that the performance of the algorithm strictly dominates that of other first-order methods, which is not the case in the proposed algorithm.
 
In the same *natural scaling* setting mentioned above, our algorithm matches the first-order lower bound by Woodworth et. al. (2021) [2] in the important sparse communication regime. Specifically, no first-order intermittent communication algorithm can guarantee suboptimality less than (ignoring constant and $\log M$ factors)
\begin{equation*}
   E F(\hat{x}) - F^* \geq \frac{HB^2}{K^2R^2} + \frac{\sigma B}{\sqrt{MKR}} + \min 
 \left(\frac{HB^2}{R^2},\frac{\alpha\sigma B^2}{\sqrt{K}R^2},\frac{\sigma B}{\sqrt{KR}}\right)\ .
\end{equation*}
In the same parameter regime as above, the lower bound reduces to
\begin{equation*}
E F(\hat{x}) - F^* \gtrsim \frac{D^2B^2}{K^2R^2} + \min\left(\frac{D^2B^2}{\sqrt{K}R^2},\frac{DB}{\sqrt{KR}}\right)\ .
\end{equation*}
Comparing this lower bound with our guarantee, we see that, when $DB = O(1)$ and the number of rounds of communication is small (e.g., $R = \mathcal{O}(\log K)$), our approximate Newton method can (ignoring $\log K$ factors) achieve an upper bound of $E F(\hat{x}) - F^* \lesssim 1/\sqrt{K}$, in which case we match the lower bound above (up to constants and logarithmic factors). We further note that we are not currently aware of any first-order methods whose guarantees match this lower bound.




>It would be great if the authors can include some comparison with second-order methods

Note that computing the stochastic Hessian-vector product information for logistic regression requires $\mathcal{O}(d)$ computation, similar to gradient based methods, and this computational advantage extends to other relevant settings (see: Pearlmutter (1994) [1]). It is therefore not immediately clear which second-order methods that compute the full Hessian would provide an appropriate comparison to our Hessian-vector product based method.

> along with some discussions on how the lower bound looks like with the extra information of stochastic Hessian-vector product, instead of comparing the algorithm directly with the lower bound for first-order methods.

We agree with the reviewer that it would be better to compare to lower bounds which are adapted to the extra information of stochastic Hessian-vector products, and as we note in the paper, obtaining such a lower bound is an interesting open direction. We would also point out that, even for first-order methods, only recently have lower bounds been shown in this setting, by Woodworth. et. al. (2021) [2], under the quasi-self-concordance assumption.

**References**

[1] Pearlmutter, Barak A. "Fast exact multiplication by the Hessian." Neural computation 6.1 (1994): 147-160.

[2] Woodworth, Blake, et al. "The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication." arXiv preprint arXiv:2102.01583 (2021).



The paper analyzes a stochastic Newton algorithm for distributed convex optimization. The upper bound is shown to achieve faster convergence rates than other first-order methods. Empirical results show that the proposed algorithm is competitive with related first-order methods. 
  The paper proposes a new stochastic Newton algorithm by expressing the Newton update as the solution to a least-squares problem which one can optimize using stochastic gradients and stochastic Hessian-vector products for the objective. I'm listing all my concerns as below:

1. The difference and connections between the three assumptions A1, A2 and A3 are not discussed sufficiently. It is unclear how the terms $\rho$, $\sigma$, $Q$ among others are related, which makes it hard to determine which algorithm has a better rate. 

2. From the rate given in the paper, I think the contribution is not significant enough compared to the previous literature. The rate for new algorithm has some counter-intuitive dependencies. For example, for all other first-order methods the algorithm will converge with all parameters fixed and the rounds of communication going to infinity. However, the upper bound in this paper suggests that the algorithm will not converge in this case. This is also inconsistent with the experiments in Figure 1 and 2, where we seem to have convergent result when the number of communication rounds gets larger. 

3. Since the algorithm uses an extra information of stochastic Hessian-vector products compared to other pure first-order methods, one shall expect that the performance of the algorithm strictly dominates that of other first-order methods, which is not the case in the proposed algorithm. I'm also interested in how this method can approach the rate for second-order methods. It would be great if the authors can include some comparison with second-order methods, along with some discussions on how the lower bound looks like with the extra information of stochastic Hessian-vector product, instead of comparing the algorithm directly with the lower bound for first-order methods.
 The authors have adequately addressed the limitations and potential negative societal impact of their work.

This paper proposed a distributed stochastic newton algorithm for convex optimization. Each Newton update is solved using a recently developed distributed least-squares algorithm with only one round of communication. The authors analyze the convergence rate of their algorithm which outperforms the convergence rates of previous work.  This paper forms the Newton updates in quasi-self-concordant convex optimization as a quadratic programming problem. It then uses recently proposed local SGD algorithm to solve this quadratic programming in a distributed setting with single round of communication. Here are some comments: 

1. The paper is mainly well-written. However, the notations are confusing and wrong. The parameter T (number of rounds for local SGD) in algorithm 3 and Theorem 3 should be K in Theorem 4 and all the comparisons in this paper. It is really confusing to mix with the number of newton updates. For some key parameters like R,M,K and T, it would be much better if the authors can restate/emphasize their meaning before main theorems/algorithm comparisons. 

2. The authors should emphasize their contributions, on the theoretical analysis and the novelty of their algorithms. Using quadratic programming to solve Newton updates is a very standard technique in recent 10 years optimization literature. From my perspective, the algorithm is just a combination of this technique and recently proposed local SGD in Woodworth et al. (2020). The theoretical analysis is also very straight-forward. The Hessian-vector product is very important in their algorithm. I am very surprised that the authors do not give any explicit form/algorithm to compute this product. They treat it as a black box with some outputs satisfying unbiasedness and bounded variance assumption. I think we need an explicit form for that (from some literature) which satisfies the assumptions proposed.

3. The Hessian-vector product is very important in their algorithm. I am very superised that the authors do not give any explicit form/algorithm to compute this product. They treat it as a black box with some outputs satisfying unbiasedness and bounded variance assumption. I think we need an explicit form for that (from some literature) which satisfies the assumptions proposed.

References mentioned above:

1. Woodworth, Blake, Kumar Kshitij Patel, Sebastian Stich, Zhen Dai, Brian Bullins, Brendan Mcmahan, Ohad Shamir, and Nathan Srebro. "Is local SGD better than minibatch SGD?." In *International Conference on Machine Learning*, pp. 10334-10343. PMLR, 2020. None. 

As far as the writing of the paper allows to summarize, this work considers parallel stochastic Newton's method for machines holding IID or shared data to train. The main idea is to replace costly computation of Newton's direction by more efficient stochastic estimates using stochastic gradient and stochastic Hessian-vector products. The paper designs a new method, called FedSN, which is claimed to outperform baseline first-order methods.  The quality and clarity of the writing is extremely low. In its current version, most part of the paper is not readable due to typos, grammar errors, odd word choices, sentence structure and lack of proper organization.

(line 69) A basic driver of progress common among…
(line 86) ..all the while dealing with technical challenges..
(lines 104-105) Leveraging this structure, in combination with distributed second-order methods, has since been shown to lead to improved communication efficiency.
(lines 159-164) ..a key difficulty comes from the fact that our Newton-type updates can only ensure small error in expectation. .. . It is details such as these in the subproblem solver ..
(Line 175) though we allow for an additive error in the subproblem solves in addition to the multiplicative error

a.k.a. means also-known-as, which appears in the caption of Algorithm 1 (FedSN). Given that FedSN is the proposed method of this work, mentioning a.k.a. makes confusion. Is FedSN a new method or known ?

In the abstract “At the heart of our approach is the recent work showing that …". That recent work should be cited there since it seems important to be mentioned in the second sentence of the abstract.

citep and citet macros are used randomly.

The steps of the Algorithms are not explained properly, notations are introduced randomly without descriptions. For example, what are $Q, C, \bar{\lambda}, r^*(\lambda), \epsilon(\lambda)$ in Theorem 2, are they defined before or inside of Theorem 2 ? Where the steps (and also notations) of Algorithm 2 are explained ?

Where the limitations of your work are described ? It is mentioned “Yes” in the checklist. I failed to find it in the appendix too.

Lines 90-91: “we provide a more practical version of our method, FedSN-Lite”. Does it mean FedSN is less practical ? What are the practical limitations of it compared to FedSN-Lite ?

The title, abstract and the intro motivate the topic by discussing distributed optimization. By distributed optimization it is typically assumed that data distributed among several machines (this is also mentioned in lines 102-104). However, the main contribution of this work (line 52, 82), FedSN method, is not analyzed under distributed setup since the key component of it, One-Shot-SGD (Algorithm 3), assumes the whole data is available to all machines (see 2nd of Algorithm 3). This is homogeneous data setup or alternatively dataset is shared across machines.

The main result of FedSN seems to be that in *some* regime it hits the lower bound obtained for first-order methods. The regime where this happens is highly unrealistic and impractical: all constants $H, B, \sigma, \alpha$ should be $1$, the number of machines must be infinitely big, the number of local steps $K$ must be sufficiently large and noise rho must be sufficiently small (as $\rho < 1/\log K$ must hold).

Why in the plots methods have so different sub-optimality at 5th round ? Are they initialized at the same point ?

Why FedSN is not compared in the plots ? I failed to find where the limitations of this work are described (main paper and the appendix) ? It is mentioned “Yes” in the checklist. 

Meta-review: This paper analyzes a stochastic Newton's method for distributed convex optimization. I share many concerns mentioned by the reviewers, including but not limited to the unusual dependence on parameters in the rate, the poor performance of the algorithm compared to first-order methods with access to more information, and inconsistency between theory and experiments. I agree that the paper is not in publishable form and I regretfully cannot recommend acceptance. 

Decision: accept