ErrorCompensatedX: error compensation for variance reduced algorithms

Abstract: Communication cost is one major bottleneck for the scalability for distributed learning. One approach to reduce the communication cost is to compress the gradient during communication. However, directly compressing the gradient decelerates the convergence speed, and the resulting algorithm may diverge for biased compression. Recent work addressed this problem for stochastic gradient descent by adding back the compression error from the previous step. This idea was further extended to one class of variance reduced algorithms, where the variance of the stochastic gradient is reduced by taking a moving average over all history gradients. However, our analysis shows that just adding the previous step's compression error, as done in existing work, does not fully compensate the compression error. So, we propose ErrorCompensateX, which uses the compression error from the previous two steps. We show that ErrorCompensateX can achieve the same asymptotic convergence rate with the training without compression. Moreover, we  provide a unified theoretical analysis framework for this class of variance reduced algorithms, with or without error compensation. 


Reviews and author responses:

The paper proposed ErrorCompensatedX, that uses compression error from the previous two steps for variance reduced algorithms and SGD. The paper provided a theoretical analysis framework to analyze error compensated algorithms, and shows that ErrorCompensatedX admits the same asymptotic convergence rate for SGD, Momentum SGD, STORM, and IGT. Numerical experiments are performed with ResNet50 on CIFAR10.  As stated in the paper most error compensation algorithms use compression error from the previous one step. The paper proposed an error compensation algorithm using compression error from the previous two steps as in equation (3) for variance reduced algorithms. The Algorithm 1 assumes a Parameter-Server communication protocol following Tang et al. and the update rule utilizes the low-pass filter by Chen et al. properly cited.

Through theoretical analysis the paper presents convergence rate for momentum SGD, STORM, IGT, same as the original algorithm. Numerical results are not very sound to me, with only experiments of ResNet50 on CIFAR10. Do ErrorCompensationX outperform Single Compensation with other datasets like ImageNet or other models or other compression method?

The paper is clearly written and well organized.

Communication is a bottleneck for scalability of distributed learning. It is important to properly address it and extend beyond SGD.

**Update**:
Thanks for the authors' response to the reviews. I didn't see full answer to my questions. Hopefully they would be addressed in the revised version.
 The paper addressed the limitations and potential negative societal impact.

The paper presents a modification to the error feedback mechanism for compression of communication in distributed optimization to handle momentum-like terms. The paper presents theoretical analyses of existing momentum-like algorithms coupled with compression based on their modifications and experimental results confirming their results.
  Improvements to compressed distributed optimization are likely to be of interest to the community. The submission identifies a problem, the compression of momentum based methods, that is likely to impact practical implementations of distributed training and seems to not have been adressed in the literature so far. 

However, the current submission suffers from major issues in its treatment of previous work and in the clarity of the assumptions and theoretical results. I cannot recommend acceptance in its current state. My main issues are as follow, with more detailled below;
- The focus and motivation on "variance-reduced methods" seems inappropriate when considering (Polyak or Nesterov) momentum, and there are bibliographic issues throughout the paper.
- The discussion of related work on variance-reduced compressed distributed optimization is lacking, making the comparative statements of the contributions of the paper inappropriate
- The theoretical results do not match the results of the original non-compressed algorithms and rely on strong assumptions that previous work did not require.   
	- For example, STORM requires a growing batch size $\propto T^{1/3}$ (Corollary 6), which defeat the purpose of its variance-reduced gradient estimator.
	- The assumption (2) that the compression error is uniformly bounded by a constant,  $\mathbb{E}_\omega[\Vert \Delta_t - C_\omega[\Delta_t]\Vert^2] \leq \frac{1}{2}\epsilon^2$, is unrealistic. It does not hold for the 1-bit compression considered in this work or  other common compression schemes (top-K, random-K). The typical assumption, used in previous, is that the error is multiplicative ($\mathbb{E}_\omega[\Vert \Delta_t - C_\omega[\Delta_t]\Vert^2] \leq c \Vert\Delta_t\Vert^2$ for some $c < 1$, see e.g. Definition 2.1 in Stich et al. (2018) or Eq. (8) in Gorbunov et al. (2020))

- The algorithms are analyzed as estimators with some specified error in finite time (where the step-size is constant and depends on the number of iterations) rather than optimization algorithms (with a decreasing step-size) where the convergence rate implies that running the algorithm for longer improves the guarantee.

---

### Detailled comments


**Related work and bibliographic issues:**

The related work and emphasis on variance reduction for the motivation of the work is confusing, as it groups methods from very different settings. The inclusion of (Polyak or Nesterov) momentum is especially confusing at those methods are designed for acceleration in the deterministic setting, not variance reduction, which is bound to confuse readers. It seems that, instead of variance reduction, the submission could be motivated in terms of momentum, with variance-reduced methods using momentum as an application.

But given the emphasis on variance reduction, the introduction should take greater care to distinguish between algorithms that are typically thought of as variance reduction (SAGA, SVRG and their more recent variants) and the more recent momentum-based variants of ROOT-SGD and STORM. Variants based on SVRG have already been adapted to the compressed setting, starting from Alistarh et al. (2017), and subsequent work in this area should be acknowledged more directly and compared against in the discussion of the proposed method and its results. 

The motivation for the proposed method could be made stronger by discussing the benefits of the STORM/ROOT-SGD over SAG or SVRG-type methods. Given that the setting is distributed optimization, where each machine only holds a subset of the data, periodic computation of the full-gradient on each machine does not seem to be a major issue.

Additionally, the claimed contribution that "Our algorithm admits a faster convergence rate compared to previous methods (Zheng et al., 2019a; Stich et al., 2018b) by fully compensating all error history." (l36-38, p2) needs to be qualified to a specific setting and the required assumptions, and include a discussion of existing variance-reduced methods.

I would suggest that for future submissions the authors proofread the bibliography and make sure that the citations in text point to the correct paper, as I found the following mistakes
- The submission cites Gorbunov et al. (2020, "Linearly Converging Error Compensated SGD") as an example of accelerated algorithm (l69, p2), but this paper does not use acceleration. There are works on acceleratiion in this setting, for example that of Qian et al. (2020) and Li et al. (2020).
- I assume that the citation for STORM, Cutkosky and Mehta (2020), "Momentum Improves Normalized SGD", was meant to be Cutkosky and Orabona (2019)?
- The citation for Momentum SGD, Zhang et al. (2015), "Deep learning with elastic averaging SGD", EAMSGD (Elastic Avg. Momentum SGD) does not discuss variance-reduction as a benefit and is instead based on Nesterov acceleration, although the description of Momentum SGD in the current submission (e.g. in Table 1) is that of Heavy Ball/Polyak momentum.
- There are also minor issues with some bibliography entries that 
	- are duplicated (Stich et al. 2018a and 2018b, Zheng et al. 2019a and 2019b)
	- do not contain publication venue (Arnold et al. 2019, Gorbunov et al. 2020, Horváth et al. 2019, Mischenko et al. 2019, Vogels et al. 2020, Yuan et al. 2020, Zhang et al.  2015, Zhao et al. 2019)
	- cite the arXiv version instead of the published work (Wu et al. 2018 at ICML)
	
**Clarity of the contributions:**

At a high level, the presentation for the "double" compensation in Section 3.2 is unclear. A presentation for the first (few) iterations, rather than the entire run ($t=0,\ldots,T$) and highlighting the failure of "single" compensation might flow better. But If I understood the method correctly, the main issue it seems to address is as follow: for methods based on a moving average momentum-term (rather than variance reduction), compressing and applying error feedback on the gradient is insufficient as it does not correct the momentum term appropriately. Given this, it is unclear to me why the method is cast as a as "double compensation" depending on the previous two timesteps, as in Eq. (3), rather than simply applying error feedback on the momentum term. For example,  

> $v_t = (1-\alpha) v_{t-1} + \alpha A(x_t)$  
> sending $C_{\omega_t}(v_t + e_t)$  
> $e_{t+1} = v_t - C_{\omega_t}(v_t + e_t)$  

If this more naïve approach does not work, a comparison and discussion of why it fails and why the "double compensation" is necessary would be a valuable addition to understand the contribution of the paper. 

**Theory**

My major issues with the theory are the strength of the assumptions (Assumption 2), which were not needed in previous work, and the resulting dependence of the hyperparameters on the problem-specific constants which do not match the uncompressed algorithms. For all results, a comparison to the original algorithms would be helpful.

For example, Theorem 5 and Corollary 6 show a dependence between the batch-size $B_0$ and the total number of iterations $T$. Such a dependence can make any algorithms "variance-reduced", and does not match the existing theory for STORM. 

Similarly, the dependence on a (constant) step-size and the total number of iteration did not appear in the original analysis, which instead used decreasing step-sizes. As the submission points out, having the parameters depend (on some power of) $1/T$ is too small and should be decreased over time instead.


**Minor comments:**

- Theorem 5: requires $\gamma \leq \frac{1}{4L} \text{ and } \alpha \geq 8(L^2+L_F^2)\gamma^2/n$. Assuming for simplicity that $\gamma$ is maximal ($\gamma = \frac{1}{4L}$) and $n=1$, $\alpha \geq \frac{1}{2}(1 + \frac{L_F^2}{L^2})$. As $L$ is the smoothness constant of $f_i$ and $L_F$ the smoothness constant of the individual components $F$ of $f_i$, $L_F \geq L$ and $\alpha \geq 1$. This runs counter to the assumption that $\alpha$ is a convex combination parameter $\in (0,1)$ (l124, p5).
	Corollary 6 also does not follow from Theorem 5 as the specified $\alpha = 8L^2\gamma^2/n$ falls outside of the range allowed for Thm 5. My guess is that the step-size should instead depend on the smoothness constant of the stochastic functions ($\propto 1/L_F$ instead of $\propto 1/L$)?
- Please be careful when discussing the dependence of the step-size on the current iteration $t$ or the total number of iterations $T$. For example, Figure 1 mentions that $\alpha_t = 1/t$ but the text before it is "However, we find that when $\alpha_t$ is very small ($O(1/T)$ for ROOT-SGD and $O(1/T^{2/3})$ for STORM, where $T$ is the total training iterations)" (l110-111, p4)
	Is it $1/T$ or $1/t$? Using $\alpha$ instead of $\alpha_t$ when $\alpha_t$ is kept constant could help avoid confusion.
- Assumption 1, bounded variance: the current assumption reads

	$\mathbb{E}_{\zeta^{(i)} \sim D_i}[\Vert \nabla F(x, \zeta^{(i)}) - \nabla f(x) \Vert^2] \leq \sigma^2, \forall x,i$

	I assume it should instead read as ($f_i$ instead of $f$)
	
	$\mathbb{E}_{\zeta^{(i)} \sim D_i}[\Vert \nabla F(x, \zeta^{(i)}) - \nabla f_i(x) \Vert^2] \leq \sigma^2, \forall x,i$

	If not, then the assumption that $\mathbb{E}_{\zeta^{(i)} \sim D_i}[F(x, \zeta^{(i)})] = f(x)$ should be explicitly stated as it is very strong (and counter to the description of Section 3, "we do not assume that all nodes can access the same data set"). Which one is actually  used in the proofs is unclear, as t
- Assumption 1 writes that "$f(\cdot)$ and $F(\cdot,\zeta)$ are assumed to be $L$-smooth" but $f$ is $L$-smooth while $F$ is $L_F$ smooth.
- Theorem 1 depends on the additional assumption that the step-size $\gamma$ is smaller than $1/L$ (l429, p18) and needs to be added to the theorem statement.
- Denoting $C_\omega$; "Here $\omega$ denotes the randomness of the compression operator" (Footnote 2, p3)  
	As I understood it, omega is the random seed used in the compression operator such, in the equation between l99 and l100, the same compression is applied to the term added to $x_t$ and $\delta_t$. Adding a subscript, $\omega_t$, or using an intermediate variable $q = C_\omega(...)$ to update $x_t$ and $\delta_t$ would make that clearer. The current notation, for example in Algorithm 1, implies that all instances the compression operator $C_\omega$ use the same random seed and is deterministic.
- Adding references to table 1 would help
- The "total batch-size $B_0$" variable in algorithm 1 seems unused.
- typo: l89 STROM (STORM)


---

**References:**
- Xun Qian, Peter Richtárik, Tong Zhang: "Error Compensated Distributed SGD can be Accelerated".   
	OPT2020: 12th Annual Workshop on Optimization for Machine Learning
- Zhize Li, Dmitry Kovalev, Xun Qian, Peter Richtárik: "Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization" ICML 2020.
- Ashok Cutkosky, Francesco Orabona: "Momentum-Based Variance Reduction in Non-Convex SGD", NeurIPS 2019



---

**Update**

I thank the autors for their response. Some of my concerns have been adressed and I increased my score. For a revision, I would the authors focus on the following points:
- The potential applications of the method are not well discussed. The paper considers stochastic, non-finite sum problems in a distributed setting where communication is the bottleneck, but does not provide real examples that are not adressed by previous work but fall in the scope of the proposed method.
- The motivation of the key contribution, the proposed modification to compression using compression buffers, needs better exposition. It is unclear, for example, how it would compare to a naïve solution of compressing a momentum-like buffer, and why the two buffers are necessary. Similarly, due to strength of Assumption 2 and the use of an increasing batch size, the comparison of the proposed method to previous work is muddled. While those may be standard techniques to simplify the proof and/or achieve better convergence rates, their inclusion makes it difficult for the reader to understand the key contribution of the submission.
 yes

This paper proposes ErrorCompensatedX, a method of adding low-pass filters to error compensation method on a type of variance reduced algorithms. A framework of proving convergence is given on some example algorithms (e.g. Momentum SGD, STORM, IGT) and experiment of Resnet-50 on CIFAR10 is conducted for comparing the convergence of algorithms.  Strength
==
Error compensation is an important technique for communication compressed model training. In practice, it is true that sometimes (especially at the beginning of training), an ill-conditioned compensation scheme would make the training unstable, especially when we are using cheap compressors such as biased 1-bit quantizer. Having said this, the motivation and the topic of this paper is good and important to the community.

Weakness
==
To begin with, it's not clear what challenge Algorithm 1 is trying to solve, or what settings will Algo 1 be applicable to. Specifically, ErrorCompensatedX requires at least three additional buffers for communication, i.e. $\delta_{t-1},\delta_{t}, e_{t-1}$, which are of model size. This very much limits the deployment of the algorithm. The reason we need communication compression is model/gradient being high-dimensional -- does it really worth using 3X additional memory just for low-precision communication? Consider the case of BERT or GPT-2, these additional buffers would take ~GBytes memory and would affect scaling the mini-batch on a modern GPU. If GBytes of memory is needed for all the compressors, why can't we use more bits (more than 1-bit) instead? 

Aside from applicability, how ErrorCompensatedX contributes in terms of theory is also unclear. In the theoretical analysis, a general method of decomposing a convergence rate into uncompressed and compressed parts is given. This is somewhat confusing as it seems to be the standard method to prove the convergence of any compressed optimization algorithm, please correct me if I'm wrong. Besides, obtaining the same $O(1/\sqrt{nT})$ rate compared to the uncompressed algorithms is not surprising, as the sample complexity is always the leading term. To the best of my knowledge, I haven't seen any compression-based algorithm breaking this term. The main design seems to be the application of low-pass filters in several algorithms, but how is it different from (Chen et al., 2020; Wu et al., 2018)?

Some technical issues:

i) Why ErrorCompensatedX can only be used on the algorithms specified in Table 1? If the main idea is to compensate the entire history, why can't other optimizers like Adam use it?

ii) The assumption on the compressor (i.e. Assumption 2) seems strong, will the algorithm still converge under the following more general assumption:

$\mathbb{E}||\mathcal{C}_\omega(x)-x||^2 \leq \omega||x||^2 + \epsilon^2$,

like the analysis in (https://arxiv.org/pdf/1907.09356.pdf)?

**Update**

I thanks the authors for clarifying the concerns on the proof and technical achievements. These adequately addressed my concerns so I increase the score slightly to reflect that. On the other hand, I'm still on the fence about the first concern, i.e., the practical usage of the algorithm. I agree with the authors that communication is the bottleneck in many distributed training problems. But I think it would be more convincing if time-wise results are shown in the paper to support the claim that trading 3X memory with bandwidth would be beneficial. No significant limitations are shown.

 Thank you for giving us constructive feedback. We will make our claims and contribution more clearly in our revised version. We have fixed all the typos as you mentioned.

+ 1: Comparison to the related work and clarification of the contribution:
    + We appreciate your suggestions for further improvement of our paper. Our paper considers a stochastic loss function, where the loss cannot be written into the finite sum formulation. Therefore previous methods (such as SVRG and SAGA or Spider) cannot be applied. For the contribution, we will clarify that "fully compensating the history error" means the compression side-effect becomes minor compared to the leading convergence rate. We will update our paper in our revised version.

+ 2: Batch size requirement ($T^{\frac{1}{3}}$) of STORM:
    + Actually this is required to get a faster convergence rate ($T^{\frac{-2}{3}}$) and it will not defeat the purpose of its variance-reduced gradient estimator. In the  first work of STORM, it does not need this but can only attain a convergence rate of ($T^{\frac{-1}{2}}$). Therefore we choose this batch size in order to improve the convergence rate.

+ 3: The algorithms are analyzed as estimators with some specified error in finite time (where the step-size is constant and depends on the number of iterations) rather than optimization algorithms (with a decreasing step-size) where the convergence rate implies that running the algorithm for longer improves the guarantee.
    + We politely point out that this is a widely used strategy for reporting the convergence rate [Chen et al., 2020, Tang et al., 2019]. Our result clearly indicates that the error bound drops when increasing the total training steps.



+ 4: Why not compressing the gradient estimator:
    + Actually we have also tried this method and its performance is still worse than ECX. We will add more explanation about what ECX uses two history compression error and why alternative methods may not work.

+ 5: Why using Assumption 2:
    + First, we politely argue that this is actually a widely used assumption in previous work [Tang et al., 2019, Wangni et al., 2018]. ECX admits a similar convergence result when using the signal-to-noise, as you suggested. We use this assumption just for the simplicity of the theoretical analysis, and we would add another set of theorems using this assumption if this is helpful.

+ 6:  The dependence on a (constant) step-size and the total number of iteration did not appear in the original analysis, which instead used decreasing step-sizes. As the submission points out, having the parameters depend (on some power of 1/T) is too small and should be decreased over time instead.
     + The full version of the convergence rate is presented in the Theorems, and in the Corollary, we purposely set the learning rate to a certain constant value to simplify the theoretical analysis. When using decreasing step-sizes, our result (Theorem 1, 2, 3, 5, and 7) can lead to the result you suggested, and we will add some discussions about this in our revised version.

+ 7: Setting of the step size:
    + Actually those are the boundaries for the step size and we can set the step to some small numbers in order to meet the rrquirements.

+ 8: Corollary 6 also does not follow from Theorem 5 as the specified $\alpha=8L^2\gamma^2/n$ falls outside of the range allowed for Thm 5. My guess is that the step-size should instead depend on the smoothness constant of the stochastic functions?
    + Yes thanks for pointing this our. This is a type and $alpha$ should be equal to men $8(L^2 +L_F^2)\gamma^2/n$ instead.

+ 9: Please be careful when discussing the dependence of the step-size on the current iteration $t$ or the total number of iterations $T$. For example, Figure 1 mentions that $\alpha_t=1/t$ but the text before it is "However, we find that when $\alpha_t$ is very small ($1/T$) for ROOT-SGD and $1/T^{2/3}$ for STORM, where $T$ is the total training iterations". Is $1/T$ it  or $1/t$? 
    + Thanks for pointing this out. Actully  $\alpha_t$ should decrease along with the training. So it should be "However, we find that when $\alpha_t$ is very small ($1/t$) for ROOT-SGD and $1/t^{2/3}$ for STORM, where $t$ is current training iteration". We will be careful about the choice for $T$ and $t$ in our revised version.

+ 10: Typos in Assumption 1:
    + Thanks for pointing this out. Yes it should be $\mathbb E_{\xi^{(i)}\sim D_i} \|\nabla F(x;\xi^{(i)}) - \nabla f(x)\|^2 \leq \sigma^2,\quad\forall x,i,$ and  $f$ is the $L$-smooth while $F$ is $L_F$-smooth.
    
+ Notation of $C_{\omega}$:
    + Thanks for your suggestion, we will change $C_{\omega}$ into $C_{\omega_t}$ in our revised version.

## Reference
Jianqiao Wangni, Jialei Wang, Ji Liu,  Tong Zhang. "Gradient Sparsification for Communication-Efficient Distributed Optimization." NIPS (2018).

Chen, C.-Y., Ni, J., Lu, S., Cui, X., Chen, P.-Y., Sun, X., Wang, N., Venkataramani, S., Srinivasan, V. V., Zhang, W.,& Gopalakrishnan, K. (2020). "Scalecom: Scalable sparsified gradient compression for communication-efficient distributed training." Advances in Neural Information Processing Systems (2021).

Hanlin Tang, Xiangru Lian, Chen Yu, Tong Zhang, Ji Liu. "DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression." ICML (2019).

 We appreciate your constructive feedback. We will add more discussion about the extra memory consumption, our contribution, algorithm design and theoretical claims according to your suggestions in our revised version.

+ 1: Extra memory consumption:
    + We politely argue that 2X of extra memory buffer is acceptable, especially when the communication cost dominates. Previous studies for error compensation (which also consumes 1X of extra memory buffer) have already proved that the total training speed would be greatly improved for the training of ResNet and BERT-Large [Tang et al., 2021] on real clusters by using 1-bit compression.

+ 2: If GBytes of memory is needed for all the compressors, why can't we use more bits (more than 1-bit) instead?
[What does this mean?]
    + Communication is the major bottleneck for distributed learning. In distributed learning, the machines may not be at the same place, and the communication time dominates the computation time. In this case, it is still reasonable to use more memory to save communication time. 

+ 3: In the theoretical analysis, a general method of decomposing a convergence rate into uncompressed and compressed parts is given. This is somewhat confusing as it seems to be the standard method to prove the convergence of any compressed optimization algorithm, please correct me if I'm wrong. 
    + We politely argue that this result is not trivial because error compensation is a non-Markov process, so that the convergence analysis is much more complicated. Previous work for error compensation did not give this type of general result [Stich et al., 2018, Zheng et al., 2019]. These general results provide an illuminative understanding of the convergence behavior of error compensation methods and can be easily applied to future VR algorithms.

+ 4: Besides, obtaining the same $O(1/\sqrt{nT})$ rate compared to the uncompressed algorithms is not surprising:
    + Actually, an asymptotic rate of $O(1/\sqrt{nT})$ was only achieved for Momentum SGD and SGD,  not for STORM or IGT, which admits different convergence rates as shown in Corollary 6 and 8.

+ 5: Why ErrorCompensatedX can only be used on the algorithms specified in Table 1? If the main idea is to compensate the entire history, why can't other optimizers like Adam use it?
    + This is because error compensation can only work for the case where the optimizer is linearly dependent on the gradient, and the direct extension to the original Adam algorithm does not work [Tang et al., 2021].

+ 6: Why need Assumption 2:
    + First, we politely argue that this is actually a widely used assumption in previous work [Tang et al., 2019, Wangni et al., 2018]. ECX admits a similar convergence result when using the signal-to-noise, as you suggested. We use this assumption just for the simplicity of the theoretical analysis, and we would add another set of theorems using this assumption if this is helpful.

## Reference
Jianqiao Wangni, Jialei Wang, Ji Liu,  Tong Zhang. "Gradient Sparsification for Communication-Efficient Distributed Optimization." NIPS (2018).

Hanlin Tang, Shaoduo Gan, A. A. Awan, Samyam Rajbhandari, Conglong Li, Xiangru Lian, Ji Liu, Ce Zhang and Yuxiong He. “1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed.” ICML (2021).

Stich, S. U., Cordonnier, J.-B., & Jaggi, M. "Sparsified sgd with memory." In Advances in Neural Information Processing Systems (2018). 

Zheng, S., Huang, Z., & Kwok, J. (2019b). "Communication-efficient distributed blockwise momentum sgd with
error-feedback." NIPS (2019).

Hanlin Tang, Xiangru Lian, Chen Yu, Tong Zhang, Ji Liu. "DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression." ICML (2019).

 We appreciate your suggestion and will try more experiments in our revised version.

 Thanks for your positive comments. We appreciate your advice and will add more details for the mathematical deduction.

+ 1: Comparison to the method of compressing momentum:
    + We have also tried this approach and found that it still runs slower than ECX. From a theoretical perspective, recently, we found that compressing the momentum is helpful for single compensation but admits the same rate with ECX when using two history compression errors.

+ 2: Mathematical deduction for line 125 and 121:
    + We apologize for the inconvenience appreciate your effort in checking. Basically, the idea is to calculate the coefficient of each $\delta$ when different compensation strategies are applied. We will add more details about this part in our revised version. 

In contrast to the existing works that only use the compression error from the last step to correct the gradient, this paper proposed to use the compression errors from the previous two steps. The paper assumes that the compression is applied to gradient before getting accumulated to momentum. Under certain conditions, the theoretical analysis shows that the proposed method achieves better bound than only using the compression error from the last iteration.  This paper proposed a general framework for analyzing different gradient estimators. The theoretical analysis is sound and convincing. The writing needs some improvement to better explain the proposed idea and how we derive its formulation. But I have one major question for the authors.

Zheng, S., Huang, Z., & Kwok, J. (2019a) applies compression to momentum instead of gradient estimator, while this paper does it in an opposite way and assumes that the compression is applied to gradient estimator instead of momentum. I am curious how do the theoretical results of the proposed method compare to theirs? From Theorem 2, we can see that the bound is the same with alpha=1 for single compensation and ErrorCompensatedX. So it is interesting to know if the bound can also be improved if we apply compression directly to momentum.

In addition, how do we derive the equation equation below line 125 from the equation below line 121? I cannot find any explanation. And, how do we obtain the equation (3) from the equation equation below line 125? Why there are extra coefficient \alpha_{t-1}/\alpha_t and \alpha_{t-2}/\alpha_{t-1}?

The experiment is not strong. But I appreciate the theoretical contributions, so I still advocate for acceptance. Yes

Meta-review: The paper presents a novel error feedback mechanism to handle momentum-like terms (most notably, SGD with momentum and STORM). The paper presents theoretical analyses of existing momentum-like algorithms coupled with compression based on their modifications and experimental results confirming their results.

The reviewer commended the new direction and find the results of interest to the community. The reviewers also find, as a minor issue, that the assumptions used in the analysis are a bit strong and, more crucially, the experiments do not yet clearly demonstrate practical usability.

The reviewers find that there is currently not enough motivation presented that would justify the 'need' of two momentum buffers. In the particular in the presence of earlier work that successfully (though mostly empirically) applied compression to momentum SGD with a single buffer [such as the cited (Zheng, S., Huang, Z., & Kwok, J. (2019a). Communication-efficient distributed blockwise momentum sgd with error-feedback) and (Vogels, T., Karimireddy, S. P., & Jaggi, M. (2019). Powersgd: Practical low-rank gradient compression for distributed optimization.)] the reviewers find that the key differences should be highlighted better. 
Additionally, I also find it strange that the paper sets off with the question 'what is the best compression method for variance reduced algorithms?', yet optimality of the proposed scheme is not discussed. 

The reviewers provided detailed and well-justified feedback and the authors are encouraged to take their comments into account in the revision. In particular, since the application of error feedback to STROM appears to be a novel direction, we are confident that this work will attract interest in the community after repositioning and recalibration.

Decision: accept