BP-Modified Local Loss for Efficient Training of Deep Neural Networks

Authors: REN Lianhai, Qianxiao Li
Primary area: optimization
Keywords: deep learning optimization, local loss training, bias-variance balance

Abstract:
The training of large models is memory-constrained, one direction to relieve
    this is training using local loss, like GIM, LoCo, and Forward-Forward
    algorithms. However, the local loss methods often face the issue of slow or
    non-convergence. In this paper, we propose a novel BP-modified local loss
    method that uses the true Backward Propagation (BP) gradient to modify the
    local loss gradient to improve the performance of local loss training. We
    use the stochastic modified equation to analyze our method and show that
    modified offset decreases the bias between the BP gradient and local loss
    gradient, but introduces additional variance, which results in a
    bias-variance balance. Numerical experiments on full-tuning and LoKr tuning
    on the ResNet-50 model and LoRA tuning on the ViT-b16 model on CIFAR-100
    datasets show 20.5\% test top-1 accuracy improvement for the Forward-Forward
    algorithm, 18.6\% improvement for LoCo algorithm and achieve only on average
    7.7\% of test accuracy loss compared to the BP algorithm, with up to 75\%
    memory savings.

Decision: accept (poster)
Reviewer scores: 6, 6, 5, 8

Reviews and discussion:
<thread>
creation date:
1729063530094

thread_type:
official_review

<thread_object>
creator: reviewer_uzvz

<content>
summary:
This paper proposes a novel BP-modified local loss to improve the performance of local loss training, which decreases the bias between the BP gradient and local loss gradient but introduces additional variance. Then authors provide a theoretical analysis using the stochastic modified equation, illustrating the bias-variance trade-off and deriving optimal scaling factors. Experiment results show that the proposed method can effectively improve the performance of local loss algorithm while increase little GPU memory.

strengths:
- The theoretical derivation of the proposed BP-Modified Local Loss algorithm is convincing.
- Experimental results indicate that the proposed method significantly improves the performance of local loss training algorithm.
- The paper is highly readable and describes the implementation details of BP-Modified Local Loss algorithm in detail.

weaknesses:
- The main motivation of local training algorithms is to train large datasets or large models in resource-constrained environments. So I think authors could scale to some datasets larger than CIFAR-100, such as the ImageNet, to better illustrate the applicability of the proposed method and meet the design motivation.
- The optimization of the Loss function tends to significantly affect the training convergence performance. So I think it is necessary to provide model training time comparison results between the BP-Modified Local Loss method and other Local Loss baseline methods, such as total epochs needed for convergence.

questions:
Overall, I think this is a good paper and I would raise rating if authors could address my concerns above. In addition, I suggest adding some figure descriptions if possible so that the readers could more clearly understand the authors’ intention.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730010975928

thread_type:
official_review

<thread_object>
creator: reviewer_eznv

<content>
summary:
In this paper, authors proposed a novel training algorithm called BP-modified local loss, which aims to address non-convergence issues and poorer performance in the previous local loss methods, e.g., GIM, LoCo, and Forward-Forward algorithm. It combines backward propagation (BP) with local loss methods by modifying the local gradient using the BP gradient to balance bias and variance.

strengths:
1. Strong empirical results: The BP-modified local loss shows significant improvement, e.g., up to 36% improvement in the test accuracy for Forward-Forward algorithm, 20% for LoCo and 11% for MPC.

2. The use of scaling factors $\alpha_{h}$ and $\lambda_{h}$ to manage the bias-variance balance is well motivated. The use of stochastic differential equations to model the modified training dynamic helps in understanding the benefits, as well as the limitations of the method.

weaknesses:
1. Generalisability is one of the major concerns, as the scope of the experiments is a bit limited. The results are primarily conducted on CIFAR-100, a relatively small dataset. While the improvements are evident, a more robust evaluation that includes larger datasets like ImageNet, as well as tasks other than image classification, would make the proposed method more convincingly.

2. The authors leveraged the Ornstein-Uhlenbck process to model the bias-variance dynamic, however, the OU process is a simplified linear model, and deep neural network training is inherently non-linear and non-convex. The validity of the approximation that discussed in the paper would require some empirical evidence to support, as well as some discussions in more depth. For example, deep neural networks exhibit different training dynamics across layers, with shallow and deep layers experiencing different gradient behaviours. Can authors provide how gradients evolve for different layers and test if the OU process holds similarly for both shallow and deep layers.

questions:
1. The method introduced several new hyper-parameters, such as batch size $B^{'}$ for the BP gradient, scaling factors $\lambda_{h}$ and $\alpha_{h}$, as well as the lazy update period $K$. I wonder how sensitive is the performance to these hyper-parameters, can authors report sensitivity analyses for these hyper-parameters? And what guidance or insight can authors provide on how to tune them for different models and datasets? Specifically, the optimal scaling factors, $\lambda_{h}$ and $\alpha_{h}$, are derived theoretically, but their practical implementation seems challenging. 

2. Although the authors adopted some strategies like lazy update and split mini-batch, computing the offset $Δg_{h,t}$ involves additional operations. I wonder what’s the overhead in terms of runtime and memory usage as the depth of the network increases? Can authors quantify the impact of these additional operations on the overall training efficiency? Such as total training time, per-epoch time as well as GPU memory usage as the depth of the network increases?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730353867051

thread_type:
official_review

<thread_object>
creator: reviewer_z11r

<content>
summary:
This paper proposes a new BP modified local loss method, which aims to adjust the local gradient by introducing an additional offset to reduce the deviation between the local gradient and the true gradient and introduce additional variance. This approach improves the performance of local loss training while maintaining memory efficiency. The theoretical analysis and experimental results of the paper show that this method can effectively improve performance on different models and tasks, while significantly reducing memory usage. This is an interesting and promising research direction with important implications for the deep learning community.

strengths:
Innovation: The proposed BP modification of local loss method improves training efficiency while reducing memory usage, which is a valuable contribution.
Theoretical analysis: The paper provides a theoretical analysis based on the random modification equation, deeply explores the bias-variance trade-off, and derives the optimal scaling factor.

weaknesses:
Impact of delayed gradient adjustment: The paper discusses the noise accumulation and bias adjustment problems that may be caused by delayed gradient adjustment, but does not provide a detailed analysis. It is recommended that the authors further study the impact of these factors on training dynamics and explore possible solutions.
More extensive experimental validation: Although the paper conducts experiments on multiple models and tasks, it is still recommended to conduct further experimental validation on larger models and datasets to demonstrate the universality and scalability of the method.

questions:
The paper mentions hyperparameters such as offset batch size B' and sampling period K, but does not discuss the selection of these parameters in detail. It is recommended that the authors provide more analysis on how these hyperparameters affect model performance and memory usage.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730666609280

thread_type:
official_review

<thread_object>
creator: reviewer_zuvr

<content>
summary:
The paper proposes a method to compute local loss based on BP which significantly improves performance of local loss based training methods with negligible memory overhead.

strengths:
Significantly improved performance compared to SoTA local loss based learning methods

Results in negligible additional memory usage

Strong theocratical foundations for proposed method

weaknesses:
Compare throughput with SoTA local loss-based learning methods 

Misc: 
Paraphrase line 31, 32, and 33 for better readibility

questions:
Is there any overhead in latency or throughput for BP-modified loss compared to other local loss-based training methods?

Is the bias-variance trade-off analysis generalizable across various model architectures and tasks, or are there conditions under which it may not apply?

Can you further explain how using true gradient information doesn’t increase memory usage?

The lazy update and mini-batch splitting techniques are introduced to manage memory and computational costs. How sensitive is the method to the choice of period K and mini-batch size B'?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732620435330

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Rebuttal

comment:
Dear Reviewers,

We would like to express our sincere gratitude for your insightful comments and suggestions. We appreciate the time and effort you have dedicated to reviewing our paper. We have carefully considered each of your comments and have made the necessary revisions to improve the quality of the paper. Below is a global response to the common concerns raised by the reviewers:

1. **Modifications of the new revision (which is shown in blue color) include:**
     - We have some expressions for better readability (e.g. Lines 31-33).
    - We have revised the last two paragraphs of Section 3 that emphasize the effect and impact of the BP-modified local loss method.
    - We have added Figure 1 which describes the idea of the BP-modified local loss method.
    - We have revised Section 5 to highlight the effect of lazy update and mini-batch splitting techniques on the memory and computation overhead.
    - We have added additional experiments on the ImageNet-Tiny dataset. The results are shown in Table 2 in Section 4.2.
    - We have moved the memory and time results in Appendix D.1 to save space and added Table 5 to show the time overhead of the BP-modified local loss method and the original methods.
    - We have added some numerical results on different period $K$ and mini-batch size $B'$ in Appendix D.2 to show the influence of these hyperparameters.
    - We have reported the first epoch that the proposed method need to achieve the final test accuracy of the original methods in Appendix D.3 to show the efficiency of the proposed method.

2. **How sensitive is the method to the choice of period K and mini-batch size B', how do these hyperparameters affect model performance and memory usage.**

    **Response:** As discussed in Appendix C, a larger period $K$, or equivalently, a lazier update of the $\Delta \tilde{g}_{h}$ will lead to larger variance since the noise of the same offset will accumulate, and enhance the delayed bias, thus reducing the performance. On the other hand, as Eq. (7) suggests, a smaller $B'$ will also increase the variance. However, a smaller $K$ and larger $B'$ mean more computation.

    We have added some numerical results on the sensitivity of the period $K$ and mini-batch size $B'$ and the results are shown in Appendix D.2. The results align with the previous discussion that larger $K$ and smaller $B'$ save some time at the expense of lower performance and vice versa.

    Additionally, if the mini-batch size $B'$ is too small, the variance of the offset will be large and the performance might be worse, while it seems that it has more tolerance for larger $K$. If the period $K$ is too small or $B'$ is too large, the computation overhead will be large with diminishing performance improvement. Our experience suggests that $B'$ should be large enough to reduce the variance of the offset while $K$ can be relatively large such that the computation overhead is small. $B'$ being $1\%$ of $BK$ are usually good choices.

    The theoretical analysis of the influence of period $K$ and mini-batch size $B'$ is a challenging task and beyond the scope of this paper. We will consider this in our future work.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a new BP modified local loss method to improve local loss training. Specifically, it addresses the poor convergence and performance of existing local loss methods by modifying local gradients. This paper provides a theoretical analysis and derives optimal scaling factors. The proposed method exhibits strong empirical performance with marginal computational overhead. This paper is well-motivated and clearly written. This paper studies an important problem, the training of large models with limited memory, via local loss methods and provides strong empirical performance and interesting theoretical results. I recommend this paper for acceptance.