Enabling Fine-Tuning of Direct Feedback Alignment via Feedback-Weight Matching

Authors: Yunseok Lee, Seulki Lee
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: direct feedback alignment, deep learning, fine tuning

Abstract:
In this paper, we introduce feedback-weight matching, a new method that facilitates reliable fine-tuning of fully connected neural networks using Direct Feedback Alignment (DFA). Although DFA has demonstrated potential by enabling efficient and parallel updates of weight parameters through direct propagation of the network's output error, its usage has been primarily restricted to training networks from scratch. We provide the first analysis showing that existing standard DFA struggles to fine-tune networks pre-trained via back-propagation. Through an analysis of weight alignment (WA) and gradient alignment (GA), we show that the proposed feedback-weight matching enhances DFA's ability and stability in fine-tuning pre-trained networks, providing insights into DFA's behavior and characteristics when applied to fine-tuning. In addition, we find that feedback-weight matching, when combined with weight decay, not only mitigates over-fitting but also further reduces the network output error, leading to improved learning performance during DFA-based fine-tuning. Our experimental results show that, for the first time, feedback-weight matching enables reliable and superior fine-tuning across various fine-tuning tasks compared to existing standard DFA, e.g., achieving 7.97\% accuracy improvement on image classification tasks (i.e., 82.67\% vs. 74.70\%) and 0.66 higher correlation score on NLP tasks (i.e., 0.76 vs. 0.10). The code implementation is available at an anonymous GitHub repository.

Decision: reject
Reviewer scores: 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1730638209943

thread_type:
official_review

<thread_object>
creator: reviewer_49zy

<content>
summary:
In this paper, the author presents a feedback-weight matching strategy that amalgamates Direct Feedback Alignment (DFA) (Nøkland, 2016) and back-propagation gradients to refine the pretrained classification models. The proposed approach attains superior performance compared to DFA, yet falls short of the performance exhibited by back-propagation based techniques. The author offers an in-depth theoretical analysis and validates the proposed method using basic network architectures.

strengths:
1) The author offers a comprehensive theoretical analysis.
2) The proposed method attains superior performance compared to the original Direct Feedback Alignment.

weaknesses:
1) The suggested approach appears to be ineffective. It yields inferior results compared to back-propagation based methods. The suggested strategy aligns the Direct Feedback weight with the back-propagation weight, resulting in enhanced performance. In my view, the improvement in results primarily stems from the back-propagation optimization. 

2) The proposed method achieves lower performance than back-propagation methods.

questions:
It appears that the enhanced efficacy still stems from the back-propagation optimization. Can the author elucidate on this?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730650942086

thread_type:
official_review

<thread_object>
creator: reviewer_tmbj

<content>
summary:
This paper proposes feedback-weight matching to improve the reliability of fin-tuning for fully connected networks using direct feedback alignmeng.

strengths:
+ It is quite interesting to investigate feasibility of using DFA for fine-tuning rather than training from scratch.
+ The proposed approach uses weight decay on top of feedback-weight matching to improve stability of fine-tuning, which makes sense. 
+ It has been shown that the proposed approach can work on transformer models for NLP tasks, which is more challenging.

weaknesses:
- It is not clear why the proposed approach hasn't been tested for other transformer models like ViT? 
- The proposed approach has been evaluated on a limited number of datasets. Can it work on models trained, or do fine-tuning on imagenet? 
- Some part of the paper is a bit difficult to follow.

questions:
* Can this approach work for other vision models, not only just the fully connected ones?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731000513263

thread_type:
official_review

<thread_object>
creator: reviewer_r6rf

<content>
summary:
In this work, the authors analyzed the direct feedback alignment (DFA) for fine-tuning network pertained via backpropagation and identified its limitations compared. Accordingly, the feedback-weight matching is proposed to enhance the ability and stability of DFA in fine-tuning and combined it with weight decay to further reduce the network output error. Experiments are conducted on various datasets.

strengths:
1. *Originality*: It is novel to recognize and scrutinize that the strong weight alignment condition for DFA in the case of fine-tuning is unlikely to satisfy. In order to address this issue, the feedback-weight matching and weight decay for it are originally proposed to enhance the fine-tuning performance.

2. *Significance*: Fine-tuning non-fully connected networks has been an actively studied topic in the recent years, especially after the debut of ChatGPT. This work is of significance to analyze DFA for fine-tuning and propose the solutions: feedback-weight matching.

weaknesses:
1. *Quality*: It would largely improve the quality of this work to articulate the proof of propositions in mathematically standard manners. Usually propositions are presented in the main content and their mathematical proofs are rigorously detailed in appendices. 

2. *Clarity*: Please also have a careful double check on the boldface numbers in Tables 1 and 2. It looks confusing and misleading to highlight the performance of the proposed method instead of the best performance. The experimental results should also be further analyzed. Please see **Questions** for details.

questions:
1. Would the authors clarify the statement "Equation (3) is referred to as weak weight alignment (WA) (Refinetti et al.,
2021), representing the state where no particular relationship exists between $\mathbf{W}_{1< l< L}^t$ and $\mathbf{F}_{l}\mathbf{A}_{l}^{t} \mathbf{F}_{l-1}^{\top}$, and between $\mathbf{W}_{L}^t$ and $\mathbf{A}_{L}^t \mathbf{F}_{L-1}^\top$? It seems that from Equation (3), $\mathbf{W}_{1< l< L}^t = \mathbf{F}_{l}\mathbf{A}_{l}^t\mathbf{F}_{l-1}^\top$, and $\mathbf{W}_{L}^{t}$ and $\mathbf{W}_{1< l< L}^t = \mathbf{A}_{L}^{t} \mathbf{F}_{L-1}^{\top}$.

2. In Table 2, the performance of DFA for training from scratch is the best among all three compared methods. It seems to be inconsistent to the performance on CIFAR-100. Could the authors explain the possible reasons? 

3. Why is the dataset STL more difficult to fine-tune on?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732785550171

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We sincerely appreciate the reviewer's effort and constructive feedback, which have significantly contributed to improving our paper. 

Below, we summarize the revisions we have made based on the reviewer's suggestions:

---

1. The proofs section, originally in the main manuscript, has been moved to Appendix A. We have also updated the relevant equations (Eq. 4 and Eq. 5) in the manuscript to reflect this change.

2. As per the reviewer's suggestion, we have revised the captions for Tables 1 and 2 to provide clearer descriptions.

3. We have revised the manuscript to improve clarity, particularly in lines 165-166, where we have refined the text to enhance readability and coherence.

---

We have made every effort to address the feedback and improve the quality of the manuscript. We kindly request the reviewers to let us know if any uncertainties remain, so we can address them before the discussion period concludes. Thank you.
</content>
</thread_object>

</thread>


Metareview:
metareview:
(a) The paper introduces feedback-weight matching, a strategy that combines Direct Feedback Alignment (DFA) and backpropagation to enhance fine-tuning of pretrained networks, demonstrating improved performance over DFA but not surpassing backpropagation-based methods.

(b) Strengths: The paper demonstrates originality by identifying the limitations of strong weight alignment in DFA during fine-tuning and proposing feedback-weight matching with weight decay as a novel solution. The significance lies in analyzing DFA for fine-tuning rather than training from scratch, which is particularly relevant given the growing focus on fine-tuning in AI, especially with models like ChatGPT. The approach improves stability and performance through weight decay and feedback-weight matching and is validated on challenging transformer models for NLP tasks, showcasing its broader applicability.

(c) Weaknesses: The paper would benefit from clearer articulation of mathematical proofs, with rigorous details provided in appendices to enhance quality. Clarity issues include misleading boldface highlights in Tables 1 and 2 and inconsistent results, such as DFA's superior performance in one task but not others, which require further analysis. The evaluation is limited to a small number of datasets and lacks testing on widely used transformer models like ViT or Imagenet-trained models, raising questions about generalizability. While the proposed method improves upon DFA, its performance remains inferior to back-propagation-based approaches, and the improvements may largely result from incorporating back-propagation optimizations. Some parts of the paper are difficult to follow, impacting overall readability.

(d) The most important reasons for reject are: The paper lacks a clear explanation of the significant technical distinctions between the proposed DFA with Weight Decay and existing methods, which constitutes its core contribution. Additionally, the superior performance of DFA in fine-tuning may largely stem from the pretrained model rather than the proposed strategy, raising questions about the true source of the improvement. Further, the applying DFA to finetuning consistently underperforms relative to back-propagation and is limited to a narrower range of architectures. We understand that, rather than aiming to surpass back-propagation, the authors focus on improving the performance of DFA. However, given that the aforementioned limitation is highly evident, what practical value or significance does it hold?