Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Thanks for author(s) for their paper. I enjoyed reading it.

This paper introduces a new method for computing the backward updates of a neural network called Direct Kolen-Pollack learning (DKP). Similar to Direct Feedback Alignment (DFA), the aim of this work is to introduce a viable alternative to back-propagation (BP) that works in parallel while achieving similar performance. Parallelization benefits comes from the fact the the backward path is unlocked. To summarize, DKP shows an interesting improvement over DFA for CNNs but results and analysis are slightly insufficient.

Kolen-Pollack learning (KP) suggests “that updates the backward matrices with the same gradient as the forward weights and uses weight decay on both the forward and backward matrices to encourage symmetry between the two.” The idea in DKP is simple: replace error signal from each layer with the final layer’s error signal in KP. ($\delta_l \rightarrow \delta_k$ in Eq. 6 and 8)
Author(s) identified the performance gap between DFA and BP in CNNs as their motivation and tested their method on image classification on Fashion-MNIST, CIFAR10 and CIFAR100 datasets.

Notes:
1. The writing is easy to understand and clear. Authors are planning to release the code soon.
2. Please introduce matrix operations used in Eq. 1 and 2 somewhere.
3. In DKP backward matrices are no longer fixed as in DFA but rather are updated after each batch with their own update rule and learning rate. These updates are still parallelizable but an analysis and/or experimentation on the speed gain with these new updates are needed, specially comparing to BP and DFA. Similarly, in the related work it’s mentioned that “[previous works except DFA] currently show no tangible benefits over backpropagation as they all have larger memory requirements” but I don’t see any discussion on that in the paper.
4. Following prior works, weight decay on both the forward and backward parameters during training are used. Author(s) mention that weight decay is important to make this method work. As another contribution they provide mathematical justifications for its use. The result of this argument is that following this update rule after enough time, the forward and backward matrices will be close to each other. Please clarify following questions:
    4. i. This argument depends on $A_{i,j}$ being the same for backward and forward update. But comparing Eq. 8 and 9, doesn’t this require $\eta_B=\eta_W$?
    4. ii. This problem is worse with DKP when comparing Eq. 6 and 9 as ${\left(-\eta_B \delta_k^T . a_{l-1} \right)}_{i,j} \neq {\left(-\eta_W \delta_l . a_{l-1}^T \right)}_{i,j}$.
5. Description of Kolen-Pollack learning is minimal and a bit too late in the paper despite the proposed method is named after it. In my opinion it would’ve been better to explain it a bit more and earlier. For example the first few paragraphs of section 2.3 could move to earlier sections.
6. Figure 2: Value of $\lambda$ is missing. Also if I am not mistaken according to Eq. 11 the rate of convergence of the two should be proportional by $(1-\lambda)^t$. I am not certain that I see this trend in the graphs, specially layers 0 and 1 that seem to be plateauing. I understand that you have similar observation at the end of section 2. The explanation seems to be that higher layers are easier to be linearly approximated. However, my counterpoint is that Eq. 11 shows an exponential decay to zero and it does not depend on layer l. Could you comment on this please?
7. Despite the fact that KP does not make any assumptions about the structure and inductive biases of the network, DKP is proposed only for CNNs and image base classifications. Why shouldn’t DKP be used for MLPs, RNNs, etc.? I would really like to see its performance compare to BP and DFA for non-CNN structures.
8. In related works, author(s) mention prior works that DFA have a hard time with VGG-16 optimization but the experiments are done AlexNet. What is the reason for this mismatch. It would have been much easier to make a direct comparison with previous works.
review_title: DKP shows an interesting improvement over DFA for CNNs but results and analysis are slightly insufficient

final_score: rating: 5: Marginally below acceptance threshold
review_content: ## Second Review

I thank authors for taking time and answering my queries. However current manuscript fails to point out key difference between Akrout 19 kolen-pollack method and DKP (proposed method). Combining FA with DKP does not add sufficient novelty. As pointed out by other reviewers, paper should highlight key differences and reasoning for such combination. I am happy to see additional results with DRTP, however it is also important to test your approach based on there methodology.   It is difficult to gauge the significance of your approach, since training protocol varies a lot. I would request authors to add more baselines and training protocols  (future submission) to show that your method is robust and can also train deeper CNNs models. Current submission missed out on many key aspects, despite having promising direction. I hope our reviews help you in strengthening this promising work.   
## Summary

This work proposes an approach to update feedback weights in DFA using modification of kolen-pollack method, which helps in training deep CNN network. 

## First Review
Citation missing for key work on assessing the scalability of bio-inspired approaches and highlighting key limitations [Bartunov 18], variants of DFA [ Moskovitz 18, Frenkel 19]  and recently an approach similar to DFA with target projection known as LRA (also has similarity with Direct Kolen-Pollack) showing promising performance on deep CNNs [ Ororbia & Mali 2020].

The update rule used in recursive-LRA is similar to what proposed in this paper
Delta_b(update for feedback weights)  = learning rate *( teaching signal(delta_k) * post-activation from layer below (a_(l-1)) 

For LRA Delta_b(update for error weights)  = learning rate *( teaching signal(error_k) * post-activation from layer below (a_(l-1)) 

For weight mirroring [ Akrout 19] Delta_b(updates for feedback weights) = = learning rate *( teaching signal(delta_(l) * delta(updates) from layer above (delta_(l+1)) 

One can see we can derive chain rule formulation with certain assumption in which feedback matrix or error matrix acts like transpose of forward weights (rotated 180 degree).

As shown in Feedback alignment(lillicrap 16) the updates for FA and LRA lie with 90-degree w.r.t BP. One can provide such plots to show how far way are your updates w.r.t. BP and other bio-inspired approach.

“We also found that the optimal hyperparameters and optimizers for the backward weight matrices in DKP seem to vary greatly from one network to the next”
Can you provide more detail about your experimental setup? What are the range of hyper-parameters and how does DKP perform w.r.t BP and other variants? It is well known that DFA in its vanilla form suffer whenever tested with deep networks on challenging benchmarks such as imagenet (akrout 19, Bartunov 18). As shown by Moskovitz 18 and Crafton 19, integrating BP or making feedback weights close to forward weights helps in learning for complex benchmarks. So, what different does DKP offer? is it robust, speeds up the convergence, always stays consistent (robust against bad initialization). Current manuscript fails to highlight these points which could make current work stronger.

Do you constraint your feedback weights, if so how? If not, then how does model ensure that feedback weights are respecting forward neural activities and helping it to converge? Won’t feedback weights grow making discrepancy between forward and backward activities, thus slowing the convergence of the network?

Comparison against other variants of DFA
We would like to see detailed comparison w.r.t various variants or family of FA(Moskovitz 18, Frenkel 19] and LRA (since update rules are similar). 





[Bartunov 18] Bartunov, S., Santoro, A., Richards, B., Marris, L., Hinton, G.E. and Lillicrap, T., 2018. Assessing the scalability of biologically-motivated deep learning algorithms and architectures. In Advances in Neural Information Processing Systems (pp. 9368-9378).

[Akrout 19] Akrout, M., Wilson, C., Humphreys, P., Lillicrap, T. and Tweed, D.B., 2019. Deep learning without weight transport. In Advances in neural information processing systems (pp. 976-984).

[Moskovitz 18] Moskovitz, T.H., Litwin-Kumar, A. and Abbott, L.F., 2018. Feedback alignment in deep convolutional networks. arXiv preprint arXiv:1812.06488.

[Frenkel 19] Frenkel, C., Lefebvre, M. and Bol, D., 2019. Learning without feedback: Direct random target projection as a feedback-alignment algorithm with layerwise feedforward training. arXiv preprint arXiv:1909.01311.

[Ororbia and Mali 20] Ororbia, A., Mali, A., Kifer, D. and Giles, C.L., 2020. Reducing the Computational Burden of Deep Learning with Recursive Local Representation Alignment. arXiv preprint arXiv:2002.03911.


review_title: iterative work, need more clarification

final_score: rating: 5: Marginally below acceptance threshold
review_content: Strength:
As mentioned in the paper, although DFA is more biologically plausible, it does not work well for deep networks and CNNs. This work proposed a possible improvement for this problem.

Weakness:
(1) My main concern is the novelty of this paper. To my understanding, the DKP and the convergence of weight decaying are first proposed in (Akrout et al., 2019). However, updating the backward matrix suffers from the problem of different dimensions. I think the main contribution of this work is the experimental demonstration of the lower angle achieved by DKP. Therefore, I hope the authors can make it clear the main contributions of this work. It is also necessary to comment on how the approach in this paper is different from (Akrout et al., 2019).

(2) In the experiments, the authors only compare the DKP with the DFA and BP. It is also necessary to compare the performance with other references. For example, how much the performance can be improved by the proposed method compared to the methods introduced in section 1.1.

(3) In addition, the network sizes are not clearly mentioned in the experiments. It is difficult for readers to make a judgment about the effectiveness of this work.



review_title: This paper proposed a direct Kolen-Pollack (DKP) method which updates the weight of the backward pass of DFA and achieves better performance.

Meta-review: This paper investigates an improvement to the direct feedback alignment (DFA) algorithm where the "backward weights" are learned instead of being fixed random matrices. The proposed approach essentially applies the technique of DFA to Kolen-Pollack learning. While reviewers found the paper reasonably clear and thought the experiments were acceptable, there were significant concerns about the novelty of the approach and the fact that the proposed approach was a straightforward combination of existing ideas. Further, the paper could have done a better job situating (and applying) the proposed method to DFA variants that have been proposed since the original DFA paper came out.
Decision: Reject