Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: This paper examines the question of learning in neural networks with random, fixed feedback weights, a technique known as “feedback alignment”. Feedback alignment was originally discovered by Lillicrap et al. (2016; Nature Communications, 7, 13276) when they were exploring potential means of solving the “weight transport problem” for neural networks. Essentially, the weight transport problem refers to the fact that the backpropagation-of-error algorithm requires feedback pathways for communicating errors that have synaptic weights that are symmetric to the feedforward pathway, which is biologically questionable. Feedback alignment is one approach to solving the weight transport problem, which as stated above, relies on the use of random, fixed weights for communicating the error backwards. It has been shown that in some cases, feedback alignment converges to weight updates that are reasonably well-aligned to the true gradient. Though initially considered a good potential solution for biologically realistic learning, feedback alignment both has not scaled up to difficult datasets and has no theoretical guarantees that it converges to the true gradient. This paper addresses both these issues.

To address these issues, the authors introduce two restrictions on the networks: (1) They enforce “monotone” networks, meaning that following the first layer, all synaptic weights are positive. This also holds for the feedback weights. (2) They require that the task in question be a binary classification task. The authors demonstrate analytically that with these restrictions, direct feedback alignment (where the errors are communicated directly to each hidden layer by separate feedback weights) is guaranteed to follow the sign of the gradient.  (Importantly, they also show that monotone nets are universal approximators.) Empirically, they back up their analysis by demonstrating that in fully connected networks that obey these two restrictions they can get nearly as good performance as back propagation on training sets, and even better performance on tests sets sometimes. However, they also demonstrate (empirically and analytically) that violating the second requirement (by introducing more classes) leads to divergence from the gradient and major impairments in performance relative to backpropagation.

Ultimately, I think this is a great paper, and I think it should be accepted at ICLR. It provides some of the first rigorous analysis of feedback alignment since the original paper came out, and unlike those original analyses, it is not restricted to linear networks (which are certainly not universal function approximators). I have looked over the proofs, and they seem to all be correct. As well, I found the paper easy to read, which was nice. However, there are a few things that could be done to clarify the contributions and situate the work within the field of biological learning algorithms better:

1) Though they do not include rigorous analyses, two previous papers have demonstrated empirically that feedback alignment works extremely well as long as the feedback weights share the same sign as the feedforward weights (see: Moskovitz, Theodore H., Ashok Litwin-Kumar, and L. F. Abbott. "Feedback alignment in deep convolutional networks." arXiv preprint arXiv:1812.06488 (2018) and Liao, Qianli, Joel Z. Leibo, and Tomaso Poggio. "How important is weight symmetry in backpropagation?." In Thirtieth AAAI Conference on Artificial Intelligence. 2016). Due to the requirement for monotone networks, this work is also providing a guarantee that the sign of feedforward and feedback weights are the same. That does not subtract substantially from the contributions of this paper, as the provision of the analytical guarantees is important. But, it is important for the authors to consider how their work relates to this past work. For example, could their analytical approach work equally well with nothing more than a sign symmetry guarantee? This should at least be discussed.

2) It should be admitted somewhere in the paper that the second requirement on the networks for binary tasks is deeply unbiological. As such, it should be recognized in discussion that this paper provides some important contributions to our understanding of feedback alignment, but does not ultimately move the question of biologically realistic learning forward all that much. Indeed, the discussion at the end about applications notably ignores biology. But, rather than just ignoring it, the biological mismatch should be openly admitted.

3) The results with the test sets are a little strange, at least for the tests with larger numbers of categories. In Bartunov et al. (2016), they reported not only better training set results with backprop, but also better test set results generally, than feedback alignment. Are the authors sure that their results, in say, Table 4, are not indicative of insufficient hyperparameter optimization?

Small notes:

- Lillicrap et al.’s paper was eventually published in Nature Communications (see citation above), and the reference should be changed to reflect this.

- The discussion on the impact of convolutions could be beefed up a little bit. In particular, it could be discussed relative to the results of Moskovitz et al. (above) who show that convnets work fine with nothing but guaranteed sign symmetry.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper presents an approach towards extending the capabilities of feedback alignment algorithms, that in essence replace the error backpropagation weights with random matrices.  The authors propose a particular type of network where all weights are constraint to positive values except the first layers, a monotonically increasing activation function, and where a single output neuron exists (i.e., for binary classification - empirical evidence for more output neurons is presented but not theoretically supported).  This is to enforce that the backpropagation of the (scalar) error signal to affect the magnitude of the error rather than the sign, while preserving universal approximation.  The authors also provide provable learning capabilities, and several experiments that show good performance, while also pointing out limitations in case of using multiple output neurons.

The strong point of the paper and main contribution is in terms of proposing the specific network architecture to facilitate scalar error propagation, as well as the proofs and insights on the topic.  The proposed network affects only magnitude rather than sign, and the authors demonstrate that it can do better than current FA and match BP performance.  This seems inspired from earlier work [1,2] - where e.g., in [2] improvements are observed when feedback weights share the sign but not the magnitude of feedforward nets.

Summarizing, I believe that this research is interesting, and can lead to improvements in FA algorithms that could potentially be more biologically plausible, and offer advantages such as full weight update parallelization (although this is more related to the fixed weights rather than the method per-se given my understanding).  However, this also seems - at the moment - to be of limited applicability.

===
Furthermore, the introduction of the network with positive weights in the 2nd layer and on is remiscent of non-negative matrix factorization algorithms.  Can the authors establish a link to these methods, where variants with backprop have also been proposed?



[1] Xiao W. et al. Biologically-Plausible Learning Algorithms Can Scale to Large Datasets, 2018
[2] Qianli Liao, Joel Z Leibo, and Tomaso Poggio.  How important is weight symmetry in backprop-agation, AAAI 2016
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: ------update------

After talking to other reviews and reading the rebuttal, I am convinced that the paper contributes sufficiently to the theoretical understanding of the FA algorithm and should be accepted as a conference paper. 

I hope that, in the next revision, the authors could include more about the limitation of their work and potential alternatives to improve the generosity of the proposed method.

-------end of the update------

The paper presented a mono-net which has only positive weights and monotonically increasing functions in between layers except for the first layer, and it can be shown that the proposed mono-net is capable of modelling any continuous functions. The modified Direct Feedback Alignment (DFA) is applied where only signs are used to update the weights. There are several issues and concerns that leads me to the following comments and concerns:

(A) Why is it necessary to construct a monotonic layer which is constrained to only be able to approximate monotonic functions? Please elaborate. If the concern is about the gradient update, then there is better way of constructing such a network with limiting it being only capable of modelling monotonic functions.

(B) If, given the proof in appendix, that the proposed mono-net is indeed a universal function approximator, then why do all the layers on top of the first layer have to contain only positive weights?

Following the proof, given a neural network with only one hidden layer and hyperbolic tanh activation functions, as long as the weights in the layer that is after the tanh functions are all non-negative or non-positive, the neural network is also a universal function approximator.

Since the chosen activation function monotonically increases in the input domain, the sign of the update calculated by DFA is the same as the gradient calculated by the chain rule. 

The aforementioned way of constructing a neural network allows it to be able to model to change the monotonicity of the function in between layers when the two sets of weights in consecutive two layers have opposite signs. Also, the update gives the sign of the gradient calculated by backpropagation.   

(C) What is the different between the proposed network along with the update rule and a network with only non-negative weights in the layers above the first layer trained with RPROP? They look exactly the same to me. 

If so, another perspective of the story is that the paper emposes a non-negative constraint on the weights in a neural network, and this could be used as a baseline.

(D) The proof that gives the universal approximation theorem explicitly defines the squashing function/the activation function to be bounded and this paper follows the setting, which is reflected in the experiments where the proposed network with ReLU activation functions don't work well. 

However, the expressiveness of ReLU networks has been shown to be strong and they are also universal approximators. I'd believe that with (B), it might lift the constraints brought by ReLU in the settings proposed by the paper.

(E) I am not sure why for now the number of output units matters that much. The training algorithm can easily ignore other output units when samples of a specific class is presented, then after learning, the algorithm can rank the output units to make predictions. 
review_title: Official Blind Review #3

Meta-review: This paper provides a rigorous analysis of feedback alignment under two restrictions 1) that all, except the first, layers are constrained to realize monotone functions and 2) the task is binary classification. Overall, all reviewers agree that this is an interesting submission providing important results on the topic and as such all agree that it should feature at the ICLR program. Thus, I recommend acceptance. However, I ask the authors to take into account the reviewers' concerns and include a discussion about limitations (and general applicability) of this work.
Decision: Accept (Poster)