Paper:
While much recent work has focused on biologically plausible variants of errorbackpropagation, learning in the brain seems to mostly adhere to a reinforcement learning paradigm; biologically plausible neural reinforcement learning frameworks however were limited to shallow networks learning from compact and abstract sensory representations. Here, we show that it is possible to generalize such approaches to deep networks with an arbitrary number of layers. We demonstrate the learning scheme -DeepAGREL -on classical and hard image-classification benchmarks requiring deep networks, namely MNIST, CIFAR10, and CIFAR100, cast as direct reward tasks, both for deep fully connected, convolutional and locally connected architectures. We show that for these tasks, DeepAGREL achieves an accuracy that is equal to supervised error-backpropagation, and the trial-and-error nature of such learning imposes only a very limited cost in terms of training time. Thus, our results provide new insights into how deep learning may be implemented in the brain.

Review:
Summary:
This paper generalizes the AGREL biologically plausible learning algorithm to deeper networks. It presents experiments to show that the method can scale to medium-sized tasks with only a modest slow down in training speed and essentially no change in classification performance.
Major comments:
This is an interesting paper which shows that a particular biologically plausible learning method can attain comparable performance to gradient descent on small and mid-size visual classification tasks. The proposed mapping to the biology is of a different flavor from many other recently proposed approaches. 
It is striking that the CIFAR10 network trains about as fast as the EBP network, while the CIFAR100 network trains much more slowly. This is presumably because randomly guessing the right answer out of 100 possibilities is the bottleneck. The paper could be strengthened by studying the speed of learning as a function of the number of output classes. For this approach to scale to ImageNet, with a 1000-way classification (or to our human visual recognition abilities with far more classes), this is an important scaling dimension to consider. 
It should be noted that other biologically plausible schemes like feedback alignment were able to solve CIFAR and other smaller image classification tasks, but struggled when applied to the larger scale ImageNet problem. The paper could be improved by pointing to this limitation of the present work, the possibility that performance could change on larger tasks, and the need to conduct larger experiments in future work.
Personally I think the statement at the beginning of the introduction that only RL occurs in animals and humans is overstated. Unsupervised learning occurs in some form in critical period plasticity, and in various unrewarded statistical learning paradigms, at a minimum.
I would also caution against categorical statements of biological plausibility, for instance, in saying that shared weights in convolutions are biologically implausible (bottom of pg 6). The same criticism has been leveled at gradient descent learning, and as the present submission shows, these intuitive judgements can be misleading.
The distinction between RL and supervised learning is a bit blurred in the present submission because it is considering a classification setting in which exactly one out of a number of possible outputs is rewarded on each trial. This looks very similar to the supervised learning scenario, and relies on stochastic softmax-like competition to select a single output. This approach is very reasonable for mutually exclusive classification tasks, but this is a small subset of the tasks that full EBP could be applied to. 
The paper is fairly clear but the explanation of the algorithm could be further streamlined and condensed. Is DeepAGREL equivalent to stochastically selecting a single unit in a softmax layer and then back propagating only its error? It seems so from what I understand, and this may be a straightforward way to explain the algorithm. 
Typos:
The text on page 7 describes MNIST performance as 99.17% but the table has 99.16%.