Paper:
This paper demonstrates that a computational neural network model using ion channel-based conductances to transmit information can solve standard computer vision datasets at near state-of-the-art performance. Although not fully biologically accurate, this model incorporates fundamental biophysical principles underlying the control of membrane potential and the processing of information by Ohmic ion channels. The key computational step employs Conductance-Weighted Averaging (CWA) in place of the traditional affine transformation, representing a fundamentally different computational principle. Importantly, CWA based networks are self-normalizing and range-limited. We also demonstrate for the first time that a network with excitatory and inhibitory neurons and nonnegative synapse strengths can successfully solve computer vision problems. Although CWA models do not yet surpass the current state-of-the-art in deep learning, the results are competitive on CIFAR-10. There remain avenues for improving these networks, e.g., by more closely modeling ion channel function and connectivity patterns found in the brain.

Review:
This paper proposes a novel neural network architecture inspired by the analysis of a steady-state solution of the Hodgkin-Huxley model. Using a few simplifying assumptions, the authors use conventional backpropagation to train DNN- and  CNN-based models and demonstrate that their accuracies are not much lower than the state-of-the-art results.
The paper is well-written, sufficiently detailed and understandable. Derived self-normalizing Conductance-Weighted Averaging (CWA) mechanism is interesting in itself, especially contrasting CWA results with those obtained for the non-Batch-Normalized networks. It is also inspiring to see that this model can be derived based on a relatively accurate biological neuron model.
My main question is actually related to the potential impact of this study. I am curious about the implications and the ways in which these results can inspire other researchers.
After reading the paper, I got an impression that:
(a) From the point of view of a machine learning practitioner, these results may not be particularly impressive. They do hint at the importance of self-normalization though, which could potentially be interesting to explore further.
(b) From the point of view of a neuroscientist, the proposed model might be too simplistic. It is my understanding, that neural systems (even at "rest") are inherently non-equilibrium (and I assume the presence of simple feedback loops could also dramatically change the stead-state of the system). Is it possible that something similar to this "steady-state inference" mode could actually take place in real biological neural systems?
(c) Presented results appear to be important from the point of view of someone who wants to transfer insights from biology into the field of deep learning. But there might be an extent to what is achievable given a simple goal of optimizing a supervised accuracy of an artificial neural network trained using gradient descent (especially considering limitations imposed by hardware). I am optimistic about the prospect of knowledge transfer between these disciplines, but it is my feeling that the study of temporal dynamics, emergent spatiotemporal encodings, "training" process of a biological neural system, etc. have potentially much more to offer to the field of machine learning. These questions do appear to be incredibly complex though and the steady-state analysis is definitely a prerequisite.