Deep learning for neuroimaging: a validation study
Authors: Sergey M. Plis, Devon R. Hjelm, Ruslan Salakhutdinov, Vince D. Calhoun
ICLR 2014

Abstract: Deep learning methods have recently enjoyed a number of successes in the tasks of classification and representation learning. These tasks are very important for brain imaging and neuroscience discovery, making the methods attractive candidates for porting to a neuroimager's toolbox. Successes are, in part, explained by a great flexibility of deep learning models. This flexibility makes the process of porting to new areas a difficult parameter optimization problem. In this work we demonstrate our results (and feasible parameter ranges) in application of deep learning methods to structural and functional brain imaging data. We also describe a novel constraint-based approach to visualizing high dimensional data. We use it to analyze the effect of parameter choices on data transformations. Our results show that deep learning methods are able to learn physiologically important representations and detect latent relations in neuroimaging data.


Peer review:
[Review]:
The paper starts by describing the success of deep learning, which has been incredible, and from the very beginning sounds like the standard paper of 'let's try a successful technique from other areas in our problems.' While interesting, it has problems to start with, not all popular techniques should be applied to all problems.

The paper presents an interesting introduction, with philosophical components, and some I either don't understand or they are not accurate.
Oracle is the theoretical limits of a problem, and if there is something we are fulling lacking is theory for deep learning (though some old one exists), so not clear what the authors mean.

Second, the authors should be aware of works on model compression that show that deep learning might actually not be needed and shallow nets can achieve almost same results, a paper on the topic even submitted to this conference.

I am not an RBM expert, only familiar with the subject, so I read it but I am not judging its correctness.

Was SM (Figure 1) ever defined? I can't find it even with my search function in the editor. This is critical.

Not critical if when comparing with ICA, you are using the last techniques developed by Smith and his team and available in the public code from Oxford (double ICA, etc). So are comparisons with the state-of-the-art?
Also, while is stated for Fig. 3 that RBM results are more supported by biology, no reference is providing to support this claim.

The multiplayer comparison in Section 3 makes no sense since parameters are significantly increasing when adding layers. The authors should look at work on model compression to see how these comparisons need to be made, and they might be surprised with the results, or maybe not, but increasing parameters by orders of magnitude is not fair.

The paper has major issues then, but the authors have done a lot of experimentation, and while under normal circumstances I would certainly recommend to reject, I think that their presence at the conference can lead to interesting discussions. The authors have not convinced me at all that their approach or deep networks in general is useful for their task, but I do believe in open discussions and I believe the community will benefit much more from the science that might come out from them defending their work at the meeting than from us rejecting outright the paper. Hopefully after the meeting we might be able to conclude if this is a direction worth investigating or not.