Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: Thanks for the very interesting paper!

I would like to ask for a clarification of the experimental procedure described in Sec. 4.2. You mention splitting the data to 5 folds: 3 for training, 1 for testing and 1 for validation. Can you please explain how these are used, especially what is the difference between testing and validation folds?
review_title: Clarification on the experimental procedure

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The problem addressed here is practically important (supervised learning with n<<d), and as far as I know, the approach is novel. I thought their proposed solution was innovative, and I enjoyed the paper. The presentation is clear and it has nice experiments. 

Comments/questions:
1) What is the dimensionality of the feature embeddings? 
2) SNPtoVec still requires training a very fat autoencoder network on X --- I suppose this doesn't contribute to the size of the final run-time model and overfitting is avoided because the parameters are fit in an unsupervised manner?


review_title: Nice paper; very clear presentation.

final_score: rating: 7: Good paper, accept
review_content: The paper addresses the important problem (d>>n) in deep learning. 
The proposed approach, based on lower-dimensional feature embeddings, is reasonable and makes applying deep learning methods to data with large d possible.
The paper is well written and the results show improvements over reasonable baselines.

review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents an application of deep learning to genomic SNP data
with a comparison of possible approaches for dealing with the very
high data dimensionality. The approach looks very interesting but the
experiments are too limited to draw firm conclusions about the
strengths of different approaches. The presentation would benefit from
more precise math.


Quality:

The basic idea of the paper is interesting and the applied deep
learning methodology appears reasonable. The experimental evaluation
is rather weak as it only covers a single data set and a very limited
number of cross validation folds. Given the significant variation in
the performances of all the methods, it seems the differences between
the better-performing methods are probably not statistically
significant. More comprehensive empirical validation could clearly
strengthen the paper.


Clarity:

The writing is generally good both in terms of the biology and ML, but
more mathematical rigour would make it easier to understand precisely
what was done. The different architectures are explained on an
intuitive level and might benefit from a clear mathematical
definition. I was ultimately left unsure of what the "raw end2end"
model is - given so few parameters it cannot work on raw 300k
dimensional input but I could not figure out what kind of embedding
was used.

The results in Fig. 3 might be clearer if scaled so that maximum for
each class is 1 to avoid confounding from different numbers of
subjects in different classes. In the text, please use the standard
italics math font for all symbols such as N, N_d, ...


Originality:

The application and the approach appear quite novel.


Significance:

There is clearly strong interest for deep learning in the genomics
area and the paper seeks to address some of the major bottlenecks
here. It is too early to tell whether the specific techniques proposed
in the paper will be the ultimate solution, but at the very least the
paper provides interesting new ideas for others to work on.


Other comments:

I think releasing the code as promised would be a must.

review_title: Interesting but somewhat early work on deep learning with very high dimensional genomic data

final_score: rating: null
review_content: It would be informative if the authors could include the performance of the model consisting of PCA preprocessor + MLP classifier in the comparison, especially as PCA is closely related to auto-encoding / reconstruction.
review_title: Comparison to PCA + MLP

Meta-review: The reviewers agreed that the paper proposed a solid and original contribution that was evaluated well. The authors would be encouraged to improve the presentation and provide a more precise mathematical presentation, as discussed by reviewer 1.
Decision: Accept (Poster)