Task: Language Identification
Input (Spanish, English): Strengths  - interesting to explore the connection between ReLU DNN and simplified SFNN - small task (MNIST)  is used to demonstrate the usefulness of the proposed training methods experimentally - the proposed, multi-stage training methods are simple to implement (despite lacking theoretical rigor)   Weaknesses  -no results are reported on real tasks with large training set  -not clear exploration on the scalability of the learning methods when training data becomes larger  -when the hidden layers become stochastic, the model shares uncertainty representation with deep Bayes networks or deep generative models (Deep Discriminative and Generative Models for Pattern Recognition , book chapter in “Pattern Recognition and Computer Vision”, November 2015, Download PDF). Such connections should be discussed, especially wrt the use of uncertainty representation to benefit pattern recognition (i.e. supervised learning via Bayes rule) and to benefit the use of domain knowledge such as “explaining away”.  -would like to see connections with variational autoencoder models and training, which is also stochastic with hidden layers
Output (English): en