Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper follows an interesting angle on optimizing neural networks. I think the write up can be improved considerably. I'm not sure if this is not the effect of having to restrict itself to 3 pages. E.g. after reading the paper, I'm not sure I know how to implement the proposed approximate Newton method in the paper.Section 3 provides some theoretical analysis of the optimization algorithm, but is not clear that those theorems sums to an algorithm that I could code up. 

I think (and maybe it is a bit harsh) that even as a workshop submission the paper is not yet clear enough (at least the pdf) and more effort needs to be put in explaining what the different constraints in the theorems mean (and how to achieve them) and in particular how do you get the algorithm.

The other aspect that I feel uncomfortable with, regarding the approach taken by the authors, is that it seems to me that the constraint of rank(P) = T * n_L, where T is the number of examples, spells out that the network memorized the training set rather than learned. IMHO, while pursuing this quest of either proving that the error surface has no "bad" local minima, or removing them is very valuable, is only so if we can get insights of why this works in a case where you *learn* a good solution, i.e. one that generalizes. I would like if the result is somehow independent of the dataset size, and has more something to do with the underlying structure of the data and nature of the network.  
review_title: Review for smooth optimization perspective paper

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents a smooth optimization perspective on feed forward neural networks and discusses a condition where the local optima does not exist in training. Next, by studying Hessian, it develops an approximate newton algorithm and provides an experiment showing the convergence attitude on the four regions classification benchmark.

Although the approach is interesting, the paper lacks some important pieces: Theorem 1 relies on the matrix P to be full rank but it does not provide any cases or sufficient conditions when this holds. also Theorem 1 assumes the global minimum w* is reachable but does not provide any insights into when this holds. even a couple of examples would be good. I understand that this is a short version but the author could easily fit this in by reducing the introduction.


review_title: Interesting approach, needs more support

Decision: Accept