Venue: ICLR 2018 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: 
-I think title is misleading, as the more concise results in this paper is about linear networks I recommend adding linear in the title i.e. changing the title to … deep LINEAR networks

- Theorems 2.1, 2.2 and the observation (2) are nice!
 
- Theorem 2.2 there is no discussion about the nature of the saddle point is it strict? Does this theorem imply that the global optima can be reached from a random initialization? Regardless of if this theorem can deal with these issues, a discussion of the computational implications of this theorem is necessary.

- I’m a bit puzzled by Theorems 4.1 and 4.2 and why they are useful. Since these results do not seem to have any computational implications about training the neural nets what insights do we gain about the problem by knowing this result? Further discussion would be helpful.

review_title: This paper studies some theoretical properties of deep linear networks. Compared to state of the art this paper has fewer assumptions and much shorter and more concise proofs. This paper does build on a few previous results but still in my view has nice contributions. In summary, this is a short, neat and concise paper, making it an ideal conference submission.

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper gives sufficient and necessary conditions for the global optimality of the loss function of deep linear neural networks. The paper is an extension of Kawaguchi'16. It also provides some sufficient conditions for the non-linear cases. 

I think the main technical concerns with the paper is that the technique only applies to a linear model, and it doesn't sound the techniques are much beyond Kawaguchi'16. I am happy to see more papers on linear models, but I would expect there are more conceptual or technical ingredients in it. As far as I can see, the same technique here will fail for non-linear models for the same reason as Kawaguchi's technique. Also, I think a more interesting question might be turning the landscape results into an algorithmic result --- have an algorithm that can guarantee to converge a global minimum. This won't be trivial because the deep linear networks do have a lot of very flat saddle points and therefore it's unclear whether one can avoid those saddle points. 
review_title: review

final_score: rating: 7: Good paper, accept
review_content: Summary:
The paper gives theoretical results regarding the existence of local minima in the objective function of deep neural networks. In particular:
- in the case of deep linear networks, they characterize whether a critical point is a global optimum or a saddle point by a simple criterion. This improves over recent work by Kawaguchi who showed that each critical point is either a global minimum or a saddle point (i.e., none is a local minimum), by relaxing some hypotheses and adding a simple criterion to know in which case we are.
- in the case of nonlinear network, they provide a sufficient condition for a solution to be a global optimum, using a function space approach.

Quality:
The quality is very good. The paper is technically correct and nontrivial. All proofs are provided and easy to follow.

Clarity:
The paper is very clear. Related work is clearly cited, and the novelty of the paper well explained. The technical proofs of the paper are in appendices, making the main text very smooth.

Originality:
The originality is weak. It extends a series of recent papers correctly cited. There is some originality in the proof which differs from recent related papers.

Significance:
The result is not completely surprising, but it is significant given the lack of theory and understanding of deep learning. Although the model is not really relevant for deep networks used in practice, the main result closes a question about characterization of critical points in simplified models if neural network, which is certainly interesting for many people.
review_title: A nice but incremental theoretical paper which contributes to better understand the optimization of deep network

Meta-review: Understanding global optimality conditions for deep nets even in the restricted case of linear layers is a valuable contribution. Please add clarifications to ways in which the paper goes beyond the results of Kawaguchi'16, which was the main concern expressed by the reviewers.
Decision: Accept (Poster)