  In this paper, we analyze the landscape of the true loss of a ReLU neural
network with one hidden layer. We provide a complete classification of the
critical points in the case where the target function is affine. In particular,
we prove that local minima and saddle points have to be of a special form and
show that there are no local maxima. Our approach is of a combinatorial nature
and builds on a careful analysis of the different types of hidden neurons that
can occur in a ReLU neural network.
