Definition: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Input: We study the connection between the highly<lb>non-convex loss function of a simple model of<lb>the fully-connected feed-forward neural net-<lb>work and the Hamiltonian of the spherical<lb>spin-glass model under the assumptions of:<lb>i) variable independence, ii) redundancy in<lb>network parametrization, and iii) uniformity.<lb>These assumptions enable us to explain the<lb>complexity of the fully decoupled neural net-<lb>work through the prism of the results from<lb>the random matrix theory. We show that for<lb>large-size decoupled networks the lowest crit-<lb>ical values of the random loss function are<lb>located in a well-defined narrow band lower-<lb>bounded by the global minimum. Further-<lb>more, they form a layered structure. We<lb>show that the number of local minima out-<lb>side the narrow band diminishes exponen-<lb>tially with the size of the network. We em-<lb>pirically demonstrate that the mathemati-<lb>cal model exhibits similar behavior as the<lb>computer simulations, despite the presence<lb>of high dependencies in real networks. We<lb>conjecture that both simulated annealing and<lb>SGD converge to the band containing the<lb>largest number of critical points, and that<lb>all critical points found there are local min-<lb>ima and correspond to the same high learn-<lb>ing quality measured by the test error. This<lb>emphasizes a major difference between large-<lb>and small-size networks where for the lat-<lb>ter poor quality local minima have non-zero<lb>probability of being recovered. Simultane-<lb>ously we prove that recovering the global<lb>minimum becomes harder as the network size<lb>increases and that it is in practice irrelevant<lb>as global minimum often leads to overfitting.<lb>
Output:
The Loss Surface of Multilayer Networks