  We apply methods of topological data analysis to loss functions to gain
insights on learning of deep neural networks and their generalization
properties. We study global properties of the loss function gradient flow. We
use topological data analysis of the loss function and its Morse complex to
relate local behavior along gradient trajectories with global properties of the
loss surface. We define neural network Topological Obstructions score,
TO-score, with help of robust topological invariants, barcodes of loss
function, that quantify the badness of local minima for gradient-based
optimization. We have made several experiments for computing these invariants,
for small neural networks, and for fully connected, convolutional and
ResNet-like neural networks on different datasets: MNIST, Fashion MNIST,
CIFAR10, SVHN. Our two principal observations are as follows. Firstly, the
neural network barcode and TO-score decrease with the increase of the neural
network depth and width. Secondly, there is an intriguing connection between
the length of minima segments in the barcode and the minima generalization
error.
