Paper:
The learnability of different neural architectures can be characterized directly by computable measures of data complexity. In this paper, we reframe the problem of architecture selection as understanding how data determines the most expressive and generalizable architectures suited to that data, beyond inductive bias. After suggesting algebraic topology as a measure for data complexity, we show that the power of a network to express the topological complexity of a dataset in its decision boundary is a strictly limiting factor in its ability to generalize. We then provide the first empirical characterization of the topological capacity of neural networks. Our empirical analysis shows that at every level of dataset complexity, neural networks exhibit topological phase transitions and stratification. This observation allowed us to connect existing theory to empirically driven conjectures on the choice of architectures for a single hidden layer neural networks.

Review:
The authors propose to use the homology of the data as a measurement of the expressibility of a deep neural network. The paper is mostly experimental. The theoretical section (3.1) is only reciting existing theory (Bianchini et al.). Theorem 3.1 is not surprising either: it basically says spaces with different topologies differ at some parts. 
As for the experiments, the idea is tested on synthetic and real data. On synthetic data, it is shown that the number of neurons of the network is correlated with the homology it can express. On real data, the tool of persistent homology is applied. It is observed that the data in the final layer do have non-trivial signal in terms of persistent homology.
I do like the general idea of the paper. It has great potentials. However, it is much undercooked. In particular, it could be improved as follows:
* 1) the main message of the paper is unclear to me.  Results observed in the synthetic experiments seem to be a confirmation of the known results by Bianchini et al.: the Betti number a network can express is linear to the number of hidden units, h, when the input dimension n is a constant. 
To be convinced, I would like to see much stronger experimental evidence: Reporting results on a single layer network is unsettling. It is known that the network expressibility is highly related to the depth (Eldan & Shamir 2016). So what about networks with more layers? Is the stratification observation statistically significant? These experiments are possible for synthetic data. 
* 2) The usage of persistent homology is not well justified. A major part of the paper is devoted to persistent homology. It is referred to as a robust computation of the homology and is used in the real data experiments. However, persistent homology itself was not originally invented to recover the homology of a fixed space. It was intended to discover homology groups at all different scales (in terms of the function value). Even with the celebrated stability theorem (Cohen-Steiner et al. 2007) and statistical guarantees (Chazal et al. 2015), the relationship between the Vietoris-Rips filtration persistent homology and the homology of the classifier region/boundary is not well established. To make a solid statement, I suggest authors look into the following papers
Homology and robustness of level and interlevel sets
P Bendich, H Edelsbrunner, D Morozov, A Patel, Homology, Homotopy and Applications 15 (1), 51-72, 2013
Herbert Edelsbrunner, Michael Kerber: Alexander Duality for Functions: the Persistent Behavior of Land and Water and Shore. Proceedings of the 28th Annual Symposium on Computational Geometry, pp. 249-258 (SoCG 2012)
There are also existing work on how the homology of a manifold or stratified space can be recovered using its samples. They could be useful. But the settings are different: in this problem, we have samples from the positive/negative regions, rather than the classification boundary. 
Finally, the gap in concepts carries to experiments. When persistent homology of different real data are reported. It is unclear how they reflect the actually topology of the classification region/boundary. There are also significant amount of approximation due to the natural computational limitation of persistent homology. In particular, LLE and subsampling are used for the computation. These methods can significantly hurt persistent homology computation. A much more proper way is via the sparsification approach. 
SimBa: An Efficient Tool for Approximating Rips-Filtration Persistence via Simplicial Batch-Collapse
T. K. Dey, D. Shi and Y. Wang. Euro. Symp. Algorithms (ESA) 2016, 35:1--35:16
* 3) Finally, to support the main thesis, it is crucial to show that the topological measure is revealing information existing ones do not. Some baseline methods such as other geometric information (e.g., volume and curvature) are quite necessary.
* 4) Important papers about persistent homology in learning could be cited:
Using persistent homology in deep convolutional neural network:
Deep Learning with Topological Signatures
C. Hofer, R. Kwitt, M. Niethammer and A. Uhl, NIPS 2017
Using persistent homology as kernels:
Sliced Wasserstein Kernel for Persistence Diagrams
Mathieu Carrière, Marco Cuturi, Steve Oudot, ICML 2017.
* 5) Minor comments:
Small typos here and there: y axis label of Fig 5, conclusion section.