Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: Summary:
The paper proposes to monitor neural networks during training by considering a momentary state of a network (MLP or CNN) as an abstract simplicial complex as defined by considering the network as a weighted directed graph. Then persistent homology features are computed resulting in persistence diagrams which are vectorized to compute distances between subsequent network states during training. The paper proposes to monitor this measure they call homological convergence to identify training stages where generalization error is low -- without the need to estimate performance by means of a validation set.
main_review: Major points:
-I find it relevant and interesting to assess generalization capabilities of neural networks by means of TDA without requiring a validation set, and I am curious to see where this line of research is developing.
- The paper is clearly written, and the non-familiar reader also receives a clear introduction to TDA.
- My biggest concern is that the main finding, i.e., that PH of the network architecture allows to monitor performance without a validation set, is not novel. For instance, in Reference [26], Rieck et al employed PH of the network architecture (albeit using a weighted undirected graph) to distinguish good from poor performing networks and even proposed an early stopping scheme with a PH-based patience approach for guiding training and monitoring of neural nets without a validation set, amongst others also using MNIST and CIFAR-10. That being said, even though the empirical observation that PH of the network correlates with generalization has been known before, to my knowledge it is still an open question *why* this is the case, i.e., the theoretical workup of this phenomenon is still pending and would be quite relevant to have addressed - which I encourage the authors of this paper to investigate, as this would considerably strengthen the paper.

One way I could imagine that potential explanations of this phenomen could be identified could be via dedicated comparison studies:
e.g. do simple statistical moments of the weight distribution also correlate with generalisation, or could there be a way to identify relevant cliques in the network which are topologically relevant and also coinciding with a "lottery winning" subnetwork, etc?

- The experiments could be strenghened by the addition of certain monitoring baselines to showcase the gained value of employing PH here.
    For instance, in [26] the PH-based measure was compared to patience with validation loss (what is the difference in performance and epochs after when early stopping was initiated). Also, it would be interesting to add simple baselines measures to check that they are not already helpful alone (like non-topological information about the weight distribution).


Minor points:
- Figure 6: it took me some time to understand the legend / caption. I would suggest to repeat "# layers" in the legend, such that is easier to understand the legend without reading the caption first.
- Line 294, typo: "We posed the of question whether"
limitations_and_societal_impact: Computation scalability was considered the main limitation in the paper. Otherwise, I could not find any discussion of potentially negative societal impact. For instance, when significant runtime due to scalability issues are in the focus, the paper could mention that this could affect CO2 expenditures of models employing these techniques.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes a topological data analysis approach to investigate the inner workings of a deep network. In particular, the generalization performance of a deep network is measured without a holdout set by analyzing the topological change of the network during training. Multiple experiments conducted over various datasets demonstrate that the method performs reasonably for various learning rates, batch sizes, and dropout probabilities. 
main_review: PROS
The approach is really sensible. In fact, I strongly believe that the topology of training dynamics can reveal why and how a neural network generalizes. In fact, in the future, by encouraging certain topological properties during training, it may be possible discover networks that generalize better. The paper delivers a clear message and is easy to understand.

CONS
- I am not entirely convinced by the experimental evaluation given in this paper. On one hand, because of the convergence properties of the (stochastic) gradient descent algorithm,  the magnitude of the 'updates' to the weights of a neural network will decrease as training progresses. On the other hand, any successful training is expected to decrease the validation error (stochastically). Hence, it is not surprising that these quantities correlate - even if not causally related. How would this be an indicator of the generalization performance? Is there any theoretical line of thought which would justify this? From an empirical standpoint, I suggest that the authors compare different architectures. Can the PD distance identify which of these networks would generalize better? 

- The proposed method seems to be agnostic to the construction of the graph. One could also construct the graph as in Corneanu [10], or potentially in many other different ways. Why do we pick the proposed method of graph construction? The impracticality claim for [10] also does not seem to be justified in this paper. Unfortunately, [2], which shares the exact similar graph construction, does not explain this either. 

- This paper seems to contain extreme resemblance to [2]. If [2] (attached in supp.) is also a submission, I would doubt the contributions of this paper. I invite the area chairs to look into this issue.

- Maybe we can have the definition of Heat vectorization in the paper?

- In general it would be good to improve the captions of the figures to be more descriptive. 

- Why are silhouette and heat distances the same in Table 1?
limitations_and_societal_impact: I do not see issues related to the societal impact of this work. However, this paper is not applicable to CNNs, which are crucial for modern deep learning. This could be added to the limitations section of the paper.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes a method to evaluate the generalization of a treined neural network using persistent homology. They measure the distance between Persistent Diagrams corresponding to Neural Networks in the training process, and show that the accumulation correlates with the Validation Curve. They also discuss which method(difinition) is appropriate for the distance between PDs.
main_review: In training NNs, the capture of generalization is very important, so we often divide the data into training set and validation set. On the other hand, there are some problems with this method, such as the dependency on the partitioning method and the small amount of training data. From these perspectives, it would be a great advantage if generality could be evaluated without a validation set. This paper represents the NN of the learning process as a weighted graph and evaluates generality by calculating persistent homology. Evaluating the network using persistent homology is a very interesting approach because it may allow us to ignore the randomness of the order of the neuron layers. On the other hand, there are some concerns about the evaluation method of Generality:
1. Since the score of the proposed method is obtained only from the training data, we cannot evaluate whether the method can be used for data whose domain is different from the training data. I think it is used to make sure that the model does not overfit the training data, but can the detection of overfit be proven by simply checking the correlation with the validation curve? In particular, it is more difficult to understand because of the normalization. I think you need more discussion.
2. In the proposed method, each score is obtained from successive NNs in the training process. Intuitively, I think this captures the fact that NNs change less with the trainning process. Since the training curve and the variation of NN are strongly correlated, isn't the proposed method looking at the convergence of the training curve? If the training set and validation set are from the same domain and there is no model overfit, the training curve and validation curve will be correlated. You may want to make sure that your experiment is not just correlating with the training curve.
3. For practical use, the proposed score has a different range of values depending on the problem. If the value of the score changes every time, it may be difficult for users to judge the generality of the score by looking at the value.

Another comment:
For example, it would be more interesting to discuss how to measure the stability of NNs based on the variation of each batch in each epoch.

Minor comments:
- The figure appears to have a solid line and a dotted line, but this is very unclear and we suggest you make it clearer.
- I think you should write what W,w is in definition of normalized weight(3) to make it easier to read.
limitations_and_societal_impact: I do not believe that the content of this paper will have any negative social impact. The ability to evaluate generarization without validation set is useful for practitioners, considering the situation where it is practically difficult to collect data. Due to the above concerns, it is still unclear whether it can be used in practice.
review_title: null

final_score: rating: 3: Clear rejection
review_content: summary: This paper analyses the training of neural networks from a topological
perspective, presenting a pipeline that can measure (pseudo) distances
between the network's weights during training. Such information is then
employed to study the generalisation error of a neural network.

In contrast to existing methods for estimating this error, this paper
does *not* require a specific hold-out data set, as topological features
of the neural network are monitored during training. This frees up
additional data for fitting, which can be highly relevant in the sparse
data regime.

main_review: ## Summary of the review

I enjoyed the ideas presented in this paper; the analysis of
generalisation performance is a highly relevant and timely.
Understanding generalisation without requiring additional data has the
potential to improve machine learning techniques to a significant
extent.

That being said, the current paper suffers from some issues, which
prevent my endorsement at this point:

1. Clarity: while background information on topological data analysis is
   provided (which I appreciate; in particular in light of the fact that
   TDA is still a rather novel occurrence at ML conferences), the method
   itself could be described in more detail. In particular, some aspects
   should be discussed in Section 4:

    - How does the calculation work in practice? The paper only provides
      a description of simplicial homology in the undirected setting but
      Section 4 appears to discuss directed graphs as well. I understand
      that the directed graph is turned into an undirected one, but this
      needs to be clarified.

    - Would it be possible to use a directed flag complex or a Dokwer
      complex here instead? Maybe this is more of an expert discussion
      topic, but if directionality matters, that could be a good option.

    - The use of additional representations for topological features
      needs to be clarified. At present, the main paper does not contain
      any description of the challenges in calculating distances or the
      need for additional representations. At the very least, a brief
      description of these topological representations is required, so
      that readers may better understand the remainder of the paper.
      I find it particularly problematic that no stability guarantees of
      these representations are discussed; they are not *just* drop-in
      replacements for the bottleneck or Wasserstein distances.

    - Overall, this section should be rewritten with a clear 'roadmap'
      in mind. What is the problem you want to tackle and how do you
      intend to tackle it in this paper?

2. Delineation to existing work: The paper by Rieck et al. [26] appears
   to already contain a large amount of the material proposed in this
   paper. A cursory reading shows that Rieck et al. even discuss
   generalisation performance in an early stopping setting (more about
   this later). It is therefore critical that the contributions of this
   paper are more clearly delineated from existing work. From my
   understanding of both papers, I would say that the current submission
   improves on the following aspects:

    - Choice of filtration for such neural networks

    - The use of other topological representations (whereas Rieck et al.
      only use a summary statistic of persistence diagrams).

  These differences (and potentially all others that I missed) should be
  briefly discussed; the advantage would be that the paper can refer to
  the previous publication as a justification of the method itself!

2. Experimental depth: the experimental section is lacking depth, in
   particular given the specific goals of this paper. The depicted plots
   and correlations are useful, but don't serve to highlight the
   benefits of the proposed method. I would therefore suggest a more
   thorough experimental setup such as the one shown in the supplement
   of Rieck et al. [26]: here, their proposed measure of network
   complexity is shown as additional early stopping criterion (without an
   hold-out validation data set as well).

   A post-hoc experiment of this sort would be extremely useful in
   demonstrating the utility of the method, and it would even enable the
   quantification of certain measures. The correlation is also useful in
   this context, but since the goal is to assess the generalisation
   error, an experiment in which the new measure is directly applied
   would be extremely worthwhile.

Please see below for detailed comments.

## Detailed comments

- The first contribution reads a little bit like a sentence fragment to
  me. I would suggest to use the terms 'silhouette distance' and 'heat
  kernel distance' instead of plain 'silhouette' and 'heat'.

- The terminology 'homological convergence' is slightly misleading
  because the paper studies persistent homology, as far as I understand.

- The introduction of topological concepts in the background section
  jumps between the 'geometrical' and the 'abstract'; I understand that
  both perspectives are valid, but I would suggest to choose only
  a single one.

- 'complex clique' --> 'clique complex'

- The discussion of chains and boundaries requires more explanations (or
  could be partially relegated to the supplementary materials). For
  instance, what is as 'signed combination' in this context? If
  I understand it correctly, the paper uses the usual $\mathbb{Z}_2$
  coefficients, so there are no signs?

- I would suggest not to use $\mathbb{B}$ and $\mathbb{Z}$ to denote the
  boundary and cycle subgroups, respectively, as the later one can be
  easily confused with the set of integers or certain characteristic-$p$
  fields. The more common terminology seems to be to just use $B$ and
  $Z$ here.

- There is a distinction between the 'homology' and the 'homology
  group'. The way the paper defines it, I assume that 'homology group'
  is meant in l. 87.

- The definition of homology group needs some refinement. The sentence
  'is the quotient space of $\mathbb{B}$ a subspace of $\mathbb{Z}$' is
  confusing to me. Please rewrite this more precisely.

- The introduction of filtrations is not motivated; the 'nested family
  of simplicial complexes' could be introduced by building more
  intuition here.

- The related work discussion should, as mentioned above, delineate this
  paper from Rieck et al. [26].

- The heading 'Algebraic Topology object' is rather confusing to me;
  I would suggest to rewrite it as 'directed flag complex', for
  instance.

- As outlined above, the distance calculation lacks information about
  stability and other guarantees.

- In the experimental section, the use of the MLP should be explained in
  more detail. What does it do and how does it work exactly?

- I don't understand the 'input order' experiment; does it refer to
  ordering of samples (i.e. on the batch level)?

- The use of additional topological descriptors makes the use of the
  word 'distance' somewhat imprecise: the bottleneck and Wasserstein
  distances are metrics in the mathematical sense; while there *is*
  a well-defined metric between, for instance, persistence landscapes or
  silhouettes, this is only (to a certain extent) an approximation to
  the bottleneck/Wasserstein metrics. This distinction should be made
  clearer in the paper.

- Figure 6 should add standard deviations of the curves. An additional
  clarification of the content of this figure would also be appreciated
  by readers; I found the discussion to be slightly confusing because
  I did not get the relevance of the cumulative distance. The statement
  'the evolution of the homological convergence [...] seems to be very
  similar to the one of the validation score' also needs some
  clarification. Except for Figure 6b, where I observe some oscillation
  behaviour and maybe the tendency to converge, I don't observe
  convergence anywhere else.

- The same applies to Figures 7–9. Adding some more details here will
  result in a clearer description of the method. Maybe some of the
  curves could also be relegated to the supplementary materials?

- I would also suggest to investigate the use of energy distances or
  energy correlations, as these measures are more flexible and capable
  of assessing more than just linear dependencies between two
  variables. See [*Energy statistics: A class of statistics based on distances*](https://www.sciencedirect.com/science/article/abs/pii/S0378375813000633)
  for more details.

## Terminology and style

For the most part, the paper is written well. I have some suggestions
concerning the style.

- Please use `sort` as a `natbib` option in order to ensure that
  citations are sorted; this makes the manuscript easier to read.

- 'contained simplicial complex' --> 'nested simplicial complexes'

- can't --> cannot

- The paper employs non-standard terminology in certain places. The
  diagram, for instance, is called 'persistence diagram', no
  'persistence homology diagram'. Please ensure consistency with
  existing papers here. The same applies to capitalisation; I personally
  see no need to write 'Persistence Diagram', but if this spelling is
  used, it should be used consistently.

- 'non-cumulative homology': I think the word 'distances' is missing
  here.

- The use of 'Means mean' and 'Deviations mean' is slightly confusing;
  I would suggest to show follow the format $\mu \pm \sigma$, with $\mu$
  being the mean and $\sigma$ being some measure of variance, such as
  the standard deviation.

- For the references list, I would suggest to carefully check which
  version of a paper is being cited. Paper #24, for instance, has
  already been published as an ICML paper.
limitations_and_societal_impact: Limitations of the method could be more prominently discussed. As in
previous work, the current method does not work well with convolutional
architectures; an additional subsection on limitations would therefore
be appreciated, in particular by readers that are not yet familiar with
TDA. This could be integrated into Section 7, which already contains
a discussion of the computational requirements.

Being mostly a theoretical analysis of generalisation performance, no
negative societal impact is bound to arise specifically from this work.
review_title: null

Meta-review: The paper proposes a novel method for evaluating the generalization of NN using persistent homology.   While the paper introduces an interesting and novel idea, there are some weak points.  The empirical validation of the proposed idea is not strong enough. As the method is not supported by theory, more careful empirical justification is needed.  Also, there are previous works that have some overlap with this submission in the direction, such as Rieck et al. [26].  More comparison is necessary to highlight the advantage of the proposed method. 
 Based on these observations, we need to judge the current paper is unfortunately not above the acceptance threshold.   However, we encourage the authors to improve the work and submit it to another conference or journal.  
Decision: Reject