Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The basic premise of this work is that topological operators are increasingly adopted in machine learning models, but the literature lacks of general differentiable topological operators.

The main idea proposed in this work is to use the topological signature directly as a loss for autoencoders. The general aim is to preserve the topological properties of data while performing dimensionality reduction. 
 
The proposed method computes the persistent homology of the 0-dimensional Vietoris–Rips by means of persistence diagrams and persistence pairing. The additional step is to computes the regularization loss, by posing the constraint that the persistent homology be similar between data space and latent space. The empirical analysis investigates the stability property with respect to the mini-batch sampling. 

The extended experiments don't provide an empirical evidence of the added value of a topological AE. The results are quite controversial if we focus our attention on MSE measures. AE and TopoAE behave in a similar way and it is not clear whether such a difference is statistically significant. 

The work is missing a mandatory comparison with Hofer et al. (2019b) that proposes a loss very similar to the one presented in Section 3, even though the authors remark that their formulation is more general. For example an interesting comparison could have been with the real-case application of classification for the images datasets (e.g. CIFAR, MNIST, ImageNet) where Hofer et al. (2019b) proves their claims.

The computational issues of the d-dimensional Vietoris-Rips with d > 0, limits the preservation of only simple structures in the point-cloud.

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: This paper shows how to train an autoencoder that preserves topologically
relevant distance across varying length scales.  It presents a new
differentiable loss term for topological distance between input and latent
space.  The topological loss has a number of nice features: compatibility with
minibatch training, and extension to higher input and latent space
dimensionality fairly easily.

Since this trainable topological loss can be applied in more general scenarios
than simply auto-encoders or visualization, I think this well-written article
is of general interest and worth publishing.

The main ideas and related work are presented clearly in sections 1--4, and the
experiments compare TopoAE with a variety of low-dimensional visualization
methods.  Embedding quality is evaluated with a wide variety of metrics on real
and synthetic datasets highlighting the preservation of global and local
topology.

My only real issue, easily fixable, was that after the minibatch stability
theory (Sec. 3.3), I really wanted to know what minibatch sizes were used in
the experimental figures, and spent twenty minutes before admitting defeat.
Please include the actual TopoAE minibatch size when presenting the
visualization figures in the article and appendices.

The appendices contain a wealth of useful reading and experiments, and the
provided source code was clearly organized and useful to browse.  I did not go
line-by-line through the appendix proofs.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: Summary
-------------

The paper proposes an approach, based on persistent homology (PH), to preserve certain topological structures of the input in the latent representations learned by an autoencoder. This is realized via an additional (i.e., in addition to reconstruction) loss term (optimized over mini-batches) which requires differentiating through the PH computation. While this has been done before (e.g., Chen et al., Hofer et al.), the authors have certainly put an interesting spin on this. The theoretical part of the work deals with the issue of using mini-batches for PH computation and whether this computation is close to the computation on the full point cloud. Experiments and comparisons on multiple datasets are presented to demonstrate that the approach, e.g., preserves nesting relationships in the input. The paper is nicely written and the content is very well presented. There are questions here and there (see below), but I do think they can be answered.

Major comments/remarks:
-------------------------------------

My first question relates to the issue that the loss only incorporates 0-dim. information. The authors do remark that higher-dim. features can be included, but the results were similar. However, after thinking about this issue quite some time, I am curious if it is possible to obtain "zero" of the topological loss (so this term is perfectly optimized), but the encoder introduces, e.g., cavities in the data which were not present in the input (e.g., 1-dim. holes).

Also, can you show formally (maybe this is trivial and I am not seeing it) that L_t = 0 would lead to 0 distance between the corresponding diagrams w.r.t. some common metric? A more formal treatment of the implications of the loss in Eq. (2) would certainly help.

Another question that immediately comes to mind is whether the computation of VR PH in the input space (e.g., CIFAR 10) makes sense, as the authors rely on ||.||_2 if I understood this correctly. I would argue that the topology of the input is basically unknown, especially for images and computing Euclidean distances among images, or vectorized images, does not make sense. For the nice results on the SPHERES data set it does, as the spheres are defined exactly using ||.||_2. If the VR PH in 0-dim. of the input is enforced upon the representations in the AE bottleneck, but the input topology is not captured well, then you might be enforcing something that you possibly do not want.

Apart from that, it is known that the Euclidean distance degenerates quickly in high dimensional spaces, e.g.,

Aggrawal et al.
On the Surprising Behavior of Distance Metrics in High Dimensional Space

Maybe this is also contributing to the fuzzy visualization of CIFAR-10 in Fig. 3 (apart from the low-dim. of the bottleneck)?

Also, maybe the authors could work out (in greater detail) the differences between their results from Thm.1/2 and the results of Chazal et al., in "Subsampling Methods for Persistent Homology". In my point of view, the results in the paper only hold if you would consider just a single batch, right? I mean, if the loss is computed from the batch, and a gradient update is performend, Z^{m} will changes (as the encoder changes as a result of the update), while the input does not. 

Finally, how were the KL divergence measures in Table 1 computed, as you need a density estimate of the input as well, not just for the representation space, right? Is this not a very crucial issue in the input space? If so, how reliable are the numbers presented for KL_{0.01},etc., given that the differences are sometimes extremely small.

Minor comments
-----------------------

Sec. 6: We presented a topological autoencoders -> We presented a topological autoencoder

Overall, I think this is a nicely done paper, but with quite some question marks at many places. I do think this is always the case for something new, though, and 
actually a good thing.
review_title: Official Blind Review #1

Meta-review: This paper introduces a new variant of autoencoders with an topological loss term.

The reviewers appreciated part of the paper and it is borderline. However, there are enough reservations to argue for it will be better for the paper to updated and submitted to next conference.

Rejection is recommended.  
Decision: Reject