Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes two measures that encapsulate the topological features of data: Persistent Intrinsic Dimension (PID) and Euclidicity. The authors claim that manifold learning with a fixed intrinsic dimension can be a restrictive assumption in general and it is worth thinking of allowing this dimension to vary for different points on the manifold. The seeming advantages are argued to be the ability to identify singularities (i.e. points that violate the manifold assumption) as well as providing a notion of local geometric complexity in the data. 

The authors spend considerable effort in theoretically elaborating their approach and providing theorems to show that their persistent homology-based method is stable to choice of parameters and sampling issues in the data. Synthetic demonstrations are made on the pinched torus, the wedges spheres as well as image datasets like MNIST and FMNIST. 

strength_and_weaknesses: Strengths 

- To the best of my non-expert understanding the formulation appears to be novel. I also did find the arguments highlighting the scenarios of “singularities” in the data to be interesting 
- Most experiments do demonstrate the characteristics of the claimed contributions, i.e. the measures PID and Euclidicity provide an interesting way of measuring the complexity of data, with singularities showing up nicely (especially in Figure 1 and 5). 
- The appendix demonstrates some interesting experiments, especially Figures 8 and 10, where a comparison is shown between some well known dimensionality estimators and the proposed approach. 

Weaknesses and Questions

- Despite the focused approach to handling singularities in data, I do feel the overall applicability of this method in practice is not well explored. Most experiments are very synthetic and tailor-made for the setting of this paper (eg. pinched torus, wedged spheres etc.) 
- The structure of the paper can be made much better. I feel a lot of the important practical results are left in the appendix. For example sections, A5, A6, and A7 should (with appropriate editing) belong to the main paper and must be argued as a core part of the contribution. 
- Unfortunately, I did not find the material to be satisfactorily self-contained, and I think that it would go a long way to make sections 2 and 4 more accessible to a general representation learning audience.   
- (suggestion) Consider improving the quality of visualizations, in terms of choosing the view in 3D (eg Figure 10, to see the point of contact between the circle and sphere).  
- What is the limit on line 111 ?

clarity,_quality,_novelty_and_reproducibility: Poor clarity, OK in quality and novelty, and reproducibility.
summary_of_the_review: Overall, I think this paper proposes an interesting contribution in topological data analysis. However, I am leaning negative considering two factors (1.) The overall confusing structure of the paper and inaccessibility of the core TDA components (2.) Very little emphasis on actual applications with significant impact.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper provides a method for detecting singularities in the data distribution in terms of the manifold hypothesis. The validity of the method is mathematically proven. Finally, it is applied to artificial data and real data to verify its effectiveness.
strength_and_weaknesses: The issues and methods proposed are fine and appear to be theoretically sound. On the other hand, the concern is whether it is appropriate in terms of AI papers. Validation with real-world data has also been done, but it observes the relationship between each data class and indicator, which is different from the original goal of detecting singularities. Therefore, we do not know if it can be used for general AI tasks such as building AI models.For example, monitoring changes in the proposed index against data interpolation (transitions between two images) performed in the generative model, and comparing it to the classifier's confidency. Other considerations might include noise and adversarial attack detection.

Minor point, but the formatting seems incorrect. I don't know why the line numbers are attached, but they should be removed.
clarity,_quality,_novelty_and_reproducibility: The proposal is a new point of view and has a high degree of novelty and originality. The content also seems accurate, but more practical verification and suggestions for usage are needed to make it appropriate for the scope.
summary_of_the_review: The problem setting is interesting and important, and the report presents a theoretical solution to the issue. On the other hand, its practical use is unclear and I question whether it is appropriate for the scope of the conference. While I believe it is still within the scope at this point, I think it should be in a form that does not raise concerns.

**Conclusion following discussion with the authors**

I believe that our biggest concern, contribution to the AI field, has been resolved by clearly showing that the manifold hypothesis detects situations in which it does not work. Since the specific application of the method remains to be determined, and since there was no score of 7 in the peer review process, the score will be stayed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a technique based on topological data analysis to: 1. estimate pointwise the intrinsic dimensionality of the data manifold in the local neighbourhood, and 2. assess if the local neighbourhood resembles a Euclidean space with the same dimensionality. In this way, singularities on the data manifold can be found. The method has a multi-scale property as it takes into account neighbourhoods of different size. In the experiments it is demonstrated the effectiveness of the approach.
strength_and_weaknesses: I think that the problem the paper aims to solve is very interesting and useful for modern machine learning, and it has been motivated well. I am not an expert in the field of topological data analysis, but I believe that the proposed approach differs from the previous related works which are properly presented. The technical content seems to be reasonable and to support the claims, however, I have not checked it thoroughly. 
clarity,_quality,_novelty_and_reproducibility: The proposed approach constitutes a significant contribution in the field, as long as it performs as expected. As regards the clarity, in general, I find papers related to topology a bit hard to access due to the terminology that is used and the background knowledge that I assumed. I think that the authors tried to give the high level idea in several places, but I believe that a suitable description (not too technical and not too abstract) of the approach is probably missing.
summary_of_the_review: Questions:
1. I think that a brief summary of the approach is missing, while in several places (e.g. "Our contributions") you summarize from a high level point of view the steps of the approach. Perhaps, you could explain in a non-technical way each step e.g. what we need to do intuitively to estimate the intrinsic dimension of the local neighbourhood around a point.
2. I believe that you could save some space e.g. by moving proofs in the appendix, such that to include some additional figures to demonstrate the steps of the approach.
3. In high dimensions a lot of data are necessary in order to approximate the data manifold. I appreciate the synthetic and the MNIST/FashionMNIST experiments, but I think that some additional real world data in high dimensions should be conducted.

I think this is an interesting contribution with potential merits, buy as a non-expert in the field I cannot criticize objectively the actual contribution of the method. However, I believe that there is some room for improvement as regards the clarity and the accessibility.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:
This paper suggests two measures to quantify the topological properties of the data manifold: Persistent intrinsic dimension and Euclidicity.

Strengths:
- The formulations are novel and appear to nicely quantify singularities.

Weaknesses:
- The applicability of the metrics remain unclear, as the experiments are mostly synthetic and proof of concept in nature. 
- In particular, the relevance for AI tasks was not clear to the reviewers.
- The paper was found a bit hard to read, and not be self contained.

In short, the reviewers appreciate the novel approach to detecting singularities in data. However, if the authors seek to publish at a mainstream machine learning conference, I recommend that even more care is taken to make the paper widely accessible and highlight the potential impact for mainstream AI to benefit from their work.
justification_for_why_not_higher_score: While two reviewers recommend weak accept, one of them has low confidence while the other questions the relevance for AI.
Decision: Reject