Question: Why do the authors choose an asymptotic regime to focus on?  My understanding is that the primary reason is easier theoretical tractability. It would help the reader to know why the paper focuses on the asymptotic setting.
Evidence: Real-world data, including images, videos, genetic expressions, natural languages, or financial statistics, usually have a high dimensionality. The intrinsic dimensionality of these data, however, is typically much lower than its ambient dimension, which property is recognized as an important underlying reason for modern machine learning to work. To capture useful information from high-dimensional data, dimensionality reduction (DR) is of both theoretical and practical value.
DR is essential for data visualization. A space with a dimensionality higher than 3, however, is already beyond our accustomed way of observing data, and our intuition in 2-dimensional or 3-dimensional space may not apply. High-dimensional space is not a trivial extension of lowdimensional space; theoretical research on high-dimensional geometry and statistics revealed a number of intriguing, non-intuitive phenomena in high-dimensional space. Imagine a hyper-sphere with a radius r in a d-dimensional Euclidean space whose central point is at the origin. Consider a "crust" of the d-dimensional hyper-sphere, which is between the surfaces of this hypersphere and a slightly smaller concentric hyper-sphere with radius (1−ϵ)r, where ϵ is small (Figure a). The ratio of the volume of the "crust" C d (r) to the hyper-sphere is V d (r) is C d (r)
V d (r) = 1 − (1 − ϵ) d . Take ϵ = 0.01, it is easy to show that when d is small, the ratio is tiny (as our intuition goes), however this ratio grows exponentially fast to near 100% with the increase of dimensionality, as illustrated in figure 1 a-b. The volume of a high-dimensional hyper-sphere is therefore counter-intuitively concentrated on a crust (1 c). Such concentration explains the "crowding problem" of DR (van der: a faithful preservation of distances in high-dimensional space would lead to crowded data points. t-distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) are among the most established DR methods today, both of which mitigate this crowding problem by modeling the low-dimensional similarity measure with a longer-tail distribution than high-dimensional similarity measure (e.g. t distribution vs. Gaussian in the case of t-SNE). Such a disparity in similarity measure amounts to a distortion of distance between two spaces, but in a highly implicit manner. Both methods perform well empirically, but the underlying distortion of distance measure in two spaces cannot be analytically expressed or validated. Furthermore, the implicity of distance transformation deters us from imposing priors in data if there is any (e.g. a Swiss Roll has an intrinsic dimensionality of 2).
Another important property of real-world high-dimensional data is that they often exhibit a hierarchical structure (sub-manifolds on large manifolds), governed by the underlying generative models. Such hierarchy demands different treatment of data points by DR at different relative position. The state-of-the-art DR methods, such as UMAP and Barnes-Hut t-SNE (van der Maaten, 2014), commonly consider on a selected neighborhood (as a hyper-parameter) while discarding the far field beyond it. Isomap, on the other hand, taking both near and far field into account by calculating distances on a connected graph. t-SNE and UMAP works well on data of disjoint manifolds such as MNIST, while Isomap works well on data of continuous manifold such as Swiss roll. However, it is generally difficult for one method to succeed in both disjoint and continuous data manifolds.
In this paper, we seek to develop a novel DR method that is based on the following two key ideas, and strive to address the two aforementioned issues:
• Space Expansion: Matching the "capacity" in high-dimensional and low-dimensional space, by explicit, quantitative transformation of distance measure;
• Hierarchical Manifold Approximation: Data-specific, hierarchical modeling of similarities in high-dimensional data, to accommodate both disjoint and continuous data manifolds.
In the setting of generally anisotropic features, this paper gives an asymptotic characterization of the generalization risk curve with more samples. The asymptotic regime is an approximation for large n, d and can also shed light on practical machine learning problems. We first introduce our problem setup.
Answer: Yes, the primary reason is easier theoretical tractability. In fact, the asymptotic regime is an approximation for large n,d and can also shed light on practical machine learning problems. We clarified this in the revision.