Title: Johnson-Lindenstrauss Lemma Beyond Euclidean Geometry
Venue: NeurIPS 2025
Accepted: yes
Keywords: Johnson-Lindenstrauss Lemma

Non-Euclidean Geometry
Abstract: The Johnson-Lindenstrauss (JL) lemma is a cornerstone of dimensionality reduction in Euclidean space, but its applicability to non-Euclidean data has remained limited. This paper extends the JL lemma beyond Euclidean geometry to handle general dissimilarity matrices that are prevalent in real-world applications. We present two complementary approaches: First, we show how the JL transform can be applied to vectors in pseudo-Euclidean space with signature $(p,q)$, providing theoretical guarantees that depend on the ratio of the $(p, q)$ norm and Euclidean norm of two vectors, measuring the deviation from Euclidean geometry. Second, we prove that any symmetric hollow dissimilarity matrix can be represented as a matrix of generalized power distances, with an additional parameter representing the uncertainty level within the data. In this representation, applying the JL transform yields multiplicative approximation with a controlled additive error term proportional to the deviation from Euclidean geometry. Our theoretical results provide fine-grained performance analysis based on the degree to which the input data deviates from Euclidean geometry, making practical and meaningful reduction in dimensionality accessible to a wider class of data. We validate our approaches on both synthetic and real-world datasets, demonstrating the effectiveness of extending the JL lemma to non-Euclidean settings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper formulates the case of the JL lemma in non Euclidean cases.\nThe authors propose two complementary approaches to achieve this generalization:\n* Pseudo-Euclidean representation, which shows how any symmetric hollow dissimilarity matrix can be represented as distances between vectors with a specific signature $(p, q)$. JL transform in this space guarantees w.p. a multiplicative factor that depends on the ratio of the squared Euclidean distance and the squared (p,q) distance. This offers a fine-grained performance analysis based on the data's deviation from Euclidean geometry.\n* Generalized power distances, which shows that any symmetric hollow dissimilarity matrix can be expressed as a matrix of generalized power distances. Here, the JL transform yields the multiplicative approximation with an additional additive error term, proportional to a parameter 'r' (radius) that quantifies the uncertainty level within the data (and its deviation from Euclidean geometry). This approach also offers a statistical interpretation using silhouette coefficients of Gaussian distributions.\n\nThe theoretical results provide error analyses that depend on parameters characterizing how the input data deviates from Euclidean geometry, allowing for practical and meaningful dimensionality reduction in a wider range of data types. \n\nThe authors validate their approaches on both synthetic and real-world datasets, showing their effectiveness and superior performance compared to the classical JL transform on non-Euclidean data.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths \nThis paper is very well written, and easy to follow.\nAdditionally, the authors provide relevant real-world applications to the proposed methods, e.g. Minkowski space and Casey’s Theorem.\n\n## Weakness\nJL transform is, by now, a very studied topic, with many efficient and scalable implementations. \nHowever, the authors chose only trivial/toy datasets to showcase their method. \nAdditionally, Table 2 presents only non-robust metrics (i.e. max and mean) for comparison with classic JL - this is slightly unfair and the bounds hold only w.p. and the worst case error might be unbounded.\n\nQUESTIONS:\n* How do you think the proposed method(s) will compare in Table 2, if we measure a robust statistic of relative error like median, ans/or some higher percentile \n* There are some existing methods that already use non Euclidean (pseudo) metrics in high dimensions, such as face recognition and information retrieval. I would appreciate a short discussion on the impact on large(r) sacle problems like these.\n\nLIMITATIONS:\nThe authors do not dedicate a section to limitations w.r.t. the current SOTA, i.e. classic JL.\nNotwithstanding, the authors do point to possible future directions and cases where the applicability of the proposed approach is not known.\nAdditionally, they note that a theoretical connection between the two proposed methods is not clear.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present variants to the Johnson Lindenstrauss lemma for non-metric settings. The first uses a generalization of metric distances by defining an inner product in $\\mathbb{R}^{p + q}$ which adds the inner product of the first p dimensions and then subtracts the inner product of the next q dimensions. The second uses a separate generalization of distances termed \"power distances\", where each point is endowed with a radius and the distances between two points correspond to the length of the tangent lines between their corresponding circles. In both cases, the authors show theoretical results which are essentially consistent with those for the standard JL transform.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- I consider both cases interesting with regards to the JL lemma but find the main results to be the evidence that any symmetric hollow matrix can be written as either a pseudo-distance matrix or a power-distance matrix. I have questions regarding one of these results, but if everything is correct then this is a nice fact which would be useful for many applications beyond simply the JL-transform.\n- The paper is very clearly wriitten and quite easy to follow. I have experience in other classical dimensionality reduction algorithms (PCA, MDS, LLE, etc.) and this text introduced the relevant ideas so that I could follow them, despite not being particularly well-versed in JL theory.\n\nWeaknesses:\n- A few steps are presented without sufficient explanation for their validity (specifically in Section 2.1)\n- I am not sure what purpose the long exposition on Ptolemaic identities serves.\n- The experimental results are presented with insufficient care in my opinion. Some small issues include:\n  - The text in the plots is tiny\n  - It is unclear what the entropic affinity is or why JL-PE suddenly performs better on this metric when it dramatically fails on the other ones.\n  - It is not clear how one applies the JL transform to a graph dataset (perhaps I missed this somewhere?)\n- Absence of a limitations section (as encouraged by the guidelines). The limitations discussed in the checklist are quite spurious I and get the sense the authors were just trying to check a box rather than provide meaningful insight into how their work could be improved upon.\n\nQUESTIONS:\nMy primary questions are regarding Section 2.1. I am *not* convinced by the provided text that one can take an arbitrary dissimilarity matrix and convert it into a set of observations in the given manner. The arguments are much too handwavy and something isn't passing the smell test for me. It would be good to write up a formal lemma or proposition and proof to clean this exposition up.\n\nFurthermore, I believe that the \"non-metric\" pseudo-euclidean dissimilarities might actually be metric, just in a complex space.\n\n### Regarding the centering:\n\n- In the case of D containing *squared* Euclidean distances, it is clear that we have the equality $-0.5 C D C = C G_X C$. However, this is explicitly due to the fact that the distances are induced by a norm and therefore we can distribute $D$.\n  - Specifically, for some dataset $\\mathbf{X} \\in \\mathbb{R}^{n \\times d}$, let $\\mathbf{\\alpha}$ be the vector of $\\alpha_i = \\langle x_i, x_i \\rangle$ and $\\mathbf{1}$ be the all ones vector. This lets us write any matrix $D$ of pairwise metric squared distances via $D = \\mathbf{1} \\mathbf{\\alpha}^\\top + \\mathbf{\\alpha} \\mathbf{1}^\\top - 2 \\mathbf{G}_X$. In this case, we get that the centering matrix cancels with the all-ones vectors and we get $-\\frac{1}{2} C D C = C G_X C$.\n  - However, in the setting considered in the paper, we can make no such assumption on the structure of $D$ since it is not obtained from an inner product space. Thus, I am not convinced that, for non-metric $D$, we can re-write it as a Gram matrix via centering. This then throws the subsequent derivations into question.\n\n- Moreover... why are the centering matrices necessary at all for the operations as they are laid out in the paper? Ostensibly, since the matrix D is already symmetric, one could immediately compute its singular value decomposition and perform all of the subsequent steps without ever applying the centering. Am I missing something where the centering is actually necessary for the downstream calculations? Something doesn't feel right.\n  - Specifically, the paper states \"Since $−CDC/2$ is a symmetric matrix, its eigenvalues are real...\" But D is also symmetric by definition. So why are we applying the centering?\n\nIn summary, it seems the centering is necessary for the theory to obtain a Gram matrix which can be analyzed accordingly. But I am not convinced that this is well-motivated in the non-metric setting. This is my primary concern and resolving it would certainly increase my score.\n\n### Regarding the complex space\n\n- To make sure I'm understanding, the $\\Lambda$ matrix in Section 2.1 could be replaced by simply having the $x$ values be complex values, right? I.e., we have some negative eigenvalues, so the $X X^T$ which produces those would consist of complex-valued observations.\n  - But then this would imply that the dissimilarity matrix being considered is actually metric; it just happens to come from points which live in a complex space. If this line of reasoning is correct, then I feel that the presentation should be amended significantly to mention that the paper presents results for JL transforms in non-metric spaces and in complex metric spaces.\n\nLIMITATIONS:\nAs mentioned above, the limitations as presented could be expanded on but it's not a big issue. I think the work could use an explicit limitations section which describes which assumptions were necessary, how these assumptions factor through, and how these assumptions may be avoided in the future.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper generalizes the Johnson-Lindenstrauss lemma to (slightly) non-Euclidean geometries. We are given as input a dissimilarity matrix D for n points, which need only be symmetric and hollow (diagonal entries 0). The existential result gives points of dimension O(log n / eps^2) such that the pairwise distances equal the entries of D up to multiplicative (1 + eps) and additive 4 * eps * r^2, where r measures how much D deviates from Euclidean distances. 


Two different versions are given. First, the desired dimensionality reduction is shown for points in "pseudo-Euclidean space" where, when computing the inner product, some coordinates are negated. Second, it is shown that for any symmetric hollow D, one can find points in "generalized power distance" space whose pairwise distances give the entries of D, and dimensionality reduction is also given for such points.

STRENGTHS AND WEAKNESSES:
Strengths
- The second result is quite general and can work for any dissimilarity matrix D
- Gives two nice generalizations of the JL lemma to non-Euclidean data
- Appear effective in a few experiments

Weaknesses
- The results, as stated, are non-algorithmic; it would be nice to know how efficiently these can be computed
- It is not clear that these are the correct ways to generalize to non-Euclidean data for ML applications. It is only hinted at what the ML applications are.

QUESTIONS:
Please write "hollow" instead of "reflexive" on line 49, since that's the word you use in the rest of the paper.

What is the running time of your algorithm for finding the points in theorem 1.3?

Is there evidence that these notions ("pseudo-Euclidean space" and "generalized power distance") are the correct ones to use in ML applications?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission aims to extend the Johnson-Lindenstrauss (JL) lemma to general non-Euclidean dissimilarity matrices by proposing two approaches: one based on pseudo-Euclidean embeddings and another based on generalized power distances. The authors provide theoretical guarantees and empirical results on both synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: Motivated Extension of JL Lemma: The work addresses an important and timely question: extending dimensionality reduction techniques beyond standard Euclidean spaces. The authors provide a structured attempt to generalize JL guarantees to non-metric or non-Euclidean dissimilarities, which are common in practice (e.g., in genomics, information retrieval, etc.). Conceptual Contribution: The connection between generalized power distance and data uncertainty is interesting and provides a new lens for interpreting structure in non-Euclidean datasets. The interpretation via eigenvalues of the Gram matrix offers some insight into when JL-like guarantees are meaningful.\n\nWeaknesses: Incremental Techniques: While the problem setup is novel, many of the tools (e.g., pseudo-Euclidean embedding via centralized Gram matrices, standard JL transforms) are not new. The contribution lies mostly in framing and packaging, rather than in fundamentally new techniques or insights. Heavy Computation Assumptions: The approach assumes full access to the dissimilarity matrix and requires SVD of the Gram matrix, which is cubic in time and not scalable to large datasets. This makes the current form more of a theoretical contribution than a practical algorithm.\n\nQUESTIONS:\nCan the authors demonstrate the utility of the proposed JL embeddings beyond preserving pairwise dissimilarities? For example, do these embeddings improve performance in downstream tasks such as clustering, classification, or nearest neighbor search on non-Euclidean data?\n\nLIMITATIONS:\nThe impossibility of worst-case dimension reduction guarantees for general non-Euclidean data.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}