Clarification on Kruskal Stress as an Optimization Target in Metric and Non-metric MDS

### Describe the issue linked to the documentation

I am working on research involving the optimization targets used in metric and non-metric MDS, and I have some questions regarding how scikit-learn's implementation of MDS defines and calculates stress, particularly Kruskal Stress. While reviewing the official documentation, I noticed that specific formulas for stress calculations are not explicitly provided, and I would appreciate some clarification.

Non-metric MDS: My understanding is that non-metric MDS typically minimizes Kruskal Stress, defined as:

![屏幕截图 2024-11-08 163808](https://github.com/user-attachments/assets/d289f4f0-afa0-4c80-a982-c2186c103e27)

in the reduced space. Could you confirm if scikit-learn's non-metric MDS implementation uses this definition, or if it employs an alternative method?

Metric MDS: Does metric MDS in scikit-learn also optimize for Kruskal Stress, or does it use a different stress formula? If a different approach is used, would it be possible to provide some insight or references on the stress function applied here?


### Suggest a potential alternative/fix


Documentation Clarification: It would be incredibly helpful if the documentation could include specific details on the stress formulas used in both metric and non-metric MDS. This addition would help researchers and users better understand the theoretical underpinnings of the algorithm in scikit-learn.
Thank you very much for your guidance and clarification on these points. Your insights would be instrumental in my work with MDS.

COMMENT:
> > 我的理解是，非公制 MDS 通常会最小化 Kruskal 应力
> 
> 否 -- 应力公式由_公制 MDS_ 优化！请注意，平方根和由原始平方距离求和的归一化不会改变损失最小值，因此本质上损失只是高微微和低微微距离之间的平方误差。这通过度量 MDS 最小化。
> 
> 非公制 MDS 允许低微距的任意单调变换，请参阅此处：[https://en.wikipedia.org/wiki/Multidimensional_scaling#Non-metric_multidimensional_scaling_（NMDS）。](https://en.wikipedia.org/wiki/Multidimensional_scaling#Non-metric_multidimensional_scaling_(NMDS))
> 
> 请注意，我认为 sklearn 中的非公制 MDS 已损坏：#27028

Thank you very much for your detailed explanation of metric and nonmetric MDS.

I have reviewed Kruskal's seminal papers from 1964: "Nonmetric multidimensional scaling: A numerical method" and "Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis". These works emphasize minimizing stress as the core optimization criterion for nonmetric MDS(As shown in the figure below), and this has also been cited in the sklearn documentation.
![屏幕截图 2024-11-18 193510](https://github.com/user-attachments/assets/85ef72b5-86ce-4f65-b491-2fd9de07f1d9)

Your explanation mentioned that metric MDS minimizes stress. Should I understand this as Kruskal stress being originally proposed for nonmetric MDS but having a formula applicable to metric MDS as well?

Additionally, you mentioned that nonmetric MDS optimizes for rank preservation instead of directly minimizing Kruskal stress. My understanding was that Kruskal introduced the stress metric specifically as a key optimization target for nonmetric MDS. Furthermore, I noticed that the Wikipedia page you shared describes a stress formula for nonmetric MDS, replacing dij
with 𝑓(𝑑𝑖𝑗) in Kruskal's formula. Is this an alternative optimization target distinct from Kruskal stress, or simply a different representation?

I apologize for my limited understanding and truly value your clarification on this matter. Thank you for your time and expertise!

Labels: Documentation, Needs Triage