KNeighborsClassifier reports different nearest neighbors and decision boundary depending on sys.platform

### Describe the bug

Training a `KNeighborsClassifier` on the iris dataset produces output that seems to depend on the system architecture (Linux, Mac, Windows tested). The order of neighboring points returned by `KNeighborsClassifier.kneighbors()` is slightly different for specific points near the decision boundary, and in some cases there are differences in the actual neighbors returned (presumably when the order is different causing a difference near `n_neighbors`).

I can theorise reasons why there might be these small differences, but I cannot find this documented anywhere, and therefore wondered if it were a bug or (if I hadn't missed something) whether something could be added to the documentation. There is no difference in the data produced by `load_iris()` and `train_test_split()`, so I know it's not that.

### Steps/Code to Reproduce

```python
# /// script
# requires-python = "==3.12.8"
# dependencies = [
#     "matplotlib",
#     "numpy==2.2.1",
#     "pandas==2.2.3",
#     "scikit-learn==1.6.1",
#     "scipy==1.15.1",
# ]
# ///

import hashlib
import sys

import numpy as np
from sklearn.datasets import load_iris
from sklearn.inspection import DecisionBoundaryDisplay
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

X, y = load_iris(as_frame=True, return_X_y=True)
X = X[["sepal length (cm)", "sepal width (cm)"]]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)

breakpoint()

model = KNeighborsClassifier(n_neighbors=20).fit(X_train, y_train)

# Pick a point that is classified differently on different platforms
point_of_interest = [6.35, 2.80]
print("classification", model.predict([point_of_interest]))
print("neighbors", model.kneighbors([point_of_interest], return_distance=False))
print("hash of training data", hashlib.md5((str(X_train) + str(y_train)).encode()).hexdigest())

display = DecisionBoundaryDisplay.from_estimator(model, X)
display.ax_.plot(*point_of_interest, 'rx')
display.ax_.set_title(sys.platform)
display.figure_.savefig(f"decision_boundary_{sys.platform}.png")
```

### Expected Results

Results as consistent across platforms.

### Actual Results

On Linux (WSL):

```
classification [1]
neighbors [[26 86 10 52 25 79 72 49  6 59 70 71 73 29 57 76 16 82 68 87]]
hash of training data 40c745f2bcd69857949d2229ac5bcdd4
```

![Decision boundary on Linux](https://github.com/user-attachments/assets/f758a181-d2fe-4da3-9e73-071358affcaf)

---

On Windows:

```
classification [2]
neighbors [[26 86 10 52 25 79 72  6 49 59 70 73 71 29 76 57 16 82 77 68]]
hash of training data 40c745f2bcd69857949d2229ac5bcdd4
```

![Decision boundary on Windows](https://github.com/user-attachments/assets/0eeb56c8-c98b-4d5a-b999-d4ed3ba0ade6)

---

On Mac:

```
classification [2]
neighbors [[26 86 10 52 25 79 72  6 49 59 70 73 71 29 76 57 16 82 77  2]]
hash of training data 40c745f2bcd69857949d2229ac5bcdd4
```

![Decision boundary on Mac](https://github.com/user-attachments/assets/e186ccff-e084-46fc-927b-71e68a07549a)

### Versions

```shell
On Linux:

System:
    python: 3.12.8 (main, Jan  5 2025, 05:33:15) [Clang 19.1.6 ]
executable: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/bin/python3
   machine: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.6.1
          pip: None
   setuptools: None
        numpy: 2.2.1
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-6bb31eeb.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/lib/python3.12/site-packages/scipy.libs/libscipy_openblas-68440149.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/lib/python3.12/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None


On Windows:

System:
    python: 3.12.8 (main, Dec 19 2024, 14:41:01) [MSC v.1942 64 bit (AMD64)]
executable: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Scripts\python.exe
   machine: Windows-11-10.0.22631-SP0

Python dependencies:
      sklearn: 1.6.1
          pip: None
   setuptools: None
        numpy: 2.2.1
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Lib\site-packages\numpy.libs\libscipy_openblas64_-43e11ff0749b8cbe0a615c9cf6737e0e.dll
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: vcomp
       filepath: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Lib\site-packages\scipy.libs\libscipy_openblas-f07f5a5d207a3a47104dca54d6d0c86a.dll
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell


On Mac:

System:
    python: 3.12.8 (main, Dec  6 2024, 19:42:06) [Clang 18.1.8 ]
executable: /Users/user/.cache/uv/archive-v0/AKRvDW3X4a5KTGwOrcQJq/bin/python3
   machine: macOS-15.2-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.1
          pip: None
   setuptools: None
        numpy: 2.2.1
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/user/.cache/uv/archive-v0/AKRvDW3X4a5KTGwOrcQJq/lib/python3.12/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
Thanks @jatonline for the very detailed issue!

I would agree with @jeremiedbb that this is a general floating point computation caveat. Can it be surprising? Definitely. Does this belong in the scikit-learn documentation? I don't think so. 

IMO a FAQ will not be so useful because it will be phrased in such a generic manner that people will not necessarily find it or understand how this applies to their use case. I think the github issue tracker is a OK enough way to surface this kind of information. We can always revisit this if we decide that we get many more questions about this and that it would be good to have a FAQ about it.

@Aryanrai23 if you are interested to contribute to scikit-learn, this is more than welcome. To get started I would suggest to start with [Issues for new contributors](https://scikit-learn.org/dev/developers/contributing.html#issues-for-new-contributors).

Labels: Documentation