KNeighborsClassifier reports different nearest neighbors and decision boundary depending on sys.platform

### Describe the bug

Training a `KNeighborsClassifier` on the iris dataset produces output that seems to depend on the system architecture (Linux, Mac, Windows tested). The order of neighboring points returned by `KNeighborsClassifier.kneighbors()` is slightly different for specific points near the decision boundary, and in some cases there are differences in the actual neighbors returned (presumably when the order is different causing a difference near `n_neighbors`).

I can theorise reasons why there might be these small differences, but I cannot find this documented anywhere, and therefore wondered if it were a bug or (if I hadn't missed something) whether something could be added to the documentation. There is no difference in the data produced by `load_iris()` and `train_test_split()`, so I know it's not that.

### Steps/Code to Reproduce

```python
# /// script
# requires-python = "==3.12.8"
# dependencies = [
#     "matplotlib",
#     "numpy==2.2.1",
#     "pandas==2.2.3",
#     "scikit-learn==1.6.1",
#     "scipy==1.15.1",
# ]
# ///

import hashlib
import sys

import numpy as np
from sklearn.datasets import load_iris
from sklearn.inspection import DecisionBoundaryDisplay
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

X, y = load_iris(as_frame=True, return_X_y=True)
X = X[["sepal length (cm)", "sepal width (cm)"]]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)

breakpoint()

model = KNeighborsClassifier(n_neighbors=20).fit(X_train, y_train)

# Pick a point that is classified differently on different platforms
point_of_interest = [6.35, 2.80]
print("classification", model.predict([point_of_interest]))
print("neighbors", model.kneighbors([point_of_interest], return_distance=False))
print("hash of training data", hashlib.md5((str(X_train) + str(y_train)).encode()).hexdigest())

display = DecisionBoundaryDisplay.from_estimator(model, X)
display.ax_.plot(*point_of_interest, 'rx')
display.ax_.set_title(sys.platform)
display.figure_.savefig(f"decision_boundary_{sys.platform}.png")
```

### Expected Results

Results as consistent across platforms.

### Actual Results

On Linux (WSL):

```
classification [1]
neighbors [[26 86 10 52 25 79 72 49  6 59 70 71 73 29 57 76 16 82 68 87]]
hash of training data 40c745f2bcd69857949d2229ac5bcdd4
```

![Decision boundary on Linux](https://github.com/user-attachments/assets/f758a181-d2fe-4da3-9e73-071358affcaf)

---

On Windows:

```
classification [2]
neighbors [[26 86 10 52 25 79 72  6 49 59 70 73 71 29 76 57 16 82 77 68]]
hash of training data 40c745f2bcd69857949d2229ac5bcdd4
```

![Decision boundary on Windows](https://github.com/user-attachments/assets/0eeb56c8-c98b-4d5a-b999-d4ed3ba0ade6)

---

On Mac:

```
classification [2]
neighbors [[26 86 10 52 25 79 72  6 49 59 70 73 71 29 76 57 16 82 77  2]]
hash of training data 40c745f2bcd69857949d2229ac5bcdd4
```

![Decision boundary on Mac](https://github.com/user-attachments/assets/e186ccff-e084-46fc-927b-71e68a07549a)

### Versions

```shell
On Linux:

System:
    python: 3.12.8 (main, Jan  5 2025, 05:33:15) [Clang 19.1.6 ]
executable: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/bin/python3
   machine: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.6.1
          pip: None
   setuptools: None
        numpy: 2.2.1
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-6bb31eeb.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/lib/python3.12/site-packages/scipy.libs/libscipy_openblas-68440149.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/user/.cache/uv/archive-v0/WfveLKj0nlDrbuzgwXgFU/lib/python3.12/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None


On Windows:

System:
    python: 3.12.8 (main, Dec 19 2024, 14:41:01) [MSC v.1942 64 bit (AMD64)]
executable: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Scripts\python.exe
   machine: Windows-11-10.0.22631-SP0

Python dependencies:
      sklearn: 1.6.1
          pip: None
   setuptools: None
        numpy: 2.2.1
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Lib\site-packages\numpy.libs\libscipy_openblas64_-43e11ff0749b8cbe0a615c9cf6737e0e.dll
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: vcomp
       filepath: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: C:\Users\user\AppData\Local\uv\cache\archive-v0\h7HWruVa4wULKAO9-PD5v\Lib\site-packages\scipy.libs\libscipy_openblas-f07f5a5d207a3a47104dca54d6d0c86a.dll
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell


On Mac:

System:
    python: 3.12.8 (main, Dec  6 2024, 19:42:06) [Clang 18.1.8 ]
executable: /Users/user/.cache/uv/archive-v0/AKRvDW3X4a5KTGwOrcQJq/bin/python3
   machine: macOS-15.2-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.1
          pip: None
   setuptools: None
        numpy: 2.2.1
        scipy: 1.15.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/user/.cache/uv/archive-v0/AKRvDW3X4a5KTGwOrcQJq/lib/python3.12/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
> Do you think this is enough of a 'frequently' asked question to warrant that? 

Hard to tell :) It's not the first time but it's not "frequent" as well, I'd say it's around once a year.

Let try to phrase it the other way around. Would it have helped you if it was already in the FAQ ? Do you think you'd have found it before opening the issue ? If not, it's maybe not worth it indeed.

Labels: Documentation