Crash in T-SNE

### Describe the bug

I got a crash using the (external) [hdbscan package](https://github.com/scikit-learn-contrib/hdbscan) in some special situation. I [debugged it](https://github.com/scikit-learn-contrib/hdbscan/issues/623) and found out that it happens in the scikit-learn package, specifically its T-SNE implementation. The hdbscan maintainer ([Leland McInnes](https://github.com/lmcinnes)!) suggested to report it here.

Is this something that should be guarded for in scikit-learn?

### Steps/Code to Reproduce

The simplest way to reproduce it (using the hdbscan package) is:
```py
import numpy as np
import hdbscan
model = hdbscan.HDBSCAN(gen_min_span_tree=True)
data = np.zeros((91, 3))
clustering = model.fit(data)
clustering.minimum_spanning_tree_.plot()
```
Note that it also happens when only a relative small proportion of points are equal (but only sometimes?), this is just the easiest way to show it. By default some warnings are displayed:
> ...\sklearn\decomposition\_pca.py:685: RuntimeWarning: invalid value encountered in divide
>   self.explained_variance_ratio_ = self.explained_variance_ / total_var
> ...\sklearn\manifold\_t_sne.py:1002: RuntimeWarning: invalid value encountered in divide
>   X_embedded = X_embedded / np.std(X_embedded[:, 0]) * 1e-4 

In the end it appears to be a problem in `sklearn.manifold._t_sne._barnes_hut_tsne.gradient()`, not (always?) being able to handle `nan` values. For example, this reproduces the crash:
```py
import numpy as np
from sklearn.manifold._t_sne import _barnes_hut_tsne
neighbors = np.array([1, 2, 0, 2, 0, 1], dtype='int64')
val_P = np.full_like(neighbors, 2 / 45, dtype='float32')
pos_output = np.full((3, 2), np.nan, dtype='float32')
forces = np.zeros_like(pos_output)
indptr = np.arange(7, step=2, dtype='int64')
_barnes_hut_tsne.gradient(val_P, pos_output, neighbors, indptr, forces, 0.5, 2, 11)
```
One layer deeper, the crash occurs inside `sklearn.neighbors._quad_tree._QuadTree.build_tree()`, as follows:
```py
import numpy as np
from sklearn.neighbors._quad_tree import _QuadTree
qt = _QuadTree(2, 11)
X = np.full((3, 2), np.nan, dtype='float32')
qt.build_tree(X)
```
The output of this (due to `verbose=11`) up to the crash is:
> [QuadTree] bounding box axis 0 : [nan, nan]
> [QuadTree] bounding box axis 1 : [nan, nan]
> [QuadTree] Inserting depth 0
> [QuadTree] inserted point 0 in cell 0
> [QuadTree] Inserting depth 0
> [QuadTree] inserted point 0 in new child 1
> [QuadTree] Inserting depth 0
> [QuadTree] Inserting depth 1
> ...
> [QuadTree] Inserting depth 6271
> [QuadTree] inserted point 0 in new child 6272
> [QuadTree] Inserting depth 6271
> [QuadTree] Inserting depth 6272

I didn't dig into the QuadTree code.

### Expected Results

I would expect anything but Python crashing.

### Actual Results

Python crashed: its console window just went away silently (on Windows).

### Versions

```shell
System:
    python: 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)]
executable: C:\Users\Public\Software\Python\python.exe
   machine: Windows-11-10.0.22631-SP0

Python dependencies:
      sklearn: 1.4.0
          pip: 23.3.2
   setuptools: 69.0.3
        numpy: 1.26.3
        scipy: 1.12.0
       Cython: 3.0.8
       pandas: 2.2.0
   matplotlib: 3.8.2
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 16
         prefix: libopenblas
       filepath: C:\Users\Public\Software\Python\Lib\site-packages\numpy.libs\libopenblas64__v0.3.23-293-gc2f4bdbb-gcc_10_3_0-2bde3a66a51006b2b53eb373ff767a3f.dll
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Cooperlake

       user_api: openmp
   internal_api: openmp
    num_threads: 16
         prefix: vcomp
       filepath: C:\Users\Public\Software\Python\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 16
         prefix: libopenblas
       filepath: C:\Users\Public\Software\Python\Lib\site-packages\scipy.libs\libopenblas_v0.3.20-571-g3dec11c6-gcc_10_3_0-c2315440d6b6cef5037bad648efc8c59.dll
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Cooperlake
```

COMMENT:
In #29127, there was a variation of this issue using only scikit-learn:

```py
import pandas as pd
import numpy as np
from sklearn.manifold import TSNE

import sklearn
sklearn.show_versions()

hashGNN_embeddings = {'projectName': {0: 'react-router', 1: 'react-router-dom', 2: 'react-router-native', 3: 'react-router-dom', 4: 'router'}, 'embedding': {0: [0.0, -0.4330126941204071, -0.4330126941204071, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.0, -1.0825317353010178, -0.21650634706020355, -0.21650634706020355, 0.6495190411806107, 0.8660253882408142, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.21650634706020355, -0.4330126941204071, -0.8660253882408142, 0.4330126941204071, 0.21650634706020355, 0.8660253882408142, 0.0, 0.4330126941204071, 0.21650634706020355, 0.21650634706020355, -0.21650634706020355, 0.0, 0.0, 0.0, -0.21650634706020355, 0.4330126941204071, 1.0825317353010178, -0.4330126941204071, 0.21650634706020355, 1.0825317353010178, -0.4330126941204071, 1.0825317353010178, 0.0, -0.8660253882408142, 0.21650634706020355, 0.8660253882408142, 0.0, 0.21650634706020355, 0.21650634706020355, 0.0, 0.21650634706020355, 0.6495190411806107, 0.6495190411806107, 0.0, -0.6495190411806107, 0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.21650634706020355, 0.6495190411806107, -0.6495190411806107, 0.4330126941204071, 0.6495190411806107, 0.0, 0.4330126941204071, 0.0, 0.21650634706020355, 0.4330126941204071], 1: [0.0, -0.4330126941204071, -0.4330126941204071, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.0, -1.0825317353010178, -0.21650634706020355, -0.21650634706020355, 0.6495190411806107, 0.8660253882408142, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.21650634706020355, -0.4330126941204071, -0.8660253882408142, 0.4330126941204071, 0.21650634706020355, 0.8660253882408142, 0.0, 0.4330126941204071, 0.21650634706020355, 0.21650634706020355, -0.21650634706020355, 0.0, 0.0, 0.0, -0.21650634706020355, 0.4330126941204071, 1.0825317353010178, -0.4330126941204071, 0.21650634706020355, 1.0825317353010178, -0.4330126941204071, 1.0825317353010178, 0.0, -0.8660253882408142, 0.21650634706020355, 0.8660253882408142, 0.0, 0.21650634706020355, 0.21650634706020355, 0.0, 0.21650634706020355, 0.6495190411806107, 0.6495190411806107, 0.0, -0.6495190411806107, 0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.21650634706020355, 0.6495190411806107, -0.6495190411806107, 0.4330126941204071, 0.6495190411806107, 0.0, 0.4330126941204071, 0.0, 0.21650634706020355, 0.4330126941204071], 2: [0.0, -0.4330126941204071, -0.4330126941204071, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.0, -1.0825317353010178, -0.21650634706020355, -0.21650634706020355, 0.6495190411806107, 0.8660253882408142, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.21650634706020355, -0.4330126941204071, -0.8660253882408142, 0.4330126941204071, 0.21650634706020355, 0.8660253882408142, 0.0, 0.4330126941204071, 0.21650634706020355, 0.21650634706020355, -0.21650634706020355, 0.0, 0.0, 0.0, -0.21650634706020355, 0.4330126941204071, 1.0825317353010178, -0.4330126941204071, 0.21650634706020355, 1.0825317353010178, -0.4330126941204071, 1.0825317353010178, 0.0, -0.8660253882408142, 0.21650634706020355, 0.8660253882408142, 0.0, 0.21650634706020355, 0.21650634706020355, 0.0, 0.21650634706020355, 0.6495190411806107, 0.6495190411806107, 0.0, -0.6495190411806107, 0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.21650634706020355, 0.6495190411806107, -0.6495190411806107, 0.4330126941204071, 0.6495190411806107, 0.0, 0.4330126941204071, 0.0, 0.21650634706020355, 0.4330126941204071], 3: [0.0, -0.4330126941204071, -0.4330126941204071, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.0, -1.0825317353010178, -0.21650634706020355, -0.21650634706020355, 0.6495190411806107, 0.8660253882408142, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.21650634706020355, -0.4330126941204071, -0.8660253882408142, 0.4330126941204071, 0.21650634706020355, 0.8660253882408142, 0.0, 0.4330126941204071, 0.21650634706020355, 0.21650634706020355, -0.21650634706020355, 0.0, 0.0, 0.0, -0.21650634706020355, 0.4330126941204071, 1.0825317353010178, -0.4330126941204071, 0.21650634706020355, 1.0825317353010178, -0.4330126941204071, 1.0825317353010178, 0.0, -0.8660253882408142, 0.21650634706020355, 0.8660253882408142, 0.0, 0.21650634706020355, 0.21650634706020355, 0.0, 0.21650634706020355, 0.6495190411806107, 0.6495190411806107, 0.0, -0.6495190411806107, 0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.21650634706020355, 0.6495190411806107, -0.6495190411806107, 0.4330126941204071, 0.6495190411806107, 0.0, 0.4330126941204071, 0.0, 0.21650634706020355, 0.4330126941204071], 4: [0.0, -0.4330126941204071, -0.4330126941204071, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.0, -1.0825317353010178, -0.21650634706020355, -0.21650634706020355, 0.6495190411806107, 0.8660253882408142, 0.21650634706020355, -0.21650634706020355, -0.4330126941204071, 0.21650634706020355, -0.4330126941204071, -0.8660253882408142, 0.4330126941204071, 0.21650634706020355, 0.8660253882408142, 0.0, 0.4330126941204071, 0.21650634706020355, 0.21650634706020355, -0.21650634706020355, 0.0, 0.0, 0.0, -0.21650634706020355, 0.4330126941204071, 1.0825317353010178, -0.4330126941204071, 0.21650634706020355, 1.0825317353010178, -0.4330126941204071, 1.0825317353010178, 0.0, -0.8660253882408142, 0.21650634706020355, 0.8660253882408142, 0.0, 0.21650634706020355, 0.21650634706020355, 0.0, 0.21650634706020355, 0.6495190411806107, 0.6495190411806107, 0.0, -0.6495190411806107, 0.21650634706020355, -0.4330126941204071, 0.4330126941204071, 0.21650634706020355, 0.6495190411806107, -0.6495190411806107, 0.4330126941204071, 0.6495190411806107, 0.0, 0.4330126941204071, 0.0, 0.21650634706020355, 0.4330126941204071]}}
embeddings = pd.DataFrame.from_dict(hashGNN_embeddings)
embeddings_as_numpy_array = np.array(embeddings.embedding.to_list())
tsne = TSNE(n_components=2, perplexity=3.0, verbose=1, random_state=50)

two_dimension_node_embeddings = tsne.fit_transform(embeddings_as_numpy_array)
```

Same warning:
```
/home/lesteve/micromamba/lib/python3.11/site-packages/sklearn/decomposition/_pca.py:793: RuntimeWarning: invalid value encountered in divide
  self.explained_variance_ratio_ = self.explained_variance_ / total_var
/home/lesteve/micromamba/lib/python3.11/site-packages/sklearn/manifold/_t_sne.py:1030: RuntimeWarning: invalid value encountered in divide
  X_embedded = X_embedded / np.std(X_embedded[:, 0]) * 1e-4
```

and also a segmentation fault with similar gdb info as https://github.com/scikit-learn/scikit-learn/issues/28368#issuecomment-1931390670.

Labels: Bug, help wanted, module:manifold, cython