Inconsistency in `DecisionTreeClassifier` Threshold Behavior

### Describe the bug

I've encountered an unexpected behavior in `DecisionTreeClassifier` when using a decision stump (a tree with one root node and two leaf children). My assumption is based on the standard decision tree logic where a feature value `x` is classified to the left child if `x <= threshold`. Therefore, I expect the following assertion to always be true:
```python
assert clf.apply([[clf.tree_.threshold[0]]]) == 1
```

This assertion is meant to test that a feature value equal to the root node's threshold is classified to the left child node, in accordance with the `x <= threshold` rule. However, I have observed that in approximately 25% of the cases, the data point is unexpectedly classified to the right child node instead of the left.

The issue persists regardless of whether the `threshold[0]` is explicitly converted to `float32` or not. Given scikit-learn's documentation stating that
> All decision trees use `np.float32` arrays internally. If training data is not in this format, a copy of the dataset will be made

this behavior is puzzling. Precise thresholding is crucial for my application. Thank you for your time and effort in maintaining this important library and for any insights you can provide regarding this issue.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.tree import DecisionTreeClassifier

correct, wrong = [], []
n_trials = 500
for seed in range(n_trials):
    rand = np.random.RandomState(seed)
    X = rand.normal(size=(100, 1))
    y = rand.randint(0, 2, size=X.shape[0])
    clf = DecisionTreeClassifier(max_depth=1, random_state=0).fit(X, y)
    
    thres = clf.tree_.threshold[0]
    # thres = np.float32(thres) # gives the same correct-wrong distribution
    if clf.apply([[thres]]) == 1:
        correct.append(thres)
    else:
        wrong.append(thres)

    assert clf.apply([[np.nextafter(np.float32(thres), np.float32(-np.inf))]]) == 1 # this is true though
print(f'{len(correct) / n_trials:%}') # 75%
print(f'{len(wrong) / n_trials:%}') # 25%
```

### Expected Results

```
100.0000%
0.0000%
```

### Actual Results

```
75.0000%
25.0000%
```

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /usr/bin/python3
   machine: Linux-6.1.58+-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 23.1.2
   setuptools: 67.7.2
        numpy: 1.23.5
        scipy: 1.11.4
       Cython: 3.0.8
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/numpy.libs/libopenblas64_p-r0-742d56dc.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 2
         prefix: libgomp
       filepath: /usr/local/lib/python3.10/dist-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
Taking back your example and enforcing `apply` to use 64-bit floating point `dtype`, I'm getting 100% of good classification.

Here is the patch:

```diff
diff --git a/sklearn/tree/_classes.py b/sklearn/tree/_classes.py
index 00d61f1847..4bee1d7168 100644
--- a/sklearn/tree/_classes.py
+++ b/sklearn/tree/_classes.py
@@ -488,7 +488,7 @@ class BaseDecisionTree(MultiOutputMixin, BaseEstimator, metaclass=ABCMeta):
                 force_all_finite = True
             X = self._validate_data(
                 X,
-                dtype=DTYPE,
+                dtype=[np.float64, np.float32],
                 accept_sparse="csr",
                 reset=False,
                 force_all_finite=force_all_finite,
diff --git a/sklearn/tree/_tree.pyx b/sklearn/tree/_tree.pyx
index ef3a99a69a..284e5360ef 100644
--- a/sklearn/tree/_tree.pyx
+++ b/sklearn/tree/_tree.pyx
@@ -990,13 +990,13 @@ cdef class Tree:
             raise ValueError("X should be in np.ndarray format, got %s"
                              % type(X))
 
-        if X.dtype != DTYPE:
-            raise ValueError("X.dtype should be np.float32, got %s" % X.dtype)
+        # if X.dtype != DTYPE:
+        #     raise ValueError("X.dtype should be np.float32, got %s" % X.dtype)
 
         # Extract input
-        cdef const float32_t[:, :] X_ndarray = X
+        cdef const float64_t[:, :] X_ndarray = X
         cdef intp_t n_samples = X.shape[0]
-        cdef float32_t X_i_node_feature
+        cdef float64_t X_i_node_feature
 
         # Initialize output
         cdef intp_t[:] out = np.zeros(n_samples, dtype=np.intp)
```

Labels: Bug