unit test failures on x64 darwin and scipy 1.12

### Describe the bug

after upgrading from scipy 1.11.4 -> 1.12 x64 darwin build fails with multiple unit test failures.
https://hydra.nixos.org/build/247540976
[raw log](https://cache.nixos.org/log/d449kn87qr1kcca138gd9pl14qg1nnxg-python3.11-scikit-learn-1.4.0.drv)

```console
=========================== short test summary info ============================
FAILED cluster/tests/test_mean_shift.py::test_parallel[float64] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED decomposition/tests/test_dict_learning.py::test_sparse_encode_shapes_omp - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED decomposition/tests/test_dict_learning.py::test_dict_learning_reconstruction_parallel - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED decomposition/tests/test_dict_learning.py::test_dict_learning_lassocd_readonly_data - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED decomposition/tests/test_dict_learning.py::test_sparse_coder_parallel_mmap - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED decomposition/tests/test_dict_learning.py::test_cd_work_on_joblib_memmapped_data - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED ensemble/tests/test_bagging.py::test_parallel_classification - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED ensemble/tests/test_bagging.py::test_parallel_regression - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED ensemble/tests/test_bagging.py::test_estimator - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED linear_model/tests/test_sgd.py::test_multi_core_gridsearch_and_early_stopping - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED ensemble/tests/test_forest.py::test_backend_respected - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED linear_model/tests/test_theil_sen.py::test_theil_sen_parallel - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED manifold/tests/test_mds.py::test_MDS - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED metrics/tests/test_classification.py::test_classification_metric_division_by_zero_nan_validaton[scoring0] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED metrics/tests/test_classification.py::test_classification_metric_division_by_zero_nan_validaton[scoring1] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED metrics/tests/test_classification.py::test_classification_metric_division_by_zero_nan_validaton[scoring2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED metrics/tests/test_classification.py::test_classification_metric_division_by_zero_nan_validaton[scoring3] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED ensemble/tests/test_voting.py::test_parallel_fit[42] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED metrics/tests/test_pairwise.py::test_sparse_manhattan_readonly_dataset[csr_matrix] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED metrics/tests/test_pairwise.py::test_sparse_manhattan_readonly_dataset[csr_array] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED feature_selection/tests/test_rfe.py::test_rfe_cv_n_jobs[42] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED gaussian_process/tests/test_gpc.py::test_multi_class_n_jobs[kernel0] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED gaussian_process/tests/test_gpc.py::test_multi_class_n_jobs[kernel1] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED model_selection/tests/test_search.py::test_random_search_bad_cv - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED gaussian_process/tests/test_gpc.py::test_multi_class_n_jobs[kernel2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED gaussian_process/tests/test_gpc.py::test_multi_class_n_jobs[kernel3] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED model_selection/tests/test_validation.py::test_cross_validate_many_jobs - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED model_selection/tests/test_validation.py::test_cross_validate_return_indices[42] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED neighbors/tests/test_kd_tree.py::test_kdtree_picklable_with_joblib[KDTree64] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED neighbors/tests/test_kd_tree.py::test_kdtree_picklable_with_joblib[KDTree32] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_permutation_importance_correlated_feature_regression[0.5-2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_permutation_importance_correlated_feature_regression[1.0-2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_robustness_to_high_cardinality_noisy_feature[0.5-2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_robustness_to_high_cardinality_noisy_feature[1.0-2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_permutation_importance_equivalence_sequential_parallel[500] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_permutation_importance_equivalence_sequential_parallel[1.0] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED inspection/tests/test_permutation_importance.py::test_permutation_importance_large_memmaped_data[array] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED neighbors/tests/test_neighbors.py::test_knn_forcing_backend[ball_tree-loky] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED neighbors/tests/test_neighbors.py::test_knn_forcing_backend[kd_tree-loky] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED neighbors/tests/test_neighbors.py::test_knn_forcing_backend[auto-loky] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_calibration.py::test_parallel_execution[True-sigmoid] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_calibration.py::test_parallel_execution[True-isotonic] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_calibration.py::test_parallel_execution[False-sigmoid] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_calibration.py::test_parallel_execution[False-isotonic] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED linear_model/tests/test_coordinate_descent.py::test_linear_models_cv_fit_with_loky[ElasticNetCV] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED linear_model/tests/test_coordinate_descent.py::test_linear_models_cv_fit_with_loky[LassoCV] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_config.py::test_config_threadsafe_joblib[loky] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_multioutput.py::test_multi_output_classification_partial_fit_parallelism - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED tests/test_pipeline.py::test_feature_union_parallel - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
FAILED utils/tests/test_parallel.py::test_configuration_passes_through_to_joblib[loky-2] - joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed...
```

### Steps/Code to Reproduce

run unit tests on x64 darwin with scipy 1.12

### Expected Results

unit tests pass

### Actual Results

https://cache.nixos.org/log/d449kn87qr1kcca138gd9pl14qg1nnxg-python3.11-scikit-learn-1.4.0.drv

### Versions

```shell
$ python -c 'import sklearn; sklearn.show_versions()'
System:
    python: 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 16.0.6 ]
executable: /nix/store/7l7g8ypn5aa6n3nvqids669rlzq9yi93-python3-3.11.7-env/bin/python3.11
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.4.0
          pip: None
   setuptools: None
        numpy: 1.26.2
        scipy: 1.12.0
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.3.2
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /nix/store/qws8gsrmyb3fks94qm4n63rgg97xa3v8-openmp-16.0.6/lib/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /nix/store/mxlhcj24il7ncsmy89hv5h1vjgy5k1nr-openblas-0.3.25/lib/libopenblasp-r0.3.25.dylib
        version: 0.3.25
threading_layer: pthreads
   architecture: Prescott
    num_threads: 8
```

COMMENT:
Looking a bit more at the raw log, it seems like this is similar to something that was seen in https://github.com/scikit-learn/scikit-learn/pull/26845#issuecomment-1639917135. In our case it was seen with numpy 1.25 for some reason and we are pinning numpy<1.25 in the problematic build for now.

I guess we would need to find a way to reproduce this ...

The mysterious problematic error is this one:
```
subprocess.CalledProcessError: Command '['/nix/store/byyb569c90slwsjr43ikhhwlhfavhiir-python3-3.11.7/bin/python3.11', '-sS', '-c', 'import platform; print(platform.mac_ver()[0])']' returned non-zero exit status 1.
```

Full stack-trace of one of the failing test:
```
____________________________ test_parallel[float64] ____________________________
[gw0] darwin -- Python 3.11.7 /nix/store/byyb569c90slwsjr43ikhhwlhfavhiir-python3-3.11.7/bin/python3.11
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 426, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/byyb569c90slwsjr43ikhhwlhfavhiir-python3-3.11.7/lib/python3.11/multiprocessing/queues.py", line 122, in get
    return _ForkingPickler.loads(res)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/__init__.py", line 87, in <module>
    from .base import clone
  File "/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/base.py", line 19, in <module>
    from .utils import _IS_32BIT
  File "/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/utils/__init__.py", line 16, in <module>
    from scipy.sparse import issparse
  File "/nix/store/sssg61dff4xx1iddd3clpvr2cyyn2vhp-python3.11-scipy-1.12.0/lib/python3.11/site-packages/scipy/__init__.py", line 67, in <module>
    from scipy._lib import _pep440
  File "/nix/store/sssg61dff4xx1iddd3clpvr2cyyn2vhp-python3.11-scipy-1.12.0/lib/python3.11/site-packages/scipy/_lib/__init__.py", line 12, in <module>
    from scipy._lib._testutils import PytestTester
  File "/nix/store/sssg61dff4xx1iddd3clpvr2cyyn2vhp-python3.11-scipy-1.12.0/lib/python3.11/site-packages/scipy/_lib/_testutils.py", line 21, in <module>
    _tags = list(sys_tags())
            ^^^^^^^^^^^^^^^^
  File "/nix/store/x21j2i6l1nvrvm6j2prk0lqff8f78j2f-python3.11-packaging-23.2/lib/python3.11/site-packages/packaging/tags.py", line 543, in sys_tags
    yield from cpython_tags(warn=warn)
  File "/nix/store/x21j2i6l1nvrvm6j2prk0lqff8f78j2f-python3.11-packaging-23.2/lib/python3.11/site-packages/packaging/tags.py", line 212, in cpython_tags
    platforms = list(platforms or platform_tags())
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/x21j2i6l1nvrvm6j2prk0lqff8f78j2f-python3.11-packaging-23.2/lib/python3.11/site-packages/packaging/tags.py", line 400, in mac_platforms
    version_str = subprocess.run(
                  ^^^^^^^^^^^^^^^
  File "/nix/store/byyb569c90slwsjr43ikhhwlhfavhiir-python3-3.11.7/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/nix/store/byyb569c90slwsjr43ikhhwlhfavhiir-python3-3.11.7/bin/python3.11', '-sS', '-c', 'import platform; print(platform.mac_ver()[0])']' returned non-zero exit status 1.
"""

The above exception was the direct cause of the following exception:

global_dtype = <class 'numpy.float64'>

    def test_parallel(global_dtype):
        centers = np.array([[1, 1], [-1, -1], [1, -1]]) + 10
        X, _ = make_blobs(
            n_samples=50,
            n_features=2,
            centers=centers,
            cluster_std=0.4,
            shuffle=True,
            random_state=11,
        )
    
        X = X.astype(global_dtype, copy=False)
    
        ms1 = MeanShift(n_jobs=2)
>       ms1.fit(X)

/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/cluster/tests/test_mean_shift.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/base.py:1351: in wrapper
    return fit_method(estimator, *args, **kwargs)
/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/cluster/_mean_shift.py:502: in fit
    all_res = Parallel(n_jobs=self.n_jobs)(
/nix/store/qh5j9kcqfbb1pydmn8sywfmdgwcg6syl-python3.11-scikit-learn-1.4.0/lib/python3.11/site-packages/sklearn/utils/parallel.py:67: in __call__
    return super().__call__(iterable_with_config)
/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1952: in __call__
    return output if self.return_generator else list(output)
/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1595: in _get_outputs
    yield from self._retrieve()
/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1699: in _retrieve
    self._raise_error_fast()
/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1734: in _raise_error_fast
    error_job.get_result(self.timeout)
/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:736: in get_result
    return self._return_or_raise()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <joblib.parallel.BatchCompletionCallBack object at 0x153af4650>

    def _return_or_raise(self):
        try:
            if self.status == TASK_ERROR:
>               raise self._result
E               joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.

/nix/store/kxa7w6ydqg8r94bf4hmchnrfc5mihx64-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:754: BrokenProcessPool
----------------------------- Captured stderr call -----------------------------
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
OSError: [Errno 9] Bad file descriptor

Current thread 0x0000000200e6a240 (most recent call first):
  <no Python frame>
```

Labels: Bug, Needs Reproducible Code