Repository: pytorch
Issue: Segmentation error for torch==2.2.1 on MacOs
Labels: needs reproduction,module: crash,triaged,module: macos,module: openmp,module: intel

### 🐛 Describe the bug

At shap, we have run into problems with our CI jobs on macOs, e.g. see [here](https://github.com/shap/shap/actions/runs/8021717954/job/21914432162#step:7:28). I tracked this down to an issue with `torch==2.2.1`.

Here is code to reproduce the issue (this works on `torch==2.2.0`):
```python
import time

import torch
from sklearn.datasets import fetch_california_housing


def test_something():
    X, y = fetch_california_housing(return_X_y=True)
    torch.tensor(X)
    time.sleep(3)
```
(execute with `python -m pytest <filename>`)

Stacktrace:
```bash
bash-3.2$ python -m pytest tests/explainers/test_segfault_minimal_example2.py                                                                                                                               
=========================================================================================== test session starts ============================================================================================
platform darwin -- Python 3.11.8, pytest-8.1.0, pluggy-1.4.0
Matplotlib: 3.8.3
Freetype: 2.6.1
rootdir: /Users/runner/work/shap/shap
configfile: pyproject.toml
plugins: cov-4.1.0, mpl-0.17.0
collected 1 item                                                                                                                                                                                           

tests/explainers/test_segfault_minimal_example2.py Fatal Python error: Segmentation fault

Thread 0x00000001140ad600 (most recent call first):
  File "/Users/runner/work/shap/shap/tests/explainers/test_segfault_minimal_example2.py", line 8 in test_something
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/_pytest/python.py", line 1769 in runtestSegmentation fault: 11
```

### Versions
```
PyTorch version: 2.2.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.7.3 (x86_64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: version 3.28.3
Libc version: N/A

Python version: 3.11.8 (v3.11.8:db85d51d3e, Feb  6 2024, 18:02:37) [Clang 13.0.0 (clang-1300.0.29.30)] (64-bit runtime)
Python platform: macOS-12.7.3-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.2.1
[pip3] torchvision==0.17.0
[conda] No relevant packages
```

cc @malfet @albanD @frank-wei @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10