Repository: pytorch
Issue: Python 3.12 CPU Build system cannot find MKL libraries
Labels: module: build,module: ci,triaged,module: mkl

### 🐛 Describe the bug

Similar to: https://github.com/pytorch/pytorch/issues/24334

Error can be seen on:
https://github.com/pytorch/pytorch/actions/runs/7835085504/job/21382465250

Log:
```
2024-02-08T21:20:46.9482141Z 
2024-02-08T21:20:46.9482362Z ==================================== RERUNS ====================================
2024-02-08T21:20:46.9482704Z ______________ TestOperatorsCPU.test_vmapvmapjvp_linalg_solve_cpu ______________
2024-02-08T21:20:46.9482866Z Traceback (most recent call last):
2024-02-08T21:20:46.9483553Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 1885, in test_vmapvmapjvp_linalg_solve
2024-02-08T21:20:46.9483902Z     batched_out = vmap(vmap(fn, in_dims=in_dims_all), in_dims=in_dims_all)(*args)
2024-02-08T21:20:46.9484109Z                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9484779Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/apis.py", line 188, in wrapped
2024-02-08T21:20:46.9485163Z     return vmap_impl(func, in_dims, out_dims, randomness, chunk_size, *args, **kwargs)
2024-02-08T21:20:46.9485371Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9486054Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 279, in vmap_impl
2024-02-08T21:20:46.9486192Z     return _flat_vmap(
2024-02-08T21:20:46.9486302Z            ^^^^^^^^^^^
2024-02-08T21:20:46.9486918Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9487060Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9487171Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9487844Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 401, in _flat_vmap
2024-02-08T21:20:46.9488066Z     batched_outputs = func(*batched_inputs, **kwargs)
2024-02-08T21:20:46.9488220Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9488875Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/apis.py", line 188, in wrapped
2024-02-08T21:20:46.9489267Z     return vmap_impl(func, in_dims, out_dims, randomness, chunk_size, *args, **kwargs)
2024-02-08T21:20:46.9489463Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9490144Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 279, in vmap_impl
2024-02-08T21:20:46.9490264Z     return _flat_vmap(
2024-02-08T21:20:46.9490374Z            ^^^^^^^^^^^
2024-02-08T21:20:46.9490998Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9491124Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9491236Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9491924Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 401, in _flat_vmap
2024-02-08T21:20:46.9492131Z     batched_outputs = func(*batched_inputs, **kwargs)
2024-02-08T21:20:46.9492278Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9492716Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 303, in wrapped
2024-02-08T21:20:46.9493043Z     primals_out, tangents_out = jvp(fn, primals_in, tangents_in)
2024-02-08T21:20:46.9493220Z                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9493940Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/eager_transforms.py", line 927, in jvp
2024-02-08T21:20:46.9494397Z     return _jvp_with_argnums(func, primals, tangents, argnums=None, strict=strict, has_aux=has_aux)
2024-02-08T21:20:46.9494613Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9495221Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9495348Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9495473Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9496282Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/eager_transforms.py", line 976, in _jvp_with_argnums
2024-02-08T21:20:46.9496430Z     result_duals = func(*duals)
2024-02-08T21:20:46.9496542Z                    ^^^^^^^^^^^^
2024-02-08T21:20:46.9496972Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 130, in wrapped
2024-02-08T21:20:46.9497115Z     result = f(*_args, **kwargs)
2024-02-08T21:20:46.9497314Z              ^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9498117Z RuntimeError: Calling torch.linalg.lu_factor on a CPU tensor requires compiling PyTorch with LAPACK. Please use PyTorch built with LAPACK support.
2024-02-08T21:20:46.9498125Z 
2024-02-08T21:20:46.9498421Z To execute this test, run the following from the base repo dir:
2024-02-08T21:20:46.9498836Z      python test/functorch/test_ops.py -k test_vmapvmapjvp_linalg_solve_cpu
2024-02-08T21:20:46.9498843Z 
2024-02-08T21:20:46.9499198Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
2024-02-08T21:20:46.9499554Z ______________ TestOperatorsCPU.test_vmapvmapjvp_linalg_solve_cpu ______________
2024-02-08T21:20:46.9499703Z Traceback (most recent call last):
2024-02-08T21:20:46.9500432Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 1885, in test_vmapvmapjvp_linalg_solve
2024-02-08T21:20:46.9500769Z     batched_out = vmap(vmap(fn, in_dims=in_dims_all), in_dims=in_dims_all)(*args)
2024-02-08T21:20:46.9500952Z                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9501605Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/apis.py", line 188, in wrapped
2024-02-08T21:20:46.9501970Z     return vmap_impl(func, in_dims, out_dims, randomness, chunk_size, *args, **kwargs)
2024-02-08T21:20:46.9502156Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9502820Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 279, in vmap_impl
2024-02-08T21:20:46.9502937Z     return _flat_vmap(
2024-02-08T21:20:46.9503059Z            ^^^^^^^^^^^
2024-02-08T21:20:46.9503650Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9503773Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9503892Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9504544Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 401, in _flat_vmap
2024-02-08T21:20:46.9504744Z     batched_outputs = func(*batched_inputs, **kwargs)
2024-02-08T21:20:46.9504904Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9505536Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/apis.py", line 188, in wrapped
2024-02-08T21:20:46.9505899Z     return vmap_impl(func, in_dims, out_dims, randomness, chunk_size, *args, **kwargs)
2024-02-08T21:20:46.9506089Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9506743Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 279, in vmap_impl
2024-02-08T21:20:46.9506931Z     return _flat_vmap(
2024-02-08T21:20:46.9507033Z            ^^^^^^^^^^^
2024-02-08T21:20:46.9507623Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9507757Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9507867Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9508520Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 401, in _flat_vmap
2024-02-08T21:20:46.9508737Z     batched_outputs = func(*batched_inputs, **kwargs)
2024-02-08T21:20:46.9508882Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9509303Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 303, in wrapped
2024-02-08T21:20:46.9509573Z     primals_out, tangents_out = jvp(fn, primals_in, tangents_in)
2024-02-08T21:20:46.9509736Z                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9510445Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/eager_transforms.py", line 927, in jvp
2024-02-08T21:20:46.9510884Z     return _jvp_with_argnums(func, primals, tangents, argnums=None, strict=strict, has_aux=has_aux)
2024-02-08T21:20:46.9511131Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9511738Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9511859Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9511965Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9512755Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/eager_transforms.py", line 976, in _jvp_with_argnums
2024-02-08T21:20:46.9512882Z     result_duals = func(*duals)
2024-02-08T21:20:46.9513002Z                    ^^^^^^^^^^^^
2024-02-08T21:20:46.9513418Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 130, in wrapped
2024-02-08T21:20:46.9513548Z     result = f(*_args, **kwargs)
2024-02-08T21:20:46.9513670Z              ^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9514444Z RuntimeError: Calling torch.linalg.lu_factor on a CPU tensor requires compiling PyTorch with LAPACK. Please use PyTorch built with LAPACK support.
2024-02-08T21:20:46.9514452Z 
2024-02-08T21:20:46.9514796Z To execute this test, run the following from the base repo dir:
2024-02-08T21:20:46.9515219Z      python test/functorch/test_ops.py -k test_vmapvmapjvp_linalg_solve_cpu
2024-02-08T21:20:46.9515228Z 
2024-02-08T21:20:46.9515573Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
2024-02-08T21:20:46.9515786Z =================================== FAILURES ===================================
2024-02-08T21:20:46.9516119Z ______________ TestOperatorsCPU.test_vmapvmapjvp_linalg_solve_cpu ______________
2024-02-08T21:20:46.9516261Z Traceback (most recent call last):
2024-02-08T21:20:46.9516973Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 1885, in test_vmapvmapjvp_linalg_solve
2024-02-08T21:20:46.9517314Z     batched_out = vmap(vmap(fn, in_dims=in_dims_all), in_dims=in_dims_all)(*args)
2024-02-08T21:20:46.9517494Z                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9518163Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/apis.py", line 188, in wrapped
2024-02-08T21:20:46.9518530Z     return vmap_impl(func, in_dims, out_dims, randomness, chunk_size, *args, **kwargs)
2024-02-08T21:20:46.9518727Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9519373Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 279, in vmap_impl
2024-02-08T21:20:46.9519489Z     return _flat_vmap(
2024-02-08T21:20:46.9519607Z            ^^^^^^^^^^^
2024-02-08T21:20:46.9520196Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9520408Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9520530Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9521181Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 401, in _flat_vmap
2024-02-08T21:20:46.9521388Z     batched_outputs = func(*batched_inputs, **kwargs)
2024-02-08T21:20:46.9521549Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9522184Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/apis.py", line 188, in wrapped
2024-02-08T21:20:46.9522566Z     return vmap_impl(func, in_dims, out_dims, randomness, chunk_size, *args, **kwargs)
2024-02-08T21:20:46.9522749Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9523398Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 279, in vmap_impl
2024-02-08T21:20:46.9523534Z     return _flat_vmap(
2024-02-08T21:20:46.9523641Z            ^^^^^^^^^^^
2024-02-08T21:20:46.9524233Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9524374Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9524481Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9525336Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 401, in _flat_vmap
2024-02-08T21:20:46.9525546Z     batched_outputs = func(*batched_inputs, **kwargs)
2024-02-08T21:20:46.9525686Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9526102Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 303, in wrapped
2024-02-08T21:20:46.9526348Z     primals_out, tangents_out = jvp(fn, primals_in, tangents_in)
2024-02-08T21:20:46.9526504Z                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9527192Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/eager_transforms.py", line 927, in jvp
2024-02-08T21:20:46.9527624Z     return _jvp_with_argnums(func, primals, tangents, argnums=None, strict=strict, has_aux=has_aux)
2024-02-08T21:20:46.9527819Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9528414Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 45, in fn
2024-02-08T21:20:46.9528534Z     return f(*args, **kwargs)
2024-02-08T21:20:46.9528640Z            ^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9529416Z   File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/_functorch/eager_transforms.py", line 976, in _jvp_with_argnums
2024-02-08T21:20:46.9529541Z     result_duals = func(*duals)
2024-02-08T21:20:46.9529658Z                    ^^^^^^^^^^^^
2024-02-08T21:20:46.9530060Z   File "/var/lib/jenkins/workspace/test/functorch/test_ops.py", line 130, in wrapped
2024-02-08T21:20:46.9530184Z     result = f(*_args, **kwargs)
2024-02-08T21:20:46.9530308Z              ^^^^^^^^^^^^^^^^^^^
2024-02-08T21:20:46.9531052Z RuntimeError: Calling torch.linalg.lu_factor on a CPU tensor requires compiling PyTorch with LAPACK. Please use PyTorch built with LAPACK support.
2024-02-08T21:20:46.9531061Z 
2024-02-08T21:20:46.9531324Z To execute this test, run the following from the base repo dir:
2024-02-08T21:20:46.9531728Z      python test/functorch/test_ops.py -k test_vmapvmapjvp_linalg_solve_cpu
2024-02-08T21:20:46.9531734Z 
2024-02-08T21:20:46.9532067Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
2024-02-08T21:20:46.9532965Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/functorch.test_ops/functorch.test_ops-de5d64171e2728c1.xml -
2024-02-08T21:20:46.9533184Z =========================== short test summary info ============================
2024-02-08T21:20:46.9534602Z FAILED [0.0228s] functorch/test_ops.py::TestOperatorsCPU::test_vmapvmapjvp_linalg_solve_cpu - RuntimeError: Calling torch.linalg.lu_factor on a CPU tensor requires compiling PyTorch with LAPACK. Please use PyTorch built with LAPACK support.
2024-02-08T21:20:46.9534698Z 
2024-02-08T21:20:46.9534977Z To execute this test, run the following from the base repo dir:
2024-02-08T21:20:46.9535367Z      python test/functorch/test_ops.py -k test_vmapvmapjvp_linalg_solve_cpu
2024-02-08T21:20:46.9535373Z 
2024-02-08T21:20:46.9535726Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
2024-02-08T21:20:46.9546398Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
2024-02-08T21:20:46.9546852Z = 1 failed, 1124 passed, 497 skipped, 53 xfailed, 2 rerun in 460.39s (0:07:40) =
2024-02-08T21:20:46.9546979Z Got exit code 1
2024-02-08T21:20:46.9547084Z Retrying...
2024-02-08T21:20:46.9547826Z Test results will be stored in test-reports/python-pytest/functorch.test_ops/functorch.test_ops-a009c67fa483d74d.xml
2024-02-08T21:20:46.9548057Z ============================= test session starts ==============================
2024-02-08T21:20:46.9548601Z platform linux -- Python 3.12.1, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.12/bin/python
2024-02-08T21:20:46.9548725Z cachedir: .pytest_cache
2024-02-08T21:20:46.9549459Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow]
2024-02-08T21:20:46.9549742Z rootdir: /var/lib/jenkins/workspace
2024-02-08T21:20:46.9549875Z configfile: pytest.ini
2024-02-08T21:20:46.9550588Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-13.0, xdist-3.3.1, xdoctest-1.1.0
2024-02-08T21:20:46.9550867Z collecting ... collected 9878 items / 1674 deselected / 8204 selected
2024-02-08T21:20:46.9551052Z stepcurrent: skipping 1674 already run items.
2024-02-08T21:20:46.9551176Z Running 1 items in this shard
```

Current Workaround is to add:
```
# Set Additional library and include path for py3.12
  if [ "$ANACONDA_PYTHON_VERSION" = "3.12" ]; then
    export CMAKE_LIBRARY_PATH="/opt/conda/envs/py_$ANACONDA_PYTHON_VERSION/lib/"
    export CMAKE_INCLUDE_PATH="/opt/conda/envs/py_$ANACONDA_PYTHON_VERSION/include/"
  fi
```
to: ``.ci/pytorch/build.sh``

### Versions

2.3.0 nightly

cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @malfet @seemethere @pytorch/pytorch-dev-infra