Repository: pytorch
Issue: Dynamo + MacOS: fatal error: 'omp.h' file not found
Labels: triaged,module: macos,oncall: pt2,oncall: cpu inductor

### 🐛 Describe the bug

`torch.compile` fails to run with MacOs with a missing "omp.h" error

libomp is available:

```bash
$ brew install libomp
Warning: libomp 15.0.7 is already installed and up-to-date.
To reinstall 15.0.7, run:
  brew reinstall libomp
```

### Error logs

Error

```python
self = <torch._dynamo.output_graph.OutputGraph object at 0x14f281850>
gm = GraphModule()

    @dynamo_timed(phase_name="backend_compile")
    def call_user_compiler(self, gm: fx.GraphModule) -> CompiledFn:
        tot = 0
        for node in gm.graph.nodes:
            if node.op in ("call_function", "call_method", "call_module"):
                tot += 1
        torch._dynamo.utils.increment_op_count(tot)
        try:
            name = (
                self.compiler_fn.__name__
                if hasattr(self.compiler_fn, "__name__")
                else ""
            )
            _step_logger()(logging.INFO, f"calling compiler function {name}")
            compiler_fn = self.compiler_fn
            # WrapperBackend needs real inputs, for now, to verify correctness
            if config.verify_correctness:
                compiler_fn = WrapperBackend(compiler_fn, self.example_inputs())
    
            # NOTE: [Real Tensors in Accuracy Evaluation]
            #
            # Today, tensors are passed to backends as fake at compile time. See the .fake_example_inputs()
            # call to compiler_fn below. At runtime, backends use real tensors.
            #
            # This should be a strong invariant we hold across all backends,
            # and generally, it is. However, for accuracy evaluation, we need real tensors at compile time,
            # for now, due to the unfortunate setup described below.
            #
            # Due to the nature of how we invoke comparison as a backend in two different ways:
            #
            # (1) Less bad, but still worth rewriting, WrapperBackend above, which takes
            # real inputs for its ctor. see the config.verify_correctnes above.
            #
            # (2) More bad, and very worth rewriting, the minifier installs accuracy comparison as
            # a true backend, and therefore needs to be compiled with real inputs. This is made trickier
            # by the fact that the minifier will spawn new processes during minification. As such, we have
            # created a global flag, MINIFIER_SPAWNED, that should be set IF AND ONLY IF this run was spawned
            # as part of accuracy minification. This flag is not a contract, and ideally will not be here long.
            #
            # The longer term PoR is to:
            # (A) Rewrite the minifier accuracy evaluation and verify_correctness code to share the same
            # correctness and accuracy logic, so as not to have two different ways of doing the same thing.
            #
            # (B) Refactor minifier accuracy backend to do its comparison fully at runtime, so as not to need to
            # pass real tensors to it at compile time.
            is_top_level_minifying = (
                config.repro_after is not None and config.repro_level == 4
            )
            if torch._dynamo.debug_utils.MINIFIER_SPAWNED or is_top_level_minifying:
                compiled_fn = compiler_fn(gm, self.example_inputs())
            elif config.DO_NOT_USE_legacy_non_fake_example_inputs:
                compiled_fn = compiler_fn(gm, self.example_inputs())
            else:
                compiled_fn = compiler_fn(gm, self.fake_example_inputs())
            _step_logger()(logging.INFO, f"done compiler function {name}")
            assert callable(compiled_fn), "compiler_fn did not return callable"
        except Exception as e:
            compiled_fn = gm.forward
>           raise BackendCompilerFailed(self.compiler_fn, e) from e
E           torch._dynamo.exc.BackendCompilerFailed: debug_wrapper raised CppCompileError: C++ compile error
E           
E           Command:
E           g++ /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/torchinductor_runner/xv/cxvjcdvblebtspcwgl5nyt4o3rch57rodfgm4uv63hhc2bqpkvtj.cpp -shared -fPIC -Wall -std=c++17 -Wno-unused-variable -I/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/torch/include -I/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/torch/include/TH -I/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/torch/include/THC -I/Users/runner/hostedtoolcache/Python/3.9.16/x64/include/python3.9 -lomp -O3 -ffast-math -fno-finite-math-only -Xclang -fopenmp -D C10_USING_CUSTOM_GENERATED_MACROS -o/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/torchinductor_runner/xv/cxvjcdvblebtspcwgl5nyt4o3rch57rodfgm4uv63hhc2bqpkvtj.so
E           
E           Output:
E           In file included from /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/torchinductor_runner/xv/cxvjcdvblebtspcwgl5nyt4o3rch57rodfgm4uv63hhc2bqpkvtj.cpp:2:
E           /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/torchinductor_runner/zt/cztcl2vp5yqlnhofzpqfficjcxgyict6e3xhfdd7sdbkipp4p44x.h:6:10: fatal error: 'omp.h' file not found
E           #include <omp.h>
E                    ^~~~~~~
E           1 error generated.
E           
E           
E           Set torch._dynamo.config.verbose=True for more information
E           
E           
E           You can suppress this exception and fall back to eager by setting:
E               torch._dynamo.config.suppress_errors = True

/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/torch/_dynamo/output_graph.py:675: BackendCompilerFailed
```


[Full stacktrace (too long to include)](https://github.com/pytorch/pytorch/files/10852078/stacktrace.txt)


### Minified repro

I have yet to create a minimal repro, but I can quickly try things as this reproduces in the Lightning CI test suite.

I would expect that this is not an issue with the actual code being run, but instead with the MacOS distribution for dynamo.

If you would still like that I provide a reproducible snippet, please ask and I will do so.

Am I missing a specific dependency? Help to find the correct incantation is appreciated.

### Versions

torch==2.0.0 (release candidate)


<details>
<summary>GitHub action workflow details</summary>

```python
Current runner version: '2.302.1'
Operating System
  macOS
  11.7.4
  [2](https://github.com/Lightning-AI/lightning/actions/runs/4294336080/jobs/7483240118#step:1:2)0G1120
Runner Image
  Image: macos-11
  Version: 202[3](https://github.com/Lightning-AI/lightning/actions/runs/4294336080/jobs/7483240118#step:1:3)0219.1
  Included Software: https://github.com/actions/runner-images/blob/macOS-11/20230219.1/images/macos/macos-11-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/macOS-11%2F20230219.1
Runner Image Provisioner
  2.0.11[7](https://github.com/Lightning-AI/lightning/actions/runs/4294336080/jobs/7483240118#step:1:8).1
```

</details>

cc @malfet @albanD @ezyang @msaroufim @bdhirsh @anijain2305 @zou3519 @chauhang @wconstab @soumith @ngimel