Unclear how to change compiler used by `torch.compile`
Labels: module: docs, triaged, oncall: pt2

### 📚 The doc issue

It is not clear from https://pytorch.org/tutorials//intermediate/torch_compile_tutorial.html, nor from the docs in `torch.compile`, nor even from looking through `_dynamo/config.py`, how one can change the compiler used by PyTorch.

Right now I am seeing the following issue. My code:

```python
import torch

@torch.compile
def f(x):
    return 0.5 * x

f(torch.tensor(1.0))
```

<details><summary>This produces the following error message (click to toggle):</summary>

```
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/codecache.py:445, in CppCodeCache.load(cls, source_code)
    444 try:
--> 445     subprocess.check_output(cmd, stderr=subprocess.STDOUT)
    446 except subprocess.CalledProcessError as e:

File /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py:421, in check_output(timeout, *popenargs, **kwargs)
    419     kwargs['input'] = empty
--> 421 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    422            **kwargs).stdout

File /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py:526, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    525     if check and retcode:
--> 526         raise CalledProcessError(retcode, process.args,
    527                                  output=stdout, stderr=stderr)
    528 return CompletedProcess(process.args, retcode, stdout, stderr)

CalledProcessError: Command '['g++', '/tmp/torchinductor_mcranmer/p4/cp42uf272g2qggmogzazkui7he4vnm4ftyfi2ghvyudtmaxxi25x.cpp', '-shared', '-fPIC', '-Wall', '-std=c++17', '-Wno-unused-variable', '-I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include', '-I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/torch/csrc/api/include', '-I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/TH', '-I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/THC', '-I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10', '-lgomp', '-march=native', '-O3', '-ffast-math', '-fno-finite-math-only', '-fopenmp', '-D', 'C10_USING_CUSTOM_GENERATED_MACROS', '-o/tmp/torchinductor_mcranmer/p4/cp42uf272g2qggmogzazkui7he4vnm4ftyfi2ghvyudtmaxxi25x.so']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

CppCompileError                           Traceback (most recent call last)
File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:676, in OutputGraph.call_user_compiler(self, gm)
    675 else:
--> 676     compiled_fn = compiler_fn(gm, self.fake_example_inputs())
    677 _step_logger()(logging.INFO, f"done compiler function {name}")

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py:1032, in wrap_backend_debug.<locals>.debug_wrapper(gm, example_inputs, **kwargs)
   1031 else:
-> 1032     compiled_gm = compiler_fn(gm, example_inputs, **kwargs)
   1034 return compiled_gm

File ~/venvs/main/lib/python3.10/site-packages/torch/__init__.py:1190, in _TorchCompileInductorWrapper.__call__(self, model_, inputs_)
   1189 with self.cm:
-> 1190     return self.compile_fn(model_, inputs_)

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/compile_fx.py:398, in compile_fx(model_, example_inputs_, inner_compile)
    393 with overrides.patch_functions():
    394 
    395     # TODO: can add logging before/after the call to create_aot_dispatcher_function
    396     # in torch._functorch/aot_autograd.py::aot_module_simplified::aot_function_simplified::new_func
    397     # once torchdynamo is merged into pytorch
--> 398     return aot_autograd(
    399         fw_compiler=fw_compiler,
    400         bw_compiler=bw_compiler,
    401         decompositions=select_decomp_table(),
    402         partition_fn=functools.partial(
    403             min_cut_rematerialization_partition, compiler="inductor"
    404         ),
    405     )(model_, example_inputs_)

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/optimizations/training.py:78, in aot_autograd.<locals>.compiler_fn(gm, example_inputs)
     77 with enable_aot_logging():
---> 78     cg = aot_module_simplified(gm, example_inputs, **kwargs)
     79     counters["aot_autograd"]["ok"] += 1

File ~/venvs/main/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:2355, in aot_module_simplified(mod, args, fw_compiler, bw_compiler, partition_fn, decompositions, hasher_type, static_argnums)
   2353 full_args.extend(args)
-> 2355 compiled_fn = create_aot_dispatcher_function(
   2356     functional_call,
   2357     full_args,
   2358     aot_config,
   2359 )
   2361 # TODO: There is something deeply wrong here; compiled_fn running with
   2362 # the boxed calling convention, but aot_module_simplified somehow
   2363 # historically returned a function that was not the boxed calling
   2364 # convention.  This should get fixed...

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/utils.py:88, in dynamo_timed.<locals>.time_wrapper(*args, **kwargs)
     87 t0 = time.time()
---> 88 r = func(*args, **kwargs)
     89 latency = time.time() - t0

File ~/venvs/main/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:2052, in create_aot_dispatcher_function(flat_fn, flat_args, aot_config)
   2050 # You can put more passes here
-> 2052 compiled_fn = compiler_fn(flat_fn, fake_flat_tensor_args, aot_config)
   2054 if not hasattr(compiled_fn, '_boxed_call'):

File ~/venvs/main/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:1307, in aot_wrapper_dedupe(flat_fn, flat_args, aot_config, compiler_fn)
   1306     if ok:
-> 1307         return compiler_fn(flat_fn, leaf_flat_args, aot_config)
   1309 # Strategy 2: Duplicate specialize.
   1310 #
   1311 # In Haskell types, suppose you have:
   (...)
   1343 #   }
   1344 #   keep_arg_mask = [True, True, False, True]

File ~/venvs/main/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:957, in aot_dispatch_base(flat_fn, flat_args, aot_config)
    956 with context(), track_graph_compiling(aot_config, "inference"):
--> 957     compiled_fw = aot_config.fw_compiler(fw_module, flat_args)
    959 @wraps(compiled_fw)
    960 def new_fn(args):

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/utils.py:88, in dynamo_timed.<locals>.time_wrapper(*args, **kwargs)
     87 t0 = time.time()
---> 88 r = func(*args, **kwargs)
     89 latency = time.time() - t0

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/compile_fx.py:373, in compile_fx.<locals>.fw_compiler(model, example_inputs)
    372 fixed = len(example_inputs) - num_example_inputs
--> 373 return inner_compile(
    374     model,
    375     example_inputs,
    376     num_fixed=fixed,
    377     cudagraphs=cudagraphs,
    378     graph_id=graph_id,
    379 )

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py:588, in wrap_compiler_debug.<locals>.debug_wrapper(gm, example_inputs, **kwargs)
    587 else:
--> 588     compiled_fn = compiler_fn(gm, example_inputs, **kwargs)
    590 return compiled_fn

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/debug.py:223, in DebugContext.wrap.<locals>.inner(*args, **kwargs)
    222 with DebugContext():
--> 223     return fn(*args, **kwargs)

File /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py:79, in ContextDecorator.__call__.<locals>.inner(*args, **kwds)
     78 with self._recreate_cm():
---> 79     return func(*args, **kwds)

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/compile_fx.py:140, in compile_fx_inner(gm, example_inputs, cudagraphs, num_fixed, is_backward, graph_id)
    139     graph.run(*example_inputs)
--> 140     compiled_fn = graph.compile_to_fn()
    142 if cudagraphs:

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/graph.py:538, in GraphLowering.compile_to_fn(self)
    537 def compile_to_fn(self):
--> 538     return self.compile_to_module().call

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/utils.py:88, in dynamo_timed.<locals>.time_wrapper(*args, **kwargs)
     87 t0 = time.time()
---> 88 r = func(*args, **kwargs)
     89 latency = time.time() - t0

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/graph.py:527, in GraphLowering.compile_to_module(self)
    525     print(code)
--> 527 mod = PyCodeCache.load(code)
    528 for name, value in self.constants.items():

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/codecache.py:468, in PyCodeCache.load(cls, source_code)
    467 mod.key = key
--> 468 exec(code, mod.__dict__, mod.__dict__)
    469 # another thread might set this first

File /tmp/torchinductor_mcranmer/a3/ca3u2aiw5tfkcpspspezlhxlp4nhdcj55j5s4mdeny7h57cnudzz.py:30
     13 kernel_cpp_0 = async_compile.cpp('''
     14 #include "/tmp/torchinductor_mcranmer/77/c7773nj5pwikpmm2pwa62rcudlf7p3if7eyqb5k4sjsvewwje4le.h"
     15 extern "C" void kernel(const float* __restrict__ in_ptr0,
   (...)
     26 }
     27 ''')
---> 30 async_compile.wait(globals())
     31 del async_compile

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/codecache.py:663, in AsyncCompile.wait(self, scope)
    662 if isinstance(result, (Future, TritonFuture)):
--> 663     scope[key] = result.result()
    664     pbar.update(1)

File /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py:458, in Future.result(self, timeout)
    457 elif self._state == FINISHED:
--> 458     return self.__get_result()
    459 else:

File /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py:403, in Future.__get_result(self)
    402 try:
--> 403     raise self._exception
    404 finally:
    405     # Break a reference cycle with the exception in self._exception

File /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/thread.py:58, in _WorkItem.run(self)
     57 try:
---> 58     result = self.fn(*self.args, **self.kwargs)
     59 except BaseException as exc:

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/codecache.py:640, in AsyncCompile.cpp.<locals>.task()
    639 def task():
--> 640     return CppCodeCache.load(source_code).kernel

File ~/venvs/main/lib/python3.10/site-packages/torch/_inductor/codecache.py:447, in CppCodeCache.load(cls, source_code)
    446     except subprocess.CalledProcessError as e:
--> 447         raise exc.CppCompileError(cmd, e.output) from e
    449 cls.cache[key] = cls._load_library(output_path)

CppCompileError: C++ compile error

Command:
g++ /tmp/torchinductor_mcranmer/p4/cp42uf272g2qggmogzazkui7he4vnm4ftyfi2ghvyudtmaxxi25x.cpp -shared -fPIC -Wall -std=c++17 -Wno-unused-variable -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/TH -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/THC -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 -lgomp -march=native -O3 -ffast-math -fno-finite-math-only -fopenmp -D C10_USING_CUSTOM_GENERATED_MACROS -o/tmp/torchinductor_mcranmer/p4/cp42uf272g2qggmogzazkui7he4vnm4ftyfi2ghvyudtmaxxi25x.so

Output:
clang: error: the clang compiler does not support '-march=native'
clang: error: unsupported option '-fopenmp'
clang: error: unsupported option '-fopenmp'


The above exception was the direct cause of the following exception:

BackendCompilerFailed                     Traceback (most recent call last)
Cell In [15], line 1
----> 1 f(torch.tensor(1.0))

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:212, in _TorchDynamoContext.__call__.<locals>._fn(*args, **kwargs)
    210 dynamic_ctx.__enter__()
    211 try:
--> 212     return fn(*args, **kwargs)
    213 finally:
    214     set_eval_frame(prior)

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:333, in catch_errors_wrapper.<locals>.catch_errors(frame, cache_size)
    330             return hijacked_callback(frame, cache_size, hooks)
    332 with compile_lock:
--> 333     return callback(frame, cache_size, hooks)

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:480, in convert_frame.<locals>._convert_frame(frame, cache_size, hooks)
    478 counters["frames"]["total"] += 1
    479 try:
--> 480     result = inner_convert(frame, cache_size, hooks)
    481     counters["frames"]["ok"] += 1
    482     return result

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:103, in wrap_convert_context.<locals>._fn(*args, **kwargs)
    101 torch.fx.graph_module._forward_from_src = fx_forward_from_src_skip_result
    102 try:
--> 103     return fn(*args, **kwargs)
    104 finally:
    105     torch._C._set_grad_enabled(prior_grad_mode)

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/utils.py:88, in dynamo_timed.<locals>.time_wrapper(*args, **kwargs)
     86     compilation_metrics[key] = []
     87 t0 = time.time()
---> 88 r = func(*args, **kwargs)
     89 latency = time.time() - t0
     90 # print(f"Dynamo timer: key={key}, latency={latency:.2f} sec")

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:339, in convert_frame_assert.<locals>._convert_frame_assert(frame, cache_size, hooks)
    336 global initial_grad_state
    337 initial_grad_state = torch.is_grad_enabled()
--> 339 return _compile(
    340     frame.f_code,
    341     frame.f_globals,
    342     frame.f_locals,
    343     frame.f_builtins,
    344     compiler_fn,
    345     one_graph,
    346     export,
    347     hooks,
    348     frame,
    349 )

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:400, in _compile(code, globals, locals, builtins, compiler_fn, one_graph, export, hooks, frame)
    398 for attempt in itertools.count():
    399     try:
--> 400         out_code = transform_code_object(code, transform)
    401         orig_code_map[out_code] = code
    402         break

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py:341, in transform_code_object(code, transformations, safe)
    338 instructions = cleaned_instructions(code, safe)
    339 propagate_line_nums(instructions)
--> 341 transformations(instructions, code_options)
    343 fix_vars(instructions, code_options)
    345 dirty = True

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:387, in _compile.<locals>.transform(instructions, code_options)
    374 nonlocal output
    375 tracer = InstructionTranslator(
    376     instructions,
    377     code,
   (...)
    385     mutated_closure_cell_contents,
    386 )
--> 387 tracer.run()
    388 output = tracer.output
    389 assert output is not None

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1684, in InstructionTranslator.run(self)
   1682 def run(self):
   1683     _step_logger()(logging.INFO, f"torchdynamo start tracing {self.f_code.co_name}")
-> 1684     super().run()

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:538, in InstructionTranslatorBase.run(self)
    533 try:
    534     self.output.push_tx(self)
    535     while (
    536         self.instruction_pointer is not None
    537         and not self.output.should_exit
--> 538         and self.step()
    539     ):
    540         pass
    541 except BackendCompilerFailed:

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:501, in InstructionTranslatorBase.step(self)
    499     if not hasattr(self, inst.opname):
    500         unimplemented(f"missing: {inst.opname}")
--> 501     getattr(self, inst.opname)(inst)
    503     return inst.opname != "RETURN_VALUE"
    504 except BackendCompilerFailed:

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1750, in InstructionTranslator.RETURN_VALUE(self, inst)
   1745 _step_logger()(
   1746     logging.INFO,
   1747     f"torchdynamo done tracing {self.f_code.co_name} (RETURN_VALUE)",
   1748 )
   1749 log.debug("RETURN_VALUE triggered compile")
-> 1750 self.output.compile_subgraph(self)
   1751 self.output.add_output_instructions([create_instruction("RETURN_VALUE")])

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:529, in OutputGraph.compile_subgraph(self, tx, partial_convert, reason)
    512     self.add_output_instructions(random_calls_instructions)
    514 if (
    515     stack_values
    516     and all(
   (...)
    526 
    527     # optimization to generate better code in a common case
    528     self.add_output_instructions(
--> 529         self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
    530         + [create_instruction("UNPACK_SEQUENCE", len(stack_values))]
    531     )
    532 else:
    533     graph_output_var = self.new_var("graph_out")

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:600, in OutputGraph.compile_and_call_fx_graph(self, tx, rv, root)
    598 assert_no_fake_params_or_buffers(gm)
    599 with tracing(self.tracing_context):
--> 600     compiled_fn = self.call_user_compiler(gm)
    601 compiled_fn = disable(compiled_fn)
    603 counters["stats"]["unique_graphs"] += 1

File ~/venvs/main/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:681, in OutputGraph.call_user_compiler(self, gm)
    679 except Exception as e:
    680     compiled_fn = gm.forward
--> 681     raise BackendCompilerFailed(self.compiler_fn, e) from e
    682 return compiled_fn

BackendCompilerFailed: debug_wrapper raised CppCompileError: C++ compile error

Command:
g++ /tmp/torchinductor_mcranmer/p4/cp42uf272g2qggmogzazkui7he4vnm4ftyfi2ghvyudtmaxxi25x.cpp -shared -fPIC -Wall -std=c++17 -Wno-unused-variable -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/TH -I/Users/mcranmer/venvs/main/lib/python3.10/site-packages/torch/include/THC -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 -lgomp -march=native -O3 -ffast-math -fno-finite-math-only -fopenmp -D C10_USING_CUSTOM_GENERATED_MACROS -o/tmp/torchinductor_mcranmer/p4/cp42uf272g2qggmogzazkui7he4vnm4ftyfi2ghvyudtmaxxi25x.so

Output:
clang: error: the clang compiler does not support '-march=native'
clang: error: unsupported option '-fopenmp'
clang: error: unsupported option '-fopenmp'



You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True
```

</details>

(For the record, I am on macOS 13.1 with an M1 processor, Python 3.10.8, PyTorch 2.0)

I installed the `brew` version of clang which does in fact have `openmp` support. However I have no idea how to force this, as the compiler used by `torch.compile` seems to be defaulting to the one which comes with macOS (at `/usr/bin/g++`), which does not have `openmp` support.

### Suggest a potential alternative/fix

It would be great if the docs specify how I can tweak the compilation settings, at the very least how to specify a compiler to use.

(For what it's worth, I did try specifying `CXX` as an environment variable, which was my first reflex - it might be a good idea to simply check for this as others might try that first as well.)

cc @svekars @carljparker @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh