"PTX .version 7.4 does not support .target sm_89" Again with latest Pytorch2.0, 4090 with CUDA12 and Ubuntu Linux 22.04

Beck_Guo: <p>I am tring to run the below code within jupyter notebook in my envirenment:<br>
<span class="hashtag">#---------------------------------------------------</span><br>
import torch<br>
import torchvision<br>
torch.set_float32_matmul_precision(‘high’)<br>
m = torchvision.models.resnet50().cuda()<br>
mm = torch.compile(m)</p>
<p>data = torch.rand(1, 3, 224, 224).cuda()<br>
o = mm(data)<br>
<span class="hashtag">#---------------------------------------------------</span></p>
<h2>
<a name="but-i-got-the-following-error-message-1" class="anchor" href="#but-i-got-the-following-error-message-1"></a>But I got the following error message:</h2>
<p>_RemoteTraceback                          Traceback (most recent call last)<br>
_RemoteTraceback:<br>
“”"<br>
Traceback (most recent call last):<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/concurrent/futures/process.py”, line 246, in _process_worker<br>
r = call_item.fn(*call_item.args, **call_item.kwargs)<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/codecache.py”, line 525, in _worker_compile<br>
kernel.precompile(warm_cache_only_with_cc=cc)<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/triton_ops/autotune.py”, line 67, in precompile<br>
self.launchers = [<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/triton_ops/autotune.py”, line 68, in <br>
self._precompile_config(c, warm_cache_only_with_cc)<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/triton_ops/autotune.py”, line 81, in _precompile_config<br>
triton.compile(<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/site-packages/triton/compiler.py”, line 1256, in compile<br>
asm, shared, kernel_name = _compile(fn, signature, device, constants, configs[0], num_warps, num_stages,<br>
File “/home/beck/miniconda3/envs/pytorch2/lib/python3.10/site-packages/triton/compiler.py”, line 901, in _compile<br>
name, asm, shared_mem = _triton.code_gen.compile_ttir(backend, module, device, num_warps, num_stages, extern_libs, cc)<br>
RuntimeError: Internal Triton PTX codegen error:<br>
ptxas /tmp/fileC5RW8H, line 6; error   : PTX .version 7.4 does not support .target sm_89<br>
ptxas fatal   : Ptx assembly aborted due to errors</p>
<p>“”"</p>
<p>The above exception was the direct cause of the following exception:<br>
[/details]</p>
<details>
<summary>
More message</summary>
<p>RuntimeError                              Traceback (most recent call last)<br>
File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:692, in OutputGraph.call_user_compiler(self, gm)<br>
691 else:<br>
 → 692     compiled_fn = compiler_fn(gm, self.fake_example_inputs())<br>
693 _step_logger()(logging.INFO, f"done compiler function {name}")</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py:1054, in wrap_backend_debug..debug_wrapper(gm, example_inputs, **kwargs)<br>
1053 else:<br>
 → 1054     compiled_gm = compiler_fn(gm, example_inputs)<br>
1056 return compiled_gm</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/<strong>init</strong>.py:1368, in <em>TorchCompileInductorWrapper.<strong>call</strong>(self, model</em>, inputs_)<br>
1367 def <strong>call</strong>(self, model_, inputs_):<br>
 → 1368     return self.compile_fn(model_, inputs_, config_patches=self.config)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/<em>inductor/compile_fx.py:426, in compile_fx(model</em>, example_inputs_, inner_compile, config_patches)<br>
421 with overrides.patch_functions():<br>
422<br>
423     # TODO: can add logging before/after the call to create_aot_dispatcher_function<br>
424     # in torch.<em>functorch/aot_autograd.py::aot_module_simplified::aot_function_simplified::new_func<br>
425     # once torchdynamo is merged into pytorch<br>
 → 426     return aot_autograd(<br>
427         fw_compiler=fw_compiler,<br>
428         bw_compiler=bw_compiler,<br>
429         decompositions=select_decomp_table(),<br>
430         partition_fn=functools.partial(<br>
431             min_cut_rematerialization_partition, compiler=“inductor”<br>
432         ),<br>
433     )(model</em>, example_inputs_)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/optimizations/training.py:66, in aot_autograd..compiler_fn(gm, example_inputs)<br>
65 with enable_aot_logging():<br>
—&gt; 66     cg = aot_module_simplified(gm, example_inputs, **kwargs)<br>
67     counters[“aot_autograd”][“ok”] += 1</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:2483, in aot_module_simplified(mod, args, fw_compiler, bw_compiler, partition_fn, decompositions, hasher_type, static_argnums)<br>
2481 full_args.extend(args)<br>
 → 2483 compiled_fn = create_aot_dispatcher_function(<br>
2484     functional_call,<br>
2485     full_args,<br>
2486     aot_config,<br>
2487 )<br>
2489 # TODO: There is something deeply wrong here; compiled_fn running with<br>
2490 # the boxed calling convention, but aot_module_simplified somehow<br>
2491 # historically returned a function that was not the boxed calling<br>
2492 # convention.  This should get fixed…</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/utils.py:162, in dynamo_timed..dynamo_timed_inner..time_wrapper(*args, **kwargs)<br>
161 t0 = time.time()<br>
 → 162 r = func(*args, **kwargs)<br>
163 time_spent = time.time() - t0</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:2180, in create_aot_dispatcher_function(flat_fn, flat_args, aot_config)<br>
2178 # You can put more passes here<br>
 → 2180 compiled_fn = compiler_fn(flat_fn, fake_flat_args, aot_config)<br>
2182 if not hasattr(compiled_fn, “_boxed_call”):</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:1411, in aot_wrapper_dedupe(flat_fn, flat_args, aot_config, compiler_fn)<br>
1410     if ok:<br>
 → 1411         return compiler_fn(flat_fn, leaf_flat_args, aot_config)<br>
1413 # Strategy 2: Duplicate specialize.<br>
1414 #<br>
1415 # In Haskell types, suppose you have:<br>
(…)<br>
1447 #   }<br>
1448 #   keep_arg_mask = [True, True, False, True]</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py:1688, in aot_dispatch_autograd(flat_fn, flat_args, aot_config)<br>
1687     with track_graph_compiling(aot_config, “forward”):<br>
 → 1688         compiled_fw_func = aot_config.fw_compiler(<br>
1689             fw_module, flat_args_with_views_handled<br>
1690         )<br>
1692 class CompiledFunction(torch.autograd.Function):</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/utils.py:162, in dynamo_timed..dynamo_timed_inner..time_wrapper(*args, **kwargs)<br>
161 t0 = time.time()<br>
 → 162 r = func(*args, **kwargs)<br>
163 time_spent = time.time() - t0</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/compile_fx.py:401, in compile_fx..fw_compiler(model, example_inputs)<br>
400 fixed = len(example_inputs) - num_example_inputs<br>
 → 401 return inner_compile(<br>
402     model,<br>
403     example_inputs,<br>
404     num_fixed=fixed,<br>
405     cudagraphs=cudagraphs,<br>
406     graph_id=graph_id,<br>
407 )</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py:594, in wrap_compiler_debug..debug_wrapper(gm, example_inputs, **kwargs)<br>
593 else:<br>
 → 594     compiled_fn = compiler_fn(gm, example_inputs)<br>
596 return compiled_fn</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/debug.py:239, in DebugContext.wrap..inner(*args, **kwargs)<br>
238 with DebugContext():<br>
 → 239     return fn(*args, **kwargs)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/contextlib.py:79, in ContextDecorator.<strong>call</strong>..inner(*args, **kwds)<br>
78 with self._recreate_cm():<br>
—&gt; 79     return func(*args, **kwds)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/compile_fx.py:155, in compile_fx_inner(gm, example_inputs, cudagraphs, num_fixed, is_backward, graph_id)<br>
154         graph.run(*example_inputs)<br>
 → 155         compiled_fn = graph.compile_to_fn()<br>
157 if cudagraphs:</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/graph.py:570, in GraphLowering.compile_to_fn(self)<br>
569 def compile_to_fn(self):<br>
 → 570     return self.compile_to_module().call</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/utils.py:162, in dynamo_timed..dynamo_timed_inner..time_wrapper(*args, **kwargs)<br>
161 t0 = time.time()<br>
 → 162 r = func(*args, **kwargs)<br>
163 time_spent = time.time() - t0</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/graph.py:559, in GraphLowering.compile_to_module(self)<br>
557     print(code)<br>
 → 559 mod = PyCodeCache.load(code)<br>
560 for name, value in self.constants.items():</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/codecache.py:504, in PyCodeCache.load(cls, source_code)<br>
503 mod.key = key<br>
 → 504 exec(code, mod.<strong>dict</strong>, mod.<strong>dict</strong>)<br>
505 # another thread might set this first</p>
<p>File /tmp/torchinductor_beck/vc/cvcdtwwo2tcoz6j2i46wspset3tq3xzax6hyw247h6bpr664ay5o.py:2032<br>
2009 triton__23 = async_compile.triton(‘’’<br>
2010 import triton<br>
2011 import triton.language as tl<br>
(…)<br>
2028     tl.store(out_ptr0 + (0 + tl.zeros([XBLOCK], tl.int32)), tmp2, None)<br>
2029 ‘’')<br>
 → 2032 async_compile.wait(globals())<br>
2033 del async_compile</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/codecache.py:691, in AsyncCompile.wait(self, scope)<br>
690 if isinstance(result, (Future, TritonFuture)):<br>
 → 691     scope[key] = result.result()<br>
692     pbar.update(1)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_inductor/codecache.py:549, in TritonFuture.result(self)<br>
548 # If the worker failed this will throw an exception.<br>
 → 549 self.future.result()<br>
550 kernel = self.kernel = _load_kernel(self.source_code)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/concurrent/futures/_base.py:458, in Future.result(self, timeout)<br>
457 elif self._state == FINISHED:<br>
 → 458     return self.__get_result()<br>
459 else:</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/concurrent/futures/_base.py:403, in Future.__get_result(self)<br>
402 try:<br>
 → 403     raise self._exception<br>
404 finally:<br>
405     # Break a reference cycle with the exception in self._exception</p>
<p>RuntimeError: Internal Triton PTX codegen error:<br>
ptxas /tmp/fileC5RW8H, line 6; error   : PTX .version 7.4 does not support .target sm_89<br>
ptxas fatal   : Ptx assembly aborted due to errors</p>
<p>The above exception was the direct cause of the following exception:</p>
<p>BackendCompilerFailed                     Traceback (most recent call last)<br>
Cell In[2], line 8<br>
5 mm = torch.compile(m)<br>
7 data = torch.rand(1, 3, 224, 224).cuda()<br>
----&gt; 8 o = mm(data)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)<br>
1496 # If we don’t have any hooks, we want to skip the rest of the logic in<br>
1497 # this function, and just call forward.<br>
1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks<br>
1499         or _global_backward_pre_hooks or _global_backward_hooks<br>
1500         or _global_forward_hooks or _global_forward_pre_hooks):<br>
 → 1501     return forward_call(*args, **kwargs)<br>
1502 # Do not call functions when jit is used<br>
1503 full_backward_hooks, non_full_backward_hooks = [], []</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:82, in OptimizedModule.forward(self, *args, **kwargs)<br>
81 def forward(self, *args, **kwargs):<br>
—&gt; 82     return self.dynamo_ctx(self._orig_mod.forward)(*args, **kwargs)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:209, in _TorchDynamoContext.<strong>call</strong>.._fn(*args, **kwargs)<br>
207 dynamic_ctx.<strong>enter</strong>()<br>
208 try:<br>
 → 209     return fn(*args, **kwargs)<br>
210 finally:<br>
211     set_eval_frame(prior)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:330, in catch_errors_wrapper..catch_errors(frame, cache_size)<br>
327             return hijacked_callback(frame, cache_size, hooks)<br>
329 with compile_lock:<br>
 → 330     return callback(frame, cache_size, hooks)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:403, in convert_frame.._convert_frame(frame, cache_size, hooks)<br>
401 counters[“frames”][“total”] += 1<br>
402 try:<br>
 → 403     result = inner_convert(frame, cache_size, hooks)<br>
404     counters[“frames”][“ok”] += 1<br>
405     return result</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:103, in wrap_convert_context.._fn(*args, **kwargs)<br>
101 torch.fx.graph_module._forward_from_src = fx_forward_from_src_skip_result<br>
102 try:<br>
 → 103     return fn(*args, **kwargs)<br>
104 finally:<br>
105     torch._C._set_grad_enabled(prior_grad_mode)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:261, in convert_frame_assert.._convert_frame_assert(frame, cache_size, hooks)<br>
258 global initial_grad_state<br>
259 initial_grad_state = torch.is_grad_enabled()<br>
 → 261 return _compile(<br>
262     frame.f_code,<br>
263     frame.f_globals,<br>
264     frame.f_locals,<br>
265     frame.f_builtins,<br>
266     compiler_fn,<br>
267     one_graph,<br>
268     export,<br>
269     hooks,<br>
270     frame,<br>
271 )</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/utils.py:162, in dynamo_timed..dynamo_timed_inner..time_wrapper(*args, **kwargs)<br>
160     compilation_metrics[key] = []<br>
161 t0 = time.time()<br>
 → 162 r = func(*args, **kwargs)<br>
163 time_spent = time.time() - t0<br>
164 # print(f"Dynamo timer: key={key}, latency={latency:.2f} sec")</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:323, in _compile(code, globals, locals, builtins, compiler_fn, one_graph, export, hooks, frame)<br>
321 for attempt in itertools.count():<br>
322     try:<br>
 → 323         out_code = transform_code_object(code, transform)<br>
324         orig_code_map[out_code] = code<br>
325         break</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py:339, in transform_code_object(code, transformations, safe)<br>
336 instructions = cleaned_instructions(code, safe)<br>
337 propagate_line_nums(instructions)<br>
 → 339 transformations(instructions, code_options)<br>
341 fix_vars(instructions, code_options)<br>
343 dirty = True</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py:310, in _compile..transform(instructions, code_options)<br>
297 nonlocal output<br>
298 tracer = InstructionTranslator(<br>
299     instructions,<br>
300     code,<br>
(…)<br>
308     mutated_closure_cell_contents,<br>
309 )<br>
 → 310 tracer.run()<br>
311 output = tracer.output<br>
312 assert output is not None</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1715, in InstructionTranslator.run(self)<br>
1713 def run(self):<br>
1714     _step_logger()(logging.INFO, f"torchdynamo start tracing {self.f_code.co_name}")<br>
 → 1715     super().run()</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:564, in InstructionTranslatorBase.run(self)<br>
559 try:<br>
560     self.output.push_tx(self)<br>
561     while (<br>
562         self.instruction_pointer is not None<br>
563         and not self.output.should_exit<br>
 → 564         and self.step()<br>
565     ):<br>
566         pass<br>
567 except BackendCompilerFailed:</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:527, in InstructionTranslatorBase.step(self)<br>
525     if not hasattr(self, inst.opname):<br>
526         unimplemented(f"missing: {inst.opname}")<br>
 → 527     getattr(self, inst.opname)(inst)<br>
529     return inst.opname != “RETURN_VALUE”<br>
530 except BackendCompilerFailed:</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py:1781, in InstructionTranslator.RETURN_VALUE(self, inst)<br>
1776 _step_logger()(<br>
1777     logging.INFO,<br>
1778     f"torchdynamo done tracing {self.f_code.co_name} (RETURN_VALUE)",<br>
1779 )<br>
1780 log.debug(“RETURN_VALUE triggered compile”)<br>
 → 1781 self.output.compile_subgraph(self)<br>
1782 self.output.add_output_instructions([create_instruction(“RETURN_VALUE”)])</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:539, in OutputGraph.compile_subgraph(self, tx, partial_convert, reason)<br>
525     self.add_output_instructions(random_calls_instructions)<br>
527 if (<br>
528     stack_values<br>
529     and all(<br>
(…)<br>
536<br>
537     # optimization to generate better code in a common case<br>
538     self.add_output_instructions(<br>
 → 539         self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)<br>
540         + [create_instruction(“UNPACK_SEQUENCE”, len(stack_values))]<br>
541     )<br>
542 else:<br>
543     graph_output_var = self.new_var(“graph_out”)</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:610, in OutputGraph.compile_and_call_fx_graph(self, tx, rv, root)<br>
608 assert_no_fake_params_or_buffers(gm)<br>
609 with tracing(self.tracing_context):<br>
 → 610     compiled_fn = self.call_user_compiler(gm)<br>
611 compiled_fn = disable(compiled_fn)<br>
613 counters[“stats”][“unique_graphs”] += 1</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/utils.py:162, in dynamo_timed..dynamo_timed_inner..time_wrapper(*args, **kwargs)<br>
160     compilation_metrics[key] = []<br>
161 t0 = time.time()<br>
 → 162 r = func(*args, **kwargs)<br>
163 time_spent = time.time() - t0<br>
164 # print(f"Dynamo timer: key={key}, latency={latency:.2f} sec")</p>
<p>File ~/miniconda3/envs/pytorch2/lib/python3.10/site-packages/torch/_dynamo/output_graph.py:697, in OutputGraph.call_user_compiler(self, gm)<br>
695 except Exception as e:<br>
696     compiled_fn = gm.forward<br>
 → 697     raise BackendCompilerFailed(self.compiler_fn, e) from e<br>
698 return compiled_fn</p>
<p>BackendCompilerFailed: debug_wrapper raised RuntimeError: Internal Triton PTX codegen error:<br>
ptxas /tmp/fileC5RW8H, line 6; error   : PTX .version 7.4 does not support .target sm_89<br>
ptxas fatal   : Ptx assembly aborted due to errors</p>
<p>Set torch._dynamo.config.verbose=True for more information</p>
<p>You can suppress this exception and fall back to eager by setting:<br>
torch._dynamo.config.suppress_errors = True</p>
</details>
<p>[/details]</p>

Beck_Guo: <p>Here are environment informaton:<br>
(pytorch2) beck@beckserver:/home$ ptxas --version<br>
ptxas: NVIDIA (R) Ptx optimizing assembler<br>
Copyright (c) 2005-2023 NVIDIA Corporation<br>
Built on Fri_Jan__6_16:43:29_PST_2023<br>
Cuda compilation tools, release 12.0, V12.0.140<br>
Build cuda_12.0.r12.0/compiler.32267302_0</p>
<p>/home$ python -m torch.utils.collect_env<br>
Collecting environment information…<br>
PyTorch version: 2.0.0.dev20230202+cu117<br>
Is debug build: False<br>
CUDA used to build PyTorch: 11.7<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 22.04.1 LTS (x86_64)<br>
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0<br>
Clang version: Could not collect<br>
CMake version: version 3.25.0<br>
Libc version: glibc-2.35</p>
<p>Python version: 3.10.9 (main, Jan 11 2023, 15:21:40) [GCC 11.2.0] (64-bit runtime)<br>
Python platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.35<br>
Is CUDA available: True<br>
CUDA runtime version: 12.0.140<br>
CUDA_MODULE_LOADING set to: LAZY<br>
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090<br>
Nvidia driver version: 525.85.12<br>
cuDNN version: Could not collect<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A<br>
Is XNNPACK available: True</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.24.1<br>
[pip3] pytorch-triton==2.0.0+0d7e753227<br>
[pip3] torch==2.0.0.dev20230202+cu117<br>
[pip3] torchaudio==2.0.0.dev20230202+cu117<br>
[pip3] torchvision==0.15.0.dev20230202+cu117<br>
[conda] blas                      1.0                         mkl<br>
[conda] mkl                       2021.4.0           h06a4308_640<br>
[conda] mkl-service               2.4.0           py310h7f8727e_0<br>
[conda] mkl_fft                   1.3.1           py310hd6ae3a3_0<br>
[conda] mkl_random                1.2.2           py310h00e6091_0<br>
[conda] numpy                     1.24.1                   pypi_0    pypi<br>
[conda] numpy-base                1.23.5          py310h8e6c178_0<br>
[conda] pytorch-triton            2.0.0+0d7e753227          pypi_0    pypi<br>
[conda] torch                     2.0.0.dev20230202+cu117          pypi_0    pypi<br>
[conda] torchaudio                2.0.0.dev20230202+cu117          pypi_0    pypi<br>
[conda] torchvision               0.15.0.dev20230202+cu117          pypi_0    pypi</p>

Beck_Guo: <p>I carefully studied the previous  post with the same problem. Actually I also tried other code without torchvision, as long as I want to use torch.compile and same error will come. I even tried with environemnt without torchvision installed from scratch, same problem</p>

Beck_Guo: <p>Anyone has succeeded with the same envirnment?<br>
Thank you in advance!</p>

eqy: <p>Have you updated your version of triton recently? It could be that your version of triton was too old to include <code>sm_89</code> support, see this post/thread for more information:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/90170#issuecomment-1378349498">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/90170#issuecomment-1378349498" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/90170#issuecomment-1378349498" target="_blank" rel="noopener nofollow ugc">Trying pytorch 2.0 snippet from the website - sm89 error</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2022-12-05" data-time="11:41:23" data-timezone="UTC">11:41AM - 05 Dec 22 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2023-01-11" data-time="06:56:52" data-timezone="UTC">06:56AM - 11 Jan 23 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/vince62s" target="_blank" rel="noopener nofollow ugc">
          <img alt="vince62s" src="https://avatars.githubusercontent.com/u/15141326?v=4" class="onebox-avatar-inline" width="20" height="20">
          vince62s
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          high priority
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: third_party
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: inductor
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 🐛 Describe the bug

trying the snippet from the website
```
import torch
<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">import torchvision.models as models

model = models.resnet18().cuda()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
compiled_model = torch.compile(model)

x = torch.randn(16, 3, 224, 224).cuda()
optimizer.zero_grad()
out = compiled_model(x)
out.sum().backward()
optimizer.step()
```

I got Value 'sm_89' is not defined for option 'gpu-name'
I am indeed using a rtx4090 and cuda 11.6
BUT works fine without torch.compile


### Versions

Collecting environment information...
PyTorch version: 1.14.0.dev20221205+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 525.60.11
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] pytorch-lightning==1.7.7
[pip3] torch==1.14.0.dev20221205+cu116
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==0.14.0.dev20221204+cu116
[pip3] torchtext==0.14.0
[pip3] torchtriton==2.0.0+0d7e753227
[pip3] torchvision==0.15.0.dev20221204+cpu
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] faiss-gpu                 1.7.2           py3.8_h28a55e0_0_cuda11.3    pytorch
[conda] libfaiss                  1.7.2           hfc2d529_0_cuda11.3    pytorch
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.1            py38hd3c417c_0  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] numpy                     1.23.4           py38h14f4228_0  
[conda] numpy-base                1.23.4           py38h31eccc5_0  
[conda] pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torch                     1.14.0.dev20221204+cu117          pypi_0    pypi
[conda] torchaudio                0.14.0.dev20221204+cu117          pypi_0    pypi
[conda] torchtriton               2.0.0+0d7e753227          pypi_0    pypi
[conda] torchvision               0.15.0.dev20221204+cpu          pypi_0    pypi


cc @ezyang @gchanan @zou3519 @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Beck_Guo: <p>Great, Problem solved, by specifying the backend supported by 4090, the default backend is not supported by 4090. code will be changed like this:</p>
<p>torch._dynamo.reset()<br>
mm = torch.compile(m,fullgraph=True, backend=‘nvprims_aten’)</p>
<p>Thank you very much!</p>

Beck_Guo: <p>In my environment, below backends are available:<br>
[‘ansor’, ‘aot_cudagraphs’, ‘aot_eager’, ‘aot_eager_decomp_partition’, ‘aot_torchxla_trace_once’, ‘aot_torchxla_trivial’, ‘aot_ts’, ‘aot_ts_nvfuser’, ‘aot_ts_nvfuser_nodecomps’, ‘cudagraphs’, ‘dynamo_accuracy_minifier_backend’, ‘dynamo_minifier_backend’, ‘eager’, ‘fx2trt’, ‘inductor’, ‘ipex’, ‘nvprims_aten’, ‘nvprims_nvfuser’, ‘onnx2tensorrt’, ‘onnxrt’, ‘onnxrt_cpu’, ‘onnxrt_cpu_numpy’, ‘onnxrt_cuda’, ‘tensorrt’, ‘torch2trt’, ‘torchxla_trace_once’, ‘torchxla_trivial’, ‘ts’, ‘tvm’, ‘tvm_meta_schedule’]</p>
<p>Any idea which one works best for GRU?<br>
‘nvprims_aten’, ‘nvprims_nvfuser’ seem to be even little slower than that without torch.compile</p>

marksaroufim: <p>Inductor should work on a 4090 - I’ll confirm that firsthand later this week but you’ll likely solve your issue by upgrading your CUDA version <a href="https://github.com/pytorch/pytorch/issues/90170" class="inline-onebox" rel="noopener nofollow ugc">Trying pytorch 2.0 snippet from the website - sm89 error · Issue #90170 · pytorch/pytorch · GitHub</a></p>

ptrblck: <p>I confirmed <a href="https://github.com/openai/triton/pull/1226">this fix</a> and also ran smoke tests for the <code>2.0.0RC5</code> on sm_86, sm_89, and sm_90 (however, additional tests cannot hurt <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> ).</p>

marksaroufim: <p>Just confirmed, I could run inductor the default backend on my 4090 - haven’t ran any perf benchmarks but nothing crashes</p>