Segmentation Fault in Triton PTX codegen with cuDNN V8 API and `eca_halonext26ts`
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

When running `--inductor` with cuDNN V8 API `TORCH_CUDNN_V8_API_ENABLED=1` a segfault appears during Triton compilation.
Repro command: ` CUDA_LAUNCH_BLOCKING=1 TORCH_CUDNN_V8_API_ENABLED=1 python -u timm_models.py --amp --training --performance --dashboard --device cuda  --inductor --only eca_halonext26ts`
I've observed that running without `TORCH_CUDNN_V8_API_ENABLED=1` first may cause the above to succeed---in order to reproduce `/tmp/torchinductor_root` should be cleared.

CC @eellison @ngimel 

### Error logs

```
cuda train eca_halonext26ts                   ERROR:common:Failed for dynamo Internal Triton PTX codegen error:
Segmentation fault (core dumped)
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/workspace/pytorch/torch/_inductor/codecache.py", line 334, in _worker_compile
    kernel.precompile(warm_cache_only_with_cc=cc)
  File "/workspace/pytorch/torch/_inductor/triton_ops/autotune.py", line 58, in precompile
    self.launchers = [
  File "/workspace/pytorch/torch/_inductor/triton_ops/autotune.py", line 59, in <listcomp>
    self._precompile_config(c, warm_cache_only_with_cc)
  File "/workspace/pytorch/torch/_inductor/triton_ops/autotune.py", line 73, in _precompile_config
    triton.compile(                                                                                                                                                                                                                            File "/usr/local/lib/python3.8/dist-packages/triton/compiler.py", line 1256, in compile
    asm, shared, kernel_name = _compile(fn, signature, device, constants, configs[0], num_warps, num_stages,
  File "/usr/local/lib/python3.8/dist-packages/triton/compiler.py", line 901, in _compile
    name, asm, shared_mem = _triton.code_gen.compile_ttir(backend, module, device, num_warps, num_stages, extern_libs, cc)
RuntimeError: Internal Triton PTX codegen error:
Segmentation fault (core dumped)
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/pytorch/benchmarks/dynamo/common.py", line 1195, in warmup
    fn(model, example_inputs)
  File "/workspace/pytorch/torch/_dynamo/eval_frame.py", line 174, in _fn
    return fn(*args, **kwargs)
  File "timm_models.py", line 305, in forward_and_backward_pass
    cloned_inputs = clone_inputs(inputs)
  File "timm_models.py", line 307, in <graph break in forward_and_backward_pass>
    with self.autocast():
  File "timm_models.py", line 312, in <graph break in forward_and_backward_pass>
    self.grad_scaler.scale(loss).backward()
  File "/workspace/pytorch/torch/_tensor.py", line 473, in backward
    torch.autograd.backward(
  File "/workspace/pytorch/torch/autograd/__init__.py", line 197, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/workspace/pytorch/torch/autograd/function.py", line 270, in apply
    return user_fn(self, *args)
  File "/workspace/pytorch/functorch/_src/aot_autograd.py", line 558, in backward
    CompiledFunction.compiled_bw = aot_config.bw_compiler(
  File "/workspace/pytorch/torch/_dynamo/optimizations/backends.py", line 555, in _wrapped_bw_compiler
    return disable(disable(bw_compiler)(*args, **kwargs))
  File "/workspace/pytorch/torch/_dynamo/eval_frame.py", line 174, in _fn
    return fn(*args, **kwargs)
  File "/workspace/pytorch/torch/_dynamo/utils.py", line 90, in time_wrapper
    r = func(*args, **kwargs)
  File "/workspace/pytorch/torch/_inductor/compile_fx.py", line 362, in bw_compiler
    return compile_fx_inner(
  File "/workspace/pytorch/torch/_dynamo/debug_utils.py", line 473, in debug_wrapper
    compiled_fn = compiler_fn(gm, example_inputs, **kwargs)
  File "/workspace/pytorch/torch/_inductor/debug.py", line 177, in inner
    return fn(*args, **kwargs)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/workspace/pytorch/torch/_inductor/compile_fx.py", line 122, in compile_fx_inner
    compiled_fn = graph.compile_to_fn()
  File "/workspace/pytorch/torch/_inductor/graph.py", line 377, in compile_to_fn
    return self.compile_to_module().call
  File "/workspace/pytorch/torch/_dynamo/utils.py", line 90, in time_wrapper
    r = func(*args, **kwargs)
  File "/workspace/pytorch/torch/_inductor/graph.py", line 367, in compile_to_module
    mod = PyCodeCache.load(code)
  File "/workspace/pytorch/torch/_inductor/codecache.py", line 305, in load
    exec(code, mod.__dict__, mod.__dict__)
  File "/tmp/torchinductor_root/ca/cca3ltcppnb4nxvkkms7vwcjj2wc33znaqbyafcj4bdu3lammdmu.py", line 6836, in <module>
    async_compile.wait(globals())
  File "/workspace/pytorch/torch/_inductor/codecache.py", line 475, in wait
    scope[key] = result.result()
  File "/workspace/pytorch/torch/_inductor/codecache.py", line 352, in result
    self.future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
RuntimeError: Internal Triton PTX codegen error:
Segmentation fault (core dumped)
```

### Minified repro

The minifier command `TORCHDYNAMO_REPRO_AFTER="dynamo" CUDA_LAUNCH_BLOCKING=1 TORCH_CUDNN_V8_API_ENABLED=1 python -u timm_models.py --amp --training --performance --dashboard --device cuda  --inductor --only eca_halonext26ts` created an intermediate `repro.py` script as below (the full run was taking multiple hours without stopping so I stopped it before it finished completely). Note that `/tmp/torchinductor_root/` should also be cleared to repro.

```

from math import inf
import torch
from torch import tensor, device
import torch.fx as fx
import torch._dynamo
from torch._dynamo.testing import rand_strided
from torch._dynamo.debug_utils import run_fwd_maybe_bwd
from torch._dynamo.debug_utils import same_two_models

# REPLACEABLE COMMENT FOR TESTING PURPOSES

args = [((), (), torch.int64, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', True), ((512,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((128,), (1,), torch.float32, 'cuda', False), ((128,), (1,), torch.float32, 'cuda', False), ((128,), (1,), torch.float32, 'cuda', True), ((128,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((128,), (1,), torch.float32, 'cuda', False), ((128,), (1,), torch.float32, 'cuda', False), ((128,), (1,), torch.float32, 'cuda', True), ((128,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', True), ((512,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', True), ((256,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', True), ((256,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', True), ((1024,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', True), ((1024,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', True), ((256,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', False), ((256,), (1,), torch.float32, 'cuda', True), ((256,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', False), ((1024,), (1,), torch.float32, 'cuda', True), ((1024,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', True), ((512,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', True), ((512,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', True), ((2048,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', True), ((2048,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', True), ((512,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', False), ((512,), (1,), torch.float32, 'cuda', True), ((512,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', False), ((2048,), (1,), torch.float32, 'cuda', True), ((2048,), (1,), torch.float32, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((128, 256, 64, 64), (1048576, 4096, 64, 1), torch.float16, 'cuda', True), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((), (), torch.int64, 'cuda', False), ((128, 512, 32, 32), (524288, 1024, 32, 1), torch.float16, 'cuda', True)]
args = [rand_strided(sh, st, dt, dev).requires_grad_(rg) for (sh, st, dt, dev, rg) in args]


from torch.nn import *
class Repro(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.self_self_stages_1_0_drop_path = Identity()
        self.self_self_stages_1_0_shortcut_conv = Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False).cuda()
        self.self_self_stages_1_0_shortcut_bn_drop = Identity()
        self.self_self_stages_1_0_shortcut_bn_act = Identity()
        self.self_self_stages_1_0_act = SiLU(inplace=True)
        self.self_self_stages_1_1_conv1_1x1_conv = Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_1_1_conv1_1x1_bn_drop = Identity()
        self.self_self_stages_1_1_conv1_1x1_bn_act = SiLU(inplace=True)
        self.self_self_stages_1_1_conv2_kxk_conv = Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=8, bias=False).cuda()
        self.self_self_stages_1_1_conv2_kxk_bn_drop = Identity()
        self.self_self_stages_1_1_conv2_kxk_bn_act = SiLU(inplace=True)
        self.self_self_stages_1_1_conv2b_kxk = Identity()
        self.self_self_stages_1_1_attn_conv = Conv1d(1, 1, kernel_size=(5,), stride=(1,), padding=(2,), bias=False).cuda()
        self.self_self_stages_1_1_conv3_1x1_conv = Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_1_1_conv3_1x1_bn_drop = Identity()
        self.self_self_stages_1_1_conv3_1x1_bn_act = Identity()
        self.self_self_stages_1_1_attn_last = Identity()
        self.self_self_stages_1_1_drop_path = Identity()
        self.self_self_stages_1_1_shortcut = Identity()
        self.self_self_stages_1_1_act = SiLU(inplace=True)
        self.self_self_stages_2_0_conv1_1x1_conv = Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_2_0_conv1_1x1_bn_drop = Identity()
        self.self_self_stages_2_0_conv1_1x1_bn_act = SiLU(inplace=True)
        self.self_self_stages_2_0_conv2_kxk_conv = Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=16, bias=False).cuda()
        self.self_self_stages_2_0_conv2_kxk_bn_drop = Identity()
        self.self_self_stages_2_0_conv2_kxk_bn_act = SiLU(inplace=True)
        self.self_self_stages_2_0_conv2b_kxk = Identity()
        self.self_self_stages_2_0_attn_conv = Conv1d(1, 1, kernel_size=(5,), stride=(1,), padding=(2,), bias=False).cuda()
        self.self_self_stages_2_0_conv3_1x1_conv = Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_2_0_conv3_1x1_bn_drop = Identity()
        self.self_self_stages_2_0_conv3_1x1_bn_act = Identity()
        self.self_self_stages_2_0_attn_last = Identity()
        self.self_self_stages_2_0_drop_path = Identity()
        self.self_self_stages_2_0_shortcut_conv = Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False).cuda()
        self.self_self_stages_2_0_shortcut_bn_drop = Identity()
        self.self_self_stages_2_0_shortcut_bn_act = Identity()
        self.self_self_stages_2_0_act = SiLU(inplace=True)
        self.self_self_stages_2_1_conv1_1x1_conv = Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_2_1_conv1_1x1_bn_drop = Identity()
        self.self_self_stages_2_1_conv1_1x1_bn_act = SiLU(inplace=True)
        self.self_self_stages_2_1_conv2_kxk = Identity()
        self.self_self_stages_2_1_self_attn_q = Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_2_1_self_attn_kv = Conv2d(256, 384, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_2_1_self_attn_pool = Identity()
        self.self_self_stages_2_1_post_attn_drop = Identity()
        self.self_self_stages_2_1_post_attn_act = SiLU(inplace=True)
        self.self_self_stages_2_1_conv3_1x1_conv = Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_2_1_conv3_1x1_bn_drop = Identity()
        self.self_self_stages_2_1_conv3_1x1_bn_act = Identity()
        self.self_self_stages_2_1_drop_path = Identity()
        self.self_self_stages_2_1_shortcut = Identity()
        self.self_self_stages_2_1_act = SiLU(inplace=True)
        self.self_self_stages_3_0_conv1_1x1_conv = Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_0_conv1_1x1_bn_drop = Identity()
        self.self_self_stages_3_0_conv1_1x1_bn_act = SiLU(inplace=True)
        self.self_self_stages_3_0_conv2_kxk = Identity()
        self.self_self_stages_3_0_self_attn_q = Conv2d(512, 128, kernel_size=(1, 1), stride=(2, 2), bias=False).cuda()
        self.self_self_stages_3_0_self_attn_kv = Conv2d(512, 640, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_0_self_attn_pool = Identity()
        self.self_self_stages_3_0_post_attn_drop = Identity()
        self.self_self_stages_3_0_post_attn_act = SiLU(inplace=True)
        self.self_self_stages_3_0_conv3_1x1_conv = Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_0_conv3_1x1_bn_drop = Identity()
        self.self_self_stages_3_0_conv3_1x1_bn_act = Identity()
        self.self_self_stages_3_0_drop_path = Identity()
        self.self_self_stages_3_0_shortcut_conv = Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False).cuda()
        self.self_self_stages_3_0_shortcut_bn_drop = Identity()
        self.self_self_stages_3_0_shortcut_bn_act = Identity()
        self.self_self_stages_3_0_act = SiLU(inplace=True)
        self.self_self_stages_3_1_conv1_1x1_conv = Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_1_conv1_1x1_bn_drop = Identity()
        self.self_self_stages_3_1_conv1_1x1_bn_act = SiLU(inplace=True)
        self.self_self_stages_3_1_conv2_kxk = Identity()
        self.self_self_stages_3_1_self_attn_q = Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_1_self_attn_kv = Conv2d(512, 640, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_1_self_attn_pool = Identity()
        self.self_self_stages_3_1_post_attn_drop = Identity()
        self.self_self_stages_3_1_post_attn_act = SiLU(inplace=True)
        self.self_self_stages_3_1_conv3_1x1_conv = Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False).cuda()
        self.self_self_stages_3_1_conv3_1x1_bn_drop = Identity()
        self.self_self_stages_3_1_conv3_1x1_bn_act = Identity()
        self.self_self_stages_3_1_drop_path = Identity()
        self.self_self_stages_3_1_shortcut = Identity()
        self.self_self_stages_3_1_act = SiLU(inplace=True)
        self.self_self_final_conv = Identity()
        self.self_self_head_global_pool_pool = AdaptiveAvgPool2d(output_size=1)
        self.self_self_head_global_pool_flatten = Flatten(start_dim=1, end_dim=-1)
        self.self_self_head_fc = Linear(in_features=2048, out_features=1000, bias=True).cuda()
        self.self_self_head_flatten = Identity()
        self.register_buffer('self_self_stages_2_1_self_attn_pos_embed_width_rel', torch.randn([23, 16], dtype=torch.float32).cuda())
        self.register_buffer('self_self_stages_2_1_self_attn_pos_embed_height_rel', torch.randn([23, 16], dtype=torch.float32).cuda())
        self.register_buffer('self_self_stages_3_0_self_attn_pos_embed_width_rel', torch.randn([23, 16], dtype=torch.float32).cuda())
        self.register_buffer('self_self_stages_3_0_self_attn_pos_embed_height_rel', torch.randn([23, 16], dtype=torch.float32).cuda())
        self.register_buffer('self_self_stages_3_1_self_attn_pos_embed_width_rel', torch.randn([23, 16], dtype=torch.float32).cuda())
        self.register_buffer('self_self_stages_3_1_self_attn_pos_embed_height_rel', torch.randn([23, 16], dtype=torch.float32).cuda())



    def forward(self, self_stages_1_0_shortcut_bn_num_batches_tracked_0 : torch.Tensor, self_stages_1_0_shortcut_bn_running_mean : torch.Tensor, self_stages_1_0_shortcut_bn_running_var : torch.Tensor, self_stages_1_0_shortcut_bn_weight : torch.Tensor, self_stages_1_0_shortcut_bn_bias : torch.Tensor, self_stages_1_1_conv1_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_1_1_conv1_1x1_bn_running_mean : torch.Tensor, self_stages_1_1_conv1_1x1_bn_running_var : torch.Tensor, self_stages_1_1_conv1_1x1_bn_weight : torch.Tensor, self_stages_1_1_conv1_1x1_bn_bias : torch.Tensor, self_stages_1_1_conv2_kxk_bn_num_batches_tracked_0 : torch.Tensor, self_stages_1_1_conv2_kxk_bn_running_mean : torch.Tensor, self_stages_1_1_conv2_kxk_bn_running_var : torch.Tensor, self_stages_1_1_conv2_kxk_bn_weight : torch.Tensor, self_stages_1_1_conv2_kxk_bn_bias : torch.Tensor, self_stages_1_1_conv3_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_1_1_conv3_1x1_bn_running_mean : torch.Tensor, self_stages_1_1_conv3_1x1_bn_running_var : torch.Tensor, self_stages_1_1_conv3_1x1_bn_weight : torch.Tensor, self_stages_1_1_conv3_1x1_bn_bias : torch.Tensor, self_stages_2_0_conv1_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_2_0_conv1_1x1_bn_running_mean : torch.Tensor, self_stages_2_0_conv1_1x1_bn_running_var : torch.Tensor, self_stages_2_0_conv1_1x1_bn_weight : torch.Tensor, self_stages_2_0_conv1_1x1_bn_bias : torch.Tensor, self_stages_2_0_conv2_kxk_bn_num_batches_tracked_0 : torch.Tensor, self_stages_2_0_conv2_kxk_bn_running_mean : torch.Tensor, self_stages_2_0_conv2_kxk_bn_running_var : torch.Tensor, self_stages_2_0_conv2_kxk_bn_weight : torch.Tensor, self_stages_2_0_conv2_kxk_bn_bias : torch.Tensor, self_stages_2_0_conv3_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_2_0_conv3_1x1_bn_running_mean : torch.Tensor, self_stages_2_0_conv3_1x1_bn_running_var : torch.Tensor, self_stages_2_0_conv3_1x1_bn_weight : torch.Tensor, self_stages_2_0_conv3_1x1_bn_bias : torch.Tensor, self_stages_2_0_shortcut_bn_num_batches_tracked_0 : torch.Tensor, self_stages_2_0_shortcut_bn_running_mean : torch.Tensor, self_stages_2_0_shortcut_bn_running_var : torch.Tensor, self_stages_2_0_shortcut_bn_weight : torch.Tensor, self_stages_2_0_shortcut_bn_bias : torch.Tensor, self_stages_2_1_conv1_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_2_1_conv1_1x1_bn_running_mean : torch.Tensor, self_stages_2_1_conv1_1x1_bn_running_var : torch.Tensor, self_stages_2_1_conv1_1x1_bn_weight : torch.Tensor, self_stages_2_1_conv1_1x1_bn_bias : torch.Tensor, self_stages_2_1_post_attn_num_batches_tracked_0 : torch.Tensor, self_stages_2_1_post_attn_running_mean : torch.Tensor, self_stages_2_1_post_attn_running_var : torch.Tensor, self_stages_2_1_post_attn_weight : torch.Tensor, self_stages_2_1_post_attn_bias : torch.Tensor, self_stages_2_1_conv3_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_2_1_conv3_1x1_bn_running_mean : torch.Tensor, self_stages_2_1_conv3_1x1_bn_running_var : torch.Tensor, self_stages_2_1_conv3_1x1_bn_weight : torch.Tensor, self_stages_2_1_conv3_1x1_bn_bias : torch.Tensor, self_stages_3_0_conv1_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_3_0_conv1_1x1_bn_running_mean : torch.Tensor, self_stages_3_0_conv1_1x1_bn_running_var : torch.Tensor, self_stages_3_0_conv1_1x1_bn_weight : torch.Tensor, self_stages_3_0_conv1_1x1_bn_bias : torch.Tensor, self_stages_3_0_post_attn_num_batches_tracked_0 : torch.Tensor, self_stages_3_0_post_attn_running_mean : torch.Tensor, self_stages_3_0_post_attn_running_var : torch.Tensor, self_stages_3_0_post_attn_weight : torch.Tensor, self_stages_3_0_post_attn_bias : torch.Tensor, self_stages_3_0_conv3_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_3_0_conv3_1x1_bn_running_mean : torch.Tensor, self_stages_3_0_conv3_1x1_bn_running_var : torch.Tensor, self_stages_3_0_conv3_1x1_bn_weight : torch.Tensor, self_stages_3_0_conv3_1x1_bn_bias : torch.Tensor, self_stages_3_0_shortcut_bn_num_batches_tracked_0 : torch.Tensor, self_stages_3_0_shortcut_bn_running_mean : torch.Tensor, self_stages_3_0_shortcut_bn_running_var : torch.Tensor, self_stages_3_0_shortcut_bn_weight : torch.Tensor, self_stages_3_0_shortcut_bn_bias : torch.Tensor, self_stages_3_1_conv1_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_3_1_conv1_1x1_bn_running_mean : torch.Tensor, self_stages_3_1_conv1_1x1_bn_running_var : torch.Tensor, self_stages_3_1_conv1_1x1_bn_weight : torch.Tensor, self_stages_3_1_conv1_1x1_bn_bias : torch.Tensor, self_stages_3_1_post_attn_num_batches_tracked_0 : torch.Tensor, self_stages_3_1_post_attn_running_mean : torch.Tensor, self_stages_3_1_post_attn_running_var : torch.Tensor, self_stages_3_1_post_attn_weight : torch.Tensor, self_stages_3_1_post_attn_bias : torch.Tensor, self_stages_3_1_conv3_1x1_bn_num_batches_tracked_0 : torch.Tensor, self_stages_3_1_conv3_1x1_bn_running_mean : torch.Tensor, self_stages_3_1_conv3_1x1_bn_running_var : torch.Tensor, self_stages_3_1_conv3_1x1_bn_weight : torch.Tensor, self_stages_3_1_conv3_1x1_bn_bias : torch.Tensor, add, add_1, add_2, add_3, add_4, add_5, add_6, add_8, add_9, add_10, self_self_stages_0_1_act, add_12, add_13, add_14, self_self_stages_1_0_attn_last):
        self_self_stages_1_0_drop_path = self.self_self_stages_1_0_drop_path(self_self_stages_1_0_attn_last);  self_self_stages_1_0_attn_last = None
        self_self_stages_1_0_shortcut_conv = self.self_self_stages_1_0_shortcut_conv(self_self_stages_0_1_act);  self_self_stages_0_1_act = None
        add_15 = self_stages_1_0_shortcut_bn_num_batches_tracked_0 + 1;  self_stages_1_0_shortcut_bn_num_batches_tracked_0 = None
        batch_norm_13 = torch.nn.functional.batch_norm(self_self_stages_1_0_shortcut_conv, self_stages_1_0_shortcut_bn_running_mean, self_stages_1_0_shortcut_bn_running_var, self_stages_1_0_shortcut_bn_weight, self_stages_1_0_shortcut_bn_bias, True, 0.1, 1e-05);  self_self_stages_1_0_shortcut_conv = self_stages_1_0_shortcut_bn_running_mean = self_stages_1_0_shortcut_bn_running_var = self_stages_1_0_shortcut_bn_weight = self_stages_1_0_shortcut_bn_bias = None
        self_self_stages_1_0_shortcut_bn_drop = self.self_self_stages_1_0_shortcut_bn_drop(batch_norm_13);  batch_norm_13 = None
        self_self_stages_1_0_shortcut_bn_act = self.self_self_stages_1_0_shortcut_bn_act(self_self_stages_1_0_shortcut_bn_drop);  self_self_stages_1_0_shortcut_bn_drop = None
        add_16 = self_self_stages_1_0_drop_path + self_self_stages_1_0_shortcut_bn_act;  self_self_stages_1_0_drop_path = self_self_stages_1_0_shortcut_bn_act = None
        self_self_stages_1_0_act = self.self_self_stages_1_0_act(add_16);  add_16 = None
        self_self_stages_1_1_conv1_1x1_conv = self.self_self_stages_1_1_conv1_1x1_conv(self_self_stages_1_0_act)
        add_17 = self_stages_1_1_conv1_1x1_bn_num_batches_tracked_0 + 1;  self_stages_1_1_conv1_1x1_bn_num_batches_tracked_0 = None
        batch_norm_14 = torch.nn.functional.batch_norm(self_self_stages_1_1_conv1_1x1_conv, self_stages_1_1_conv1_1x1_bn_running_mean, self_stages_1_1_conv1_1x1_bn_running_var, self_stages_1_1_conv1_1x1_bn_weight, self_stages_1_1_conv1_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_1_1_conv1_1x1_conv = self_stages_1_1_conv1_1x1_bn_running_mean = self_stages_1_1_conv1_1x1_bn_running_var = self_stages_1_1_conv1_1x1_bn_weight = self_stages_1_1_conv1_1x1_bn_bias = None
        self_self_stages_1_1_conv1_1x1_bn_drop = self.self_self_stages_1_1_conv1_1x1_bn_drop(batch_norm_14);  batch_norm_14 = None
        self_self_stages_1_1_conv1_1x1_bn_act = self.self_self_stages_1_1_conv1_1x1_bn_act(self_self_stages_1_1_conv1_1x1_bn_drop);  self_self_stages_1_1_conv1_1x1_bn_drop = None
        self_self_stages_1_1_conv2_kxk_conv = self.self_self_stages_1_1_conv2_kxk_conv(self_self_stages_1_1_conv1_1x1_bn_act);  self_self_stages_1_1_conv1_1x1_bn_act = None
        add_18 = self_stages_1_1_conv2_kxk_bn_num_batches_tracked_0 + 1;  self_stages_1_1_conv2_kxk_bn_num_batches_tracked_0 = None
        batch_norm_15 = torch.nn.functional.batch_norm(self_self_stages_1_1_conv2_kxk_conv, self_stages_1_1_conv2_kxk_bn_running_mean, self_stages_1_1_conv2_kxk_bn_running_var, self_stages_1_1_conv2_kxk_bn_weight, self_stages_1_1_conv2_kxk_bn_bias, True, 0.1, 1e-05);  self_self_stages_1_1_conv2_kxk_conv = self_stages_1_1_conv2_kxk_bn_running_mean = self_stages_1_1_conv2_kxk_bn_running_var = self_stages_1_1_conv2_kxk_bn_weight = self_stages_1_1_conv2_kxk_bn_bias = None
        self_self_stages_1_1_conv2_kxk_bn_drop = self.self_self_stages_1_1_conv2_kxk_bn_drop(batch_norm_15);  batch_norm_15 = None
        self_self_stages_1_1_conv2_kxk_bn_act = self.self_self_stages_1_1_conv2_kxk_bn_act(self_self_stages_1_1_conv2_kxk_bn_drop);  self_self_stages_1_1_conv2_kxk_bn_drop = None
        self_self_stages_1_1_conv2b_kxk = self.self_self_stages_1_1_conv2b_kxk(self_self_stages_1_1_conv2_kxk_bn_act);  self_self_stages_1_1_conv2_kxk_bn_act = None
        mean_3 = self_self_stages_1_1_conv2b_kxk.mean((2, 3))
        view_6 = mean_3.view(128, 1, -1);  mean_3 = None
        self_self_stages_1_1_attn_conv = self.self_self_stages_1_1_attn_conv(view_6);  view_6 = None
        sigmoid_3 = self_self_stages_1_1_attn_conv.sigmoid();  self_self_stages_1_1_attn_conv = None
        view_7 = sigmoid_3.view(128, -1, 1, 1);  sigmoid_3 = None
        expand_as_3 = view_7.expand_as(self_self_stages_1_1_conv2b_kxk);  view_7 = None
        mul_3 = self_self_stages_1_1_conv2b_kxk * expand_as_3;  self_self_stages_1_1_conv2b_kxk = expand_as_3 = None
        self_self_stages_1_1_conv3_1x1_conv = self.self_self_stages_1_1_conv3_1x1_conv(mul_3);  mul_3 = None
        add_19 = self_stages_1_1_conv3_1x1_bn_num_batches_tracked_0 + 1;  self_stages_1_1_conv3_1x1_bn_num_batches_tracked_0 = None
        batch_norm_16 = torch.nn.functional.batch_norm(self_self_stages_1_1_conv3_1x1_conv, self_stages_1_1_conv3_1x1_bn_running_mean, self_stages_1_1_conv3_1x1_bn_running_var, self_stages_1_1_conv3_1x1_bn_weight, self_stages_1_1_conv3_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_1_1_conv3_1x1_conv = self_stages_1_1_conv3_1x1_bn_running_mean = self_stages_1_1_conv3_1x1_bn_running_var = self_stages_1_1_conv3_1x1_bn_weight = self_stages_1_1_conv3_1x1_bn_bias = None
        self_self_stages_1_1_conv3_1x1_bn_drop = self.self_self_stages_1_1_conv3_1x1_bn_drop(batch_norm_16);  batch_norm_16 = None
        self_self_stages_1_1_conv3_1x1_bn_act = self.self_self_stages_1_1_conv3_1x1_bn_act(self_self_stages_1_1_conv3_1x1_bn_drop);  self_self_stages_1_1_conv3_1x1_bn_drop = None
        self_self_stages_1_1_attn_last = self.self_self_stages_1_1_attn_last(self_self_stages_1_1_conv3_1x1_bn_act);  self_self_stages_1_1_conv3_1x1_bn_act = None
        self_self_stages_1_1_drop_path = self.self_self_stages_1_1_drop_path(self_self_stages_1_1_attn_last);  self_self_stages_1_1_attn_last = None
        self_self_stages_1_1_shortcut = self.self_self_stages_1_1_shortcut(self_self_stages_1_0_act);  self_self_stages_1_0_act = None
        add_20 = self_self_stages_1_1_drop_path + self_self_stages_1_1_shortcut;  self_self_stages_1_1_drop_path = self_self_stages_1_1_shortcut = None
        self_self_stages_1_1_act = self.self_self_stages_1_1_act(add_20);  add_20 = None
        self_self_stages_2_0_conv1_1x1_conv = self.self_self_stages_2_0_conv1_1x1_conv(self_self_stages_1_1_act)
        add_21 = self_stages_2_0_conv1_1x1_bn_num_batches_tracked_0 + 1;  self_stages_2_0_conv1_1x1_bn_num_batches_tracked_0 = None
        batch_norm_17 = torch.nn.functional.batch_norm(self_self_stages_2_0_conv1_1x1_conv, self_stages_2_0_conv1_1x1_bn_running_mean, self_stages_2_0_conv1_1x1_bn_running_var, self_stages_2_0_conv1_1x1_bn_weight, self_stages_2_0_conv1_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_2_0_conv1_1x1_conv = self_stages_2_0_conv1_1x1_bn_running_mean = self_stages_2_0_conv1_1x1_bn_running_var = self_stages_2_0_conv1_1x1_bn_weight = self_stages_2_0_conv1_1x1_bn_bias = None
        self_self_stages_2_0_conv1_1x1_bn_drop = self.self_self_stages_2_0_conv1_1x1_bn_drop(batch_norm_17);  batch_norm_17 = None
        self_self_stages_2_0_conv1_1x1_bn_act = self.self_self_stages_2_0_conv1_1x1_bn_act(self_self_stages_2_0_conv1_1x1_bn_drop);  self_self_stages_2_0_conv1_1x1_bn_drop = None
        self_self_stages_2_0_conv2_kxk_conv = self.self_self_stages_2_0_conv2_kxk_conv(self_self_stages_2_0_conv1_1x1_bn_act);  self_self_stages_2_0_conv1_1x1_bn_act = None
        add_22 = self_stages_2_0_conv2_kxk_bn_num_batches_tracked_0 + 1;  self_stages_2_0_conv2_kxk_bn_num_batches_tracked_0 = None
        batch_norm_18 = torch.nn.functional.batch_norm(self_self_stages_2_0_conv2_kxk_conv, self_stages_2_0_conv2_kxk_bn_running_mean, self_stages_2_0_conv2_kxk_bn_running_var, self_stages_2_0_conv2_kxk_bn_weight, self_stages_2_0_conv2_kxk_bn_bias, True, 0.1, 1e-05);  self_self_stages_2_0_conv2_kxk_conv = self_stages_2_0_conv2_kxk_bn_running_mean = self_stages_2_0_conv2_kxk_bn_running_var = self_stages_2_0_conv2_kxk_bn_weight = self_stages_2_0_conv2_kxk_bn_bias = None
        self_self_stages_2_0_conv2_kxk_bn_drop = self.self_self_stages_2_0_conv2_kxk_bn_drop(batch_norm_18);  batch_norm_18 = None
        self_self_stages_2_0_conv2_kxk_bn_act = self.self_self_stages_2_0_conv2_kxk_bn_act(self_self_stages_2_0_conv2_kxk_bn_drop);  self_self_stages_2_0_conv2_kxk_bn_drop = None
        self_self_stages_2_0_conv2b_kxk = self.self_self_stages_2_0_conv2b_kxk(self_self_stages_2_0_conv2_kxk_bn_act);  self_self_stages_2_0_conv2_kxk_bn_act = None
        mean_4 = self_self_stages_2_0_conv2b_kxk.mean((2, 3))
        view_8 = mean_4.view(128, 1, -1);  mean_4 = None
        self_self_stages_2_0_attn_conv = self.self_self_stages_2_0_attn_conv(view_8);  view_8 = None
        sigmoid_4 = self_self_stages_2_0_attn_conv.sigmoid();  self_self_stages_2_0_attn_conv = None
        view_9 = sigmoid_4.view(128, -1, 1, 1);  sigmoid_4 = None
        expand_as_4 = view_9.expand_as(self_self_stages_2_0_conv2b_kxk);  view_9 = None
        mul_4 = self_self_stages_2_0_conv2b_kxk * expand_as_4;  self_self_stages_2_0_conv2b_kxk = expand_as_4 = None
        self_self_stages_2_0_conv3_1x1_conv = self.self_self_stages_2_0_conv3_1x1_conv(mul_4);  mul_4 = None
        add_23 = self_stages_2_0_conv3_1x1_bn_num_batches_tracked_0 + 1;  self_stages_2_0_conv3_1x1_bn_num_batches_tracked_0 = None
        batch_norm_19 = torch.nn.functional.batch_norm(self_self_stages_2_0_conv3_1x1_conv, self_stages_2_0_conv3_1x1_bn_running_mean, self_stages_2_0_conv3_1x1_bn_running_var, self_stages_2_0_conv3_1x1_bn_weight, self_stages_2_0_conv3_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_2_0_conv3_1x1_conv = self_stages_2_0_conv3_1x1_bn_running_mean = self_stages_2_0_conv3_1x1_bn_running_var = self_stages_2_0_conv3_1x1_bn_weight = self_stages_2_0_conv3_1x1_bn_bias = None
        self_self_stages_2_0_conv3_1x1_bn_drop = self.self_self_stages_2_0_conv3_1x1_bn_drop(batch_norm_19);  batch_norm_19 = None
        self_self_stages_2_0_conv3_1x1_bn_act = self.self_self_stages_2_0_conv3_1x1_bn_act(self_self_stages_2_0_conv3_1x1_bn_drop);  self_self_stages_2_0_conv3_1x1_bn_drop = None
        self_self_stages_2_0_attn_last = self.self_self_stages_2_0_attn_last(self_self_stages_2_0_conv3_1x1_bn_act);  self_self_stages_2_0_conv3_1x1_bn_act = None
        self_self_stages_2_0_drop_path = self.self_self_stages_2_0_drop_path(self_self_stages_2_0_attn_last);  self_self_stages_2_0_attn_last = None
        self_self_stages_2_0_shortcut_conv = self.self_self_stages_2_0_shortcut_conv(self_self_stages_1_1_act);  self_self_stages_1_1_act = None
        add_24 = self_stages_2_0_shortcut_bn_num_batches_tracked_0 + 1;  self_stages_2_0_shortcut_bn_num_batches_tracked_0 = None
        batch_norm_20 = torch.nn.functional.batch_norm(self_self_stages_2_0_shortcut_conv, self_stages_2_0_shortcut_bn_running_mean, self_stages_2_0_shortcut_bn_running_var, self_stages_2_0_shortcut_bn_weight, self_stages_2_0_shortcut_bn_bias, True, 0.1, 1e-05);  self_self_stages_2_0_shortcut_conv = self_stages_2_0_shortcut_bn_running_mean = self_stages_2_0_shortcut_bn_running_var = self_stages_2_0_shortcut_bn_weight = self_stages_2_0_shortcut_bn_bias = None
        self_self_stages_2_0_shortcut_bn_drop = self.self_self_stages_2_0_shortcut_bn_drop(batch_norm_20);  batch_norm_20 = None
        self_self_stages_2_0_shortcut_bn_act = self.self_self_stages_2_0_shortcut_bn_act(self_self_stages_2_0_shortcut_bn_drop);  self_self_stages_2_0_shortcut_bn_drop = None
        add_25 = self_self_stages_2_0_drop_path + self_self_stages_2_0_shortcut_bn_act;  self_self_stages_2_0_drop_path = self_self_stages_2_0_shortcut_bn_act = None
        self_self_stages_2_0_act = self.self_self_stages_2_0_act(add_25);  add_25 = None
        self_self_stages_2_1_conv1_1x1_conv = self.self_self_stages_2_1_conv1_1x1_conv(self_self_stages_2_0_act)
        add_26 = self_stages_2_1_conv1_1x1_bn_num_batches_tracked_0 + 1;  self_stages_2_1_conv1_1x1_bn_num_batches_tracked_0 = None
        batch_norm_21 = torch.nn.functional.batch_norm(self_self_stages_2_1_conv1_1x1_conv, self_stages_2_1_conv1_1x1_bn_running_mean, self_stages_2_1_conv1_1x1_bn_running_var, self_stages_2_1_conv1_1x1_bn_weight, self_stages_2_1_conv1_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_2_1_conv1_1x1_conv = self_stages_2_1_conv1_1x1_bn_running_mean = self_stages_2_1_conv1_1x1_bn_running_var = self_stages_2_1_conv1_1x1_bn_weight = self_stages_2_1_conv1_1x1_bn_bias = None
        self_self_stages_2_1_conv1_1x1_bn_drop = self.self_self_stages_2_1_conv1_1x1_bn_drop(batch_norm_21);  batch_norm_21 = None
        self_self_stages_2_1_conv1_1x1_bn_act = self.self_self_stages_2_1_conv1_1x1_bn_act(self_self_stages_2_1_conv1_1x1_bn_drop);  self_self_stages_2_1_conv1_1x1_bn_drop = None
        self_self_stages_2_1_conv2_kxk = self.self_self_stages_2_1_conv2_kxk(self_self_stages_2_1_conv1_1x1_bn_act);  self_self_stages_2_1_conv1_1x1_bn_act = None
        self_self_stages_2_1_self_attn_q = self.self_self_stages_2_1_self_attn_q(self_self_stages_2_1_conv2_kxk)
        reshape = self_self_stages_2_1_self_attn_q.reshape(-1, 16, 2, 8, 2, 8);  self_self_stages_2_1_self_attn_q = None
        permute = reshape.permute(0, 1, 3, 5, 2, 4);  reshape = None
        reshape_1 = permute.reshape(1024, 16, -1, 4);  permute = None
        transpose = reshape_1.transpose(1, 3);  reshape_1 = None
        self_self_stages_2_1_self_attn_kv = self.self_self_stages_2_1_self_attn_kv(self_self_stages_2_1_conv2_kxk);  self_self_stages_2_1_conv2_kxk = None
        pad = torch.nn.functional.pad(self_self_stages_2_1_self_attn_kv, [2, 2, 2, 2]);  self_self_stages_2_1_self_attn_kv = None
        unfold = pad.unfold(2, 12, 8);  pad = None
        unfold_1 = unfold.unfold(3, 12, 8);  unfold = None
        reshape_2 = unfold_1.reshape(1024, 48, 4, -1);  unfold_1 = None
        permute_1 = reshape_2.permute(0, 2, 3, 1);  reshape_2 = None
        split = torch.functional.split(permute_1, [16, 32], dim = -1);  permute_1 = None
        getitem = split[0]
        getitem_1 = split[1];  split = None
        transpose_1 = getitem.transpose(-1, -2);  getitem = None
        matmul = transpose @ transpose_1;  transpose_1 = None
        mul_5 = matmul * 0.25;  matmul = None
        reshape_3 = transpose.reshape(-1, 8, 8, 16);  transpose = None
        self_self_stages_2_1_self_attn_pos_embed_width_rel = self.self_self_stages_2_1_self_attn_pos_embed_width_rel
        transpose_2 = self_self_stages_2_1_self_attn_pos_embed_width_rel.transpose(-1, -2);  self_self_stages_2_1_self_attn_pos_embed_width_rel = None
        matmul_1 = reshape_3 @ transpose_2;  transpose_2 = None
        reshape_4 = matmul_1.reshape(-1, 8, 23);  matmul_1 = None
        pad_1 = torch.nn.functional.pad(reshape_4, [0, 1]);  reshape_4 = None
        flatten = pad_1.flatten(1);  pad_1 = None
        pad_2 = torch.nn.functional.pad(flatten, [0, 15]);  flatten = None
        reshape_5 = pad_2.reshape(-1, 9, 23);  pad_2 = None
        getitem_2 = reshape_5[(slice(None, None, None), slice(None, 8, None), slice(11, None, None))];  reshape_5 = None
        reshape_6 = getitem_2.reshape(4096, 8, 1, 8, 12);  getitem_2 = None
        expand = reshape_6.expand(-1, -1, 12, -1, -1);  reshape_6 = None
        permute_2 = expand.permute((0, 1, 3, 2, 4));  expand = None
        transpose_3 = reshape_3.transpose(1, 2);  reshape_3 = None
        self_self_stages_2_1_self_attn_pos_embed_height_rel = self.self_self_stages_2_1_self_attn_pos_embed_height_rel
        transpose_4 = self_self_stages_2_1_self_attn_pos_embed_height_rel.transpose(-1, -2);  self_self_stages_2_1_self_attn_pos_embed_height_rel = None
        matmul_2 = transpose_3 @ transpose_4;  transpose_3 = transpose_4 = None
        reshape_7 = matmul_2.reshape(-1, 8, 23);  matmul_2 = None
        pad_3 = torch.nn.functional.pad(reshape_7, [0, 1]);  reshape_7 = None
        flatten_1 = pad_3.flatten(1);  pad_3 = None
        pad_4 = torch.nn.functional.pad(flatten_1, [0, 15]);  flatten_1 = None
        reshape_8 = pad_4.reshape(-1, 9, 23);  pad_4 = None
        getitem_3 = reshape_8[(slice(None, None, None), slice(None, 8, None), slice(11, None, None))];  reshape_8 = None
        reshape_9 = getitem_3.reshape(4096, 8, 1, 8, 12);  getitem_3 = None
        expand_1 = reshape_9.expand(-1, -1, 12, -1, -1);  reshape_9 = None
        permute_3 = expand_1.permute((0, 3, 1, 4, 2));  expand_1 = None
        add_27 = permute_3 + permute_2;  permute_3 = permute_2 = None
        reshape_10 = add_27.reshape(1024, 4, 64, -1);  add_27 = None
        add_28 = mul_5 + reshape_10;  mul_5 = reshape_10 = None
        softmax = add_28.softmax(dim = -1);  add_28 = None
        matmul_3 = softmax @ getitem_1;  softmax = getitem_1 = None
        transpose_5 = matmul_3.transpose(1, 3);  matmul_3 = None
        reshape_11 = transpose_5.reshape(-1, 8, 8, 2, 2);  transpose_5 = None
        permute_4 = reshape_11.permute(0, 3, 1, 4, 2);  reshape_11 = None
        contiguous = permute_4.contiguous();  permute_4 = None
        view_10 = contiguous.view(128, 256, 16, 16);  contiguous = None
        self_self_stages_2_1_self_attn_pool = self.self_self_stages_2_1_self_attn_pool(view_10);  view_10 = None
        add_29 = self_stages_2_1_post_attn_num_batches_tracked_0 + 1;  self_stages_2_1_post_attn_num_batches_tracked_0 = None
        batch_norm_22 = torch.nn.functional.batch_norm(self_self_stages_2_1_self_attn_pool, self_stages_2_1_post_attn_running_mean, self_stages_2_1_post_attn_running_var, self_stages_2_1_post_attn_weight, self_stages_2_1_post_attn_bias, True, 0.1, 1e-05);  self_self_stages_2_1_self_attn_pool = self_stages_2_1_post_attn_running_mean = self_stages_2_1_post_attn_running_var = self_stages_2_1_post_attn_weight = self_stages_2_1_post_attn_bias = None
        self_self_stages_2_1_post_attn_drop = self.self_self_stages_2_1_post_attn_drop(batch_norm_22);  batch_norm_22 = None
        self_self_stages_2_1_post_attn_act = self.self_self_stages_2_1_post_attn_act(self_self_stages_2_1_post_attn_drop);  self_self_stages_2_1_post_attn_drop = None
        self_self_stages_2_1_conv3_1x1_conv = self.self_self_stages_2_1_conv3_1x1_conv(self_self_stages_2_1_post_attn_act);  self_self_stages_2_1_post_attn_act = None
        add_30 = self_stages_2_1_conv3_1x1_bn_num_batches_tracked_0 + 1;  self_stages_2_1_conv3_1x1_bn_num_batches_tracked_0 = None
        batch_norm_23 = torch.nn.functional.batch_norm(self_self_stages_2_1_conv3_1x1_conv, self_stages_2_1_conv3_1x1_bn_running_mean, self_stages_2_1_conv3_1x1_bn_running_var, self_stages_2_1_conv3_1x1_bn_weight, self_stages_2_1_conv3_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_2_1_conv3_1x1_conv = self_stages_2_1_conv3_1x1_bn_running_mean = self_stages_2_1_conv3_1x1_bn_running_var = self_stages_2_1_conv3_1x1_bn_weight = self_stages_2_1_conv3_1x1_bn_bias = None
        self_self_stages_2_1_conv3_1x1_bn_drop = self.self_self_stages_2_1_conv3_1x1_bn_drop(batch_norm_23);  batch_norm_23 = None
        self_self_stages_2_1_conv3_1x1_bn_act = self.self_self_stages_2_1_conv3_1x1_bn_act(self_self_stages_2_1_conv3_1x1_bn_drop);  self_self_stages_2_1_conv3_1x1_bn_drop = None
        self_self_stages_2_1_drop_path = self.self_self_stages_2_1_drop_path(self_self_stages_2_1_conv3_1x1_bn_act);  self_self_stages_2_1_conv3_1x1_bn_act = None
        self_self_stages_2_1_shortcut = self.self_self_stages_2_1_shortcut(self_self_stages_2_0_act);  self_self_stages_2_0_act = None
        add_31 = self_self_stages_2_1_drop_path + self_self_stages_2_1_shortcut;  self_self_stages_2_1_drop_path = self_self_stages_2_1_shortcut = None
        self_self_stages_2_1_act = self.self_self_stages_2_1_act(add_31);  add_31 = None
        self_self_stages_3_0_conv1_1x1_conv = self.self_self_stages_3_0_conv1_1x1_conv(self_self_stages_2_1_act)
        add_32 = self_stages_3_0_conv1_1x1_bn_num_batches_tracked_0 + 1;  self_stages_3_0_conv1_1x1_bn_num_batches_tracked_0 = None
        batch_norm_24 = torch.nn.functional.batch_norm(self_self_stages_3_0_conv1_1x1_conv, self_stages_3_0_conv1_1x1_bn_running_mean, self_stages_3_0_conv1_1x1_bn_running_var, self_stages_3_0_conv1_1x1_bn_weight, self_stages_3_0_conv1_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_3_0_conv1_1x1_conv = self_stages_3_0_conv1_1x1_bn_running_mean = self_stages_3_0_conv1_1x1_bn_running_var = self_stages_3_0_conv1_1x1_bn_weight = self_stages_3_0_conv1_1x1_bn_bias = None
        self_self_stages_3_0_conv1_1x1_bn_drop = self.self_self_stages_3_0_conv1_1x1_bn_drop(batch_norm_24);  batch_norm_24 = None
        self_self_stages_3_0_conv1_1x1_bn_act = self.self_self_stages_3_0_conv1_1x1_bn_act(self_self_stages_3_0_conv1_1x1_bn_drop);  self_self_stages_3_0_conv1_1x1_bn_drop = None
        self_self_stages_3_0_conv2_kxk = self.self_self_stages_3_0_conv2_kxk(self_self_stages_3_0_conv1_1x1_bn_act);  self_self_stages_3_0_conv1_1x1_bn_act = None
        self_self_stages_3_0_self_attn_q = self.self_self_stages_3_0_self_attn_q(self_self_stages_3_0_conv2_kxk)
        reshape_12 = self_self_stages_3_0_self_attn_q.reshape(-1, 16, 2, 4, 2, 4);  self_self_stages_3_0_self_attn_q = None
        permute_5 = reshape_12.permute(0, 1, 3, 5, 2, 4);  reshape_12 = None
        reshape_13 = permute_5.reshape(1024, 16, -1, 4);  permute_5 = None
        transpose_6 = reshape_13.transpose(1, 3);  reshape_13 = None
        self_self_stages_3_0_self_attn_kv = self.self_self_stages_3_0_self_attn_kv(self_self_stages_3_0_conv2_kxk);  self_self_stages_3_0_conv2_kxk = None
        pad_5 = torch.nn.functional.pad(self_self_stages_3_0_self_attn_kv, [2, 2, 2, 2]);  self_self_stages_3_0_self_attn_kv = None
        unfold_2 = pad_5.unfold(2, 12, 8);  pad_5 = None
        unfold_3 = unfold_2.unfold(3, 12, 8);  unfold_2 = None
        reshape_14 = unfold_3.reshape(1024, 80, 4, -1);  unfold_3 = None
        permute_6 = reshape_14.permute(0, 2, 3, 1);  reshape_14 = None
        split_1 = torch.functional.split(permute_6, [16, 64], dim = -1);  permute_6 = None
        getitem_4 = split_1[0]
        getitem_5 = split_1[1];  split_1 = None
        transpose_7 = getitem_4.transpose(-1, -2);  getitem_4 = None
        matmul_4 = transpose_6 @ transpose_7;  transpose_7 = None
        mul_6 = matmul_4 * 0.25;  matmul_4 = None
        reshape_15 = transpose_6.reshape(-1, 4, 4, 16);  transpose_6 = None
        self_self_stages_3_0_self_attn_pos_embed_width_rel = self.self_self_stages_3_0_self_attn_pos_embed_width_rel
        transpose_8 = self_self_stages_3_0_self_attn_pos_embed_width_rel.transpose(-1, -2);  self_self_stages_3_0_self_attn_pos_embed_width_rel = None
        matmul_5 = reshape_15 @ transpose_8;  transpose_8 = None
        reshape_16 = matmul_5.reshape(-1, 4, 23);  matmul_5 = None
        pad_6 = torch.nn.functional.pad(reshape_16, [0, 1]);  reshape_16 = None
        flatten_2 = pad_6.flatten(1);  pad_6 = None
        pad_7 = torch.nn.functional.pad(flatten_2, [0, 19]);  flatten_2 = None
        reshape_17 = pad_7.reshape(-1, 5, 23);  pad_7 = None
        getitem_6 = reshape_17[(slice(None, None, None), slice(None, 4, None), slice(11, None, None))];  reshape_17 = None
        reshape_18 = getitem_6.reshape(4096, 4, 1, 4, 12);  getitem_6 = None
        expand_2 = reshape_18.expand(-1, -1, 12, -1, -1);  reshape_18 = None
        permute_7 = expand_2.permute((0, 1, 3, 2, 4));  expand_2 = None
        transpose_9 = reshape_15.transpose(1, 2);  reshape_15 = None
        self_self_stages_3_0_self_attn_pos_embed_height_rel = self.self_self_stages_3_0_self_attn_pos_embed_height_rel
        transpose_10 = self_self_stages_3_0_self_attn_pos_embed_height_rel.transpose(-1, -2);  self_self_stages_3_0_self_attn_pos_embed_height_rel = None
        matmul_6 = transpose_9 @ transpose_10;  transpose_9 = transpose_10 = None
        reshape_19 = matmul_6.reshape(-1, 4, 23);  matmul_6 = None
        pad_8 = torch.nn.functional.pad(reshape_19, [0, 1]);  reshape_19 = None
        flatten_3 = pad_8.flatten(1);  pad_8 = None
        pad_9 = torch.nn.functional.pad(flatten_3, [0, 19]);  flatten_3 = None
        reshape_20 = pad_9.reshape(-1, 5, 23);  pad_9 = None
        getitem_7 = reshape_20[(slice(None, None, None), slice(None, 4, None), slice(11, None, None))];  reshape_20 = None
        reshape_21 = getitem_7.reshape(4096, 4, 1, 4, 12);  getitem_7 = None
        expand_3 = reshape_21.expand(-1, -1, 12, -1, -1);  reshape_21 = None
        permute_8 = expand_3.permute((0, 3, 1, 4, 2));  expand_3 = None
        add_33 = permute_8 + permute_7;  permute_8 = permute_7 = None
        reshape_22 = add_33.reshape(1024, 4, 16, -1);  add_33 = None
        add_34 = mul_6 + reshape_22;  mul_6 = reshape_22 = None
        softmax_1 = add_34.softmax(dim = -1);  add_34 = None
        matmul_7 = softmax_1 @ getitem_5;  softmax_1 = getitem_5 = None
        transpose_11 = matmul_7.transpose(1, 3);  matmul_7 = None
        reshape_23 = transpose_11.reshape(-1, 4, 4, 2, 2);  transpose_11 = None
        permute_9 = reshape_23.permute(0, 3, 1, 4, 2);  reshape_23 = None
        contiguous_1 = permute_9.contiguous();  permute_9 = None
        view_11 = contiguous_1.view(128, 512, 8, 8);  contiguous_1 = None
        self_self_stages_3_0_self_attn_pool = self.self_self_stages_3_0_self_attn_pool(view_11);  view_11 = None
        add_35 = self_stages_3_0_post_attn_num_batches_tracked_0 + 1;  self_stages_3_0_post_attn_num_batches_tracked_0 = None
        batch_norm_25 = torch.nn.functional.batch_norm(self_self_stages_3_0_self_attn_pool, self_stages_3_0_post_attn_running_mean, self_stages_3_0_post_attn_running_var, self_stages_3_0_post_attn_weight, self_stages_3_0_post_attn_bias, True, 0.1, 1e-05);  self_self_stages_3_0_self_attn_pool = self_stages_3_0_post_attn_running_mean = self_stages_3_0_post_attn_running_var = self_stages_3_0_post_attn_weight = self_stages_3_0_post_attn_bias = None
        self_self_stages_3_0_post_attn_drop = self.self_self_stages_3_0_post_attn_drop(batch_norm_25);  batch_norm_25 = None
        self_self_stages_3_0_post_attn_act = self.self_self_stages_3_0_post_attn_act(self_self_stages_3_0_post_attn_drop);  self_self_stages_3_0_post_attn_drop = None
        self_self_stages_3_0_conv3_1x1_conv = self.self_self_stages_3_0_conv3_1x1_conv(self_self_stages_3_0_post_attn_act);  self_self_stages_3_0_post_attn_act = None
        add_36 = self_stages_3_0_conv3_1x1_bn_num_batches_tracked_0 + 1;  self_stages_3_0_conv3_1x1_bn_num_batches_tracked_0 = None
        batch_norm_26 = torch.nn.functional.batch_norm(self_self_stages_3_0_conv3_1x1_conv, self_stages_3_0_conv3_1x1_bn_running_mean, self_stages_3_0_conv3_1x1_bn_running_var, self_stages_3_0_conv3_1x1_bn_weight, self_stages_3_0_conv3_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_3_0_conv3_1x1_conv = self_stages_3_0_conv3_1x1_bn_running_mean = self_stages_3_0_conv3_1x1_bn_running_var = self_stages_3_0_conv3_1x1_bn_weight = self_stages_3_0_conv3_1x1_bn_bias = None
        self_self_stages_3_0_conv3_1x1_bn_drop = self.self_self_stages_3_0_conv3_1x1_bn_drop(batch_norm_26);  batch_norm_26 = None
        self_self_stages_3_0_conv3_1x1_bn_act = self.self_self_stages_3_0_conv3_1x1_bn_act(self_self_stages_3_0_conv3_1x1_bn_drop);  self_self_stages_3_0_conv3_1x1_bn_drop = None
        self_self_stages_3_0_drop_path = self.self_self_stages_3_0_drop_path(self_self_stages_3_0_conv3_1x1_bn_act);  self_self_stages_3_0_conv3_1x1_bn_act = None
        self_self_stages_3_0_shortcut_conv = self.self_self_stages_3_0_shortcut_conv(self_self_stages_2_1_act);  self_self_stages_2_1_act = None
        add_37 = self_stages_3_0_shortcut_bn_num_batches_tracked_0 + 1;  self_stages_3_0_shortcut_bn_num_batches_tracked_0 = None
        batch_norm_27 = torch.nn.functional.batch_norm(self_self_stages_3_0_shortcut_conv, self_stages_3_0_shortcut_bn_running_mean, self_stages_3_0_shortcut_bn_running_var, self_stages_3_0_shortcut_bn_weight, self_stages_3_0_shortcut_bn_bias, True, 0.1, 1e-05);  self_self_stages_3_0_shortcut_conv = self_stages_3_0_shortcut_bn_running_mean = self_stages_3_0_shortcut_bn_running_var = self_stages_3_0_shortcut_bn_weight = self_stages_3_0_shortcut_bn_bias = None
        self_self_stages_3_0_shortcut_bn_drop = self.self_self_stages_3_0_shortcut_bn_drop(batch_norm_27);  batch_norm_27 = None
        self_self_stages_3_0_shortcut_bn_act = self.self_self_stages_3_0_shortcut_bn_act(self_self_stages_3_0_shortcut_bn_drop);  self_self_stages_3_0_shortcut_bn_drop = None
        add_38 = self_self_stages_3_0_drop_path + self_self_stages_3_0_shortcut_bn_act;  self_self_stages_3_0_drop_path = self_self_stages_3_0_shortcut_bn_act = None
        self_self_stages_3_0_act = self.self_self_stages_3_0_act(add_38);  add_38 = None
        self_self_stages_3_1_conv1_1x1_conv = self.self_self_stages_3_1_conv1_1x1_conv(self_self_stages_3_0_act)
        add_39 = self_stages_3_1_conv1_1x1_bn_num_batches_tracked_0 + 1;  self_stages_3_1_conv1_1x1_bn_num_batches_tracked_0 = None
        batch_norm_28 = torch.nn.functional.batch_norm(self_self_stages_3_1_conv1_1x1_conv, self_stages_3_1_conv1_1x1_bn_running_mean, self_stages_3_1_conv1_1x1_bn_running_var, self_stages_3_1_conv1_1x1_bn_weight, self_stages_3_1_conv1_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_3_1_conv1_1x1_conv = self_stages_3_1_conv1_1x1_bn_running_mean = self_stages_3_1_conv1_1x1_bn_running_var = self_stages_3_1_conv1_1x1_bn_weight = self_stages_3_1_conv1_1x1_bn_bias = None
        self_self_stages_3_1_conv1_1x1_bn_drop = self.self_self_stages_3_1_conv1_1x1_bn_drop(batch_norm_28);  batch_norm_28 = None
        self_self_stages_3_1_conv1_1x1_bn_act = self.self_self_stages_3_1_conv1_1x1_bn_act(self_self_stages_3_1_conv1_1x1_bn_drop);  self_self_stages_3_1_conv1_1x1_bn_drop = None
        self_self_stages_3_1_conv2_kxk = self.self_self_stages_3_1_conv2_kxk(self_self_stages_3_1_conv1_1x1_bn_act);  self_self_stages_3_1_conv1_1x1_bn_act = None
        self_self_stages_3_1_self_attn_q = self.self_self_stages_3_1_self_attn_q(self_self_stages_3_1_conv2_kxk)
        reshape_24 = self_self_stages_3_1_self_attn_q.reshape(-1, 16, 1, 8, 1, 8);  self_self_stages_3_1_self_attn_q = None
        permute_10 = reshape_24.permute(0, 1, 3, 5, 2, 4);  reshape_24 = None
        reshape_25 = permute_10.reshape(1024, 16, -1, 1);  permute_10 = None
        transpose_12 = reshape_25.transpose(1, 3);  reshape_25 = None
        self_self_stages_3_1_self_attn_kv = self.self_self_stages_3_1_self_attn_kv(self_self_stages_3_1_conv2_kxk);  self_self_stages_3_1_conv2_kxk = None
        pad_10 = torch.nn.functional.pad(self_self_stages_3_1_self_attn_kv, [2, 2, 2, 2]);  self_self_stages_3_1_self_attn_kv = None
        unfold_4 = pad_10.unfold(2, 12, 8);  pad_10 = None
        unfold_5 = unfold_4.unfold(3, 12, 8);  unfold_4 = None
        reshape_26 = unfold_5.reshape(1024, 80, 1, -1);  unfold_5 = None
        permute_11 = reshape_26.permute(0, 2, 3, 1);  reshape_26 = None
        split_2 = torch.functional.split(permute_11, [16, 64], dim = -1);  permute_11 = None
        getitem_8 = split_2[0]
        getitem_9 = split_2[1];  split_2 = None
        transpose_13 = getitem_8.transpose(-1, -2);  getitem_8 = None
        matmul_8 = transpose_12 @ transpose_13;  transpose_13 = None
        mul_7 = matmul_8 * 0.25;  matmul_8 = None
        reshape_27 = transpose_12.reshape(-1, 8, 8, 16);  transpose_12 = None
        self_self_stages_3_1_self_attn_pos_embed_width_rel = self.self_self_stages_3_1_self_attn_pos_embed_width_rel
        transpose_14 = self_self_stages_3_1_self_attn_pos_embed_width_rel.transpose(-1, -2);  self_self_stages_3_1_self_attn_pos_embed_width_rel = None
        matmul_9 = reshape_27 @ transpose_14;  transpose_14 = None
        reshape_28 = matmul_9.reshape(-1, 8, 23);  matmul_9 = None
        pad_11 = torch.nn.functional.pad(reshape_28, [0, 1]);  reshape_28 = None
        flatten_4 = pad_11.flatten(1);  pad_11 = None
        pad_12 = torch.nn.functional.pad(flatten_4, [0, 15]);  flatten_4 = None
        reshape_29 = pad_12.reshape(-1, 9, 23);  pad_12 = None
        getitem_10 = reshape_29[(slice(None, None, None), slice(None, 8, None), slice(11, None, None))];  reshape_29 = None
        reshape_30 = getitem_10.reshape(1024, 8, 1, 8, 12);  getitem_10 = None
        expand_4 = reshape_30.expand(-1, -1, 12, -1, -1);  reshape_30 = None
        permute_12 = expand_4.permute((0, 1, 3, 2, 4));  expand_4 = None
        transpose_15 = reshape_27.transpose(1, 2);  reshape_27 = None
        self_self_stages_3_1_self_attn_pos_embed_height_rel = self.self_self_stages_3_1_self_attn_pos_embed_height_rel
        transpose_16 = self_self_stages_3_1_self_attn_pos_embed_height_rel.transpose(-1, -2);  self_self_stages_3_1_self_attn_pos_embed_height_rel = None
        matmul_10 = transpose_15 @ transpose_16;  transpose_15 = transpose_16 = None
        reshape_31 = matmul_10.reshape(-1, 8, 23);  matmul_10 = None
        pad_13 = torch.nn.functional.pad(reshape_31, [0, 1]);  reshape_31 = None
        flatten_5 = pad_13.flatten(1);  pad_13 = None
        pad_14 = torch.nn.functional.pad(flatten_5, [0, 15]);  flatten_5 = None
        reshape_32 = pad_14.reshape(-1, 9, 23);  pad_14 = None
        getitem_11 = reshape_32[(slice(None, None, None), slice(None, 8, None), slice(11, None, None))];  reshape_32 = None
        reshape_33 = getitem_11.reshape(1024, 8, 1, 8, 12);  getitem_11 = None
        expand_5 = reshape_33.expand(-1, -1, 12, -1, -1);  reshape_33 = None
        permute_13 = expand_5.permute((0, 3, 1, 4, 2));  expand_5 = None
        add_40 = permute_13 + permute_12;  permute_13 = permute_12 = None
        reshape_34 = add_40.reshape(1024, 1, 64, -1);  add_40 = None
        add_41 = mul_7 + reshape_34;  mul_7 = reshape_34 = None
        softmax_2 = add_41.softmax(dim = -1);  add_41 = None
        matmul_11 = softmax_2 @ getitem_9;  softmax_2 = getitem_9 = None
        transpose_17 = matmul_11.transpose(1, 3);  matmul_11 = None
        reshape_35 = transpose_17.reshape(-1, 8, 8, 1, 1);  transpose_17 = None
        permute_14 = reshape_35.permute(0, 3, 1, 4, 2);  reshape_35 = None
        contiguous_2 = permute_14.contiguous();  permute_14 = None
        view_12 = contiguous_2.view(128, 512, 8, 8);  contiguous_2 = None
        self_self_stages_3_1_self_attn_pool = self.self_self_stages_3_1_self_attn_pool(view_12);  view_12 = None
        add_42 = self_stages_3_1_post_attn_num_batches_tracked_0 + 1;  self_stages_3_1_post_attn_num_batches_tracked_0 = None
        batch_norm_29 = torch.nn.functional.batch_norm(self_self_stages_3_1_self_attn_pool, self_stages_3_1_post_attn_running_mean, self_stages_3_1_post_attn_running_var, self_stages_3_1_post_attn_weight, self_stages_3_1_post_attn_bias, True, 0.1, 1e-05);  self_self_stages_3_1_self_attn_pool = self_stages_3_1_post_attn_running_mean = self_stages_3_1_post_attn_running_var = self_stages_3_1_post_attn_weight = self_stages_3_1_post_attn_bias = None
        self_self_stages_3_1_post_attn_drop = self.self_self_stages_3_1_post_attn_drop(batch_norm_29);  batch_norm_29 = None
        self_self_stages_3_1_post_attn_act = self.self_self_stages_3_1_post_attn_act(self_self_stages_3_1_post_attn_drop);  self_self_stages_3_1_post_attn_drop = None
        self_self_stages_3_1_conv3_1x1_conv = self.self_self_stages_3_1_conv3_1x1_conv(self_self_stages_3_1_post_attn_act);  self_self_stages_3_1_post_attn_act = None
        add_43 = self_stages_3_1_conv3_1x1_bn_num_batches_tracked_0 + 1;  self_stages_3_1_conv3_1x1_bn_num_batches_tracked_0 = None
        batch_norm_30 = torch.nn.functional.batch_norm(self_self_stages_3_1_conv3_1x1_conv, self_stages_3_1_conv3_1x1_bn_running_mean, self_stages_3_1_conv3_1x1_bn_running_var, self_stages_3_1_conv3_1x1_bn_weight, self_stages_3_1_conv3_1x1_bn_bias, True, 0.1, 1e-05);  self_self_stages_3_1_conv3_1x1_conv = self_stages_3_1_conv3_1x1_bn_running_mean = self_stages_3_1_conv3_1x1_bn_running_var = self_stages_3_1_conv3_1x1_bn_weight = self_stages_3_1_conv3_1x1_bn_bias = None
        self_self_stages_3_1_conv3_1x1_bn_drop = self.self_self_stages_3_1_conv3_1x1_bn_drop(batch_norm_30);  batch_norm_30 = None
        self_self_stages_3_1_conv3_1x1_bn_act = self.self_self_stages_3_1_conv3_1x1_bn_act(self_self_stages_3_1_conv3_1x1_bn_drop);  self_self_stages_3_1_conv3_1x1_bn_drop = None
        self_self_stages_3_1_drop_path = self.self_self_stages_3_1_drop_path(self_self_stages_3_1_conv3_1x1_bn_act);  self_self_stages_3_1_conv3_1x1_bn_act = None
        self_self_stages_3_1_shortcut = self.self_self_stages_3_1_shortcut(self_self_stages_3_0_act);  self_self_stages_3_0_act = None
        add_44 = self_self_stages_3_1_drop_path + self_self_stages_3_1_shortcut;  self_self_stages_3_1_drop_path = self_self_stages_3_1_shortcut = None
        self_self_stages_3_1_act = self.self_self_stages_3_1_act(add_44);  add_44 = None
        self_self_final_conv = self.self_self_final_conv(self_self_stages_3_1_act);  self_self_stages_3_1_act = None
        self_self_head_global_pool_pool = self.self_self_head_global_pool_pool(self_self_final_conv);  self_self_final_conv = None
        self_self_head_global_pool_flatten = self.self_self_head_global_pool_flatten(self_self_head_global_pool_pool);  self_self_head_global_pool_pool = None
        self_self_head_fc = self.self_self_head_fc(self_self_head_global_pool_flatten);  self_self_head_global_pool_flatten = None
        self_self_head_flatten = self.self_self_head_flatten(self_self_head_fc);  self_self_head_fc = None
        return (self_self_head_flatten, add, add_1, add_2, add_3, add_4, add_5, add_6, add_8, add_9, add_10, add_12, add_13, add_14, add_15, add_17, add_18, add_19, add_21, add_22, add_23, add_24, add_26, add_29, add_30, add_32, add_35, add_36, add_37, add_39, add_42, add_43)



mod = Repro()
opt_mod = torch._dynamo.optimize("inductor")(mod)


with torch.cuda.amp.autocast(enabled=True):
    ref = run_fwd_maybe_bwd(mod, args)
    res = run_fwd_maybe_bwd(opt_mod, args)
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh