Repository: pytorch
Issue: Composer inductor errors
Labels: oncall: distributed,triaged,module: fsdp,oncall: pt2

As a followup to https://github.com/pytorch/torchdynamo/issues/887 which worked with eager

## Repro

`pip install mosaicml`

```python
from torch.utils.data import DataLoader
from torchvision import datasets, transforms

from composer import Trainer
from composer.algorithms import ChannelsLast, CutMix, LabelSmoothing
from composer.models import mnist_model
import torchdynamo

transform = transforms.Compose([transforms.ToTensor()])
train_dataset = datasets.MNIST("data", download=True, train=True, transform=transform)
eval_dataset = datasets.MNIST("data", download=True, train=False, transform=transform)
train_dataloader = DataLoader(train_dataset, batch_size=128)
eval_dataloader = DataLoader(eval_dataset, batch_size=128)

trainer = Trainer(
    model=mnist_model(),
    train_dataloader=train_dataloader,
    eval_dataloader=eval_dataloader,
    max_duration="2ep",
    algorithms=[
        ChannelsLast(),
        CutMix(alpha=1.0),
        LabelSmoothing(smoothing=0.1),
    ]
)

@torchdynamo.optimize("inductor")
def train():
    trainer.fit()

import time 
tic = time.time()
train()
toc = time.time()
print(toc - tic)

```

Tried running the same file on inductor and a bunch of warnings are coming up,
* import deepspeed is not working even though I don't think this program is using deepspeed? 
* A few `UserDefinedObjectVariable` not a constant errors
* An assertion error on buffer strides
* numpy int64 not implemented
* keyerror _IPYTHON
* torch op returned non tensor


## Logs

```(dynamo) ubuntu@ip-172-31-31-152:~/tests$ python comp.py 
Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz
100%|██████████████████| 9912422/9912422 [00:00<00:00, 34569393.90it/s]
Extracting data/MNIST/raw/train-images-idx3-ubyte.gz to data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to data/MNIST/raw/train-labels-idx1-ubyte.gz
100%|█████████████████████| 28881/28881 [00:00<00:00, 139076571.55it/s]
Extracting data/MNIST/raw/train-labels-idx1-ubyte.gz to data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to data/MNIST/raw/t10k-images-idx3-ubyte.gz
100%|███████████████████| 1648877/1648877 [00:00<00:00, 7789551.97it/s]
Extracting data/MNIST/raw/t10k-images-idx3-ubyte.gz to data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to data/MNIST/raw/t10k-labels-idx1-ubyte.gz
100%|████████████████████████| 4542/4542 [00:00<00:00, 34637325.03it/s]
Extracting data/MNIST/raw/t10k-labels-idx1-ubyte.gz to data/MNIST/raw

/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py:852: UserWarning: No optimizer was specified. Defaulting to DecoupledSGDW(lr=0.1)
  warnings.warn(('No optimizer was specified. Defaulting to '
                                                                      torchdynamo.symbolic_convert: [WARNING] Graph break: call_function UserDefinedObjectVariable(EnumMeta) [EnumVariable(<enum 'TimeUnit'>)] {} from user code at   File "comp.py", line 29, in train
    trainer.fit()
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1379, in fit
    if self.state.max_duration <= self.state.timestamp.get(self.state.max_duration.unit) and not reset_time:
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/time.py", line 559, in get
    unit = TimeUnit(unit)

torchdynamo.symbolic_convert: [WARNING] Graph break: inline in skipfiles: info /opt/conda/envs/dynamo/lib/python3.8/logging/__init__.py from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1555, in _train_loop
    log.info('Using precision %s', self.state.precision)

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function UserDefinedObjectVariable(EnumMeta) [EnumVariable(<enum 'Event'>)] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1563, in <graph break in _train_loop>
    self.engine.run_event(Event.FIT_START)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 210, in run_event
    event = Event(event)

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function BuiltinVariable(int) [UserDefinedObjectVariable(Time)] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 212, in <graph break in run_event>
    self._debug_log(event, 'Running event')
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 419, in _debug_log
    timestamp = f'[ep={int(self.state.timestamp.epoch)}][ba={int(self.state.timestamp.batch)}]'

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function BuiltinVariable(sorted) [ListVariable()] {'key': NestedUserFunctionVariable()} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 315, in _run_algorithms
    algorithms_to_run = self._compile(algorithms_to_run, event)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 369, in _compile
    algorithms_to_run = passes(algorithms_to_run, event)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/passes.py", line 91, in sort_selective_backprop_first
    return sort_to_front(algorithms, cls=SelectiveBackprop)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/passes.py", line 54, in sort_to_front
    return sorted(list_to_sort, key=lambda x: not isinstance(x, cls))

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function in skip_files /opt/conda/envs/dynamo/lib/python3.8/collections/__init__.py from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 317, in <graph break in _run_algorithms>
    trace = _setup_trace(algorithms_to_run, event)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 147, in _setup_trace
    return OrderedDict([(f'{algo}/{event}', Trace(name=algo.__class__.__name__)) for algo in algorithms])

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function UserDefinedObjectVariable(EnumMeta) [EnumVariable(<enum 'Event'>)] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 382, in _run_callbacks
    event = Event(event)

torchdynamo.symbolic_convert: [WARNING] Graph break: inline in skipfiles: __init__ /opt/conda/envs/dynamo/lib/python3.8/contextlib.py from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/engine.py", line 405, in <graph break in _run_callbacks>
    ctx = cast(ContextManager, contextlib.nullcontext()) if marker is None else marker

torchdynamo.convert_frame: [ERROR] WON'T CONVERT run_event /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/callback.py line 87 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/base.py", line 144, in as_python_constant
    raise NotImplementedError(f"{self} is not a constant")
NotImplementedError: GetAttrVariable(EnumVariable(<enum 'Event'>), value) is not a constant

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/callback.py", line 95, in run_event
    event_cb = getattr(self, event.value)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT <graph break in _train_loop> /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py line 1563 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1565, in <graph break in _train_loop>
    use_grad_scaling = self._use_grad_scaling(self.state.precision, self.state.scaler)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT _use_grad_scaling /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py line 2435 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 2451, in _use_grad_scaling
    if self.deepspeed_enabled:

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT deepspeed_enabled /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py line 1149 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1155, in deepspeed_enabled
    return is_model_deepspeed(self.state.model)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT is_model_deepspeed /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/misc.py line 16 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/misc.py", line 19, in is_model_deepspeed
    import deepspeed

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.symbolic_convert: [WARNING] Graph break: inline in skipfiles: debug /opt/conda/envs/dynamo/lib/python3.8/logging/__init__.py from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1514, in _spin_dataloaders
    log.debug('Spinning the dataloaders')

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function BuiltinVariable(int) [TensorVariable()] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/datasets/mnist.py", line 138, in __getitem__
    img, target = self.data[index], int(self.targets[index])

torchdynamo.symbolic_convert: [WARNING] Graph break: Tensor.numpy from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/datasets/mnist.py", line 142, in <graph break in __getitem__>
    img = Image.fromarray(img.numpy(), mode="L")

torchdynamo.symbolic_convert: [WARNING] Graph break: call_method GetAttrVariable(NumpyVariable(), __array_interface__) __getitem__ (ConstantVariable(str),) {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/datasets/mnist.py", line 142, in <graph break in __getitem__>
    img = Image.fromarray(img.numpy(), mode="L")
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/PIL/Image.py", line 2944, in fromarray
    shape = arr["shape"]

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function UserDefinedObjectVariable(Compose) [UserDefinedObjectVariable(Image)] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/datasets/mnist.py", line 145, in <graph break in __getitem__>
    img = self.transform(img)

torchdynamo.convert_frame: [ERROR] WON'T CONVERT to_tensor /opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/transforms/functional.py line 122 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/tensor.py", line 268, in create
    assert (
AssertionError: torch.* op returned non-Tensor dtype call_function <built-in function get_default_dtype>

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 142, in to_tensor
    default_float_dtype = torch.get_default_dtype()

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [WARNING] torchdynamo hit config.cache_size_limit (64)
   function: '__getitem__' (/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchvision/datasets/mnist.py:130)
   reasons:  ['index == 0']
to diagnose recompilation issues, see https://github.com/pytorch/torchdynamo/blob/main/TROUBLESHOOTING.md.
torchdynamo.symbolic_convert: [WARNING] Graph break: hasattr: TensorVariable() from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchmetrics/metric.py", line 591, in __hash__
    if hasattr(val, "__iter__") and not isinstance(val, Tensor):

torchdynamo.symbolic_convert: [WARNING] Graph break: hasattr: TensorVariable() from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchmetrics/metric.py", line 594, in <graph break in __hash__>
    hash_vals.append(val)

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function BuiltinVariable(hash) [TupleVariable()] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchmetrics/metric.py", line 596, in <graph break in __hash__>
    return hash(tuple(hash_vals))

torchdynamo.convert_frame: [ERROR] WON'T CONVERT epoch_start /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/loggers/progress_bar_logger.py line 308 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/tensor.py", line 268, in create
    assert (
AssertionError: torch.* op returned non-Tensor bool call_function <function is_available at 0x7f1edb2a0430>

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/loggers/progress_bar_logger.py", line 309, in epoch_start
    if self.show_pbar and not self.train_pbar:

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT show_pbar /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/loggers/progress_bar_logger.py line 167 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/tensor.py", line 268, in create
    assert (
AssertionError: torch.* op returned non-Tensor bool call_function <function is_available at 0x7f1edb2a0430>

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/loggers/progress_bar_logger.py", line 169, in show_pbar
    return self._show_pbar and dist.get_local_rank() == 0

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT _get_distributed_config_var /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/dist.py line 75 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/tensor.py", line 268, in create
    assert (
AssertionError: torch.* op returned non-Tensor bool call_function <function is_available at 0x7f1edb2a0430>

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/dist.py", line 81, in _get_distributed_config_var
    if not dist.is_available():

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT _build_pbar /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/loggers/progress_bar_logger.py line 234 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 446, in LOAD_GLOBAL
    value = self.f_globals[name]
KeyError: '__IPYTHON__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 567, in load_builtin
    assert inst.argval in self.f_builtins
AssertionError

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/loggers/progress_bar_logger.py", line 244, in _build_pbar
    position = None if is_notebook() else 1

Set torchdynamo.config.verbose=True for more information
==========
                                                                      torchinductor.ir: [WARNING] DeviceCopy                    | 0/469 [00:00
torchinductor.ir: [WARNING] DeviceCopy
torchinductor.compile_fx: [WARNING] skipping cudagraphs due to multiple devices
torchdynamo.symbolic_convert: [WARNING] Graph break: hasattr: TensorVariable() from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/data_spec.py", line 197, in _default_get_num_samples_in_batch
    if not hasattr(t, 'shape'):

torchdynamo.symbolic_convert: [WARNING] Graph break: hasattr: TensorVariable() from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/data_spec.py", line 202, in <graph break in _default_get_num_samples_in_batch>
    dim0_sizes.append(t.shape[0])

torchdynamo.symbolic_convert: [WARNING] Graph break: setattr(UserDefinedObjectVariable) <function Metric.__setattr__ at 0x7f1eed36b550> from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1788, in _train_batch
    metric.reset()
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchmetrics/metric.py", line 402, in reset
    self._update_called = False

torchdynamo.symbolic_convert: [WARNING] Graph break: setattr(UserDefinedObjectVariable) <function Metric.__setattr__ at 0x7f1eed36b550> from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torchmetrics/metric.py", line 411, in <graph break in reset>
    setattr(self, attr, default.detach().clone().to(current_val.device))

torchdynamo.convert_frame: [ERROR] WON'T CONVERT <graph break in _train_batch> /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py line 1788 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1795, in <graph break in _train_batch>
    if self._use_closures():

Set torchdynamo.config.verbose=True for more information
==========
torchinductor.ir: [WARNING] DeviceCopy
torchinductor.compile_fx: [WARNING] skipping cudagraphs due to multiple devices
torchdynamo.convert_frame: [ERROR] WON'T CONVERT _use_closures /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py line 2496 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 2502, in _use_closures
    if self.deepspeed_enabled:

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.symbolic_convert: [WARNING] Graph break: inline in skipfiles: __init__ /opt/conda/envs/dynamo/lib/python3.8/contextlib.py from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1856, in _train_microbatches
    with context():

torchdynamo.variables.torch: [WARNING] Profiler will be ignored
torchdynamo.symbolic_convert: [WARNING] Graph break: copy.copy/copy.deepcopy called on non-tensor from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1905, in _train_microbatch
    device_batch = deepcopy(self.state.batch)

torchdynamo.symbolic_convert: [WARNING] Graph break: call_id with args (ListVariable(),) from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/copy.py", line 137, in deepcopy
    d = id(x)

torchdynamo.symbolic_convert: [WARNING] Graph break: call_id with args (ListVariable(),) from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/copy.py", line 202, in _deepcopy_list
    memo[id(x)] = y

torchdynamo.symbolic_convert: [WARNING] Graph break: copy.copy/copy.deepcopy does not have 1 argument from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/copy.py", line 205, in <graph break in _deepcopy_list>
    append(deepcopy(a, memo))

torchdynamo.symbolic_convert: [WARNING] Graph break: call_function UserDefinedObjectVariable(_default_get_num_samples_in_batch) [ListVariable()] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1907, in <graph break in _train_microbatch>
    microbatch_num_samples = self._train_data_spec.get_num_samples_in_batch(self.state.batch)

torchdynamo.convert_frame: [ERROR] WON'T CONVERT <graph break in _train_microbatch> /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py line 1907 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/symbolic_convert.py", line 530, in IMPORT_NAME
    value = __import__(
ModuleNotFoundError: No module named 'deepspeed'

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1908, in <graph break in _train_microbatch>
    sync_context = contextlib.nullcontext() if self.deepspeed_enabled else ddp_sync_context(

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT apply /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py line 219 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/base.py", line 144, in as_python_constant
    raise NotImplementedError(f"{self} is not a constant")
NotImplementedError: UserDefinedObjectVariable(_GenericAlias) is not a constant

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py", line 220, in apply
    input = state.batch_get_item(key=self.input_key)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT batch_get_item /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/state.py line 388 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/base.py", line 144, in as_python_constant
    raise NotImplementedError(f"{self} is not a constant")
NotImplementedError: UserDefinedObjectVariable(_GenericAlias) is not a constant

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/state.py", line 404, in batch_get_item
    return batch_get(self.batch, key)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT batch_get /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/batch_helpers.py line 12 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/base.py", line 144, in as_python_constant
    raise NotImplementedError(f"{self} is not a constant")
NotImplementedError: UserDefinedObjectVariable(_GenericAlias) is not a constant

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/batch_helpers.py", line 40, in batch_get
    if (isinstance(key, Sequence) and not isinstance(key, str) and _is_key_get_and_set_fn_pair(key)):

Set torchdynamo.config.verbose=True for more information
==========
torchinductor.ir: [WARNING] Using FallbackKernel: aten.randperm
torchdynamo.symbolic_convert: [WARNING] Graph break: numpy from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py", line 118, in cutmix_batch
    cutmix_lambda = _gen_cutmix_coef(alpha)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py", line 325, in _gen_cutmix_coef
    cutmix_lambda = np.random.beta(alpha, alpha)

torchdynamo.symbolic_convert: [WARNING] Graph break: partial tensor op: BuiltinVariable(getitem) [TensorVariable(), TupleVariable()] {} from user code at   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py", line 136, in <graph break in cutmix_batch>
    X_cutmix[:, :, rx:rw, ry:rh] = X_cutmix[shuffled_idx, :, rx:rw, ry:rh]

torchinductor.compile_fx: [WARNING] skipping cudagraphs due to multiple devices
torchdynamo.convert_frame: [ERROR] WON'T CONVERT <graph break in cutmix_batch> /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py line 136 
due to: 
Traceback (most recent call last):
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torch/fx/proxy.py", line 166, in create_arg
    raise NotImplementedError(f"argument of type: {type(a)}")
NotImplementedError: argument of type: <class 'numpy.int64'>

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/algorithms/cutmix/cutmix.py", line 136, in <graph break in cutmix_batch>
    X_cutmix[:, :, rx:rw, ry:rh] = X_cutmix[shuffled_idx, :, rx:rw, ry:rh]

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT batch_set_item /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/state.py line 406 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/base.py", line 144, in as_python_constant
    raise NotImplementedError(f"{self} is not a constant")
NotImplementedError: UserDefinedObjectVariable(_GenericAlias) is not a constant

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/core/state.py", line 425, in batch_set_item
    self.batch = batch_set(self.batch, key=key, value=value)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT batch_set /opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/batch_helpers.py line 61 
due to: 
Traceback (most recent call last):
  File "/home/ubuntu/torchdynamo/torchdynamo/variables/base.py", line 144, in as_python_constant
    raise NotImplementedError(f"{self} is not a constant")
NotImplementedError: UserDefinedObjectVariable(_GenericAlias) is not a constant

from user code:
   File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/utils/batch_helpers.py", line 97, in batch_set
    if (isinstance(key, Sequence) and not isinstance(key, str) and _is_key_get_and_set_fn_pair(key)):

Set torchdynamo.config.verbose=True for more information
==========
Traceback (most recent call last):
  File "comp.py", line 33, in <module>
    train()
  File "/home/ubuntu/torchdynamo/torchdynamo/eval_frame.py", line 166, in _fn
    return fn(*args, **kwargs)
  File "comp.py", line 27, in train
    @torchdynamo.optimize("inductor")
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1218, in fit
    def fit(
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1457, in <graph break in fit>
    self._train_loop()
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1552, in _train_loop
    def _train_loop(self) -> None:
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1555, in <graph break in _train_loop>
    log.info('Using precision %s', self.state.precision)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1613, in <graph break in _train_loop>
    total_loss_dict = self._train_batch(use_grad_scaling)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1766, in _train_batch
    def _train_batch(self, use_grad_scaling: bool) -> Dict[str, torch.Tensor]:
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1805, in <graph break in _train_batch>
    self._train_microbatches(microbatches, total_loss_dict)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1835, in _train_microbatches
    def _train_microbatches(self,
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1873, in <graph break in _train_microbatches>
    microbatch_loss_dict = self._train_microbatch(use_grad_scaling, current_batch_size, is_final_microbatch)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1891, in _train_microbatch
    def _train_microbatch(self, use_grad_scaling: bool, current_batch_size: int,
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1905, in <graph break in _train_microbatch>
    device_batch = deepcopy(self.state.batch)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/trainer/trainer.py", line 1919, in <graph break in _train_microbatch>
    self.state.outputs = self.state.model(self.state.batch)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/composer/models/tasks/classification.py", line 104, in forward
    def forward(self, batch: Tuple[Tensor, Any]) -> Tensor:
  File "/home/ubuntu/torchdynamo/torchdynamo/eval_frame.py", line 166, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/functorch/_src/aot_autograd.py", line 904, in forward
    return compiled_f(
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/functorch/_src/aot_autograd.py", line 895, in new_func
    return compiled_fn(args)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/functorch/_src/aot_autograd.py", line 228, in g
    return f(*args)
  File "/home/ubuntu/torchdynamo/torchdynamo/eval_frame.py", line 166, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/functorch/_src/aot_autograd.py", line 354, in forward
    fw_outs = call_func_with_args(
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/functorch/_src/aot_autograd.py", line 250, in call_func_with_args
    out = normalize_as_list(f(args))
  File "/opt/conda/envs/dynamo/lib/python3.8/site-packages/functorch/_src/aot_autograd.py", line 228, in g
    return f(*args)
  File "/home/ubuntu/torchdynamo/torchinductor/compile_fx.py", line 125, in run
    compiled_fn = cudagraphify_impl(model, new_inputs, static_input_idxs)
  File "/home/ubuntu/torchdynamo/torchinductor/compile_fx.py", line 165, in cudagraphify_impl
    model(*inputs)
  File "/tmp/torchinductor_ubuntu/tl/ctlxby7hf5bmlaxx2jqq3rgece36mjri4jn7smw54hxr7lgtrkaq.py", line 517, in call
    assert buf5.stride() == (18432, 576, 24, 1)
AssertionError

train          Epoch   0:    0%|                         | 0/469 [02:4
```


cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @ezyang @bdhirsh @anijain2305 @zou3519 @kiukchung @lucasllc @d4l3k @soumith @ngimel