Repository: pytorch
Issue: [Inductor] Device-specific padding settings
Labels: triaged,oncall: pt2,module: inductor

### 🚀 The feature, motivation and pitch

### Problem

In the past, Inductor only used padding on GPU devices. In order to support other devices such as MTIA, recent PRs such as #133939 and #135280 have made padding configurable via Inductor's global `config.py`. While this approach works, it would be cleaner if each device could provide its own padding configuration as opposed to setting 4-5 global configs before running Inductor. 

One pain point that illustrates the issue is `config.disable_padding_cpu`. By default, Inductor enables padding only on GPU devices. We disable this behavior when we want to pad on MTIA, essentially allowing padding on all devices, and not just MTIA. We don't currently have a way to tell PyTorch to pad specifically on MTIA, since MTIA's backend is closed source, so PyTorch is not aware of it. 

### Proposal

To get around this, I was hoping to make some common interface where each device can provide information to Inductor about what kind of code it should make. I'm aware of two places where this is currently done:
   - [Device Interface](https://github.com/pytorch/pytorch/blob/main/torch/_dynamo/device_interface.py#L36) - exposes constants such as how many processors the device has, e.g. the number of CPU cores or GPU SMs.
   - [Backend Feature](https://github.com/pytorch/pytorch/blob/a2cb9b7331524eb0d9e62b38c57d38d8725cbc1b/torch/_inductor/codegen/common.py#L163) - I'm less familiar with this, but it seems to say whether specific patterns are available for a given backend. E.g., can we do a sort. This is currently implemented as an `Enum`. Note this backend-specific (e.g. Halide vs Triton), not device-specific (e.g. H100 vs A100). 
   
Neither of these seems to be a perfect fit for the padding config. I like that `BackendFeature` is internal to Inductor, whereas `DeviceInterface` is shared with other parts of PyTorch which don't need to know about padding. On the other hand, `BackendFeature` is an `Enum`, so it's not expressive enough to represent padding configs which tend to be integers. Also, we will probably want to configure padding differently per device, as opposed to per backend. (For example, `CudaCombinedScheduling` is used for all generations of NV GPUs, which might want different padding settings.)

Right now, I'm leaning towards adding a new API to `DeviceInterface` to expose device-specific Inductor settings. But I'm open to other options. Is there a better way?

### Alternatives

### Option 1: bury heuristics inside of Inductor

Rather than exposing this to `DeviceInterface`, we could bury these heuristics within Inductor, such as
```
if isinstance(scheduling, MTIATritonScheduling):
    padding_config = {
        "comprehensive_padding" True,
    }
```

This is undesirable for closed source backends like MTIA, since it doesn't make much sense for PyTorch to have code dealing with a backend which it can't import. It's also less scalable since Inductor's codebase would need to handle all possible hardware backends. It would be cleaner if we could expose these options via some common interface that out-of-tree backends can implement themselves.

### Option 2: use existing DeviceProperties

`DeviceInterface` already has access to `DeviceProperties`, which provides similar hardware-specific parameters. The problem is that PyTorch does not seem to control these settings, since they are queried from CUDA. So we can't easily extend them.
[https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html#group__CUDART__DEVICE_1g1bf9d625a931d657e08db2b4391170f0](https://www.google.com/url?q=https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html%23group__CUDART__DEVICE_1g1bf9d625a931d657e08db2b4391170f0&sa=D&source=docs&ust=1726083264874585&usg=AOvVaw0D0T4jCiOutrb8yAFqDqqH)

### Additional context

_No response_

cc @ezyang @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire