Repository: pytorch
Issue: [RFC] Add Intel GPU Support to Torch Test Cases
Labels: triaged,module: testing,module: xpu

### 🚀 The feature, motivation and pitch

### Motivation
For [[RFC]Add Intel GPU Support in PyTorch CI/CD,](https://github.com/pytorch/pytorch/issues/114850) we already enabled Intel GPU support in test cases of Inductor ([#122866,](https://github.com/pytorch/pytorch/pull/122866) [#124147](https://github.com/pytorch/pytorch/pull/124147)) and Profiler ([#134316](https://github.com/pytorch/pytorch/pull/134316/files)). As a step forward, we’d like to enable Intel GPU in other Torch test cases to achieve better test coverage.

### Approach 
Because Intel GPU functionality is on par with other GPU devices, in Inductor and Profiler tests we did generalization for device specific cases to reuse them. We’d like to extend the generalization approach to other test cases.  

We assume only one kind of GPU device will be active in the test at one time, for example no CUDA and Intel GPU cases will run in the same test. "XPU" is the device type of Intel GPU in PyTorch. To generalize the Inductor tests we have introduced a GPU_TYPES list for all the GPU devices on par functionally, and GPU_TYPE to indicate the current GPU device. For test parameterization with instantiate_device_type_tests(), we will adopt the following approaches for device generalization: 
![upstream](https://github.com/user-attachments/assets/35c73c75-35be-47ed-ab98-3f3fc1488ab6)

- We have introduced XPUTestBase class for the Intel GPU test and supported it in instantiate_device_type_tests(), PR #[120891](https://github.com/pytorch/pytorch/pull/120891).  
- For decorators, we do generalization, for example, change onlyCUDA to onlyGPU so that all the devices in GPU_TYPES can run the tests. 
- We also make op_db general for all GPU devices, introduce dtypesIfGPU to replace dytpesIfCUDA; for “skips” with device_type specified we unify them with device_type=GPU_TYPE. For example, change

![opdb](https://github.com/user-attachments/assets/212543d1-0910-49a4-8fd6-067c5d81d9e4)
into
![opdb2](https://github.com/user-attachments/assets/fb7ea320-14fc-43f6-8945-99d0f813b955)

- For API on device module, we can generalize it with torch.get_device_module(), for example torch.get_device_module().FloatTensor. If needed, we would also use [torch.accelerator](https://github.com/pytorch/pytorch/pull/132204) interface.  
- If needed, we could add new code or wrapper functions to remove the device specific code. For example, add get_gpu_autocast() for torch.cuda.amp.autocast:
```
def get_gpu_autocast(): 
    return torch.cuda.amp.autocast if HAS_CUDA else torch.amp.autocast 
```
For code
```
with torch.backends.cudnn.flags(enabled=True): 
```
we could update it with 
```
context = torch.backends.cudnn.flags(enabled=True) if HAS_CUDA else contextlib.nullcontext()         
with context: 
```
### Plan
We are going to make all the modules Intel GPU supported device agnostic. We have enabled the “XPU” backend in test infrastructure ([#120891](https://github.com/pytorch/pytorch/pull/120891)) and enabled inductor ([#122866,](https://github.com/pytorch/pytorch/pull/122866) [#124147](https://github.com/pytorch/pytorch/pull/124147)) and profiler tests ([#134316](https://github.com/pytorch/pytorch/pull/134316/files)). As a staging plan for the next step, we will

- Work on the generalization of Aten:ops tests starting from test_ops.py and test_nn.py. 
- Work on tests for runtime, distributed, quantization and sparsity, etc.  
- More modules will be added according to new feature planning.  

### Alternatives

_No response_

### Additional context

_No response_

cc @gujinghui @EikanWang @fengyuan14 @guangyey