Fix test_eager_matches_sdpa_inference for XPU backend
Author: dvrogozh
Pull request: yes
Labels: name: Tests

name: bug

Included fixes:
* Use `torch.nn.attention.sdpa_kernel` instead of deprecated `torch.backends.cuda.sdp_kernel`
* Use `torch.amp.autocast` instead of deprecated `torch.cuda.amp.autocast` in nemotron
* Reuse CUDA MATH thresholds in for XPU (as of PyTorch 2.5 XPU backend supports only `torch.nn.attention.SDPBackend.MATH`)

Fixes: #34888
CC: @amyeroberts @ydshieh

Comments:
Looks like Friday evening is not the best time to run ci. Pushed same code 3 times, seeing different errors on each run:). Not related to the change I think. Will continue on Monday :).

Clarified XPU backend behavior for `torch.backends.cuda.sdp_kernel`. As of PyTorch 2.5 (and today PyTorch main), XPU backend supports only `torch.nn.attention.SDPBackend.MATH` implementation of which is device agnostic with respect to implementation of each individual aten operator. So, we can reuse CUDA or CPU MATH weights for XPU (for all cases in disrespect to SDP backend). That's the change I did in the last version. @faaany, @EikanWang : fyi.

Thanks @dvrogozh ! LGTM.

Have you run (some of) the relevant tests on a GPU (cuda) and XPU (cc @faaany  for this part maybe) machine?




> Have you run (some of) the relevant tests on a GPU (cuda) and XPU (cc @faaany for this part maybe) machine?

I ran `python3 -m pytest --pspec -k test_eager_matches_sdpa_inference tests/models`:
* On Nvidia A10, CUDA, passing
* On Intel PVC, upstream pytorch XPU (without IPEX), passing
* On Intel PVC, with IPEX 2.3.110+xpu (pytorch 2.3.1), pasing
I also think that Transformers ci executes these CUDA tests and they are passing.

Above being said, we synced with @faany offline and she did try this PR on her side. I believe she saw tests passing for her w/o IPEX, but she mentioned that some tests are failing for her with IPEX. The later point is different from test results I have. I suspect that's due to the different IPEX versions we tried: I think @faaany tried IPEX with later version (she's offline now, I will check with her later on that).

Note that without this PR `test_eager_matches_sdpa_inference` tests fail for both upstream pytorch XPU and IPEX. The difference between upstream pytorch XPU and IPEX is which attention algorithms are supported. I know the current status for upstream pytorch XPU since I discussed that with relevant folks already. But I did not have such discussion for IPEX - that's what I will need to do. I know that IPEX might register additional operations and algorithms on top of upstream pytorch XPU, but whether it does so for SDP attention and in which version - I do not know (though I do have some understanding from the results I see so far).

For myself: need to follow up with #34941 (sdpa tests for beit) which is being reviewed in parallel.

```
FAILED tests/models/xlm/test_modeling_xlm.py::XLMModelTest::test_batching_equivalence - AssertionError: tensor(False) is not true : Batched and Single row outputs are not equal in XLMForQuestionAnswering for key=end_top_index. Difference=1.
```
Failure on ci seems unrelated. I also see some flakiness on main ci results. I tried to rebase couple times, but this did not help. No changes from last review.

yes we do have some flaky tests. (trying to fix them but in a slow peace)

There is no need to have a green CI if you think some failure are irrelevant. Ping me for a double check then we could wait for a core maintainer's review.

> There is no need to have a green CI if you think some failure are irrelevant. Ping me for a double check then we could wait for a core maintainer's review.

Yeah, I think that's the case here. I rebased today w/o any changes. Another test failed this time, I believe unrelated as well. Sorry, I am just paranoid about green ci - I consider that's my responsibility to achieve that on my PRs.

```
FAILED tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_tags - KeyError: 'url'
FAILED tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_with_saves_each_epoch - AssertionError: no logs of level WARNING or higher triggered on root
```

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34889). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.