Running into AttributeErrorAttributeError from 4.37.0
Author: ningziwen
Pull request: no

### System Info

Only happens from 4.37.0
```
- `transformers` version: 4.37.0
- Platform: Linux-5.10.192-183.736.amzn2.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: 0.26.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 1.13.1+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

Switching back to 4.36.2 and it works well.
```
- `transformers` version: 4.36.2
- Platform: Linux-5.10.192-183.736.amzn2.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.20.2
- Safetensors version: 0.4.1
- Accelerate version: 0.26.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 1.13.1+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

### Who can help?

@fxmarty, @michaelbenayoun, @amyeroberts

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Run `torchrun` against one the simple file.

```
if __name__ == '__main__':
    from transformers.utils.fx import HFTracer
```

Running into

```
Traceback (most recent call last):
Traceback (most recent call last):
  File "/test/bin/pytorch_tests/testCustom", line 2, in <module>
  File "/test/bin/pytorch_tests/testCustom", line 2, in <module>
        from transformers.utils.fx import HFTracerfrom transformers.utils.fx import HFTracer

  File "/usr/local/lib/python3.10/site-packages/transformers/utils/fx.py", line 611, in <module>
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/fx.py", line 611, in <module>
        torch.nn.functional.scaled_dot_product_attention: torch_nn_functional_scaled_dot_product_attention,torch.nn.functional.scaled_dot_product_attention: torch_nn_functional_scaled_dot_product_attention,

AttributeErrorAttributeError: : module 'torch.nn.functional' has no attribute 'scaled_dot_product_attention'module 'torch.nn.functional' has no attribute 'scaled_dot_product_attention'. Did you mean: '. Did you mean: '_scaled_dot_product_attention_scaled_dot_product_attention'?'?

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 31) of binary: /usr/local/bin/python3.10
Traceback (most recent call last):
  File "/usr/local/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/usr/local/lib/python3.10/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/usr/local/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/usr/local/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
/test/bin/pytorch_tests/testCustom FAILED 
```

### Expected behavior

Should succeed.

Comments:
Seems related to this recent change. https://github.com/huggingface/transformers/pull/28447

Found the PT commit adding this interface. https://github.com/pytorch/pytorch/commit/df14650f0b14b80db132b0c1797dc595fbee1054

This is only added in PT 2.0. PT 1.13.1 does not have it. https://github.com/pytorch/pytorch/blob/49444c3e546bf240bed24a101e747422d1f8a0ee/torch/nn/functional.py#L4808

Even if PT 1.10 is not supported from 4.37.0, PT 1.13.1 is still supported right?

Hi @ningziwen thank you, indeed I did not add a guard on torch>=2.0. Will fix, thank you.

Hi @ningziwen, this is fixed on main with https://github.com/huggingface/transformers/pull/28774. Thank you!