Error when loading models to DirectML after PR#24505
Author: pengan1987
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.10.13
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.3.3
- Accelerate version: 0.23.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgu

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I'm trying to run Phi-1.5 using Pytorch with DirectML `torch-directml==0.2.0.dev230426`, my script is here:
https://github.com/pengan1987/DirectML-demos/blob/main/Phi_1_5.py
When `transformers<=4.30.2`, it works fine, when ` transformers>=4.31.0`, I got an exception like this when loading the model
It seems `modeling_utils.py` has been changed a lot in PR#24505 and caused this exception, is there any fix/workarond to address this issue?
```
E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\torch\utils\_device.py:62: UserWarning: TypedStorage is deprecated. It will be removed in the future and Unty
pedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_sto
rage() instead of tensor.storage()
  return func(*args, **kwargs)
Traceback (most recent call last):
  File "E:\StableDiffusion\DirectML-demos\Phi_1_5.py", line 14, in <module>
    model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\models\auto\auto_factory.py", line 488, in from_pretrained
    return model_class.from_pretrained(
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\modeling_utils.py", line 2903, in from_pretrained
    ) = cls._load_pretrained_model(
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\modeling_utils.py", line 3061, in _load_pretrained_model
    id_tensor = id_tensor_storage(tensor) if tensor.device != torch.device("meta") else id(tensor)
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\pytorch_utils.py", line 287, in id_tensor_storage
    return tensor.device, storage_ptr(tensor), storage_size(tensor)
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\safetensors\torch.py", line 33, in storage_size
    return tensor.untyped_storage().nbytes()
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\torch\utils\_device.py", line 62, in __torch_function__
    return func(*args, **kwargs)
NotImplementedError: Cannot access storage of OpaqueTensorImpl
```

### Expected behavior

model loading correctly like `transformers<=4.30.2`

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @pengan1987, thanks for raising this issue! 

Two comments: 
* There have been a few recent changes to loading of weights in recent releases. Could you try with the latest version of transformers to see if that works? 
* Could you provide a minimal code snippet which reproduces the error? Without being able to get the error on our side we won't be able to help

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.