Wav2Vec2ForCTC architecture models load incorrectly with torch 2.1 and later
Author: eindenbom
Pull request: no

### System Info

- `transformers` version: 4.35.2
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.8.0
- Huggingface_hub version: 0.19.4
- Safetensors version: 0.4.0
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.1.1 (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

 @sanchit-gandhi, @ezyang 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In python console run:
```
from transformers import AutoModelForCTC
model = AutoModelForCTC.from_pretrained("facebook/wav2vec2-base-100h")
```
Output:
```
Some weights of the model checkpoint at facebook/wav2vec2-base-100h were not used when initializing Wav2Vec2ForCTC: ['wav2vec2.encoder.pos_conv_embed.conv.weight_v', 'wav2vec2.mask_time_emb_vector', 'wav2vec2.encoder.pos_conv_embed.conv.weight_g']
- This IS expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-base-100h and are newly initialized: ['wav2vec2.encoder.pos_conv_embed.conv.parametrizations.weight.original1', 'wav2vec2.masked_spec_embed', 'wav2vec2.encoder.pos_conv_embed.conv.parametrizations.weight.original0']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

Root cause for the problem: PR #24030. New parametrization changes weight names from weight_g => parametrizations.weight.original0, weight_v => parametrizations.weight.original1.

Although new parametrization code does contain backward compat load hook, it it not used by transformers loading meachanics.

### Expected behavior

All weights are correctly loaded from checkpoint.

Comments:
rip. Not sure if there's a way to fix this without updating transformers. Have you filed a bug there?

This is transformers repo and PR breaking backward compatibility has landed here. Where should I file a bug then?

The major problem is that transformers itself lack ANY provision for backward compatibility hooks in model loading.

Oh oops, this is the right spot haha, my bad!

cc @sanchit-gandhi 

Duplicate of https://github.com/huggingface/transformers/issues/26796 - let's discuss on that thread to keep track of the issue! (note that the weights are loaded correctly, it's just the warning that needs to be updated)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.