Loading repository after rename does not work (with old name)
Author: osanseviero
Pull request: no
Labels: name: Feature request

### System Info

```shell
- `transformers` version: 4.19.2
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- Huggingface_hub version: 0.7.0
- PyTorch version (GPU?): 1.11.0+cu113 (False)
- Tensorflow version (GPU?): 2.8.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

cc @LysandreJik

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Context
```
FROM LOCATION (old location): https://huggingface.co/KB/bert-base-swedish-cased
TO LOCATION (new location): https://huggingface.co/KBLab/bert-base-swedish-cased
```

The following breaks
```py
from transformers import AutoModel
model = AutoModel.from_pretrained("KB/bert-base-swedish-cased")
```

Error
```
OSError: Can't load config for 'KB/bert-base-swedish-cased'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'KB/bert-base-swedish-cased' is the correct path to a directory containing a config.json file
```

What works here?
:white_check_mark: going to original path redirects to new path
:white_check_mark: doing git clone
:white_check_mark: using huggingface_hub model_info("KB/bert-base-swedish-cased")
:x: using transformers AutoModel.from_pretrained("KB/bert-base-swedish-cased")

### Expected behavior

```shell
Loading the model with the old name would still work so users loading a transferred model don't have an impacted experience
```


Comments:
We have since the issue was opened here duplicated some of the most used repositories and re-uploaded them. The above code example will therefore work now (because we duplicated the repo). 

Here is a fresh example of a transferred repository we haven't manually duplicated to replicate the redirection error:

```python
from transformers import AutoModel
model = AutoModel.from_pretrained("KB/bert-base-swedish-cased-alpha") # New location is KBLab//bert-base-swedish-cased-alpha
```



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I think this issue needs to be addressed still, so I will provide some extra context that may not have been apparent in the original post. 

### Context

We previously hosted our models on a user account named [KB](https://huggingface.co/KB). After organization accounts became available, we created an organization called [KBLab](https://huggingface.co/KBLab) where we uploaded our newer models. In the beginning of June, we decided to transfer all our models from the user [KB](https://huggingface.co/KB) to the organization [KBLab](https://huggingface.co/KBLab). Under the "Settings" tab for individual models, it is stated that models will be **automatically redirected** to the new location after the renaming or transferring of the models:

![image](https://user-images.githubusercontent.com/7157234/177817202-3a61a9ac-39d7-4a80-841c-6655170eb8f9.png)

### Expected behavior

We interpret "automatic redirection" being an assurance of the the following functionality working after a rename/transfer:

1. Old URLs to models will redirect to new URL in browser. 
2. Git operations on old URLs will redirect to new URL. 
3. Using Huggingface Hub functions with old URL will redirect properly. 
4. Using `transformers` package with old URL and `AutoModel.from_pretrained("OLD_URL")` will redirect to new URL and successfully load all transferred models. 

In reality cases 1, 2 and 3 work well and provide automatic redirection :heavy_check_mark:, whereas case number 4 does *not* provide functioning automatic redirection :x: .

### Reproduction and errors

Cases 1, 2,  and 3 from above work fine when transferring a model repository to a new organization. However, case number 4, using the `transformers` package with `Automodel.from_pretrained()`, does *not* redirect the models properly. 

To reproduce this error:

* Create a model in any repository you have read/write access to.
* Transfer that same model to another user/organization you also have read/write access to. 
* Try loading the model with the old URL using `Automodel.from_pretrained()`.

We can provide an example from our own transfer of models below. 

We transferred the repository previously located at [https://huggingface.co/KB/bert-base-swedish-cased-alpha](https://huggingface.co/KB/bert-base-swedish-cased-alpha) to the organization KBLab at [https://huggingface.co/KBLab/bert-base-swedish-cased-alpha](https://huggingface.co/KBLab/bert-base-swedish-cased-alpha). Notice how the first link (KB/bert-base-swedish-cased-alpha) automatically redirects to the second link in the browser (KBLab/bert-base-swedish-cased-alpha). 

However, in python, when trying to load the model with the `transformers` package, the old URL does not work:

```python
from transformers import AutoModel
model = AutoModel.from_pretrained("KB/bert-base-swedish-cased-alpha")
```
Leads to the error

```
OSError: Can't load config for 'KB/bert-base-swedish-cased-alpha'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'KB/bert-base-swedish-cased-alpha' is the correct path to a directory containing a config.json file
```

Would appreciate if @LysandreJik  or someone else could take a look at this. I imagine other users and organizations will expect the transferring and renaming of models  to provide automatic redirection when loading the models in the `transformers` package.

Also cc @sgugger for visibility

This is because Transformers does not rely on `huggingface_hub` for loading models since its loading mechanism predates that library. We will switch soon to using `huggingface_hub` behind the scenes, which should solve this issue.

See issue linked above, Transformers now uses `hf_hub_download` behind the scenes, but that function does not handle redirections (yet). The bug will be fully solved once this issue is resolved.