OPT-350m cannot be loaded from local files generated using the save_pretrained method
Author: greg2451
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.19.3
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.9.7
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO
```


### Who can help?

@younesbelkada @patrickvonplaten @Lys

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Load opt-350m model from the hub with `AutoModelForCausalLM.from_pretrained('facebook/opt-350m')`
2. Save the model using `model.save_pretrained('save_dir/')`
3. Try loading back the model with `AutoModelForCausalLM.from_pretrained('save_dir/')`

Full code:


```python
from transformers import AutoModelForCausalLM

opt350m = AutoModelForCausalLM.from_pretrained('facebook/opt-350m')
opt350m.save_pretrained("local_save_dir/")
loaded_opt = AutoModelForCausalLM.from_pretrained('local_save_dir/')
```

### Expected behavior

```shell
A RuntimeError will be raised when loading the model from save_dir/

Logs:

Traceback (most recent call last):
  File "/Users/gregoireretourne/opt/miniconda3/envs/health/lib/python3.9/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 5, in <module>
  File "lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 446, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "lib/python3.9/site-packages/transformers/modeling_utils.py", line 2059, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model(
  File "lib/python3.9/site-packages/transformers/modeling_utils.py", line 2251, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for OPTForCausalLM:
	size mismatch for lm_head.weight: copying a param with shape torch.Size([50272, 512]) from checkpoint, the shape in current model is torch.Size([50272, 1024]).
```


Comments:
From what I understand, this is a related to a known issue that is unique to opt-350m amongst the opt models, because the `hidden_size` (512) is different from the `word_embed_proj_dim` (1024).

When I load the opt model using the hub and do `opt350m.lm_head.weight.shape` I have `torch.Size([50272, 512])`. 

When I manually load the weight files saved by the `save_pretrained` method, and go to the lm_head weight, it is also `torch.Size([50272, 512])`

But for some reasons, as the lm_head is a `Linear(in_features=1024, out_features=50272, bias=False)`, there is a problem loading the weights in the model.

I have tried to reproduce with other models (namely `opt-125m` and `opt-1.3b`), but it works well with them.

Hmm sorry I did not see it was fixed in last release. Duplicate of #17389. Closing

Just to clarify, I just ran the command with Transformers >= 4.20.1 and it works as expected. Are there still any problems?

Yes @patrickvonplaten, now it works as expected, just had to bump my transformers version