Improve mismatched sizes management when loading a pretrained model
Author: regisss
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Examples currently fail when the loaded model head has different dimensions from the expected ones. For instance, in the image classification example, if a pretrained classification head has different dimensions from the classification head to fine-tune, the current implementation will lead to this error:
```
Traceback (most recent call last):
  File "run_image_classification.py", line 377, in <module>
    main()
  File "run_image_classification.py", line 267, in main
    model = AutoModelForImageClassification.from_pretrained(
  File "/home/regis/HuggingFace/dev/transformers/venv/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 446, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "/home/regis/HuggingFace/dev/transformers/venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2067, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model(
  File "/home/regis/HuggingFace/dev/transformers/venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2276, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for SwinForImageClassification:
	size mismatch for classifier.weight: copying a param with shape torch.Size([1000, 1024]) from checkpoint, the shape in current model is torch.Size([3, 1024]).
	size mismatch for classifier.bias: copying a param with shape torch.Size([1000]) from checkpoint, the shape in current model is torch.Size([3]).
```

To reproduce this, you can run the image classification example with Swin, such as:
```
python run_image_classification.py \
  --dataset_name beans \
  --output_dir /tmp/beans_outputs/ \
  --remove_unused_columns False \
  --do_train \
  --per_device_train_batch_size 8 \
  --model_name_or_path microsoft/swin-base-patch4-window7-224
```

The solution is to add the argument `ignore_mismatched_sizes=True` to the `AutoModelForXXX.from_pretrained` method. Thus, this PR does the following:
- expand the error message and suggest a solution when the error is raised
- for all classification examples, an argument `--ignore_mismatched_sizes` can now be given to adapt the dimensions of the classification head when they are different from the expected ones


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I see that all tests corresponding to mismatched sizes failed. Looking at [test_modeling_common.py](https://github.com/huggingface/transformers/blob/ee393c009a243bbb86fa11d2efa771a1704d85cf/tests/test_modeling_common.py#L2191), I guess the current behaviour is expected.

Shall I close this PR and open a new issue regarding the fact that some models will fail in examples (such as Swin for image classification as explained in the first message)?

I'm not sure why you would open a new issue: this is not a bug and there is an option to load a model with a pretrained head that has different shapes from the checkpoint. Or do you mean the examples should have a flag to activate that option? 

> I'm not sure why you would open a new issue: this is not a bug and there is an option to load a model with a pretrained head that has different shapes from the checkpoint. Or do you mean the examples should have a flag to activate that option?

After using it a bit more I realized it's not a bug indeed, sorry for the confused wording. The problem is that just changing the model used in the example may break it and I couldn't fine anywhere in the doc or in the READMEs how to solve this, I had to take a look at the code. I suggest one of the following to make it more user-friendly:
- the error message suggests to add the argument `ignore_mismatched_sizes=True` to `AutoModelForXXX.from_pretrained`
- adding a flag to activate that option as you propose, with a mention in the README
- changing the default value of `ignore_mismatched_sizes` to `True` since a warning is displayed when sizes are different, but I guess I'm lacking of context here and I'm just considering this example use case

Again I'm certainly lacking of context here but I would be happy to modify this PR so that it makes using a different model in examples less tedious when the pretrained head has different dimensions :)

>     * the error message suggests to add the argument `ignore_mismatched_sizes=True` to `AutoModelForXXX.from_pretrained`

This is definitely something we can add and would help the user!

>     * adding a flag to activate that option as you propose, with a mention in the README

Yes, another welcome improvement!

>     * changing the default value of `ignore_mismatched_sizes` to `True` since a warning is displayed when sizes are different, but I guess I'm lacking of context here and I'm just considering this example use case

This can't be done for backward compatibility reasons. In further work in `from_pretrained`, we might have a default that does this, but only for the head of the model. It's dangerous to have it enabled by default on the whole body.

Great, I'm going to modify this PR accordingly!

I just modified the *classification* examples because I'm not sure about the types of head used in other scenarios.

Also, I noticed that VSCode automatically trimmed extra whitespaces (because I configured it this way). Let me know if this is an issue and I'll revert that.