`Trainer` loads model weights twice when resuming from checkpoint
Author: JamesDeAntonis
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.10.0
- Platform:
- Python version: 3.8
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
@sgugger 

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): `T5` (`t5-3b`)

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Try to do training with resumption from checkpoint
2. Observe that model weights are loaded twice

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

The model weights are loaded once [here](https://github.com/huggingface/transformers/blob/v4.10.0/src/transformers/trainer.py#L260) as passed to the constructor, and then again [here](https://github.com/huggingface/transformers/blob/v4.10.0/src/transformers/trainer.py#L1074).

My proposal: I think the constructor should simply take the class of the model itself.  No model weights are loaded until `self.train`.  Inside `self.train`, the model weights path, optimizer path, etc are deduced using logic of whether `resume_from_checkpoint` is provided, `model_name_or_path`, etc

EDIT: I have also investigated the `model_init` param, but that seems to have the same problems, because it loads the model first [inside the constructor](https://github.com/huggingface/transformers/blob/v4.10.0/src/transformers/trainer.py#L296) and then again in `train`

Comments:
On a similar note, why do we put the optimizer on cuda [here](https://github.com/huggingface/transformers/blob/v4.10.0/src/transformers/trainer.py#L1621)?  Can we instead keep the optimizer on cpu?  That would save more memory

Your proposal would be a major breaking change as the `Trainer` initialization wouldn't take the model anymore, and the model attribute of the `Trainer` would not be available after the init anymore. We don't do breaking changes between minor releases, and it might even be too big for a major release.

As for your question about the the optimizer, it needs to be on CUDA like the model to be able to perform the optimizer step. I invite you to remove the line and look for yourself the error message of mismatched devices you will get.

If you want to save memory this way, you can have a look at the ZeRO-offload integration with DeepSpeed, which does offload the optimzier stat and the gradients to the CPU.

Thanks for the quick reply.  That's good to know; maybe we can live without the change.

>I invite you to remove the line and look for yourself the error message of mismatched devices you will get.

Somehow, my teammate @dblakely is able to remove the line with no error.  Not sure why.

EDIT: the reason we're able to train without specifying map location, but not with, is that without specify the map location, the optimizer gets distributed.  With map location, the optimizer gets sent to one gpu.

In short, I think the map_location shouldn't be specified at all.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.