1x model size CPU memory usage for `from_pretrained` 
Author: stas00
Pull request: yes

The problem is that currently to load a model using `from_pretrained` requires 2x model size on CPU memory and for those odd cases where a user has large GPU memory, but very little CPU memory it's a problem.

This PR is trying to solve the puzzle of loading "EleutherAI/gpt-j-6B", with revision="float16" and torch_dtype=torch.float16 for the model in full FP16. it shouldn't use more than 12.1GB of cpu RAM to work on colab.

So this should use 12.1GB of CPU memory including peak memory:
```
m = GPTJForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", torch_dtype=torch.float16, low_cpu_mem_usage=True)
```

This of course won't be enough to be used on the free google colab since it has a total of 12GB of CPU RAM for everything, so by the time we come to `from_pretrained` a few GBs will be already consumed by python/torch/transformers, leaving less than 10GB of CPU RAM.

Here is what this PR does:

1. load state_dict and register which keys we have
2. drop state_dict
3. switch to the meta device all params/buffers that are going to be replaced from the loaded state_dict
4. load state_dict 2nd time
5. replace the params/buffers from the state_dict

See the snapshot showing the memory use of just 1x model size and a bit more for temps (this includes peak memory usage).

(The automatic cpu/gpu memory usage reporting in the snapshot is courtesy of https://github.com/stas00/ipyexperiments)
![snapshot_27](https://user-images.githubusercontent.com/10676103/132447001-61652e49-1e79-4d7c-adad-4de0d685194b.png)

TODO:

- if this gets a green light - need to write a test that measure 1x memory size

@LysandreJik, @sgugger 

Comments:
> It looks like the very first state_dict is only instantiated to get the list of keys in the state dict, maybe we should cache that (same path as `resolved_archive_path` but with some extension) so the subsequent loads are a bit faster? What do you think?

I'm all ears if you have suggestions. I'm not sure how we can peek at the list of keys w/o loading the full state_dict. The key is that we can't cache the values, as they consume memory. But you're proposing to just save the keys somewhere on the filesystem.

On the other hand I'm not sure this will be widely used, so perhaps it doesn't have to be super-efficient. I'm not sure.

If this proposal https://github.com/pytorch/pytorch/issues/64327 results in a solution then we should be able to load state_dict's keys w/o loading the dict itself down the road.


Also note that this experimental function doesn't handle `missing_keys`, `unexpected_keys`, `mismatched_keys` - so it's more of an experiment still for those who urgently wanted the functionality.

@stas00 Is this ready to be merged? Sorry for nudging - I tried running `stas00:low-mem-load` and it errors out with an error ("LayerNormKernelImpl" not implemented for 'Half') for which the fix was merged into `huggingface/transformers`.

I think we have been waiting for the EAI group to confirm that this is what they want since they asked for this feature in the first place, but otherwise there is no reason not to merge it.

I guess let's just do it.