Access denied to pretrained GPT2 model
Author: ciwang
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): GPT2

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [ ] the official example scripts: (give details)
* [x] my own modified scripts: I cannot load the GPT2 small pretrained model.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: I am trying to instantiate a GPT2 pretrained model.

## To Reproduce

Steps to reproduce the behavior:

```
from pytorch_transformers import AutoModel
model = AutoModel.from_pretrained('gpt2')
```

This only happens with the 'gpt2' shortcut, not others ('gpt2-medium', etc.)

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

Error message: 
```
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
<ipython-input-14-e057f5f0ba3e> in <module>
      1 from pytorch_transformers import AutoModel
----> 2 model = AutoModel.from_pretrained('gpt2')

~/pipeline/.venv/lib/python3.7/site-packages/pytorch_transformers/modeling_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    246             return OpenAIGPTModel.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
    247         elif 'gpt2' in pretrained_model_name_or_path:
--> 248             return GPT2Model.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
    249         elif 'transfo-xl' in pretrained_model_name_or_path:
    250             return TransfoXLModel.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)

~/pipeline/.venv/lib/python3.7/site-packages/pytorch_transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    537 
    538         if state_dict is None and not from_tf:
--> 539             state_dict = torch.load(resolved_archive_file, map_location='cpu')
    540         if from_tf:
    541             # Directly load from a TensorFlow checkpoint

~/pipeline/.venv/lib/python3.7/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    385         f = f.open('rb')
    386     try:
--> 387         return _load(f, map_location, pickle_module, **pickle_load_args)
    388     finally:
    389         if new_fd:

~/pipeline/.venv/lib/python3.7/site-packages/torch/serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    562             f.seek(0)
    563 
--> 564     magic_number = pickle_module.load(f, **pickle_load_args)
    565     if magic_number != MAGIC_NUMBER:
    566         raise RuntimeError("Invalid magic number; corrupt file?")

UnpicklingError: invalid load key, '<'.
```

Contents of the downloaded file:
```
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>D027DE3363BB3D26</RequestId><HostId>GPDcAN+fZerpFZ5ZR9ZnATk3XIJ4GgLjCDMLnzvs48MRKG8soooyb8HM+zjBA0Gnn7HJc4CRqpA=</HostId></Error>%
```

## Expected behavior

Successfully load the pretrained model.

## Environment

* OS: macOS Catalina
* Python version: 3.7.4
* PyTorch version: 1.1.0
* PyTorch Transformers version (or branch): 1.1.0
* Using GPU ? No
* Distributed of parallel setup ? No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
I'm having the same error

There is a known (temporary) issue with our `gpt2` model – can you guys use `gpt2-medium` or `distilgpt2` instead for now?

cc @LysandreJik @thomwolf @n1t0 @clmnt 

Sure thing! Thanks for letting us know :)

(should be fixed now)

Thank you!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.