save_pretrained doesn't work with GPT2FastTokenizer
Author: bilal2vec
Pull request: no
Labels: name: Core: Tokenization

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2TokenizerFast

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
> from transformers import *
> tok = GPT2TokenizerFast.from_pretrained('distilgpt2')
> tok.save_pretrained('./')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bilal/Documents/transformers/src/transformers/tokenization_utils.py", line 519, in save_pretrained
    vocab_files = self.save_vocabulary(save_directory)
  File "/Users/bilal/Documents/transformers/src/transformers/tokenization_utils.py", line 529, in save_vocabulary
    raise NotImplementedError
NotImplementedError
```

## Expected behavior

The tokenizer should be able to be saved

## Environment info
     
- `transformers` version: 2.4.1
- Platform: Darwin-19.3.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.3.1 (False)
- Tensorflow version (GPU?): 2.0.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

Comments:
After upgrading to 2.5.0, the code now throws

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bilal/Documents/transformers/src/transformers/tokenization_utils.py", line 587, in save_pretrained
    return vocab_files + (special_tokens_map_file, added_tokens_file)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'tuple'
```

Hi @bkkaggle , 

Thanks for reporting the issue, it should be fixed through https://github.com/huggingface/transformers/pull/2918 and land very soon on master.

I'll be included in the first maintenance release following 2.5

Morgan

Saving tokenizers works now, but restoring them doesn't
```
> from transformers import *
> tok = GPT2TokenizerFast.from_pretrained('distilgpt2')
> tok.save_pretrained('./')
('./vocab.json-vocab.json', './vocab.json-merges.txt', './special_tokens_map.json', './added_tokens.json')
> tok = GPT2TokenizerFast.from_pretrained('./')
RobertaTokenizerFast has an issue when working on mask language modeling where it introduces an extra encoded space before the mask token.See https://github.com/huggingface/transformers/pull/2778 for more information.
> tok.tokenize('test')
[]
```

I can't reproduce on my side, using your code:

```python
tok = GPT2TokenizerFast.from_pretrained('distilgpt2')
tok.save_pretrained('./')
> ('./vocab.json-vocab.json', './vocab.json-merges.txt', './special_tokens_map.json', './added_tokens.json')

tok = GPT2TokenizerFast.from_pretrained('./')
tok.tokenize('test')
> ['test']
```

I made a colab notebook to reproduce the error
The error appears when installing from source on the master branch

colab: https://colab.research.google.com/drive/1OJdm6LzVtyb-biVR1ky6joSX7gBgl6St

Thanks, I'm able to reproduce now, I'll have a look hopefully tomorrow morning.

I'll keep you posted here 👀 

Fixed