Issue: Datasets' cache not re-used

Comment:
`tokenizer.save_pretrained(training_args.output_dir)` produces a different tokenizer hash when loaded on restart of the script. When I was debugging before I was terminating the script prior to this command, then rerunning. 

I compared the tokenizer items on the first and second runs, there are two different items:
1st:
```
('_additional_special_tokens', [AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True)])

...

('tokens_trie', <transformers.tokenization_utils.Trie object at 0x7f4d6d0ddb38>)
```

2nd:
```
('_additional_special_tokens', [AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True), AddedToken("
", rstrip=False, lstrip=False, single_word=False, normalized=True)])

...

('tokens_trie', <transformers.tokenization_utils.Trie object at 0x7efc23dcce80>)
```

 On every run of this the special tokens are being added on, and the hash is different on the `tokens_trie`. The increase in the special tokens category could be cleaned, but not sure about the hash for the `tokens_trie`. What might work is that the call for the tokenizer encoding can be translated into a function that strips any unnecessary information out, but that's a guess.