Fix tokenizer saving during training with `Trainer`
Author: SaulLu
Pull request: yes

**EDIT:** If you have already read this message, you can see the update directly in [this message](https://github.com/huggingface/transformers/pull/12806#issuecomment-899570500)

# What does this PR do?

Fixes #12762. 

As shown in this issue, a training launched with the `Trainer` object can fail at the time of the saving of the tokenizer if this last one belongs to a tokenizer class that has a slow version and a fast version but that it was instantiated from a folder containing only the files which permit to initialize the fast version.

I think it is important to solve this problem because this case will happen "more often" with the addition of the new feature to train new tokenizers from a known architecture.

To be even more precise, it does not fail in all cases, it depends on the implementation of the `save_vocabulary` method of the slow version. For example, `BertTokenizer` knows how to generate the slow files but `AlbertTokenizer` copies the files from the files in the folder where the tokenizer was initialized.

My PR proposes to add a test in the trainer tests to test this version that fails. 

The fix I propose is really to be discussed because it changes the generated files in some cases (Bert's for example) but it seems to me to be the simplest fix.

If we ever want the behavior not to be modified in the case of the Bert tokenizer, we should find a way to know if the `save_vocabulary` method is able to generate the slow version from the fast version or not. I'm not sure I see a simple way to do this - unless we use a `try/except block. What I see as an alternative would be to add a property to all tokenizer classes that would allow knowing if the `save_vocabulary` method can be executed even without the slow versions present in the initial folder.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. @LysandreJik and @sgugger , I would love to hear your thoughts on this PR even though it is certainly not in its final form.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
I don't think this is the right fix: it just hides the problem as `tokenizer.save_pretrained()` should always work. And it should always save all the possible files, to make it easy for users to share their tokenizer with other users that might not have the `tokenizers` or the `sentencepiece` library installed.

Adding a new class property that tells the tokenizer whether it can save the vocabulary from the fast tokenizer sounds the best fix to me, with the try-excepts blocks in the `save_pretrained` method as the second best option.

**Update**: I have therefore redone this PR to take into account the return of @sgugger. This time, I propose to add a new `can_save_slow_tokenizer` attribute to all fast tokenizers:
- if the tokenizer fast is able to rebuild the files of the tokenizer slow, the attribute is set to `True`
- otherwise it is set to `False`

I use this attribute in ` _save_pretrained` of `PreTrainedTokenizerFast` so that with the default arguments:
- if possible the slow and fast files are saved
- otherwise only fast files are saved

For information, all slow tokenizers using `spm` have now the `can_save_slow_tokenizer`attribute set to `False`, all other tokenizers have their attribute left at `True`.

Failing tests: I don't think this is related to the current PR but I'll have a closer look :thinking: 

You'll probably need to rebase for the tests to pass.

@sgugger, @LysandreJik, Thank you very much for your reviews! I have normally applied all your suggestions and after the rebase the tests all pass well :smile: 

In that case, feel free to merge your PR :-)