Unclear error "NotImplementedError:  "while saving tokenizer. How fix it?
Author: MLDovakin
Pull request: no

Here is my tokenizer code and how I save it to a json file" /content/bert-datas7.json"

````
from tokenizers import normalizers
from tokenizers.normalizers import Lowercase, NFD, StripAccents

bert_tokenizer.pre_tokenizer = Whitespace()

from tokenizers.processors import TemplateProcessing

bert_tokenizer.post_processor = TemplateProcessing(
    single="[CLS] $A [SEP]",
    pair="[CLS] $A [SEP] $B:1 [SEP]:1",
    special_tokens=[
        ("[CLS]", 1),
        ("[SEP]", 2),
        ("[PAD]", 3),
    ],
    
)
from tokenizers.trainers import WordPieceTrainer

trainer = WordPieceTrainer(
    vocab_size=30522, special_tokens=["[UNK]", "[CLS]", "[SEP]", "[PAD]", "[MASK]"], pad_to_max_length=True
)
files = [f"/content/For_ITMO.txt" for split in ["test", "train", "valid"]]
bert_tokenizer.train(trainer, files)

model_files = bert_tokenizer.model.save("data", "/content/For_ITMO.txt")

bert_tokenizer.model = WordPiece.from_file(*model_files, unk_token="[UNK]",  pad_to_max_length=True)

bert_tokenizer.save("/content/bert-datas7.json") 
````

When I output tokenizer name_or_path = nothing is displayed. This is normal?


````
tokenizer = PreTrainedTokenizerFast(tokenizer_file='/content/bert-datas7.json')
tokenizer.add_special_tokens({'pad_token': '[PAD]'})

print(tokenizer)
>>> PreTrainedTokenizerFast(name_or_path='', vocab_size=1435, model_max_len=1000000000000000019884624838656, is_fast=True, padding_side='right', special_tokens={'pad_token': '[PAD]'})
````
Also, when I try to save my tokenizer, I get an error without explanation. How can I rewrite the code so that all this???
#9658 
#10039 
[For_ITMO.txt-vocab (1) (1).txt](https://github.com/huggingface/transformers/files/5945659/For_ITMO.txt-vocab.1.1.txt)
  
````
tokenizer.save_pretrained("/content/tokennizerrrr")

NotImplementedError                       Traceback (most recent call last)
<ipython-input-11-efc48254a528> in <module>()
----> 1 tokenizer.save_pretrained("/content/tokennizerrrr")

2 frames
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py in save_vocabulary(self, save_directory, filename_prefix)
   2042             :obj:`Tuple(str)`: Paths to the files saved.
   2043         """
-> 2044         raise NotImplementedError
   2045 
   2046     def tokenize(self, text: str, pair: Optional[str] = None, add_special_tokens: bool = False, **kwargs) -> List[str]:

NotImplementedError: 
````


Comments:
Maybe @n1t0 can chime in here!

> When I output tokenizer name_or_path = nothing is displayed. This is normal?

I think it is yes, you are loading using `tokenizer_file=` instead of using the normal path with `from_pretrained`. No need to worry about this.

Concerning the error, I think the way to avoid it is by specifying `legacy_format=False`:
```python
tokenizer.save_pretrained("/content/tokennizerrrr", legacy_format=False)
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.