BertTokenizer with BertJapaneseTokenizer pretrained model generates unintended tokenization.
Author: europeanplaice
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.9.0.dev0
- Platform: Windows-10-10.0.19043-SP0
- Python version: 3.9.5
- PyTorch version (GPU?): 1.9.0+cpu (False)
- Tensorflow version (GPU?): 2.5.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

`BertTokenizer` with `BertJapaneseTokenizer` pretrained model generates unintended tokenization without any caution.



## To reproduce

Steps to reproduce the behavior:

Run
```python
EXAMPLE_BERT_JAPANESE_ID = "cl-tohoku/bert-base-japanese"
tokenizer = BertTokenizer.from_pretrained(EXAMPLE_BERT_JAPANESE_ID)
print(tokenizer.tokenize("今日はいい天気ですね"))
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

```python
not_correct = BertTokenizer.from_pretrained(EXAMPLE_BERT_JAPANESE_ID)
correct = BertJapaneseTokenizer.from_pretrained(EXAMPLE_BERT_JAPANESE_ID)
print(not_correct.tokenize("今日はいい天気ですね"))
print(correct.tokenize("今日はいい天気ですね"))
```

Because the two tokenizers were made from the same pretrained model, the output should have been
```
['今日', 'は', 'いい', '天気', 'です', 'ね']
['今日', 'は', 'いい', '天気', 'です', 'ね']
```

or `BertTokenizer.from_pretrained(EXAMPLE_BERT_JAPANESE_ID)` should have raised an error.
However, the actual result was

```
['今', '日', 'はい', '##い', '天', '気', 'です', '##ね']
['今日', 'は', 'いい', '天気', 'です', 'ね']
```
and no error or warning raised.

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
I think you're raising a good issue that tokenizers will not tell you when they're instantiated from a checkpoint that doesn't have the same tokenizer architecture.

However, I think this should be resolved for all tokenizers rather than for a single one, probably by checking the `tokenizer_class` inside the `config.json` and `tokenizer_config.json`. cc @SaulLu @sgugger 

> However, I think this should be resolved for all tokenizers rather than for a single one, probably by checking the `tokenizer_class` inside the `config.json` and `tokenizer_config.json`.

I agree. `AutoTokenizer` can choose a tokenizer automatically, so until it is solved, I think that recommending a user uses `AutoTokenizer` is a better way to prevent the silent error. 



Thank you very much for reporting this problem @europeanplaice :+1:. Indeed, I share your opinion, it would be better if a warning was logged if ever the class tokenizer used to load a pretrained tokenizer is not the same type.

I also agree with @LysandreJik, it should be possible to find this information in `config.json` and/or` tokenizer_config.json` and this would allow to have a logged warning for all types of tokenizers. 

@europeanplaice, would you like to work on this? If you want, I could of course help you. Or if you don't have the time/want to, I can take over your PR in the next days and adapt it to this new approach. What do you think? :blush: 

@SaulLu Thank you for your offer. I want to try to tackle this problem.
I plan to add something like below
https://github.com/huggingface/transformers/blob/122d7dc34fd0e397a08b8a584a632fc57d3fd5d0/src/transformers/models/auto/tokenization_auto.py#L527-L551

to `from_pretrained` in `PreTrainedTokenizerBase (tokenization_utils_base.py)` to make sure that we can check whether a user is trying to use different tokenizers between `cls` and `config.json or tokenizer_config.json`'s class before a tokenizer returns.
If this detected conflicts between them, a warning would be logged, or an error would occur.

I want my PR to be in line with your overall plan, so I hope to get your opinion about this comment.


Thank you very much for offering to take care of this issue! 

From my point of view, what you described above sounds really great! :+1: 

I opened a new pull request about this issue. However, there is a point I couldn't overcome.
If `config.json` and/or `tokenizer_config.json` don't have information about the tokenizer's class, it's impossible to specify which model is correct.

In `AutoTokenizer`, it seems that TOKENIZER_MAPPING is used in this pattern, so I first intended to import `AutoTokenizer` in tokenization_utils_base.py, but it was a circular import. 😂