Error:"TypeError: 'NoneType' object is not callable" with model specific Tokenizers
Author: afcruzs
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.17.0.dev0 but also current latest master
- Platform: Colab
- Python version:
- PyTorch version (GPU?): 
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

Models:

- XGLM (although I don't think this is model specific)

## To reproduce

Steps to reproduce the behavior:

1. Try to create a tokenizer from class-specific tokenizers (eg XGLMTokenizer), it fails.

```
from transformers import XGLMTokenizer
tokenizer = XGLMTokenizer.from_pretrained("facebook/xglm-564M")
```

## Expected behavior

Should work, but it fails with this exception: 
```
Error:"TypeError: 'NoneType' object is not callable" with Tokenizers
```

However, creating it with `AutoTokenizer` just works, this is fine, but there're a lot of examples for specific models which do not use AutoTokenizer (I found out this by pasting an example from XGLM)

Comments:
cc @SaulLu for tokenizers

Oops #15827 existed already, this seems to be widespread to more than XGLMTokenizer. XLMRobertaTokenizer doesn't seem to work either. AutoTokenizer seems to work but because it's a separate class altogether (`PreTrainedTokenizerFast`)

Hi @afcruzs ! `XGLMTokenizer` depends on `sentencepiece`, if it's not installed then a None object is imported.

`pip install sentencepiece` should resolve this.

> AutoTokenizer seems to work but because it's a separate class altogether (PreTrainedTokenizerFast)

yes, by default `AutoTokenizer` returns fast tokenizer.

Oh yes, this works indeed. Thanks! It'd be great to have a meaningful error message though, is not at all evident.

If I understood correctly, you tried with the version of transformers on master. After running:
```python
from transformers import XGLMTokenizer
tokenizer = XGLMTokenizer.from_pretrained("facebook/xglm-564M")
```
normally you should see this error message after:
```python
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-3-34ff8e92d7d6>](https://localhost:8080/#) in <module>()
----> 1 tokenizer = XGLMTokenizer.from_pretrained("facebook/xglm-564M")

1 frames
[/usr/local/lib/python3.7/dist-packages/transformers/file_utils.py](https://localhost:8080/#) in requires_backends(obj, backends)
    846     failed = [msg.format(name) for available, msg in checks if not available()]
    847     if failed:
--> 848         raise ImportError("".join(failed))
    849 
    850 

ImportError: 
XGLMTokenizer requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/google/sentencepiece#installation and follow the ones
that match your environment.


---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------
```

Can you share with us a little more about what you did so we can understand why you didn't receive this error? :slightly_smiling_face: 

@SaulLu I've tried to repro it again and i couldn't, it seems to work just like you said 😐. I'll take a closer look and share a colab if I find something, otherwise will close this. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing this issue as it seems to be solved :smile: 

> Oh yes, this works indeed. Thanks! It'd be great to have a meaningful error message though, is not at all evident.

Mine does not work still with distilbert tokenizer

@aiendeveloper , that would be super helpful if you can share with us your environment info (that you can get with the `transformers-cli env` command). Thank you!

![image](https://user-images.githubusercontent.com/90980083/181121007-e3182991-0320-4ab8-89c5-d8ff31f89538.png)
I am getting this error for xlm-roberta-base model. Any help will be very helpful.

@tanuj2212, My hunch is that you are missing the `sentencepiece` dependency. Could you check if you don't have it in your virtual environment or install it with for example `pip install sentencepiece`? 

What version of `transformers` are you using? I would have expected you to get the following message the first time you tried to use `XLMRobertaTokenizer` :
```bash
XLMRobertaTokenizer requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/google/sentencepiece#installation and follow the ones
that match your environment.
```

I am having the same issue with T5Tokenizer - I have installed sentencepiece and I still get the NoneType error. I am working on a Google Colab file and here is my environment info from the `transformers-cli env` command:

- `transformers` version: 4.24.0
- Platform: Linux-5.10.133+-x86_64-with-glibc2.27
- Python version: 3.8.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.12.1+cu113 (False)
- Tensorflow version (GPU?): 2.9.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

Also, `pip install sentencepiece` gave the following output: `Successfully installed sentencepiece-0.1.97`

What I found it works perfectly with the CPU, this problem arises with GPU. 

I ran my code in colab and on the first try it also gave the same error. But then I restarted the runtime, then it worked.

@tianshuailu same thing, restarting the kernel worked. Pretty buggy I would say.

I am also getting the same error message even though I have sentencepiece installed

Friendly ping to @ArthurZucker 

Hey @91jpark19 could you share the settings in which this is happening or a notebook? 
I can't reproduce any of the errors. See this notebook: 
https://colab.research.google.com/drive/1RFdXqNwA-kdyx0S_ALRrGzZMR8dk48Ie?usp=sharing