ImportError: tokenizers>=0.11.1,!=0.11.3,<0.14 is required for a normal functioning of this module, but found tokenizers==0.10.3. Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main
Author: Justinfungi
Pull request: no

### System Info

I git the newest version

OS ubuntu system

python3

```
ImportError: tokenizers>=0.11.1,!=0.11.3,<0.14 is required for a normal functioning of this module, but found tokenizers==0.10.3.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main

```

### Who can help?

@SaulLu

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Traceback (most recent call last):
  File "AI_clone.py", line 11, in <module>
    from transformers import AutoModelForCausalLM, AutoTokenizer
  File "/home/fish/anaconda3/envs/comp3340/lib/python3.7/site-packages/transformers/__init__.py", line 30, in <module>
    from . import dependency_versions_check
  File "/home/fish/anaconda3/envs/comp3340/lib/python3.7/site-packages/transformers/dependency_versions_check.py", line 41, in <module>
    require_version_core(deps[pkg])
  File "/home/fish/anaconda3/envs/comp3340/lib/python3.7/site-packages/transformers/utils/versions.py", line 123, in require_version_core
    return require_version(requirement, hint)
  File "/home/fish/anaconda3/envs/comp3340/lib/python3.7/site-packages/transformers/utils/versions.py", line 117, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/home/fish/anaconda3/envs/comp3340/lib/python3.7/site-packages/transformers/utils/versions.py", line 51, in _compare_versions
    f"{requirement} is required for a normal functioning of this module, but found {pkg}=={got_ver}.{hint}"
ImportError: tokenizers>=0.11.1,!=0.11.3,<0.14 is required for a normal functioning of this module, but found tokenizers==0.10.3.

Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main

### Expected behavior

How to use the Transformer lib without this error

Comments:
Hi @Justinfungi,

Could you share the command you run that generates this error? Possibly the associated python script?

The error suggests that you have a version problem in your python environment. How did you install transformers? Did you try to run the recommended command "Try: `pip install transformers -U` or `pip install -e '.[dev]'` if you're working with git main" ? 

```
import customtkinter as ctk
import os
import torch
#import torchaudio
from transformers import AutoModel, AutoTokenizer
import tortoise
#from tortoise.utils.audio import load_voice
#import vlc
#from tkVideoPlayer import TkinterVideo

import tkinter as tk
from tkinter import ttk

# ✅ Works
app = tk.Tk()
app.geometry("700x700")
app.title("Justris")
ctk.set_appearance_mode("dark")

```

This is my python script.
i use conda forge to install the transformer
I have run ``` pip install transformers -U ```. It don't generate any error

Thanks for these details. :hugs: 

Given your error, I think that what should solve your problem is to install a version of tokenizer compatible as indicated in the error message., e.g. `pip install tokenizers==0.13.2`. Let me know if this works

[Voice - Jupyter Notebook.pdf](https://github.com/huggingface/transformers/files/10089433/Voice.-.Jupyter.Notebook.pdf)

i dont know why this error still exist

Couldn't the problem be that the package is not installed in your virtual environment since you are running the start from your jupyter notebook? 

Cf [this thread](https://stackoverflow.com/questions/38368318/installing-a-pip-package-from-within-a-jupyter-notebook-not-working)

> Couldn't the problem be that the package is not installed in your virtual environment since you are running the start from your jupyter notebook? 
> 
> Cf [this thread](https://stackoverflow.com/questions/38368318/installing-a-pip-package-from-within-a-jupyter-notebook-not-working)

i think i have pip install both in jupyter and terminal. But it doesnt work. it is so werid

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

that's works
 for me pip install tokenizers==0.13.2