ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported.

#9
by aigeek0x0 - opened

Is this model supported by transformers yet? I am using the latest transformers version built using the main branch of the github repo.

Installed the latest version of transformers again and that solved the problem.

!pip install -q -U git+https://github.com/huggingface/transformers.git

osanseviero changed discussion status to closed

!pip install -q -U git+https://github.com/huggingface/transformers.git after this the problem is same
ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported.

Try this:
pip install "torch>=2.1.1" -U

I did the same [pip install "torch>=2.1.1" -U] but also got the same error!😔

Hey, I have got the same error, any solutions??

Hi, just restart the session again but before that,
remove any pip install transformers command and replace it with below line.
!pip -q install git+https://github.com/huggingface/transformers.git

Install other packages like trl or peft seperately using !pip install trl peft
It started working for me..

The solutions provided here did not work for me. I've done pip install the git repository and did restart the session a few times but still I get ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported..

I did "pip install -U transformers" installed 4.38.2
It worked for me.

Thank you for this. I checked transformers.__version__ and saw something older than 4.38.2 . Jupyter-lab was from a conda installation, not my python venv. So, I installed jupyterlab in the right python venv and then I was able to utilize the right transformers library!

"pip install -U transformers" can resolve this question, and version update form 4.37.1 to 4.38.2.

Updating transformers to 4.38.2 resolves the issue.

Updating transformers should resolve it:

pip install -q -U transformers

Sign up or log in to comment