google

/

gemma-2b

like 738

Hi there! Could you make sure to pip install the latest transformers version please?

pip install -U transformers
This will update the transformers to the latest version and then you can download it

pip install -U transformers
This will update the transformers to the latest version and then you can download it

Nope. Still the same.

I've tried the following ways to update the Transformers:
!pip install -U transformers huggingface_hub -q
!pip install git+https://github.com/huggingface/transformers -q

On both I'm getting this:

ValueError Traceback (most recent call last)
in <cell line: 3>()
1 from transformers import AutoTokenizer, AutoModelForCausalLM
2
----> 3 tokenizer = AutoTokenizer.from_pretrained("google/gemma-2b")

ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported.

Hi there! Could you make sure to pip install the latest transformers version please?

Sorry. Still the same issue. I've tried the following ways to update the Transformers:
!pip install -U transformers huggingface_hub -q
!pip install git+https://github.com/huggingface/transformers -q

Hi there!
I fixed this error by using the demo code of the GPU one. I think the CPU demo code has an error about these three lines:
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(**input_text, return_tensors="pt")
outputs = model.generate(input_ids)
which should be updated to :
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**input_ids)

Hi there!
By running this code:
input_ids = tokenizer(**input_text, return_tensors="pt").to("cuda")

I got this error:
TypeError: GemmaTokenizerFast(name_or_path='google/gemma-2b', vocab_size=256000, model_max_length=1000000000000000019884624838656, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'bos_token': '', 'eos_token': '', 'unk_token': '', 'pad_token': ''}, clean_up_tokenization_spaces=False), added_tokens_decoder={
0: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
1: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
2: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
3: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
} argument after ** must be a mapping, not str

@arslankas Did you restart the session after updating the transformers ? Just restart the session after updating the transformers and it must run fine

@arslankas Did you restart the session after updating the transformers ? Just restart the session after updating the transformers and it must run fine

Oops. Thank you. After restarting the session I ran into another error:

TypeError Traceback (most recent call last)
in <cell line: 10>()
8
9 input_text = "Write me a poem about Machine Learning."
---> 10 input_ids = tokenizer(**input_text, return_tensors="pt")
11
12 outputs = model.generate(input_ids)

TypeError: GemmaTokenizerFast(name_or_path='google/gemma-2b', vocab_size=256000, model_max_length=1000000000000000019884624838656, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'bos_token': '', 'eos_token': '', 'unk_token': '', 'pad_token': ''}, clean_up_tokenization_spaces=False), added_tokens_decoder={
0: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
1: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
2: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
3: AddedToken("", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
} argument after ** must be a mapping, not str

@arslankas
I think you can just try to replace your code:
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(**input_text, return_tensors="pt")
outputs = model.generate(input_ids)
to :
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**input_ids)

It works for me.

@junchenfu yah that's the fix for that issue. You need to pass in the tensor to the model.generate()

@arslankas
I think you can just try to replace your code:
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(**input_text, return_tensors="pt")
outputs = model.generate(input_ids)
to :
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**input_ids)

It works for me.

Thank you it works :) :)

@arslankas
I think you can just try to replace your code:
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(**input_text, return_tensors="pt")
outputs = model.generate(input_ids)
to :
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**input_ids)

It works for me.

Is there a way to increase the response? All I get is just 3,4 words as output

Edit: After increasing max_new_tokens=150 the model keeps loading on....kinda stuck at this point

You might want to use a streamer tout output the text on the fly see here

Hi all! Closing this discussion as the tokenization issue is resolved. Feel free to begin a new discussion if you want to discuss ways to speed up generation.

same issue, but on my windows labtop

ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported.

Hi there, please make sure to upgrade the transformers version to the latest with pip install -U transformers

The official huggingface TGI Docker image at version 1.4.2 (released yesterday) is also hitting this (ghcr.io/huggingface/text-generation-inference:1.4.2)

(I poked the GitHub as well: https://github.com/huggingface/text-generation-inference/issues/1590#issuecomment-1960042694)