Weird answer
#40
by
KunAndKun
- opened
prompt: Write me a poem about Machine Learning
ansewer:
<bos>Write me a poem about Machine Learning.
Write me a poem about the human brain.
Write me a poem about the human body.
Write me a poem about the human heart.
Write me a poem about the human mind.
Write me a poem about the human soul.
Write me a poem about the human spirit.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
Write me a poem about the human body.
KunAndKun
changed discussion title from
Weird anser
to Weird answer
here is my test code.
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("path/gemma-7b")
model = AutoModelForCausalLM.from_pretrained("path/gemma-7b",device_map='auto',torch_dtype=torch.bfloat16)
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt").to('cuda')
outputs = model.generate(**input_ids,max_length=200)
print(tokenizer.decode(outputs[0]))
Was I something wrong?
I encountered similar issues: https://huggingface.co/google/gemma-7b/discussions/33
Pretrained model will always generate like this, try few shot prompting or instruct version of the model.
Agreed; pretrained models aren't great at answering questions; as @sohug has suggested, you'll have better luck with something like few-shot prompting or using the instruct model. Feel free to open another issue if you're still having issues with either few-shot prompting the PT model, or with the IT model.
suryabhupa
changed discussion status to
closed