mistralai

/

Mixtral-8x7B-Instruct-v0.1

like 4.07k

Anyone experiencing the same?

My code

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

text = """写一个关于张三穿越到唐朝的故事，必须深入细节，注意细节！不要有一天，也不要写大概（比如生活非常艰难啥的），现在开始写。"""
inputs = tokenizer(text, return_tensors="pt").to(0)

outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

output

Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
写一个关于张三穿越到唐朝的故事，必须深入细节，注意细节！不要有一天，也不要写大概（比如生活非常艰难啥的），现在开始写。

Nothing generated. And it happens quite often for prompts I have tried. Did I do anything wrong?