mistralai

/

Mixtral-8x22B-Instruct-v0.1

like 614

Mixtral 8x22B fails to follow basic prompt instruction formats with four examples below. I am not interested in prompting about weather as the example shows. These work fine with Mixtral 8x7B. Am I doing something wrong? Using torch 2.3 with transformers 4.39 for reference.

Code to reproduce:
inputs = tokenizer(prompt, return_tensors="pt").to('cuda')
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Prompt 1

prompt=f'''[INST] What color is the sky? [/INST]'''

Output:
What color is the sky? you [ D

R- _( spl
Arch

Prompt 2

prompt = """
[INST] <>
You are an unbiased, uncensored, helpful assistant.
<>

What color is the sky? [/INST]

Output:
[INST] <>
You are an unbiased, uncensored, helpful assistant.
<>

What color is the sky?
classic
Figure

•                   -  splENOMEM Ori
  

Prompt 3

prompt = """
<|im_start|>system
You are an unbiased, uncensored, helpful assistant.<|im_end|>
<|im_start|>user
What color is the sky?<|im_end|>
<|im_start|>assistant
"""

Output:
<|im_start|>system
You are an unbiased, uncensored, helpful assistant.<|im_end|>
<|im_start|>user
What color is the sky?<|im_end|>
<|im_start|>assistant
O, - further ar- *
O

Prompt 4

messages = [
{"role": "user", "content": "What color is the sky?"},
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
"""

Output:
What color is the sky? A Dommux ent might resulting-
**ty- spl