mistralai

/

Mistral-Nemo-Instruct-2407

like 426

mistral-demo ./nemo/
Traceback (most recent call last):
  File "/home/cloudyu/.local/bin/mistral-demo", line 8, in <module>
    sys.exit(mistral_demo())
  File "/home/cloudyu/.local/lib/python3.10/site-packages/mistral_inference/main.py", line 207, in mistral_demo
    fire.Fire(demo)
  File "/home/cloudyu/.local/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/cloudyu/.local/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/cloudyu/.local/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/cloudyu/.local/lib/python3.10/site-packages/mistral_inference/main.py", line 191, in demo
    generated_words.append(tokenizer.decode(encoded_prompts[i] + x))
  File "/home/cloudyu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py", line 224, in decode
    return "".join(self._decode_all(tokens, special_token_policy=self._special_token_policy))
  File "/home/cloudyu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py", line 203, in _decode_all
    raise ValueError(f"Special tokens not allowed in this context: {list(group)}")
ValueError: Special tokens not allowed in this context: [1]

mistral_common 1.3.1
mistral_inference 1.3.0

I spent whole day to install packages and test new models
both mamba-codestral-7B-v0.1 and Mistral-Nemo-Instruct-2407 don't work.

https://huggingface.co/mistralai/mamba-codestral-7B-v0.1/discussions/4#6699047cd77a8b28703e029c

I'm having the same issue with special tokens even running the function calling with texts in Portuguese example in the README:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[16], line 33
     30 tokens = tokenizer.encode_chat_completion(completion_request).tokens
     32 out_tokens, _ = generate([tokens], model, max_tokens=256, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
---> 33 result = tokenizer.decode(out_tokens[0])
     35 print(result)

File /usr/local/lib/python3.10/dist-packages/mistral_common/tokens/tokenizers/mistral.py:148, in MistralTokenizer.decode(self, tokens)
    147 def decode(self, tokens: List[int]) -> str:
--> 148     return self.instruct_tokenizer.decode(tokens)

File /usr/local/lib/python3.10/dist-packages/mistral_common/tokens/tokenizers/sentencepiece.py:199, in InstructTokenizerBase.decode(self, tokens)
    198 def decode(self, tokens: List[int]) -> str:
--> 199     return self.tokenizer.decode(tokens)

File /usr/local/lib/python3.10/dist-packages/mistral_common/tokens/tokenizers/tekken.py:224, in Tekkenizer.decode(self, tokens)
    223 def decode(self, tokens: List[int]) -> str:
--> 224     return "".join(self._decode_all(tokens, special_token_policy=self._special_token_policy))

File /usr/local/lib/python3.10/dist-packages/mistral_common/tokens/tokenizers/tekken.py:203, in Tekkenizer._decode_all(self, tokens, special_token_policy)
    201 if is_special:
    202     if special_token_policy == SpecialTokenPolicy.RAISE:
--> 203         raise ValueError(f"Special tokens not allowed in this context: {list(group)}")
    204     elif special_token_policy == SpecialTokenPolicy.KEEP:
    205         decoded.extend(self._all_special_tokens[t] for t in group)

ValueError: Special tokens not allowed in this context: [9]