license: apache-2.0
This is a replicate of https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca
But in safetensor format
Prompt Template
To use the prompt for further training and inference, please use OpenAI's Chat Markup Language (ChatML) format, with <|im_start|> and <|im_end|> tokens added to support this.
This means that, e.g., in oobabooga the "MPT-Chat" instruction template should work, as it also uses ChatML.
This formatting is also available via a pre-defined Transformers chat template,
which means that lists of messages can be formatted for you with the apply_chat_template() method:
chat = [
{"role": "system", "content": "You are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!"}
{"role": "user", "content": "How are you?"},
{"role": "assistant", "content": "I am doing well!"},
{"role": "user", "content": "Please tell me about how mistral winds have attracted super-orcas."},
]
tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
which will yield:
<|im_start|>system
You are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!
<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
I am doing well!<|im_end|>
<|im_start|>user
Please tell me about how mistral winds have attracted super-orcas.<|im_end|>
<|im_start|>assistant
If you use tokenize=True and return_tensors="pt" instead, then you will get a tokenized
and formatted conversation ready to pass to model.generate().
Inference
See this notebook for inference details.
Note that you need the development snapshot of Transformers currently, as support for Mistral hasn't been released into PyPI yet:
pip install git+https://github.com/huggingface/transformers