cmarkea
/

Mixtral-8x7B-Instruct-v0.1-4bit

Text Generation Transformers Safetensors 5 languages mixtral Mixture of Experts conversational Inference Endpoints text-generation-inference 4-bit precision

Model card Files Files and versions Community

Edit model card

Converted version of Mixtral Instruct to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.

Prompt Pattern

Here is a reminder of the command pattern to interact with the model if the add_special_tokens option is disabled (otherwise, do not include the BOS symbol and space at the beginning of the sequence):

<s> [INST] {user_prompt_1} [/INST] model_answer_1</s> [INST] {user_prompt_2} [/INST] model_answer_2</s>

Downloads last month: 51

Safetensors

Model size

24.2B params

Tensor type

F32

BF16