Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

A very capable chat model built on top of the new Mistral MoE model, trained on the SlimOrca dataset for 1 epoch, using QLoRA.


import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mattshumer/mistral-8x7b-chat", low_cpu_mem_usage=True, device_map="auto", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("mattshumer/mistral-8x7b-chat")
x = tok.encode(PROMPT_GOES_HERE, return_tensors="pt").cuda()
x = model.generate(x, max_new_tokens=512).cpu()

Prompt Template:

You are an AI assistant.<|im_end|>
Hi, how are you?<|im_end|>
I'm doing well, thanks for asking!<|im_end|>
Write me a poem about AI.<|im_end|>

Trained w/ Axolotl on 6x H100s for nine hours.

Downloads last month
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Spaces using mattshumer/mistral-8x7b-chat 7