duyntnet/bagel-8b-v1.0-imatrix-GGUF

Quantizations of https://huggingface.co/jondurbin/bagel-8b-v1.0

From original readme

Prompt formatting

This model uses the llama-3-instruct prompt template, and is provided in the tokenizer config. You can use the apply_chat_template method to accurate format prompts, e.g.:

import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("jondurbin/bagel-8b-v1.0", trust_remote_code=True)
chat = [
  {"role": "system", "content": "You are Bob, a friendly AI assistant."},
  {"role": "user", "content": "Hello, how are you?"},
  {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
  {"role": "user", "content": "I'd like to show off how chat templating works!"},
]
print(tokenizer.apply_chat_template(chat, tokenize=False))