Edit model card
Mii-LLM

Maestrale chat beta ༄

By @efederici and @mferraretto

Model description

  • Language Model: Mistral-7b for the Italian language, continued pre-training for Italian on a curated large-scale high-quality corpus, merged with occiglot.
  • Fine-Tuning: SFT performed on 1.7M convs/instructions for 2 epochs.
  • DPO: Aligned with DPO on multiple datasets.

v0.4

  • Agent
  • Improved truthfullness
  • Improved Math & Reasoning capabilities
  • Mermaid mindmaps
  • More latin translations, poems, ...

This model uses ChatML prompt format:

<|im_start|>system
Sei un assistente utile.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Scores

Tasks Version Filter n-shot Metric Value Stderr
hellaswag_it 1 none 0 acc 0.5270 ± 0.0052
none 0 acc_norm 0.7037 ± 0.0048
arc_it 1 none 0 acc 0.1771 ± 0.0112
none 0 acc_norm 0.5218 ± 0.0146
m_mmlu_it 0 none 5 acc 0.5623 ± 0.0043

Usage:

from transformers import (
    AutoTokenizer, 
    AutoModelForCausalLM, 
    GenerationConfig,
    TextStreamer
)
import torch

tokenizer = AutoTokenizer.from_pretrained("mii-llm/maestrale-chat-v0.4-beta")
model = AutoModelForCausalLM.from_pretrained("mii-llm/maestrale-chat-v0.4-beta", load_in_8bit=True, device_map="auto")

gen = GenerationConfig(
    do_sample=True,
    temperature=0.7,
    repetition_penalty=1.2,
    top_k=50,
    top_p=0.95,
    max_new_tokens=500,
    pad_token_id=tokenizer.eos_token_id,
    eos_token_id=tokenizer.convert_tokens_to_ids("<|im_end|>")
)

streamer = TextStreamer(tokenizer, skip_prompt=True)

messages = [
    {"role": "system", "content": "Sei un assistente utile."},
    {"role": "user", "content": "{prompt}"}
]

with torch.no_grad():
    temp = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(temp, return_tensors="pt").to("cuda")

    _ = model.generate(
        **inputs,
        streamer=streamer,
        generation_config=gen
    )

Examples

Mindmaps

messages = [
  {"role": "system", "content": "Fornisci una mindmap Mermaid sull'argomento in input."},
  {"role": "user", "content": "Argomento: [argomento]"}
]

SQL

schema = "[db schema]"
messages = [
  {"role": "system", "content": f"Sei un assistente SQL e il tuo compito è convertire la domanda dell'utente in codice SQL valido rispetto allo schema del database fornito.\n\nSchema:\n```sql\n{schema}\n```"},
  {"role": "user", "content": "Conta il numero di X prodotti dall'azienda Y"}
]

Article from index

messages = [
  {"role": "system", "content": "Sei un assistente utile."},
  {"role": "user", "content": (
    "Scrivi un articolo a partire dal titolo e dall'indice dei contenuti.\n\n"
    "Titolo: [titolo]\n\n"
    "Indice:\n\n"
    "1. Introduzione\n"
    "2. [heading]\n"
    "..."
  )}
]

Intended uses & limitations

It's a beta version; it's quite safe, and it can refuse to answer to toxic questions.

Built with Axolotl

Downloads last month
1,822
Safetensors
Model size
7.24B params
Tensor type
BF16
·
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Spaces using mii-llm/maestrale-chat-v0.4-beta 2