--- language: - it license: cc-by-nc-4.0 tags: - sft - it - mistral - chatml - axolotl prompt_template: <|im_start|>system {system_message}<|im_end|> <|im_start|>user {prompt}<|im_end|> <|im_start|>assistant model-index: - name: maestrale-chat-v0.3-beta results: [] ---
Mii-LLM

Want to contribute? Please donate! This will let us work on better datasets and models!


# Maestrale chat beta ༄ By @efederici and @mferraretto ## Model description - **Language Model**: Mistral-7b for the Italian language, continued pre-training for Italian on a curated large-scale high-quality corpus. - **Fine-Tuning**: SFT performed on convs/instructions for three epochs. **v0.3** - Function calling - Reduced default system prompt to avoid wasting tokens (pre-alignment) This model uses ChatML prompt format: ``` <|im_start|>system Sei un assistente utile.<|im_end|> <|im_start|>user {prompt}<|im_end|> <|im_start|>assistant ``` ## Usage: ```python from transformers import ( AutoTokenizer, AutoModelForCausalLM, GenerationConfig, TextStreamer ) import torch tokenizer = AutoTokenizer.from_pretrained("mii-llm/maestrale-chat-v0.3-beta") model = AutoModelForCausalLM.from_pretrained("mii-llm/maestrale-chat-v0.3-beta", load_in_8bit=True, device_map="auto") gen = GenerationConfig( do_sample=True, temperature=0.7, repetition_penalty=1.2, top_k=50, top_p=0.95, max_new_tokens=500, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.convert_tokens_to_ids("<|im_end|>") ) messages = [ {"role": "system", "content": "Sei un assistente utile."}, {"role": "user", "content": "{prompt}"} ] with torch.no_grad(), torch.backends.cuda.sdp_kernel( enable_flash=True, enable_math=False, enable_mem_efficient=False ): temp = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(temp, return_tensors="pt").to("cuda") streamer = TextStreamer(tokenizer, skip_prompt=True) _ = model.generate( **inputs, streamer=streamer, generation_config=gen ) ``` ## Intended uses & limitations It's a beta sft version, but it's not `aligned`. It's a first test. We are working on alignment data and evals. [Built with Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl)