Text Generation
Transformers
Safetensors
GGUF
mistral
text-generation-inference
Inference Endpoints
emnakamura's picture
Update README.md
2870c71 verified
metadata
library_name: transformers
license: apache-2.0
base_model:
  - flammenai/flammen27-mistral-7B
datasets:
  - flammenai/FlameMix-DPO-v1
  - flammenai/Grill-preprod-v1_chatML
  - flammenai/Grill-preprod-v2_chatML

image/png

Mahou-1.2a-mistral-7B

Mahou is our attempt to build a production-ready conversational/roleplay LLM.

Future versions will be released iteratively and finetuned from flammen.ai conversational data.

1.2a is rebased and retrained to improve comphresion and coherence.

Chat Format

This model has been trained to use ChatML format.

<|im_start|>system
{{system}}<|im_end|>
<|im_start|>{{char}}
{{message}}<|im_end|>
<|im_start|>{{user}}
{{message}}<|im_end|>

Roleplay Format

  • Speech without quotes.
  • Actions in *asterisks*
*leans against wall cooly* so like, i just casted a super strong spell at magician academy today, not gonna lie, felt badass.

ST Settings

  1. Use ChatML for the Context Template.
  2. Turn on Instruct Mode for ChatML.
  3. Use the following stopping strings: ["<", "|", "<|", "\n"]

Method

Finetuned using an A100 on Google Colab.

Fine-tune a Mistral-7b model with Direct Preference Optimization - Maxime Labonne

Configuration

LoRA, model, and training settings:

# LoRA configuration
peft_config = LoraConfig(
    r=16,
    lora_alpha=16,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj']
)

# Model to fine-tune
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    load_in_4bit=True
)
model.config.use_cache = False

# Reference model
ref_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    load_in_4bit=True
)

# Training arguments
training_args = TrainingArguments(
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    gradient_checkpointing=True,
    learning_rate=5e-5,
    lr_scheduler_type="cosine",
    max_steps=2000,
    save_strategy="no",
    logging_steps=1,
    output_dir=new_model,
    optim="paged_adamw_32bit",
    warmup_steps=100,
    bf16=True,
    report_to="wandb",
)

# Create DPO trainer
dpo_trainer = DPOTrainer(
    model,
    ref_model,
    args=training_args,
    train_dataset=dataset,
    tokenizer=tokenizer,
    peft_config=peft_config,
    beta=0.1,
    force_use_ref_model=True
)

# Fine-tune model with DPO
dpo_trainer.train()