nkuniyil/mistral-7b-sft

Mistral-7B SFT Model

This is a LoRA adapter for Mistral-7B trained using Supervised Fine-Tuning (SFT) on the Alpaca-GPT4 dataset.

Training Details

• Base Model: mistralai/Mistral-7B-v0.1
• Method: LoRA (r=16, alpha=32)
• Dataset: Alpaca-GPT4 (10K examples)
• Epochs: 3
• Learning Rate: 2e-5
• Training Time: ~6 hours on A100

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "nkuniyil/mistral-7b-sft")

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("nkuniyil/mistral-7b-sft")

# Generate
prompt = "### Instruction:\nWrite a haiku about AI.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

Part of SFT vs DPO Comparison Study

This model is part of a study comparing Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO).

See also:

• SFT+DPO model: nkuniyil/mistral-7b-sft-dpo
• Results: nkuniyil/sft-dpo-comparison-results