Slmkrnz/promptmon-llama-3.2-3b-lora

PromptMon LoRA v2 — Llama 3.2 3B Instruct

A LoRA adapter that teaches Llama 3.2 3B Instruct a short, concrete, goal-driven dialogue style for the PromptMon mobile duel game. Each turn is a single ≤25-word reply with no metaphor stacking and no opening- quote tic — the failure modes the base model showed on persona role-play.

Training recipe

• Base: mlx-community/Llama-3.2-3B-Instruct (bf16)
• Framework: MLX-LM on Apple Silicon
• LoRA: rank 8, scale 20.0, dropout 0.1, on self_attn.{q,k,v,o}_proj
• Trainable params: ~10M (≈0.3% of base)
• Optimizer: AdamW, lr 1e-4, warmup 50, batch 1
• Iters: 250 (best checkpoint at iter 100, val loss 0.576)
• Data: 170 train / 19 eval examples from synthetic persona duels (judge-filtered; Qwen-Plus as generator, Gemini-2.5-Flash as judge)

adapters.safetensors is the pinned best checkpoint (iter 100). Train loss kept dropping after that but val loss plateaued around 0.60 — a typical small-dataset early-stop.

Usage (MLX-LM)

from mlx_lm import load, generate

model, tokenizer = load(
    "mlx-community/Llama-3.2-3B-Instruct",
    adapter_path="./",  # path to this repo checkout
)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello."}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, verbose=True))

For on-device mobile use (fllama / llama.cpp), fuse into the base and convert to GGUF — see the PromptMon repo's ml/training/ scripts.

License

Inherits Llama 3.2 Community License from the base model. This adapter adds no additional restrictions.