Ouchhh/AFLM-v3-10k

AFLM Model v3 (10k) — LoRA Adapter for Gemma 2B (4-bit)

This repository contains a PEFT / LoRA adapter trained on top of unsloth/gemma-2b-bnb-4bit (a 4-bit quantized Gemma 2B base model). It is not a standalone full model checkpoint — you load the base model first, then attach this adapter.

What’s inside

• adapter_model.safetensors — LoRA adapter weights
• adapter_config.json — PEFT configuration
• tokenizer.* + special_tokens_map.json — tokenizer files
• tokenizer_config.json — includes a chat template for <start_of_turn> / <end_of_turn> formatting

Adapter details

• Base model: unsloth/gemma-2b-bnb-4bit
• Task: Causal Language Modeling (CAUSAL_LM)
• LoRA rank (r): 16
• LoRA alpha: 16
• LoRA dropout: 0.0
• Target modules: q_proj, k_proj, v_proj, o_proj, up_proj, gate_proj, down_proj
• Adapter file SHA256: 5f5423552c53c9cd0ace4fe55f33a841f93dbcdf752bfa388fc9692bd889130e

Quickstart (Transformers + PEFT)

Requirements: transformers, peft, accelerate, safetensors

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "unsloth/gemma-2b-bnb-4bit"
adapter_dir = "./"  # this folder

tokenizer = AutoTokenizer.from_pretrained(adapter_dir, use_fast=True)

base = AutoModelForCausalLM.from_pretrained(
    base_id,
    device_map="auto",
    torch_dtype=torch.float16,
)

model = PeftModel.from_pretrained(base, adapter_dir)
model.eval()

messages = [
    {"role": "user", "content": "Write a short paragraph about data-driven art."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Prompting format

This adapter expects a Gemma-style chat formatting using:

• <start_of_turn>user ... <end_of_turn>
• <start_of_turn>model ... <end_of_turn>

A ready-to-use chat_template is included in tokenizer_config.json, so you can rely on:

• tokenizer.apply_chat_template(messages, add_generation_prompt=True)

If you do not use apply_chat_template, your manual prompt should look like:

<start_of_turn>user
YOUR QUESTION HERE<end_of_turn>
<start_of_turn>model

Intended use

• Use for experimentation and internal evaluation of the fine-tuned behavior.
• If you plan to publish/deploy, add:
  • a clear dataset description,
  • safety constraints,
  • quantitative + qualitative evaluation.

Limitations & safety notes

• This is not a medical/clinical tool and must not be used for diagnosis.
• Outputs may be incorrect, incomplete, or biased. Always validate in high-stakes contexts.

Training & evaluation (fill in)

Please document your training dataset(s), filtering steps, and evaluation here.

• Training steps: 10k (as implied by name)
• Train/eval split: (add)
• Evaluation sets: (add)
• Metrics (perplexity / task scores): (add)

Changelog

• 2026-02-02: Added a Gemma-compatible chat_template and a complete README/model card.