adarrshDev/gemma-2-9b-it-plc-sft-r16

Gemma-2 9B Instruct — IEC 61131-3 Structured Text SFT (LoRA r=16)

A QLoRA fine-tuned adapter for unsloth/gemma-2-9b-it-bnb-4bit specialised in generating and reasoning about IEC 61131-3 Structured Text (ST) programs for industrial PLC applications.

Note: This is a Gemma-2 9B adapter. It is not compatible with Ornith-1.5-9B (Qwen3.5 family, different architecture). Load it only on a Gemma-2 9B base.

Model Details

Property	Value
Base model	unsloth/gemma-2-9b-it-bnb-4bit
Architecture	Gemma2ForCausalLM
PEFT method	QLoRA (4-bit base + bf16 adapters)
Rank (r)	16
Alpha (α)	32
Dropout	0.0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable params	54,018,048 / 9,295,724,032 (0.58%)
Training duration	124.3 s on Tesla T4
Epochs	3
Effective batch	8 (2 per device × 4 gradient accumulation)
Learning rate	2e-4 (cosine decay)
Sequence length	2048
Framework	Unsloth 2026.8.22 + TRL + HuggingFace Transformers

Training Loss

Step	Epoch	Loss
1/9	0.4	21.30
3/9	1.0	19.72
6/9	2.0	18.53
8/9	2.8	16.22
9/9	3.0	16.52

Average train loss: 18.58

How to Load

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "unsloth/gemma-2-9b-it-bnb-4bit"
adapter_id = "adarrshDev/gemma-2-9b-it-plc-sft-r16"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

Limitations

• 20 training samples — proof-of-concept only, not production-ready
• No held-out eval run yet against the base model
• 2048 token context limit during training
• Inherits Gemma license