delimitter/qwen25-coder-7b-synoema-iot

Qwen2.5-Coder-7B — Synoema IoT Fine-tune

LoRA adapter on Qwen2.5-Coder-7B-Instruct trained to generate correct Synoema IoT automation rules.

What is Synoema?

Synoema is an LLM-native programming language designed for IoT/edge automation:

• 33 BPE-aligned operators (cl100k_base) — no tokenizer misalignment
• GBNF grammar for constrained decoding (100% syntactic correctness)
• Cranelift JIT + WebAssembly targets
• Contract annotations (requires/ensures) for formal verification

Model Details

Property	Value
Base model	unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit
Adapter type	LoRA (QLoRA 4-bit)
LoRA rank	r=8, alpha=32
Training epochs	3
Train examples	5,479
Eval suite	7-task IoT sandbox (T1–T7)
Best score	5/7 (Cycle 22, 2026-05-03)
Training hardware	AMD RX 7900 GRE (ROCm, unsloth)

Eval Results — IoT 7-Task Suite

Task	Status	Description
T1-bearing-protection	✅ PASS	API 670 bearing temperature/vibration protection relay
T2-irrigation-interlock	✅ PASS	Soil moisture + rain sensor irrigation interlock
T3-async-sensor-poll	✅ PASS	Async GPIO sensor polling with cancellation token
T4-bearing-anomaly	❌ FAIL	Bearing anomaly diagnosis (lubrication failure pattern)
T5-hvac-setback	❌ FAIL	HVAC occupancy setback (BACnet/Modbus control)
T6-vitals-alert	✅ PASS	Patient vitals alert (HR/SpO2 thresholds)
T7-co2-anomaly	✅ PASS	CO2 anomaly detection (NDIR sensor, ventilation logic)

Score: 5/7 (71.4%) — Training ongoing, target 7/7.

Continuous Training

This model is trained in a continuous improvement loop:

• 22+ cycles completed (2026-04-18 → 2026-05-03)
• Each cycle: corpus expansion → QLoRA fine-tune → IoT eval → targeted corpus generation
• Corpus: 5,479 training examples across 15 domains

Corpus Composition

Category	Examples
Anchor format / doc annotations	329
Async IoT patterns	104
Contract codegen	48
Doc interrogation (MCP)	500
IoT aggregation	48
IoT verticals (agri/building/industrial/medical)	112
IoT hypothesis patterns	502
Multi-step chains	500
Multi-step training	3,242
Pkg interrogation	200
Syntax drills/fixes	160
Targeted fixes (T2–T5)	320+

Usage

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained(
    "synoema/qwen25-coder-7b-synoema-iot",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
    "synoema/qwen25-coder-7b-synoema-iot"
)

prompt = """Generate a Synoema IoT rule that monitors bearing temperature.
Alert if temperature > 85°C, trip if > 95°C."""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Training Setup

# Training command (AMD ROCm, unsloth)
TORCHDYNAMO_DISABLE=1 python3 train_mcp_finetune.py \
  --model unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit \
  --output output/ci-c22-qwen25-7b \
  --train corpus/mcp_train.jsonl \
  --epochs 3 \
  --batch-size 1 \
  --grad-accum 16 \
  --max-seq-len 2048 \
  --lora-r 8

Note: TORCHDYNAMO_DISABLE=1 required to work around unsloth_zoo CE loss decorator on AMD ROCm (TorchDynamo shape mismatch with s97 vs s7 tensors in cross_entropy).

Known Issues

• T4 / T5 still failing — targeted corpus expansion in progress
• TORCHDYNAMO_DISABLE required on AMD ROCm — unsloth_zoo CE loss applies @torch._dynamo.optimize() at import time; must monkey-patch or disable globally

Links

• Synoema language
• GitHub
• Continuous improvement loop