netcfg-8b-lora

A LoRA adapter for Qwen3-8B that converts raw network device configuration text into a normalized JSON object, across 17 vendor platforms.

Trained entirely on a single laptop (AMD Ryzen AI Max+ 395 / Strix Halo) with no cloud compute and no proprietary data.

Turkce ozet: Ag cihazi konfigurasyonlarini normalize JSON'a ceviren, Qwen3-8B uzerine egitilmis bir LoRA adaptoru. 17 farkli uretici platformu kapsiyor. Tamamen tek bir dizustu bilgisayarda, bulut kullanmadan egitildi.

What it does

Input: a device configuration (Cisco IOS/IOS-XR/NX-OS, Arista EOS, Nokia SR Linux/SR OS, MikroTik RouterOS 7, FortiOS, FRR, Aruba CX, Dell OS10, SONiC, Arcos, ...).

Output: a single JSON object describing hostname, enabled modules, interfaces (name, address, type, OSPF/BGP attachment), and routing protocol state.

Results

Held-out set: 236 configurations from topologies never seen in training (the split is by topology seed, not by sample, so no config from a training topology appears in evaluation).

Both rows use the identical prompt, the identical Q4_K_M quantization, the identical serving stack (LM Studio), and temperature=0. The only variable is the adapter.

Metric	Qwen3-8B baseline	+ this adapter	delta
Field F1	0.5752	0.9219	+0.3467
JSON validity	0.9492	1.0000	+0.0508
Exact match	0.0000	0.1992	+0.1992
Hallucination rate	0.0104	0.0073	-0.0031
Parseable outputs	224 / 236	236 / 236	+12

Precision 0.8975, recall 0.9610.

Without the schema in the prompt, the base model scores F1 0.0227 - it cannot guess the target shape. The 0.5752 baseline above is the fair one: the base model is given the full schema as a system prompt, the same one the adapter was trained against.

General capability check

Fine-tuning a model on a narrow task often destroys its general ability. Measured on 200 MMLU questions across 8 subjects:

	Baseline	+ adapter
MMLU (200 q)	0.630	0.640

The difference is 2 questions out of 200 (z = 0.21). The correct reading is no measurable regression, not an improvement. 35% of the training mix was general instruction data (FineTome-100k) specifically to protect against this.

Pre-registered thresholds

Thresholds were fixed in writing before training started:

Threshold	Target	Result
Field F1	>= 0.85	0.9219	pass
MMLU	>= 0.60	0.640	pass
Hallucination	<= 0.005	0.0073	missed

Hallucination improved but did not reach the target. Reported as measured.

Known limitations

It fills in plausible defaults. The clearest failure mode: on an SR Linux config with no MTU statement, the model emitted "mtu": 1500 on every interface. 1500 is the correct default for Ethernet - it is simply not in the source text. This is the main source of the remaining precision gap.

It occasionally drops a module. On one EOS config the expected module list was ["bfd", "bgp", "mpls"] and the model produced ["bfd", "bgp"], missing MPLS, while getting every BGP neighbor, AS number, and community list exactly right.

Synthetic training data. All configurations were generated with netlab - real, vendor-valid syntax produced by netlab's own templates, but not drawn from production networks. Real-world configs carry comments, dead blocks, local conventions, and vendor quirks that this set does not represent. Expect degradation on production text.

Coverage is uneven. RouterOS 7 (400 samples) and EOS (367) are well represented; SONiC (3) is not.

Training

Base	Qwen/Qwen3-8B
Method	LoRA, r=32, alpha=64, dropout=0.05
Trainable	87,293,952 params (1.05%)
Data	2,541 task samples + 889 general (FineTome-100k, 35%)
Epochs	2 (422 optimizer steps)
LR	2e-4, cosine, 3% warmup
Batch	1 x 16 gradient accumulation
Max length	4096
Precision	bf16
Best eval loss	0.0126

Hardware

Trained on an ASUS ProArt PX13 - AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151), Radeon 8060S, 128 GB unified memory - under WSL2 / Ubuntu 24.04 with ROCm 7.2.1, librocdxg 1.2.0, and torch 2.9.1+rocm7.2.1.

About 72 s per optimizer step, 7.5 hours total.

A note for anyone attempting this: AMD's own ROCm documentation states "No ML training support" for Ryzen APUs under native Windows, and native-Windows training on gfx1151 did in fact fail here - every GPU-to-CPU scalar read returned a stale host buffer, producing a silently frozen loss and a zero-valued saved adapter. WSL2 works. An 11-test data-path integrity check (scalar reads, boolean masking, masked loss vs CPU reference, forward, backward, optimizer step, weight readback) passes under WSL2 and fails under native Windows. Run that check before trusting any training run on this hardware.

Usage

For llama.cpp / LM Studio / Ollama, use the pre-quantized GGUF build: huseyinturkdogan/netcfg-8b-GGUF

Append /no_think to the user message to suppress Qwen3's reasoning mode. All numbers above were measured with it.

Dataset and evaluation code

huseyinturkdogan/netcfg-config-json