ManojP09/logfix-smollm2-360m

LogFix V4 — Failure-Driven Dataset

A LoRA adapter for HuggingFaceTB/SmolLM2-360M-Instruct that answers a Python error or traceback in four sections: ERROR TYPE, CAUSE, FIX, PREVENTION. Trained with LogFix Studio.

Model

• Base model: HuggingFaceTB/SmolLM2-360M-Instruct (original weights; this adapter was trained from them, not from an earlier LogFix version)
• Training method: LoRA (PEFT); only the adapter weights were trained
• Version: V4; run run_20260925_010; trained 2026-09-25T19:22:11+00:00
• Trainable parameters: 8683520

Training data

• Dataset: LogFix Dataset V4 (logfix-v4)
• Size: 519 training examples; validation 20; test 20
• Description: Failure-driven dataset: 250 de-duplicated Dataset V2 records plus new hard examples and new examples targeting the weaknesses measured on the frozen evaluation. Validation and test are the shared frozen sets.
• Source: V2 records: synthetic-authored for V2. New records: synthetic-authored for V4 by authors who never saw the frozen set, checked by scripts (schema, format, length, duplicates, leakage); not human-reviewed.
• Composition: diversity 169, foundational 81, failure-targeted 135, hard-example 134
• Training file sha256: 53eb20dde1aab000ba41ecd4161e856ff728775ab9de0d319778ca60451a3047
• The training records themselves are not included in this export unless the exporter selected them.

Training parameters

Setting	Value
epochs	3.0
learning_rate	0.0001
batch_size	4
gradient_accumulation	2
lora_rank	16
lora_alpha	32
lora_dropout	0.05
max_seq_length	512
seed	42

Experiment exp_dataset_v4 (Failure-Driven Dataset Improvement): changed dataset_id = logfix-v4 relative to V3.

Evaluation

Methodology: a frozen 20-example test set that no version trains on, sealed by hashes. Every version is evaluated with the same prompts, greedy decoding (temperature 0), 256 new tokens and batches of 4. Checks are deterministic: whether all four sections are present, whether the ERROR TYPE matches an accepted label, and which required fix concepts the FIX section mentions (substring rules). Test loss is the cross-entropy of the reference answers. These checks measure format and mentions, not whether a fix is correct.

Metric	Value
Format compliance (all four sections)	95.0%
Error type accuracy	80.0%
Required fix concept coverage	27.5%
Field completeness (sections present, of 4)	3.80
Test loss, whole answer	1.9937
Test loss, CAUSE/FIX/PREVENTION	1.9997
Mean generation time (CPU, batched)	7.82 s

From evaluation eval_20260925_192222 on 20 frozen test examples.

Intended use

Explaining common Python errors to learners and developers as a starting point for debugging, with a human checking the suggested fix before applying it.

Limitations

• SmolLM2-360M-Instruct is a small model. Complex debugging tasks (long or multi-cause tracebacks, framework internals, version-specific behaviour) may exceed its capabilities, and it can state wrong causes or fixes confidently.
• The evaluation set has only 20 examples, so one example moves a rate by 5 percentage points.
• Training data is synthetic (written for this project and checked by scripts), not collected from real users.
• Not production-ready: training and evaluation completing does not establish correctness or safety of fixes.
• Responses can repeat themselves until the token limit on some inputs.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct")
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct"), "ManojP09/logfix-smollm2-360m")