rigidhat/llama-3.2-3b-construction-codecite-v2

Llama 3.2 3B — Construction Code-Citation v2 (AutoScientist)

LoRA adapter on top of Llama 3.2 3B-Instruct, trained by AutoScientist from Adaption Labs on the construction-code-corpus-v1 dataset. Predicts OIICS hazard codes (event, source, nature, body) and OSHA 29 CFR 1926 citations from construction-site incident narratives.

Built for the Adaption Labs AutoScientist Challenge, "All Other Domains" category. Credit to Adaptive Data by Adaption.

Story: Base Llama 3.2 3B vs AutoScientist-adapted

AutoScientist reported a 77% win rate for the adapted model vs the base Llama 3.2 3B on the held-out task test set. Data adaption alone lifted quality 6.0 → 9.1 (grade C → A, +51.7% relative). The training recipe (rank/alpha/schedule/mixture) was chosen by AutoScientist end-to-end — not hand-tuned.

Inputs / Outputs

Input: free-text construction-site incident narrative. Output: strict JSON with hazards[] (4 OIICS codes + severity) and citations[] (verified OSHA 1926 standards).

{
  "hazards": [
    {
      "code_event":  {"id": "21",   "title": "Slip or fall without fall to lower level"},
      "code_source": {"id": "5510", "title": "Ice, snow"},
      "code_nature": {"id": "220",  "title": "Fractures"},
      "code_body":   {"id": "280",  "title": "Trunk"},
      "severity": "high"
    }
  ],
  "citations": [
    {"standard": "1926.501", "section_heading": "Duty to have fall protection", "verified": true}
  ]
}

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-3.2-3B-Instruct"
adapter = "rigidhat/llama-3.2-3b-construction-codecite-v2"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter)

For the full RAG-augmented prompt template + citation verifier, see gradio_app/app.py in the source repo.

Training recipe (from AutoScientist)

Field	Value
Base model	meta-llama/Llama-3.2-3B-Instruct
Training method	SFT (LoRA)
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.0
Target modules	all-linear (gate_proj, k_proj, up_proj, down_proj, q_proj, o_proj, v_proj)
Optimizer	AdamW (cosine LR, warmup 3%)
Peak LR	2.0e-4
Weight decay	0.01
Grad clip	1.0
Epochs	3
Steps	66
Train loss	1.10 → 0.84
Validation loss	1.00 → 0.90

Data

Trained on rigidhat/construction-code-corpus-v1 — 18,122 SFT records built from the OSHA Severe Injury Reports corpus (2015–2025), stratified 70/15/15 by NAICS subsector, with test split hash-pinned SHA-256 c9490ed3... and never opened during development.

Adaption applied: Reasoning Traces, Hallucination Mitigation, Prompt Rephrase, Prompt Deduplication via AutoScientist's data-adaption recipe.

Demo

Live Gradio Space: rigidhat/construction-code-cite

Baseline for comparison

Same data, smaller manual QLoRA baseline: rigidhat/qwen-2.5-construction-codecite-v1 (Qwen 2.5 1.5B). Provided for methodological comparison — the v2 story is that AutoScientist end-to-end produced a stronger model than hand-tuned QLoRA.

Source

Full pipeline (data pull, verifier, RAG index, eval harness): github.com/snakezilla/construction-code-llm

License

Llama 3.2 Community License applies to the base model. LoRA adapter weights released under MIT.

Citation

@misc{construction-codecite-v2-2026,
  title  = {Llama 3.2 3B - Construction Code-Citation v2 (AutoScientist)},
  author = {Oversite Innovations},
  year   = {2026},
  note   = {Trained by AutoScientist for the Adaption Labs AutoScientist Challenge, All Other Domains category}
}