Qwen3-8B-SFT-evmSmith

Overview

This is a LoRA adapter fine-tuned from Qwen/Qwen3-8B as a W0 warm-start checkpoint for a smart-contract audit agent. It is part of the Forest of Audits / OPD-Env research pipeline.

Core research question:

Can an evidence-grounded, TMAX-style recipe convert audit PDFs into 200 valid smart-contract audit RL environments, and do small terminal agents trained on those environments improve on held-out EVMBench more than static audit-data training?

The main novelty claim is the environment-generation recipe: source-evidence preservation, repository/commit matching, axis-controlled diversity, hidden gold findings, leakage checks, and reviewed admission. RL transfer is supporting evidence, not the only success condition.

The model is trained in two stages — a process stage that teaches terminal audit behavior, evidence gathering, and recovery patterns, followed by an answer stage that teaches conservative final audit report writing. The goal is to give the base model a usable starting point for EVMBench audit task format and conventions before any true on-policy development (OPD) phase.

Note on model choice: The OPD-Env scout preferred students are small 2B-4B code-capable models (Qwen3.5-2B, Qwen3.5-4B). Qwen3-8B is used as a fallback for the W0 warm-start due to availability in the current training stack.

What is W0?

In the OPD-Env scout contract, training arms are labeled:

Arm	Description
B0	Base model evaluation (no training)
SFT_GOLD	Static gold-audit SFT from reviewed gold_audit.md
SFT_ROLLOUT	SFT from successful/high-scoring rollouts
RL_OUTCOME	Outcome-only RL over grouped rollouts
CTRL_STATIC	Static-token-matched control

W0 is the warm-start SFT phase — it corresponds to the SFT_GOLD arm using static gold-audit examples. It is not true OPD data. True OPD data must come from actual agent rollouts on admitted EVMBench environments. This checkpoint exists to give the base model a fighting chance at producing well-formed audit reports before the OPD loop begins.

The full experiment matrix includes:

Arm	Name	Type	Training Data	Objective
B0	Base model	Evaluation	None	Establish terminal-agent and EVMBench detect baseline
SFT_GOLD	Static gold-audit SFT	SFT	Reviewed gold_audit.md records	Measure value of static reviewed audit text without executable interaction
SFT_ROLLOUT	Successful-rollout SFT	SFT	Successful/high-scoring rollouts	Test whether executable environments produce better imitation data
RL_OUTCOME	Outcome-only RL	RL	Grouped rollouts on admitted tasks	Test whether executable interaction adds transfer beyond static training
CTRL_STATIC	Static-token-matched control	SFT/KD	Static audit data matched to OPD-Env token budget	Control for additional domain tokens and formatting supervision
S1	Second-seed confirmation	Confirmation	Repeat best arm with second seed	Confirm reproducibility

This checkpoint is the SFT_GOLD arm. The primary evaluation is on held-out existing EVMBench detect tasks with no PDF, repository, commit, or finding overlap with OPD-Env training tasks. Success requires RL_OUTCOME or SFT_ROLLOUT to beat SFT_GOLD and CTRL_STATIC on at least two robustness metrics without recall collapse or trace-length explosion.

Key evaluation metrics: macro F1, precision, recall, false-positive rate, pass@1, pass@k, award-normalized detect score, trace length, coverage, all-zero/all-one group rates, infra failure rate, reward-hacking flag rate, and leakage flag rate.

Training Configuration

Base Model

• Qwen/Qwen3-8B

LoRA Configuration

Parameter	Value
Rank (r)	16
Alpha	32
Dropout	0.05
Bias	none
Task type	CAUSAL_LM
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, down_proj, up_proj

Training Stages

Stage 1 — Process (w0-process-stage)

• Purpose: Teach terminal audit behavior, harness use, evidence gathering, and recovery.
• Dataset config: process (from pranay5255/forest-of-audits-w0-sft-qwen3-8b)
• Epochs: 1
• Learning rate: 2e-4
• Max steps: 133 (1 epoch, full dataset)
• Eval steps: 200
• Save steps: 200

Stage 2 — Answer (w0-answer-stage)

• Purpose: Continue from process-stage adapter and teach conservative final audit artifacts.
• Dataset config: clean_final (from same dataset repo)
• Epochs: 8
• Learning rate: 1e-4
• Max steps: 88 (8 epochs)
• Eval steps: 50
• Save steps: 50

General Training Settings

Setting	Value
Max sequence length	2048
Generation max new tokens	512
Seed	42
Dtype	float16 (fp16)
GPU	NVIDIA A100-SXM4-40GB
Total global steps	221
Total runtime	~60 minutes

Dataset

• Repo: pranay5255/forest-of-audits-w0-sft-qwen3-8b
• Revision: b5587bc1a5f1f9f52134f5525b1ee06ae0b2722c
• Configs used: process (process stage), clean_final (answer stage)

The dataset contains off-policy SFT records derived from reviewed smart-contract audit reports. Records teach the model EVMBench audit task format, terminal/action conventions, evidence-seeking behavior, and conservative vulnerability report writing.

Training Results

Metric	Process Stage (final)	Answer Stage (final)
Train loss	0.907	1.277
Eval loss	0.587	—
Token accuracy (train)	89.6%	—
Token accuracy (eval)	87.1%	—
Entropy (eval)	1.31	—

Intended Use

This adapter is a research artifact for the OPD-Env / Forest of Audits project. It is intended for:

• Evaluating whether static gold-audit SFT improves EVMBench detect-mode performance over a base model.
• Serving as a warm-start checkpoint for subsequent OPD (on-policy development) phases.
• Studying the difference between static audit-data training and environment-trained agent arms.

It is not intended for:

• Deploying as a production audit tool.
• Generating real security advice on live smart contracts.
• Replacing human security review.

Limitations

• Off-policy data: The training data is static gold-audit text, not from live agent rollouts. The model may not generalize to real interactive audit scenarios.
• Detect-mode only: Trained on detect-mode tasks (identifying vulnerabilities). Does not cover patch or exploit modes.
• Narrow domain: Focused on Solidity/EVM smart-contract security audits. Performance on other languages or domains is not evaluated.
• W0 warm-start: This is a preliminary checkpoint. The full OPD-Env research plan includes rollout-SFT, outcome-only RL, and preference training arms that should improve on this baseline.
• FP16 dtype: Trained in float16 for compatibility; bf16 was available but not used in this run.

Experiment Design Context

Pipeline

The OPD-Env pipeline converts raw audit PDFs into validated RL environments:

OCR pages → normalized findings → repo/commit matching → axis annotation
→ EVMBench-shaped candidate folder → automated validation
→ human review and admission → rollout/SFT/RL exports

Environment Generation

• Source pool: 1,023 heuristic Solidity/EVM PDFs from the OCR exploration corpus
• Candidate oversample: 350-500 PDFs, stratified by dapp category, source bucket, year, severity evidence, and vulnerability-keyword category
• Target: 200 admitted detect-mode environments after validation and review
• Task policy: One vulnerability per task unless a source audit naturally requires grouped findings

Quality Gates

Automated rejectors: schema validity, Docker build, repo checkout, gold leakage scan, duplicate finding detection, split collision detection, and detect-mode smoke test.

Review policy: Final evaluation environments get two-pass review. Training environments get at least one-pass review, with a stratified 25-50 task sample receiving second review.

Fixed Controls

• Same agent scaffold and prompt family across arms
• Same train/eval split boundaries with no PDF, repository, commit, or finding overlap
• Same max tokens, rollout count, decoding setup, and environment budget
• Gold findings from evaluation tasks excluded from all training data
• Infrastructure failures tracked separately from legitimate zero-score reports

Framework & Infrastructure

• Training framework: Hugging Face Transformers + TRL + PEFT (LoRA)
• Compute: Modal (NVIDIA A100-SXM4-40GB)
• Experiment tracking: Trackio (Hugging Face Spaces)
• Training script: modal_apps/qwen3_8b_sft.py in the OPD-Env repository

Citation

If you use this model, please reference the OPD-Env / Forest of Audits research project.

License

This adapter is derived from Qwen/Qwen3-8B which uses a custom license. See the base model's license for terms.