rishr/qwen2.5-coder-7b-fairchild-v5

Qwen2.5-Coder-7B — Fairchild v5 (Verilog/RTL LoRA)

A LoRA adapter for Qwen/Qwen2.5-Coder-7B-Instruct specialized for Verilog / SystemVerilog RTL tasks: spec→RTL generation, lint/error repair, and error explanation. Trained as part of the EvalChip/Fairchild project.

This is an adapter only — the base model is not redistributed here. PEFT, vLLM, and transformers load it on top of the base via base_model_name_or_path in adapter_config.json.

Results

Metric	Value
eval_loss	0.2486
token accuracy	0.926

(Held-out 300-example validation split. For reference, the project's run-1 adapter scored eval_loss 0.3117 and an earlier v5 attempt 0.277.)

Training

• Method: bf16 LoRA (PEFT), r=16, α=32, dropout=0.05, targets = q/k/v/o_proj + gate/up/down_proj.
• Hyperparameters: 2 epochs, lr 1e-4 cosine, warmup 0.05, effective batch 16 (8 × 2 grad-accum), max_len 2048, assistant_only_loss, packing=False.
• Hardware: 1× H100 80GB. Note: attn_implementation="eager" was required to avoid a bf16 attention-backward NaN on H100.
• Data (~39k examples): spec2rtl (MG-Verilog + RTL-Coder Resyn27k), lint/semantic repair, and error-explanation, normalized and decontaminated against the held-out VerilogEval + RTLLM benchmark tasks.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen2.5-Coder-7B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "rishr/qwen2.5-coder-7b-fairchild-v5")

Serve with vLLM as a hot-swappable LoRA module:

vllm serve Qwen/Qwen2.5-Coder-7B-Instruct \
  --enable-lora --lora-modules tuned=rishr/qwen2.5-coder-7b-fairchild-v5 \
  --max-lora-rank 16 --enforce-eager

License & data provenance (read before use)

Released under CC-BY-NC-4.0 (non-commercial). The base model Qwen2.5-Coder-7B-Instruct is Apache-2.0, but part of the training mixture (RTL-Coder Resyn27k) is GPT-3.5-generated and carries no upstream license — so this adapter is intended for research / non-commercial use only.

Limitations

• Lint-level correctness was the primary training signal; functional/simulation correctness is not guaranteed.
• May still produce subtly incorrect RTL (e.g. reset polarity, sync vs async) on tricky specs — verify with a real toolchain (Verilator/iverilog) before use.