jajamken/DLTP_project

LoRA adapters for natural-language CAD program editing (IE 643 course project, team Tired Perceptron)

• patch_lora/: the model replies with a unified diff that edits the CadQuery program (ours).
• rewrite_lora/: the same training, but the model replies with the whole edited program (baseline).

Training (identical for both): Qwen2.5-Coder-7B-Instruct, LoRA r=16, alpha=32, dropout 0.05 on q/k/v/o/gate/up/down, lr 2e-4 cosine, 3% warmup, 1 epoch over 27,839 triplets (dataset repo jajamken/DLTP_project), effective batch 32, bf16, loss only on the reply. Prompt format: see prompts.py in the project code.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct", dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(base, "jajamken/DLTP_project", subfolder="patch_lora")