qwen-devops-lora

LoRA adapter for Qwen2.5-0.5B-Instruct, fine-tuned on a hand-written DevOps/SRE troubleshooting dataset (Kubernetes, Linux, Terraform, AWS, Docker, CI/CD).

• Base model: Qwen2.5-0.5B-Instruct (494M params)
• Method: LoRA, rank 16, alpha 32, applied to q_proj/k_proj/v_proj/o_proj
• Trainable params: 2,162,688 (0.44% of base model)
• Dataset: 60 hand-written instruction/response pairs, 54 train / 6 val

Results

metric	value
train loss (start → end)	3.20 → 2.01
final eval loss	2.31
epochs	6
effective batch size	8

Full config and loss history: run_info.json. Code + dataset: https://github.com/shivam2003-dev/qwen-devops-lora

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base, "Shivam3002/qwen-devops-lora")
tokenizer = AutoTokenizer.from_pretrained("Shivam3002/qwen-devops-lora")

messages = [
    {"role": "system", "content": "You are a senior DevOps/SRE engineer. Give concise, practical troubleshooting steps."},
    {"role": "user", "content": "A pod is stuck in CrashLoopBackOff. How do I debug it?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

Trained on only 60 examples — this is a demo of the LoRA fine-tuning workflow, not a production-quality DevOps assistant. The base model's general knowledge still does most of the work; the adapter nudges style/format toward the training examples.