chauben/advisorai-qwen2.5-14b-stevens

AdvisorAI — Qwen2.5-14B Stevens (Fine-Tuned)

Fine-tuned Qwen/Qwen2.5-14B-Instruct for AdvisorAI, an academic advising assistant for Stevens Institute of Technology. This checkpoint is the merged full model (DoRA adapter fused into the base weights).

Model Hub: chauben/advisorai-qwen2.5-14b-stevens

Model summary

Base model	Qwen/Qwen2.5-14B-Instruct
Parameters	~15B (BF16)
Fine-tuning	QDoRA (4-bit NF4 + DoRA r=64 + rsLoRA) + NEFTune (α=5)
Training	2× NVIDIA RTX 3090, DDP, TRL SFTTrainer
Domain	Stevens academic advising (courses, faculty, programs, admissions, etc.)
Format	Safetensors, Qwen ChatML template

What this model does

Answers student-style questions about Stevens in a helpful, markdown-formatted advising tone:

• Courses and prerequisites
• Programs and degree requirements
• Faculty and teaching (when covered in training data)
• Admissions, financial aid, campus life, and general advising

Training data

Split	Examples
Train	71,883
Eval	7,988
Total	79,871

Built from Stevens-related sources and LLM-assisted Q&A generation (Gemini + Qwen scoring), formatted as multi-turn chat JSONL. Approximate mix: 95% single-turn, ~5% multi-turn; categories dominated by course and general (65% combined).

Training details

Parameter	Value
Epochs	2
Effective batch size	32
Learning rate	8e-5
Max seq length	2048
LoRA rank	64
LoRA alpha	128
DoRA / rsLoRA	enabled
NEFTune α	5
Optimizer	paged_adamw_8bit

Run name: advisorai-qwen25-14b-qdora-neftune-v1

Post-training: DoRA adapter merged into base → uploaded as this Hub checkpoint.

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

MODEL_ID = "chauben/advisorai-qwen2.5-14b-stevens"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "system",
        "content": (
            "You are AdvisorAI, a knowledgeable academic advisor for "
            "Stevens Institute of Technology. Be specific — cite course codes "
            "and requirements when available. Use markdown."
        ),
    },
    {"role": "user", "content": "What are the requirements for the CS MS program at Stevens?"},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.3,
        top_p=0.9,
        do_sample=True,
        repetition_penalty=1.05,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))


Citation:
@misc{advisorai-qwen25-14b-stevens-2026,
  title        = {AdvisorAI: Fine-Tuned Qwen2.5-14B for Stevens Institute Academic Advising},
  author       = {Nitin Chaube},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/chauben/advisorai-qwen2.5-14b-stevens}},
  note         = {Fine-tuned from Qwen/Qwen2.5-14B-Instruct; QDoRA + NEFTune}
}