rohit21789/genailab

Phi-2 Fine-Tuned for Technical Support Response Generation

This is a LoRA adapter fine-tuned on top of microsoft/phi-2 to generate helpful, on-brand responses to technical/customer support queries.

Task

Given an informally- or formally-phrased customer support question, generate a helpful, empathetic support-agent-style response.

Base Model

• Base: microsoft/phi-2 (2.7B parameters)
• Quantization for training: 4-bit (QLoRA), NF4, double quantization
• Fine-tuning method: LoRA (rank 16, alpha 32, dropout 0.05)
• Target modules: q_proj, k_proj, v_proj, dense
• Trainable parameters: 10.48M / 2.79B (0.376%)

Dataset

Bitext Customer Support LLM Chatbot Training Dataset — 2,000 examples sampled (1,600 train / 200 validation / 200 test).

Training Configuration

• Epochs: 3
• Batch size: 4 (effective 16 via gradient accumulation)
• Learning rate: 2e-4
• Precision: fp16 mixed precision
• Hardware: single NVIDIA T4 (Google Colab)
• Training time: ~35.5 minutes

Results

Evaluated on 30 held-out test examples, generation via greedy decoding.

Metric	Base Model	Fine-Tuned (this adapter, fp16 merge)
ROUGE-1	0.1815	0.5068
ROUGE-L	0.1330	0.3410
Inference latency (avg)	2.21s	4.60s
Peak GPU memory	2.53 GB	9.94 GB

Note on merging: merging this adapter directly into a 4-bit quantized base model causes measurable quality degradation from compounding quantization error. Merge into a full-precision (fp16) copy of the base model instead — see usage below.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-2", torch_dtype=torch.float16, trust_remote_code=True
)
model = PeftModel.from_pretrained(base_model, "deepeshd12/phi2-support-lora")
tokenizer = AutoTokenizer.from_pretrained("deepeshd12/phi2-support-lora")

prompt = "Instruction: I want to check my order status\nResponse:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• Trained on a 2,000-example subset of a larger dataset; broader coverage would benefit from more data.
• ROUGE is a lexical overlap metric and under-represents genuine quality improvements when phrasing differs from the reference (observed directly during development — see repo for details).
• Evaluated on 30 test examples; not a large-scale benchmark.