Poster676767/meridian-router-lora

Meridian inbox router - TinyLlama LoRA

A six-category email-routing classifier fine-tuned for the INFO 642 Meridian inbox exercise. This repository contains a trained LoRA adapter and its six-way classification head, tokenizer files, ordered labels, and measured training/evaluation metadata. The base-model weights are downloaded separately when loading the adapter.

Labels

Label order is part of the model interface and is also saved in labels.json.

ID	Label
0	shortage
1	delivery
2	recall
3	billing
4	marketing
5	confirmation

Training

The training source is the supplied INFO 642 TinyLlama LoRA notebook and the course meridian_inbox.csv dataset. The run used the dataset's existing split: 270 training examples and 90 test examples. Test examples were not used for gradient updates.

Setting	Value
Base model	TinyLlama/TinyLlama-1.1B-Chat-v1.0
Base revision	fe8a4ea1ffedaf415f4da2f062534de366a451e6
Architecture	TinyLlama sequence classification with a six-way score head
PEFT task	SEQ_CLS
LoRA rank / alpha / dropout	8 / 16 / 0.05
Target projections	q_proj, k_proj, v_proj, o_proj
Additional saved module	score
Completed epochs	3
Optimizer	AdamW
Learning rate / weight decay	0.0002 / 0.01
Effective batch size	8
Microbatch size	8
Gradient accumulation	1 microbatches per full effective batch; final partial batch is normalized by its actual size
Device / precision	cuda / float32
Gradient checkpointing	False
Maximum input length	96 tokens
Padding	Right padding; EOS is the padding token
Random seed	0

This run uses the effective batch size directly, without gradient accumulation across microbatches. Floating-point operations can vary by hardware and library version, so this run does not claim bit-for-bit equivalence with a separate notebook run. Baseline models and the Groq API comparison are not needed to train this adapter and were not run.

Dataset SHA-256: e4137cd41283f02a13ca050e3123314646fd8d40beede2e6240efa74f35694d0.

Measured evaluation

Held-out test accuracy: 83.33% (75/90). These results are from the completed training run, not estimates.

Label	Precision	Recall	F1	Test support
shortage	0.833	0.667	0.741	15
delivery	0.789	1.000	0.882	15
recall	0.882	1.000	0.938	15
billing	0.682	1.000	0.811	15
marketing	1.000	0.867	0.929	15
confirmation	1.000	0.467	0.636	15

Epoch	Mean training loss	Optimizer steps
1	0.92506	34
2	0.00922	34
3	0.00119	34

Full metrics, including the confusion matrix, are in metrics.json. Rows of the confusion matrix are true labels and columns are predicted labels, in the label order above.

A separate process on cuda reloaded the saved adapter and tokenizer and reproduced all 90 saved test predictions. Confidence differences were within 0.0001. See verification.json for the checks.

Load and classify

import json
import torch
from huggingface_hub import hf_hub_download
from peft import PeftConfig, PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo_id = 'Poster676767/meridian-router-lora'
adapter_config = PeftConfig.from_pretrained(repo_id)
tokenizer = AutoTokenizer.from_pretrained(repo_id)
with open(hf_hub_download(repo_id, "labels.json"), encoding="utf-8") as f:
    labels = json.load(f)

base = AutoModelForSequenceClassification.from_pretrained(
    adapter_config.base_model_name_or_path,
    revision=adapter_config.revision,
    num_labels=len(labels),
    torch_dtype=torch.float32,
)
base.config.pad_token_id = tokenizer.pad_token_id
model = PeftModel.from_pretrained(base, repo_id).cpu().eval()

def classify(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=96)
    with torch.inference_mode():
        probabilities = model(**inputs).logits.softmax(-1)[0]
    label_id = int(probabilities.argmax())
    return {"label": labels[label_id], "confidence": float(probabilities[label_id])}

An initial warning about a newly initialized classification head is expected while constructing the base classifier; the trained head is then restored from the adapter.

Scope and limitations

This is a course demonstration evaluated on the supplied test split. Evaluation does not establish performance on other inboxes or writing styles. The classifier always selects one of six labels; it has no out-of-scope category. Inputs are truncated to 96 tokens. Confidence is the model's softmax score and has not been calibrated.

Runtime versions

Package	Version
torch	2.11.0+cu130
transformers	4.49.0
peft	0.14.0
accelerate	1.4.0
safetensors	0.8.0
numpy	2.1.3

Base-model license

The TinyLlama base-model repository lists Apache 2.0 as its license. No separate license has been selected for this new adapter in this repository.