sutd-bge-large-aug98

Finetuned BAAI/bge-large-en-v1.5 for job-description-to-module retrieval in the SUTD Course Recommendation Chatbot (MLOps Group 9). This is the production embedding model used in the final Hybrid pipeline.

Given a job description, this model retrieves relevant SUTD elective modules. It is the dense retrieval backbone in the RAG and Hybrid pipelines.

Model Details

Property	Value
Base model	BAAI/bge-large-en-v1.5
Embedding dimension	1024
Max sequence length	512
Similarity function	Cosine
Loss	MultipleNegativesRankingLoss

Training Data

Trained on 98 (job description, relevant SUTD module) pairs spanning four pillars: ASD, EPD, ESD, ISTD/CSD:

• 67 hand-annotated pairs — original dataset
• 31 augmented pairs — sourced from MyCareersFuture postings, labelled by Claude

After train/validation splitting and hard-negative expansion by the Sentence Transformers trainer, this produces 831 training samples and 146 validation samples.

A version trained on the smaller 67-pair dataset is available at henreads/sutd-bge-large-ft67.

Training Setup

• Hardware: Modal A10G (24 GB VRAM)
• Epochs: up to 10 with early stopping (patience 4); converged at epoch 3
• Effective batch size: 16 (per-device batch 4, gradient accumulation 4)
• Learning rate: 2e-5
• Tracking: Weights & Biases (sutd-mlops-bge-finetune)

Evaluation

Evaluated on a 10-job held-out retrieval set (completely separate from training). This model achieves the best retrieval NDCG@10 among all embedding configurations.

Model	NDCG@10
BGE-large-en-v1.5 (base)	0.679
sutd-bge-large-ft67	0.747
sutd-bge-large-aug98 (this model)	0.770

When paired with the matched henreads/sutd-reranker-aug98, the Hybrid pipeline achieves a Gemini LLM-as-judge chat quality score of 4.06 / 5.0, exceeding the project success criterion of 4.0.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("henreads/sutd-bge-large-aug98")

job_description = "Data Scientist at GovTech. Build ML models with Python..."
module_passage = "50.007 Machine Learning — Topics: supervised learning, neural networks..."

embeddings = model.encode([job_description, module_passage], normalize_embeddings=True)
similarity = embeddings[0] @ embeddings[1]
print(similarity)

Project

Part of the SUTD Course Recommendation Chatbot — MLOps Group 9.
Code: github.com/henreads/sutd-mlops-group9