minilm-fastapi-docs-retrieval

sentence-transformers/all-MiniLM-L6-v2 fine-tuned for question-to-passage retrieval over FastAPI documentation. It maps text to 384-dimensional vectors and was built for the RAG pipeline in multimodal-rag-copilot.

Training

• Data: 743 (question, passage) pairs. Passages are chunks of the FastAPI tutorial documentation (MIT licensed). The questions were generated by an LLM (Gemini) from each passage.
• Method: MultipleNegativesRankingLoss (other passages in a batch act as negatives), 3 epochs, batch size 32, learning rate 2e-5, trained with sentence-transformers on PyTorch.
• Split: documents were split by file into train and held-out sets, so the held-out documents were never seen in training.

Evaluation

Held-out documents: 256 synthetic queries searched over 64 passages.

Model	Recall@1	Recall@3	Recall@5	MRR@10
sentence-transformers/all-MiniLM-L6-v2 (base)	0.566	0.789	0.863	0.695
This model	0.730	0.906	0.965	0.831

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Sushmitha2003/minilm-fastapi-docs-retrieval")
embeddings = model.encode(["How do I add a prefix to a router in FastAPI?"], normalize_embeddings=True)

Limitations

• Tuned for one domain (FastAPI documentation); gains are not expected to transfer to other domains.
• Evaluated on LLM-generated queries from the same documentation, not on real user queries.
• Small evaluation pool and a single training run, so no confidence intervals.