riverguide-retriever-full

Qwen3-Embedding-0.6B fine-tuned to find Indian River State College web pages for students' questions, abbreviations, and nicknames (RN, rad tech, BSN, plumber). One forward pass per query, no generated tokens. Built for RiverGuide, the College's student assistant, in https://github.com/joelcook/riverguide-retriever.

full: trained on every generated query, including ones with the 75 test abbreviations, so its own abbreviation score (73/75) is contaminated. The honest number comes from a twin trained without any query containing them ("strict"): 67/75 on its own, and in RiverGuide's agent loop it linked the right page for 71 of 75, the most of any search tested, on abbreviations it never saw.

Use

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("joeltcook/riverguide-retriever-full")
INSTRUCT = "Instruct: Given a college student's search query, retrieve the College web page that answers it\nQuery: "
q = model.encode(INSTRUCT + "rad tech", normalize_embeddings=True)
d = model.encode(["<page title> > <section>\n\n<chunk text>"], normalize_embeddings=True)
scores = d @ q

Queries need the instruction prefix; chunks are embedded as plain text. A page's score is its best chunk's. The weights are bf16; on a CPU, load with model_kwargs={"dtype": torch.float32} for speed (RiverGuide serves it on a CPU at about 150 ms a query).

Results (right page in the top 5)

	keyword (BM25)	stock 0.6B	this model	hybrid
abbreviations	51/75	65/75	73/75	73/75
golden	18/21	19/21	19/21	19/21
val	631/942	757/942	909/942	909/942

val holds out whole pages, so it is the clean comparison.

In RiverGuide (2026-10-24)

Real students' questions. 523 distinct first questions that students typed into the College's chat assistant (Sept–Oct 2026). For each, the top 5 pages from each search were judged blind (shuffled, unlabeled) by Qwen3.8-27B: did any page give the student what they need? Only aggregate counts were kept, never the questions.

Search on the student's own words	Answered
keyword (BM25, production at the time)	309 (59%)
stock Qwen3-Embedding-0.6B	414 (79%)
this model	458 (88%)

Head to head, this model answered 160 questions that keyword search missed; keyword search answered 11 that it missed.

Searching first. RiverGuide's answering model (Qwen3.8-27B) works in a tool loop. With this model behind the search tool, and run on the student's question before the first step (except questions about their own records), RiverGuide's own golden gate (54 questions, the production Worker running locally) went from 43.7 to 45.5 passed (averages over 3 and 2 runs), with 32% fewer tool calls (111 to 75.5) and a 26% lower median answer time (31.7 s to 23.5 s, four questions at a time).

Caveats: a few runs each (about ±2 questions of run-to-run noise on the golden set), and the real-question judge is from the same model family that wrote the training queries.

Training

• Data: about 31,000 student-style queries written by Qwen3.8-27B for 5,923 chunks of the College's public website and catalog (the 2026-10-22 crawl, 2,389 pages): a full question, a 2–5 word search, the shorthand or abbreviation, and a misspelling per chunk, plus six "how people refer to this program" queries for each of 377 program pages.
• Judge: Qwen3-Reranker-0.6B dropped queries their own chunk doesn't answer, and kept hard negatives only when it judged they don't answer the query either. The College repeats itself across its site and catalog, so a close chunk is often another right answer.
• Objective: InfoNCE (τ = 0.05) against the right chunk, two hard negatives, and the rest of a 64-query batch. In-batch candidates are masked when they are another chunk of the right page, the same text, or scored above the right chunk by the base model (GISTEmbed's idea).
• Run: one epoch, lr 1e-5, bf16, on one RTX 4090 (about 40 minutes). The best step was picked by MRR@10 on held-out pages.

Intended use and limits

• Use it to find Indian River State College pages for students' questions in English, as the first stage of a search or RAG system.
• Facts live in the pages, not the model. When a fee, date, or policy changes, re-embed the changed pages; no retraining is needed. Retrain when the College's vocabulary changes (new programs, new names). RiverGuide re-embeds nightly and checks for drift weekly.
• Other colleges: it should still beat keyword search on general college questions, but it learned IRSC's names and programs. Retrain on your own pages for the full gain.
• Queries need the instruction prefix above.