Zephyr RTOS documentation RAG
A retrieval index over the Zephyr RTOS documentation, built to give a local coding assistant current, citable answers about Zephyr — offline, on CPU, with every claim traceable to the page it came from.
- Space: https://huggingface.co/spaces/eoinedge/zephyrproject-rag
- Model repo: https://huggingface.co/eoinedge/zephyrproject
- Source: https://github.com/eoinjordan/zephyrproject-rag
Why this exists
Zephyr is large and moves quickly. A general coding model trained months ago
will confidently produce a Kconfig symbol that was renamed, a devicetree
binding that never existed, or a west invocation from an old release — and on
embedded work that costs hours on hardware before the mistake shows itself.
Retrieval fixes the currency problem without retraining anything. The index is rebuilt from the documentation source, so it is exactly as current as the commit it was built from, and every answer carries the file it came from.
Current index
| Documents | 829 |
| Chunks | 6,949 |
| Embedding model | sentence-transformers/all-MiniLM-L6-v2 (384-dim) |
| Index | FAISS IndexFlatIP, cosine over normalised vectors |
| Source | zephyrproject-rtos/zephyr main @ 580547d |
Quick start
git clone https://github.com/eoinjordan/zephyrproject-rag
cd zephyrproject-rag
python -m venv .venv && .venv\Scripts\Activate.ps1
pip install -r requirements.txt
python scripts/fetch_docs.py # 829 docs from doc/ (RST)
python scripts/build_index.py # chunk, embed, FAISS
python scripts/ask.py "How do I define a devicetree binding?" --retrieve-only
--retrieve-only skips the generator entirely: no model download, CPU-only,
answers in under a second. It is genuinely useful on its own — most of the time
"which page documents this?" is the question.
Drop the flag to generate an answer with Qwen Coder over the retrieved passages, or run the Gradio app:
python app.py # http://localhost:7860
How the corpus is built
Source: the RST in doc/, not the rendered HTML. Zephyr publishes no
llms.txt (docs.zephyrproject.org/llms.txt 404s), so the choice was between
crawling rendered pages via sitemap.xml or reading the documentation source.
Rendered HTML carries navigation, version banners and the whole Doxygen API
surface, all of which lands in chunks and competes with prose during retrieval.
The RST is authoritative, versioned and diffable.
Chunking follows document structure. RST carries its own section underlines, so chunks are split on headings rather than on a character count. That keeps a chunk to one topic and lets every passage cite its section. Chunks are bounded at 1,600 characters, with a line-level backstop for sections that contain no paragraph breaks.
Non-prose is filtered. A first pass indexed a JavaScript 404 page, a pip requirements file, and produced a single 126 KB chunk. Files that are not documentation are skipped by name; sections that are lists of GitHub issue IDs are skipped by heading; and chunks that are mostly punctuation or short tokens are dropped by a prose heuristic.
Release notes are down-weighted at query time. They are a quarter of the
corpus and are written in the same vocabulary as the docs they describe, so
"what does west build actually do?" retrieved a v0.7.0 changelog entry above
develop/west/build-flash-debug. They are penalised unless the question is
about a version or a change, in which case the penalty lifts.
Fine-tuning the retriever
all-MiniLM-L6-v2 is trained on general web text, where "binding" means a
contract and "west" is a direction. scripts/train_embeddings.py adapts it to
Zephyr's vocabulary using pairs mined from the docs' own structure — a section
heading is a natural query for the body beneath it — with
MultipleNegativesRankingLoss supplying in-batch negatives.
python scripts/train_embeddings.py --eval-only # baseline recall@5
python scripts/train_embeddings.py --epochs 1
python scripts/build_index.py --model data/embedding-model
It reports recall@5 on held-out pairs before and after, and refuses to save a model that did not beat the baseline. A negative result is a real result.
Retriever and generator are different problems
Worth stating plainly, because the two get conflated: fine-tuning the embedding model does not teach Qwen Coder anything about Zephyr. It makes the passages Qwen is handed more likely to be the right ones. Both are worth doing and they compose — better retrieval helps any generator, including one that was never fine-tuned — but a LoRA on Qwen is a separate piece of work, not this one.
For an offline assistant, retrieval is the higher-leverage half: it is cheap, it is current the moment you rebuild the index, and it is auditable.
Offline use
Everything runs locally once fetched. The index is ~17 MB; the embedding model
is ~90 MB; Qwen2.5-Coder-1.5B-Instruct is ~3 GB. No network calls at query
time and no API keys.
Licence
Apache-2.0, matching Zephyr. The indexed content is Zephyr documentation, copyright its contributors, under Apache-2.0 — this repository redistributes derived chunks with their source paths intact so attribution survives retrieval.