SpaceEmbed-Qwen3-0.6B-Domain

A text-embedding model for space-domain retrieval: Qwen3-Embedding-0.6B fine-tuned on question→passage pairs from space-engineering and space-science documents (Wikipedia, NASA, ESA, ISRO). This Domain variant is the fully fine-tuned model: the best space-domain retrieval, with a somewhat larger drop on general-domain text. For mixed workloads use the Balanced variant, SpaceEmbed-Qwen3-0.6B. Try it in the Space Tools desktop app (SpaceEmbed).

Usage

Queries take an instruction prefix (stored as the query prompt); passages are embedded as-is. Passages formatted as title | section\ntext work best.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("llmlabcse/SpaceEmbed-Qwen3-0.6B-Domain")
queries = ["How is momentum unloaded from reaction wheels?"]
passages = ["Attitude control | Momentum management\nMagnetic torquers dump excess wheel momentum ..."]
q = model.encode(queries, prompt_name="query", normalize_embeddings=True)
p = model.encode(passages, normalize_embeddings=True)
print(q @ p.T)

The query prompt is "Instruct: Given a space-domain search query, retrieve the passage that answers it\nQuery: ". GGUF versions for llama.cpp: SpaceEmbed-Qwen3-0.6B-GGUF.

Training

• Corpus: 906 space-domain documents (598 Wikipedia articles, 308 NASA/ESA/ISRO reports and handbooks) screened for technical content, cleaned and cut into 27,119 section-aware passages (~250 words). Documents were split 80/10/10 into train/validation/test by document family (editions and near-duplicates kept together), so test documents were never seen in training.
• Data: GPT-OSS-120B wrote one natural question and one keyword query per sampled passage (8,186 training passages → 16,372 queries). One hard negative per pair was mined with the base model from other training documents.
• Recipe: sentence-transformers, CachedMultipleNegativesRankingLoss, batch 128, lr 2e-5, 1 epoch, bf16, best checkpoint by validation nDCG@10.

Evaluation

Held-out test split: 1,888 queries, each searching all 27,119 passages.

Model	Recall@1	Recall@10	MRR@10	nDCG@10
BM25	0.659	0.926	0.752	0.794
Qwen3-Embedding-0.6B (base)	0.670	0.943	0.766	0.809
SpaceEmbed Balanced	0.724	0.972	0.820	0.858
SpaceEmbed Domain (this model)	0.734	0.973	0.828	0.864

(sentence-transformers, max_seq_length 512. With full-length passages in llama.cpp the Domain GGUF reaches MRR@10 0.843 vs 0.763 for the base model.) On the 140 test queries where BM25 fails (answer not in its top 10), MRR@10 rises from 0.501 (base) to 0.632.

General-domain retrieval (NanoBEIR, 7 datasets, mean nDCG@10): base 0.592 → this model 0.560.

Limitations

The test queries were generated by the same LLM procedure as the training queries, so part of the gain may reflect that query style; human-written queries have not yet been evaluated. English only. The corpus over-represents NASA, ESA and ISRO missions.

Licence

Apache-2.0, as the base model. Training text came from Wikipedia (CC BY-SA 4.0) and public agency documents; no training text is redistributed in this repository.