m2v-bge-m3-edu — static embeddings for educational content

A fast, torch-free static embedding model tuned for educational metadata. It is a Model2Vec distillation of BAAI/bge-m3 enriched with a domain vocabulary of curated education-domain keyword tags, so that subject terms and multi-word phrases (e.g. "Quadratische Gleichung", "Erneuerbare Energien") are represented as atomic units.

The base model (bge-m3) is multilingual, so the model still embeds many languages; the added domain vocabulary, however, is primarily German, so the strongest gains are on German educational text. Inference is numpy-only — no PyTorch required — and much faster and lighter than a transformer.

How it was built

• Base model: BAAI/bge-m3 (the Model2Vec team's recommended base for multilingual data).
• Distillation: model2vec.distill with CLS / first pooling (matching bge-m3's native pooling — this choice has a large effect on quality), pca_dims=512, float32, SIF weighting (sif_coefficient=1e-4).
• Domain vocabulary: ~70,000 cleaned keyword tags (document frequency ≥ 3) extracted from the keyword field of ~340k educational resources — mostly German subject terms — added on top of bge-m3's subword vocabulary. Subword fallback is kept for out-of-vocabulary text.

Evaluation

Evaluated on an educational subject-classification task and compared against other embedding models.

Methodology (identical across models — only the embedding model differs): 25,068 educational items with 47 subject labels and a fixed train / validation / held-out test split. Each embedding model is frozen; a small MLP classification head is trained on top (Optuna hyperparameter search on validation), per-label decision thresholds are tuned on validation, and F1 is reported on the held-out test split that is never used for training or tuning (honest, non-optimistic evaluation).

embedding model	dim	F1-macro	F1-micro	inference
intfloat/multilingual-e5-small	384	0.587	0.777	requires torch
m2v-bge-m3-edu (this model)	512	0.581	0.746	torch-free (numpy)
m2v-gte-multilingual-768 (generic static)	768	0.559	0.747	torch-free (numpy)

What this shows: the education-domain vocabulary lifts macro-F1 over a generic multilingual static baseline (0.559 → 0.581). A transformer (e5-small) remains slightly ahead on this task — an expected trade-off for static embeddings — but this model needs no PyTorch and is far lighter at inference. In our CPU benchmark it encoded ~400 texts/s single-threaded (BLAS capped to one core); throughput scales with cores.

For this specific classification task, a plain TF-IDF + linear classifier scores higher still (macro ≈ 0.62). So the intended value of this model is fast, torch-free semantic embeddings for the education domain, not being the single best classifier for one dataset. These are domain-specific benchmark numbers, not a general-purpose (e.g. MTEB) score.

Intended use

Fast embedding of (mostly German) educational metadata — titles, descriptions, keywords — for classification, clustering, semantic search and retrieval, especially in CPU-only or low-resource deployments where a transformer is too heavy.

Usage

from model2vec import StaticModel

model = StaticModel.from_pretrained("JanSchachtschabel/m2v-bge-m3-edu")
embeddings = model.encode(["Arbeitsblatt zur Bruchrechnung, Klasse 6"])

Or via Sentence Transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("JanSchachtschabel/m2v-bge-m3-edu")
embeddings = model.encode(["Arbeitsblatt zur Bruchrechnung, Klasse 6"])

How Model2Vec works

Model2Vec passes a vocabulary through a sentence-transformer, reduces dimensionality with PCA, and weights the token embeddings with SIF weighting. At inference it takes the (weighted) mean of the static token vectors of a text — no transformer forward pass.

License

Inherits the base model's license (bge-m3: MIT).

Citation

@article{minishlab2024model2vec,
  author = {Tulkens, Stephan and {van Dongen}, Thomas},
  title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year = {2024},
  url = {https://github.com/MinishLab/model2vec}
}