m2v-gte-256-int8-edu — ultra-compact int8 static embeddings for educational content

The smallest member of the family: an 80 MB, int8-quantized static embedding model for educational metadata, built for the tightest deployments (edge, embedded, memory-limited). It is the int8 variant of m2v-gte-256-edu — same Model2Vec distillation of Alibaba-NLP/gte-multilingual-base (mean pooling, 256-d, ~70k German education keyword vocabulary), then quantized to int8.

numpy-only inference, no PyTorch. The base is multilingual; the domain vocabulary is primarily German.

Size / accuracy trade-off (measured, honest)

Quantizing the 256-d table from float32 to int8 cuts the model to a quarter of the size, at a small, measured cost on our German subject-classification benchmark (25k items, 47 labels, honest held-out test):

variant	size	F1-macro	F1-micro
m2v-gte-256-edu (float32)	321 MB	0.568	0.743
m2v-gte-256-int8-edu (this, int8)	80 MB	0.558	0.737
Δ (int8 − float32)	−75 % size	−0.010	−0.006

So int8 costs about 1 macro-F1 point for a 4× smaller model. (Note: the per-vector cosine similarity between int8 and float32 embeddings is ~0.9997 — very high — yet the downstream classifier is slightly more sensitive than that suggests, hence the ~0.01 drop. Measure, don't assume.) For memory/bandwidth-constrained applications this is usually a good deal; where accuracy matters more, use the float32 sibling.

Intended use

Fast embedding of (mostly German) educational metadata — titles, descriptions, keywords — in the smallest possible footprint: edge devices, embedded systems, serverless/cold-start, or anywhere an 80 MB model beats a 321 MB one. For classification, clustering, semantic search and retrieval.

Usage

from model2vec import StaticModel

model = StaticModel.from_pretrained("JanSchachtschabel/m2v-gte-256-int8-edu")
embeddings = model.encode(["Arbeitsblatt zur Bruchrechnung, Klasse 6"])

Or via Sentence Transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("JanSchachtschabel/m2v-gte-256-int8-edu")
embeddings = model.encode(["Arbeitsblatt zur Bruchrechnung, Klasse 6"])

Family

• m2v-bge-m3-edu — 512-d, strongest macro-F1.
• m2v-e5-large-edu — 512-d.
• m2v-gte-256-edu — 256-d float32 (this model's source).
• m2v-gte-256-int8-edu — 256-d int8, smallest (this model).

How Model2Vec works

Model2Vec passes a vocabulary through a sentence-transformer, reduces dimensionality with PCA, and weights the token embeddings with SIF weighting. At inference it takes the (weighted) mean of the static token vectors of a text — no transformer forward pass.

License

Inherits the base model's license (gte-multilingual-base: Apache-2.0).

Citation

@article{minishlab2024model2vec,
  author = {Tulkens, Stephan and {van Dongen}, Thomas},
  title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year = {2024},
  url = {https://github.com/MinishLab/model2vec}
}