bge-m3-swedish-superlim

Swedish contrastive fine-tune of BAAI/bge-m3 (568M). It beats KBLab/sentence-bert-swedish-cased on the two SuperLim tasks that card published: SweParaphrase (STS) and SweFAQ (in-category retrieval).

Default use: keep the first 768 dimensions and re-L2. BGE-M3 is Matryoshka-trained; that cut keeps the STS win and lifts SweFAQ to the Qwen3-Embedding-8B zero-shot number.

Test (SuperLim-2, held-out)

Same frozen sbx/superlim-2 splits as the KBLab re-run. Spearman of cosine vs human 0–5 on SweParaphrase. SweFAQ is in-category accuracy (109 items).

Model	Params	SweParaphrase ρ	SweFAQ
KBLab v2.0 (card)	110M	0.8213	0.5871
KBLab v2.0 (our re-run)	110M	0.8207	0.5780
BGE-M3 zero-shot	568M	0.8165	0.6055
this @768 (recommended)	568M	0.8354	0.6514
this @1024	568M	0.8353	0.6422

SweParaphrase test was never in train. SweFAQ used the official train split (plus Swedish healthcare Q–A). Do not call SweFAQ zero-shot.

MTEB(Scandinavian, v1)

Same five models, mteb 2.19.3, max_seq_length=512 (KBLab 256). 27/28 tasks — DKHateClassification is gated and was skipped. Official SweFAQ score is nDCG@10, not SuperLim in-category accuracy. SweFAQ-train overlap applies to this checkpoint; Swedn / SweRec / Dalaj / SwedishSentiment do not.

Swedish tasks:

Task	KBLab	BGE-M3	this @768
DalajClassification	0.5003	0.4979	0.4993
SweRecClassification	0.6858	0.8121	0.7647
SwedishSentimentClassification	0.8737	0.9572	0.9293
SweFaqRetrieval nDCG@10 †	0.7331	0.7170	0.8221
SwednRetrieval	0.7067	0.7335	0.7809
SwednClusteringP2P	0.3641	0.3654	0.3554
SwednClusteringS2S	0.2486	0.1483	0.1991
MassiveIntent (sv)	0.6587	0.6915	0.6824
MassiveScenario (sv)	0.7475	0.7494	0.7553
SV mean (9)	0.6132	0.6303	0.6432

† Supervised for this model (SweFAQ-train). SwednRetrieval is the clean Swedish retrieval number: +0.047 vs vanilla BGE, +0.074 vs KBLab.

Nordic category means (27 tasks):

	KBLab	BGE-M3	this @768
Classification (12)	0.5243	0.6115	0.5960
Retrieval (7)	0.4466	0.6424	0.6915
Clustering (6)	0.4091	0.3953	0.4101
Bitext (2)	0.4338	0.6924	0.6876
Overall (27)	0.4719	0.5774	0.5862

The Swedish mix is a retrieval lift. Classification dropped vs vanilla BGE (SweRec, SwedishSentiment). Not a general “best Nordic embedder” claim.

Raw task JSON: mteb/MTEB_Scandinavian_v1/. SuperLim test dump: superlim/test.json. These files document the run; they do not put the model on the official MTEB leaderboard.

Use

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("oxfrug/bge-m3-swedish-superlim")
QUERY = "Represent this sentence for searching relevant passages: "

def embed(texts, query=False, dim=768):
    texts = [QUERY + t if query else t for t in texts]
    vec = model.encode(texts, normalize_embeddings=True)
    if dim and vec.shape[1] > dim:
        vec = vec[:, :dim]
        vec /= np.clip(np.linalg.norm(vec, axis=1, keepdims=True), 1e-12, None)
    return vec

# STS / clustering: no prefix
s = embed(["Ett plan lyfter.", "Ett flygplan lyfter."])
print(float(s[0] @ s[1]))

# Retrieval: prefix the query only
q = embed(["När får jag föräldrapenning?"], query=True)
d = embed(["Du kan ansöka på Mina sidor."], query=False)

Train data

One epoch, MultipleNegativesRankingLoss, from vanilla BAAI/bge-m3:

• SweParaphrase train pairs with score ≥ 4
• SweNLI entailment (40k)
• SweFAQ train Q–A (BGE query prefix)
• Swedish healthcare Q–A

SuperLim dev/test strings were blocked. One epoch, MNRL, full-FT in bf16.

Limits

• 568M / ~1.1 GB bf16. Slower to encode than 110M KBLab.
• SweFAQ test is 109 items. @768 is +8 correct vs the re-run.
• SuperLim claim is SweParaphrase + SweFAQ only. MTEB Scandinavian is extra context: retrieval up, classification down vs vanilla BGE.
• 4-bit NF4 gives the STS win back (0.820). Ship bf16.

License

MIT, same as BGE-M3. Fine-tune by oxfrug.