glossia/Rambley-V2.1-350M-Embed

Rambley V2.1 350M Embed

Finetuned version of glossia/Rambley-V2.1-350M for embedding purposes on a subset of the nomic-embed training data.

Architecture

• Last-token pooling + L2 normalization
• 896 dimensional embeddings
• NO Matryoshka Representation Learning (embeddings will not maintain semantic relevance when truncated)
• NO quantization-aware training (embeddings may not maintain semantic relevance when quantized)
• See the original model card for details on the backbone architecture

Benchmarks

MTEB

Task type	Tasks	Average
Classification	8	72.79
Clustering	8	44.83
Pair classification	3	78.93
Reranking	2	45.16
Retrieval	10	43.28
Semantic textual similarity	9	75.81
Summarization	1	28.63
Mean across tasks	41	58.83
Mean across task types	7	55.63

Task	Score
Classification
AmazonCounterfactualClassification	66.87
Banking77Classification	77.05
ImdbClassification	81.14
MTOPDomainClassification	90.78
MassiveIntentClassification	65.46
MassiveScenarioClassification	74.05
ToxicConversationsClassification	65.88
TweetSentimentExtractionClassification	61.12
Clustering
ArXivHierarchicalClusteringP2P	56.26
ArXivHierarchicalClusteringS2S	53.24
BiorxivClusteringP2P.v2	40.01
MedrxivClusteringP2P.v2	37.40
MedrxivClusteringS2S.v2	36.00
StackExchangeClustering.v2	56.99
StackExchangeClusteringP2P.v2	37.01
TwentyNewsgroupsClustering.v2	41.76
Pair classification
SprintDuplicateQuestions	91.64
TwitterSemEval2015	60.95
TwitterURLCorpus	84.21
Reranking
AskUbuntuDupQuestions	59.13
MindSmallReranking	31.18
Retrieval
ArguAna	51.00
CQADupstackGamingRetrieval	51.22
CQADupstackUnixRetrieval	33.17
ClimateFEVERHardNegatives	26.31
FEVERHardNegatives	68.32
FiQA2018	27.98
HotpotQAHardNegatives	48.40
SCIDOCS	16.66
TRECCOVID	54.87
Touche2020Retrieval.v3	54.82
Semantic textual similarity
BIOSSES	81.10
SICK-R	68.63
STS12	68.19
STS13	79.81
STS14	72.89
STS15	82.00
STS17	86.60
STS22.v2	64.54
STSBenchmark	78.52
Summarization
SummEvalSummarization.v2	28.63

Detailed logs in glossia/Rambley-V2.1-350M-Embed-mteb-results

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("glossia/Rambley-V2.1-350M-Embed", trust_remote_code=True)
embeddings = model.encode(["Example text"])