|
--- |
|
pipeline_tag: sentence-similarity |
|
tags: |
|
- feature-extraction |
|
license: mit |
|
language: |
|
- fr |
|
- en |
|
model-index: |
|
- name: Solon-embeddings-base-0.1 |
|
results: |
|
- task: |
|
type: sentence-similarity |
|
name: Passage Retrieval |
|
dataset: |
|
type: unicamp-dl/mmarco |
|
name: mMARCO-fr |
|
config: french |
|
split: validation |
|
metrics: |
|
- type: recall_at_500 |
|
name: Recall@500 |
|
value: 90.9 |
|
- type: recall_at_100 |
|
name: Recall@100 |
|
value: 80.6 |
|
- type: recall_at_10 |
|
name: Recall@10 |
|
value: 52.5 |
|
- type: map_at_10 |
|
name: MAP@10 |
|
value: 27.4 |
|
- type: ndcg_at_10 |
|
name: nDCG@10 |
|
value: 33.5 |
|
- type: mrr_at_10 |
|
name: MRR@10 |
|
value: 27.9 |
|
--- |
|
|
|
# Solon Embeddings — Base 0.1 |
|
SOTA Open source french embedding model. |
|
|
|
**Instructions :** |
|
Add "query : " before the *query* to retrieve to increase performance of retrieval. |
|
No instructions needed for *passages*. |
|
|
|
|
|
| Model | Mean Score | |
|
| --- | --- | |
|
| **OrdalieTech/Solon-embeddings-large-0.1** | 0.7490 | |
|
| cohere/embed-multilingual-v3 | 0.7402 | |
|
| **OrdalieTech/Solon-embeddings-base-0.1** | 0.7306 | |
|
| openai/ada-002 | 0.7290 | |
|
| cohere/embed-multilingual-light-v3 | 0.6945 | |
|
| antoinelouis/biencoder-camembert-base-mmarcoFR | 0.6826 | |
|
| dangvantuan/sentence-camembert-large | 0.6756 | |
|
| voyage/voyage-01 | 0.6753 | |
|
| intfloat/multilingual-e5-large | 0.6660 | |
|
| intfloat/multilingual-e5-base | 0.6597 | |
|
| Sbert/paraphrase-multilingual-mpnet-base-v2 | 0.5975 | |
|
| dangvantuan/sentence-camembert-base | 0.5456 | |
|
| EuropeanParliament/eubert_embedding_v1 | 0.5063 | |
|
|
|
These results have been obtained through 9 french benchmarks on a variety of text similarity tasks (classification, reranking, STS) : |
|
- AmazonReviewsClassification (MTEB) |
|
- MassiveIntentClassification (MTEB) |
|
- MassiveScenarioClassification (MTEB) |
|
- MTOPDomainClassification (MTEB) |
|
- MTOPIntentClassification (MTEB) |
|
- STS22 (MTEB) |
|
- MiraclFRRerank (Miracl) |
|
- OrdalieFRSTS (Ordalie) |
|
- OrdalieFRReranking (Ordalie) |
|
|
|
We created OrdalieFRSTS and OrdalieFRReranking to enhance the benchmarking capabilities of French STS and reranking assessments. |
|
|
|
(evaluation script available here : github.com/OrdalieTech/mteb) |
