Hugging Face's logo

antoinelouis
/
biencoder-camembert-L2-mmarcoFR

Sentence Similarity
sentence-transformers
PyTorch
Safetensors
French
camembert
passage-retrieval
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
1
antoinelouis commited on
Commit
d15ff9e
•
1 Parent(s): 90f5aab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -8
README.md CHANGED
@@ -113,14 +113,14 @@ print(similarity)
113
 
114
  We evaluate the model on the smaller development set of [mMARCO-fr](https://ir-datasets.com/mmarco.html#mmarco/v2/fr/), which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compare the model performance with other CamemBERT-based biencoder models fine-tuned on the same dataset. We report the mean reciprocal rank (MRR), normalized discounted cumulative gainand (NDCG), mean average precision (MAP), and recall at various cut-offs (R@k).
115
 
116
- | | model | Vocab. | #Param. | Size | R@500 | R@100(↑) | R@10 | MRR@10 | NDCG@10 | MAP@10 |
117
- |---:|:-------------------------------------------------------------------------------------------------------------|:-------|--------:|------:|-------:|---------:|-------:|-------:|--------:|-------:|
118
- | 1 | [biencoder-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-base-mmarcoFR) | 🇫🇷 | 111M | 445MB | 89.1 | 77.8 | 51.5 | 28.5 | 33.7 | 27.9 |
119
- | 2 | [biencoder-camembert-L10-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L10-mmarcoFR) | 🇫🇷 | 96M | 386MB | 87.8 | 76.7 | 49.5 | 27.5 | 32.5 | 27.0 |
120
- | 3 | [biencoder-camembert-L8-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L8-mmarcoFR) | 🇫🇷 | 82M | 329MB | 87.4 | 75.9 | 48.9 | 26.7 | 31.8 | 26.2 |
121
- | 4 | [biencoder-camembert-L6-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L6-mmarcoFR) | 🇫🇷 | 68M | 272MB | 86.7 | 74.9 | 46.7 | 25.7 | 30.4 | 25.1 |
122
- | 5 | [biencoder-camembert-L4-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L2-mmarcoFR) | 🇫🇷 | 54M | 216MB | 85.4 | 72.1 | 44.2 | 23.7 | 28.3 | 23.2 |
123
- | 6 | **biencoder-camembert-L2-mmarcoFR** | 🇫🇷 | 40M | 159MB | 81.0 | 66.3 | 38.5 | 20.1 | 24.3 | 19.7 |
124
 
125
  ***
126
 
 
113
 
114
  We evaluate the model on the smaller development set of [mMARCO-fr](https://ir-datasets.com/mmarco.html#mmarco/v2/fr/), which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compare the model performance with other CamemBERT-based biencoder models fine-tuned on the same dataset. We report the mean reciprocal rank (MRR), normalized discounted cumulative gainand (NDCG), mean average precision (MAP), and recall at various cut-offs (R@k).
115
 
116
+ | | model | #Param. | Size | R@500 | R@100(↑) | R@10 | MRR@10 | NDCG@10 | MAP@10 |
117
+ |---:|:-------------------------------------------------------------------------------------------------------------|--------:|------:|-------:|---------:|-------:|-------:|--------:|-------:|
118
+ | 1 | [biencoder-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-base-mmarcoFR) | 111M | 445MB | 89.1 | 77.8 | 51.5 | 28.5 | 33.7 | 27.9 |
119
+ | 2 | [biencoder-camembert-L10-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L10-mmarcoFR) | 96M | 386MB | 87.8 | 76.7 | 49.5 | 27.5 | 32.5 | 27.0 |
120
+ | 3 | [biencoder-camembert-L8-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L8-mmarcoFR) | 82M | 329MB | 87.4 | 75.9 | 48.9 | 26.7 | 31.8 | 26.2 |
121
+ | 4 | [biencoder-camembert-L6-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L6-mmarcoFR) | 68M | 272MB | 86.7 | 74.9 | 46.7 | 25.7 | 30.4 | 25.1 |
122
+ | 5 | [biencoder-camembert-L4-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L2-mmarcoFR) | 54M | 216MB | 85.4 | 72.1 | 44.2 | 23.7 | 28.3 | 23.2 |
123
+ | 6 | **biencoder-camembert-L2-mmarcoFR** | 40M | 159MB | 81.0 | 66.3 | 38.5 | 20.1 | 24.3 | 19.7 |
124
 
125
  ***
126