Commit
•
5b8ccf5
1
Parent(s):
283fa86
Update README.md
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ library_name: sentence-transformers
|
|
| 20 |
<a href=#usage>🛠️ Usage</a> |
|
| 21 |
<a href="#evaluation">📊 Evaluation</a> |
|
| 22 |
<a href="#train">🤖 Training</a> |
|
| 23 |
-
<a href="#citation">🔗 Citation</a>
|
| 24 |
<p>
|
| 25 |
</h4>
|
| 26 |
|
|
@@ -113,14 +113,14 @@ print(similarity)
|
|
| 113 |
|
| 114 |
We evaluate the model on the smaller development set of [mMARCO-fr](https://ir-datasets.com/mmarco.html#mmarco/v2/fr/), which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compare the model performance with other CamemBERT-based biencoder models fine-tuned on the same dataset. We report the mean reciprocal rank (MRR), normalized discounted cumulative gainand (NDCG), mean average precision (MAP), and recall at various cut-offs (R@k).
|
| 115 |
|
| 116 |
-
| | model |
|
| 117 |
-
|
| 118 |
-
| 1 | [biencoder-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-base-mmarcoFR) |
|
| 119 |
-
| 2 | [biencoder-camembert-L10-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L10-mmarcoFR) |
|
| 120 |
-
| 3 | [biencoder-camembert-L8-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L8-mmarcoFR) |
|
| 121 |
-
| 4 | [biencoder-camembert-L6-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L6-mmarcoFR) |
|
| 122 |
-
| 5 | **biencoder-camembert-L4-mmarcoFR** |
|
| 123 |
-
| 6 | [biencoder-camembert-L2-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L2-mmarcoFR) |
|
| 124 |
|
| 125 |
***
|
| 126 |
|
|
|
|
| 20 |
<a href=#usage>🛠️ Usage</a> |
|
| 21 |
<a href="#evaluation">📊 Evaluation</a> |
|
| 22 |
<a href="#train">🤖 Training</a> |
|
| 23 |
+
<a href="#citation">🔗 Citation</a>
|
| 24 |
<p>
|
| 25 |
</h4>
|
| 26 |
|
|
|
|
| 113 |
|
| 114 |
We evaluate the model on the smaller development set of [mMARCO-fr](https://ir-datasets.com/mmarco.html#mmarco/v2/fr/), which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compare the model performance with other CamemBERT-based biencoder models fine-tuned on the same dataset. We report the mean reciprocal rank (MRR), normalized discounted cumulative gainand (NDCG), mean average precision (MAP), and recall at various cut-offs (R@k).
|
| 115 |
|
| 116 |
+
| | model | #Param. | Size | R@500 | R@100(↑) | R@10 | MRR@10 | NDCG@10 | MAP@10 |
|
| 117 |
+
|---:|:-------------------------------------------------------------------------------------------------------------|--------:|------:|-------:|---------:|-------:|-------:|--------:|-------:|
|
| 118 |
+
| 1 | [biencoder-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-base-mmarcoFR) | 111M | 445MB | 89.1 | 77.8 | 51.5 | 28.5 | 33.7 | 27.9 |
|
| 119 |
+
| 2 | [biencoder-camembert-L10-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L10-mmarcoFR) | 96M | 386MB | 87.8 | 76.7 | 49.5 | 27.5 | 32.5 | 27.0 |
|
| 120 |
+
| 3 | [biencoder-camembert-L8-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L8-mmarcoFR) | 82M | 329MB | 87.4 | 75.9 | 48.9 | 26.7 | 31.8 | 26.2 |
|
| 121 |
+
| 4 | [biencoder-camembert-L6-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L6-mmarcoFR) | 68M | 272MB | 86.7 | 74.9 | 46.7 | 25.7 | 30.4 | 25.1 |
|
| 122 |
+
| 5 | **biencoder-camembert-L4-mmarcoFR** | 54M | 216MB | 85.4 | 72.1 | 44.2 | 23.7 | 28.3 | 23.2 |
|
| 123 |
+
| 6 | [biencoder-camembert-L2-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-L2-mmarcoFR) | 40M | 159MB | 81.0 | 66.3 | 38.5 | 20.1 | 24.3 | 19.7 |
|
| 124 |
|
| 125 |
***
|
| 126 |
|