Commit
•
ab216b6
1
Parent(s):
ef545c8
Update README.md
Browse files
README.md
CHANGED
|
@@ -47,8 +47,9 @@ scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Que
|
|
| 47 |
### BEIR (NDCG@10)
|
| 48 |
I've run tests on different BEIR datasets. Cross Encoders rerank top100 BM25 results.
|
| 49 |
|
|
|
|
| 50 |
| | bm25 | jina-reranker-v1-turbo-en | bge-reranker-v2-m3 | mxbai-rerank-base-v1 | ms-marco-MiniLM-L-6-v2 | MiniLM-L-6-rerank-m3 |
|
| 51 |
-
|
| 52 |
| nq* | 0.305 | 0.533 | **0.597** | 0.535 | 0.523 | 0.580 |
|
| 53 |
| fever* | 0.638 | 0.852 | 0.857 | 0.767 | 0.801 | **0.867** |
|
| 54 |
| fiqa | 0.238 | 0.336 | **0.397** | 0.382 | 0.349 | 0.364 |
|
|
@@ -61,12 +62,13 @@ I've run tests on different BEIR datasets. Cross Encoders rerank top100 BM25 res
|
|
| 61 |
| quora | 0.787 | 0.858 | 0.858 | 0.747 | 0.825 | **0.871** |
|
| 62 |
| climate-fever | 0.163 | 0.233 | **0.314** | 0.253 | 0.244 | 0.309 |
|
| 63 |
|
|
|
|
| 64 |
\* Training splits of NQ and Fever were used as part of the training data.
|
| 65 |
|
| 66 |
Comparison with [ablated model](https://huggingface.co/juanluisdb/MiniLM-L-6-rerank-m3-ablated) trained only on MSMarco:
|
| 67 |
|
| 68 |
| | ms-marco-MiniLM-L-6-v2 | MiniLM-L-6-rerank-m3-ablated |
|
| 69 |
-
|
| 70 |
| nq | 0.5234 | **0.5412** |
|
| 71 |
| fever | 0.8007 | **0.8221** |
|
| 72 |
| fiqa | 0.349 | **0.3598** |
|
|
|
|
| 47 |
### BEIR (NDCG@10)
|
| 48 |
I've run tests on different BEIR datasets. Cross Encoders rerank top100 BM25 results.
|
| 49 |
|
| 50 |
+
|
| 51 |
| | bm25 | jina-reranker-v1-turbo-en | bge-reranker-v2-m3 | mxbai-rerank-base-v1 | ms-marco-MiniLM-L-6-v2 | MiniLM-L-6-rerank-m3 |
|
| 52 |
+
|:---------------|:-------:|:----------------------------:|:---------------------:|:-----------------------:|:-------------------------:|:------------------------------:|
|
| 53 |
| nq* | 0.305 | 0.533 | **0.597** | 0.535 | 0.523 | 0.580 |
|
| 54 |
| fever* | 0.638 | 0.852 | 0.857 | 0.767 | 0.801 | **0.867** |
|
| 55 |
| fiqa | 0.238 | 0.336 | **0.397** | 0.382 | 0.349 | 0.364 |
|
|
|
|
| 62 |
| quora | 0.787 | 0.858 | 0.858 | 0.747 | 0.825 | **0.871** |
|
| 63 |
| climate-fever | 0.163 | 0.233 | **0.314** | 0.253 | 0.244 | 0.309 |
|
| 64 |
|
| 65 |
+
|
| 66 |
\* Training splits of NQ and Fever were used as part of the training data.
|
| 67 |
|
| 68 |
Comparison with [ablated model](https://huggingface.co/juanluisdb/MiniLM-L-6-rerank-m3-ablated) trained only on MSMarco:
|
| 69 |
|
| 70 |
| | ms-marco-MiniLM-L-6-v2 | MiniLM-L-6-rerank-m3-ablated |
|
| 71 |
+
|:---------------|:-------------------------:|:--------------------------------------:|
|
| 72 |
| nq | 0.5234 | **0.5412** |
|
| 73 |
| fever | 0.8007 | **0.8221** |
|
| 74 |
| fiqa | 0.349 | **0.3598** |
|