Commit
•
9837751
1
Parent(s):
dfd64c8
Update README.md
Browse files
README.md
CHANGED
|
@@ -45,6 +45,20 @@ scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Que
|
|
| 45 |
### BEIR (NDCG@10)
|
| 46 |
I've run tests on different BEIR datasets. Cross Encoders rerank top100 BM25 results.
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
| | nq* | fever* | fiqa | trec-covid | scidocs | scifact | nfcorpus | hotpotqa | dbpedia-entity | quora | climate-fever |
|
| 49 |
|:--------------------------|:----------|:----------|:----------|:-------------|:----------|:----------|:-----------|:-----------|:-----------------|:----------|:----------------|
|
| 50 |
| bm25 | 0.305 | 0.638 | 0.238 | 0.589 | 0.150 | 0.676 | 0.318 | 0.629 | 0.319 | 0.787 | 0.163 |
|
|
|
|
| 45 |
### BEIR (NDCG@10)
|
| 46 |
I've run tests on different BEIR datasets. Cross Encoders rerank top100 BM25 results.
|
| 47 |
|
| 48 |
+
| | bm25 | jina-reranker-v1-turbo-en | bge-reranker-v2-m3 | mxbai-rerank-base-v1 | ms-marco-MiniLM-L-6-v2 | MiniLM-L-6-rerank-refreshed-ablated | MiniLM-L-6-rerank-refreshed |
|
| 49 |
+
|:---------------|-------:|----------------------------:|:---------------------|:-----------------------|-------------------------:|--------------------------------------:|:------------------------------|
|
| 50 |
+
| nq | 0.305 | 0.533 | **0.597** | 0.535 | 0.523 | 0.541 | 0.580 |
|
| 51 |
+
| fever | 0.638 | 0.852 | 0.857 | 0.767 | 0.801 | 0.822 | **0.867** |
|
| 52 |
+
| fiqa | 0.238 | 0.336 | **0.397** | 0.382 | 0.349 | 0.36 | 0.364 |
|
| 53 |
+
| trec-covid | 0.589 | 0.774 | 0.784 | **0.830** | 0.741 | 0.733 | 0.738 |
|
| 54 |
+
| scidocs | 0.15 | 0.166 | 0.169 | **0.171** | 0.164 | 0.163 | 0.165 |
|
| 55 |
+
| scifact | 0.676 | 0.739 | 0.731 | 0.719 | 0.688 | 0.738 | **0.750** |
|
| 56 |
+
| nfcorpus | 0.318 | 0.353 | 0.336 | **0.353** | 0.349 | 0.35 | 0.350 |
|
| 57 |
+
| hotpotqa | 0.629 | 0.745 | **0.794** | 0.668 | 0.724 | 0.758 | 0.775 |
|
| 58 |
+
| dbpedia-entity | 0.319 | 0.421 | **0.445** | 0.416 | 0.445 | 0.438 | 0.444 |
|
| 59 |
+
| quora | 0.787 | 0.858 | 0.858 | 0.747 | 0.825 | 0.862 | **0.871** |
|
| 60 |
+
| climate-fever | 0.163 | 0.233 | **0.314** | 0.253 | 0.244 | 0.245 | 0.309 |
|
| 61 |
+
|
| 62 |
| | nq* | fever* | fiqa | trec-covid | scidocs | scifact | nfcorpus | hotpotqa | dbpedia-entity | quora | climate-fever |
|
| 63 |
|:--------------------------|:----------|:----------|:----------|:-------------|:----------|:----------|:-----------|:-----------|:-----------------|:----------|:----------------|
|
| 64 |
| bm25 | 0.305 | 0.638 | 0.238 | 0.589 | 0.150 | 0.676 | 0.318 | 0.629 | 0.319 | 0.787 | 0.163 |
|