Commit
•
79370ce
1
Parent(s):
c008de6
update eval results
Browse files
README.md
CHANGED
@@ -33,7 +33,7 @@ It was initialized from the [HerBERT-base](https://huggingface.co/allegro/herber
|
|
33 |
|
34 |
| **Model** | **Average [Acc]** | **Average [NDCG]** | [**PolQA**](https://huggingface.co/datasets/ipipan/polqa) **[Acc]** | [**PolQA**](https://huggingface.co/datasets/ipipan/polqa) **[NDCG]** | [**Allegro FAQ**](https://huggingface.co/datasets/piotr-rybak/allegro-faq) **[Acc]** | [**Allegro FAQ**](https://huggingface.co/datasets/piotr-rybak/allegro-faq) **[NDCG]** | [**Legal Questions**](https://huggingface.co/datasets/piotr-rybak/legal-questions) **[Acc]** | [**Legal Questions**](https://huggingface.co/datasets/piotr-rybak/legal-questions) **[NDCG]** |
|
35 |
|--------------------:|------------:|-------------:|------------:|-------------:|------------:|-------------:|------------:|-------------:|
|
36 |
-
| BM25 | 74.87 | 51.81 | 61.35 | 24.51 | 66.89 | 48.71 |
|
37 |
| BM25 (lemma) | 80.46 | 55.44 | 71.49 | 31.97 | 75.33 | 55.70 | 94.57 | 78.65 |
|
38 |
| [MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) | 62.62 | 39.21 | 37.24 | 11.93 | 71.67 | 51.25 | 78.97 | 54.44 |
|
39 |
| [LaBSE](https://huggingface.co/sentence-transformers/LaBSE) | 64.89 | 39.47 | 46.23 | 15.53 | 67.11 | 46.71 | 81.34 | 56.16 |
|
@@ -42,7 +42,8 @@ It was initialized from the [HerBERT-base](https://huggingface.co/allegro/herber
|
|
42 |
| [ST-DistilRoBERTa](https://huggingface.co/sdadas/st-polish-paraphrase-from-distilroberta) | 73.78 | 48.29 | 48.43 | 16.73 | 84.89 | 64.39 | 88.02 | 63.76 |
|
43 |
| [ST-MPNet](https://huggingface.co/sdadas/st-polish-paraphrase-from-mpnet) | 76.66 | 49.99 | 56.80 | 21.55 | 86.00 | 65.44 | 87.19 | 62.99 |
|
44 |
| [HerBERT-QA](https://huggingface.co/ipipan/herbert-base-qa-v1) | 84.23 | 54.36 | 75.84 | 32.52 | 85.78 | 63.58 | 91.09 | 66.99 |
|
45 |
-
| [
|
|
|
46 |
|
47 |
Legend:
|
48 |
- **Acc** is the Accuracy at 10
|
|
|
33 |
|
34 |
| **Model** | **Average [Acc]** | **Average [NDCG]** | [**PolQA**](https://huggingface.co/datasets/ipipan/polqa) **[Acc]** | [**PolQA**](https://huggingface.co/datasets/ipipan/polqa) **[NDCG]** | [**Allegro FAQ**](https://huggingface.co/datasets/piotr-rybak/allegro-faq) **[Acc]** | [**Allegro FAQ**](https://huggingface.co/datasets/piotr-rybak/allegro-faq) **[NDCG]** | [**Legal Questions**](https://huggingface.co/datasets/piotr-rybak/legal-questions) **[Acc]** | [**Legal Questions**](https://huggingface.co/datasets/piotr-rybak/legal-questions) **[NDCG]** |
|
35 |
|--------------------:|------------:|-------------:|------------:|-------------:|------------:|-------------:|------------:|-------------:|
|
36 |
+
| BM25 | 74.87 | 51.81 | 61.35 | 24.51 | 66.89 | 48.71 | 96.38 | **82.21** |
|
37 |
| BM25 (lemma) | 80.46 | 55.44 | 71.49 | 31.97 | 75.33 | 55.70 | 94.57 | 78.65 |
|
38 |
| [MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) | 62.62 | 39.21 | 37.24 | 11.93 | 71.67 | 51.25 | 78.97 | 54.44 |
|
39 |
| [LaBSE](https://huggingface.co/sentence-transformers/LaBSE) | 64.89 | 39.47 | 46.23 | 15.53 | 67.11 | 46.71 | 81.34 | 56.16 |
|
|
|
42 |
| [ST-DistilRoBERTa](https://huggingface.co/sdadas/st-polish-paraphrase-from-distilroberta) | 73.78 | 48.29 | 48.43 | 16.73 | 84.89 | 64.39 | 88.02 | 63.76 |
|
43 |
| [ST-MPNet](https://huggingface.co/sdadas/st-polish-paraphrase-from-mpnet) | 76.66 | 49.99 | 56.80 | 21.55 | 86.00 | 65.44 | 87.19 | 62.99 |
|
44 |
| [HerBERT-QA](https://huggingface.co/ipipan/herbert-base-qa-v1) | 84.23 | 54.36 | 75.84 | 32.52 | 85.78 | 63.58 | 91.09 | 66.99 |
|
45 |
+
| [Silver Retriever v1](https://huggingface.co/ipipan/silver-retriever-base-v1) | 92.45 | 66.72 | 87.24 | 43.40 | **94.56** | 79.66 | 95.54 | 77.10 |
|
46 |
+
| [Silver Retriever v1.1](https://huggingface.co/ipipan/silver-retriever-base-v1.1) | **93.18** | **67.55** | **88.60** | 44.88 | 94.00 | **79.83** | **96.94** | 77.95 |
|
47 |
|
48 |
Legend:
|
49 |
- **Acc** is the Accuracy at 10
|