ruMTEB (v1.1) results
#1
by BorisTM - opened
| Task | Score |
|---|---|
| GeoreviewClassification | 21.2158 |
| HeadlineClassification | 23.4912 |
| InappropriatenessClassification | 51.4648 |
| KinopoiskClassification | 34.0133 |
| MassiveIntentClassification | 15.1560 |
| MassiveScenarioClassification | 19.1995 |
| RuReviewsClassification | 39.0039 |
| RuSciBenchGRNTIClassification | 5.8154 |
| RuSciBenchOECDClassification | 5.0244 |
| GeoreviewClusteringP2P | 19.6829 |
| RuSciBenchGRNTIClusteringP2P | 9.3429 |
| RuSciBenchOECDClusteringP2P | 9.2425 |
| CEDRClassification | 33.2678 |
| SensitiveTopicsClassification | 17.6758 |
| TERRa | 52.5268 |
| MIRACLReranking | 11.4410 |
| RuBQReranking | 25.0730 |
| MIRACLRetrievalHardNegatives.v2 | 1.1190 |
| RiaNewsRetrievalHardNegatives.v2 | 1.3040 |
| RuBQRetrieval | 0.4220 |
| RUParaPhraserSTS | 41.2728 |
| STS22 | −5.2536 |
| RuSTSBenchmarkSTS | 44.1540 |
| Mean(Task) | 20.6807 |
| Mean(TaskType) | 22.9293 |