ruMTEB v1.1 results
#1
by BorisTM - opened
| Task | Score |
|---|---|
| GeoreviewClassification | 21.8066 |
| HeadlineClassification | 21.8848 |
| InappropriatenessClassification | 50.1855 |
| KinopoiskClassification | 34.2933 |
| MassiveIntentClassification | 12.2657 |
| MassiveScenarioClassification | 17.2162 |
| RuReviewsClassification | 37.1680 |
| RuSciBenchGRNTIClassification | 4.9414 |
| RuSciBenchOECDClassification | 4.3750 |
| GeoreviewClusteringP2P | 19.9622 |
| RuSciBenchGRNTIClusteringP2P | 7.8454 |
| RuSciBenchOECDClusteringP2P | 8.3841 |
| CEDRClassification | 32.0138 |
| SensitiveTopicsClassification | 17.7783 |
| TERRa | 49.6673 |
| MIRACLReranking | 14.3370 |
| RuBQReranking | 26.0150 |
| MIRACLRetrievalHardNegatives.v2 | 1.9620 |
| RiaNewsRetrievalHardNegatives.v2 | 2.8130 |
| RuBQRetrieval | 0.4280 |
| RUParaPhraserSTS | 29.6974 |
| STS22 | −3.9589 |
| RuSTSBenchmarkSTS | 37.2306 |
| Mean(Task) | 19.4918 |
| Mean(TaskType) | 21.7442 |