Is there a way to download CSV?

#133
by cheebw1 - opened

Is there a way to download a CSV of results? I want to create a custom ranking based on specific datasets of interest eg medical. I didn't see a good way to download raw data. Thanks!

Massive Text Embedding Benchmark org
edited Jul 9

Hi cheebw1, all results can be found here. We are also currently working on developing code to more easily access the results. In a future version of mteb it will be available as something like:

# this is not yet implemented
import mteb
results = mteb.load_results() # downloads and loads in results using MTEBResults

# format will be:
results: dict[MODEL_NAME_STR, dict[REVISION_STR, list[MTEBResult]]

We will likely also add utilities for converting this format to e.g. a data frame

Massive Text Embedding Benchmark org

Adding to @KennethEnevoldsen great reply, we also plan to change the leaderboard soon to look something like this: https://github.com/embeddings-benchmark/mteb/discussions/674#discussioncomment-9404987 which would allow you to create custom rankings based on specific datasets. Help on this is very welcome if you are interested :)

Sign up or log in to comment