Spaces:
AIR-Bench
/
Running on CPU Upgrade

Add metadata to leaderboard

#22
by Vinno97 - opened

Hi,
It would be nice if the leaderboard also shows information like the model size and embedding size to make it easier to compare models.

AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark org

hey, Vinno,

We've thought about this previously. But our conclusion was that adding such information might add too much overhead for the submitter. As we don't require all the models must have a repo with model cards on HF, it is difficult to get the correct model information.

Though I'd personally say it's a very small amount of overhead, I understand where you're coming from and that it was a properly considered choice. Thanks for the clear response :)

Vinno97 changed discussion status to closed

Sign up or log in to comment