Spaces:

ml-energy

/

leaderboard

like 8

Running

Thanks for your interest.

I think it's clear enough. Not many people will think the 7 means 7 parameters; most people will understand it's 7 billion. It's fair to point out that the column should have been named parameters (B), but our column names need to avoid having spaces or parentheses to allow custom metrics (in the box below the table). Basically, if it were parameters (B), df.eval will require extra customization to understand that that variable, which we thought this wasn't worth the effort.

Thanks for your interest.

I think it's clear enough. Not many people will think the 7 means 7 parameters; most people will understand it's 7 billion. It's fair to point out that the column should have been named parameters (B), but our column names need to avoid having spaces or parentheses to allow custom metrics (in the box below the table). Basically, if it were parameters (B), df.eval will require extra customization to understand that that variable, which we thought this wasn't worth the effort.

Thanks! But how about million-level-parameter models? Would you use decimal such as "0.3"? I think it is still worth the efforts. @jaywonchung

Yep, I think so. But if that time comes and if we realize it's too ugly, we'll try to think of something better. I thought for now it's safe to assume that any Large Language Model will have at least one billion parameters.

Yep, I think so. But if that time comes and if we realize it's too ugly, we'll try to think of something better. I thought for now it's safe to assume that any Large Language Model will have at least one billion parameters.

@jaywonchung I think MLLM smaller than 1B parameter has been released out there yet, check https://arxiv.org/pdf/2404.11459 and https://www.nexa4ai.com/apply
Thus, I recommend refining the column name to something like parameters (B).
Here is my pull request for your review: https://huggingface.co/spaces/ml-energy/leaderboard/discussions/3