Spaces:
Running
Running
Jae-Won Chung
commited on
Commit
•
a2463c2
1
Parent(s):
6a28b83
Add formula for energy_eff
Browse files- LEADERBOARD.md +1 -1
LEADERBOARD.md
CHANGED
@@ -10,7 +10,7 @@ That is, when asked the same thing, different models answer in different lengths
|
|
10 |
|
11 |
- `gpu`: NVIDIA GPU model name. Note that NLP evaluation was only run once on our A40 GPUs, so this column only changes system-level measurements like latency and energy.
|
12 |
- `task`: Name of the task. See *Tasks* below for details.
|
13 |
-
- `energy_eff`: Our definition of energy efficiency: Average NLP evaluation metric attained per Joule of energy.
|
14 |
- `energy` (J): The average energy consumed by the model to generate a response.
|
15 |
- `nlp_average`: The arithmetic average of the NLP evaluation metrics we obtained. See *NLP evaluation metrics* below for details.
|
16 |
- `throughput` (token/s): The average number of tokens generated per second.
|
|
|
10 |
|
11 |
- `gpu`: NVIDIA GPU model name. Note that NLP evaluation was only run once on our A40 GPUs, so this column only changes system-level measurements like latency and energy.
|
12 |
- `task`: Name of the task. See *Tasks* below for details.
|
13 |
+
- `energy_eff`: Our definition of energy efficiency: Average NLP evaluation metric attained per Joule of energy (`nlp_average / energy`).
|
14 |
- `energy` (J): The average energy consumed by the model to generate a response.
|
15 |
- `nlp_average`: The arithmetic average of the NLP evaluation metrics we obtained. See *NLP evaluation metrics* below for details.
|
16 |
- `throughput` (token/s): The average number of tokens generated per second.
|