Jae-Won Chung commited on
Commit
a2463c2
1 Parent(s): 6a28b83

Add formula for energy_eff

Browse files
Files changed (1) hide show
  1. LEADERBOARD.md +1 -1
LEADERBOARD.md CHANGED
@@ -10,7 +10,7 @@ That is, when asked the same thing, different models answer in different lengths
10
 
11
  - `gpu`: NVIDIA GPU model name. Note that NLP evaluation was only run once on our A40 GPUs, so this column only changes system-level measurements like latency and energy.
12
  - `task`: Name of the task. See *Tasks* below for details.
13
- - `energy_eff`: Our definition of energy efficiency: Average NLP evaluation metric attained per Joule of energy.
14
  - `energy` (J): The average energy consumed by the model to generate a response.
15
  - `nlp_average`: The arithmetic average of the NLP evaluation metrics we obtained. See *NLP evaluation metrics* below for details.
16
  - `throughput` (token/s): The average number of tokens generated per second.
 
10
 
11
  - `gpu`: NVIDIA GPU model name. Note that NLP evaluation was only run once on our A40 GPUs, so this column only changes system-level measurements like latency and energy.
12
  - `task`: Name of the task. See *Tasks* below for details.
13
+ - `energy_eff`: Our definition of energy efficiency: Average NLP evaluation metric attained per Joule of energy (`nlp_average / energy`).
14
  - `energy` (J): The average energy consumed by the model to generate a response.
15
  - `nlp_average`: The arithmetic average of the NLP evaluation metrics we obtained. See *NLP evaluation metrics* below for details.
16
  - `throughput` (token/s): The average number of tokens generated per second.