Jae-Won Chung commited on
Commit
7109f43
1 Parent(s): 7bdf0cd

Add instructions for running

Browse files
Files changed (1) hide show
  1. README.md +25 -13
README.md CHANGED
@@ -2,24 +2,36 @@
2
 
3
  How much energy do LLMs consume?
4
 
5
- ## Devs
 
6
 
7
- Current setup in `ampere02`:
8
 
9
- 1. Find model weights in `/data/leaderboard/weights/`, e.g. subdirectory `llama` and `vicuna`.
10
- 2. Let's share the Huggingface Transformer cache:
11
 
12
- ```bash
13
- export TRANSFORMERS_CACHE=/data/leaderboard/hfcache
14
- ```
15
 
16
- Run benchmarks like this:
17
 
18
  ```console
19
- $ docker build -t leaderboard:latest .
20
- $ docker run -it --name jw-leaderboard --gpus all --cap-add SYS_ADMIN -v /data/leaderboard:/data/leaderboard -v $HOME/workspace/leaderboard:/workspace/leaderboard leaderboard:latest bash
 
 
 
 
 
 
 
 
 
21
 
22
- # cd leaderboard
23
- # python scripts/benchmark.py --model-path /data/leaderboard/weights/lmsys/vicuna-7B --input-file /data/leaderboard/sharegpt/sg_90k_part1_html_cleaned_lang_first_sampled.json
24
- # python scripts/benchmark.py --model-path databricks/dolly-v2-12b --input-file /data/leaderboard/sharegpt/sg_90k_part1_html_cleaned_lang_first_sampled.json
 
 
 
 
25
  ```
 
2
 
3
  How much energy do LLMs consume?
4
 
5
+ This README focuses on explaining how to run the benchmark yourself.
6
+ The actual leaderboard is here: https://ml.energy/leaderboard.
7
 
8
+ ## Setup
9
 
10
+ ### Model weights
 
11
 
12
+ - For models that are directly accessible in Hugging Face Hub, you don't need to do anything.
13
+ - For other models, convert them to Hugging Face format and put them in `/data/leaderboard/weights/lmsys/vicuna-13B`, for example. The last two path components (e.g., `lmsys/vicuna-13B`) are taken as the name of the model.
 
14
 
15
+ ### Docker container
16
 
17
  ```console
18
+ $ git clone https://github.com/ml-energy/leaderboard.git
19
+ $ cd leaderboard
20
+ $ docker build -t ml-energy:latest .
21
+ # Replace /data/leaderboard with your data directory.
22
+ $ docker run -it \
23
+ --name leaderboard \
24
+ --gpus all \
25
+ -v /data/leaderboard:/data/leaderboard \
26
+ -v $HOME/workspace/leaderboard:/workspace/leaderboard \
27
+ ml-energy:latest bash
28
+ ```
29
 
30
+ ## Running the benchmark
31
+
32
+ ```console
33
+ # Inside the container
34
+ $ cd /workspace/leaderboard
35
+ $ python scripts/benchmark.py --model-path /data/leaderboard/weights/lmsys/vicuna-13B --input-file sharegpt/sg_90k_part1_html_cleaned_lang_first_sampled.json
36
+ $ python scripts/benchmark.py --model-path databricks/dolly-v2-12b --input-file sharegpt/sg_90k_part1_html_cleaned_lang_first_sampled.json
37
  ```