Spaces:
Running
Running
Update content.py
Browse files- content.py +5 -2
content.py
CHANGED
@@ -19,12 +19,15 @@ SUBMISSION_TAB_TITLE_MARKDOWN = """
|
|
19 |
1. Head down to our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness).
|
20 |
Follow the instructions and evaluate your model on all π¨πΏ BenCzechMark tasks, while logging your lm harness outputs into designated folder.
|
21 |
|
22 |
-
2. Use our script [
|
23 |
Example usage:
|
24 |
- Download sample outputs for csmpt7b from [csmpt_logdir.zip](https://czechllm.fit.vutbr.cz/csmpt7b/sample_results/csmpt_logdir.zip).
|
25 |
- Unzip.
|
26 |
-
- Run the script
|
27 |
```bash
|
|
|
|
|
|
|
28 |
python compile_log_files.py \
|
29 |
-i "<your_local_path_to_folder>/csmpt_logdir/csmpt/eval_csmpt7b*" \
|
30 |
-o "<your_local_path_to_outfolder>/sample_submission.json"
|
|
|
19 |
1. Head down to our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness).
|
20 |
Follow the instructions and evaluate your model on all π¨πΏ BenCzechMark tasks, while logging your lm harness outputs into designated folder.
|
21 |
|
22 |
+
2. Use our script from [benczechmark-leaderboard](https://github.com/MFajcik/benczechmark-leaderboard) repository for processing log files from your designated folder into single compact submission file that contains everything we need.
|
23 |
Example usage:
|
24 |
- Download sample outputs for csmpt7b from [csmpt_logdir.zip](https://czechllm.fit.vutbr.cz/csmpt7b/sample_results/csmpt_logdir.zip).
|
25 |
- Unzip.
|
26 |
+
- Run the script from leaderboard repository with python (with libs jsonlines and tqdm)
|
27 |
```bash
|
28 |
+
git clone https://github.com/MFajcik/benczechmark-leaderboard.git
|
29 |
+
cd benczechmark-leaderboard/
|
30 |
+
export PYTHONPATH=$(pwd)
|
31 |
python compile_log_files.py \
|
32 |
-i "<your_local_path_to_folder>/csmpt_logdir/csmpt/eval_csmpt7b*" \
|
33 |
-o "<your_local_path_to_outfolder>/sample_submission.json"
|