Add one more commit to lm-evaluation-harness

deployment/benchmark.Dockerfile

@@ -26,7 +26,7 @@ RUN cd /workspace/leaderboard && pip install -e .[benchmark]
 RUN cd /workspace \
       && git clone https://github.com/EleutherAI/lm-evaluation-harness.git \
       && cd lm-evaluation-harness \
-      && git checkout e9f1af36d2f6f8449e3cd132e6885d3b010ec838 \
+      && git checkout d1537059b515511801ae9b742f8e949f1bfcd010 \
       && rm -r .git \
       && pip install -e .
 

docs/leaderboard.md

@@ -42,7 +42,7 @@ Find our benchmark script for one model [here](https://github.com/ml-energy/lead
 - PyTorch 2.0.1
 - [Zeus](https://ml.energy/zeus) -- For GPU time and energy measurement
 - [FastChat](https://github.com/lm-sys/fastchat) -- For running inference on various models
-- [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/e9f1af36d2f6f8449e3cd132e6885d3b010ec838) -- For NLP evaluation metrics
+- [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/d1537059b515511801ae9b742f8e949f1bfcd010) -- For NLP evaluation metrics
 
 ### Hardware
 

pegasus/README.md

@@ -65,7 +65,7 @@ After all the tasks finish, aggregate all the data into one node and run [`compu
 
 ## NLP benchmark
 
-We'll use [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/e9f1af36d2f6f8449e3cd132e6885d3b010ec838) to run models through three NLP datasets: ARC challenge (`arc`), HellaSwag (`hellaswag`), and TruthfulQA (`truthfulqa`).
+We'll use [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/d1537059b515511801ae9b742f8e949f1bfcd010) to run models through three NLP datasets: ARC challenge (`arc`), HellaSwag (`hellaswag`), and TruthfulQA (`truthfulqa`).
 
 Use Pegasus to run benchmarks for all the models across all nodes.