Commit
•
2a6e114
1
Parent(s):
d11fb56
Update README.md
Browse files
README.md
CHANGED
@@ -142,7 +142,8 @@ Meta-Llama-3-70B-Instruct 9.006250
|
|
142 |
of the [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness/).
|
143 |
The commit used by the leaderboard has a significant issue that impacts models that
|
144 |
tend to use `:` in their responses due to a bug in the stop word configuration for
|
145 |
-
GSM8K. The issue is
|
|
|
146 |
The score for both Llama-3 and this model are significantly different when evaluated
|
147 |
with the updated harness as the issue with stop words has been addressed.
|
148 |
|
|
|
142 |
of the [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness/).
|
143 |
The commit used by the leaderboard has a significant issue that impacts models that
|
144 |
tend to use `:` in their responses due to a bug in the stop word configuration for
|
145 |
+
GSM8K. The issue is covered in more detail at
|
146 |
+
[GSM8K eval issue discussion](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/770).
|
147 |
The score for both Llama-3 and this model are significantly different when evaluated
|
148 |
with the updated harness as the issue with stop words has been addressed.
|
149 |
|