Spaces:
Running
Running
Clémentine
commited on
Commit
•
afd9528
1
Parent(s):
00663b9
added iframes to visualizers
Browse files- dist/index.html +7 -2
- src/index.html +7 -2
dist/index.html
CHANGED
@@ -115,7 +115,7 @@
|
|
115 |
<p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
|
116 |
<p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
|
117 |
|
118 |
-
|
119 |
|
120 |
<h3>Why did we choose these subsets?</h3>
|
121 |
<p>In summary, our criterion were: </p>
|
@@ -172,6 +172,11 @@
|
|
172 |
<p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
|
173 |
<p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
|
174 |
<p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
|
|
|
|
|
|
|
|
|
|
|
175 |
<p>This should allow us to keep our version up to date with new features added in the future!</p>
|
176 |
<p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
|
177 |
|
@@ -202,7 +207,7 @@
|
|
202 |
|
203 |
<h3>Better and simpler interface</h3>
|
204 |
<p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
|
205 |
-
<p>This is thanks to the work of the Gradio team, notably Freddy Boulton, who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a component that you can re-use yourself in your own leaderboard!</p>
|
206 |
<p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
|
207 |
|
208 |
<h2>New leaderboard, new results!</h2>
|
|
|
115 |
<p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
|
116 |
<p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
|
117 |
|
118 |
+
<iframe src="https://open-llm-leaderboard/sample_viewer.hf.space"></iframe>
|
119 |
|
120 |
<h3>Why did we choose these subsets?</h3>
|
121 |
<p>In summary, our criterion were: </p>
|
|
|
172 |
<p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
|
173 |
<p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
|
174 |
<p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
|
175 |
+
|
176 |
+
<iframe src="https://open-llm-leaderboard/GenerationVisualizer.hf.space"></iframe>
|
177 |
+
|
178 |
+
<p>You can explore the visualiser we used here!</p>
|
179 |
+
|
180 |
<p>This should allow us to keep our version up to date with new features added in the future!</p>
|
181 |
<p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
|
182 |
|
|
|
207 |
|
208 |
<h3>Better and simpler interface</h3>
|
209 |
<p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
|
210 |
+
<p>This is thanks to the work of the Gradio team, notably [Freddy Boulton](https://huggingface.co/freddyaboulton), who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a [component](https://huggingface.co/spaces/freddyaboulton/gradio_leaderboard) that you can re-use yourself in your own leaderboard!</p>
|
211 |
<p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
|
212 |
|
213 |
<h2>New leaderboard, new results!</h2>
|
src/index.html
CHANGED
@@ -115,7 +115,7 @@
|
|
115 |
<p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
|
116 |
<p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
|
117 |
|
118 |
-
|
119 |
|
120 |
<h3>Why did we choose these subsets?</h3>
|
121 |
<p>In summary, our criterion were: </p>
|
@@ -172,6 +172,11 @@
|
|
172 |
<p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
|
173 |
<p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
|
174 |
<p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
|
|
|
|
|
|
|
|
|
|
|
175 |
<p>This should allow us to keep our version up to date with new features added in the future!</p>
|
176 |
<p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
|
177 |
|
@@ -202,7 +207,7 @@
|
|
202 |
|
203 |
<h3>Better and simpler interface</h3>
|
204 |
<p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
|
205 |
-
<p>This is thanks to the work of the Gradio team, notably Freddy Boulton, who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a component that you can re-use yourself in your own leaderboard!</p>
|
206 |
<p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
|
207 |
|
208 |
<h2>New leaderboard, new results!</h2>
|
|
|
115 |
<p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
|
116 |
<p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
|
117 |
|
118 |
+
<iframe src="https://open-llm-leaderboard/sample_viewer.hf.space"></iframe>
|
119 |
|
120 |
<h3>Why did we choose these subsets?</h3>
|
121 |
<p>In summary, our criterion were: </p>
|
|
|
172 |
<p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
|
173 |
<p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
|
174 |
<p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
|
175 |
+
|
176 |
+
<iframe src="https://open-llm-leaderboard/GenerationVisualizer.hf.space"></iframe>
|
177 |
+
|
178 |
+
<p>You can explore the visualiser we used here!</p>
|
179 |
+
|
180 |
<p>This should allow us to keep our version up to date with new features added in the future!</p>
|
181 |
<p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
|
182 |
|
|
|
207 |
|
208 |
<h3>Better and simpler interface</h3>
|
209 |
<p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
|
210 |
+
<p>This is thanks to the work of the Gradio team, notably [Freddy Boulton](https://huggingface.co/freddyaboulton), who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a [component](https://huggingface.co/spaces/freddyaboulton/gradio_leaderboard) that you can re-use yourself in your own leaderboard!</p>
|
211 |
<p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
|
212 |
|
213 |
<h2>New leaderboard, new results!</h2>
|