Clémentine commited on
Commit
afd9528
1 Parent(s): 00663b9

added iframes to visualizers

Browse files
Files changed (2) hide show
  1. dist/index.html +7 -2
  2. src/index.html +7 -2
dist/index.html CHANGED
@@ -115,7 +115,7 @@
115
  <p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
116
  <p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
117
 
118
- <!-- TODO: Interactive prompts exploration -->
119
 
120
  <h3>Why did we choose these subsets?</h3>
121
  <p>In summary, our criterion were: </p>
@@ -172,6 +172,11 @@
172
  <p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
173
  <p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
174
  <p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
 
 
 
 
 
175
  <p>This should allow us to keep our version up to date with new features added in the future!</p>
176
  <p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
177
 
@@ -202,7 +207,7 @@
202
 
203
  <h3>Better and simpler interface</h3>
204
  <p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
205
- <p>This is thanks to the work of the Gradio team, notably Freddy Boulton, who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a component that you can re-use yourself in your own leaderboard!</p>
206
  <p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
207
 
208
  <h2>New leaderboard, new results!</h2>
 
115
  <p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
116
  <p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
117
 
118
+ <iframe src="https://open-llm-leaderboard/sample_viewer.hf.space"></iframe>
119
 
120
  <h3>Why did we choose these subsets?</h3>
121
  <p>In summary, our criterion were: </p>
 
172
  <p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
173
  <p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
174
  <p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
175
+
176
+ <iframe src="https://open-llm-leaderboard/GenerationVisualizer.hf.space"></iframe>
177
+
178
+ <p>You can explore the visualiser we used here!</p>
179
+
180
  <p>This should allow us to keep our version up to date with new features added in the future!</p>
181
  <p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
182
 
 
207
 
208
  <h3>Better and simpler interface</h3>
209
  <p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
210
+ <p>This is thanks to the work of the Gradio team, notably [Freddy Boulton](https://huggingface.co/freddyaboulton), who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a [component](https://huggingface.co/spaces/freddyaboulton/gradio_leaderboard) that you can re-use yourself in your own leaderboard!</p>
211
  <p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
212
 
213
  <h2>New leaderboard, new results!</h2>
src/index.html CHANGED
@@ -115,7 +115,7 @@
115
  <p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
116
  <p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
117
 
118
- <!-- TODO: Interactive prompts exploration -->
119
 
120
  <h3>Why did we choose these subsets?</h3>
121
  <p>In summary, our criterion were: </p>
@@ -172,6 +172,11 @@
172
  <p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
173
  <p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
174
  <p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
 
 
 
 
 
175
  <p>This should allow us to keep our version up to date with new features added in the future!</p>
176
  <p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
177
 
@@ -202,7 +207,7 @@
202
 
203
  <h3>Better and simpler interface</h3>
204
  <p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
205
- <p>This is thanks to the work of the Gradio team, notably Freddy Boulton, who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a component that you can re-use yourself in your own leaderboard!</p>
206
  <p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
207
 
208
  <h2>New leaderboard, new results!</h2>
 
115
  <p>🤝 <strong>IFEval</strong> (Instruction Following Evaluation, <a href="https://arxiv.org/abs/2311.07911">paper</a>). IFEval is a fairly interesting dataset, which tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions, rather than the actual contents generated, which allows the use of strict and rigorous metrics.</p>
116
  <p>🧮 🤝 <strong>BBH</strong> (Big Bench Hard, <a href="https://arxiv.org/abs/2210.09261">paper</a>). BBH is a subset of 23 challenging tasks from the BigBench dataset, which 1) use objective metrics, 2) are hard, measured as language models not originally outperforming human baselines, 3) contain enough samples to be statistically significant. They contain multistep arithmetic and algorithmic reasoning (understanding boolean expressions, svg for geometric shapes, etc), language understanding (sarcasm detection, name disambiguation, etc), and some world knowledge. Performance on BBH has been on average very well correlated with human preference. We expect this dataset to provide interesting insights on specific capabilities which could interest people.</p>
117
 
118
+ <iframe src="https://open-llm-leaderboard/sample_viewer.hf.space"></iframe>
119
 
120
  <h3>Why did we choose these subsets?</h3>
121
  <p>In summary, our criterion were: </p>
 
172
  <p>For the new version of the Open LLM Leaderboard, we have therefore worked together with the amazing EleutherAI team (notably Hailey Schoelkopf, so many, huge kudos!) to update the harness.</p>
173
  <p>Features side, we added in the harness support for delta weights (LoRA finetuning/adaptation of models), a logging system compatible with the leaderboard, and the highly requested use of chat templates for evaluation.</p>
174
  <p>On the task side, we took a couple of weeks to manually check all implementations and generations thoroughly, and fix the problems we observed with inconsistent few shot samples, too restrictive end of sentence tokens, etc. We created specific configuration files for the leaderboard task implementations, and are now working on adding a test suite to make sure that evaluation results stay unchanging through time for the leaderboard tasks.</p>
175
+
176
+ <iframe src="https://open-llm-leaderboard/GenerationVisualizer.hf.space"></iframe>
177
+
178
+ <p>You can explore the visualiser we used here!</p>
179
+
180
  <p>This should allow us to keep our version up to date with new features added in the future!</p>
181
  <p>Enough said on the leaderboard backend and metrics, now let’s turn to the models and model selection/submission.
182
 
 
207
 
208
  <h3>Better and simpler interface</h3>
209
  <p>If you’re among our regular users, you may have noticed in the last month that our front end became much faster.</p>
210
+ <p>This is thanks to the work of the Gradio team, notably [Freddy Boulton](https://huggingface.co/freddyaboulton), who developed a Leaderboard <code>gradio</code> component! It notably loads data client side, which makes any column selection or search virtually instantaneous! It’s also a [component](https://huggingface.co/spaces/freddyaboulton/gradio_leaderboard) that you can re-use yourself in your own leaderboard!</p>
211
  <p>We’ve also decided to move the FAQ and About tabs to their own dedicated documentation page!</p>
212
 
213
  <h2>New leaderboard, new results!</h2>