Clémentine Fourrier's picture

Clémentine Fourrier

clefourrier

·

http://clefourrier.github.io

AI & ML interests

None yet

Articles

BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks

CyberSecEval 2 - A Comprehensive Evaluation Framework for Cybersecurity Risks and Capabilities of Large Language Models

Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens tokens and 11 languages

Let's talk about LLM evaluation

Introducing the Open Arabic LLM Leaderboard

Introducing the Open Leaderboard for Hebrew LLMs!

Bringing the Artificial Analysis LLM Performance Leaderboard to Hugging Face

Improving Prompt Consistency with Structured Generations

Introducing the Open Chain of Thought Leaderboard

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

Introducing the LiveCodeBench Leaderboard - Holistic and Contamination-Free Evaluation of Code LLMs

Introducing the Chatbot Guardrails Arena

Introducing ConTextual: How well can your Multimodal model jointly reason over text and image in text-rich scenes?

TTS Arena: Benchmarking Text-to-Speech Models in the Wild

Introducing the Red-Teaming Resistance Leaderboard

Introducing the Open Ko-LLM Leaderboard: Leading the Korean LLM Evaluation Ecosystem

NPHardEval Leaderboard: Unveiling the Reasoning Abilities of Large Language Models through Complexity Classes and Dynamic Updates

Introducing the Enterprise Scenarios Leaderboard: a Leaderboard for Real World Use Cases

The Hallucinations Leaderboard, an Open Effort to Measure Hallucinations in Large Language Models

A guide to setting up your own Hugging Face leaderboard: an end-to-end example with Vectara's hallucination leaderboard

2023, year of open LLMs

Open LLM Leaderboard: DROP deep dive

Overview of natively supported quantization schemes in 🤗 Transformers

What's going on with the Open LLM Leaderboard?

Introduction to Graph Machine Learning

Organizations

clefourrier's activity

New activity in open-llm-leaderboard/open_llm_leaderboard about 4 hours ago

Dear huggingface leaderboard

#820 opened about 6 hours ago by

New activity in demo-leaderboard-backend/backend about 12 hours ago

Renaming environ variable

#6 opened about 19 hours ago by

meg

Changing to REPO_ID that exists

#5 opened about 20 hours ago by

meg

Fix minor import error

#4 opened about 20 hours ago by

meg

adds variables EvalRequest is searching for

#8 opened about 19 hours ago by

meg

New activity in open-llm-leaderboard/open_llm_leaderboard 1 day ago

Re-Evaluate models with old Llama 3 generation config

#815 opened 2 days ago by

Running MMLU-Pro with Eleuther LM-Eval

#814 opened 2 days ago by

New activity in open-llm-leaderboard/requests 1 day ago

Update migtissera/Tess-v2.5-Phi-3-medium-128k-14B_eval_request_False_float16_Original.json

#23 opened 2 days ago by

Update migtissera/Llama-3-70B-Synthia-v3.5_eval_request_False_float16_Original.json

#24 opened 2 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 2 days ago

Model failure: migtissera/Tess-v2.5.2-Qwen2-72B model

#799 opened 7 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 5 days ago

Models disappearing from eval queue?

#805 opened 6 days ago by

fix-merged-column

#810 opened 5 days ago by

Leaderboard isn't updating its model list.

#809 opened 5 days ago by

📑 Raw results link goes to old leaderboard results dataset

#808 opened 5 days ago by

Some models are tagged with incorrect model types

#806 opened 5 days ago by

Archive of the last leaderboard

#807 opened 5 days ago by

MarxistLeninist

Some suggestions for evaluation priority voting mechanism

#801 opened 7 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 6 days ago

RecurrentGemma - add the rest of the models!

#800 opened 7 days ago by

submission-fix

#803 opened 6 days ago by

Failed evaluation for 70B model

#804 opened 6 days ago by

sarath-shekkizhar

Bug with gui

#802 opened 6 days ago by

New activity in open-llm-leaderboard/GenerationVisualizer 6 days ago

Leaks GPQA Questions

#2 opened 6 days ago by

New activity in open-llm-leaderboard/requests 6 days ago

Renaming Model Qwen/Qwen2-72B-Instruct to Qwen/qwen-test-rename2

#22 opened 6 days ago by

New activity in open-llm-leaderboard/results 6 days ago

Renaming Model Qwen/Qwen2-72B-Instruct to Qwen/qwen-test-rename2

#2 opened 6 days ago by

New activity in open-llm-leaderboard/requests 6 days ago

Renaming Model Qwen/Qwen2-72B-Instruct to Qwen/qwen-test-rename

#21 opened 6 days ago by

New activity in open-llm-leaderboard/results 6 days ago

Renaming Model Qwen/Qwen2-72B-Instruct to Qwen/qwen-test-rename

#1 opened 6 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 6 days ago

Feature request: Add non AutoModelForCausalLM models

#391 opened 8 months ago by

KnutJaegersberg

New activity in open-llm-leaderboard/requests 6 days ago

Update altomek/YiSM-34B-0rn_eval_request_False_float16_Original.json

#19 opened 6 days ago by

Update altomek/CodeRosa-70B-AB1_eval_request_False_float16_Original.json

#20 opened 6 days ago by

Update altomek/YiSM-34B-0rn_eval_request_False_float16_Original.json

#17 opened 6 days ago by

Update altomek/CodeRosa-70B-AB1_eval_request_False_float16_Original.json

#18 opened 6 days ago by

New activity in demo-leaderboard-backend/leaderboard 6 days ago

Is this space supposed to work in tandem with Lighteval?

#5 opened about 1 month ago by

The leaderboard is not working..

#6 opened 21 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 6 days ago

Upvote to evaluate deepseek-coder-v2

#793 opened 7 days ago by

New activity in open-llm-leaderboard/requests 6 days ago

Update VAGOsolutions/SauerkrautLM-1.5b_eval_request_False_bfloat16_Original.json

#6 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-7b-HerO_eval_request_False_bfloat16_Original.json

#7 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-7b-LaserChat_eval_request_False_bfloat16_Original.json

#8 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-Gemma-2b_eval_request_False_bfloat16_Original.json

#9 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-Gemma-7b_eval_request_False_bfloat16_Original.json

#10 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-Mixtral-8x7B-Instruct_eval_request_False_bfloat16_Original.json

#11 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-Qwen-32b_eval_request_False_bfloat16_Original.json

#14 opened 7 days ago by

Update VAGOsolutions/SauerkrautLM-SOLAR-Instruct_eval_request_False_bfloat16_Original.json

#15 opened 7 days ago by

Update migtissera/Tess-v2.5.2-Qwen2-72B_eval_request_False_float16_Original.json

#16 opened 7 days ago by

Update VAGOsolutions/Llama-3-SauerkrautLM-70b-Instruct_eval_request_False_bfloat16_Original.json

#5 opened 7 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 6 days ago

Resource: Understanding the new benchmarks

#796 opened 7 days ago by

Question: Will open LLM leadboard add some explicit coding benchmark?

#797 opened 7 days ago by

Congrats to the new leaderboard! Accidentally uploaded the wrong modeltype.

#794 opened 7 days ago by

Tweaks / improvements: Add column with parameter count and mouseover labels for datasets

#795 opened 7 days ago by

Issues with Model Submissions , Pending Stuck on 180 from 8 hours

#798 opened 7 days ago by

Models that used Nectar dataset

#749 opened about 2 months ago by

New activity in gaia-benchmark/leaderboard 7 days ago

Update app.py

#18 opened 7 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 7 days ago

Reproduction

#792 opened 7 days ago by

What happened with qwen2 ratings?

#786 opened 23 days ago by

Wanted to check status of evaluation on leaderboard

#791 opened 7 days ago by

New activity in open-llm-leaderboard/blog 7 days ago

Final pass

#2 opened 7 days ago by

Minor grammar changes

#1 opened 7 days ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 8 days ago

No good way to identify number of activated parameters causes MIxtral evaluation failures

#680 opened 3 months ago by

TRI-ML/mamba-7b-rw failed

#704 opened 2 months ago by

GPTQ and Mixtral models will need to be relaunched

#692 opened 2 months ago by

ALL Jamba models failing

#690 opened 3 months ago by