Question: According to the image, in the benchmark from the general category where Llama 3 8B performs the worst, what is the name of the third author of that benchmark?
Date: 2024-08-08

Search results:

MMLU-Pro: A More Robust and Challenging Multi-Task Language ...
Our experimental results show that MMLU-Pro not only raises the challenge, causing a significant drop in accuracy by 16% to 33% compared to MMLU but also demonstrates greater stability under varying prompts. With 24 different prompt styles tested, the sensitivity of model scores to prompt variations decreased from 4-5% in MMLU to just 2% in ...

MMLU-Pro: A More Robust and Challenging Multi-Task Language ...
Authors: Yubo Wang, Xueguang Ma, Ge Zhang, ... This paper introduces MMLU-Pro, an enhanced dataset designed to extend the mostly knowledge-driven MMLU benchmark by integrating more challenging, reasoning-focused questions and expanding the choice set from four to ten options. Additionally, MMLU-Pro eliminates the trivial and noisy questions in ...

MMLU-Pro: A More Robust and Challenging Multi-Task Language ...
The MMLU-Pro dataset, while enhancing the complexity of MMLU by incorporating more challenging, reasoning-focused questions, remains constrained by the limitations of the multiple-choice format. This format may not capture the depth of comprehension and creative response generation as effectively as open-ended answers, which better reflect real ...

MMLU-Pro: A More Robust and Challenging Multi-Task Language ...
MMLU-Pro is introduced, an enhanced dataset designed to extend the mostly knowledge-driven MMLU benchmark by integrating more challenging, reasoning-focused questions and expanding the choice set from four to ten options, indicating that MMLU-Pro includes more complex reasoning questions. In the age of large-scale language models, benchmarks like the Massive Multitask Language Understanding ...

MMLU-Pro: A More Robust and Challenging Multi-Task Language ...
MMLU-Pro is more discriminative than MMLU in distinguishing the nuances between models. For example, the gap between GPT -4o and GPT-4-T urbo is 1% on MMLU, while it becomes 9% on MMLU-Pro.

MMLU-Pro Dataset | Papers With Code
The MMLU-Pro dataset is an enhanced version of the Massive Multitask Language Understanding (MMLU) benchmark. It's designed to be more robust and challenging, aiming to rigorously benchmark large language models' capabilities in language comprehension and reasoning across diverse domains. Here are some key features of the MMLU-Pro dataset: Increased Complexity: It includes more reasoning ...

MMLU-Pro: A More Robust and Challenging Multi-Task Language ...
Upload an image to customize your repository's social media preview. Images should be at least 640×320px (1280×640px for best display).

README.md · TIGER-Lab/MMLU-Pro at main - Hugging Face
In MMLU-Pro, CoT can be 20% higher than PPL. By increasing the distractor numbers, we significantly reduce the probability of correct guess by chance to boost the benchmark's robustness. Specifically, with 24 different prompt styles tested, the sensitivity of model scores to prompt variations decreased from 4-5% in MMLU to just 2% in MMLU-Pro. 2.

Answer: Ge Zhang